USING MULTICULTURAL CONSULTATION TO SUPPORT THE IMPLEMENTATION AND DE-IMPLEMENTATION OF SCHOOL-BASED PRACTICES TO IMPROVE OUTCOMES FOR MARGINALIZED YOUTH By Andryce Clinkscales A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of School Psychology – Doctor of Philosophy 2024 ABSTRACT Racially and ethnically minoritized students experience a wide range of negative outcomes when compared to their White counterparts. School psychologists are called to address these disparate outcomes for racially and ethnically minoritized students and are uniquely positioned to use their consultation training as a mechanism to improve them. Consultation is a systematic problem-solving process in which consultants (e.g., school psychologists) collaborate with consultees (e.g., teachers, administrators) to address clients’ (e.g., students’) problems. All of these individuals can have different racial and ethnic backgrounds; and yet, there is little attention placed on how these racial differences might impact the effectiveness of consultation. A consultation model that explicitly considers the cultural backgrounds of those within the consultation triad (i.e., the consultant, consultee, client) is multicultural consultee-centered consultation, which is the theoretical framework that informs all three papers within this dissertation. The three papers in this dissertation explore how multicultural consultee-centered consultation might be used to improve outcomes for racially and ethnically minoritized youth. The first two papers examine how racial match (i.e., when students and teachers share the same race) influences school-based consultation. The third paper focuses on student race, and (a) investigates how de-implementation is conceptualized in the literature, (b) how de- implementation impacts racially and ethnically minoritized populations, and (c) how multicultural consultee-centered consultation can be leveraged to promote de-implementation that supports positive student outcomes. Each of the three papers provide recommendations for how research and practice can integrate race and ethnicity, and other cultural factors, into school- based consultation. Copyright by ANDRYCE CLINKSCALES 2024 To those who dream. iv ACKNOWLEDGEMENTS You have probably heard that getting a doctoral degree “takes a village,” that it is not possible alone. I’m here to tell you that this is true. I would not be writing this today without the support of my family, friends, doctoral advisor and committee members, and my loving husband. To my family: Thank you for continuously cheering me on over the past four and a half years. I did not always know that this achievement would be possible for me, but you did. To my siblings, Madison and Eli, I hope that I have shown you the power of perseverance and that you can achieve anything you set your mind to (and more). To my grandparents, your encouragement helped make the challenges I faced seem less intimidating. To my mom, we did it. To my friends: I am forever grateful for the patience and love you provided me along this journey. Whether I needed a listening ear, time away from my computer and a good laugh, or accountability to get things done, you were there. The memories I have made with you while completing this dissertation will be ones I will look back on with gratitude and nostalgia. To my doctoral advisor and committee members: Thank you for pushing me to be the professional that I am today. I have grown so much over the last several years and you all played a crucial role in helping me develop my identity as a scholar and practitioner. Dr. Barrett, thank you for your affirming mentorship and intentional guidance. To my husband: Daniel, thank you for believing in me, sometimes when I didn’t even believe in myself. It’s difficult to put into words just how much you supported me, sacrificed for us, and cared for me as I reached for this goal. There is no doubt in my mind that this dissertation would not have been possible without you. I only hope that I can return to you in kind the love and support you have given me throughout this journey. v TABLE OF CONTENTS CHAPTER 1: INTRODUCTION……………………………………………………………...….1 CHAPTER 2 (PAPER 1): THE ROLE OF RACIAL MATCH BETWEEN STUDENTS AND TEACHERS IN SCHOOL-BASED CONSULTATION ...............................................................6 CHAPTER 3 (PAPER 2): WHAT’S RACE GOT TO DO WITH IT? RELATIONSHIPS BETWEEN RACIAL MATCH AND SCHOOL PSYCHOLOGISTS’ PERCEPTIONS OF SCHOOL-BASED CONSULTATION ........................................................................................33 CHAPTER 4 (PAPER 3): HOW DOES CULTURE FIT INTO DE-IMPLEMENTATION? A SCOPING REVIEW OF EMPIRICAL RESEARCH...................................................................53 CHAPTER 5: CONCLUSION…………………………………………………………………..75 REFERENCES………………......................................................................................................82 APPENDIX A: AUTHORSHIP DISCLOSURES .......................................................................93 APPENDIX B: TABLES FOR PAPER 1 ....................................................................................94 APPENDIX C: FIGURES FOR PAPER 1 .................................................................................100 APPENDIX D: RECRUITMENT EMAIL FOR PAPERS 1-2...................................................105 APPENDIX E: IRB PROPOSAL FOR PAPERS 1-2 ................................................................106 APPENDIX F: VIGNETTES FOR PAPERS 1-2........................................................................108 APPENDIX G: SURVEY ITEMS FOR PAPERS 1-2………………………..…………….….111 APPENDIX H: TABLES FOR PAPER 2 ...................................................................................113 APPENDIX I: FIGURE FOR PAPER 2 .....................................................................................119 APPENDIX J: TABLES FOR PAPER 3 ....................................................................................120 APPENDIX K: FIGURES FOR PAPER 3…………………………………………………….131 vi CHAPTER 1: INTRODUCTION Students from historically marginalized backgrounds (e.g., racially and ethnically minoritized students) consistently experience poor academic, behavioral, and socioemotional outcomes in comparison to their White peers (Kozlowski, 2015). Specifically, racially and ethnically minoritized students are less likely to succeed academically, report a strong sense of school belonging, and develop positive attitudes toward education (Ford & Moore, 2013). These negative outcomes impact more than their educational careers, as racially and ethnically minoritized youth, when compared to their White peers, go on to experience racism and discrimination (Leath et al., 2019), less access to mental health supports (Snowden, 2012), lower quality of life (Jones et al., 2020), higher mortality rates (Smedley, 2012), and lower income (Myers, 2009) throughout adulthood. Prior research has suggested that one of the underlying mechanisms for why these poor outcomes persist is the opportunity gap, by which White students incur more opportunities compared to their Black and Brown counterparts, which results in more favorable outcomes (Flores, 2007). School psychologists are well-suited to address these perpetuated racial disparities given their training in consultation, a systematic problem-solving process in which consultants (e.g., school psychologists) and consultees (e.g., teachers) collaborate to resolve client (e.g., student) problems (Bergan & Kratochwill, 1990). Through this expertise in consultation, school psychologists are apt to address when racially and ethnically marginalized students are provided fewer opportunities than their White peers. School psychologists are In U.S. schools today, it is likely that consultation occurs between White school psychologists and teachers, and students from racially and ethnically marginalized backgrounds (Egalite et al., 2015; Walcott et al., 2018). This is because there is an increasingly diverse student population, with more than 50% of 1 students identifying as students of color (National Center for Education Statistics, [NCES], 2019). Multicultural consultation explicitly acknowledges the influence of historical marginalization on the members of the consultation triad (i.e., school psychologists, teachers, and students) particularly as it relates to the variety of cultural backgrounds amongst them (Ingraham, 2003). In this consultation model, the consultant must intentionally incorporate the cultural backgrounds of all members of the consultation triad throughout the problem-solving process. This approach differs from other consultation models, such as behavioral (Bergan & Kratochwill, 1990) and instructional (Rosenfield, 1987), because of its emphasis on the potential influence of racial, ethnic, and cultural differences on consultation outcomes (Behring & Ingraham, 1998). Utilizing this consultation model in schools is critical. For instance, recent United States Supreme Court decisions repealing affirmative action (Regents of the University of California v. Bakke, 1978) and banning educational practices related to Critical Race Theory (Bell, 1995; Stefancic and Delgado, 2010) will have implications for narrowing the opportunity gap for years to come. Multicultural consultation, particularly multicultural consultee-centered consultation (MCCC), provides the common theoretical frameworks for the three papers in this dissertation. The first paper is titled “The Role of Racial Match Between Students and Teachers in School-Based Consultation” and is published online first in School Psychology Review. Chapter 2 of this dissertation includes a preprint of the article. This study examined how racial match between students and teachers influences three consultation outcomes, mainly collaboration, teacher expectations, and student-teacher relationship quality. The study answered the following research question: To what extent do student race, teacher race, and racial match influence the 2 perceptions school psychologists have of collaboration with the teacher, perceptions of teacher expectations of students, and perceptions of the quality of student-teacher relationships in the context of school consultation? 83 participants watched brief videos of teachers describing a student's concerns, which reflects the way student problems are often identified in schools. Then, participants were assessed using a standardized measure of collaboration (Johnson et al., 2016), teacher expectations (van den Bergh et al., 2010), and the student-teacher relationship (Pianta & Nimetz, 1991). The videos varied by student and teacher race, and referral concern (i.e., behavior and reading). Participants were randomly assigned to one of four conditions, two of which reflected a racial match between the teacher and student (White teacher, White student; Black teacher, Black student) and two of which reflected a racial mismatch between the teacher and student (White teacher, Black student; Black teacher, White student) conditions. Multivariate analysis of variance (MANOVA) was used to examine the main and interaction effects between student and teacher race on school psychologists’ perceptions on the three outcome measures. Results suggested that variations between student race and teacher race influenced school psychologists’ perceptions of the consultation process. Given that perceptions differed across conditions, recommendations included using multicultural consultee-centered consultation to reduce the potential influence of implicit bias in the consultation process when working within a consultation triad where its members are racially diverse. The second paper is titled, “What’s Race Got to Do With It? Relationships Between Racial Match and School Psychologists’ Perceptions in School-Based Consultation.” Chapter 3 of this dissertation includes the full manuscript. Paper 2 analyzed the data collected for Paper 1 to investigate the relationship between racial match and the perceptions school psychologists have of collaboration, teacher expectations, and student-teacher relationship quality. This study 3 looked at how consultants’ representations (i.e., perceptions and the interactions between them) of student problems varied across racial match conditions. The following research questions were answered: (1) To what extent are consultants’ representations of collaboration, teacher expectations, and student-teacher relationship quality correlated? (2) To what extent do racially matched (White teacher-White student, Black teacher-Black student) and racially mismatched (White teacher-Black student, Black teacher-White student) teachers and students affect consultants’ representations of collaboration, teacher expectations, and student-teacher relationship quality? And (3) To what extent is self-reported cultural competence associated with consultants’ representations of collaboration, teacher expectations, and student-teacher relationship quality? Multiple regression analyses were used to determine the relative contribution of racial match/mismatch, cultural competence, and the three representation variables (i.e., collaboration, teacher expectations, and student-teacher relationship). Consultants’ representations of the problem varied based on the racial diversity in the consultation triad, with racial match between a White teacher and a White student contributing to the variability in ratings of the collaborative and student-teacher relationships. We provided implications for school-based practitioners regarding how MCCC might help them become aware of how race influences their representations of student problems. The third paper is titled, “How Does Culture Fit into De-Implementation? A Scoping Review of Empirical Research.” Chapter 4 of this dissertation includes the full manuscript. This study was a scoping review of empirical research related to de-implementation (i.e., the discontinuation of harmful or ineffective practices; Walsh-Bailey et al., 2021) and how de- implementation relates to disparate outcomes for racially and ethnically minoritized populations. The purpose of this study was to investigate how prior de-implementation literature incorporated 4 culture, and various cultural factors (e.g., race/ethnicity). It is unknown how school psychologists might have chosen to intervene after seeing the teachers describing student concerns in Papers 1 and 2. It is often the responsibility of a school psychologist to determine the appropriate intervention for students’ needs. To do so, they must recognize how some evidence- based programs (EBPs) were not developed with students of color in mind and may need to be de-implemented. Thus, we reviewed empirical research related to the de-implementation of EBPs in education and related fields (e.g., healthcare). Then, we made explicit connections regarding how de-implementation research can be applied across settings, particularly within healthcare and education. Finally, this study explored how consultants can use MCCC to address de-implementation and promote equitable educational outcomes for racially and ethnically minoritized youth. In sum, this dissertation involves three papers, each of which explores how racial diversity amongst members of the consultation triad might be recognized and supported using multicultural consultation. The references for all chapters of this dissertation can be found at the end, separated by chapter. Appendices for all chapters can also be found at the end, separated by chapter. 5 CHAPTER 2 (PAPER 1): THE ROLE OF RACIAL MATCH BETWEEN STUDENTS AND TEACHERS IN SCHOOL-BASED CONSULTATION Abstract Disparities between the educational outcomes of students of color and their White peers have persisted across decades. Multicultural consultee-centered consultation has the potential to mitigate negative outcomes for students of color by improving the knowledge and skills of teachers. However, more empirical research is needed to understand the role of diversity within the consultative relationship and problem-solving process. The purpose of this study was to examine the effects of racial diversity between students and teachers during school consultation on the perceptions school psychologists have of collaboration, teacher expectations of students, and student-teacher relationships, as school psychologists commonly serve as school-based consultants. The study employed an experimental design, in which 83 practicing school psychologists were randomly assigned to watch videos of a teacher describing a student referral problem in a racial match or racial mismatch condition. Results suggested school psychologists’ perceptions of the collaborative relationship were more positive when students and teachers were Black compared to when they were White. Implications and suggestions for future research are discussed. Keywords: multicultural consultee-centered consultation, racial match, school-based consultation, collaboration, teacher expectations, student-teacher relationship 6 In the United States, students of color consistently fall behind their White peers (Kozlowski, 2015), with lower levels of academic achievement, poorer evaluations of classroom behavior, and lower rates of high school graduation and college enrollment (i.e., achievement gap; Ford & Moore, 2013). This inequity can be attributed, in part, to teacher perceptions, including lower expectations for Black students compared to White students, lower quality student-teacher relationships between Black students and White teachers, and unfavorable perceptions of Black students resulting in increased disciplinary referrals (Bates & Glick, 2013; Edwards et al., 2019; Pianta & Nimetz, 1991). Therefore, teacher perceptions play a critical role in student outcomes and achieving racial equity in schools. Examining these perceptions and the role of race in educational contexts is timely given increased awareness of social inequities and discrimination against marginalized populations through the expansion of the Black Lives Matter movement which began in 2014 (Taylor, 2016), healthcare disparities highlighted by the COVID-19 pandemic (Bambra et al., 2020), and the regulation of federal policies that provide additional funding to schools in order to close the achievement gap (Hunter & Bartee, 2003). As such, it is essential to understand the myriad of factors that influence teacher perceptions, including both students’ and teachers’ race. We use the term race to refer to “physical differences that groups and cultures consider socially significant” (i.e., Black or White perceived race; American Psychological Association, 2020, p. 142). Research has suggested that when students and teachers are the same race (i.e., racial match between a Black student and Black teacher; Oates, 2003), then students tend to experience more favorable outcomes. Redding (2019) conducted a systematic review of 37 studies and found that when Black students were racially matched with their teachers, they received more positive behavior ratings, had higher academic achievement, lower rates of exclusionary discipline and school dropout, greater 7 assignment to gifted/talented programs, and better attendance. However, some studies did find neutral or negative effects of racial match on the previously mentioned outcomes (e.g., Pigott & Cowen, 2000). Overall, the largest effects of racial match have been noted in perceptions of Black students by White teachers, when compared to perceptions of Black students by Black teachers (Downer et al., 2016), thus this study further examines disparities between perceptions of Black and White student-teacher dyads, although students from a variety of diverse backgrounds may be racially matched or mismatched with the adults that educate them. Consultation is one potential mechanism for working with teachers to improve the outcomes of racially minoritized students (Ingraham, 2000) and occurs when consultants (e.g., school psychologists) and consultees (e.g., teachers) engage in a collaborative problem-solving process to improve a client’s (e.g., student’s) academic, behavioral, or social-emotional outcomes (Barrett et al., 2017). These individuals (i.e., students, teachers, and consultants) make up the consultation triad, all of whom may vary in their race (see Figure 1 in the Online Supplemental Materials). Put another way, the consultation triad can involve various forms of racial match or mismatch among students, teachers, and consultants (e.g., racial mismatch between the student and teacher, with racial match between the teacher and consultant; Ingraham, 2000). There are various approaches to consultation, such as behavioral (Bergan & Kratochwill, 1990), conjoint-behavioral (Sheridan, 1997), instructional (Rosenfield, 1987), mental health (Caplan & Caplan, 1993), and multicultural (Ingraham, 2000) models that have distinct orientations toward collaboration, attention given to consultees’ perspectives and learning (e.g., consultee-centered; Hylander, 2012), and structure of the problem-solving process (Barrett et al., 2017). For example, behavioral consultation employs an expert approach in that the consultant often directs the problem-solving process (Bergan & Kratochwill, 1990); whereas 8 mental health focuses on nonhierarchical relationships through which activities may be collaboratively developed, implemented, and analyzed by both the teacher and consultant together (Caplan & Caplan, 1993). However, relationship building, which is generally defined as the collaborative bond between the teacher and consultant (Johnson et al., 2016), is the foundation of the consultation process, and is emphasized across all models, because the relationship developed during this process influences both the student and the teacher (Newman & Ingraham, 2017). Consultation models also vary in their approaches to diversity. Multicultural consultee- centered consultation is defined as a school-based consultation model that explicitly incorporates factors of diversity (e.g., race) in the problem-solving process and the teacher’s perception and skills in working with a diverse range of students (Ingraham, 2000; Ingraham, 2017). It emphasizes the implications of racial, ethnic, and cultural differences among the consultation triad (Behring & Ingraham, 1998). This model has several key components, (1) consultant learning and development (2) consultee learning and development (3) cultural variation in the consultation constellation (4) contextual and power influences and (5) supporting consultee and client success (Ingraham, 2000). It is most likely that school-based consultation will involve a racial match between White teachers and White consultants, and a racial mismatch between students of color, White teachers and/or White consultants (Walcott & Hyson, 2018). This is due to the racially homogenous population of school psychologists and the teacher workforce (Egalite et al., 2015), and the growing racial diversity of students in schools (National Center for Education Statistics [NCES], 2019). Multicultural consultee-centered consultation is therefore particularly well-suited to address disparate outcomes among students of color that may result from negative perceptions from educators in the triad. In this manner, consultants are in a unique 9 position to facilitate a conceptual shift among teachers through the teacher-consultant relationship (Hylander, 2012). Although much work has been done on the importance of cultural competence and culturally responsive practice (Ingraham, 2017), the individual contribution of race alone in the consultation context has yet to be fully examined. The purpose of this experimental study was to address this gap in the literature and examine the effects of student race, teacher race, and racial match between students and teachers on the perceptions consultants have of collaboration, teacher expectations, and the quality of student-teacher relationships during the problem-solving process. Consultants’ perceptions are important to examine because they may influence how the consultant conceptualizes the student concern, how they help teachers conceptualize the student concern, and how they subsequently guide the focus of the consultation sessions. This study makes a scientific contribution to the field in that it is the only experimental study conducted since 2000, to the best of our knowledge, to explicitly examine the influence of racial differences within the school-based consultation context. Theoretical Frameworks In this study, we examined racial variations among the consultation triad, but recognize that race, ethnicity, and culture may be influenced by each other and the broader societal context (Ingraham, 2000). Research has frequently conflated these constructs (Quintana et al, 2006), even using the terms interchangeably for several decades (Betancourt & Lopez, 1993). Additionally, recent research has examined intersectionality, the influence of multiple identities on interactions with social environments (Crenshaw, 1989), and describes the intersection of race, culture, language, and disability that impacts education services for students of color (Blanchett et al., 2009). This study seeks to examine the influence of race in the context of 10 consultation to provide insight into the particular strategies consultants can use to make the consultation process more effective, as perceived racial differences have been shown to influence perceptions of trustworthiness (Gordon et al., 2006) and the quality of services received in related human-services settings (e.g., healthcare; Gregg & Saha, 2006). The majority of research in education has examined racial inequities as a result of interactions between students and teachers. A systematic review of 37 studies examining racial match between students and teachers indicated that teachers rated students’ behavior more positively when racial match was present within the relationship. Moreover, results showed that Black and Latinx students had higher academic achievement when they were paired with Black and Latinx teachers respectively (Redding, 2019). Additional research has suggested that teachers’ implicit racial bias may contribute to disproportionate educational outcomes for White students in comparison to students of color (Staats, 2016). Implicit racial biases are stereotypes about racial groups held by individuals at an unconscious level (Peck et al., 2013), which may manifest during the consultation process. Implicit racial biases are developed by stereotypes regarding members of various racial groups and activated when individuals encounter members of those races (Staats, 2016). These implicit racial biases are automatic, operate at an unconscious level, and are one component of the anti-blackness present in education within the Unites States (Marcucci, 2020). When educators are faced with time constraints, lack information necessary to think critically about a situation, or are experiencing mental fatigue, unconscious implicit biases are more likely to impact decisions and perceptions (Staats, 2016). Implicit racial bias is most commonly examined among teachers about students, but may also occur between consultants and teachers, and consultants and students. 11 Although implicit racial bias is unconscious, people do have the ability to reflect and modify their thought processes (Staats, 2016). Multicultural consultee-centered consultation encourages this self-awareness and self-reflection so that consultants can employ intentional strategies to mitigate the negative impacts of implicit racial bias (Ingraham, 2000). For example, this may involve reflection on ones’ own strengths and areas of improvement (Ingraham, 2003), in order to increase objectivity of the problem (Hylander, 2012). Further, consultants might explicitly discuss race and racial identity, and their relation to the student’s education in order to mitigate any potential implicit biases, stereotypes, or assumptions that the teacher or consultant may hold. Finally, by using these strategies, consultants recognize that when members of the consultation triad share the same race, this does not necessarily mean these individuals will have the same worldview, cultural values and beliefs, or perceptions (Ingraham, 2000; Redding, 2019). In sum, two theoretical mechanisms may underlie the effects of race within school consultation: implicit bias and explicit or intentional use of culturally responsive consultation strategies. These may be more or less salient in consultation, related to variations in the racial composition of the consultation triad (see Figure 1 in the Online Supplementary Materials), the consultation model employed (i.e., Multicultural consultee-centered consultation), individual characteristics (e.g., identity, intersectionality), and the broader societal context (e.g., Black Lives Matter movement). For example, the activation of a consultant’s implicit racial biases about a teacher may be more likely in consultation triads where the teacher and consultant are racially mismatched, thereby decreasing the quality of the teacher-consultant relationship. In another example, in consultation triads where the student and teacher are racially matched, but the consultant is not, the consultant may employ self-awareness and self-reflection strategies to 12 address implicit biases that they may have about the student or teacher, which may improve consultation outcomes. Literature Review Very few empirical studies have been conducted to examine the influence of race in school consultation. Three studies, all of which were conducted over 20 years ago, were found in the review of the literature and are summarized next. First, Naumann and colleagues (1996) employed experimental methods to examine the role of racial match between consultants and students on preservice teachers’ perceptions of consultant credibility and intervention acceptability. They found that race appeared to have no impact on preservice teachers’ perceptions of consultant credibility or intervention acceptability. In addition, Rogers (1998) also employed experimental methods, with undergraduate students, to examine how race and consultant verbal behavior influenced perceptions of consultant competence and multicultural sensitivity. Results indicated that White consultants that addressed race (i.e., race sensitive; one strategy promoted through Multicultural consultee-centered consultation) were perceived as more competent and multiculturally sensitive, on average. However, Black participants found consultants who were race blind (race not addressed) to be more competent and multiculturally sensitive, regardless of consultant race. Additionally, White consultants were rated as more competent than Black consultants in race sensitive conditions, while Black consultants were rated as more multiculturally sensitive than White consultants in race blind conditions. Finally, Ingraham (2000) described cross-cultural cases in which there was a racial match between the student and consultant, but the teacher was of a different racial background. In these cases, teachers experienced “intervention paralysis” and were fearful of making culturally inappropriate recommendations (p. 333). Thus, their self-confidence decreased, and they became 13 increasingly dependent on the consultant’s understanding of their shared identity with the student. In contrast, other teachers became defensive and projected preconceived stereotypes about the student and consultants’ race into the working relationship, prompting the consultant to use strategies to increase teacher objectivity and build rapport with the teacher. In this manner, Multicultural consultee-centered consultation was particularly useful to address teachers’ preconceived beliefs (e.g., implicit biases) that prevented them from clearly conceptualizing student problems (Hylander, 2012). Current Study This study built upon the literature in several important ways. First, the empirical studies conducted in the consultation context were conducted over 20 years ago and there has been a significant shift in U.S. culture related to improving racial equity through systemic change (American Psychological Association, 2020). Therefore, it is important to conduct research to gain an updated understanding of the role of race and racial match in contemporary school-based consultation. Second, none of the studies included in-service school psychologists, who typically engage in school-based consultation. Graduate or undergraduate student samples, as were used in the prior studies, may not generalize to in-service school psychologist perceptions or their indirect service delivery. This study was the first to empirically examine in-service school psychologists’ perceptions in the context of school consultation, which may have important implications for practice. For example, if school psychologists perceive the student-teacher relationship to be weak, then they may guide the process toward addressing or improving that relationship. This study answered the following overarching research question: To what extent do student race, teacher race, and racial match influence the perceptions school psychologists have 14 of collaboration with the teacher, perceptions of teacher expectations of students, and perceptions of the quality of student-teacher relationships in the context of school consultation? Based on prior empirical research and theory, which provides general support for positive effects of racial match (i.e., Redding, 2019, at the student-teacher level; Rogers, 1998 and Ingraham, 2000, at the teacher-consultant level), we tested the following hypotheses: When there is a racial match between the student and teacher, and the teacher and consultant, school psychologists will report (a) more positive perceptions of collaboration (b) more positive perceptions of teacher expectations and (c) more positive perceptions of the quality of the student-teacher relationship. Method We conducted a true experiment, in which participants (i.e., practicing school psychologists) were randomly assigned to watch videos in one of four possible conditions: (1) a White teacher describing a referral problem about a White student, (2) a White teacher describing a referral problem about a Black student, (3) a Black teacher describing a referral problem about a White student, or (4) a Black teacher describing a referral problem about a Black student. Participants watched two different videos within each condition, one of which was of a teacher describing a reading problem and the other of a teacher describing a behavior problem (see Procedure for additional details). Then, participants responded to items measuring the dependent variables (i.e., perceptions of collaboration with the teacher, teacher expectations of students, and the quality of student-teacher relationships described in detail in the Measures section). The study was approved by the [blinded] Institutional Review Board (IRB) prior to data collection. Data were collected in between August 2020 and December 2020, which coincided with the Black Lives Matter protests that began globally in May 2020. 15 Recruitment and Participants This study used two sampling methods for recruitment: (1) a random sample of one thousand National Association of School Psychologists (NASP) members through a mailing list and (2) four school districts from each state were sampled from the Common Core of Data (CCD) from the National Center for Education Statistics (N = 200 districts). The largest district in each state was selected first in order to increase the potential sample size and the remaining three districts in each state were sampled at random to increase generalizability and minimize the overrepresentation of larger districts. Districts were asked to send the survey to their currently employed school psychologists. When the largest district declined, the next largest district in the state was contacted. When a randomly sampled district declined, another randomly sampled district in the state was contacted. In this manner, sampling occurred without replacement in that the same district could not be sampled more than once. These recruitment strategies were chosen to increase generalizability through random sampling (i.e., NASP recruitment and random sample of CCD districts) and sample size (i.e., the largest CCD districts). Both recruitment methods provided potential participants an anonymous link to the survey via Qualtrics, a description of the study, and informed consent. In order to participate in the study, participants met the following inclusionary criteria: (a) currently practicing full-time, as a school psychologist, in a public-school district at the time of the study and (b) had received a graduate degree in school psychology, excluding related fields such as special education. Participants were not provided financial compensation for their participation but were informed that a $1 donation was made to the NASP Minority Scholarship Fund for each survey response received. 16 The NASP recruitment method yielded 57 participants and the CCD recruitment method yielded 59 participants. Preliminary analyses indicated the two samples were not significantly different in terms of race, gender, education level, age, or years of experience (p values > .26) and were, therefore, combined for the remaining analyses. A total of 33 participants were excluded because they discontinued the survey immediately after responding to the inclusion criteria, did not watch any of the videos, and did not complete the outcome measures. This yielded an analysis sample of 83 participants. Participants were mostly White (95%), worked in suburban locations (43%), elementary settings (57%), had more than 11 years of experience (59%), had a Specialist’s level degree (45%), and a slight majority had their National Certification in School Psychology (52%). The sample was not significantly different from the population of NASP members described by Walcott and Hyson (2018) in terms of race, gender, education level, or age (p values > .08). Table 1 presents additional sample demographics and information about the participants’ work settings. Measures Teacher-consultant relationship. A subscale from the Coach-Teacher Alliance Scale (Johnson et al., 2016) was modified to address the collaborative, working relationship between the teacher and consultant. The subscale included 5 items, such as “The teacher and I could work together collaboratively.” Participant responses were on a 5-point frequency scale (1 = Never to 5 = Always). Subscale scores were calculated as the arithmetic mean of the 5 items (α = .88 for original; α = .84 for reading videos, α = .92 for behavior videos). Teacher expectations. The teacher expectations scale from van den Bergh et al. (2010) was adapted to identify perceptions of teacher expectations of a student’s performance, ability, and level of educational attainment. Participants completed 6 items (e.g., “The teacher seems to 17 think [student] is a smart student.”), but 1 item was excluded because of the high rate of missingness on the item in both the behavioral and academic vignettes (i.e., “The teacher seems to think [student] will have a high score on the final school achievement tests;” Behavior = 40% missing; Reading = 57% missing). Participant responses were on a 5-point Likert scale (1 = Not applicable to 5 = Totally applicable). Subscale scores were calculated as the arithmetic mean of the 5 items ( α = .97 for original; α = .72 for reading videos, α = .68 for behavior videos). Student-teacher relationship. The closeness and conflict subscales from the Student- Teacher Relationship Scale-Short Form (Pianta & Nimetz, 1991) were adapted for this study to measure perceptions of the quality of the student-teacher relationship. Three items measured closeness, such as “The teacher seems to share an affectionate, warm relationship with [student]” and 4 items measured conflict, such as “The teacher seems to easily become angry with [student]” (Mashburn et al., 2006). Participant responses were on a 5-point Likert scale (1 = Definitely does not apply to 5 = Definitely applies), with conflict items reverse coded for scoring. Subscale scores were calculated as the arithmetic mean of the 7 items (α = .85 for original; α = .84 for reading videos, α = .73 for behavior videos). Procedures First, two different vignettes were written, one describing a reading-focused problem and the other describing a behavior-focused problem (interested readers are invited to contact the first author for the scripts). Reading and behavior problems were both included because they are the most common types of referral problems school psychologists receive in schools, have been linked to disproportionality, and there is variability in how these problems are conceptualized (Benson et al., 2020; Skiba et al., 2011). For instance, reading problems can be viewed as the result of inappropriate curriculum and instruction, a lack of cognitive strategies and skills (e.g., 18 phonological awareness; Afflerbach et al., 2013), or insufficient exposure to reading before school-age (McCardle et al., 2002), all of which may be related to the cultural context of both students and teachers. As another example, research suggests that behavioral concerns in schools are often prone to implicit bias and subjectivity, requiring a judgment to be made by the teacher, and the perceived severity of these concerns may depend on the students’ gender, race,ethnicity, or school climate (Girvan et al., 2017; Morris & Perry, 2017). The videos were created using computer software (Zoom) and lasted approximately 2 minutes. In the videos, an acting “teacher” read the scripts describing the student concerns. Four teachers were in the videos, two of whom were White and two of whom were Black. The four acting teachers were similar in terms of age, socioeconomic status, gender (i.e., cisgender women), and wore similar business casual clothing. All of the teachers were women because of the high proportion of women in the elementary teacher workforce (NCES, 2021) and to control for the potential influence of teacher gender. In order to make the student’s race more salient within the videos, student names (Cody, Greg, Darius, and Jamal) were selected based on previous research identifying the perceived ethnic and non-ethnic typicality of these chosen names (Bertrand & Mullainathan, 2004; Noltemeyer et al., 2012). Across all scripts, the student was consistently a boy, as boys are more likely to be referred for special education services (Wehmeyer & Schwartz, 2001) and to control for the potential influence of student gender. Each teacher described a White student and Black student, as well as both reading and behavior problems. The reading and behavior videos were counterbalanced to reduce carryover or priming effects across vignettes (see Figure 2 in the Online Supplementary Materials; Kazdin, 2016). Students’ and teachers’ cultures were controlled for in that students’ culture (e.g., family 19 characteristics) was described identically across student race and teachers made no references to their own cultural backgrounds. There were no systematic differences across conditions, except for student race and teacher race, as teachers read identical scripts in the videos. However, there may have been differences in the acting teachers unrelated to the variables of interest in the current study (e.g., nonverbal behavior). Despite those potential differences, it is likely that variability in how participants responded to the dependent variables across conditions can be attributed to differences in student race and teacher race opposed to the referral concern or the order of the vignettes. Then, the survey was developed in Qualtrics. Cognitive interviews were conducted with 3 graduate school psychology students to understand how items on the survey might be interpreted by participants, to gain insight into the reliability and validity of the measures presented within the survey, and to receive feedback on the quality of the survey (Furr & Bacharach, 2017; DeSimone & LeFloch, 2004). After feedback was incorporated, data collection began. Prior to watching each video of the teacher describing the referral problem, participants saw a slide providing the name of the student, student race, student sex, student grade level, and name (a pseudonym) of the teacher. Then, participants watched the video of the teacher reading the script that described the referral problem. After each video, participants answered the following items: (1) What was the race/ethnicity of the student in the video? (2) To the best of your knowledge, what was the race/ethnicity of the teacher in the video? (3) What was the student’s name? (4) What was the student’s sex? (5) What was the student’s grade level? (6) What most accurately summarizes the referral problem described in the video? Participants were not asked to recall the teachers’ names. These items served as manipulation checks to determine the extent to which participants 20 could recall the information presented and if they were aware of the race of the student and teacher in the video (Naumann et al., 1996). If participants did not answer the manipulation check about both student and teacher race correctly, then we cannot infer that their responses on the dependent variables can be attributed to student and teacher race; those participants were excluded from subsequent analyses. This process was consistent across both reading and behavior vignettes. After the manipulation checks, participants filled out the subscales measuring the dependent variables. Data Analysis Two-way multivariate analysis of variance (MANOVA) was used to answer the research questions examining the main effects of teacher race, student race, and the interaction between the two (i.e., racial match and racial mismatch) on school psychologists’ perceptions of the collaborative relationship, teacher expectations, and the quality of the student-teacher relationship. Two separate two-way MANOVAs were conducted, one for responses to the reading vignette and the other for responses to the behavior vignette. In order to keep the family- wise Type I error rate at 5% (α = .025), a Bonferroni adjustment was applied (Cohen, 1988), resulting in α = .0125 (.025/2) for the omnibus test in each MANOVA. Finally, a post-hoc power analysis was conducted after data were collected to determine the minimum detectable effect size. Results indicated the study had sufficient power to detect a minimum effect size of 0.16 (d; power = .80, N = 83, α = .0125). Results The means, standard deviations, skew, and kurtosis values for the three dependent variables are presented in Table 2. Preliminary analyses were conducted to ensure the data met assumptions for MANOVA. All variables were normally distributed based on skew and kurtosis 21 values (see Table 2) and visual analysis of Q-Q plots. Homogeneity of variance was assessed using Levene’s test, with results indicating no significant differences in variances across conditions. Table 3 presents bivariate correlations between the variables, which indicated there were no substantive issues with multicollinearity (r < .90; Tabachnick & Fidell, 2012). Responses to the manipulation checks indicated that the majority of participants remembered the vignettes accurately and were aware of the race of the student and teacher (student race = 88% correct, n = 73; teacher race = 86% correct, n = 71; student name = 100% correct, n = 83; student grade = 92% correct, n = 76; student sex = 100% correct, n = 83; referral problem = 99% correct, n = 82). However, participants were less likely to accurately remember the students’ race and teachers’ race, compared to other facts about the student and referral problem (χ2 = 47.82, df = 1, p < .001; χ2 = 41.94, df = 1, p < .001). A Kruskal-Wallis H test found there were no statistically significant differences in the percentage of accurate responses to the manipulation checks across the four conditions (p values > .53), indicating that student race and teacher race did not influence participants’ accurate recall of the manipulation checks. Lastly, the demographics of the student population (e.g., diversity) within participants’ school districts or their years of experience did not influence responses for reading or behavior problems (p values > .20). Table 4 presents the omnibus test for the reading-focused referral problem, which indicated there were significant differences across one outcome variable. The low representation of participants of color (N = 4) allowed for partial examination of the first hypothesis, in which there was one type of racial match (i.e., White teachers and White participants) and one type of racial mismatch (i.e., Black teachers and White participants). Results indicated there were significant main effects for teacher race, such that participants, almost all of whom were White, 22 rated higher levels of collaboration when teachers were Black (p = .007; d = .77; see Figure 3 in the Online Supplementary Materials). Further, there were no significant main effects for student race or teacher race on perceptions of teacher expectations. Finally, results indicated there were no significant main effects for student race or teacher race on perceptions of the student-teacher relationship. For all three outcome variables, there were no significant interaction effects and, therefore, the hypotheses positing that perceptions of collaboration, teacher expectations, and the quality of the student-teacher relationship would differ between racial match and racial mismatch conditions were not supported. Table 4 also presents the omnibus test for the behavior-focused referral problem. Results indicated there were significant differences in ratings of collaboration across conditions, but no significant differences in teacher expectations or the student-teacher relationship. Similar to the reading-focused referral problem, there were significant main effects for teacher race, such that participants rated higher levels of collaboration when teachers were Black (p < .001; d = 1.10; see Figure 3 in the Online Supplementary Materials). However, unlike in the reading-focused vignettes, results indicated there were also significant main effects for student race, where participants rated higher levels of collaboration when students were Black (p < 0.001; d = 1.01; see Figure 3). Finally, there was a significant interaction effect between student and teacher race, such that participants reported lower levels of collaboration when both students and teachers were White (p = .001; d = .10 to 2.60; see Figure 4 in the Online Supplementary Materials). Discussion The current study examined how racial match between students and teachers impacted the perceptions school psychologists have of collaboration, teacher expectations, and the quality of the student-teacher relationship. With the increase in the racial diversity of the school-aged 23 population (NCES, 2019), school psychologists will need to engage in consultation where the consultation triad (i.e., student, teacher, and consultant) is increasingly diverse; thus, it is important to understand the role race plays in this process. Additionally, considering that school- based consultation is a mechanism by which marginalized populations can achieve more equitable outcomes (Barrett et al., 2017), and racial match has been known to play a role in educational contexts (Redding, 2019), it is paramount to understand how these factors influence the consultation process in schools. The current study demonstrated that variations between student race and teacher race did, in fact, influence school psychologists’ perceptions of the consultation process, providing updated empirical evidence of the importance of racial diversity in consultation. First, it is important to note that although there was a low representation of school psychologists of color in the sample (N = 4), it was representative of the population of NASP members (Walcott & Hyson, 2018). Although there have been efforts to diversify the field of school psychology in recent years (Grapin et al., 2016), this study highlights the need to further increase recruitment efforts of diverse individuals in order to be able to fully examine the influence of race in school consultation. Second, results from the manipulation checks indicated that participants were significantly less likely to accurately remember, or report, student and teacher race compared to the student’s name, grade level, gender, or referral concern. Within the context of Multicultural consultee-centered consultation, culturally responsive service delivery, and culturally responsive teaching, it is crucial to acknowledge and understand the racial identity of students and teachers. Individuals’ racial identities are as equally important as other identities (e.g., gender and grade level). In order to engage in Multicultural consultee-centered consultation, school psychologists must first acknowledge the race of students and teachers and 24 then ask open-ended questions during the problem identification session to gain a better understanding of how race may influence student outcomes (Ingraham, 2017). Then, consultants may collaborate with teachers to support culturally responsive practices, such as clearly modeling expectations and modifying communication patterns based on students’ racial backgrounds (Gay, 2018). In regard to collaboration, results indicated there was a main effect for teacher race across both reading and behavior vignettes, such that participants rated their level of collaboration with Black teachers more positively than White teachers. This aligns with Rogers (1998) findings and may be because the teachers did not discuss their own racial background and were seen as “race blind” (Rogers, 1998). This may have negatively impacted the participants’ responses regarding White teachers, due to a perceived lack of knowledge or skills (Ingraham, 2000); while Black teachers were seen as more competent and multiculturally sensitive (Rogers, 1998). Alternatively, participants may have been employing self-awareness or self-reflection strategies in order to address implicit biases that may have been activated while watching the video, subsequently rating Black teachers more positively than White teachers. Finally, participants may have overcompensated for their implicit biases, rating White teachers more negatively to align with what they may have perceived to be more socially acceptable (Marcucci, 2020). Furthermore, results indicated a main effect for student race in regard to perceptions of collaboration, such that participants rated collaboration more positively when the student was Black and described as having a behavioral concern. With this in mind, consultants might incorporate aspects of Multicultural consultee-centered consultation to examine how both individual and systemic issues influence their work with the teacher as they progress through the problem-solving stages (Ingraham, 2003). For instance, the consultant may ask the teacher 25 pointed questions to better understand their worldview (e.g., beliefs, values, and attitudes) and how they wish to support the student’s needs (Ingraham, 2017). The significant interaction effect between student race and teacher race for perceptions of collaboration in the behavior vignette suggests that racial match between students and teachers may influence the working relationship, at least when presented with behavioral concerns. Interestingly, results indicated that participants reported the weakest level of collaboration when all members of the triad were White (i.e., participants, students and teachers), which was in the opposite direction of literature supporting the benefits of racial match (e.g., Redding, 2019). It is possible that participants were aware of the complexities introduced when racial diversity was present and subsequently rated the teachers in this condition the lowest, as they may have expected teachers to be able to address behavioral problems themselves when they were racially matched to their students. Another potential explanation is that the participants overcompensated for implicit racial biases that were present in the remaining three conditions, thereby lowering their ratings in this condition, which has been described as one form of social desirability (Marcucci, 2020). In cases where all members of the consultation triad are White, consultants may still wish to use Multicultural consultee-centered consultation to establish a sense of “we-ness” (Ingraham, 2000, p. 335), where an area of common ground is created to foster the collaborative relationship. This may involve reflecting on the similarities, rather than differences, between oneself and the teacher (e.g., gender, professional status) in order to build rapport, and addressing beliefs, values, and perspectives given cultural variability within racial groups. When consultants have less positive feelings of collaboration with some teachers, it is important for them to engage in self-reflection lest a self-fulfilling prophecy influences their behavior in such a 26 way that would impair the effectiveness of consultation to the teacher going forward (e.g., be generally uncooperative; Edwards et al., 2019). Implications for Practice This study yields several important implications for practice. First, school psychologists’ differences in their perceptions of collaboration, after interacting with a teacher for just two minutes, suggests a need for practitioners to reflect on their own implicit biases within the consultation process. This self-reflective practice may help practitioners positively influence the consultation process in additional sessions with the teacher (Edwards et al., 2019), even when all members of the consultation triad are racially matched. This may include discussions about the teachers’ and consultants’ racial identities within consultation, which has proved beneficial at the problem-solving stage in previous research (Newell & Looser, 2018). Given that White teachers, who comprise a majority of the teacher workforce in schools (Egalite et al., 2015), were rated more poorly in terms of collaboration, this also necessitates increased efforts to build effective relationships. For example, school psychologists might reflect on the following questions to ensure they are able to meaningfully connect with the teachers they work with, regardless of race: (1) Do I understand the teacher’s perspective? (2) Do we have a shared understanding of the problem? (3) Do I respect the teacher’s knowledge and skills? (4) How can we create a trusting environment for our consultation sessions? (Ingraham, 2017). Second, when school psychologists believe racial differences may be impacting teacher perceptions, we suggest creating a space to have discussions about the potential implications of these misconceptions with the teacher. For example, school psychologists may open a discussion about racial diversity in the classroom environment by asking questions such as: (a) In what ways do your students identify with a particular race? (b) Do you have a good understanding of 27 the family backgrounds of your students? (c) How do your students’ races relate to their culture? (d) Do the cultural norms of your students differ from your own? (McKenney et al., 2017). In this manner, school psychologists might build the teacher’s capacity to affirm each student’s racial identity and understand how it relates to their unique culture, which has been associated with culturally responsive teaching (Gay, 2018). Limitations There are several strengths of the study, such as employing experimental methods in order to disentangle race and culture. However, there are limitations that must be noted. First, priming may have occurred during recruitment, such that participants might have been prompted to consider the racial injustices within our education system with reference to the NASP Minority Scholarship Fund donations in the recruitment email and informed consent. Second, there may have been selection bias, as school psychologists that agreed to participate may have been more interested in consultation compared to those that did not choose to participate. Moreover, participants’ experience, interest, and enjoyment in providing consultation services were not assessed but may have influenced results. Third, there was a low response rate among the NASP-recruitment sample, which may have been related to the COVID-19 global pandemic, and the response rate could not be calculated for the CCD-recruitment sample. Therefore, the extent to which the sample parameters reflect population parameters may be limited. Next, there may have been social desirability bias, such that participants’ responses reflected what they perceived to be the most appropriate response, opposed to what they honestly believed or what they would do in practice. Further, there may have been a historical threat to validity, as participants completed the study during or in close proximity to several national events which may have influenced responses (e.g., the Black Lives Matter movement, COVID-19 global 28 pandemic). Additionally, as this study focused on Black and White student-teacher dyads, we were unable to capture the influence of racial match among other racial variations (e.g., Latinx or Asian American students and teachers) which may generate different findings. Further, the teachers may have differed in related psychosocial characteristics, such as attractiveness or familiarity, which may have influenced the results. However, the within-teacher differences (e.g., the same teacher described both Black students and White students; the same teacher was assessed across several outcome measures), which were the crux of racial match and mismatch conditions, could be attributed to variations in teacher race, opposed to confounding teacher characteristics. Lastly, the study included simulated videos of teachers describing referral problems, which may not generalize to face-to-face consultation contexts as the consultation process occurs over time (not just during a two-minute teacher interaction) and research has suggested differences in emotional understanding in virtual contexts (Caridakis et al., 2008), which may influence perceptions of collaboration. However, the study may be applicable to teleconsultation (Fischer et al., 2016), which may have been common during the COVID-19 global pandemic. Recommendations for Future Research The consideration of race in school-based consultation is extremely important, yet there have been very few, if any, empirical studies explicitly examining the role of race in consultation conducted in the past 20 years. This study provides a jumping off point for researchers to explore the complexities of race, ethnicity, and culture, among other facets of diversity, in school consultation. We make a call for updated, high-quality research to understand diversity in consultation that employs a variety of methodological approaches. 29 Future studies might employ qualitative (e.g., interviews, case studies), mixed methods, or other types of quantitative research (e.g., observational, survey, randomized controlled trials) to gain a more comprehensive understanding of (1) how school psychologists’, teachers’, and students’ conceptualize race and how this influences consultation; (2) how other facets of diversity influence consultation (e.g., ethnicity, culture, gender, socioeconomic status, native language); (3) how school psychologists’ experiences with indirect service delivery influences their understanding of race within consultation, and (4) the types of strategies consultants can use to improve student outcomes in variations of racial match within the consultation triad. Future experimental studies may wish to include a visual representation of the student (e.g., video or observation) to provide participants with more information or use technology to manipulate the appearance of students and teachers in order to isolate additional psychosocial factors. Moreover, future replications and extensions of this experimental study may help to disentangle which findings from the current study were influenced by the historical context (i.e., Black Lives Matter movement, COVID-19 pandemic), and which findings were more representative of typical school-based consultation practice. These examples of experimental studies offer the ability to isolate variables of interest through the use of tightly controlled manipulations, but may be less generalizable to actual practice. Future research that observes or audiotapes multiple consultation sessions that occur over time may be useful to complement experimental studies, despite the potential for a large number of confounding variables at the student-level (e.g., referral concern), teacher-level (e.g., familiarity), and school-level (e.g., school policies, school culture). Future studies might also employ a variety of measures to address the potential for social desirability bias. For example, extensions of this study may wish to pair school psychologists’ 30 self-reported use of Multicultural consultee-centered consultation strategies with process logs, audiotapes, or observations of consultation sessions. Moreover, student or teacher reports of the extent to which school psychologists use Multicultural consultee-centered consultation strategies may provide additional information on how, when, and why school psychologists use such tools. For example, observations of the interactions between teachers and school psychologists during consultation sessions may give greater insight into their relationship, the influence of racial match or mismatch, and could account for socially desirable survey responses. Researchers may also consider conducting interviews with school psychologists, using both direct and indirect questioning (Fisher, 1993), to reduce the impact of social desirability. However, an interview may also be subject to social desirability bias given the nature of the relationship with the interviewer (e.g., racial match and mismatch between the interviewer and interviewee), school culture, and even time of year (e.g., end-of year workload). By combining multiple methods and measures to examine racial match and mismatch, researchers may be able to further address the limitations of self-reports due to social desirability bias. Finally, future studies may wish to oversample from under-represented groups in order to yield a larger sample of diverse school psychologists or engage in more targeted recruiting from particular subgroups, such as the NASP African American and Multicultural Committees. Further, snowball sampling methods for recruitment might be used, where members of these multicultural communities recruit other members for participation in the study (Goodman, 1961). However, we emphasize the need for school psychology as a field to diversify, so that more selective methods are not needed. 31 Conclusion As consultants uniquely positioned to serve marginalized populations (Newell & Looser, 2018), school psychologists have the ability to make meaningful change within educational systems. This study adds empirical support to the importance of understanding racial diversity in school-based consultation (Ingraham, 2017). Findings suggested that student race, teacher race, and the interaction between the two (i.e., racial match and racial mismatch) do play a role in school psychologists’ perceptions during the consultation process. Although advances in research and practice to further the consideration and understanding of race in school consultation will not be effortless, they are critical to acknowledge how racial differences influence classroom contexts and ensure that school-based consultation fulfills its potential to mitigate negative outcomes among students of color. 32 CHAPTER 3 (PAPER 2): WHAT’S RACE GOT TO DO WITH IT? RELATIONSHIPS BETWEEN RACIAL MATCH AND SCHOOL PSYCHOLOGISTS’ PERCEPTIONS OF SCHOOL-BASED CONSULTATION Abstract Multicultural consultee-centered consultation has the potential to improve outcomes for historically marginalized students given its intentional consideration of their unique backgrounds and strengths. In this approach, consultants aim to create shared understanding of the problem with consultees and help them conceptualize how multicultural factors might influence the problem. This change, or “turning point,” in their conceptualization aims to improve their ability to solve future problems on their own. However, more research is needed to determine how consultants themselves, as school psychologists, conceptualize student problems, especially when members of the consultation triad differ in their racial identities. The current study uses extant data from a larger study examining racial match in consultation (N = 83), in which participants were randomly assigned to see a teacher describing a student referral problem across four conditions: (a) White teacher describing a White student, (b) White teacher describing a Black student, (c) Black teacher describing a White student, and (d) Black teacher describing a Black student. Multiple regression analyses were used to determine the relationships between school psychologists’ perceptions of collaboration, teacher expectations, and student-teacher relationship quality. These analyses illustrated whether consultants’ representations of the problem differed based on the racial diversity of the consultation triad. Implications for research and practice are provided. Keywords: multicultural consultee-centered consultation, racial match, school-based consultation, perceptions 33 School psychologists frequently engage in consultative services to improve student outcomes (Rosenfield, 2014), and consultation is considered a critical professional competence across educational and clinical settings (National Association of School Psychologists, [NASP], 2020). Consultation occurs when consultants (e.g., school psychologists) and consultees (e.g., teachers) collaborate to solve clients’ (e.g., students’) problems (Bergan & Kratochwill, 1990). The consultant, consultee, and client form the consultation triad, all of whom can vary in their race/ethnicity and cultural backgrounds (Ingraham, 2000). Through consultation, the consultant and consultee engage in a collaborative, systematic process to identify a problem, collect data, plan an intervention, implement the intervention, and evaluate its effectiveness (Caplan & Caplan, 1993). Multicultural consultation has the potential to improve outcomes for historically marginalized students and narrow the opportunity gap (i.e., White children having more opportunities when compared to Black and Brown children; Flores, 2007). This approach specifically acknowledges the biases of consultation triad members, and it can improve teachers’ skills in working with racially and ethnically diverse students (Ingraham, 2017). Consultants themselves might have their own biases informing their worldviews, and multicultural consultation techniques promote self-awareness and reflection on these potential biases (Ingraham, 2017). Multicultural consultation is consultee-centered, which means that the consultee is ultimately responsible for client outcomes. Consultee-centered approaches are useful when the goal is to build consultees’ skills (Hylander, 2012). Multicultural consultee-centered consultation (MCCC) provides the theoretical framework for the current study in which we examine racial match and racial mismatch in school-based consultation (see Figure 1; Ingraham, 2000). Racial match occurs when members of the consultation triad share the same race while 34 racial mismatch occurs when there are racial differences between any members of the consultation triad (Oates, 2003). This multicultural consultee-centered framework purports that consultants and consultees engage in a learning process in which they develop knowledge and skills related to multiculturalism (e.g., one’s own culture, the culture of others). Then, they utilize this enhanced objectivity and confidence in multicultural consultation and intervention methods to support consultee and client success. However, cultural variations (i.e., racial match/mismatch) and factors related to the consultation context (e.g., the larger society) and power differences between the consultant and consultee can influence the extent to which consultant-consultee learning and development actually benefits consultees and clients. For example, when the consultant and client share an aspect of their culture (e.g., racial match), the consultee may fear making a mistake when working with the client (e.g., accidentally offending the client given the cross-cultural nature of the case; Ingraham, 2000). The current study draws from the differentiation made by Hylander (2012) between representations and presentations in consultee-centered consultation. Briefly, representations are the consultee’s thoughts and feelings about the problem, while presentations are how the consultee presents the problem to others (Hylander, 2012). We use the term representation to refer to an individual’s thinking about or feelings toward a problem, rather than the term “perceptions” used in prior research (e.g., Gutkin, 1980, 1986; Kaiser et al., 2009), to align with research focused on consultee-centered consultation (e.g., Hylander, 2000, 2003). The current study assumes that representations are made up of multiple perceptions that interact and influence each other. By conceptualizing representations in this way, the current study provides a novel approach to examining consultants’ thinking in school-based consultation. 35 Few empirical studies have examined the influence of racial match on the consultation process and no previous studies have examined how racial match influences consultants’ representations of student concerns. Consultants’ representations of the problem are critical to understand because they may influence how consultants subsequently engage in the consultation process. For example, consultants may focus on teacher-level change if they hypothesize that a racial mismatch between a student and teacher decreases teacher expectations of the student (Egalite et al., 2015). As another example, consultants may associate racial mismatch between a student and teacher with a lower quality student-teacher relationship, which can have negative effects on student achievement (Redding, 2019). In a final example, a consultant may believe that teacher expectations and student-teacher relationship quality are related, thus choosing to address one construct to influence the other. All of these conceptualizations may be points of intervention that the consultant chooses to address. The current study contributes to the literature by using a novel methodology to examine the relationships between racial match and school psychologists’ representations regarding collaboration, teacher expectations, and student-teacher relationship quality. Consultee-Centered Consultation There are four primary tenets of consultee-centered consultation: (1) a nonhierarchical relationship between a consultant and a consultee, (2) the consultee has responsibility for client outcomes, (3) the primary task of consultation is to reframe knowledge about, and solutions for, the problem, and (4) the goal is to develop a new way of conceptualizing the problem to build consultee skills and aid their professional development (Lambert, 2004; Newman & Ingraham, 2017). In general, a nonprescriptive approach is used within consultee-centered consultation to create a conceptual change in the consultee’s representation of the problem (Hylander, 2012). 36 The construction of the client’s problem is based on the consultee’s presentation to the consultant, or how the consultee presents the issue (Hylander, 2012). However, the consultee’s presentation may not always align with their representation of the problem, or how the consultee thinks and feels about the problem, as well as how prepared they are to act to address the problem (Hylander, 2000). Thus, the consultee may not be describing the true problem (i.e., the consultant may not know the consultee’s representation), which has implications for the entirety of the problem-solving process (Hylander, 2012). In order for the consultant to understand the consultee’s representation, the consultant must develop an environment in which a safe space, and positive interpersonal relationship, has been established for the consultee to share their thoughts and feelings about the problem. The consultee must feel confident in sharing their representation with the consultant (Hylander, 2012). Given the potential for cultural differences between consultants and consultees to influence the collaborative relationship and communication (Ingraham, 2000), it is critical to examine how presentations and representations of client problems might differ based on the diversity of the consultation triad. In addition, prior research has failed to fully examine consultants’ representations or how they think and feel about student problems. Next, we summarize the literature regarding how racial match influences the collaborative relationship, as well as prior research examining perceptions of consultants and consultees. The Influence of Racial Match on Collaborative Relationships A small but growing body of research has examined relationships between members of the consultation triad broadly (e.g., Ingraham, 2000) and the consultant-consultee relationship specifically (e.g., Johnson et al., 2016). Racial match or mismatch within the consultation triad may influence the collaborative relationship because communication styles can differ based on 37 the cultural backgrounds of each member (Gay, 2018). Consultants use communication strategies, such as paraphrasing and open-ended questions, to build these relationships and understand the consultee’s perspective (Rosenfield, 2004). Multicultural consultee-centered approaches may be useful when racial diversity is present within the consultation triad because the consultant can explicitly address and incorporate these communication-related differences (Ingraham, 2000, 2017). For instance, a consultant may perceive a teacher to be inattentive and unmotivated to collaborate for failing to make eye contact with them during consultation sessions, but in the teacher’s culture it may be considered inappropriate to make direct eye contact with authority figures (Gay, 2018). The extent to which consultants and consultees engage in consultation varies across school setting, and school level (i.e., elementary, middle, high) has been found to be associated with between-teacher variance in referrals for individualized academic (e.g., Shapiro, 2022), behavioral (e.g., Kaufman et al., 2009), and mental health services (e.g., Green et al., 2022). Across school level, prior research indicates that increases in teacher expectations and demands also occur (Weinstein et al., 1987). As a result, the current study includes work setting as a demographic characteristic. At any school level, consultants must understand potential cultural differences when working with consultees because these misunderstandings may influence the consultant-consultee relationship. As mentioned, these relationships can have implications for the consultant’s representations throughout the problem-solving process and the overall effectiveness of consultation. For example, the quality of the collaborative relationship may influence the consultee’s willingness to share their representations of the problem with the consultant (Hylander, 2012). 38 The Influence of Racial Match on Consultant and Consultee Perceptions Consultants’ perceptions are underexamined in the field of school psychology, despite their potential to have a tremendous impact on student outcomes. Most of the literature has examined consultees’ perceptions of consultants and consultative services. In regard to consultees’ perceptions, research has illustrated that racial match influences consultees’ perceptions of the consultant’s competence and utility of consultation. For instance, Naumann and colleagues (1996) found that consultee perceptions of consultant credibility and intervention acceptability varied based on racial match. The interaction between consultant race and verbal behavior, and how these influence consultee perceptions of consultant competence and multicultural sensitivity, has also been examined (e.g., Rogers, 1998). Further, consultees may perceive a consultant to have greater expertise when there is a racial match between the consultant and student, and racial mismatch between the consultee and student (Ingraham, 2000). In therapeutic contexts, racial and ethnic differences between consultants and consultees have been shown to negatively impact these relationships (Ramirez et al., 1998). Implications of this finding include relationship-building difficulties (e.g., Vontress, 1981), negative thoughts and feelings toward treatment (e.g., Atkinson & Lowe, 1995), and even termination of treatment (e.g., Sue & Sue, 1977). A more recent study by Clinkscales and colleagues (2022) found that racial match and mismatch between students and teachers influenced school psychologists’ perceptions of collaboration. In situations where student problems were related to behavioral difficulties, school psychologists rated collaboration more positively when students and teachers were Black, compared to when they were White. As such, racial diversity in the consultation triad has implications for various outcomes. Very few studies have examined consultants’ perceptions 39 throughout the consultation process or how they may be influenced by the racial composition of the consultation triad. The current study extends prior research by examining school psychologists’ representations of the collaborative relationship, teacher expectations, and student-teacher relationship quality. In addition, the field of school psychology has a limited understanding of how consultants’ cultural backgrounds influence their perceptions. Ingraham (2000) described the role of a consultant’s culture (one’s affiliation with certain values and beliefs) in consultation with White and Black students and teachers. In these cross-cultural contexts, the consultation process may be subject to the influence of implicit bias and stereotypes. Implicit biases are unconscious, or unknown, stereotypes held about various cultural groups and members of those groups (Staats, 2016). Since these implicit biases are automatic, they may manifest in the consultant’s perceptions during consultation (Ingraham, 1995) and are often a type of filter used to view situations, guide thoughts, and determine behaviors (Cushner & Brislin, 1997). Finally, in a qualitative case study of three MCCC cases, consultants shared their perceptions throughout the consultation process, how aspects of the case related to each other, and consultation outcomes (Ingraham, 2003). All three consultants found that their representation of the problem (e.g., the consultee’s lack of skills or the cultural mismatch between the student and the teacher) changed over time. In this manner, the particular point in each case where the consultants’ perceptions shifted or “turned” was documented. Current Study The current study adds to the scientific understanding of consultee-centered consultation in a few ways. First, a very limited number of empirical studies have examined consultants’ representations as they relate to providing consultation in school contexts. In a review of the 40 literature, no quantitative studies were identified that have investigated consultants’ representations. The few studies identified used qualitative methodologies, such as interviews and content analysis, and examined perceptions of the consultation process. Additionally, no prior studies used quantitative methods to understand how consultants’ representations of student problems may vary across racial match and mismatch groups. Quantitative approaches might complement our understanding of how consultants’ representations and racial match are related to each other and further the ability to conduct research on this topic. This study answered the following research questions: (1) To what extent are consultants’ representations of collaboration, teacher expectations, and student-teacher relationship quality correlated? (2) To what extent do racially matched (White teacher-White student, Black teacher- Black student) and racially mismatched (White teacher-Black student, Black teacher-White student) teachers and students affect consultants’ representations of collaboration, teacher expectations, and student-teacher relationship quality? And (3) To what extent is self-reported cultural competence associated with consultants’ representations of collaboration, teacher expectations, and student-teacher relationship quality? Method Data analyzed in this study were part of a larger experimental study on the role of racial match in school-based consultation (Clinkscales et al., 2022). Sample Participants included 83 school psychologists. They were mostly White (95%), worked in suburban locations (43%), elementary settings (57%), had more than 11 years of experience (59%), had a Specialist’s level degree (45%), and had their National Certification in School Psychology (NCSP; 52%). See Table 1 for the demographic characteristics of the sample. 41 Measures Collaboration. A subscale from the Coach-Teacher Alliance Scale (Johnson et al., 2016) addressed the collaborative, working relationship between the teacher and consultant. The subscale included 5 items, such as “The teacher and I could work together collaboratively.” Participant responses were on a 5-point Likert-type scale (1 = Never to 5 = Always; α = .88 for original; α = .84 for reading videos, α = .92 for behavior videos). Teacher expectations. The teacher expectations scale from van den Bergh et al. (2010) identified perceptions of teacher expectations of a student’s performance, ability, and level of educational attainment. The scale included 6 items, such as “The teacher seems to think [student] is a smart student.” The fifth item (“The teacher seems to think [student] will have a high score on the final school achievement tests.”) was removed due to missingness. Participant responses were on a 5-point Likert-type scale (1 = Not applicable to 5 = Totally applicable; α = .97 for original; α = .72 for reading videos, α = .68 for behavior videos). Student-teacher relationship. The closeness and conflict subscales from the Student- Teacher Relationship Scale-Short Form (Pianta & Nimetz, 1991) measured perceptions of the student-teacher relationship quality. Three items measured closeness, such as “The teacher seems to share an affectionate, warm relationship with [student],” and four items measured conflict, such as “The teacher seems to easily become angry with [student]” (Mashburn et al., 2006). Conflict items were reverse coded. Participant responses were on a 5-point Likert-type scale (1 = Definitely does not apply to 5 = Definitely applies; α = .85 for original; α = .84 for reading videos, α = .73 for behavior videos). Paired samples t-tests indicated there were no significant differences between the reading and behavior vignettes for collaboration (p = 0.76) and teacher expectations (p = 0.170). 42 However, there was a statistically significant difference between them for student-teacher relationship (p < 0.001). Thus, composite scores for collaboration and teacher expectations were calculated as the mean of all of the items across both vignettes. Student-teacher relationship composite scores remained separated for reading and behavior concerns and calculated as the mean of the four items for each vignette. Cultural competence. The Multicultural School Psychology Counseling Competency Scale (Rogers & Ponterotto, 1997) was adapted for this study to identify school psychologists’ self-perceptions of cultural competence in the consultation context. The scale included 10 items, such as “I am aware of my own cultural heritage and values.” Participants responded to items on a 5-point Likert-type scale (1 = Strongly disagree to 5 = Strongly agree;  = 0.88 for original;  = 0.42 for current study). Composite scores for cultural competence were calculated as the mean of all of the items across each vignette. Demographic information. Demographic data included age, gender, race/ethnicity, degree, school setting, and years of experience. School setting was coded as 1 = Elementary school; 2 = Middle school; 3 = High school; 4 = Early childhood/Preschool; 5 = Other). Elementary schools were used as the reference group. Procedure Participants were randomly assigned to one of four conditions: White teacher-White student (WW); White teacher-Black student (WB); Black teacher-Black student (BB); and Black teacher-White student (BW). Within each condition, participants watched two two-minute videos where an acting “teacher” described concerns about a 4th grade boy, resembling the problem identification session within school consultation. Four teachers were included in the videos (two White teachers and two Black teachers). In each video, a teacher read an identical script to 43 describe a White student and a Black student, as well as both reading and behavior problems. In one video, a teacher described a student’s reading problem, and in the other video, a teacher described a different student’s behavior problem. These videos were counterbalanced (Kazdin, 2016). Prior to each video, participants were given the name of the student, student race, student sex, student grade level, and a pseudonym for the teacher. The ethnic typicality of the students’ names (Bertrand & Mullainathan, 2004) was also used to denote the race/ethnicity of the student (Greg and Cody were White students; Darius and Jamal were Black students). After each video, participants answered items (e.g., “What was the name of the student in the video?”) to determine accurate recall of the information in the video and identification of student and teacher race (Naumann et al., 1996). Finally, participants filled out the three representation measures regarding collaboration, teacher expectations, and student-teacher relationship quality. Data Analysis For cases with a total of one missing item (n = 16), scores were calculated as the arithmetic mean of the remaining items in the corresponding scale. Cases with a total of more than one missing item (n = 10) were excluded using listwise deletion for final analyses to produce unbiased results (Kang, 2013). A missing values analysis was conducted to determine whether data were missing completely at random (MCAR). Little’s test was used, with an alpha level of 0.05. Expectation-maximization estimates indicated a 𝜒2 statistic of 26 (df = 21), implying that data were missing completely at random (p > 0.05; Li, 2013). In order to answer the first research question, Pearson correlation coefficients were estimated to determine the strength of the relationship between school psychologists’ representations of the collaborative relationship, teacher expectations, student-teacher relationship, and cultural competence. To answer the second and third research questions, 44 multiple regression was appropriate to understand the relative contributions of racial match, racial mismatch, and cultural competence in explaining the variance in collaboration, teacher expectations, and the student-teacher relationship. Data were checked to ensure they met the assumptions of multiple regression prior to estimating the models. The multiple regression models were built using a hierarchical procedure with four blocks: (a) school setting, (b) cultural competence, (c) two out of the three representation variables (collaboration, teacher expectations, and student-teacher relationship) and (d) dummy codes for the racial match (WW, BB) and racial mismatch (WB, BW) conditions. The racial match condition of a White teacher and a Black student was used as the reference group to challenge the standard practice of using White (i.e., White teacher-White student) as the reference group in racial disparities studies (Elliott et al., 2022) and in the social sciences generally (Johfre & Freese, 2021). Further, this represents common school consultation scenarios given the growing population of students of color and predominately White teacher workforce (Egalit et al., 2015). A Bonferroni-adjustment resulted in an alpha level of 0.0125 (α = 0.05/4) to reduce the family-wise error rate and avoid making a Type 1 error (Cohen, 1988). Finally, we determined the model of best fit and looked for the most parsimonious model, indicated by a statistically significant change in the R2 statistic. A post-hoc power analysis indicated the study had sufficient power, with three predictors, to detect a minimum effect size of 0.14 (power = .80; N = 83; α = .05). Results Preliminary analyses indicated the data met the assumptions for multiple regression. Visual analysis of scatterplots indicated linear relationships between each of the independent variables and dependent variables, and there were no issues with homoscedasticity. Using 45 Cook’s distance, no outliers were identified. A histogram and Normal P-P plots showed a normal distribution of residuals. There was no multicollinearity (r < 0.90; Tabachnick & Fidell, 2012). Table 2 presents the descriptive statistics for the variables of interest, which illustrates that the continuous variables were normally distributed. Table 3 presents the bivariate correlations to answer the first research question. In answer to the first research question, correlation coefficients for the representation variables were in the moderate range (r = 0.36 - 0.57; p < 0.01), meaning that collaboration, teacher expectations, and student-teacher relationship were moderately associated with each other in terms of participants’ representation of the problem. In terms of the third research question, school psychologists’ self-reported cultural competence was not significantly associated with representations of collaboration, teacher expectations, or the student-teacher relationship (p > 0.0125). What emerges from the results for the second research question is more complex. Racial match and mismatch were found to contribute to perceptions of collaboration (see Table 3). More specifically, a racial match between a White teacher and White student was found to negatively influence school psychologists’ perceptions of collaboration (B = - 0.56; p < 0.001). This suggests that school psychologists seeing a White teacher describing a White student reported a collaborative relationship 0.56 points lower than those seeing a White teacher describing a Black student, on average. Ratings of the student-teacher relationship also accounted for part of the variation in perceptions of collaboration (B = 0.31; p < 0.001). Model 3, which included participant demographics, cultural competence, and two out of the three representation variables, fit the data best according to a statistically significant difference in the R2 statistic (R2 = 0.41 in Model 3; p < 0.001), in comparison to the other three models. 46 Ratings of collaboration and the student-teacher relationship influenced teacher expectations (B = 0.16 for collaboration, p < 0.01; B = 0.24 for student-teacher relationship, p < 0.001; see Table 4). This finding indicates that when school psychologists’ perceptions of collaboration and the quality of the student-teacher relationship changed, so did their perceptions of the teacher’s expectations for their student. In addition, the middle school setting influenced teacher expectations (B = 0.21; p = 0.01). This suggests that school psychologists who worked in middle school settings rated the teacher as having higher expectations by 0.21 points when compared to those who worked in elementary school settings, on average. Model 3 fit the data best for teacher expectations with a statistically significant change in the R2 statistic (R2 = 0.52; p < 0.001). Teacher expectations contributed to the variation in ratings of the student-teacher relationship in the behavior vignette (B = 0.19; p < 0.01; see Table 5). This means that when school psychologists’ perceptions of the student-teacher relationship increased, their perceptions of the teacher’s expectations also increased. Again, Model 3 fit the data best for the student- teacher relationship in the behavior vignette, due to a statistically significant change in the R2 statistic (R2 = 0.39; p < 0.001). Ratings of collaboration contributed to changes in perceptions of the student-teacher relationship in the reading vignette (B = 0.56; p < 0.001; see Table 6). When school psychologists’ perceptions of their ability to collaborate with the teacher increased, their perceptions of the student-teacher relationship also increased. Racial match between a White student and a White teacher was also found to contribute to the variation in ratings of the student-teacher relationship quality in the reading vignette (B = 0.88; p < 0.001). This suggests that when school psychologists saw a White teacher describing a White student, their reports of the student-teacher relationship were greater by 0.88 points on average, in comparison to when 47 school psychologists saw a White teacher describing a Black student. Model 4 fit the data best for student-teacher relationship in the reading vignette (R2 = 0.44; p < 0.01). In summary, the three representation variables were moderately correlated, racial match between a White student and teacher affected school psychologists’ representations, and cultural competence was not associated with the three representation variables. Together these results provide important insights into how racial diversity within the consultation triad contributes to school psychologists’ representations of student problems. Discussion The current study made several contributions to the consultation literature. First, we used a novel approach for examining how consultants think about the school-based consultation process and have started to fill the gap in the literature related to consultants’ representations of student problems. Second, the current study was the first to quantitatively investigate how consultants’ thoughts about student problems vary based on the racial diversity of the consultation triad. School psychologists who serve as school-based consultants, where they encounter teachers of a variety of different racial backgrounds, need to be aware that their thoughts regarding student problems can change based on student and teacher race. Third, our findings indicated that school psychologists’ perceptions of collaboration, teacher expectations, and the student-teacher relationship influence one another. The current study conceptualized representations as the interaction between multiple perceptions. Thus, findings suggest that school psychologists must be aware of how their perceptions of aspects of the consultation process (e.g., student and teacher race) might inform their representation of student problems. For collaboration and the student-teacher relationship, we found that consultants’ perceptions of the quality of the collaborative and student-teacher relationship varied when 48 White teachers described Black students compared to White students. Consultants perceived the collaborative relationship more poorly when thinking about working within a consultation triad comprised of a White teacher and White student. On the other hand, when reporting on the quality of the student-teacher relationship between a White teacher and White student in the reading vignette, consultants rated the relationship stronger than one between a White teacher and Black student. These variations in perceptions about collaboration versus the student-teacher relationship could mean that the consultants were considering the role of student and teacher race in the vignette. For example, they could have been reflecting on research findings indicating that White teachers often report poorer quality student-teacher relationships with their Black students in comparison to White students (e.g., Saft & Pianta, 2001; Redding, 2019). Practitioners should be mindful of how their thoughts about the problem can vary when they are working with students and teachers of different races in order to choose the most appropriate route for addressing student concerns and closing the opportunity gap. Consultants’ reports of teacher expectations were influenced by perceptions of collaboration and the student-teacher relationship. This finding makes evident how perceptions of one facet of the consultation process influences another. In addition, consultants’ work setting influenced their perceptions in that those who worked in middle school reported higher teacher expectations when compared to those who worked in elementary school. This aligns with prior research indicating that teacher expectations increase in middle school with higher demands placed on students (Weinstein et al., 1987). Perhaps school psychologists were reflecting on their in vivo experience when rating teacher expectations in the vignettes. Although this study did not find that cultural competence influenced consultants’ representations, school psychologists providing school-based consultation should be mindful of their ongoing development toward 49 cultural competence and the various cultural factors involved in the consultation process. It could be that cultural competence later influences consultants’ representations (i.e., within subsequent stages of the consultation process, such as analysis or implementation). Simultaneously, practitioners must (a) recognize that there is no “threshold” for becoming culturally competent, and (b) strive to engage in continuous learning about others and their cultural experience(s) throughout each stage of the consultation process. School-Based Implications School psychologists can use MCCC to recognize how their own cultural backgrounds may be influencing their perceptions (Ingraham, 2017). For instance, they could ask themselves why they believe a White teacher might have a stronger relationship with a White student than a Black student (i.e.., self-reflect on their assumptions). The school psychologist might have an explicit discussion about race (and how it impacts the student-teacher relationship) with the teacher to identify whether their representation of the problem is accurate. Within these discussions, it is critical for consultants to attempt to develop a shared cultural understanding with the consultee, as this might improve their working relationship (e.g., Redding, 2019). They must also be prepared to encounter some resistance or fear on behalf of the consultee when discussing race (Gay, 2018). In addition, MCCC highlights both differences and similarities among consultation triad members (Ingraham, 2017); and White school psychologists might try to recognize what they have in common with White teachers to prevent weak collaborative relationships from negatively impacting students (Johnson et al., 2016). Limitations Despite the strengths of this study, there are a few limitations that should be noted. First, it is unknown what steps the consultant would have taken after seeing the teacher describe the 50 referral concern (i.e., the problem identification stage). Given that consultants could choose to improve collaboration with the teacher, change teacher expectations, or how the teacher interacts with the student, it is unclear the extent to which each of these potential routes might influence student outcomes. Second, our small sample size reduces our ability to make broad claims about how consultants’ think about consultation, even though our sample was representative of the general population of school psychologists (Walcott & Hyson, 2018). Third, the low reliability of the cultural competence scale may have introduced a high amount of measurement error and prevented us from determining the true effect of cultural competence. However, this issue may be the result of how the construct of cultural competence was operationalized. It is unclear whether participants’ definition of cultural competence might have differed from that within the measure. Although self-report measures have many advantages (e.g., providing depth of information), they also have disadvantages. Self-perception bias, or how accurate one’s judgment of their skills actually is (Robins & John, 1997), could have led to low variability across participants. Further, selection bias may have occurred, such that school psychologists particularly interested in consultation may have been more likely to participate and may have also had systematically different representations than those that did not. Finally, it is possible that White participants rated the Black teacher-Black student and/or Black teacher-White student more favorably, and/or overexaggerated their cultural competence, due to social desirability bias (Marcucci, 2020). Recommendations for better understanding these nuances, particularly those related to cultural competence, are presented in the following section. Suggestions for Future Research Future research may wish to examine whether explicit discussions about culture shift consultants’ and consultees’ representations of the problem, influence ensuing behavior (e.g., 51 intervention selection), and the extent to which changes in perceptions lead to improved student outcomes. Further investigation of the conditions under which these discussions are effective would be fruitful. For instance, future researchers may wish to examine (a) how the type of consultation approach (i.e., expert or consultee-centered) influences consultants’ perceptions and (b) how effective explicit discussions about culture are in eliciting positive student outcomes. As mentioned, representations entail both thoughts and feelings about the problem (Hylander, 2000). Given that this study examined consultants’ thoughts about student problems, future studies should also investigate consultants’ feelings about them to achieve a more well-rounded understanding of consultants’ representations. Findings indicated the need for more rigorous approaches to measure cultural competence throughout the consultation process. These might include various methodologies (e.g., qualitative interviews, observation) and outcomes (e.g., route taken in consultation, or the intervention chosen). Conclusion The current study contributed to the multicultural consultee-centered consultation literature and employed a novel approach for understanding consultants’ representations. Results suggested that consultants’ perceptions of collaboration, teacher expectations, and student- teacher relationship quality influence each other. These variations in perceptions may impact consultants’ representations of student problems, and subsequent stages of the consultation process. Increasing consultants’ awareness of how their perceptions might vary depending on student and teacher race could increase the effectiveness of consultation, narrow the opportunity gap, and lead to more culturally competent practitioners in the field. 52 CHAPTER 4 (PAPER 3): HOW DOES CULTURE FIT INTO DE-IMPLEMENTATION? A SCOPING REVIEW OF EMPIRICAL RESEARCH Abstract Research indicates that historically marginalized populations are more likely to receive low-value care in comparison to White populations. It is crucial to de-implement (i.e., replace or discontinue use of) these practices to make room for more effective alternatives and reduce disparities in outcomes. However, there is limited understanding of how, when, and why practices are de-implemented. De-implementation has been shown to improve client outcomes through a variety of mechanisms, such as improved service quality. This scoping review had two aims. First, we examined empirical articles related to de-implementation to determine whether culture (the practitioner’s or client’s values and beliefs) and race/ethnicity were assessed or reported. Second, we examined how these factors were considered throughout the de- implementation process. A total of 20 empirical studies were identified through five online databases, prior reviews, and hand-searching the references of studies that met our inclusionary criteria. Results suggested there is a gap in the literature regarding how culture and race/ethnicity may impact de-implementation and ensuing outcomes. We translated research across settings to highlight the importance of considering culture and race/ethnicity when engaging in de- implementation. Lastly, we provide suggestions for future research and discuss implications for practitioners. Keywords: school psychologists; de-implementation; discontinuation; de-adoption; implementation; culture; race; ethnicity 53 Historically marginalized populations, such as those from racially and ethnically minoritized backgrounds, experience disparate negative outcomes in comparison to their White peers, often beginning during childhood (Franklin et al., 2006). An opportunity gap (Akiba et al., 2007) results in children and adolescents from diverse backgrounds having lower academic achievement in schools (Lee, 2004), increased exposure to racism and discrimination (Leath et al., 2019), and experiencing high mortality rates (Geiger, 2006). These disparities have received the attention of researchers, policymakers, and practitioners for decades. Much research has sought to develop, test, and implement evidence-based programs and practices (EBPs) to improve outcomes for historically marginalized populations. People of color frequently experience barriers to accessing quality care (Walsh-Bailey et al., 2021). When they do access much needed services, they may receive ineffective or low-value services. Research suggests that Black and Brown individuals are more likely to receive low- value care compared to their White counterparts (McKay et al., 2018). Subsequently, some scholars argue that the study of de-implementation may be equally as important as the study of the adoption and implementation of new EBPs (e.g., Prasad & Ioannidis, 2014), given associations between implementation and population health (Lobb & Colditz, 2013). De- implementation is the replacement or discontinuation of ineffective programs. Resources saved by replacing ineffective practices may be re-allocated to other areas to improve racial equity in service provision (e.g., community outreach, providing free services for low-income patients; Levin et al., 2018). The de-implementation process may also improve service quality within a system, particularly for marginalized populations (Owens et al., 2014). The purpose of this scoping review was to explore how the de-implementation literature (a) examined culture within the de-implementation process in order to promote positive and 54 equitable outcomes for marginalized populations and (b) reported disparities in outcomes based on the race/ethnicity of practitioners and/or clients. Moreover, we sought to provide implications for including culture in the de-implementation process for practitioners, particularly those working in the school setting. We use the term culture to refer to one’s self-identified values and beliefs, and the term cultural factors to refer to various aspects of one’s culture such as one’s identity (e.g., race/ethnicity, gender; Gay, 2018). Next, we describe what is known about the de- implementation process. De-Implementation: What Is It and How Does It Work? De-implementation has been defined in many ways in the literature (see Table 1). Results of prior reviews (e.g., Niven et al., 2015) suggest a lack of consensus about how de- implementation is defined, and many of the articles included in these reviews did not reference one’s cultural context (e.g., Wang et al., 2018). Generally, de-implementation involves discontinuing and/or replacing an ineffective or harmful EBPs. Consultants serve a critical role in the adoption, implementation, and sustainability of EBPs (Roach et al., 2006); thus, they can consequently spearhead de-implementation initiatives. In the educational context, school psychologists often serve as consultants, and help school administrators in the adoption process (Newman et al., 2015). They might advocate for the selection of EBPs, deemed effective for marginalized populations (National Association of School Psychologists, [NASP], 2020), that adhere to federal and state mandates (Barrett et al., 2017). In the healthcare context, many EBPs are adopted via a “top-down” model in which practices are mandated or strongly encouraged by the federal and state governments (United States Public Health Service, 2023). Medical doctors may consult with their support staff and/or patients regarding appropriate EBPs for treatment, varying their consultative services based on staff and patient characteristics. For instance, a 55 physician might consult with a local clinic about implementing combination therapy, aimed at treating hypertension, for Black patients given findings from prior research that this therapy may be more effective for this population (e.g., Helmer et al., 2018). School psychologists have begun to spread their skills beyond the school setting (Eklund et al., 2017); and it is possible that school psychologists might find themselves providing consultation within a healthcare system in order to support the use of EBPs (Hoagwood & Johnson, 2003). Multicultural approaches to consultation (e.g., Ingraham, 2017) could be used to intentionally incorporate culture into the problem-solving process. Multicultural consultation is unique in that it specifically emphasizes the role of diversity throughout the problem-solving process (Ingraham, 2000). Across settings, school psychologists might support racially and ethnically minoritized youth by leveraging multicultural consultation techniques throughout the four stages of de-implementation, which are described next. The De-Implementation Process De-implementation is a complex process that develops over time. According to Niven et al. (2015), de-implementation occurs in four stages: (1) identify and prioritize interventions appropriate for de-implementation; (2) assess the barriers and facilitators to de-implementation; (3) develop strategies to successfully support de-implementing the interventions; and (4) evaluate outcomes of de-implementation by assessing changes in practice, outcomes, and cost. De-implementation occurs within a socio-ecological context, in which individual and organizational systems interact and influence one another (Pinto & Witte, 2019). Nilsen and colleagues (2020) also described the ecological context of de-implementation in this way in their scoping review (N = 10 studies). Finally, another scoping review by Walsh-Bailey and colleagues (2021; N = 27 studies) found similar results when examining literature related to the 56 de-implementation of medical care practices. Healthcare and education systems are embedded within communities and are influenced by the racial/ethnic composition and inequality amongst them (Arum, 2000). These prior scoping reviews did not discuss how these cultural factors interact within the ecological framework presented, nor did they fully consider how culture manifests within individual and systems-level determinants of de-implementation (e.g., Walsh- Baily et al, 2021). These determinants are described next. Determinants of De-Implementation Individual and systems-level determinants can facilitate or inhibit de-implementation. In other words, some determinants are more likely to lead to successful de-implementation, while others contribute to the complexity of the de-implementation and make this process more difficult. Individual-level determinants can be either intrapersonal or interpersonal (Walsh-Bailey et al., 2021). Additionally, systems-level determinants can occur either internally or externally to the organization (Walsh-Bailey et al., 2021). Individual-Level Determinants Intrapersonal determinants are defined as within-individual (e.g., cognitive processes such as unlearning), while interpersonal determinants involve interactions between individuals (e.g., shared decision-making between a patient and provider). For example, in schools, teacher beliefs may be an intrapersonal determinant where a teacher believes that the EBP in question is fundamental to their traditional teaching practices (e.g., whole-language based approaches to reading; McKay et al., 2018). This, in turn, may lead to resistance between an administrator aiming to de-implement the EBP and the teacher, which would be an interpersonal determinant (Nilsen et al., 2020). In another interpersonal example, the sociocultural context of the 1970s and 1980s inspired greater participation from patients in medical treatment planning and decision- 57 making, impacting the de-implementation of certain breast cancer treatments (i.e., radical mastectomy; Montini & Graham, 2015). System-Level Determinants The following are examples of internal determinants that occur at the systems-level, of which include: (a) internal operations, (b) decision-making processes, (c) structures, (d) resources, and (e) internal organizational history. External determinants include community factors, historical events, the societal culture of healthcare consumption and professional medicine, the broader practice environment, and other societal influences (e.g., economy, politics; Nilsen et al., 2020). These internal and external systems-level determinants can influence the de-implementation process. For example, an EBP that has been used for several years (i.e., the historical context) may be more difficult to de-implement due to staff inertia (McKay et al., 2018). Additionally, certain EBPs may be viewed as more profitable at a given time and thus more difficult to de-implement (Rogers et al., 2021). Also, a higher financial investment may increase the likelihood the EBP will be continued (Barrett et al., 2023). The intricacies of these determinants are evident and the ecological interactions within and across individuals and systems makes de-implementation inherently nuanced. Current Study: Cultural Considerations for De-Implementation The purpose of this study is to explicitly focus on how culture and race/ethnicity have been considered in the de-implementation literature. Prior scoping reviews have primarily focused on healthcare settings (e.g., Walsh-Bailey et al., 2021), but there is also a need for de- implementation within the context of education (Shaw, 2021). There are many similarities between education and other health services fields (e.g., public health), such as interdisciplinary collaboration (Fewster-Thuente et al., 2008) and the promotion of evidence-based practices. 58 However, what is considered “harmful” may differ across schools and healthcare, which is one of the primary reasons for de-implementation (Gnidjic & Elsaug, 2015). Research on “lower quality education” for marginalized students, or disparities in education (e.g., poor quality student-teacher relationships for students of color) might be similar in this regard. It is possible to translate research findings across disciplines (Abrams, 2006; Grimshaw et al., 2012), and because school psychologists might work within both education and healthcare systems, we later discuss implications across settings. Second, none of the prior scoping reviews evaluated studies’ inclusion of culture or cultural factors, which is critical if de-implementation is to serve as a mechanism to close the opportunity gap. The de-implementation process is influenced by both individual- and systems- level determinants, and both individuals and systems have cultural characteristics. For example, intrapersonal and interpersonal individual determinants are likely influenced by culture (e.g., communication styles; Gay, 2018). External systems-level determinants (e.g., community, neighborhoods) may also be influenced by culture and cultural factors. For instance, the economic context might contribute to the inappropriate continuation of ineffective or low-value EBPs given the complex financial consequences of de-implementation (Montini & Graham, 2015). Previous scoping reviews have not evaluated the literature with this explicit emphasis on how culture permeates the de-implementation process. Moreover, these prior reviews did not emphasize education, nor were they attuned to the unique role of school psychologists and how they can bridge the gap between education and health services for marginalized populations (Shaw, 2003). This scoping review answers the following research question: How is culture incorporated into empirical research examining de-implementation? A scoping review is a 59 synthesis of the literature on a particular topic in order to “identify key concepts; gaps in the literature; and types of evidence to inform practice, policymaking, and research” (Daudt et al., 2013, as cited in Pham et al., 2014, p. 373). Thus, a scoping review was appropriate given that our goal was to identify the available empirical evidence of how to include culture within the de- implementation process (Munn et al., 2018). The current study contributes to the literature as the only known scoping review related to de-implementation that highlights culture and various cultural factors, such as race/ethnicity. In addition, this is the first review to discuss implications for consultants across systems (i.e., healthcare and education). Method This study was registered using the Open Science Framework platform and it meets the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (see Online Supplementary Materials for the PRISMA [2018] checklist). Search Strategy and Inclusion Criteria A search of the literature was conducted using five electronic databases: ERIC, PsycINFO, EBSCOHost, Academic Search Complete, and JSTOR. Search terms included school OR education AND strategic abandonment OR de-implementation OR de-adoption. Duplicates were removed after the initial identification of potential articles for inclusion. Then, the first and third authors reviewed the titles and abstracts of the articles to determine if they met the inclusionary criteria: (a) published in a peer-reviewed academic journal, (b) in between 2005 and 2021, (c) in English, (d) in K-12 public school or healthcare settings, and (e) included empirical data (quantitative or qualitative). The first year of publication included was 2005 to align with findings from the most recent scoping review by Walsh-Bailey and colleagues (2021). The 60 rationale for including healthcare settings was that this is a nascent body of literature and excluding these related settings would have limited our understanding of how culture fits into de- implementation. Next, the first and third authors reviewed the full text of the articles. Only empirical articles were included to assess how participant demographics (i.e., race/ethnicity) and the study setting (e.g., school) influenced de-implementation and make generalizable conclusions (Sifers et al., 2002). Studies were excluded if they were (a) published in a language other than English, (b) did not include empirical data (e.g., conceptual or theoretical articles), or (c) conducted in any other setting (e.g., business). There were no criteria regarding the aims of programs for de-implementation, and studies could include any outcome measures (e.g., academic or medical). These inclusionary and exclusionary criteria were chosen to meet the purpose of this study, which was to understand the extent to which empirical literature regarding de-implementation included culture or the race/ethnicity of key stakeholders. After identifying articles through the databases, the first and third authors hand searched the references of the 13 studies that met the inclusionary criteria to identify additional relevant articles. The articles retrieved via hand searching (n = 46) were divided amongst all three authors. The titles and abstracts of these articles were read, and the same inclusionary and exclusionary criteria were applied to determine whether to include these articles in the review. There was only one discrepancy regarding the type of methodology used in one article and whether it met inclusionary criteria. Data Coding In order to begin to understand how culture and cultural factors were included in the de- implementation literature and develop a coding scheme, the first and third authors read through 61 two articles each while individually noting their general thoughts in narrative form (i.e., memoing; Birks et al., 2008). Next, all authors met to discuss these notes and narratives. All authors coded 65% of the articles for the following, (a) author and year of publication, (b) setting in which the study took place (e.g., education, healthcare), (c) study characteristics, including the measures (e.g., scales used), methodology (e.g., quantitative) and research design (e.g., survey), (d) sample characteristics (i.e., practitioner and client race) and (e) culture, defined as the attitudes, beliefs, and behaviors of a particular social group (Pachter, 1994). When the authors coded for culture, they also used the ADDRESSING model (Hays, 1996) to guide their coding of the attitudes, beliefs, and behaviors of a particular social group included in the articles, as this is a framework that delineates between ten different “categories” of culture to highlight the complexities of an individual’s cultural identity. The ADDRESSING framework includes: age, developmental, disability, religion, ethnicity and race, socioeconomic status, sexual orientation, nationality, and gender (Hays, 2008. Interobserver agreement was calculated twice, first based on the study characteristics and then based on the inclusion of culture. Interobserver agreement was calculated (by dividing the total number of agreements by the number of agreements plus disagreements and multiplying by 100) across both coding iterations and indicated 95% agreement. To determine the quality of the included articles, the first and second author critically appraised the articles for the following: (a) appropriateness of methodology and measures, (b) implementation fidelity of interventions, and (c) interpretation of results (Higgins et al., 2019). 62 Results Figure 1 illustrates the number of articles meeting inclusionary criteria throughout the screening process. The database searches resulted in 100 articles to be reviewed and hand searching resulted in 46 additional articles to be reviewed for inclusionary and exclusionary criteria. A total of 20 studies met inclusionary criteria. Table 1 presents a summary of the findings. Of the 20 total papers, 85% (n = 17) were conducted in healthcare settings, 10% (n = 2) were conducted in community mental health/social service settings, and 5% (n = 1) were conducted in a school setting. A total of 55% (n = 11) employed quantitative methods, 15% (n = 3) employed qualitative methods, and 35% (n = 7) employed mixed methods. Only 20% (n = 4) of the articles reported the race/ethnicity of the practitioners in the study; McKay and colleagues (2017) also reported the race/ethnicity of the clients. Of these four articles, two studies were quantitative, one used mixed methods, and one was qualitative. The majority of participants were White in most of these studies (e.g., 79.5% in Padek et al., 2021). 35% (n = 7) of the articles included individual or organizational culture in some way. Organizational culture was broadly related to the ecological system of the organization; and specifically referred to the values, attitudes, and acceptability of evidence-based practices amongst practitioners and leadership characteristics (e.g., support; Harris et al., 2017). One study examined individual culture (Cuttler, 2005). Next, we describe how organizational and individual culture were examined in the studies. Organizational Culture In the only school-based study, Nadeem and Ringle (2016) focused on the de- implementation of Cognitive Behavioral Intervention for Trauma in Schools (CBITS). They highlighted that organizational culture and support (or lack thereof) for an EBP might influence 63 the de-implementation process and overall sustainability of the EBP. For example, the authors described how the positive culture regarding EBPs, that had existed for years, was important for the sustainability of the new program. Findings from this study seem to support the idea of aligning the timeline of de-implementation with the culture of the school, or even the larger district. For instance, staff resistance to a new practice may warrant an extended timeline for de- implementation to gain buy-in from staff (i.e., de-implementing CBITS spanned two years). Harris and colleagues (2017a, 2017b, 2017c) published a series of mixed methods studies examining the effect of organizational culture on de-implementation in a large healthcare organization, noting that organizational culture could negatively influence the de-implementation process if practitioners were resistant to change. More specifically, they described three mechanisms that might influence de-implementation and its outcomes. The first mechanism related to how de-implementation was introduced and whether de-implementation was part of the routine decision-making processes of the organization. Whether practitioners were aware of the de-implementation decision (i.e., transparent) or unaware (i.e., decision-making was hidden) influenced practitioners’ acceptance of de-implementation. Second, results indicated that differences in organizational cultures across sites of the health service network may have required patients and staff members to adapt to de-implementation. Regarding the third mechanism, Harris and colleagues (2017c) indicated the amount of support from staff, and the perceived value of the EBP to be de-implemented, could influence a variety of outcomes, such as practitioner recommendations and the sustainability of the replacement program. Padek and colleagues (2021) gathered data from state health department practitioners and found that the organizational culture surrounding decision-making influenced which program(s) were selected for de-implementation. Further, the number of layers of authority impeded the 64 decision-making processes across state health departments. They found differences in organizational culture (i.e., hierarchy amongst employees) and leadership (e.g., competence in managing change) could lead to inappropriately terminating an effective program or practice. Organizational culture was a significant protective factor against continuing a practice that was ineffective or harmful. Finally, several articles noted the importance of funding in the de-implementation process, highlighting how resource allocation and prioritization were integrated into organizational culture (e.g., Goodwin & Frew, 2013). For example, Padek and colleagues (2021) found that the most common reason for inappropriately de-implementing an EBP was a change in funding priorities or the end of funding (reported by 87.6% of participants). On the other hand, the most common reasons for failing to de-implement an ineffective EBP was funder priorities to maintain the program (reported by 43.4% of participants). Authors noted that government funding might not lend itself to an organizational culture supportive of de-implementation and effective resource allocation. Goodwin and Frew (2013) interviewed 13 primary care trust staff members (i.e., managers, clinicians) involved in de-implementation. Their results indicated: (a) all key stakeholders should be involved in de-implementation and (b) joint planning and decision- making might strengthen staff members’ desire to collaborate and take ownership over the outcome of de-implementation. Finally, McKay and colleagues (2017) noted inconsistent policy and changes in funding priorities as some of the primary mechanisms behind the de- implementation of evidence-based practices in public health systems. They described how RESPECT, a brief counseling intervention utilized alongside HIV testing, was gradually 65 replaced over three months due to loss of grant funding. Clients were informed of this transition and could elect to complete the remainder of the RESPECT intervention if desired. Individual Culture Only one study referenced the role of individual culture in the de-implementation process. Cuttler and colleagues (2005) found that the cultural values and attitudes of physicians may have influenced their perceptions of treatment, in this case the use of growth hormone therapy. In turn, the clinicians’ perceptions influenced their recommendations to continue or de- implement treatment, which was their main outcome measure. Authors suggested these findings have implications for practice guidelines provided by professional organizations (i.e., differing values and attitudes amongst clinicians may make it difficult to develop widely accepted practices for growth hormone therapy). Perhaps the most striking finding is that none of the articles in this scoping review discussed how the race/ethnicity of the practitioners and/or clients might have influenced the de-implementation process. For example, although McKay and colleagues (2017) reported the race/ethnicity of clients receiving the RESPECT intervention prior to de-implementation, they did not report how outcome variables (e.g., continuation of services) may have differed based on client race/ethnicity. Discussion The topic of de-implementation is broadly understudied; and we found a dearth of literature that incorporated culture and race/ethnicity in this process. Interestingly, only one empirical study identified in this review was conducted in the school setting. This may be the result of the complex nature of de-implementing programs in educational contexts. However, understanding de-implementation, and how culture fits into this process, is critical; failure to do so may impact patient/student outcomes, particularly those from marginalized backgrounds. 66 Moving forward, researchers, practitioners, and policymakers alike must incorporate culture into the de-implementation process to close the opportunity gap by ensuring that effective programs are not inappropriately de-implemented and ineffective programs are not sustained for people of color. Because of the lack of de-implementation studies in schools and other educational settings, we are hesitant to make any strong claims about school-based de-implementation at this time. Instead, we theorize about implications for school psychologists serving as consultants across settings, with the hope that these will be used as jumping off points for future examination. Importantly, our findings suggested that both organizational and individual culture might influence de-implementation in critical ways. These are described next, followed by suggestions for future research and implications for practice. Organizational Culture Our findings showed that when prior studies did include culture, it was often organizational culture that was emphasized. Harris and colleagues (2017b, 2017c) described how various approaches to de-implementation might interact with the cultural environment of the organization. For instance, depending on the values and attitudes of staff, and whether the organizational culture is supportive of change (Harris et al., 2017c), a transparent or hidden approach might influence perceptions of de-implementation (Harris et al., 2017). Padek and colleagues (2021) found that organizational culture was a protective factor for appropriately de- implementing ineffective or harmful practices. In both education and healthcare, the values and attitudes of staff might be a key component to consider before beginning the de-implementation process. Consultants could discuss the importance of de-implementation during staff meetings to potentially foster an environment supportive of de-implementation. Harris and colleagues (2017; 2018) suggested evaluations of organizational culture to (a) determine if staff values and 67 attitudes may influence the de-implementation process and (b) ensure that all the necessary stakeholders are involved in decision-making. Our findings indicated that policy changes and funding support are potential influencers for de-implementation. Funding is particularly relevant to educational and healthcare systems operating in economically disadvantaged communities with limited resources. Resources in schools include funding to purchase evidence-based programs, wages for trained and qualified staff, safe and secure facilities, and supplies, such as textbooks and manipulatives (Hanushken, 2006). In healthcare, resources include funding for training and related materials, medical equipment, information technology infrastructure, and data management systems (Colombo, 2018; Marwaha et al., 2022). Montini and Graham (2015) described how physicians, hospitals, the pharmaceutical industry, and device manufacturers could all be invested in the continued implementation of a single EBP. There may be a well-established path of revenue for an organization because of this inappropriate continuation of the ineffective EBP. This could be a powerful barrier to de-implementation in the healthcare context. Alluding to the other side of this issue, Nadeem and Ringle (2016) suggested that a program may not be built sustainably for a particular setting, such as when the cost to purchase a program is too high and an organization is instead losing revenue because of continued implementation of an ineffective EBP. Thus, funding is a likely determinant of de-implementation that is especially relevant to organizational culture that school psychologists should keep in mind. Anderson and colleagues (2003) suggested that healthcare systems that are culturally competent might “have the potential to reduce racial and ethnic health disparities” (p. 68). Culturally competent systems have the awareness, knowledge, and skills to address, discuss, and incorporate a client’s cultural identities in practice (Hook & Watkins, 2015). It is critical for 68 empirical research related to de-implementation to better incorporate the individual cultural backgrounds of clients. Moreover, both healthcare and schools are multicultural and ecological systems, comprised of the various cultures of not only patients/students, but also families, staff, and even the larger community in which the organization is embedded, incorporating individual culture in this process may be even more important. Online resources (e.g., the United States Department of Education) are often used for the selection of EBPs; but there is variability in standards of evidence across these organizations (Flay et al., 2005), and it is unclear what individual cultural factors may be included within these studies and online resources (Nilsen et al., 2020; Walsh-Bailey et al., 2021). For instance, the National Institutes of Health requires reporting of race/ethnicity in federally funded research related to biomedicine (Konkel, 2015), but not all organizations will have this same requirement. Therefore, school psychologists might unknowingly choose an EBP that is not suitable for their particular context or population, given the lack of efficacy studies to determine its utility, and de- implementation of the EBP may need to occur (Nadeem & Ringle, 2016). When school psychologists can recognize the interaction between organizational culture and individual culture, they may provide their racially and ethnically minoritized patients/students with more effective EBPs. In sum, considering how individual culture interacts with organizational culture may help to avoid inappropriate de-implementation of effective programs, and inappropriate continuation of harmful ones. Individual Culture Very few studies reported the race/ethnicity of practitioners and their clients, let alone discussed how individual culture influenced de-implementation. For instance, Cuttler (2005) discussed how the values and attitudes of clinicians regarding certain practices influenced their 69 ensuing recommendations for their patients. This parallels how teachers’ values and attitudes regarding certain practices, such as whole word reading, might influence their ensuing recommendations for a student struggling with reading. Multicultural consultation encourages school psychologists to consider how their own culture might be influencing the de- implementation process (i.e., do they themselves have beliefs that are hindering the process?). In addition, school psychologists utilizing multicultural consultation would be intentional about developing systems that consider the individual culture of patients/students, given that they might be more likely to reduce outcome disparities this way, specifically between White and Black individuals (Anderson et al., 2003). In the absence of a consultation approach that includes culture in this way, school psychologists providing consultative services might suggest the inappropriate continuation of an EBP for a student of color and fail to de-implement an ineffective or harmful EBP. Gaps in the Literature and Suggestions for Future Research From this review, it is apparent that practitioner- and client-specific variables, especially differences based on race/ethnicity, have been overlooked. For instance, the transition to a new program in the study conducted by McKay and colleagues (2017) led to a disconnect between RESPECT staff and the community in that potential clients continued to request RESPECT after its de-implementation. And yet, it is unknown how this transition to a new program might have disproportionately impacted Black and Brown clients. Further, evidence-based medicine practices should include patients’ values and preferences in the process of developing treatment plans (Masic et al., 2008). Prior literature has failed to fully include these variables, and it is especially critical for future research to consider differential impacts based on practitioner- and client-specific variables when thinking about de-implementation. For instance, previous studies 70 indicate that immigrant children and families are less likely to seek community-based resources for mental health (Snowden, 2012). If the social-emotional learning program at a school was replaced, these students might be disproportionally affected. More research is needed to examine these differential effects. It is also necessary to determine the extent to which there are significant differences in patient/student outcomes based on their racial or ethnic identities. This might prevent schools and healthcare organizations from inappropriately continuing an ineffective or harmful program, or inappropriately terminating an effective program for people of color. Further, culture and various cultural factors, such as race/ethnicity, are inherently nuanced. Future researchers must be cognizant of the heterogeneity present within all cultural groups in their studies. Future research might employ qualitative methodologies to better understand the de- implementation process. For example, case studies might be conducted to document specific steps taken by stakeholders in the de-implementation process, and how culture influenced this process. There is more to be understood about how decision-making practices occur, and how they influence de-implementation. Randomized control trials regarding the de-implementation of an EBP might also be conducted in which various outcomes could be compared (Lilienfeld, 2007). Moreover, practitioner- and client-specific variables need further examination, as only one study examined practitioner-specific outcomes. This might also help to identify underlying cultural mediators regarding outcomes of de-implementation. It may also be useful to examine the perspectives of multiple stakeholders within a given study, such as patients/students, practitioners/teachers, and parents, given the broader systems-level impact of de-implementation. 71 Implications for including culture throughout each stage of the de-implementation process are described next. Implications for School Psychologists Serving as Consultants Next, we provide a few implications related to how culture might be situated within the four-stage de-implementation process. Following findings from Harris and colleagues (2017a, 2017b, 2017c) regarding how decision-making might influence de-implementation, school psychologists providing consultative services could encourage conversations regarding who the appropriate decision makers are for de-implementation in their contexts. In addition, individual culture might be included in the identification stage by identifying the unique needs of particular cultural groups (Harris et al., 2017a, 2017b, 2017c). To do so within education, school psychologists, administrators, and other key school staff (e.g., teachers) could examine student outcome data related to a specific intervention across subgroups. In healthcare settings, school psychologists might analyze current care practices across subgroups as well, examining differential outcomes based on cultural factors, such as race/ethnicity (Voorn et al., 2018). In the determinants stage, it is particularly important to address the various individual and systems-level determinants that might be present. Healthcare and educational contexts are complex in that individual and organizational culture are often intertwined (e.g., they involve multi-level interactions between patients/students, parents, and staff), and these interactions could manifest in cultural differences between stakeholders. To better recognize these potential differences, school psychologists might help to identify whether, (1) the organizational culture is supportive of de-implementation and (2) staff attitudes and values differ based on their individual cultural backgrounds. During the timeline and process stage, school psychologists engaging in consultation could again prompt conversations regarding the potential impact on 72 patients/students when determining the de-implementation timeline, particularly if many individuals will be affected by this change. Finally, to include culture in the evaluation stage, school psychologists should disaggregate data based on the individual cultural factors, such as race/ethnicity, socioeconomic status, and native language, of their clients. We have discussed the inherent nuances involved in the de-implementation of programs, especially in schools (e.g., funding restrictions, lack of available alternatives and/or resources to launch a new program, etc.). Thus, school psychologists must be mindful of situations where adaptations to an ineffective program should be made before completely de-implementing the program. In other words, a key question of abandonment is related to what the replacement of the program will be, particularly when serving marginalized populations. School psychologists can aid in this decision-making about making adaptations, in place of de-implementation, by ensuring that evidence-based programs are implemented with fidelity. Limitations While the current study had many strengths, such as the cross-examination of multidisciplinary literature, there were also a few limitations. First, this scoping review was limited to peer-reviewed articles, which excluded gray literature and other sources of information. However, limiting the review to peer-reviewed articles might have strengthened the quality of the studies found. Second, conceptual articles were not included in this review and these papers may have added to the broader understanding of de-implementation. We were most interested in how culture influenced de-implementation, and to what extent race/ethnicity was reported, which necessitated empirical research. Lastly, it is unknown how including culture and race/ethnicity in the reviewed articles would have improved de-implementation outcomes. Future 73 research should explicitly define de-implementation outcomes of interest (e.g., Prusazyk et al., 2020). Conclusion Results of this scoping review indicated that more research is needed regarding the de- implementation of evidence-based programs, particularly in education. This study examined how culture might fit into the de-implementation process and discussed implications for school psychologists working in different settings. In order to narrow the opportunity gap, further examination of how to de-implement ineffective EBPs for Black and Brown populations must occur. Moreover, future research should examine how, when, and why de-implementation occurs in educational contexts to ensure that schools are able to promote equitable outcomes for all students. 74 CHAPTER 5: CONCLUSION The three papers within this dissertation investigated the importance of recognizing racial diversity in consultation, and how multicultural consultation could be used to support consultation efforts aimed at narrowing the opportunity gap. The first two papers examined how consultants might use multicultural consultee-centered consultation when members of the consultation triad are racially matched (i.e., share the same race; Oates, 2003), and provided recommendations for research and practice to improve the implementation of school-based consultation that emphasizes diversity. The third paper explored how the race/ethnicity of clients in the consultation triad is important to consider when evaluating evidence-based programs. More specifically, the third paper (a) investigated how culture is incorporated in the de- implementation literature and (b) explored how de-implementation practices can include race/ethnicity in order to improve positive outcomes for racially and ethnically minoritized youth. Major findings from Papers 1 and 2 indicated that the perceptions school psychologists’ have of aspects of the consultation process are influenced by student race and teacher race. More specifically, participants in Paper 1 more positively perceived their ability to collaborate with Black teachers than White teachers. Further, when students were described as having a behavioral concern and students were Black, participants rated collaboration more positively overall. Finally, there was a significant interaction effect between student race and teacher race, specifically in the behavior vignette, for ratings of collaboration. Again, participants reported the weakest level of collaboration when the participants, students and teachers were all White. This was unexpected, but it is possible that the participants responded in a way they thought was socially desirable, overcompensating for their implicit racial biases that were present in the 75 White teacher-Black student, Black teacher-Black student, and Black teacher-White student conditions (Marcucci, 2020). Paper 2 revealed similar discoveries. Participant perceptions of their level of collaboration and the quality of the student-teacher relationship varied when White teachers described Black students. This is in comparison to White students specifically. Collaboration was rated more poorly by participants when both teachers and students were White. However, when the student was described as having a reading problem, the quality of the student-teacher relationship was rated more positively when both teachers and students were White, when compared to the student-teacher relationship between a White teacher and Black student. This variability indicate that participants considered how student and teacher race might be impacting the student-teacher relationship, specifically between a White teacher and Black student when reading was a concern. Participants’ perceptions of teacher expectations were influenced by their perceptions of collaboration and the student-teacher relationship. This suggests that perceptions do interact to inform one’s representation of the problem. Lastly, work setting was a demographic characteristic of participants that influenced their perceptions. In comparison to participants who worked in elementary school, those who worked in middle school reported higher teacher expectations. Results from Paper 3 showed that consultee and client race/ethnicity have not been intentionally incorporated within the de-implementation literature. Only one de-implementation study was found that was conducted in the school setting. And yet, the implementation of evidence-based programs in schools is encouraged, if not required, by federal and state regulations (e.g., ESSA, 2015). Organizational culture was emphasized in studies that did include these cultural variables. A few studies (e.g., Harris et al., 2017b) described how 76 organizational culture (e.g., is the culture supportive of change, is decision-making transparent) influenced de-implementation. Some studies highlighted the importance of understanding the values and attitudes of staff before de-implementation. For instance, consultants should determine whether staff might be resistant to change. In addition, policy changes and funding support are potential facilitators for de-implementation in terms of organizational culture. One study emphasized individual culture, describing how the race/ethnicity of practitioners and their clients’ influenced recommendations regarding de-implementation of specific practices. It is paramount for consultants to help develop culturally responsive systems that include clients’ individual culture (i.e., race/ethnicity) in the de-implementation process (e.g., Anderson et al., 2003); and they must consider whether the resources they are using to determine whether de- implementation is appropriate have also considered cultural factors (Flay et al., 2005). Overall, both individual culture and organizational culture are necessary to consider when thinking about de-implementation. In conjunction with one another, the three papers of this dissertation highlight how school psychologists must consider how their own culture as a consultant, the consultee’s culture, and the client’s culture might manifest within the consultation process. Findings demonstrated that practitioners should be mindful of how their thoughts about client problems can vary when the consultation triad is racially diverse. Moreover, findings suggest that client race is particularly important to consider when selecting methods to support their success. For instance, client race should be taken into account when selecting interventions and when choosing to discontinue programming. School psychologists can utilize the MCCC framework to be more intentional about including race/ethnicity, and other cultural identities, within their work as consultants. 77 School-Based Implications School psychologists can use MCCC to recognize how their own cultural backgrounds may be influencing their perceptions (Ingraham, 2017). For instance, they could ask themselves why they believe a White teacher might have a stronger relationship with a White student than a Black student (i.e.., self-reflect on their assumptions). The school psychologist might have an explicit discussion about race (and how it impacts the student-teacher relationship) with the teacher to identify whether their representation of the problem is accurate. Within these discussions, it is critical for consultants to attempt to develop a shared cultural understanding with the consultee, as this might improve their working relationship (e.g., Redding, 2019). They must also be prepared to encounter some resistance or fear on behalf of the consultee when discussing race (Gay, 2018). In addition, MCCC highlights both differences and similarities among consultation triad members (Ingraham, 2017); and White school psychologists might try to recognize what they have in common with White teachers to prevent weak collaborative relationships from negatively impacting students (Johnson et al., 2016). Limitations Despite the strengths of each of the three individual studies, there are a few limitations of the totality of work that should be noted. First, utilizing quantitative methodologies to examine the influence of race/ethnicity, and culture overall, may not fully capture the breadth and depth of racial/cultural diversity in school psychological practice. There is inherent nuance involved in examining a social construct such as race/ethnicity. Second, the small sample size, and limited racial diversity within the sample for Papers 1 and 2 may reflect a simpler understanding of the problem than if we had been able to recruit a more diverse sample of practitioners. That is, the voice of those from diverse backgrounds, such as racially and ethnically minoritized school 78 psychologists, was not completely gathered. It will be crucial for future research to ensure that these voices are heard and represented. However, our sound recruitment methods speak to the need for the field of school psychology to further diversify. Suggestions for Future Research Based on the findings of the three papers, we offer several suggestions for future research related to multicultural consultation. First, studies might use multiple methodologies (e.g., qualitative, community-based participatory action research) to provide a deeper understanding of how the race/ethnicity and cultural backgrounds of consultation triad members, and the match amongst them, influences the consultation process. Future research studies might also involve a more rigorous investigation of what a conceptual shift during consultation entails and how to measure it appropriately (e.g., observations, process logs). These studies might use purposive sampling strategies in order to recruit a sufficient sample size of school psychologists of color. Second, future research must consider the historical context in which the research is conducted. For instance, Paper 1 was situated within the global events of the Black Lives Matter Movement. This prompted deeper reflection regarding the importance of history when examining the influence of race/ethnicity in empirical studies. Outside of major historical events like these global movements, all research exists within a historical context in some way or another. As an example, much of the de-implementation research in Paper 3 discussed how changes to federal and state laws and regulations regarding evidence-based practice influenced the use of these strategies in particular settings (e.g., Walsh-Bailey et al., 2020). Further, the landscape of advocacy and social justice within school psychology and education is rapidly changing. Federal and state policies, such as those related to antiracist teaching practices (e.g., cultural responsiveness), necessitate investigations related to how these policies lead to disparate 79 outcomes between racially and ethnically minoritized students and their White peers. Given that school psychologists might be limited in their ability to rectify these policies, de-implementation efforts might impact students of color more positively than others. These examples reflect how the broader historical and societal contexts should be acknowledged when conducting research. For instance, future studies must intentionally consider how they frame their reference group and comparisons when evaluating EBPs (Johfre & Freese, 2021). Moreover, the American Psychological Association (2022) has recently released statements regarding the negative impact psychological research has had on racially and ethnically minoritized populations because of White-centered approaches to research. In other words, the historical context of psychological practice and education should also be discussed when conducting research on racially and ethnically minoritized populations. For example, future research related to Papers 1 and 2 might wish to use critical theories (e.g., Critical Race Theory; Bell, 1995) to explore how consultants’ social conceptions of race and ethnicity are shaped by various factors (e.g., national origin, political movements). Implications for Practice There are several implications for practice following findings from these three papers. First, practitioners can use the principles of multicultural consultation when providing consultative services. As mentioned in Papers 1 and 2, they must have self-awareness of their own cultural background and how it might influence perceptions and interactions between members of the consultation triad because both have implications for student outcomes. Relatedly, practitioners must continually examine if they are complicit in perpetuating racial disparities when implementing EBPs or failing to de-implement them. For example, few studies included in Paper 3 discussed terms related to culture (e.g., race versus culture), let alone 80 reported on the race/ethnicity of participants included in the sample. Our findings generally suggest that understanding how racial diversity manifests for all members of the consultation triad is critical. Consultants must develop a shared understanding of the problem, not only as a key tenet of consultee-centered consultation, but also because consultants might fail to de- implement an EBP that is ineffective or harmful given biases and beliefs. If the consultant can understand the consultee’s orientation, and their willingness to change (Hylander, 2012), they can ensure that racial differences amongst members of the consultation triad are not leading to ineffective consultative services. In addition, they can ensure that the clients’ race/ethnicity is also intentionally considered when transitioning to the later stages of consultation. Finally, consultants must advocate for the development of culturally responsive systems to support the implementation of EBPs that are effective for all students. These systems would support the use of culturally responsive practices (Chin, 2000), and might further narrow the opportunity gap for Black and Brown populations (Andreson, 2003). For example, school policies that are ineffective for particular minoritized groups should be de-implemented and replaced with a culturally responsive alternative that addresses a similar concern, such as replacing school policing with culturally responsive positive behavior interventions and supports for behavior problems (Bal, 2018; Petteruti, 2011). In conclusion, systemic barriers have consistently prevented racially and ethnically minoritized students from accessing the same opportunities as White students (Flores, 2007). In order to achieve racial equity in education, inequity cannot be ignored (Justice Sonia Sotomayor, 2023). Utilizing multicultural consultation might promote and sustain systems change and help all students reach their full potential, regardless of their race or ethnicity. 81 REFERENCES Abrams, D. B. (2006). Applying transdisciplinary research strategies to understanding and eliminating health disparities. Health Education & Behavior, 33(4), 515–531. Afflerbach, P., Cho, B.Y., Jong‐Yun, K., Crassas, M., Doyle, B. (2013). Reading: What else matters besides strategies and skills? The Reading Teacher, 66(6), 440-448. Akiba, M., LeTender, G. K., Scribner, J. P. (2007). Teacher quality, opportunity gap, and national achievement in 46 countries. Educational Researcher, 36(7), 369-387. American Psychological Association. (2020). APA calls for true systemic change in U.S. culture. Anderson, L. M., Scrimshaw, S. C., Fullilove, M. T., Fielding, J. E., Normand, J. (2003). Culturally competent healthcare systems: A systematic review. American Journal of Preventive Medicine, 24(3), 68-79. Arum, R. (2000). School and communities: Ecological and institutional dimensions. Annual Review of Sociology, 26, 395-418. Bambra, C., Riordan, R., Ford, J., Matthews, F. (2020). The covid‐19 pandemic and health inequalities. Journal of Epidemiology and Community Health, 74(11), 964 ‐ 968. Barrett, C.A., Hazel, C.E., & Newman, D.S. (2017). Training confident school-based consultants: The role of course content, process, and supervision. Training and Education in Professional Psychology, 11, 41-48. Barrett, C. A., Sleesman D. J., Spear S. E., Clinkscales A., Amin, T. (2023). Sticking with programs that do not work: The role of escalation of commitment in schools. Prevention Science, 24(3), 567-576. Bates, L.A., & Glick, J.E. (2013). Does it matter if teachers and schools match the student? Racial and ethnic disparities in problem behaviors. Social Science Research, 42, 1180- 1190. Behring, S.T., & Ingraham, C.L. (1998). Culture as a central component of consultation: A call to the field. Journal of Educational & Psychological Consultation, 9, 57-72. Benson, N.F., Floyd, R.G., Kranzler, J.H., Maki, K.E., Eckert, T.L., & Fefer, S.A. (2020). A national survey of school psychologists’ practices in identifying specific learning disabilities. School Psychology, 35, 146-157. Bergan, J. R., & Kratochwill, T. R. (1990). Behavioral consultation and therapy. New York, NY: Plenum Press. Bertrand, M., & Mullainathan, S. (2004). Are emily and greg more employable than lakisha 82 and jamal? A field experiment on labor market discrimination. American Economic Review, 94(4), 991-1013. Betancourt, H., & López, S. R. (1993). The study of culture, ethnicity, and race in american psychology. American Psychologist, 48(6), 629–637. Birks, M., Chapman, Y., Francis, K. (2008). Memoing in qualitative research: Probing data and processes. Journal of Research in Nursing, 13(1), 68-75. Blanchett, W. J., Klingner, J. K., & Harry, B. (2009). The intersection of race, culture, language, and disability: Implications for urban education. Urban Education, 44(4), 389-409. Brooks-Gunn, J., & Markman, L. B. (2005). The contribution of parenting to ethnic and racial gaps in school readiness. The Future of Children, 15(1), 139–168. *Brownson, R. C., Allen, P., Jacob, R. R., Harris, J. K., Duggan, K., Hipp, P. R., Erwin, P. C. (2015). Understanding mis-implementation in public health practice. American Journal of Preventive Medicine, 48(5), 543-551. Caplan, G., & Caplan, R. B. (1993). Mental health consultation and collaboration. San Francisco, CA: Jossey-Bass. Caridakis, G., Karpouzis, K., Kollias, S. (2008). User and context adaptive neural networks for emotion recognition. Neurocomputing, 71(13–15), 2553-2562. Cohen J, 1988. Statistical power analysis for the behavioral sciences. (2nd ed.). New York: Routledge. Crenshaw, K. (1989). Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. University of Chicago Legal Forum, 1, 139–167. Critical Appraisal Skills Programme (2022). CASP Checklist. [online]. Retrieved from: https://casp-uk.net/referencing/. Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24(4), 349–354. *Cuttler, L., Silvers, J. B., Singh, J., Tsai, A. C., & Radcliffe, D. (2005). Physician decisions to discontinue long‐term medications using a two‐stage framework: The case of growth hormone therapy. Medical Care, 43(12), 1185–1193. Desimone, L., & Le Floch, K. (2004). Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis, 26, 1-22. Downer, J.T., Goble, P., Myers, S.S., & Pianta, R.C. (2016). Teacher-child racial/ethnic match within pre-kindergarten classrooms and children’s early school adjustment. Early 83 Childhood Research Quarterly, 37, 26-38. Edwards, O.W., McKinzie Bennett, C., & Johnson, B. (2019). School consultation to counter stereotype threat. Journal of Educational and Psychological Consultation, 29, 188- 205. Egalite, A.J., Kisida, B., & Winters, M.A. (2015). Representation in the classroom: The effect of own-race teachers on student achievement. Economics of Education Review, 45, 44- 52. *Erwin, C. P., Padek, M. M., Allen, P. Smith, R.; Brownson, C. R. (2020). The association between evidence-based decision making and accreditation of state health departments. Journal of Public Health Management and Practice, 26(5), 419 - 427. ESSA (2015). Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015- 2016). Retrieved from https://www.congress.gov/114/plaws/publ95/PLAW- 114publ95.pdf. Farmer, R. L., Zaheer, I., Schulte, M. (2022). Disentangling low-value practices from pseudoscience in health service psychology. Philosophical Psychology, 1-24. Fewster-Thuente, L., Velsor-Friedrich, B. (2008). Interdisciplinary collaboration for healthcare professionals. Nursing Administration Quarterly, 32(1), 40-48. Fischer, A.J., Schultz, B.K., Collier-Meek, M.A., Zoder-Martell, K.A., & Erchul, W.P. (2016). A critical review of videoconferencing software to support school consultation. International Journal of School & Educational Psychology, 6, 12-22. Fisher, J. R. (1993). Social desirability bias and the validity of indirect questioning. Journal of Consumer Research, 20(2), 303-315. Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151-175. Ford, D.Y., & Moore, J.L. (2013). Understanding and reversing underachievement, low achievement, and achievement gaps among high-ability african american males in urban school contexts. The Urban Review, 45, 399-415. Franklin, A. J., Boyd-Franklin, N., & Kelly, S. (2006) Racism and invisibility. Journal of Emotional Abuse, 6(2-3), 9-30. Furr, R.M., & Bacharach, V.R. (2017). Psychometrics: An introduction (3rd ed.). Los Angeles, CA: SAGE Publications, Inc. Gay, G. (2018). Culturally responsive teaching: Theory, research, and practice. New York: Teachers College Press. 84 Geiger, H. J. (2006). Health disparities: What do we know? What do we need to know? What should we do? In A. J. Schulz & L. Mullings (Eds.), Gender, race, class, & health: Intersectional approaches (pp. 261–288). Jossey-Bass/Wiley. Girvan, E. J., Gion, C., McIntosh, K., & Smolkowski, K. (2017). The relative contribution of subjective office referrals to racial disproportionality in school discipline. School Psychology Quarterly, 32(3), 392–4. Gnjidic, D., Elshaug, A. (2015). De-adoption and its 43 related terms: Harmonizing low-value care terminology. BMC Medicine, 13(1), 1-3. Goodman, L. (1961). Snowball sampling. The Annals of Mathematical Statistics, 32(1), 148- 170. *Goodwin, E., & Frew, E. J. (2013). Using programme budgeting and marginal analysis (PBMA) to set priorities: Reflections from a qualitative assessment in an English primary care trust. Social Science and Medicine, 98, 162–168. Gordon K. H., Brattole M. M., Wingate L. R., Joiner T. E. (2006). The impact of client race on clinician detection of eating disorders. Behavior Therapy, 37(4), 319-25. Grapin, S., Bocanegra, J., Green, T., Lee, E., Jaafar, D. (2016). Increasing diversity in school psychology: Uniting the efforts of institutions, faculty, students, and practitioners. Contemporary School Psychology, 20. Gregg, J., Saha, S. (2006). Losing culture on the way to competence: The use and misuse of culture in medical education. Academic Medicine, 81(6), 542-547. Grimshaw, J.M., Eccles, M.P, Lavis, J.N., Hill, S.J., & Squires, J.E. (2012). Knowledge translation of research findings. Implementation Science, 7(1), 50-66. *Gupta, D.M., Boland, R.J. & Aron, D.C. (2017). The physician’s experience of changing clinical practice: a struggle to unlearn. Implementation Science, 12, 28. Hanushken, A. E., & Welch, F. (2006). Handbook of the Economics of Education, 2, 865-908. *Harris, C., Allen, K., Brooke, V., Dyer, T., Waller, C., King, R., Ramsey, W., & Mortimer, D. (2017). Sustainability in health care by allocating resources effectively (SHARE) 6: Investigating methods to identify, prioritise, implement and evaluate disinvestment projects in a local healthcare setting. BMC Health Services Research, 17(1). *Harris, C., Allen, K., Waller, C., & Brooke, V. (2017). Sustainability in health care by allocating resources effectively (SHARE) 3: Examining how resource allocation decisions are made, implemented and evaluated in a local healthcare setting. BMC Health Services Research, 17(1). 85 *Harris, C., Green, S., & Elshaug, A. G. (2017). Sustainability in health care by Allocating Resources Effectively (SHARE) 10: Operationalising disinvestment in a conceptual frame- work for resource allocation. BMC Health Services Research, 17(1). *Harris C, Allen K, Ramsey W, King R, Green S. (2018). Sustainability in health care by allocating resources effectively (SHARE) 11: Reporting outcomes of an evidence‐driven approach to disinvestment in a local healthcare setting. BMC Health Services Research, 18(1). Hays, P. (1996; 2008). Addressing cultural complexities in practice: Assessment, diagnosis, and therapy (7th ed.). American Psychological Association. Helmer, A., Slater, N., & Smithgall, S. (2018). A review of ACE inhibitors and ARBs in black patients with hypertension. Annals of Pharmacotherapy, 52(11), 1143-1151. Higgins, J. P. T., Savović, J., Page, M. J., Sterne, J. A. C. (2019). Revised cochrane risk-of-bias tool for randomized trials (RoB-2). RoB-2 Development Group. Hoagwood, K., & Johnson, J. (2003). School psychology: A public health framework. From evidence-based practices to evidence-based policies. Journal of School Psychology, 41, 3-21. Hook, J. N., & Watkins, C. E., Jr. (2015). Cultural humility: The cornerstone of positive contact with culturally different individuals and groups? American Psychologist, 70(7), 661–662. Hunter, R. C., & Bartee, R. (2003). The achievement gap: Issues of competition, class, and race. Education and Urban Society, 35(2), 151–160. Hylander, I. (2012). Conceptual change through consultee-centered consultation: A theoretical model. Consulting Psychology Journal: Practice and Research, 64, 29–45. Ingraham, C.L. (2000). Consultation through a multicultural lens: Multicultural and cross- cultural consultation in schools. School Psychology Review, 29, 320-343. Ingraham, C.L. (2003). Multicultural consultee-centered consultation: When novice consultants explore cultural hypotheses with experienced teacher consultees. Journal of Educational and Psychological Consultation, 14, 329-362. Ingraham, C.L. (2017). Educating consultants for multicultural practice of consultee-centered consultation. Journal of Educational and Psychological Consultation, 27, 72-95. Ingvarsson, S., Augustsson, H., Hasson, H., Ulrica von Thiele Schwarz, P. N., & von Knorring, M. (2020). Why do they do it? A grounded theory study of low-value care among primary care physicians. Implementation Science, 15(93), 1-10. Johnson, S.R., Pas, E.T., & Bradshaw, C.P. (2016). Understanding and measuring coach- teacher alliance: A glimpse inside the ‘black box’. Prevention Science, 17, 439-449. 86 Kazdin, A.E. (2016). Research design in clinical psychology (5th ed.). Boston, MA: Pearson Education. Konkel, L. (2015). Racial and ethnic disparities in research studies: The challenge of creating more diverse cohorts. Environmental Health Perspectives, 123(12), 297-302. Kozlowski, K.P. (2015). Culture or teacher bias? Racial and ethnic variation in student–teacher effort assessment match/mismatch. Race and Social Problems, 7, 43-59. Leath, S., Mathews, C., Harrison, A., & Chavous, T. (2019). Racial identity, racial discrimination, and classroom engagement outcomes among black girls and boys in predominantly black and predominantly white school districts. American Educational Research Journal, 56(4), 1318–1352. Lee, J. (2004) Multiple facets of inequity in racial and ethnic achievement gaps. Peabody Journal of Education, 79(2), 51-73. Levin, H. M., McEwan, P. J., Belfield, C., Bowden, A. B., & Shand, R. (2018). Economic evaluation in education: Cost-effectiveness analysis and benefit-cost analysis (3rd Ed.). Thousand Oaks, CA: Sage. Lilienfeld, S. O. (2007). Psychological treatments that cause harm. Perspectives on Psychological Science, 2(1), 53-70. Lobb, R., & Colditz A. G. (2013). Implementation science and its application to population health. Annual Review of Public Health, 34(1), 235-251. Mallett, R. K., Mello, Z. R., Wagner, D. E., Worrell, F., Burrow, R. N., & Andretta, J. R. (2011). Do I belong? It depends on when you ask. Cultural Diversity and Ethnic Minority Psychology, 17(4), 432–436. Marcucci, O. (2020). Implicit bias in the era of social desirability: Understanding antiblackness in rehabilitative and punitive school discipline. The Urban Review, 52(1), 47–74. Mashburn, A.J., Hamre, B. K., Downer, J.T., & Pianta, R.C. (2006). Teacher and classroom characteristics associated with teachers’ ratings of prekindergartners’ relationships and behaviors. Journal of Psychoeducational Assessment, 24, 367-380. Masic, I., Miokovic, M., & Muhamedagic, B. (2008). Evidence based medicine - New approaches and challenges. Journal of Academy of Medical Sciences of Bosnia & Herzegovina, 16(4), 219–225. *Massatti, R. R., Sweeney, H. A., Panzano, P. C., & Roth, D. (2008). The de-adoption of innovative mental health practices (IMHP): Why organizations choose not to sustain an IMHP. Administration and Policy in Mental Health, 35, 50–65. McCardle, P., Scarborough, H., & Catts, H. (2002). Predicting, explaining, and preventing children's reading difficulties. Learning Disabilities Research & Practice, 16, 230-239. 87 *McKay, V. R., Margaret Dolcini, M., & Hoffer, L. D. (2017). The dynamics of de-adoption: A case study of policy change, de-adoption, and replacement of an evidence-based HIV intervention. Translational Behavioral Medicine, 7, 821–831. McKay, V. R., Morshed, A. B., Brownson, R. C., Proctor, E. K., & Prusaczyk, B. (2018). Letting go: Conceptualizing intervention de-implementation in public health and social service settings. American Journal of Community Psychology, 62(1–2), 189–202. McKenney, E. L. W., Mann, K. A., Brown, D. L., & Jewell, J. D. (2017). Addressing cultural responsiveness in consultation: An empirical demonstration. Journal of Educational and Psychological Consultation, 27, 289-316. Montini, T., & Graham, I.D. (2015). Entrenched practices and other biases: Unpacking the historical, economic, professional, and social resistance to de-implementation. Implementation Science, 10(24), 1-8. Morris, E.W., & Perry, B.L. (2017). Girls behaving badly? Race, gender, and subjective evaluation in the discipline of African American girls. Sociology of Education, 90(2), 127–148. Munn, Z., Peters, M.D.J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18, 143- 150. *Nadeem, E., & Ringle, V. A. (2016). De-adoption of evidence-based trauma intervention in schools: A retrospective report from an urban school district. School Mental Health, 8, 132-143. Nastasi, B. K. (2000). School psychologists as health-care providers in the 21st century: Conceptual framework, professional identity, and professional practice. School Psychology Review, 29(4), 540-554. National Association of School Psychologists Professional Standards. (2020). National Center for Education Statistics. (2019). Racial/ethnic enrollment in public schools. National Center for Education Statistics. (2021). Characteristics of public school teachers. Naumann, W. C., Gutkin, T. B., & Sandoval, S. R. (1996). The impact of consultant race and student race on perceptions of consultant effectiveness and intervention acceptability. Journal of Educational & Psychological Consultation, 7, 151-160. Nederhof, A. J. (1985). Methods of coping with social desirability bias: A review. European Journal of Social Psychology, 15(3), 263–280. Newell, M. L., & Looser, J. (2018). Does context matter? An analysis of training in multicultural assessment, consultation, and intervention between school psychologists in urban and 88 rural contexts. Psychology in the Schools, 55, 85-92. Newman, D. S., Barrett, C. A., & Hazel, C. (2015). School consultation practices in the early career: Does training matter? Consulting Psychology Journal: Practice and Research, 67, 326 –347. Newman, D. S., & Ingraham, C. L. (2017). Consultee-centered consultation: Contemporary perspectives and a framework for the future. Journal of Educational & Psychological Consultation, 27, 1-12. Nilsen P., Ingvarsson S., Hasson H., von Thiele S. U., Augustsson H. (2020). Theories, models, and frameworks for de-implementation of low-value care: A scoping review of the literature. Implementation Research and Practice, 1, 1-15. Niven, D. J., Mrklas, K. J., Holodinsky, J. K., Straus, S. E., Hemmelgarn, B. R., Jeffs, L. P., & Stelfox, H. T. (2015). Towards understanding the de-adoption of low-value clinical practices: A scoping review. BMC Medicine, 13, 1-21. Noltemeyer, A., Kunesh, C., Hostutler, C., Frato, P. & Sarr-Kerman, B. (2012). The effects of student and teacher characteristics on teacher impressions of- and responses to- student behaviors. International Education Studies, 5, 96-111. Norton, W. E. & Chambers, D. A. (2020). Unpacking the complexities of de-implementing inappropriate health interventions. Implementation Science, 15(2), 1-7. *Norton, W. E., Kennedy, A. E., & Chambers, D. A. (2017). Studying de-implementation in health: an analysis of funded research grants. Implementation Science, 12(144), 1-13. Oates, G. L. S. C. (2003). Teacher-student racial congruence, teacher perceptions, and test performance. Social Science Quarterly, 84, 508-525. Owens, J. S., Lyon, A. R., Brandt, N. E., Warner, C. M., Nadeem, E., Spiel, C., & Wagner, M. (2014). Implementation science in school mental health: Key constructs in a developing research agenda. School Mental Health, 6, 99-111. *Padek, M. M., Mazzucca1, S., Allen, P., Rodriguez Weno, E., Tsai, E., Luke, D. A., & Brownson, R. C. (2021). Patterns and correlates of mis-implementation in state chronic disease public health practice in the United States. BMC Public Health, 21(101), 1-11. Peck, C. T.., Seinfeld, S., Aglioti, M. S., Slater, M. (2013). Putting yourself in the skin of a black avatar reduces implicit racial bias. Consciousness and Cognition, 22(3), 779-787. Pham, M. T., Rajić A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., McEwen, S. A. (2014). A scoping review of scoping reviews: Advancing the approach and enhancing the consistency. Research Synthesis Methods, 5(4), 371-85. Pianta, R. C., & Nimetz, S. L. (1991). Relationships between children and teachers: Associations 89 with classroom and home behavior. Journal of Applied Developmental Psychology, 12, 379-393. Pigott, R. L., & Cowen, E. L. (2000). Teacher race, child race, racial congruence, and teacher ratings of children’s school adjustment. Journal of School Psychology, 38, 177-195. Pinto, R. M., & Witte, S. S. (2019). No easy answers: Avoiding potential pitfalls of de- implementation. American Journal of Community Psychology, 63(1–2), 239–242. *Pinto, M. R & Park E. S. (2019). De-Implementation of evidence-based interventions: Implications for organizational and managerial research. Human Service Organizations: Management, Leadership & Governance, 43(4), 336-343. Prasad, V., & Ioannidis, J., 2014. Evidence-based de-implementation for contradicted, unproven, and aspiring healthcare. Implementation Science, 9(1), 1-5. Prusaczyk, B., Swindle, T., & Curran, G. (2020). Defining and conceptualizing outcomes for de- implementation: Key distinctions from implementation outcomes. Implementation Science Communications, 1(43), 1-10. Quintana, S., Aboud, F., Chao, R., Grau, J., Cross, W., Hudley, C., Hughes, D., Liben, L., Gall, S., Vietze, D. (2006). Race, ethnicity, and culture in child development: Contemporary research and future directions. Child Development, 77, 1129-1141. Redding, C. (2019). A teacher like me: A review of the effect of student-teacher racial/ethnic matching on teacher perceptions of students and student academic and behavioral outcomes. Review of Educational Research, 89, 499-535. Roach, A. T., Kratochwill, T. R., & Frank, J. L. (2009). School-based consultants as change facilitators: Adaptation of the concerns-based adoption model (CBAM) to support the implementation of research-based practices. Journal of Educational and Psychological Consultation, 19(4), 300-320. Rogers, M. R. (1998). The influence of race and consultant verbal behavior on perceptions of consultant competence and multicultural sensitivity. School Psychology Quarterly, 13, 265-280. Rogers, A. A., Ha, T., & Ockey, S. (2021). Adolescents' perceived socio-emotional impact of COVID-19 and implications for mental health: Results from a U.S.-based mixed-methods study. The Journal of Adolescent Health: Official publication of the Society for Adolescent Medicine, 68(1), 43–52. Rosenfield, S. (1987). Instructional consultation (1st ed.). New York: Routledge. Saft, E. W., & Pianta, R. C. (2001). Teachers’ perceptions of their relationships with students: Effects of child age, gender, and ethnicity of teachers and children. School Psychology Quarterly, 16, 125-141. 90 Sheridan, S. M. (1997). Conceptual and empirical bases of conjoint behavioral consultation. School Psychology Quarterly, 12(2), 119–133. Shaw, S. R. (2021). Implementing evidence-based practices in school psychology: Excavation by de-implementing the disproved. Canadian Journal of School Psychology, 36(2), 91-97. Sifers, S. K., Puddy, R. W., Warren, J. S., Roberts, M. C. (2002). Reporting of demographics, methodology, and ethical procedures in journals in pediatric and child psychology. Journal of Pediatric Psychology, 27(1), 19–25. Skiba, R. J., Horner, R. H., Chung, C.-G., Rausch, M. K., May, S. L., & Tobin, T. (2011). Race is not neutral: A national investigation of African American and Latino is proportionality in school discipline. School Psychology Review, 40(1), 85–107. *Skolarus, T. A., Hawley, S. T., Wittmann, D. A., Forman, J., Metreger, T., Sparks, J. B., Zhu, K., Caram, M. E. V., Hollenbeck, B. K., Makarov, D. V., Leppert, J. T., Shelton, J. B., Shahinian, V., Srinivasaraghavan, S., Sales, A. E. (2018). De-implementation of low value castration for men with prostate cancer: Protocol for a theory-based, mixed methods approach to minimizing low value androgen deprivation therapy (DeADT). Implementation Science, 13(1), 144-155. Snowden, L. R. (2012). Health and mental health policies’ role in better understanding and closing African American-White American disparities in treatment access and quality of care. American Psychologist, 67(7), 524-531. Staats, C. (2016). Understanding implicit bias: What educators should know. The American Educator, 39(4), 29-43. Tabachnick, B.G. & Fidell, L.S. (2012) Using multivariate statistics. (6th ed.). Pearson Education. *Tangpong, C., Abebe, M., & Li, Z. (2015). A temporal approach to retrenchment and successful turnaround in declining firms. Journal of Management Studies, 52(5), 647-677. Taylor, K. (2016). From #blacklivesmatter to black liberation. Chicago, IL: Haymarket Books. United States Census Bureau. (2020). Inequalities persist despite decline in poverty for all major race and hispanic origin groups. *Upvall, M.J., & Bourgault, A.M. (2018). De‐implementation: A concept analysis. Nursing Forum, 53, 376–382. van den Bergh, L., Denessen, E., Hornstra, L., Voeten, M., & Holland, R. W. (2010). The implicit prejudiced attitudes of teachers: Relations to teacher expectations and the ethnic achievement gap. American Educational Research Journal, 47, 497-527. *Voorn, V. M. A., Marang-van de Mheen, P. J., van der Hout, A. Stefanie N. Hofstede1, S. N., 91 So-Osman, C., van den Akker-van Marle, M. E., Kaptein, A. A., Stijnen, T., Koopman- van Gemert, A. W. M. M., Dahan, A., Vlieland, T. P. M. M. V., Nelissen, R. G. H. H. & van Bodegom-Vos, L. The effectiveness of a de-implementation strategy to reduce low- value blood management techniques in primary hip and knee arthroplasty: A pragmatic cluster-randomized controlled trial. Implementation Science, 12(72), 1-11. *Voorn, V. M. A., van Bodegom-Vos, L., & So-Osman C. (2018). Towards a systematic approach for (de)implementation of patient blood management strategies. Transfusion Medicine, 28(2), 158-167. Walcott, C. M., & Hyson, D. (2018). Results from the NASP 2015 membership survey, part one: Demographics and employment conditions [Research report]. Bethesda, MD: National Association of School Psychologists. Walsh‐Bailey, C., Tsai, E., Tabak, R. G., Morshed, A. B., Norton, W. E., McKay, V. R., Brownson, R. C., & Gifford, S. (2021). A scoping review of de‐implementation frameworks and models. Implementation Science, 16(1), 100-118. *Wang, V., Gamble, G., & Grey, A. (2015). Responses of specialist societies to evidence for reversal of practice. American Medical Association, 175(5), 845-848. Wang, V., Maciejewski, M. L., Helfrich, C. D., & Weiner, B. J. (2018). Working smarter not harder: Coupling implementation to de-implementation. Healthcare, 6(2), 104–107. Wehmeyer, M., & Schwartz, M. (2001). Disproportionate representation of males in special education services: Biology, behavior, or bias? Education and Treatment of Children, 24(1), 28-45. 92 APPENDIX A: AUTHORSHIP DISCLOSURES Paper 1 • Shelbie E. Spear was third author on the published article, because of her participation and support cleaning the data. Paper 2 • There are no disclosures to report. Paper 3 • Bryn Endres will be third author on the manuscript, because she supported the calculation of IOA (i.e., screening articles, determining whether articles met the inclusionary criteria of the study). 93 APPENDIX B: TABLES FOR PAPER 1 Table 1 Sample Characteristics School Psychologist Demographics Race/Ethnicity White Black/African American Asian/Pacific Islander Multiracial Missing Age 22-30 30-38 38-48 48-56 56-64 65 + Highest Level of Education in School Psychology Master’s Education Specialist Doctoral Years of Experience Less than 10 years More than 11 years School District Demographics Geographic Locale Urban Suburban Rural Practice Setting Early Childhood/Preschool Elementary School Middle School High School Other N 78 1 1 2 1 13 24 15 13 13 5 23 37 23 34 49 N 25 36 22 4 47 11 15 6 M % of Racial Minority Students % of Students of Low SES 43.4 59.8 94 % 94.0 1.2 1.2 2.4 1.2 15.7 28.9 18.1 15.7 15.7 6.0 27.7 44.6 27.7 41.0 59.0 % 30.1 43.4 26.5 4.8 56.6 13.3 18.1 7.2 SD 32.7 27.0 Table 2 Descriptive Statistics for the Dependent Variables Measures Coach-Teacher Alliance Scale Reading Behavior Teacher Expectations Scale Reading Behavior Student-Teacher Relationship Scale Reading Behavior N M SD Skew Kurtosis 86 83 67 79 86 84 4.05 3.95 2.88 2.90 3.43 3.02 0.59 0.70 0.43 0.40 0.61 0.55 -0.09 -0.18 0.06 -0.10 -0.26 -0.01 -0.64 -0.65 -0.17 0.12 -0.55 -0.54 95 Table 3 Bivariate Correlations between Dependent Variables Variable 1. Collaboration- Behavior 2. Collaboration- Reading 1 1 .52** 1 2 3 4 5 6 3. Teach Expectations- .22 .06 1 Behavior 4. Teacher Expectations – Reading .32* .57** .12 1 5. Student-Teacher Rel. .44** .19 .63** .18 1 – Behavior 6. Student-Teacher Rel. .07 .66** - – Reading .06 1 .58** . 0 1 **Correlation is significant at the p < 0.01 level. *Correlation is significant at the p < 0.05 level. 96 Table 4 Main Effects of Student and Teacher Race on Perceptions of Collaboration, Teacher Expectations, and Student-Teacher Relationship Measure df Student M SD F p ηp 2 Race Reading Collaboration Teacher Expectations Student-Teacher Relationship 1 1 1 Black White 4.01 4.13 0.68 0.44 Black White 2.87 2.81 0.46 0.45 Black White 3.38 3.63 0.66 0.50 0.56 0.46 .005 0.24 0.63 .012 0.14 0.14 0.05 Behavior Teacher Race M SD F p ηp 2 Black White 4.29 3.85 0.68 0.44 Black White 2.96 2.71 0.46 0.45 Black White 3.58 3.42 0.66 0.50 7.96 .007** 0.15 3.83 0.06 0.08 0.93 0.34 0.02 Measure df Student M SD F P ηp 2 Race Teacher Race M SD F p ηp 2 Collaboration Teacher Expectations 1 1 Black White 4.27 3.65 0.45 0.74 Black White 2.96 2.91 0.40 0.42 23.89 < .001** 0.31 0.21 0.65 .004 Student-Teacher 3.17 3.10 Relationship ** p < 0.0125 due to Bonferroni adjustment Black White 1 0.50 0.50 0.01 0.51 0.53 97 Black White 4.29 3.63 0.45 0.74 Black White 3.00 2.91 0.40 0.42 Black White 3.15 3.10 0.51 0.53 26.50 < .001** 0.33 0.32 0.57 0.01 0.14 0.71 .003 Table 5 Language Modifications for All Measures (CTAS, TE, STRS) Measure Original Item Modified Item Coach-Teacher Alliance Scale (Working Relationship) The teacher and I trust one another. The teacher was approachable. Teacher Expectations Student-Teacher Relationship Scale The teacher and I worked together collaboratively. Overall, the teacher showed a sincere desire to improve his/her classroom. The services I provided incorporated the teacher’s view. He or she will probably have a good school report at the end of this school year. He or she performs well in school. He or she will probably have a successful school career. He or she is an intelligent student. He or she will probably have a high score on the final elementary school achievement tests. I share an affectionate, warm relationship with this child. This child and I always seem to be struggling with each other. This child values his/her relationship with me. This child easily becomes angry with me. 98 The teacher and I could trust one another. The teacher would be approachable. The teacher and I could work together collaboratively. Overall, the teacher seems to show a sincere desire to improve her classroom. The services I would provide would incorporate the teacher’s view. The teacher seems to think he will probably have a good school report at the end of the year. The teacher seems to think he performs well in school. The teacher seems to think he will probably have a successful school career. The teacher seems to think he is an intelligent student. The teacher seems to think he will probably have a high score on the final school achievement tests. The teacher seems to share an affectionate, warm relationship with the child. The child and the teacher seem to always struggle with each other. The teacher seems to value her relationship with the child. The teacher seems to easily become angry with the child. Table 5 (cont’d) It is easy to be in tune with what this child is feeling. Dealing with this child drains my energy. When this child is in a bad mood, I know we’re in for a long and difficult day. This child’s feelings toward me can be unpredictable or can change suddenly. It seems easy for the teacher to be in tune with what the child is feeling. Dealing with the child seems to drain the teacher’s energy. When the child is in a bad mood, the teacher knows they are in for a long and difficult day. The teacher’s feelings toward the child seem unpredictable or that they change suddenly. 99 APPENDIX C: FIGURES FOR PAPER 1 Figure 1 Variations of Racial Match within the Consultation Triad Student Teacher Consultant Racial match among all members Racial match among all members Racial mismatch between students and teachers; racial match between teachers and consultants Racial mismatch between students and teachers; racial match between teachers and consultants Racial match between students and teachers; racial mismatch between teachers and consultants Racial match between students and teachers; racial mismatch between teachers and consultants Racial match between students and consultants; racial mismatch between teachers and consultants Racial match between students and consultants; racial mismatch between teachers and consultants Race A Race B Race A Race B Race A Race B Race A Race B Race A Race B Race B Race A Race A Race B Race B Race A Race A Race B Race B Race A Race B Race A Race A Race B Note. Figure adapted with permission from Ingraham (2000). This study examines the racial match and mismatch between White and Black members of the consultation triad. However, in 100 Figure 1 (cont’d) practice, other races may be represented (e.g., Indigenous, Latinx, Asian American) and consultation triads may include multiple races simultaneously (e.g., Race C), which have not been illustrated for simplicity. 101 Figure 2 Depiction of Counterbalancing across Reading and Behavior Vignettes Condition Reading Behavior 1 2 3 4 White Teacher A White Student A White Teacher B White Student B White Teacher B Black Student C White Teacher A Black Student D Black Teacher C Black Student D Black Teacher D Black Student C Black Teacher D White Student B Black Teacher C White Student A 102 Figure 3 Significant Main Effects across Collaboration s e r o c S n a e M 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 Behavior Black Student White Student Black Teacher Reading White Teacher 103 Figure 4 Significant Interaction Effect on Collaboration s e r o c S n a e M 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Black Student - Black Teacher Black Student - White Teacher White Student - Black Teacher White Student - White Teacher Student-Teacher Interaction 104 APPENDIX D: RECRUITMENT EMAIL FOR PAPERS 1-2 Dear NASP member (NAME for districts), I am a doctoral student at Michigan State University, under the supervision of Dr. Courtenay Barrett. I am conducting a research study seeking to understand the perspective of practicing school psychologists on the school-based consultation process. Information from this study may benefit other school psychologists, public-school teachers, and school-aged children in the future. I am inviting your participation in this study, which will involve taking an online survey that can be accessed directly through this link: http://XXX. The survey will take approximately 30 minutes. To be eligible to participate you must have received a degree, or training in School Psychology and currently practice school psychological services in a public-school setting. Your participation in this study is voluntary. There is no penalty if you choose not to participate or to withdraw from the study at any time. If you choose to withdraw from the study, all data collected from you will be destroyed through deletion of files. Compensation for participating in this study is not available. Your responses will be anonymous, and confidential. That is, personal identifiers will not be collected in order to maintain your confidentiality. The results of this study will be used in my research apprenticeship, and may be used in reports, presentations, or publications - but your name will not be known or used. There are no foreseeable risks or discomforts to your participation. If you have any questions concerning the research study, please contact Andryce Clinkscales by phone at 616-644-8111 or via email at: clinksc4@msu.edu, or her graduate advisor Dr. Courtenay Barrett at morsicou@msu.edu. If you have any questions about your rights as a participant in this research, or if you feel you have been placed at risk, you can contact the Chair of the Institutional Review Board at irb@msu.edu. Thank you for your time and consideration, Andryce Clinkscales Doctoral student, College of Education, Michigan State University 105 APPENDIX E: IRB PROPOSAL FOR PAPERS 1-2 RESEARCH PARTICIPATION AND CONSENT FORM Study Title: School Psychologists’ Perceptions of Consultation Researcher and Title: Andryce Clinkscales, Doctoral Student; Dr. Courtenay Barrett, Supervisor Department and Institution: Department of Counseling, Educational Psychology, and Special Education/Michigan State University Contact Information: email: clinksc4@msu.edu; phone: 616-644-8111 Sponsor: Graduate Advisor: Courtenay Barrett, Ph.D.; email: morsicou@msu.edu BRIEF SUMMARY You are being asked to participate in a research study. Researchers are required to provide a consent form to inform you about the research study, to convey that participation is voluntary, to explain risks and benefits of participation including why you might or might not want to participate, and to empower you to make an informed decision. You should feel free to discuss and ask the researchers any questions you may have. You are being asked to participate in a research study that seeks to understand the perspective of practicing school psychologists on the school-based consultation process. Your participation in this study will take about 30 minutes. You will be asked to respond to survey questions in an online format. The are no foreseeable risks of participating in this study. You will likely not directly benefit from your participation in this study. However, your participation in this study may contribute to the understanding of school psychologists’ perspective of providing consultation services in schools. PURPOSE OF RESEARCH The purpose of this research study is to learn more about school psychologists’ perspective on collaborative relationships with teachers, perceptions of teacher expectations for their students, and the quality of student-teacher relationships. WHAT YOU WILL BE ASKED TO DO You will watch four videos describing student referral problems and be asked to respond to online survey items after watching these videos. You are free to skip any questions that you would prefer not to answer. You will submit your survey responses online, using a computer or smartphone. You will fill out this survey only one time. POTENTIAL BENEFITS You may not benefit personally from being in this study. However, we hope that, in the future, other people might benefit, such as other school psychologists, public-school teachers, and school-aged children because this study adds to the empirical research on school-based consultation. POTENTIAL RISKS There are no foreseeable risks involved in participation for this study. There is potential discomfort that may come from providing survey responses for 30 minutes if you are not accustomed to viewing your computer screen for that length of time. PRIVACY AND CONFIDENTIALITY Data and any identifying information, including signed consent forms, will be kept for a period of 6 months after electronic data is withdrawn from the survey system. The electronic data will be stripped of identifying information, such as demographic information (i.e., gender, age, race/ethnicity, geographical region, work setting, and degree obtained) and survey responses, 106 stored in an encrypted file on a password-protected computer; this information may be used in future research without anyone knowing it is your information. YOUR RIGHTS TO PARTICIPATE, SAY NO, OR WITHDRAW You have the right to say no to participate in the research. You can stop at any time after it has already started. There will be no consequences if you stop and you will not be criticized. You will not lose any benefits that you normally receive. COSTS AND COMPENSATION FOR BEING IN THE STUDY Aside from the time it will take you to complete the online survey, there are no additional costs for your participation. You will not receive money or any other form of compensation for participating in this study. RESEARCH RESULTS You may request findings from this study if desired. FUTURE RESEARCH Information that identifies you might be removed from the demographic characteristics collected (i.e., gender, age, race/ethnicity, geographical region, work setting, and degree obtained) and survey responses. After such removal, the demographic characteristics and survey responses may be used for future research studies or distributed to another investigator for future research studies without additional informed consent from you. CONTACT INFORMATION If you have concerns or questions about this study, such as scientific issues, how to do any part of it, or to report an injury, please contact the researchers Andryce Clinkscales by phone at 616- 644-8111 or via email at: clinksc4@msu.edu or her graduate advisor Dr. Courtenay Barrett via email at: morsicou@msu.edu. If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at 4000 Collins Rd, Suite 136, Lansing, MI 48910. DOCUMENTATION OF INFORMED CONSENT Your signature below means that you voluntarily agree to participate in this research study. Signature________________________________________ Date_____________________________ You will be given a copy of this form to keep. 107 APPENDIX F: VIGNETTES FOR PAPERS 1-2 Video 1 Teacher Race: White Student Name: Cody (White) Script: I have a student, Cody, that is really struggling with his behavior in my class. I am also concerned with his relationships with peers. He doesn’t show interest in engaging with others during play time or at recess. When he does interact with them, Cody is bossy and manipulative with peers. I want to find a way for Cody to interact with his peers, in a more positive light, to build positive relationships with them. For example maybe he could have a buddy throughout the day. This last week has been especially hard for Cody though, I do not know why. I call him an energy consumer because he requires a lot of my energy throughout the day. He is constantly moving and has a hard time paying attention. Like, this past week, during independent seat work, Cody would make comments to his peers to encourage them to get off-task. For example he would say “Hey you’re not doing that right, watch me” like he’s taunting or teasing them to get attention from his peers. I try to redirect all of my students to get back on-task when he does this, but it only works some of the time. I am trying not to get frustrated with him and treat him the same as his classmates but it’s hard. (Ingraham, 2003; Intervention Central) Video 2 Teacher Race: White Student: Darius (Black) Script: My student, Darius, is having trouble with building peer relationships and completing assignments in class. He does not initiate relationships and has very low self-esteem in this area. Darius does interact with peers somewhat, but these interactions are often negative. For example 108 he has said to me before that no one likes him. When I tell the class to begin assignments, Darius will verbally refuse, saying “I don’t have to” and calling out comments like “My mom says that I am already good at reading so I can play.” Darius has not been responding to the consequences I have in place for his problem behaviors. I usually reprimand him and say that he should be following directions, but it is hard to find something motivating enough for him to listen. I want him to make gains in his social relationships and also develop a plan to address his behavior problems. I want to help him be a well-behaved student, but I do not know how, hopefully you can give me some ideas (Ingraham, 2003; Intervention Central). Video 3: Teacher Race: Black Student: Greg (White) Script: Greg is one of my more difficult students. I don’t have a very large class but it seems like he needs to be disciplined as much as the rest of the class combined. He often misses out on activities because of his problem behavior. He has had a tough life, but I do not know much about his home environment. Greg’s parents never come to conferences, so I have not had a chance to talk to them about his problem behaviors. During transitions between academic activities, Greg is inattentive and sits at his desk, with his head down. He often needs a reminder to gather the required materials or transition to the front carpet for the next activity. He seems to want to be in control of the situation and will silently refuse to listen by staying at his desk. He also rarely makes eye contact even when I am standing right in front of him and talking. I want to find a way to change Greg’s behaviors so that he will actively participate and be engaged in class, for example using a sticker chart for good behavior. I do not know what else I can do to promote positive changes (Ingraham, 2003; Intervention Central). 109 Video 4: Teacher Race: Black Student: Jamal (Black) Script: Jamal is a student in my class with behavioral issues. I need help dealing with them, I do not know what to do with him. He is timid and shy. I wish he was more enthusiastic about our activities, like his peers. Jamal requests bathroom breaks so often that I think he is just trying to escape the activities. One time he asked to go the bathroom 20 times in one day, I counted. He fails to complete his work, like he has only turned in homework assignments a handful of times, and his now very behind his peers. I try to have a conference with him to talk about why he is not doing his work, but he does not talk to me. Jamal will stand in front of me with his head down, looking at the floor. During large group instruction, he plays with his shoes and looks around the room. He often does not respond when I give my students the opportunity to during this large group time. For example, I asked everyone to share what they had to eat for lunch today and he just stared out the window. I want Jamal to communicate with me more and turn in his homework assignments (Ingraham, 2003; Intervention Central). 110 APPENDIX G: SURVEY ITEMS FOR PAPERS 1-2 Inclusionary Criteria: Participants must respond “Yes” to BOTH questions to be included in this study. • Have you received a degree or training in school psychology, excluding related fields such as special education? o Yes o No • Are you currently practicing full-time as a school psychologist in a public-school district? o Yes o No Demographic Characteristics • Age ____________ • Gender __________ • Please choose your race/ethnicity from the following options AND print origins: o White __________ o African American __________ o American Indian or Alaska Native __________ o Asian or Pacific Islander __________ o Hispanic, Latino, or Spanish origin __________ o More than two races __________ o Some other race __________ (based on data from U.S. Census Bureau, 2020) • What is your highest level of education completed in school psychology? o Master’s (e.g., M.A., M.S., M.Ed) o Education Specialist (e.g., Ed.S. or A.G.S.) o Doctoral (e.g., Psy.D, Ph.D or Ed.D) • How many years have you been a practicing school psychologist? ______ years • Are you currently practicing as a school psychologist in a school setting? (e.g., public or private school) o Yes o No o If yes, please specify the state you currently work in: _____ o If yes, please specify the type of setting you currently work in: ▪ Elementary School ▪ Middle School ▪ High School 111 • Which option best describes the region of the school you currently work in? o Urban o Suburban o Rural • Please describe characteristics of the student population (e.g., race/ethnicity composition, SES) ________________________________ 112 APPENDIX H: TABLES FOR PAPER 2 Table 6 Descriptive Statistics for the Outcome Variables Measures Coach-Teacher Alliance Scale Teacher Expectations N 71 73 Min Max M SD Skew Kurtosis 3.10 5.00 4.00 .55 .35 -.93 2.00 3.60 2.90 .31 .16 .39 Student-Teacher Relationship Scale - Behavior 72 1.71 4.29 3.04 .55 -.13 -.50 Student-Teacher Relationship Scale - Reading Cultural Competence 73 73 2.00 4.57 3.44 .62 -.27 -.57 4.00 5.90 5.13 .34 -.05 1.03 Note: Collaboration, Teacher Expectations, and Student-Teacher Relationship scales on a 5-point Likert scale; Cultural Competence scale on a 6-point Likert scale. 113 Table 7 Bivariate Correlations between Variables Variable 1. Collaboration 1 1 2. Teacher Expectations .45** 1 2 3 4 5 1 .14 .10 1 -.01 1 3. Student-Teacher Rel. – Behavior 4. Student-Teacher Rel. – Reading .40** .57** .38** 5. Cultural Competence .14 ** p < 0.001 level (two-tailed). * p < 0.01 level (two-tailed). .36* -.01 114 Table 8 Regression Models for Collaboration Model 1 Model 2 Model 3 Model 4 Variables B SE B SE B SE B Intercept Middle school High school Early childhood/Preschool Other setting Cultural competence Teacher expectations Student-teacher relationship (Behavior) Student-teacher relationship (Reading) White teacher-White student Black teacher-Black student Black teacher-White student 3.937 .013 .388 .288 -.070 .088 .193 .179 .285 .238 3.021 -.027 .360 .305 -.070 .181 1.026 .198 .182 .286 .238 .202 -.065 -.139 .218 .454 -.032 .182 .658* .167 1.039 .171 .155 .237 .200 .169 .244 .123 .417 -.055 .182 314 -.053 .167 .417 .154 .203 .098 .312** -.563** .115 .145 SE .895 .143 .129 .200 .168 .143 .206 .104 .089 .146 .138 .135 R2 R2 change .083 .083 .095 .011 .414 .320** .624 .210** Note: α = 0.0125; p < 0.01*; p < 0.001** 115 Table 9 Regression Models for Teacher Expectations Variables Intercept Middle school High school Early childhood/Preschool Other setting Cultural competence Collaboration Student-teacher relationship (Behavior) Student-teacher relationship (Reading) White teacher-White student White teacher-Black student Black teacher-Black student R2 R2 change Note: α = 0.0125; p < 0.01*; p < 0.001** Model 1 Model 2 Model 3 Model 4 B SE B SE B SE B SE 2.834 .179 .129 -.126 .033 .048 .106 .099 .157 .131 3.081 .190 .137 -.131 .033 -.049 .568 .110 .101 .158 .132 .112 1.774 .212 .065 -.136 .107 -.122 .162* .237** .090 .462 .081 .078 .120 .098 .083 .060 .054 .049 1.770 .208 .060 -.123 .119 -.119 .159 .234** .092 -.004 .023 -.021 .502 .084 .080 .125 .103 .088 .078 .057 .059 .101 .005 .084 .074 .074 .076 .003 .520 .444** .523 .003 116 Table 10 Regression Models for Student-Teacher Relationship (Behavior) Variables B SE B SE B SE B SE Model 1 Model 2 Model 3 Model 4 Intercept Middle school High school Early childhood/Preschool Other setting Cultural competence Collaboration Teacher expectations Student-teacher relationship (Reading) White teacher-White student White teacher-Black student Black teacher-Black student R2 R2 change Note: α = 0.0125; p < 0.01*; p < 0.001** 3.086 -.029 -.027 -.122 -.229 .091 .199 .186 .295 .247 2.144 -.071 -.056 -.105 -.229 .186 1.063 .205 .189 .296 .247 .209 -1.060 -.258 -.219 -.054 -.263 .204 .174 1.010** -.126 1.054 .173 .159 .250 .202 .173 .129 .230 .103 -.391 -.291 -.213 -.024 -.236 .129 .239 .952** -.216 .085 -.231 -.076 1.172 .174 .160 .253 .207 .179 .160 .233 .119 .204 .170 .162 .015 .015 .027 .012 .386 .359** .417 .031 117 Table 11 Regression Models for Student-Teacher Relationship (Reading) Variables Intercept Middle school High school Early childhood/Preschool Other setting Cultural competence Collaboration Teacher expectations Student-teacher relationship (Behavior) White teacher-White student White teacher-Black student Black teacher-Black student R2 R2 change Note: α = 0.0125; p < 0.01*; p < 0.001** Model 1 Model 2 Model 3 Model 4 B SE B SE B SE B SE 3.436 -.008 .302 -.222 -.103 .098 .215 .200 .318 .266 3.443 -.007 .302 -.222 -.103 -.001 .1153 .222 .205 .321 .268 .227 1.086 -.122 .095 -.264 -.143 .004 .323 .581 -.192 1.303 .217 .199 .306 .251 .215 .156 .317 .156 1.233 -.179 .056 -.200 -.109 -.151 .558** .432 -.250 .880** .273 .292 1.188 .190 .175 .271 .225 .192 .160 .279 .137 .186 .182 .168 .052 .052 .052 .000 .227 .175* .442 .216** 118 APPENDIX I: FIGURE FOR PAPER 2 Figure 5 Multicultural Consultee-Centered Consultation Theory of Change 119 Table 12 Description of Articles Included in Review APPENDIX J: TABLES FOR PAPER 3 Author(s)/Year Setting Type of Article/Methodology De-implementation Themes Prasad & Ioannidis (2014) Healthcare Editorial and conceptual analysis Gupta et al. (2017) Healthcare Norton et al. (2017) Healthcare Wang et al. (2018) Healthcare Qualitative study (grounded theory): 30-minute semi-structured interviews (N = 15 primary care physicians) Systematic review of de- implementation research grants (N = 20) funded across all 27 NIH Institutes and Centers (ICs) and the Agency for Healthcare Research and Quality (AHRQ) from 2000 to 2017 Conceptual analysis Definition: Elimination of ineffective and harmful practices. Benefits: More efficient resource allocation and cost-effective health outcomes. Influences: Cognitive, political, evidence-base. Definition: Unlearning of clinical practices. Influences: Ease of implementation, personal discomfort, evidence-base Influences: Few NIH- and AHRQ- funded research grants have focused on studying de-implementation – more research is needed Definition/Process: partial reduction, complete reversal, substitution with related replacement, and substitution with unrelated replacement 120 Table 12 (cont’d) Upvall & Bourgault (2018) Healthcare Systematic review and conceptual analysis Pinto & Witte (2019) Healthcare Conceptual analysis Erwin et al. (2020) Healthcare Cross-sectional survey of state health department practitioners (N = 1329) Norton & Chambers (2020) Healthcare Conceptual analysis Prusaczyk et al. (2020) Multiple Conceptual analysis Definition: Complex process that involves the use of specific strategies to terminate clinical practices that do not promote positive outcomes Process: Bidirectional influence by many parts of a system, involves the consideration of the political climate (e.g., the promotion of more-profitable interventions) Definition: Mis-implementation, ending effective interventions (inappropriate termination) and continuing ineffective interventions (inappropriate continuation) Process: Complex, nuanced process due to multi-level factors (e.g., intervention + patient + health professional + organization) involved and the context in which de-implementation occurs Influences: acceptability, adoption, appropriateness, feasibility, fidelity, cost, penetration, and sustainability related to de-implementation; and when applicable, outcome targets of measurement include practice, process, and stakeholders 121 Table 12 (cont’d) Massatti et al. (2008) McKay et al. (2018) Mixed methods: Survey administration and semi-structured interviews of key stakeholders (administration, implementers, and other staff; N = 21 de-adopters, 30 implementers) Conceptual analysis Community Mental Health/Public Health and Social Service Community Mental Health/Public Health and Social Service Pinto & Park (2019) Community Mental Health/Public Health and Social Service Qualitative study (grounded theory): reviewed records of the implementation of HIV prevention services (N = 36 agencies in New York City) Influences: financial resources, external influences (e.g., community members), staffing issues, IMHP characteristics, system characteristics, and implementation fidelity Process: may be viewed as an outcome itself, and involves intervention sustainability/replacement Influences: contextual factors (e.g., internal and external) Definition: discontinuation of interventions that should be stopped Influences: environmental (e.g., regulations and resources), organizational (e.g., capacity for staff training), and both provider- and client- level factors (e.g., provider’s experience and client’s preferences); and it influences implementation of a new approach (e.g., practices, organizational structures, contexts) 122 Table 12 (cont’d) Nadeem & Ringle (2016) Education Qualitative study (grounded theory): 30-minute semi-structured interviews (N = 14 school social workers and 2 central district administrators) Prasad & Cifu (2011) Healthcare Conceptual Wang et al. (2015) Healthcare Secondary data analysis of responses from specialist societies and journals about evidence for reversal of practice (N = 177 responses) 123 Process: phase 1, partial de-adoption; phase 2, complete de-adoption (district- wide). Phase 1 emphasizes inner context or intra-organizational factors (e.g., workforce stability). Phase 2 emphasizes outer context or macro- level challenges to sustainment (e.g., district leadership). Large, urban school district; CBITS program implemented for 3 years. Definition: replacement (practice is supplanted by one that works better) versus reversal (new clinical practice contradicts prior one that did not work/was harmful) Influences: financial rewards/incentives/cost, desire/beliefs, undermines trust in medical system, potential benefits Definition: practice is inferior to a prior standard and is inconsistently translated into practice Influences: profitability, importance to individuals, bias, investment; specialist societies, journals, conservatism; group values Table 12 (cont’d) Harvey & McInnes (2015) Healthcare Editorial Davidoff (2015) Healthcare Conceptual Analysis 124 Process: Role of specialist societies and journals in de-implementation and timeline Definition: disinvestment – ineffective, inappropriate or unnecessary care Benefits: potential reduction of economic constraints Influences: “combination of individual (e.g., beliefs about evidence, past experience), interprofessional (e.g., influence of peers), and contextual (e.g., economic, industry and marketing influences) factors, policies (at the national and local level) – historical, economic, political, and social contexts – “knowledge brokers or evidence- based practice mentors… facilitators and opinion leaders, using a combination of strategies such as interactive education, audit and feedback, reminder systems, and patient-mediated interventions” Definition: Undiffusion is a process involving abandoning established practices Table 12 (cont’d) McKay et al. (2017) Public Health, Healthcare Mixed Methods: Systematic review & Case study with an agency who used RESPECT program (Staff, N=7) Admon & Hyzy (2017) Critical Care, Healthcare Conceptual Analysis Roman, BR., Asch, DA. (2014) Healthcare Conceptual Analysis Ubel & Asch (2015) Healthcare Conceptual Niven et al. (2016) Healthcare Conceptual 125 Influences/Process: The knowledge that drives diffusion is socially constructed, and rarely smooth. Slow and tightly controlled removal is necessary as opposed to rapid/unselective diffusion. Definition: De-adoption occurs when activities associated with an evidence- based practice conclude or are abandoned. Process: Historical engagement in informal de-adoption using clinical guidelines Influences: Performance measures for de-adoption Influences: Finances, biases; psychological biases that are asymmetrical in favoring adoption over de-implementation Process: Messaging approaches Definition: Eliminating entrenched, often costly practices that have lost value because of new evidence or competing approaches Influences: Psychological biases, Definition: Discontinuance of practice or service found to be ineffective or harmful following previous adoption Table 12 (cont’d) Scott & Elshaug (2013) Brownson et al. (2015) Healthcare Conceptual Public Health Cross-sectional survey of public health practitioners (N = 944) Process: Identification and prioritization, coordination, evaluating effects of de-implementation, sustaining de-implementation Influences: Ethics Influences: Psychological biases Definition: Mis-implementation, de- adoption of effective practices or continuation of ineffective practices Influences: Funding, support, continued by other organization 126 Table 13 PRISMA 2018 Checklist for Scoping Reviews SECTION ITEM PRISMA-ScR CHECKLIST ITEM 1 Identify the report as a scoping review. REPORTED ON PAGE # Click here to enter text. TITLE Title ABSTRACT Structured summary INTRODUCTION Rationale Objectives METHODS Protocol and registration 2 3 4 5 Eligibility criteria 6 Information sources* Search 7 8 Provide a structured summary that includes (as applicable): background, objectives, eligibility criteria, sources of evidence, charting methods, results, and conclusions that relate to the review questions and objectives. Describe the rationale for the review in the context of what is already known. Explain why the review questions/objectives lend themselves to a scoping review approach. Provide an explicit statement of the questions and objectives being addressed with reference to their key elements (e.g., population or participants, concepts, and context) or other relevant key elements used to conceptualize the review questions and/or objectives. Indicate whether a review protocol exists; state if and where it can be accessed (e.g., a Web address); and if available, provide registration information, including the registration number. Specify characteristics of the sources of evidence used as eligibility criteria (e.g., years considered, language, and publication status), and provide a rationale. Describe all information sources in the search (e.g., databases with dates of coverage and contact with authors to identify additional sources), as well as the date the most recent search was executed. Present the full electronic search strategy for at least 1 database, including any limits used, such that it could be repeated. 1 3 3 4 4-5 4-5 4 127 Table 13 (cont’d) Selection of sources of evidence† Data charting process‡ Data items Critical appraisal of individual sources of evidence§ Synthesis of results RESULTS Selection of sources of evidence Characteristics of sources of evidence Critical appraisal within sources of evidence Results of individual sources of evidence Synthesis of results DISCUSSION Summary of evidence 9 10 11 12 13 14 15 16 17 18 19 State the process for selecting sources of evidence (i.e., screening and eligibility) included in the scoping review. Describe the methods of charting data from the included sources of evidence (e.g., calibrated forms or forms that have been tested by the team before their use, and whether data charting was done independently or in duplicate) and any processes for obtaining and confirming data from investigators. List and define all variables for which data were sought and any assumptions and simplifications made. If done, provide a rationale for conducting a critical appraisal of included sources of evidence; describe the methods used and how this information was used in any data synthesis (if appropriate). Describe the methods of handling and summarizing the data that were charted. Give numbers of sources of evidence screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally using a flow diagram. For each source of evidence, present characteristics for which data were charted and provide the citations. If done, present data on critical appraisal of included sources of evidence (see item 12). For each included source of evidence, present the relevant data that were charted that relate to the review questions and objectives. Summarize and/or present the charting results as they relate to the review questions and objectives. 4-5; Figure 1 N/a 5 5 5 Figure 1 Table 1 6 6-11 6-11; Table 1 Summarize the main results (including an overview of concepts, themes, and types of evidence available), link to the review questions 11-15 128 Table 13 (cont’d) Limitations Conclusions 20 21 and objectives, and consider the relevance to key groups. Discuss the limitations of the scoping review process. Provide a general interpretation of the results with respect to the review questions and objectives, as well as potential implications and/or next steps. 14 15 FUNDING Funding 22 Describe sources of funding for the included sources of evidence, as well as sources of funding for the scoping review. Describe the role of the funders of the scoping review. From: Tricco, AC, Lillie, E, Zarin, W, O'Brien, KK, Colquhoun, H, Levac, D, Moher, D, Peters, MD, Horsley, T, Weeks, L, Hempel, S et al. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annals of Internal Medicine, 169(7), 467-473. N/a 129 Table 14 Definitions of De-implementation in Prior Research Definition Author "De-adoption is the abandonment of an innovation at any stage of implementation" Massati et al., 2008, p. 50 “The unlearning of an outmoded clinical practice and learning of a new one” Gupta et al., 2017, p. 2 "The discontinuation of interventions that should no longer be provided" Mckay et al., 2018, p. 190 "The process of identifying and removing harmful and low- value practices based on tradition and without scientific support" Upvall & Bourgalt, 2018, p. 379 “De-implementation involves interventions that are (a) harmful or not effective, (b) not the most effective or proficient to provide, or (c) no longer necessary” "Mis-implementation includes both ending effective interventions (inappropriate termination) and continuing ineffective interventions (inappropriate continuation)" Pinto & Witte, 2019, p. 239 Erwin et al., 2020, p. 6 “The abandonment of an outmoded or disproven clinical practice” Wang et al., 2018, p. 104 130 APPENDIX K: FIGURES FOR PAPER 3 Figure 6 Flow Diagram of Identification Procedures 131 PsycInfo (n = 43)IdentificationERIC (n = 18)EBSCOHost (n = 946)Academic Search Complete (n = 376)JSTOR (n = 1,004)Total records identified (n = 2,387)Non-peer reviewed studies removed (n = 322)Studies published prior to 2005 removed (n = 680)ScreeningRecords after non-peer reviewed and those published prior to 2005 removed (n = 1,385)Studies published outside of the US removed (n = 1158)Duplicates removed (n = 4)Did not meet inclusion criteria (n = 123)Titles and abstracts reviewed (n = 100)EligibilityRecords after screening(n = 13)Did not meet inclusion criteria (n = 85)Inaccessible (n = 2)Studies identified via hand searching references (n = 12)IncludedStudies included (n = 25) Figure 7 Conceptual Model of Including Culture during De-implementation in Schools 132