CHANGE FOR THE BETTER: ASSESSING READINESS TO ADOPT APPLIED BEHAVIOR ANALYSIS IN MICHIGAN PUBLIC SCHOOLS By Brian Patrick Brutzman A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology - Doctor of Philosophy 2024 ABSTRACT Many students diagnosed with autism spectrum disorder (ASD) need evidence-based intervention in school, and applied behavior analysis (ABA) offers an array of treatment options shown to improve outcomes. However, in Michigan, ABA has not been thoroughly integrated into public schools. The present dissertation assessed the mechanisms thought to predict organizational readiness for change (ORC) among Michigan public educators faced with the potential adoption of specialized ABA classrooms. Findings from a representative sample of Michigan K-12 principals and administrators (N = 346) suggest a two-factor structure (i.e., commitment and efficacy) for organizational readiness for change, (χ2 = 118.86, df = 26, p < .001, RMSEA = 0.10, SRMR = 0.05, CFI = 0.97, TLI = 0.95). Findings suggest that the system climate of a school or district has a direct positive relationship with change commitment, (b = 0.28, p < .001) and an indirect relationship with change commitment, mediated by change valence, (b = 0.23, p < .001, 95% CI [0.13, 0.34]). Findings also suggest that system climate has a direct positive relationship with change efficacy, (b = 0.39, p < .001), and an indirect relationship with change efficacy, mediated by resource perceptions, (b = 0.14, p < .001, 95% CI [0.07, 0.22]). Implications for future research include the applicability of the two-factor structure of ORC to the educational field, the salience of system climate, change valence, and resource perceptions in readiness for change, and the importance of operationalization and measurement of ORC and related constructs when mapping them onto a specific change in a unique context. Implications for practice include a need to focus on both stakeholder commitment and efficacy, the potential importance of alignment between a school's climate and the proposed intervention, and an attendance to the mechanisms related to organizational change which will likely make the adoption of ABA classrooms in Michigan public schools more successful. This dissertation is dedicated first and foremost to my wife Jenna, whose tireless support and willingness to uproot her life for my dreams made this work possible. I would also like to dedicate this dissertation to my two sons, James and Luke. Underlying this research and my entire professional and personal life is a deep desire to make them proud. Finally, I would like to dedicate this dissertation to the students, families, educators, and colleagues that I have worked with over the course of my career. They are a constant inspiration for me to do more. iii AKNOWLEDGEMENTS I would first like to thank Dr. Jennifer Watling Neal for her incredible support for this project from its inception to its conclusion. Her patience, insight, generosity, and mentorship has improved every piece of this project and provided a truly exceptional graduate school experience. I will be forever thankful for it. I would also like to thank Dr. Zachary P. Neal for his amazing support, especially as it relates to the analytic approach and code for the present dissertation. He provided the clarity necessary for me to tackle the present analyses, brought order to an otherwise unruly first attempt at coding this project, and was extremely generous with his time in the process. I would next like to thank Dr. Kaston D. Anderson Jr. for his thoughtful support on this project, his handwritten notes on my dissertation proposal, and the way that he challenged me to translate this research for a behavior analytic audience. As a result, I believe this dissertation is more oriented toward communities of practice and provides more actionable insight for stakeholders. Next, I would like to thank Dr. J. Kevin Ford, who proposed the idea of an organizational readiness for change approach when I brought up the issue of ABA integration in Michigan public schools. His insight provided a way into this large-scale, seemingly intractable organizational challenge. It also allowed me to study a problem that has deep personal and professional meaning. Finally, I would like to thank the undergraduate research assistants from Michigan State University who selflessly gave their time and energy to recruit respondents from across the state over a period of several years. This work would not have been possible without them. iv TABLE OF CONTENTS CHAPTER 1: INTRODUCTION AND THEORETICAL FRAMEWORK .......................1 CHAPTER 2: ORGANIZATIONAL READINESS FOR CHANGE & ABA CLASSROOMS .......................................................................................................24 CHAPTER 3: CONTEXTUAL FACTORS, CHANGE VALENCE, & CHANGE COMMITMENT ..........................................................................................54 CHAPTER 4: CONTEXTUAL FACTORS, INFORMATIONAL ASSESSMENT, & CHANGE EFFICACY ..................................................................................................75 CHAPTER 5: CONCLUSION ..........................................................................................97 REFERENCES ................................................................................................................112 APPENDIX A: IRB APPROVAL ...................................................................................121 APPENDIX B: RECRUITMENT EMAIL ......................................................................126 APPENDIX C: SURVEY INSTRUMENT FOR ISD ADMINISTRATORS .................127 APPENDIX D: SURVEY INSTRUMENT FOR SCHOOL PRINCIPALS ...................152 APPENDIX E: STANDARDIZED REGRESSION COEFFICIENTS ...........................178 APPENDIX F: SENSITIVITY ANALYSIS ...................................................................179 v CHAPTER 1: INTRODUCTION AND THEORETICAL FRAMEWORK INTRODUCTION Despite recent statewide adoption of licensure for professionals in the field of applied behavior analysis (ABA), Michigan public schools have not thoroughly integrated ABA into special education classrooms for students with challenges related to behavior and/or skill acquisition. This is particularly troubling for students diagnosed with autism spectrum disorder (ASD) and other developmental disabilities, given that the U.S. Surgeon General has endorsed intensive behavioral intervention for these individuals, stating: “Thirty years of research demonstrated the efficacy of applied behavioral methods in reducing inappropriate behavior and increasing communication, learning, and appropriate social behavior” (U.S. Department of Health and Human Services, 1999, p.164). In addition, Wilczynski et al. (2009) found that these treatments have the strongest research support and highest level of effectiveness for persons faced with these specific challenges. More than twenty thousand students enrolled in Michigan public schools met eligibility for ASD in the 2017-2018 school year, and this number reportedly grew to over twenty five thousand in the 2022-2023 academic year (Michigan Department of Education, 2017, n.d.-a). A more thorough integration of ABA into the public-school setting could improve services for these students. One way to approach this challenge is through an organizational readiness for change (ORC) perspective. Widespread integration of ABA into public schools will require a massive organizational shift, and educators will need to demonstrate that they are willing and able to take substantive steps (e.g., adopting a specialized ABA classroom) to meet the needs of their students. In Weiner’s (2009) widely cited, two-factor theory of ORC, the colloquial terms willing and able are operationalized as “change commitment” and “change efficacy,” respectively. 1 Essentially, this approach allows for the assessment of key stakeholders’ attitudes toward a particular organizational change, with the assumption that pledged commitment and a strong endorsement of the efficacy of a particular change together predict implementation. Drawing from ABA, developmental and organizational psychology, education, and implementation science, this dissertation examines whether Michigan educators are willing and able to adopt applied behavior analytic classrooms from an organizational readiness for change (ORC) perspective. I focus on Weiner’s (2009) two-factor theory of organizational readiness for change and test it in this specific context, using a representative sample of Michigan K-12 school principals and intermediate school district (ISD) administrators. Principals and ISD administrators were chosen because they are the key decision-makers who are best positioned to catalyze the adoption of ABA services in their schools or districts. This research informs future interventions that might make ABA services more accessible to thousands of students across the state. In this chapter, I will orient the reader to the major components of the dissertation. First, I will begin with an introduction to applied behavior analysis and a discussion of its potential efficacy in schools. Second, I will discuss potential barriers to the integration of ABA in Michigan public schools. Third, I will contextualize the choice of ABA classrooms as the specific organizational change for this project. Fourth, I will discuss Weiner’s (2009) theoretical framework of ORC in detail and apply it to the adoption of ABA classrooms in Michigan public schools. ABA in Schools Applied behavior analysis is “the science in which tactics derived from the principles of behavior are applied to improve socially significant behavior and experimentation is used to 2 identify the variables responsible for the improvement in behavior” (Cooper, Heron, & Heward, 2007, p. 690). Philosophically grounded in the science of behavior, and following in the tradition of Skinnerian experimental analysis, ABA is perhaps best understood as an overarching scientific framework that seeks to improve socially significant behavior in all contexts. This type of modification is possible through an analysis of the functional relationship between environmental variables and behavior itself, such that alterations in the environment yield different behavioral outcomes. For example, increases in socially significant behavior are often attributable to changes in behavioral contingencies, whereby problematic behavior no longer receives reinforcement (e.g., attention from others, access to preferred items), and alternative behavior (e.g., a communicative response) is reinforced instead. In addition to the broad evidence base that ABA-related interventions enjoy across environments (e.g., Steinbrenner et al., 2020), more specific evidence speaks to the effectiveness of intensive behavioral treatment delivered specifically in school settings for students with ASD. For example, McGarrell et al. (2009) found that intensive behavioral intervention delivered in a specialized school led to improvements in IQ and adaptive behavior in students with ASD and subsequent placement in mainstream education. These findings are comparable to Eikeseth et al. (2002; 2007) who noted that ASD students who received intensive behavioral treatment in mainstream public schools showed significant improvement on standardized tests over peers in an eclectic treatment group, and that these gains persisted at a 3-year follow up. Findings pertinent to specialized ABA classrooms within the mainstream school setting are similarly encouraging. Grindle et al. (2012) found that ASD students (aged 3-7 years) in an ABA classroom showed significant improvements on standardized measures of IQ and adaptive behavior compared to ASD counterparts who received “education as usual.” 3 ABA and Michigan Public Schools In the 2017-2018 school year, 20,595 children enrolled in Michigan schools met eligibility as a student with autism spectrum disorder (Michigan Department of Education, 2017). By the 2022-2023 school year, that number grew to 25,147 (Michigan Department of Education, n.d. -a). Compared to other states that boast a continuum of school placement options that integrate principles of ABA for students with ASD, Michigan has not yet incorporated ABA into its schools. Findings from a representative sample of educators working with ASD students in Michigan show that while 59% endorse the utilization of some form of ABA in their classrooms, half of these respondents only utilize ABA for five hours per week or less (Ferreri & Bolt, 2011). Within this same study, only 14% of classrooms employed some form of data collection (an essential component of ABA) at any point during a site visit from the research team (Ferreri & Bolt, 2011). The delivery of ABA programming in Michigan public schools falls short of recent clinical best practices outlined by the Council of Autism Service Providers (CASP). Specifically, CASP (2020) designates 10-25 hours per week as focused ABA treatment and 30-40 hours per week as comprehensive ABA treatment. These best practices also require 2 hours of supervision with a board-certified behavior analyst for every 10 hours of direct treatment (CASP, 2020). The intermittent utilization of ABA strategies in Michigan schools is problematic for another reason. Despite the wide availability of “eclectic” interventions (i.e., combining multiple instructional approaches) in public early intervention and special education placements, these have consistently proven inferior to an intensive behavior analytic approach for children with ASD across various domains of skill acquisition (Eikeseth et al., 2002; Howard et al., 2005; 4 Zachor et al., 2007). The available evidence points to the efficacy of a sustained, intensive ABA- based approach to maximize the likelihood of positive outcomes for students with ASD. Given the resource constraints and competing priorities in public schools, best practices in ABA might not be feasible in the school setting. However, school-based service provision is not necessarily bound by the same rules as ABA provided by an individual’s insurance company and allows for the flexibility to treat larger groups of students with fewer time constraints. For example, in an ABA classroom with 10 students, paraprofessionals could implement the majority of direct intervention to approximate the hours of focused ABA treatment that a student would receive in an insurance-funded clinic. These paraprofessionals could operate under the supervision of a teacher for a sizeable portion of the time, and a BCBA could spend an hour each week per child providing consultation and feedback as needed. This would equate to about 10 hours per week of BCBA time in one ABA classroom. In this scenario, the number of hours of ABA intervention and ratio of BCBA supervision would likely be decided by a student’s interdisciplinary team at school and codified annually in their IEP, rather than mandated by private insurance or Medicare/Medicaid. Absent any steps to meaningfully integrate behavioral treatment, the gap between Michigan public schools’ execution of behavioral interventions and best practices in ABA is wide. This discrepancy could hamper the effect of otherwise valuable practices, and at the very least bears closer examination. Given that ASD is a pervasive condition that can impede development across domains, enhancing access to school-based services could lead to improvements in skill acquisition and maladaptive behavior, placement in less restrictive school environments, and successful educational and vocational placement later in life. 5 Barriers to ABA in Michigan Public Schools The information available points to several complicating factors which could inhibit the successful integration of ABA into Michigan public schools. These include the way in which ABA services are funded, how these services might be categorized on a student’s individualized education program (IEP), and the general conceptualization of ABA programming by the Michigan Department of Education. Michigan’s Autism Insurance Reform Legislation, which went into effect on October 15, 2012 (State of Michigan, 2022) mandates that members of for-profit, commercial, HMO, and non-profit health insurance companies receive coverage for services related to ASD diagnosis and treatment (e.g., applied behavior analysis) through the age of 18 (State of Michigan, 2022). This was part of a package of laws enacted in 2012, known as the Autism Insurance Benefit (AIB) which ensured coverage of ABA by private and public insurance in the state of Michigan for children with ASD (MAASE, 2021). Despite being a critical step to enshrine the provision of vital services for children with developmental disabilities, this type of legislation puts the onus on insurance alone to provide funding for treatment. Other states take a more balanced approach, where schools are often tasked with providing ABA services during the school day according to a given student’s individualized education plan (IEP) and their insurance company might be responsible for ABA services outside of school hours. Thus, school systems elsewhere are incentivized to do things like hire full-time board certified behavior analysts (BCBAs) or set up specialized ABA classrooms in mainstream public schools to meet the needs of students who have a certain number of daily hours of ABA instruction mandated by their IEP. In Michigan, the inclusion of ABA programming on a student’s IEP is complicated by the need to discriminate between special education programs/services and teaching 6 methodology. If ABA is included as a special education program/service, then an outside provider could conceivably provide those services in school. However, funding would most likely have to come from the student’s private insurance or Medicaid, and services would have to “supplement” but not “supplant” the instruction they were already receiving (MAASE, 2017). If ABA is categorized as a teaching methodology, it might be purposely left off the student’s IEP to give special education teachers enhanced flexibility to modify their strategies as they see fit (MAASE, 2017). This is problematic because intensive ABA intervention in school likely requires systematic protocols and ongoing consultation with credentialed professionals, which might not be outlined in an IEP where ABA is considered a teaching methodology. There is also evidence to suggest that the Michigan Department of Education takes a narrow view of ABA programming, which might not acknowledge the possibility of comprehensive integration of ABA into the classroom environment: “ABA can be effective for changing behaviors and developing skills in both children and adults. Some families may want their children to be in an ABA program during the school day. Research demonstrates placing students with disabilities in schools alongside their peers promotes more successful transitions to community living. Additionally, if a student has an individualized family service plan (IFSP) or individualized education program (IEP), being in an ABA program during the school day may violate their right to a free appropriate public education (FAPE) in the least restrictive environment (LRE) according to the Individuals with Disabilities Education Act (IDEA)” (Michigan Department of Education, 2022; p.2). This language seems to suggest that ABA programming during the school day might somehow violate IDEA by being unnecessarily restrictive. However, this type of programming does not necessarily require removing a student from school or sequestering them from their 7 peers. Rather, a more thorough integration of this programming into the school environment could help students access appropriate curriculum, develop and maintain peer relationships, minimize inappropriate behavior, and thrive in the least restrictive setting possible. Specialized ABA Classrooms Despite the challenges, many students in Michigan could potentially benefit from a more thorough integration of ABA programming in schools. One way to get students what they need in a least restrictive environment would be to adopt specialized ABA classrooms in mainstream public schools. These classrooms are more common in other parts of the country and allow students to remain engaged in appropriate curriculum during the school day, mainstream with general education peers for specials and other subject areas as appropriate, and work on problem behavior mitigation with trained staff in a supportive environment. Furthermore, this type of model allows educators to seamlessly integrate applied behavior analytic principles into their academic assessments, curriculum, classroom environment, and social interactions. In the current dissertation, the adoption of an ABA classroom is a concrete and tangible step that school administrators might take to address the needs of their students. In contrast to hiring a BCBA consultant on an ad-hoc basis or offering a professional development workshop on the principles of ABA, adopting this type of classroom would constitute a lasting organizational change. In an ABA classroom, teachers are encouraged to use assessments specifically geared toward core deficits of ASD, rather than relying on generic assessments developed for general education students or self-generated assessments that tend to be more subjective in nature. Two of the most robust and popular assessment tools are the Assessment of Basic Language and Learning Skills-Revised (ABLLS-R; Partington, 2010) and the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg, 2008). A major advantage of these 8 assessments is that they also include tailored curricula to meet the needs of ASD students and lend themselves to straightforward, quantitative data collection. These assessments/curricula break larger tasks down into smaller, manageable pieces, so that incremental progress toward larger goals is emphasized and benchmarks are less daunting. Although individualized assessment, tailored curriculum development, and rigorous data collection are foundational to any ABA classroom, there are several more advantages derived from the clinical therapeutic model. Best practices dictate regular supervision/consultation by a BCBA at the master’s or doctoral level. This allows for valuable knowledge transfer about how best to execute instruction for the full class, small group, and individual students. It also allows for the development and implementation of functional behavior assessments and behavior intervention plans for students struggling with complex and challenging problem behavior. Furthermore, teachers in ABA classrooms collaborate with behavior analysts to design elements (e.g., physical space, activities, data collection) in ways that maximize socially significant behavior. What constitutes “socially significant behavior” is collaboratively determined by the parents, teachers, BCBA, and if possible, the students themselves. For example, if a student has difficulty with expressive language (i.e., identifying wants and needs), their classroom, individual workspace, and interaction with staff would be thoughtfully designed to elicit, reinforce, and shape verbal behavior. Expressive language rehearsal might be integrated into their individualized curriculum and afford them the opportunity to improve through sustained, generalized practice. There would also be dedicated time for the student’s behavior analyst to consult with and garner input from their special education teacher, speech-language pathologist (SLP), and any other relevant stakeholder to build a curriculum around their expressive language needs. 9 Appropriateness and Feasibility of ABA Classrooms in Michigan It should be noted that specialized ABA classrooms are unlikely to be appropriate placements for all students in Michigan who meet criteria for ASD. Rather, these classrooms in public schools might be thought of as one point on a spectrum of ABA integration, which also includes ABA-based private school placements on the more restrictive end of the spectrum and ad-hoc consultation with BCBAs on the less restrictive end. Specialized ABA classrooms in public schools are likely most appropriate for students with ASD who present with comorbid intellectual impairment and/or significantly challenging problem behavior. These students likely comprise a subset of the roughly 25,000 students in Michigan who presently meet criteria for ASD. Though it is not known exactly how many students might fall into this category, Maenner et al. (2021) estimate that approximately 35% of children diagnosed with ASD present with an intellectual disability. Given the prevalence of students with ASD and an intellectual disability, it likely will not be necessary to adopt an ABA classroom in every public school in Michigan (N = 2,989). It might be more appropriate to aim for one or more ABA classrooms in every intermediate school district (N = 57) with additional ABA classrooms for more populous districts. Thus, it is critical to survey both ISD administrators and school principals because an ABA classroom that serves students across an ISD will likely be housed within a single school and thus under the purview of the principal. Given the need for credentialed personnel to provide consultative support in the lead up to and implementation of ABA classrooms, an assessment of the number of available BCBAs to assist is also critical. The available evidence suggests that there should be enough BCBAs in Michigan to meet the demand of these specialized classrooms as they emerge, setting aside 10 geographic location of providers which is difficult to assess. According to one report, there were 1,174 certified behavior analysts in Michigan in 2019 and this number has grown considerably over the last several years (MDHHS, 2019). Specifically, the number of behavior analysts in Michigan has increased by approximately 150 each year on average since 2012. At present, many BCBAs in Michigan are likely treating school-aged students in the clinic or home setting during school hours. If more students can access behavioral treatment in the school setting, then they will not have to leave school each day for clinics and home-based treatment, which could free up BCBAs during daytime hours to oversee the treatment of larger numbers of students in the school setting. It should also be noted that a key principle of this treatment modality is to fade services as students become more independent. So, if these classrooms are successful, many of these students could re-integrate with general education peers as soon as they are able. In conclusion, the cultivation of ABA classrooms would be an important step to bridge the gap between what is currently available in Michigan public schools and the ABA therapy that takes place largely outside of school settings. It could alleviate concerns about unnecessary restrictiveness and provide educators with the tools to meet the needs of students with significant challenges as the result of a developmental disability. That said, widespread adoption of these classrooms would constitute a big shift, drawing upon monetary resources for professional and curriculum development, the purchase of data collection systems, dedicated space, staff training for the safe management of complex and challenging problem behavior, regular consultation with credentialed behavior analysts, etc. As such, it is important to ascertain whether ISD administrators and principals in Michigan public schools are both willing and able to make this important shift toward evidence-based practice. 11 The current study’s focus on ISD administrators and school principals is responsive to the structural makeup of local special education service delivery, the nature of the organizational change itself, and the nascent stage of ABA integration in Michigan public schools. In the state of Michigan, special education services largely fall within the purview of intermediate school districts, even though the implementation of these services occurs within the schools themselves. The adoption of a new type of special education service would thus require the support of the ISD administrator and the school principal to effectively get the program off the ground. The nature of the change is also unique. Unlike the typical rollout of a new program to be implemented by existing staff, the adoption of ABA classrooms would likely require new personnel to fill novel roles in brand new classrooms. Therefore, it is not appropriate to survey current special education teachers, because it is not clear which of them (if any) will be involved in the implementation of this new type of classroom. Finally, since the adoption of ABA classrooms would be a totally new frontier for most ISDs and schools in Michigan, the critical stakeholders at this stage are the ISD administrators and school principals at the top of the organizational hierarchy. Since they would make the initial decision to enact this change, they are the most pertinent people to survey about the adoption of ABA classrooms. One way to assess this is through the lens of organizational readiness for change (ORC). THEORETICAL FRAMEWORK Organizational Readiness for Change: The Weiner Model Organizational change, when it occurs intentionally, refers to a shift from an organization’s present state to a more desirable future state in which organizational effectiveness is improved (Weiner et al., 2008). To be successful, organizational stakeholders need to be ready to enact this shift. “As a working definition, we consider organizational readiness for change as 12 the extent to which organizational members are psychologically and behaviorally prepared to implement organizational change” (Weiner et al., 2008, p. 381). ORC theory is largely rooted in Lewin’s (1947; 1951) three-stage model of change, which involves “unfreezing” the present level of group performance, “moving” to a higher level of group performance, and “freezing” performance at the higher level. Promoting readiness for change within an organization is tantamount to “unfreezing” it in preparation for movement toward a higher level of performance (Weiner et al., 2008). Thus, organizational members’ readiness is reflected in being willing and able to implement change. In Weiner’s widely cited, two-factor theory of ORC, the colloquial terms willing and able are operationalized as “change commitment” and “change efficacy,” respectively (2009). These are the constituent factors of the ORC construct. Drawing upon motivation theory, Weiner (2009) discusses change commitment as a function of change valence, or the value that organizational stakeholders attribute to the specific impending change. The present study follows this approach. Applied to the present dissertation, a school culture which is supportive of evidence-based practices for special education will likely lead stakeholders to perceive ABA classrooms as more beneficial. In turn, they will be more committed to adopt these classrooms. As shown in the model in Figure 1, contextual factors (e.g., system climate and family-school collaboration) contribute to change valence (i.e., the extent to which principals and ISD administrators believe that ABA classrooms are needed, beneficial, important, and worthwhile), which in turn affects how committed they are to the organizational change. 13 Figure 1 Adapted Model of Organizational Readiness for Change Note: Figure modified from Weiner (2009). Constructs of interest are presented in bold font. Corresponding measurement instruments are presented in blue and refer to the Usage Rating Profile – Intervention Revised (URP- IR; Briesch et al., 2013) and the Organizational Readiness for Implementing Change instrument (ORIC; Shea et al., 2014). Weiner’s model relies on social cognitive theory, which holds that change efficacy is the function of stakeholder’s appraisal of task demands, resource availability, and situational factors (Gist and Mitchell, 1992; Weiner, 2009). In the context of this study, task demands might encompass principals’ appraisal of the steps necessary to adopt ABA classrooms. Resource perceptions might pertain to the appraisal of monetary, physical space, time, knowledge, and skill requirements necessary to implement this change. Finally, situational factors could pertain to the appraisal of organizational idiosyncrasies inherent to the adoption of this intervention within Michigan schools. For instance, an educator in a district that is very supportive of special 14 education services might be more apprised of the steps necessary to open an ABA classroom, more confident in the available resources (e.g., money, space, time) to implement this change, and more positive about the situational factors at play. All these factors will likely contribute to an increased sense of efficacy to adopt an ABA classroom. As shown in the model in Figure 1, contextual factors are thought to contribute to the informational assessment of the impending organizational change, which in turn affects stakeholder perceptions of efficacy to implement the change. Weiner (2009) acknowledges that prior conceptualization and measurement of organizational change has been ambiguous, in part because it can be theorized, assessed, and studied at various levels of analysis. In some ways, the current model is a reaction to this ambiguity. After all, organizations are comprised of departments, units, groups, and individuals. However, Weiner (2009) is clear that this multi-level construct is not homologous, meaning that at each level of analysis, the construct’s meaning, measurement, and relationship with other variables would necessarily differ. Thus, the model focuses on a supra-individual level of analysis, where theory is most heavily concerned with determinants and outcomes of an organizational nature (Weiner, 2009). Although Weiner (2009) emphasizes the ‘shared team property’ of ORC, which suggests the need for data collection from all stakeholders involved in a given change, there is precedent for testing this model with one representative from a given organization (e.g., Helfrich et al., 2018). Weiner’s (2009) theory was chosen as a vehicle for the current study in part because the relevant determinants and outcomes are pertinent to a given public school as an organizational whole. However, the circumstances surrounding ABA classroom adoption make it impossible to identify all potential stakeholders that will be involved in the change. Rather, the emphasis is on 15 organizational leadership (i.e., school principals and ISD administrators), who will make the initial determination to adopt ABA classrooms for the benefit of students with ASD. The ORIC Instrument Aside from being widely cited and congruent with the organizational determinants and outcomes within schools, another strength of this model is the availability of direct measurement. In keeping with the original model, the Organizational Readiness for Implementing Change (ORIC) instrument purports to measure the two factors of ORC in a way that distinguishes them from their hypothesized determinants (Shea et al., 2014). The hypothesized determinant of change commitment is change valence, and the hypothesized determinant of change efficacy is the informational assessment of task knowledge, resource availability, and situational factors (Shea et al., 2014). This instrument is the product of a collaboration with Weiner and stands as an alternative to previous measures (Shea et al., 2014). These previous measures showed little consistency in the definition or measurement of organizational readiness for change, reflected disparate factor structure, and suffered from a lack of evidence for reliability and validity (Shea et al., 2014; Weiner 2008). Specifically, prior to the release of the ORIC, a systematic review of the psychometric properties of 26 other measures of ORC was conducted and similarly found little evidence for the reliability or validity of these measures in the healthcare domain (Gagnon et al., 2014). In addition, only 62% of the measures were grounded in a discernable theoretical foundation (Gagnon et al., 2014). Although the present dissertation does not take place in the healthcare domain, the ORIC measure has been used with school administrators and educators before. In an assessment of ORC for implementing a nutrition curriculum in the early care and education setting, Swindle et al. (2018) found that the ORIC demonstrated high reliability (α = .97) and convergent validity in 16 a sample similar to the one proposed in the present dissertation. Swindle et al. (2018) did not find support for Weiner’s (2009) two-factor structure of ORC, and there was some indication of careless responding on the part of the participants, potentially due to the length of the ORIC. However, this research demonstrates precedent for the use of the ORIC in educational settings and highlights the need for further exploration (Swindle et al., 2018). Present Research This dissertation seeks to fill gaps in the literature regarding ORC in educational settings, specifically pertinent to the adoption of specialized ABA classrooms in Michigan public schools to meet the needs of students with ASD. In doing so, I will answer the following questions: (RQ1) Do findings support a two-factor structure (change commitment and change efficacy) for organizational readiness for change? (RQ2) What is the relationship between possible contextual factors and change commitment? (RQ3) What is the relationship between possible contextual factors and change efficacy? Chapter 2 will focus on RQ1 by reviewing the literature relevant to ORC (with a focus on change commitment and change efficacy) and the literature relevant to ABA in schools. Chapter 2 will also describe the confirmatory factor analytic approach which will be used to assess whether the data supports a two-factor structure for ORC. I hypothesize that confirmatory factor analysis will support a two-factor structure for organizational readiness for change, indicative of change commitment and change efficacy as constituent factors of the supraordinate construct of ORC (H1). Chapter 3 will address RQ2 by reviewing the literature relevant to contextual factors and change valence. Chapter 3 will further discuss the measurement of contextual factors and change valence using the system climate, family-school, and acceptability subscales of the Usage Rating 17 Profile – Intervention Revised (URP-IR; Briesch et al., 2013). Chapter 3 will also outline the structural equation modeling (SEM) approach which will be used to assess whether change valence mediates the relationship between contextual factors and change commitment. I expect change valence to mediate the relationship between contextual factors and change commitment. More specifically, positive endorsement of the contextual factor of system climate will be associated with increases in participants’ ratings of the acceptability of ABA (i.e., change valence), which in turn will be associated with higher levels of change commitment (H2a). Similarly, positive endorsement of the contextual factor of family-school collaboration will be associated with increases in participants’ ratings of the acceptability of ABA (i.e., change valence), which in turn will be associated with higher levels of change commitment (H2b). Chapter 4 will address RQ3 by reviewing the literature relevant to informational assessment and discuss the measurement of this construct using the understanding, feasibility, and system support subscales of the Usage Rating Profile – Intervention Revised (URP-IR; Briesch et al., 2013). Chapter 4 will also outline the SEM approach which will be used to assess whether informational assessment mediates the relationship between contextual factors and change efficacy. I expect that participants’ informational assessment of ABA will mediate the relationship between contextual factors and change efficacy. First, positive endorsement of the contextual factor of system climate will be associated with a more positive informational assessment in the form of grasping task demands (i.e. understanding), which will be associated with higher levels of change efficacy (H3a). Second, positive endorsement of system climate will be associated with a more positive informational assessment of resource perceptions (i.e., feasibility), which in turn will be associated with higher levels of change efficacy (H3b). Third, positive endorsement of system climate will be 18 associated with a more positive informational assessment of situational factors (i.e., system support), which in turn will be associated with higher levels of change efficacy (H3c). Fourth, positive endorsement of the contextual factor of family-school collaboration will be associated with a more positive informational assessment of grasping task demands (i.e., understanding), which in turn will be associated with higher levels of change efficacy (H3d). Fifth, positive endorsement of family-school collaboration will be associated with a more positive informational assessment of resource perceptions (i.e., feasibility), which in turn will be associated with higher levels of change efficacy (H3e). Finally, positive endorsement of family- school collaboration will be associated with a more positive informational assessment of situational factors (i.e., system support), which in turn will be associated with higher levels of change efficacy (H3f). Significance This dissertation is expected to contribute to the literature in several ways. First, there is a dearth of research on readiness for change in the educational setting, and even less that is pertinent to the adoption of applied behavior analytic interventions for students with ASD. Second, this study could help to identify more general mechanisms by which educators adopt evidence-based practices, and thus be applicable to domains beyond ABA. Third, through a representative sample of educators in the state of Michigan, this study has the potential to assess their readiness to initiate an organizational change that would more closely align current practice in Michigan public schools with the needs of ASD students. This change would undoubtedly have a direct and positive impact on the lives of students, families, and stakeholders. Finally, the adoption of ABA classrooms would make Michigan public schools a more equitable option for students with disabilities and help keep pace with other states that are leveraging behavioral science to enhance student achievement. 19 Translating the Present Research in the Language of ABA Given the importance of credentialed behavioral practitioners in the potential adoption of ABA classrooms in Michigan public schools, it is useful to examine the present research from an ABA perspective. Given the tendency in ABA to eschew phenomena and processes that are not directly observable to outside observers, it might be a challenge to explain the ORC construct to a behavior analytic audience. However, one approach might be to frame ORC from a lens of private events. Skinner (1945) first introduced the concept of private events to account for internal stimuli and responses to that stimuli which occur, but are observable only to the individual (e.g., thinking "I am hungry" in response to hunger pangs). From this perspective, readiness for change would encompass some of the private events (e.g., thoughts, feelings) in the lead up to the potential adoption of ABA classrooms. This dovetails nicely with one definition of ORC as a “cognitive precursor” to behavior that either supports or counteracts organizational change efforts (Armenakis et al., 1993; p. 681). Educators should logically experience some private events pertaining to a given change before their publicly observable behavior in support of this change can occur. Moving beyond the definition of ORC, it is also important to contextualize the mechanisms thought to predict ORC for ABA practitioners. In behavior analytic terms, this dissertation examines the structure and nature of setting events and motivating operations which could precede the adoption of specialized ABA classrooms in Michigan public schools. Since setting events and motivating operations pertain to the availability and value of reinforcement, I will also consider the reinforcing consequences likely required to sustain this organizational change. 20 Expanding upon the three-term contingency of operant behavior (i.e., antecedent stimulus, response, consequence) first put forth by Skinner (1938), setting events and motivating operations are related but distinct concepts used to describe phenomena that set the stage for an operant contingency to occur (Nosik & Carr, 2015). Essentially, setting events and motivating operations precede the distinct antecedent stimuli in the three-term operant contingency. Distinct antecedents are not a focus of the present dissertation, but example stimuli that would be expected to have an overt effect on the adoption of an ABA classroom might be a budget increase for special education services or the hiring of an ISD administrator who happens to be a board certified behavior analyst. The response at the center of the three-term contingency in this case would be the adoption of ABA classrooms, and the consequence would be the reinforcing or punishing stimuli or events that would follow in the aftermath of adoption. Setting events (i.e., "setting factors") describe antecedent conditions that are broader in scope, more temporally distal, and/or more complex than the distinct antecedent stimuli that overtly affect behavioral responses (Bijou & Baer, 1961; Kantor, 1959; Nosik & Carr, 2015). In the present dissertation, contextual factors constitute the clearest examples of setting events. For example, the system climate of a school as it pertains to the adoption of new initiatives for students with special needs would likely have meaningful, though distal effects on the adoption of ABA classrooms. Relatedly, the level of collaboration between families and schools might have a similar type of distal impact on the adoption of ABA classrooms. For example, if parents of students with ASD serve as intermediaries between ABA professionals and teachers, parents could collaborate to bring ABA practices from the therapeutic setting into their child's school. If this were found to be beneficial, school staff might be more amenable to the adoption of a specialized ABA classroom where these practices were more fully integrated. 21 Similar to setting events, motivating operations describe antecedent phenomena that are thought to affect behavior (Keller & Schoenfeld, 1950). However, motivating operations pertain more specifically to the value-altering effect (i.e., increasing or decreasing reinforcing effectiveness) that environmental variables have on stimuli, objects, or events, and the potential behavior-altering effect as well (i.e., the increase or decrease in frequency of behavior reinforced by a given stimulus) (Michael, 2007). One example from the present dissertation would be change valence toward the adoption of ABA classrooms. If educators believe that the adoption of ABA classrooms is necessary, beneficial, and worthwhile, then the benefits that they experience after adoption (e.g., decreases in maladaptive behavior, improvements in skill acquisition, etc.) will likely have more reinforcing value. Educator's informational assessment of the task demands, resource perceptions, and situational factors necessary to adopt an ABA classroom would likely function as a motivating operation as well. For example, these factors would likely have a behavior-altering effect. If educators know what is required, have the necessary resources, and the situation permits, they will likely engage in behavior that is supportive of adopting an ABA classroom at a higher rate. The ORC factors of change commitment and change efficacy can also be contextualized through the behavior analytic lens of motivating operations. For example, change commitment and change efficacy are likely to have a behavior-altering effect on the adoption of ABA classrooms, such that enhanced commitment and efficacy will be associated with an increased likelihood for educators to engage in behaviors to bring about the change. Educators who are more committed to this change and feel empowered to bring it about might be more likely to advocate for the adoption of an ABA classroom, seek out professional development opportunities to learn more about ABA practices, consult with ABA professionals, etc. 22 The perceived benefits of specialized ABA classrooms to students, caregivers, teachers, and administrators would likely serve as the reinforcing consequences when conceptualizing this change from a behavior analytic lens. For example, if students can access an education that better meets their needs, their caregivers might be more supportive of the teachers and classroom activities as a result of this perceived effectiveness. If the ABA classroom can address challenges in behavior management and skill acquisition that staff were otherwise struggling with, then their activities in support of the new classroom model will be reinforced. Finally, if the ABA classroom can help address some of the administrative concerns inherent to the education of students with special needs, (e.g., maintaining safety in a least restrictive environment, appropriate curricula based on ability level, empowering staff with the knowledge, skills, and abilities to make an impact, etc.), then administrators' efforts in the development and maintenance of an ABA classroom would similarly be reinforced. In conclusion, the framing of this potential organizational change from a behavior analytic lens could be useful when translating this research to professionals within the field of ABA, thereby improving accessibility and utility for a key audience. Since this dissertation is meant to address a gap in services for students with developmental disabilities, speaking directly to service providers and other stakeholders is a critical consideration. 23 CHAPTER 2: ORGANIZATIONAL READINESS FOR CHANGE & ABA CLASSROOMS Organizational Readiness for Change LITERATURE REVIEW Organizational readiness for change (ORC) is present when environmental characteristics, structural elements, and personal attitudes reflect receptivity to a change on the horizon (Holt et al., 2007a). Although a consensus definition of this construct has been elusive, one highly cited paper defines it as “organizational members’ beliefs, attitudes, and intentions regarding the extent to which changes are needed and the organization’s capacity to successfully make those changes. Readiness is the cognitive precursor to the behaviors of either resistance to, or support for, a change effort” (Armenakis et al., 1993; p. 681-682). Organizational readiness for change has rarely been discussed from a behavior analytic perspective. However, McGee and Crowley-Koch (2021) describe assessment of this construct in the context of organizational behavior management (i.e., an offshoot of ABA) across four categories. These include the extent to which organizational stakeholders understand the change and their role in implementing it (i.e., clarity and alignment), their competency to perform new or different behaviors to support the change (i.e., skills and abilities), whether stakeholders have the required resources to implement the change (i.e., time, tools, and resources), and finally whether feedback and reinforcement systems are in place to sufficiently prompt and reward the change in behavior associated with the organizational shift (McGee and Crowley-Koch, 2021). The following section will discuss discrepancies between various theoretical conceptualizations of the construct, issues with measurement reliability and validity stemming from this ambiguity, and the potential challenges associated with examining this construct across various types of organizational change within different contexts. It will conclude with a deeper dive into Weiner’s 24 (2009) model of ORC and its constituent factors, which provide the theoretical basis for this dissertation. Defining ORC Evaluating the merit of various conceptual definitions of ORC is difficult because many articles in the ORC literature lack such a definition. In one systematic review, Weiner et al. (2008) found that 55% of the 106 publications reviewed gave no conceptual definition of ORC. The articles that did provide a conceptual definition largely fell into two categories. The first category defined the construct in psychological terms, focusing on the cognitions or attitudes that individuals hold. Many of these were based on Armenakis et al.’s (1993) definition cited above (Weiner et al., 2008). Several others that took the individual psychological approach relied upon Prochaska and DiClemente’s (1983) transtheoretical model, which describes five stages implicated in behavior change (i.e., precontemplation, contemplation, preparation, action, and maintenance; Weiner et al., 2008). Alternatively, the second category defined ORC in structural terms that focused on the capabilities and resources of an organization rather than individual employee cognitions or attitudes (Weiner et al., 2008). Other differences in the theoretical conceptualization of ORC pertain to the level at which the construct is applied (Bouckenooghe, 2010; Weiner, 2008). Although it is generally understood that ORC is a multilevel construct (e.g., readiness can be assessed at the individual, team, and organizational level), most ORC research has examined the construct only at the individual level and overlooked critical elements such as organization-level outcomes (Bouckenooghe, 2010; Rafferty et al., 2013; Weiner, 2009), In many cases, overlap has also been observed between the unit of analysis and the way ORC was defined. For instance, Weiner et al. (2008) found that 46% of articles pertained to the individual level of analysis, which was 25 frequently associated with ORC being defined in psychological terms. In the same review, 55% of the articles discussed ORC as an organizational construct and these included all papers which defined the construct in structural terms (Weiner et al., 2008). Weiner et al. (2008) noted differences in whether authors described ORC as a generalized state of existence within an organization or as a specific state of preparedness for a particular impending organizational change. This finding is similar to that of Bouckenooghe (2010), who noted tension between conceptualizations of ORC as planned and episodic versus emergent and continuous. Other discrepancies were noted about the particular stage in the change process to which the ORC applies. Although it is generally understood that complex organizational changes unfold gradually over time and do not often follow a linear trajectory, it is sometimes useful to map these processes onto discrete linear stages for explanatory or analytic purposes. In doing so, several authors applied ORC at different stages (i.e., initiation, implementation, or both), or applied it in a way in which the relevant stage could not be determined (Weiner et al., 2008). Finally, tension has been noted in whether authors focus on a positive versus negative framing of change (i.e., enabling, motivating, and facilitating readiness versus resistance and cynicism toward readiness), as well as the finding that different conceptualizations of ORC lend themselves to different methodological approaches (i.e., variance versus process; Bouckenooghe, 2010). These discrepancies in the literature have contributed to a lack of clarity about this ORC’s conceptual definition and led some to assert that it has fallen prey to the “jingle” fallacy, whereby various terms have been used synonymously with ORC (e.g., preparedness, willingness), as well as the “jangle” fallacy, whereby ORC has been defined and measured in different ways (Bouckenooghe, 2010; Gagnon et al., 2014; Miyake-Lye et al., 2020). 26 Issues with Reliability and Validity in ORC Measurement The discrepancies endemic to the ORC literature suggest that this construct suffers from conceptual ambiguity which could stymie efforts to develop acceptably reliable and valid measurement, advance knowledge, and inform communities of practice on how best to approach organizational change efforts (Weiner, 2009). Successive reviews have found limited evidence for the validity and reliability of ORC measures (e.g., Gagnon et al., 2014, Holt et al., 2007a; Weiner et al., 2008; Weiner et al., 2020), which could be a downstream consequence of its conceptual ambiguity. For example, out of 32 ORC instruments examined by Holt et al., (2007a), only two presented evidence of content, construct, and predictive validity. Furthermore, Gagnon et al. (2014) found that only one instrument out of 26 presented evidence of all four subtypes of construct validity assessed. Estimates of reliability for ORC measures have been similarly problematic. Gagnon et al. (2014) found that only 69% of reviewed papers provided estimates of reliability, and of those, all but one relied upon internal consistency reliability. Furthermore, five of the 26 instruments (19%) included no information about reliability and validity (Gagnon et al., 2014). In a recent systematic review, Miyake-Lye, et al. (2020) concluded that “readiness assessments must bridge the gap between measuring a theoretical construct and factors of importance to a particular implementation” (p. 1). However, limited evidence for reliability and validity suggests that a shaky conceptual foundation could stymie the application of ORC to implementation efforts across domains. Different Types of Change in Various Domains Nevertheless, measures of ORC have been used to examine organizational changes in healthcare and social services (e.g., Gagnon et al., 2014; Miyake-Lye et al., 2020), as well as 27 education, organizational science, government, business, and elsewhere (e.g., Holt et al., 2007a; Weiner et al., 2008). These different contexts reflect a broad range of initiatives which include the adoption of new technologies, programs, and organizational structure. For example, ORC measures have been used to assess readiness to implement electronic health record keeping in long term care facilities (Cherry, 2011), readiness to adopt a program for screening, brief intervention, and referral to treatment for substance misuse in community health programs and emergency centers (Bohman et al., 2008), readiness to implement a new organizational structure at a governmental body that develops information systems for the U.S. Department of Defense (Holt et al., 2007b), readiness to implement an enterprise resource planning (ERP) system at an aircraft manufacturing firm (Abdinnour-Helm et al., 2003), and many more contexts. Thus, the operationalization and measurement of ORC is often tailored to a particular context, which impedes the emergence of a “gold standard” of assessment (Miyake-Lye, et al., 2020). This adaptation might include the modification of an existing assessment for a particular setting or the development of an entirely new instrument. One recent systematic review in the health and social services sectors examined 29 uses of such assessments and found that 62% (18/29) of these were separate instruments tailored for a specific context (Miyake-Lye, et al., 2020). Therefore, when it comes to bridging the gap between ORC theory and its application to specific changes in various domains, there are challenges on both sides. On the one hand, the construct is often ambiguously conceptualized and not thoroughly defined. This has likely led to a lack of evidence for the reliability and validity of many available ORC instruments. On the other hand, measures of ORC have been developed or modified for many specific applications. 28 Thus, the ORC literature reflects an uncertain conceptual core and broad application of the construct in many places for many purposes. The result is an ongoing lack of clarity and will most likely require a high degree of discernment for researchers to successfully choose measures that are both conceptually sound and applicable to the context of interest. The Weiner Model To overcome many of the aforementioned conceptual and methodological challenges, the present study will use Weiner’s (2009) theory of ORC. There are several advantages in doing so. First, the Weiner model benefits from an established theory of ORC which seeks to clarify much of the ambiguity in the literature. Within this model, ORC is clearly defined as “organizational members’ shared resolve to implement a change (change commitment) and shared belief in their collective capacity to do so (change efficacy)” (Weiner, 2009, p. 1). Second, Weiner’s theoretical model translates directly into operationalization and measurement using the ORIC instrument, which has demonstrated adequate reliability and validity (Shea et al., 2014). Third, there is precedent for using Weiner’s (2009) model and the ORIC instrument in educational settings (Swindle et al., 2018). Fourth, the Weiner model conceptualizes ORC in psychological rather than structural terms, positing that stakeholders consider the available resources and deficits of their organization to judge change efficacy (Weiner, 2009). Rather than neglecting these structural elements, the model seeks to integrate and more successfully ground the structural determinants of ORC in theory than prior conceptualizations have. For example, the model posits that contextual elements such as organizational structure and resources shape perceptions of readiness. Finally, Weiner’s (2009) model describes ORC as situational rather than a more generalized state of affairs within an organization. Thus, ORC is heavily dependent on both the 29 content and the context of the change (Weiner, 2009). This type of approach is critical for a highly specific type of change in a unique organizational setting (e.g., the adoption of ABA classrooms in Michigan public schools). It also dovetails with evidence from a recent systematic review which highlights the importance of specific situational factors in the conceptualization and measurement of ORC (Miyake-Lye, 2020). It should be noted that the original Weiner (2009) model encompasses outcomes that extend beyond ORC (i.e., change-related effort and implementation effectiveness). Those distal outcomes are beyond the scope of the present dissertation and will not be examined. Factor Structure of ORC Weiner’s concept of ORC is concerned with whether organizational stakeholders are willing and able to make change. Willingness is operationalized as change commitment, which “refers to organizational members’ shared resolve to pursue the courses of action involved in change implementation” (Weiner, 2009, p. 2). Drawing from Bandura’s (1997) concept of goal commitment, change commitment is predicated on the idea that change is more likely to occur when individuals are invested in the goal and determined to act. An example item from the ORIC measure for change commitment would be “people who work here want to make this change” (Shea et al., 2014). Ability is operationalized as change efficacy, which “refers to organizational members' shared beliefs in their collective capabilities to organize and execute the courses of action involved in change implementation” (Weiner, 2009, p. 2). Change efficacy draws from Bandura’s (1997) concept of collective efficacy as “…not simply the sum of the efficacy beliefs of individuals. Rather, it is an emergent group-level attribute that is the product of coordinative and interactive dynamics” (p. 7). An example item from the ORIC for change efficacy would be 30 “people who work here feel confident that they can handle the challenges that might arise in implementing this change” (Shea et al., 2014). Change commitment and change efficacy thus comprise the two factors of ORC in Weiner’s (2009) model. Hypothesis 1 of this dissertation holds that the findings will support a two-factor structure for ORC. There has been limited exploration of the factor structure of ORC in school settings. The available evidence is not supportive of two-factor structure in school- based organizational change (e.g., Swindle et al., 2018). However, Swindle et al. (2018) piloted a nutritional intervention in schools. A contribution of the present work is that this model will be tested in an important new context with a novel instructional paradigm for students with ASD. Since ABA classrooms are equipped to deliver many evidence-based interventions, this work can potentially shed light on how Michigan schools might better meet the diverse needs of thousands of students with ASD. ABA Classrooms ABA is an overarching framework that has given rise to many useful interventions in various settings. But despite this broad efficacy, ABA can be difficult to contextualize in schools. Michigan educators might be aware of ABA as a clinic or home-based therapeutic approach, since many children with ASD receive insurance-funded ABA outside of school. However, much of what teachers do daily (consciously or otherwise) is also rooted in behavior analytic research. These strategies have been enumerated by Twyman (2014) and include shaping, group contingencies, incidental teaching, differential reinforcement, and many more. Other interventions and frameworks that are rooted in behavioral research such as functional behavior assessment (FBA) and positive behavior support (PBS) are widely utilized in Michigan schools at various levels (i.e., individual student, classroom, school, and district; 31 Michigan Department of Education, 2021; Sugai et al., 2000). However, these interventions are rarely attributed to behavior analytic research explicitly, so it is unlikely that educators would realize the common thread that binds their everyday practices (e.g., classroom reinforcement), the strategies used for a student struggling with problem behavior (e.g., functional behavior assessment), the overarching framework to increase socially significant behavior across their entire school (e.g., positive behavior support), and the more systematic application of ABA that their students with ASD might receive outside of school hours. Since these interventions and frameworks are connected by foundational principles rooted in behavioral research, but this connection is likely unknown by many Michigan educators, the adoption of ABA classrooms is a highly unique context in which to test the Weiner (2009) model of ORC. Although ABA classrooms are expected to capably deliver a variety of behavioral interventions depending on students’ needs, educators might be reluctant to adopt them if they are unaware of the empirical basis that these classrooms share with other, more familiar interventions and frameworks. This section will outline how an ABA classroom might be defined and explore the aspects of this unique context that could affect principals’ and ISD administrators’ perceptions of change commitment and efficacy. Defining an ABA Classroom Despite the lack of a consensus over what constitutes an “ABA classroom” and differentiates it from other classroom settings (e.g., special education classroom, general education classroom, etc.) there have been some attempts to outline key aspects of behavior analytic practice in educational settings. These are as follows: “(1) the methods of science are used to guide practice; (2) behavior change procedures are applied systematically and are technologically replicable; (3) only procedures conceptually derived from the basic principles of 32 behavior are claimed or used; (4) socially significant behavior is the focus; (5) meaningful improvement in behaviors relevant to the individual is made; and (6) the factors responsible for improvement are analyzed” (Cooper et al., 2007, as cited in Twyman, 2014, p. 534). Other characteristics of effective ABA school programs might include empirically-based and highly structured interventions, functional behavior assessments for challenging behavior, academic targets that are operationally defined, set criteria for meeting instructional objectives, systematic prompts, identification of specific skills to teach, deliberate social skill instruction, peer tutoring, individualization of reinforcement schedules and lesson plans, and purposeful attempts to generalize learned skills (Anderson, 2000; Foxx, 2008). Commitment to Adopt ABA Classrooms Weiner (2009) posits that organizational stakeholders will be committed to making a change if they think the change is necessary, beneficial, or worth the effort. In this way, change commitment is conceptualized largely as a function of change valence (Weiner, 2009). In the context of ABA classrooms, school principals and ISD administrators who serve many students with ASD might be more likely to view this type of classroom as a necessity. For example, it might make sense to have a classroom that can execute various types of interventions to meet the diverse needs of students with ASD in a comprehensive and systematic way, rather than try to facilitate these interventions in a piecemeal or eclectic fashion. Similarly, school principals and ISD administrators who struggle to meet the needs of students with severe behavioral challenges due to ASD might view ABA classrooms as more beneficial, since behavior management is integrated into instruction and coping skills can be explicitly taught and reinforced. If participants perceive that certain students are struggling in the available settings, then perhaps they will be more amenable to a paradigm shift. For example, in 33 the traditional classroom setting, educators often must take time out from whole group instruction to redirect problematic behavior. However, ABA classrooms typically have the staffing and resources to address problematic behavior continuously and systematically. This type of approach helps students get on track more quickly than the intermittent and unsystematic treatment that often occurs in a traditional public school classroom. If educators see the benefit of systematic behavioral treatment, they might be more committed to the adoption of ABA classrooms. Finally, the cost and effort involved in adopting an ABA classroom needs to be perceived as worthwhile for educators to be committed to making this change. Educators will most likely have to be convinced that an ABA classroom can do a better job of meeting the needs of their students with ASD than what is currently available in their school or district. This might be easier if they have experienced the benefits of ABA-based instruction firsthand. However, ABA classrooms require a great deal of upfront investment and sustained effort, given the rigor of instruction, continuous data collection, and high level of structure. If participants do not perceive that these classrooms are worth the time, effort, and monetary cost to better educate students with ASD, then their commitment to make this change will likely be low. Perceived Efficacy and ABA Classrooms Change efficacy is thought to be a function of organizational stakeholders’ appraisal of task demands, resource availability, and various situational factors (Gist and Mitchell, 1992; Weiner, 2009). It is “a comprehensive summary or judgement of perceived capability to perform a task” (Gist and Mitchell, 1992, p. 184, as cited in Weiner, 2009). Educators who understand the task demands and appropriate sequence of activities associated with adopting an ABA classroom will likely feel better positioned to make this change (Weiner, 2009). For example, a school 34 principal or ISD administrator who feels equipped to navigate the activities involved will likely endorse higher change efficacy. These activities might include keeping track of progress, coordinating tasks, or supporting stakeholders during the change (Shea et al., 2014). In the context of ABA classrooms, keeping track of progress might involve reviewing students’ mastery of benchmarks or monitoring teachers as they deliver ABA-based instruction. Various tasks involved in this change would include the procurement of data collection systems, conducting ABA-based assessments, and seeking the expertise of ABA consultants. Finally, stakeholders will likely need support in the form of regular check-ins and management of individual issues exhibited by students and staff members as they acclimate to the change. School principals and ISD administrators will also need to perceive adequate resource availability to successfully implement the adoption of ABA classrooms. In this context, significant financial investment will be necessary but not sufficient to enact change. ABA classrooms will additionally require dedicated physical space, significant time commitments for training and professional development, ongoing consultation with credentialed ABA professionals, and skillful case management to explain the paradigm shift to parents and cultivate their support. School principals and ISD administrators who believe that they can marshal the necessary resources to successfully implement this change are expected to endorse higher change efficacy. Finally, school principals and ISD administrators will likely endorse higher change efficacy if they have a favorable assessment of situational factors that could affect the adoption of ABA classrooms. Weiner (2009) mentions time constraints and the internal political environment as potentially salient situational factors. In the present context, the timing of the change will need to be thoughtfully implemented in accordance with the realities of the school 35 calendar. It is unlikely that all the activities involved in adopting an ABA classroom will be possible to complete during the summer months leading up to the start of a new school year. There are also staffing concerns and constraints around teaching union contracts that would likely complicate summer preparation. Thus, the planning and implementation for this change would probably have to begin early in the preceding school year to maximize the chances of success. The political climate is also likely to be a factor. For example, if a given school or district is hostile to ABA integration or reticent to abandon the status quo in favor of a new educational paradigm, then perceived change efficacy is likely to suffer. There could be vested interest on the part of certain constituencies to keep ABA out of Michigan public schools. If this is the case, it is possible that school and district administrators will have to convince parents, staff, and school board members that this change is necessary, beneficial, and worthwhile. If the political climate is not supportive of evidence-based instruction delivered in a rigorously systematic way, then public opinion will be yet another barrier to overcome on the path to organizational readiness to adopt ABA classrooms. METHOD The unique context of adopting and implementing ABA classrooms in schools presents an interesting case for testing Weiner’s (2009) two-factor model of ORC. This section describes the methodology used to address Research Question (1) Do findings support a two-factor structure (change commitment and change efficacy) for organizational readiness for change? I hypothesize that confirmatory factor analysis will support a two-factor structure for organizational readiness for change, indicative of change commitment and change efficacy as constituent factors of the supraordinate construct of ORC (Hypothesis 1). 36 Setting Data collection for the present dissertation comprised part of the 2021-2022 school year, the entire 2022-2023 school year, and the beginning of the 2023-2024 school year (i.e., January 2021 – October 2023). As of the 2023-2024 school year, the Michigan public school system is comprised of 2,989 total public schools that fall under the purview of 541 local education authorities (LEAs) and 57 Intermediate School Districts (ISDs), in addition to 375 public school academies and 4 state facilities (Michigan Department of Education, n.d.-b). In the 2021-2022 school year, there were 10,957.71 full-time equivalent (FTE) administrators in Michigan public schools serving 1,443,456 students (Michigan Department of Education, n.d. -c, n.d. -d). Of these administrators, 54.6% identified as female and 45.4% identified as male. Most administrators (73.2%) were in the 40 to 59-year-old age range. Most administrators (83.3%) identified as White, with the next highest group identifying as Black or African American (14.2%) (Michigan Department of Education, n.d. -c). The data for the 2022- 2023 school year suggests that the overall enrollment of Michigan public schools decreased to 1,437,279 (Michigan Department of Education, n.d. -d). However, the number of FTE administrators increased to 11,475.98 by the 2022-2023 school year (Michigan Department of Education, n.d. -c). The demographic breakdown of these administrators did not change significantly. In the 2022-2023 school year, 55.2% of FTE administrators identified as female and 44.8% identified as male (Michigan Department of Education, n.d. -c). Most administrators (73.9%) were in the 40 to 59-year-old age range. Similarly, in the 2022-2023 school year, 82.7% of Michigan FTE administrators identified as White and 14.8% identified as Black or African American (Michigan Department of Education, n.d. -c). Demographic data for Michigan school 37 administrators in the 2023-2024 school year was not available in time to include in the present dissertation. Sample Participants in this study comprised a representative sample of K-12 principals (n = 335) from Michigan public schools and a representative sample of intermediate school district (ISD) administrators (n = 65). The choice to include both groups is reflective of the fact that special education services might be overseen by school principals and/or administrators (superintendents, special education directors, etc.) at the ISD level in Michigan. Eligible participants were employed by public schools or intermediate school districts (ISDs) in Michigan at the time of recruitment. A sampling frame was created from the Center for Educational Performance and Information’s Educational Entity Master. Then, principals and ISD administrators were randomly selected for recruitment. Recruitment for the present study was complicated by the COVID-19 pandemic, which likely made school principals and administrators harder to reach due to additional challenges on the job (e.g., overseeing the transition to and from online instruction, managing public health measures in the school buildings, etc.) and perhaps also contributed to increased turnover in these roles. When recruitment began in January 2021, efforts were made to restrict the size of the sampling frame to elicit the highest possible response rate. I began with a randomly selected sampling frame of 300 principals and 100 ISD administrators. Over time, low response rates necessitated expanding the sampling frame to reach the desired sample size of 400 participants. It also became clear that ISD administrators were especially difficult to recruit, and that participation from 100 of them would likely be unattainable within a reasonable amount of time. This complication necessitated the recruitment of extra LEA principals to make up the difference 38 and achieve the desired total sample size. After the sampling frame was expanded several times, the final sampling frame for principals totaled 750 and the sampling frame for ISD administrators included 158 (i.e., all persons listed in these roles in the Center for Educational Performance and Information’s Educational Entity Master). This yielded a response rate of 45% for LEA principals, 41% for ISD administrators, and a total sample response rate of 44%. Responses were counted as complete if participants successfully navigated to the end of the survey. Recruitment concluded in October 2023. Missing data was handled with a complete-case analysis approach. Respondents were omitted listwise from the sample for any missingness on key study variables (i.e., ORIC measure, URP-IR scales, etc.). This missingness pertained to 13.5% of the sample. One participant who identified as gender "non-binary" was also omitted. Since gender is included as a binary control variable in later analyses it was not practical to retain them. This left an analytic sample of 346 complete cases, comprised of 293 LEA principals and 53 ISD administrators. Missing participants did not differ significantly by gender (i.e., 24 males and 27 females omitted) or age (M = 48) than their counterparts whose responses were retained for analysis. The vast majority of omitted respondents achieved their highest degree at the master's level (n = 45), which mirrored the disproportionate number of master's degrees in the analytic sample. Regarding race, the participants with missing data comprised about 11% of all White respondents and 26% of all non-White respondents. More specifically, 24% of Black respondents and 39% of participants who identified as "other" were omitted from the analytic sample. The seemingly high proportion of missingness in non-White respondents should be placed within the larger context of a representative sample that is predominantly White. Any non-White participant 39 lost to missingness would have a disproportionate impact on the demographic breakdown of the analytic sample compared with a White counterpart who was omitted. In the final analytic sample, 51% identified as male and 49% identified as female. Eighty- nine percent identified as White, 7% identified as Black or African American, 0.6% identified as American Indian or Alaska Native, 0.3% identified as Asian, and 3% identified as "other" or a member of multiple racial groups. The vast majority (79%) attained a master's degree as their highest level of education. Eighty percent of all respondents had some awareness of applied behavior analysis before taking the survey. Additional descriptive information (i.e., demographic characteristics broken down by position as a school-level principal or district-level administrator) can be found in Table 1. Table 1 Participant Demographic Characteristics Variable LEA Principals ISD Admin Full Sample Gender Male Female Race White Black or African American American Indian or Alaska Native Asian Other/Multiple Groups Education Level Two-year Degree Four-year Degree Master's Degree Education Specialist Doctorate ABA Awareness Yes No n 27 26 51 2 0 0 0 0 0 40 4 9 50 3 % n % 50.9 49.1 96.2 3.8 0 0 0 0 0 75.5 7.5 17 94.3 5.7 178 168 308 25 2 1 10 1 3 272 30 40 278 68 51.4 48.6 89 7.2 0.6 0.3 2.9 0.3 0.9 78.6 8.7 11.6 80.3 19.7 n % 151 142 257 23 2 1 10 1 3 232 26 31 228 65 51.5 48.5 87.7 7.8 0.7 0.3 3.4 0.3 1 79.2 8.9 10.6 77.8 22.2 40 Data Collection Approval for this project was secured from the Michigan State University Institutional Review Board on 10/28/20 (MSU Study ID: STUDY00005237, see Appendix A). Data collection began on 1/14/21 and ended on 10/22/2023. Participants were recruited by phone and email from a sampling frame derived from the Center for Educational Performance and Information’s Educational Entity Master, which is a public repository of contact information for educational entities in Michigan. This study employed an online survey format using Qualtrics (see Appendices C and D), with an estimated response time of 22 minutes. Principals and ISD administrators clicked the embedded link in the recruitment email (see Appendix B), provided informed consent in the Qualtrics system, and completed the survey which included questions about their awareness and experience with ABA, the Evidence-Based Practice Attitude Scale (EBPAS) (Aarons, 2004), the Organizational Readiness for Implementing Change (ORIC) instrument (Shea et al., 2014), the Usage Rating Profile – Intervention Revised (URP-IR) (Briesch et al., 2013), and a demographic questionnaire. Participants who were unresponsive to email received follow-up phone calls at approximately biweekly intervals to boost response rates. Evidence suggests that follow-up phone calls are an effective way to increase response rates among school principals, who are typically difficult to reach (Neal et al., 2020). These phone calls also served to elicit updated contact information if a recruitment email bounced. If a principal or ISD administrator left their position before completing the survey, they were replaced in the sampling frame by their successor. Upon completion of the survey, each participant received a $10 Amazon.com e-gift card at an email address of their choice. Participant information was de-identified though the 41 assignment of an ID number. Only the research team has access to identifiable information, and it will not be shared with anyone else, at any time, for any purpose. Once all data was collected and participants were compensated, analysis took place using the de-identified data set. Measures ORIC Instrument. To measure organizational readiness for change, participants completed the Organizational Readiness for Implementing Change (ORIC) instrument. The ORIC was created in collaboration with Weiner and measures change commitment and change efficacy as constituent factors of organizational readiness for change (Shea et al., 2014). The ORIC change commitment subscale includes 4 items such as “People who work here are committed to implementing this change” (Shea et al., 2014). The ORIC change efficacy subscale includes 5 items such as “People who work here feel confident that they can handle the challenges that might arise in implementing this change” (Shea et al., 2014). ORIC items were modified slightly for the present study (e.g., “Teachers, administrators and staff who work in my school/ISD/ESA would be committed to implementing this change” and “Teachers, administrators and staff who work in my school/ISD/ESA would feel confident that they can handle the challenges that might arise in implementing this change”). Responses are assessed on a 5-point Likert-type scale from (1) ‘disagree’ to (5) ‘agree.’ The change commitment subscale (α = .93) and change efficacy subscale (α = .92) exhibited adequate internal consistency reliability in the present study. The overall instrument was found to have adequate internal consistency reliability as well (α = .94). Shapiro-Wilk tests were conducted to examine univariate normality for each individual ORIC item. Test statistic values ranged from .81 - .89 with p < .05 for each. These findings indicate that the distribution of each ORIC item differs significantly from a normal distribution. 42 A closer look indicates negative skewness values across all nine ORIC items (i.e., a propensity for respondents to endorse higher levels of change commitment and change efficacy). This is not unusual, given the potential social desirability for educators to positively endorse ORC for the benefit of students with special needs. Although some skewness was found, it is useful to contextualize skewness and kurtosis values from the ORIC items in the present study. Curran et al. (1996) categorized univariate skewness in CFA as 0 for normal, |2| for moderately nonnormal, and |3| for severely nonnormal distributions. In addition, Curran et al. (1996) categorized univariate kurtosis in CFA as 0 for normal, |7| for moderately nonnormal, and |21| for severely nonnormal distributions respectively. Across ORIC items in the present study, skewness values range from -0.42 to -0.92. Kurtosis values range from -0.29 to 0.83. Since these values do not approximate |2| for skew and |7| for kurtosis, they would be considered closer to normal than moderately nonnormal by Curran et al.'s (1996) standards and are thus not expected be problematic in the present analyses. Demographics. Participants completed demographic measures which assessed length of time in current placement and position, years of educational experience, age, gender, race, educational level, and year of last degree. Data Analysis Data was analyzed using the lavaan package for R (R Core Team, 2017; Rosseel, 2012). Confirmatory factor analysis (CFA) was used to test Hypothesis 1. CFA is a statistical approach that falls under the general category of structural equation modeling and explores the relationship between empirical indicators (i.e., individual ORIC items) and latent factors (i.e., change commitment and change efficacy; Brown & Moore, 2012). In the present dissertation, I analyzed the covariance matrix of relevant variables. The full de-identified dataset is available for download at https://osf.io/ub254/. 43 CFA is the appropriate approach to use regarding Hypothesis 1 because it uses variation and covariation among empirical indicators to assess the extent to which these indicators “load” onto a smaller number of factors (i.e., whether the 9 ORIC items load onto the two latent factors of ORC; Brown & Moore, 2012). CFA is also the statistical approach used in the initial assessment of the ORIC measure by Shea et al., (2014) after an exploratory factor analysis suggested that the two-factor model was a reasonably good fit. Therefore, the use of CFA in this dissertation also has replicative value. While some research has supported a one-factor structure for ORC as measured by the ORIC measure, (e.g., Lindig et al., 2020), a majority of studies seem to support the two-factor structure (i.e., change commitment and change efficacy) of ORC (e.g., Ruest et al., 2019; Shea et al., 2014; Storkholm et al., 2018). Therefore, a correlated two-factor solution is put forward. The first latent factor (i.e., change commitment) has four associated indicators. The second latent factor (i.e., change efficacy) has five associated indicators, resulting in a total of nine factor loadings to be estimated (see Figure 2). The covariance between latent factors (i.e., change commitment and change efficacy) is also estimated, as well as the residual variances of the nine observed variables and each of the two latent variables. This results in 21 free parameters to be estimated. However, in order to interpret all factor loadings and compare their direction and magnitude with Shea et al. (2014), the residual variances of the latent factors were standardized, leaving a total of 19 free parameters. Maximum likelihood estimation was utilized in lavaan. With the model and all relevant parameters outlined, I turn next to sample size and statistical power. The total sample of 346 participants is expected to have sufficient statistical power for confirmatory factor analysis. An a priori power analysis was conducted using the semPower 44 package in R (Moshagen & Erdfelder, 2016). Results of this power analysis suggest that a minimum sample size of 141 is adequate to detect effects similar to those in Shea et al. (2014) (α = .05, β = .8, df = 26, RMSEA = .08). Although there is some disagreement about how best to conduct a preliminary power analysis for CFA, a sample of 346 approximates the median sample size for confirmatory factor analysis (n = 389) discussed by Jackson et al. (2009) and exceeds the largest sample (n = 311) in Shea et al.’s (2014) psychometric assessment of the ORIC instrument used in the present study. Descriptive statistics such as means and standard deviations were calculated for the ORIC measure, as well as Cronbach’s alphas, correlations, variances, and covariances. As in Shea et al. (2014), various goodness-of-fit indices were calculated, such as the comparative fit index (CFI), the Tucker-Lewis Fit Index (TLI), the standard root mean square residual (SRMR), and the root mean square residual of approximation (RMSEA). These estimates were then interpreted and compared with those discussed in Shea et al. (2014). RESULTS Descriptives Pearson correlations for continuous study variables can be found in Table 2. Correlations generally align with expectations, given the variables presented and the context. For example, participant age correlated strongly with the number of years that they have spent as an educator (r = .78, p < .001). Age also correlated positively with the number of years since a degree was attained (r = .57, p < .001), the number of years in a given school or district (r = .35, p < .001), and the number of years in a participant's present role (r = .34, p < .001). Age had a small, but statistically significant positive relationship with the overall ORIC measure (r = .13, p = .01), as did years since degree (r = .11, p = .04), years in a particular role (r = .13, p = .01), and years as 45 an educator (r = .15, p = .01). Finally, the overall ORIC correlated very strongly with its change commitment subscale (r = .92, p < .001) and its change efficacy subscale (r = .95, p < .001). The ORIC change commitment and change efficacy subscales also correlated strongly with one another, (r = .75, p < .001). Confirmatory Factor Analysis Confirmatory factor analysis was used to examine the structure of change commitment and change efficacy within the superordinate construct of organizational readiness for change. A visual of the measurement model, including factor loadings and the estimated covariance between change commitment and change efficacy can be found in Figure 2. Due to the use of only complete cases, no additional respondents were eliminated for missingness (N = 346). Given the high correlation between the ORIC subscales, a correlated two-factor solution is put forward. To estimate the factor loadings of each item, a variance standardization approach was used to set residual variances for each factor to 1.0 and then freely estimate the covariance between factors. Factor loadings were comparable to those in Shea et al.'s (2014) examination of the psychometric properties of the ORIC measure. For example, loadings for change commitment in Shea et al.'s (2014) study ranged from .75 to .93. In the present study, loadings for change commitment were slightly lower, ranging from .68 to .84. Shea et al.'s (2014) loadings for change efficacy ranged from .65 to .83. In the present study, factor loadings for change efficacy ranged from .76 to .81. Several different indices offer evidence of acceptable model fit, (χ2 = 118.86, df = 26, p< .001, RMSEA = 0.10, SRMR = 0.05, CFI = 0.97, TLI = 0.95). Estimates for the SRMR, CFI, and TLI all fall within conventional cutoffs laid out by Hu & Bentler (1999). These estimates are also 46 nearly identical to those reported by Shea et al. (2014). The RMSEA of 0.10 in the present study was a bit higher than the conventional cutoff of < .06 put forth by Hu & Bentler (1999). MacCallum, Browne, & Sugawara (1996) describe RMSEA estimates between 0.08 and 0.10 as indicative of "mediocre fit," and the estimates in the present study (RMSEA = 0.10) and the RMSEA of .08 reported by Shea et al. (2014) would both fit into this "mediocre" category. However, a slightly elevated RMSEA might not necessarily be a problem. Kenny et al. (2015) contend that MacCallum, Browne, and Sugawara's (1996) criteria pertained to population RMSEA values and "not as cutoffs to empirically establish good- and bad-fitting models" (p. 488). Kenny et al. (2015) also discouraged the interpretation of point estimates of RMSEA and comparison with an arbitrary cutoff point. As an alternative, McCallum, Browne, and Sugawara (1996) and Kenny et al. (2015) agree on the importance of confidence intervals for RMSEA to account for the inherent imprecision of the estimate. The confidence interval for the RMSEA estimate in the present study, 90% CI [.08, .12], suggests the potential for overlap with Shea et al.'s (2014) findings. Also, a slightly elevated RMSEA estimate might not be problematic within the larger context of other fit indices, both incremental and absolute, that seem to align with conventional cutoffs. Given the sensitivity of any individual fit index to bias due to various factors (e.g., sample size, estimation method, model complexity, etc.), it is critical to examine several different fit indices and interpret findings holistically. Following guidance from Jackson et al. (2009), I have presented several different fit indices regardless of whether individual findings are less supportive of my original hypothesis. Taken together, evidence from the present study provides support for Hypothesis 1, which holds that items from the ORIC measure generally map onto a 47 two-factor structure for organizational readiness for change, indicative of change commitment and change efficacy as constituent factors of the supraordinate construct of ORC. Table 2 Chapter 2 Means, Standard Deviations, and Correlations Between Continuous Variables Variable M SD 1 2 3 4 5 6 7 1. Age 47.55 7.05 2. Years Since Degree 3. Years at Placement 12.19 7.70 .57** 12.81 9.31 .35** .39** 4. Years in Role 6.03 4.98 .34** .41** .55** 5. Years as Educator 6. ORIC - Overall 7. ORIC – Commitment 8. ORIC – Efficacy 21.86 6.51 .78** .63** .40** .39** 3.81 0.76 .13* .11* .09 .13* .15** 3.91 0.81 .11* .08 .08 .09 .13* .92** 3.73 0.81 .14** .13* .08 .15** .15** .95** .75** Note. * indicates p < .05. ** indicates p < .01. 48 Figure 2 Factor Structure of Organizational Readiness for Change Note. Figure 2 includes empirical indicators of change commitment and change efficacy from the ORIC measure, modified for the present study (Shea et al., 2014). Residual variances were set to 1.0 for change commitment and change efficacy. DISCUSSION The present study examined the factor structure of ORC as it pertained to educators' adoption of applied behavior analytic classrooms in Michigan public schools. ORC is conceptually vague and suffers from ambiguity which has likely complicated straightforward operationalization and measurement (Bouckenooghe, 2010; Gagnon et. al, 2014; Miyake-Lye, 2020; Weiner et al., 2008). However, Weiner's (2009) two-factor theory of ORC provides some conceptual clarity and forms the basis for the ORIC measure, which has demonstrated adequate reliability and validity when applied to organizational change in healthcare settings (Shea et al., 2014). The adoption of ABA classrooms in schools provides an interesting test case for Weiner's (2009) model as well as the ORIC measure. Educators' commitment to adopting these 49 classrooms will require them to perceive the change as necessary, beneficial, or worthwhile to meet the needs of special education students in their school or district. Educators' perceived efficacy to enact this change requires an understanding of the task demands in their proper sequence, as well as adequate resources (e.g., time, money, expertise, physical space), and a favorable assessment of the situational factors that could affect the adoption of these specialized classrooms. There is a strong empirical basis for the efficacy of applied behavior analysis to mitigate maladaptive behavior and bolster skill acquisition for students with autism spectrum disorder, but a lack of integration of ABA into Michigan public schools. Therefore, it is critical to examine Michigan educators' perceived willingness (i.e., commitment) and ability (i.e., efficacy) to integrate behavioral science into special education through the adoption of specialized ABA classrooms. Evidence for a two-factor structure of ORC as operationalized by the ORIC measure provides support for the utility of this measure in the educational sphere. It also provides the basis for further examination of Weiner's (2009) model in subsequent analyses, which differentiate change commitment and change efficacy as discrete outcome variables. The present study began with a preliminary correlational analysis of the relevant variables, primarily to inform the subsequent CFA and assess for correlated factors. The small, but statistically significant positive correlations between the overall ORIC measure and age, years since degree, years in current role, and years as an educator are notable. These findings suggest that older, more experienced educators who have been in their position longer tend to endorse more readiness for change. This cuts against the idea that people who have been in their positions for a long time are complacent or intransigent when it comes to new initiatives. On average, participants had spent 12.81 years in a given school or district, and therefore this might 50 be taken as an encouraging sign. Longer tenure at a particular school or district is associated with increased openness to change for the benefit of the students. The CFA results of the present study supported the presence of two unique factors (i.e., change commitment and change efficacy) that constitute organizational readiness for change. This lends support to Weiner's (2009) model, which differentiates change commitment from change efficacy and outlines the unique contribution of each to the overarching construct of ORC. The successful replication of the two-factor structure previously observed in healthcare and therapeutic settings (e.g., Shea et al., 2014; Storkholm et al., 2018; Ruest et al., 2019) suggests that this two-factor structure generalizes to the educational domain. Implications for Practice The applicability of ORC to educational settings and the extension of the ORIC measure to educational research are important for a few reasons. As stated previously, the context dependence of ORC can complicate operationalization and measurement. The fact that the ORC construct and its related instrument map onto a highly specific organizational change in the unique context of special education is encouraging. These findings suggest that we are a bit closer to understanding how organizational change occurs in schools. If educators being willing and able to implement change are the key, then policymakers and stakeholders have a roadmap for what it takes to undertake new initiatives successfully. For instance, if leadership decides on a district-wide shift to a new curriculum, they would do well to assess whether educators are willing and able to undertake this change before allocating resources to make the shift. If it is determined that the educators do not believe the change is worthwhile or that necessary resources do not exist to facilitate this change, then leadership will likely need to satisfy these preconditions before adoption of the curriculum would be expected to succeed. 51 Limitations and Future Directions Limitations and future directions that pertain to all studies in this dissertation will be presented in Chapter 5. However, one potential limitation specific to Chapter 2 is the narrow focus on specialized ABA classrooms as the organizational change to assess educators' readiness. I chose the adoption of ABA classrooms as an organizational change for several reasons. First, it would allow districts to meet the needs of the most profoundly disabled students with ASD in a way that would potentially keep resource expenditures low (e.g., by minimizing the need for specialized placements out of district). Second, ABA classrooms in general education public schools would also potentially allow students with ASD to maximize skill acquisition, minimize challenging behavior, and do so in the least restrictive environment which provides contact with general education peers. Third, the use of ABA classrooms as an organizational change would be more concrete, observable, and permanent than other ways to integrate ABA practice into Michigan public schools (e.g., through intermittent consultation with credentialed professionals, etc.). Although the adoption of ABA classrooms is expected to meet the needs of schools and districts in most cases, it might not always be the best option. For example, small, rural schools might not have an appropriate number of students to occupy such a classroom. Additionally, if challenging behavior is so severe that students pose a significant risk to themselves or others, a more restrictive setting might be necessary to effectively integrate ABA into the existing public education system (e.g., specialized school). Future research might incorporate other types of behavioral services (e.g., consultation with credentialed professionals, the development of specialized ABA schools or treatment facilities) as the basis for organizational change in public schools. Additionally, public education is replete with examples of new initiatives, proposed 52 policy shifts, and new areas of focus to improve educational outcomes. Another direction for future research might be to apply the ORIC instrument to other types of organizational changes in educational settings to assess whether educators' commitment to change and perceived efficacy similarly constitute readiness for change. In the next chapter, I will delve into some of the mechanisms that are thought to contribute to change commitment. These will include potential contextual factors (e.g., school climate, interactions between families and schools) which could affect educators' willingness to adopt ABA classrooms. I will also discuss change commitment as a function of change valence, or the value that stakeholders place on a proposed change. This will provide a more thorough examination of various components of the Weiner (2009) model in the present context, as well as an evaluation of the model's structural utility when applied to the adoption of ABA classrooms in Michigan public schools. 53 CHAPTER 3: CONTEXTUAL FACTORS, CHANGE VALENCE, & CHANGE COMMITMENT LITERATURE REVIEW Contextual Factors Weiner’s (2009) theory of ORC posits that the contextual factors of an organization most likely have an indirect effect on readiness to change. For example, contextual factors (e.g., organizational culture) could affect change valence (i.e., whether stakeholders value the impending change), which in turn could influence how committed they are to the change. This chapter will review the literature relevant to each of these constructs, discuss how these constructs might be operationalized and measured in the educational context, and outline the methods that will be employed to assess their relationship. Contextual factors pertain to the specific and potentially idiosyncratic variables that describe the personnel, structure, and function of every organization. These might include organizational culture, policies and procedures which inform the climate of an organization, and past experience of change (Weiner, 2009). One widely adopted view of organizational culture is that it consists of assumptions (i.e., deeply held beliefs about human nature and the organizational environment), values (i.e., shared beliefs and rules that dictate employee attitudes and behavior), and artifacts (i.e., observable language, behavior, and symbols; Jones et al., 2005; Schein, 1990). Organizational culture which values strong human relations (e.g., Jones et al., 2005), promotes employee satisfaction (e.g., Ingersoll et al., 2000), and is oriented toward entrepreneurship (i.e., risk tolerance and innovation; Chonko et al., 2002) is thought to be supportive of ORC. Employee perception of an organizational system that is adaptable enough to accommodate change is another determinate of ORC. Eby et al. (2000) found that contextual 54 variables such as flexible policies and procedures, logistics and systems support, and trust in management were the most useful for understanding ORC when compared to individual or work group variables. These contextual determinants speak directly to the climate of an organization and are thought to support ORC and facilitate successful change. Past experience is also a key contextual variable which is thought to influence ORC. Some suggest that successful past experiences with organizational change can facilitate ORC (e.g., Armenakis et al., 1993; Weiner, 2009), and Hamilton et al. (2007) found that exemplars of past successful organizational change facilitated ORC among healthcare workers implementing a new assessment for acute stroke care. Similar findings suggest that previous experience with change efforts affects ORC in the specialty mental health domain as well. Hamilton et al. (2010) found that among mental health care providers, prior experience improved their readiness to implement an intervention for outpatients with schizophrenia. Taken together, these findings support the intuitively appealing notion that past success can set the stage for future ORC. Operationalizing Contextual Factors In the context of Michigan public schools, one might expect that principals and ISD administrators who believe deeply in a school’s responsibility to educate all children regardless of their needs, value special education, and use language and other symbols that indicate a commitment to neurodiversity and inclusion would embody a culture that is amenable to the adoption of ABA classrooms. This type of culture might emphasize social relationships between teachers, students, parents, and colleagues, as well as employee satisfaction, and a willingness to tolerate risk and innovate to ensure that student needs are met. Similarly, a school system that is perceived to be flexible enough to handle the challenges that would accompany the adoption of ABA classrooms would likely benefit from increased 55 ORC among stakeholders. If employees can rely on the logistical and systems support necessary at the school and/or ISD level, and trust in their supervisors to help navigate challenges, then they will likely feel confident enough to implement this change. Finally, a track record of well- managed change at a given school or ISD will likely also play a role. For example, if a school principal has been able to successfully implement other interventions to benefit students with special needs, they might be more ready to adopt an ABA classroom. Although it is reasonably straightforward to map the kinds of contextual factors discussed in the literature onto the present scenario, finding reliable and valid measurement is a challenge. Several items were developed for this dissertation to assess principals’ and ISD administrators’ past experiences with ABA- based intervention. Unfortunately, due to missingness on many of these variables, likely due to limited experience with ABA services in schools, most of these items were not retained for analysis. However, other relevant contextual determinants were carefully considered to ensure suitable operationalization and measurement. URP-IR System Climate and Family-School Subscales To operationalize contextual factors for reliable and valid measurement, this dissertation used the revised Usage Rating Profile – Intervention (URP-IR) system climate and family-school subscales (Briesch et al., 2013). The URP-IR has several key advantages: (1) it is an instrument that was specifically developed to assess perceptions of school-based interventions; (2) it has demonstrated adequate reliability and validity in samples of public-school educators; and (3) the system climate and family-school subscales elicit context-specific information that is critical for the successful implementation of interventions in schools. For example, the system-climate subscale assesses the compatibility of a given intervention with various aspects of the environment (e.g., the school’s mission, administrative support, job expectations, etc.; Briesch et 56 al., 2013). A flexible school environment that can accommodate implementation with support from principals and school administrators is expected to contribute to the value that stakeholders place on the adoption of ABA classrooms. This flexibility is expected to help align aspects of an organization (e.g., mission, policies and procedures, job expectations, etc.) with the requirements of this intervention. The family-school subscale of the URP-IR pertains to the critical role of school and family collaboration in any successful school-based initiative. It highlights contextual factors such as communication between educators and parents and the importance of a positive home- school relationship (Briesch et al., 2013). Although not discussed explicitly in the literature relevant to contextual determinants of ORC, family-school collaboration could be a critical component of the adoption of ABA classrooms in Michigan public schools. First, parents of students with ASD would almost certainly have to give permission for their children to attend this different kind of classroom. They might even advocate for the adoption of an ABA classroom, especially if they have seen the benefits of insurance-funded ABA in the home or clinic setting. Due to the parent training component of insurance-funded ABA therapy, parents are frequently called upon to help students generalize skills to the home or community environments. So, it is possible that parents of students with ASD who have had some exposure to ABA methodology will be more involved in their child’s skill acquisition and perhaps more interested in the collaborative approach espoused by ABA-based instruction. Change Valence Change valence is thought to be the mechanism by which contextual factors affect organizational stakeholders’ commitment to make a change (Weiner, 2009). If the situation on the ground necessitates an urgent shift, or if stakeholders believe that a given change might solve 57 a critical problem in the organization, then they are likely to value the impending change (Weiner, 2009). An example in the present context would be if parents of ASD students in Michigan started demanding ABA-based instruction in their individualized education plans (IEPs) on par with what students in other states receive. To meet this demand, schools and districts would have to either fund individualized ABA providers for every student or move to an ABA classroom model for the sake of efficiency. Stakeholders might also value the anticipated benefits for or an organizational change for the sake of themselves, their organization, or the clients they serve (Weiner, 2009). The obvious example here would be that school principals and ISD administrators would value ABA classrooms because of the benefits they would provide to students with ASD. Stakeholders might also value a given change because it amplifies their core values or because it is supported by their peers, superiors, or others in positions of influence (Weiner, 2009). If educators perceive support for the adoption of ABA classrooms by people whose opinions they trust, they will be more likely to value the change and commit to it themselves. Operationalizing Change Valence In the present context, change valence is the value that school principals and ISD administrators place on the proposed adoption of ABA classrooms (i.e., whether they perceive the change as important, beneficial, worthwhile, etc.). Given the positionality of change valence as a mediator, the value that stakeholders place on this change is expected to be highly context dependent. For example, if a school principal is stationed at a very small school which does not include any students with ASD in the population, they would not be expected to interpret the adoption of this highly specialized instructional methodology as particularly important, 58 beneficial, or worthwhile. Instead, they would be expected to devalue this approach since it does not meet a need or solve a problem at their school. URP-IR Acceptability Subscale In a similar manner to the way in which contextual factors mapped onto relevant URP-IR subscales, change valence will be measured using the URP-IR acceptability subscale (Briesch et al., 2013). This subscale gauges interest and enthusiasm for the implementation of a particular intervention in schools while also accounting for potential disruption to students, how well the intervention might fit into current practices, and an appraisal of the intervention’s ability to address a variety of problems in the classroom environment (Briesch et al., 2013). Thus, the URP-IR acceptability subscale is expected to effectively capture the critical elements of change valence specific to the school context for the adoption of ABA classrooms. Change Commitment A logical path can be traced from contextual factors to change valence and subsequently to change commitment. Circumstances at the school or district level will likely dictate the value that principals and administrators place on the adoption of ABA classrooms, which in turn could determine their level of commitment to act. This commitment might be expressed as motivation, determination, or desire to break from the status quo and commit to a rigorously systematic and empirically validated approach to instruction for students with ASD (Shea et al., 2014). The motivational aspect of change commitment implies intentionality and suggests that the broader concept of ORC should move beyond an attitudinal or evaluative appraisal of impending change (Weiner et al., 2020). The present conceptualization of change commitment holds that school principals and ISD administrators who are committed to the adoption of ABA classrooms are well-positioned to enact this change. They have concluded that the environmental conditions in 59 their school or district are suboptimal for students with ASD, they value a change that will improve this situation, and they are poised for action. Operationalizing Change Commitment As in the previous chapter, change commitment will be operationalized using four items from the ORIC instrument (Shea et al., 2014). These items will explicitly gauge the commitment of participants to adopt an ABA classroom in their school or district, as well as their desire, determination, and motivation to implement this change. If participants want this specific change to take place, and if they are determined and motivated to make it happen, then their commitment to adopt an ABA classroom will likely be high. METHOD This section describes the methodology used to address Research Question (2) What is the relationship between possible contextual factors and change commitment? I hypothesize that change valence will mediate the relationship between contextual factors and change commitment. More specifically, positive endorsement of contextual factors (i.e., system climate and family-school collaboration) will be associated with increases in participants’ ratings of the positive valence (i.e., acceptability) of ABA, which in turn will be associated with higher levels of change commitment (Hypothesis 2). 60 Figure 3 Relationship Between Possible Contextual Factors and Change Commitment Note. Figure modified from Weiner (2009). Constructs of interest are presented in bold font. Corresponding measurement instruments are presented in blue (Briesch et al., 2013; Shea et al., 2014). Sample and Data Collection This study relied on the same analytic sample, sampling approach, and data collection protocols as those outlined in Chapter 2. This left an analytic sample of 346 complete cases, comprised of 293 LEA principals and 53 ISD administrators from public schools across the state of Michigan. It focused on the ORIC change commitment subscale, which was previously discussed in Chapter 2 (Shea et al., 2014), as well as the URP-IR system climate and family- school subscales to measure possible contextual factors and the URP-IR acceptability subscale to measure change valence (Briesch et al., 2013). Measures URP-IR System Climate Subscale The system climate subscale was used to assess possible contextual factors. It is composed of 5 items on a Likert-type scale from (1) ‘strongly disagree’ to (6) ‘strongly agree.’ It 61 demonstrated adequate internal consistency reliability in the present study (α = .89). Participants who score high on this scale believe that a given intervention will be welcome, given a high degree of compatibility with the existing school environment (Briesch, et al., 2013). It contains items such as “Use of this intervention would be consistent with the mission of my school” (Briesch, et al., 2013). A mean composite score of the constituent items on the URP-IR system climate subscale was calculated and used in subsequent analyses. URP-IR Family-School Subscale The family-school subscale was used to assess possible contextual factors. It is composed of 3 items on a Likert-type scale from (1) ‘strongly disagree’ to (6) ‘strongly agree.’ It demonstrated an adequate level of internal consistency reliability in the present study (α = .88). Individuals who score higher on this subscale believe that successful interventions are largely made possible by a productive collaboration between families and school personnel (Briesch et al., 2013). It contains items such as “Parental collaboration is required in order to use this intervention” (Briesch et al., 2013). A mean composite score of the constituent items on the URP-IR family-school subscale was calculated and used in subsequent analyses. URP-IR Acceptability Subscale The acceptability subscale was used to assess change valence. It is composed of 9 items on a Likert-type scale from (1) ‘strongly disagree’ to (6) ‘strongly agree.’ It demonstrated adequate internal consistency reliability in the present study (α = .89). Participants who score high on this scale typically believe that a given intervention is appropriate and they are enthusiastic about implementing it (Briesch et al., 2013). It contains items such as “This intervention is a good way to handle students’ behavior problems” (Briesch et al., 2013). A mean 62 composite score of the constituent items on the URP-IR acceptability subscale was calculated and used in subsequent analyses. ORIC Instrument Change commitment was measured using the ORIC change commitment subscale. In line with findings from Chapter 2, change commitment was treated as a latent factor. A marker indicator approach (i.e., reference variable method, fixed marker scaling) was used to set the factor loading of the first indicator on the ORIC change commitment subscale to 1.0 and to allow for the factor variance to be scaled to the first indicator (Kline, 2016; Klopp & Klößner, 2021). This allows for the interpretation of regression coefficients when the latent change commitment factor is included as the outcome variable. A visual of the model can be found in Figure 3. For descriptives such as Pearson correlations, a mean composite score of the ORIC change commitment subscale was calculated and used. Control Variables It is possible that individual characteristics could affect participants’ perception of change commitment. For example, participants who have attained a postgraduate degree might have more exposure to research in support of evidence-based intervention for students with ASD and are thus more committed to the adoption of ABA classrooms. Conversely, educators who obtained their highest degree many years ago might have less exposure to supportive evidence for ABA classrooms. As such, demographic covariates such as length of time in current placement and position, years of educational experience, age, gender, race, educational level, and years since last degree were included in the model to control for potential effects on the outcome. Categorical covariates were binarized and recoded for inclusion in regression models. For example, the race variable was binarized into White (0) and not White (1). Gender was binarized 63 into male (0) and female (1). Highest degree (i.e., educational level) was binarized into no doctorate (0) and doctorate (1). In addition to the above control variables, the survey also included the Evidence-Based Practice Attitude Scale (EBPAS) to gauge participants’ stance toward evidence-based practices more generally (Aarons, 2004). The EBPAS demonstrated adequate internal consistency reliability in the present study (α = .82). Normality As in Chapter 2, Shapiro-Wilk tests were conducted to examine univariate normality for each relevant model variable (i.e., contextual factors and change valence). Similar to Chapter 2, each of the Shapiro-Wilk tests of key study variables yielded p-values < .05, indicating a significant departure from the univariate normal distribution. However, just as in Study 2, skewness and kurtosis values were rather small, ranging from -0.4 to -0.68 for skewness and - 0.07 to 0.45 for kurtosis. Therefore, non-normality is not expected to be a problem in the present analysis. Data Analysis Descriptive statistics such as means and standard deviations were calculated, as well as Pearson correlations between each of the continuous variables. These can be found in Table 3. As in Chapter 2, missing data was not a concern due to the use of only complete cases (N = 346). Predictors (i.e., contextual factors operationalized using the URP-IR system climate and family- school collaboration subscales) and the mediator (i.e., change valence operationalized by the URP-IR acceptability subscale) were mean-centered prior to analysis. The two exogenous variables (i.e., system climate and family-school collaboration) were treated as correlated in the structural equation model to account for a statistically significant correlation between the 64 variables and to account for the fact that they represent subscales of the same overall URP-IR measure. Several covariates (e.g., length of time in current placement and position, years of educational experience, age, gender, race, educational level, and years since last degree, and the EBPAS measure; Aarons, 2004) were included to control for various external factors that could otherwise affect participant endorsement of change commitment. Data was analyzed using a structural equation modeling (SEM) approach with mediated paths using the lavaan package for R and maximum likelihood (ML) estimation (R Core Team, 2017; Rosseel, 2012). SEM with mediated paths is the appropriate approach to use regarding Hypothesis 2 because it simultaneously assesses potential mechanisms by which one or more predictors (i.e., possible contextual factors) are associated with outcomes (i.e., change commitment) directly and through a mediating variable (i.e., change valence). It also accommodates the use of latent and non-latent variables within the same model and provides estimates of overall fit in addition to estimates of direct and indirect effects. As such, SEM with mediated paths offers a straightforward way to test the hypothesized mediating pathways in Weiner’s (2009) model of ORC. A similar approach has also been used to test the Weiner model previously (e.g., Hannon et al., 2017), so its present use builds on prior research in a new context. The present analysis relied on raw data to estimate variances and covariances among study variables using ML estimation which can be found at https://osf.io/ub254/. The SEM analysis unfolded in several steps. First, I interpreted the overall model goodness-of-fit using several indices provided by the lavaan package in R (e.g., χ2, CFI, TLI, RMSEA). I then obtained individual standardized and unstandardized regression coefficients for each model component along with percentile bootstrap confidence intervals from 1,000 bootstrapped resamples. Next, I interpreted the relative magnitude and statistical significance of 65 these coefficients. Finally, I assessed the percentile bootstrap confidence intervals for the indirect effects. Descriptives RESULTS Means, standard deviations and Pearson correlations for continuous study variables can be found in Table 3. Across key study variables such as system climate, family-school collaboration, and change valence (i.e., acceptability), average endorsement was above the midpoint of the relevant scale. Correlations generally align with expectations, given the variables presented and the context. For example, there was a statistically significant positive correlation between the contextual factors (i.e., system climate and family-school collaboration) (r = .25, p < .001). There was also a significant positive correlation between family-school collaboration and change valence (i.e., acceptability) (r = .27, p < .001) and a very strong positive correlation between system climate and change valence (r = .79, p < .001). Change commitment was positively correlated with the EBPAS (r = .33, p < .001), as well as system climate (r = .63, p < .001), family-school collaboration (r = .21, p < .001), and change valence (r = .64, p < .001). 66 Table 3 Chapter 3 Means, Standard Deviations, and Correlations Between Continuous Variables Variable M SD 1 2 3 4 5 6 7 8 9 1. Age 47.55 7.05 2. Years Since Degree 12.19 7.70 .57** 3. Years at Placement 4. Years in Role 5. Years as Educator 12.81 9.31 .35** .39** 6.03 4.98 .34** .41** .55** 21.86 6.51 .78** .63** .40** .39** 6. EBPAS 4.12 0.45 -.04 -.02 .07 .06 .02 7. System Climate 8. Family- School Collaboration 9. Change Valence 10. Change Commitment 4.72 0.80 .06 .07 .09 .15** .12* .36** 5.13 0.79 -.03 .04 -.02 -.01 .02 .23** .25** 4.57 0.75 .03 .02 .08 .05 .07 .41** .79** .27** 3.91 0.81 .11* .08 .08 .09 .13* .33** .63** .21** .64** Note. * indicates p < .05. ** indicates p < .01. SEM with Mediated Paths Structural equation modeling with mediated paths was used to examine the mechanisms by which contextual factors (i.e., system climate and family-school collaboration) might be associated with change commitment, both directly and indirectly through change valence. Various fit indices reflected good overall fit (χ2 = 119.65, df = 56, p < .001, RMSEA = 0.06, 67 SRMR = 0.07, CFI = 0.96, TLI = 0.95) according to conventional cutoffs (e.g., Hu & Bentler, 1999; MacCallum, Brown, & Sugawara, 1996). Given the treatment of change commitment as a latent factor and the marker indicator approach used, it should be noted that unstandardized regression coefficients pertinent to change commitment are scaled to the first indicator (i.e., a single unit shift in the predictor amounts to an estimated shift in change commitment based upon the scale of its first indicator). Unstandardized regression coefficients are presented in Fig. 4. Each regression controls for the effects of age, gender, race, years since degree, years in current school or district, years in current role, years as an educator, highest degree, and attitudes toward evidence-based practices as measured by the EBPAS instrument (Aarons, 2004). The direct effect of system climate on change valence was statistically significant and positive, b = 0.69, p < .001, as was the direct effect of change valence on change commitment, b = 0.34, p < .001. The direct effect of system climate on change commitment was similarly statistically significant and positive, b = 0.28, p < .001. The direct effect of family-school collaboration on change valence was not significant, b = 0.05, p = .18, nor was the direct effect of family-school collaboration on change commitment, b = 0.02, p = .57. Standardized regression coefficients can be found in Table 5, Appendix E. 68 Figure 4 Parameter Estimates for Direct Effects on Change Commitment Note. Regression coefficients are unstandardized. The relationship between system climate and family-school collaboration is expressed as a correlation. Dashed lines represent non-significant paths. *** = p < .001 The overall indirect effect of system climate on change commitment, mediated by change valence was found to be positive and statistically significant, b = 0.23, p < .001, 95% CI [0.13, 0.34], as indicated by percentile bootstrap confidence intervals that do not cross zero. No significant indirect effect of family-school collaboration on change commitment, mediated by change valence was found, b = 0.02, p = .19, 95% CI [-0.01, 0.05]. Taken together, evidence from the present study provides partial support for Hypothesis 2: change valence mediates the relationship between system climate and change commitment, but not the relationship between family-school collaboration and change commitment. 69 DISCUSSION The present study examined the relationship between contextual factors (i.e., system climate and family-school collaboration) and change commitment, and potential mechanisms by which contextual factors might indirectly influence change commitment through change valence. Weiner's (2009) theory of ORC posits that contextual factors likely have an indirect effect on change commitment. Furthermore, change commitment is thought to emerge as a function of change valence, or the extent to which stakeholders believe that a given change is necessary, beneficial, or worthwhile. Thus, it is plausible that contextual factors such as system climate (i.e., the compatibility of an intervention with environmental aspects of a school or district) and family-school collaboration (i.e., positive home/school communication that might facilitate the success of an intervention) might contribute to change valence, which in turn would influence educators' commitment to change. A more thorough understanding of these potential mechanisms is critical in the present context, because the adoption of specialized ABA classrooms in Michigan public schools will require significant commitment from educators. The present study seeks to uncover the underlying contributors to that commitment, thereby providing a potential lever for administrators, policymakers, and other stakeholders to use in service of organizational change. The present study began with an examination of descriptive statistics. Descriptives in the present study suggest an average tendency for educators to positively endorse the compatibility of ABA classrooms with the climate of their school or district. On average, participants also endorsed the necessity for good home/school collaboration to properly implement ABA classrooms. Finally, educators endorsed positive valence toward the adoption of ABA classrooms which suggests that on average, they believe that such a change would be necessary, 70 beneficial, and worthwhile. These are encouraging signs that educators seem generally supportive of ABA classrooms in Michigan public schools and perceive a potential benefit from the adoption of these classrooms. Respondents also seem to perceive an overall alignment between the mission of their schools/districts and the adoption of ABA classrooms, and the importance of caregiver relationships in getting this intervention off the ground. A correlational analysis between relevant study variables suggests a very strong positive relationship between the climate of a given school or district and the likelihood that respondents would endorse positive valence toward ABA classrooms. This provides some preliminary evidence for the potential importance of alignment between the mission of a school or district in meeting the needs of students with ASD and the perception of stakeholders that a shift toward ABA classrooms would be worthwhile. System climate also exhibited a strong positive relationship with change commitment in the present context. Building off these findings, the overall good model fit lends support to the inclusion of relevant study variables and the relationships between these variables. However, a closer look suggests a discrepancy between the contextual factors. Although system climate seems to play a meaningful role in both change valence and change commitment, family-school collaboration does not seem to play a role. System climate is positively associated with change valence, such that an educational climate which supports evidence-based initiatives bolsters educators' belief that the adoption of an ABA classroom would be necessary, beneficial, or worthwhile, which in turn is associated with endorsement of higher change commitment. By contrast, despite the most robust positive endorsement (M = 5.13 on a 6-point scale) of all predictor variables, family- school collaboration did not exhibit a significant direct or indirect relationship with the outcome of change commitment. Thus, the evidence did not support family-school collaboration having a 71 relevant association with change commitment. Taken together, the results of the present study provide partial support for H2, specifically H2a, which holds that positive endorsement of contextual factors (i.e., system climate) is associated with increases in participants’ ratings of the acceptability of ABA (i.e. change valence), which in turn is associated with higher levels of change commitment. Implications for Practice These findings are noteworthy for a few reasons. First, they suggest that the alignment between the organizational climate of a school or district and the adoption of ABA classrooms might be an important consideration for encouraging the successful adoption of ABA classrooms. These findings also suggest that system climate might directly contribute to change commitment as well as contributing indirectly through change valence. As such, it may be necessary for organizational changemakers to focus on both the system climate as a whole and the specific valence that stakeholders attribute to ABA classrooms when trying to enact this change. An example might be a school administrator focusing resources on building a school climate that is aligned with evidence-based practices and also making efforts to demonstrate the specific utility of ABA classrooms in meeting the specific needs of special education students. These efforts would align the intervention (i.e., ABA classrooms) with the overall mission of the school while also bringing individual benefits to light to ensure that educators perceive the intervention as worthwhile. The lack of support for an association between family-school collaboration and change valence or change commitment is notable as well. Since the intervention will take place in school with potentially limited input from parents, it is possible that their role in the implementation of ABA classrooms is de-emphasized by educators. Also, since parental influence on the adoption 72 of ABA classrooms would likely be distal (e.g., through advocacy at school board meetings, etc.), it makes sense that parental collaboration would play a small role in the present context. Finally, it is also possible that the parent-school collaboration variable suffered from ceiling effects and a resulting lack of variation which might suggest that although educators endorse the general importance of collaboration with families at a high level, it might not play a role in shaping commitment to adopt ABA classrooms. One takeaway for organizational changemakers might be to inform parents of the potential benefits of a specialized ABA classroom and then focus on sharing positive outcomes with them as the school year unfolds, rather than seeking collaboration at the outset. Given the specialized nature of this intervention and the need for highly trained personnel and a structured environment, it might be more beneficial to inform parents about how they might carry over best practices to the home setting and promote generalization. Limitations and Future Directions Limitations and future directions that pertain to all studies in this dissertation will be discussed in Chapter 5. However, one potential limitation specific to Chapter 3 is the potential for misalignment between the measures used in the present study with Weiner's (2009) model. For example, the Weiner (2009) model is vague when it comes to describing which contextual factors might predict organizational readiness for change. I chose system climate and family- school collaboration for two reasons. First, they seemed plausible contextual variables that might contribute to the adoption of ABA classrooms. Second, they mapped onto the subscales of a validated measure that has been widely used to assess interventions in schools (i.e., URP-IR, Briesch et al., 2013). However, there are likely a plethora of contextual factors that could affect perceptions of valence and change commitment in the present context. These might include 73 socioeconomic factors, the composition and priorities of local school boards, federal, state, and local legislation regarding special education, the demographic composition of a given school or district (e.g., the number of students whose diagnostic profile makes them a good candidate for an ABA classroom), and perhaps even the political landscape. A related potential limitation is the choice to map the change valence construct onto the URP-IR acceptability subscale. Even though this construct and its analogue seem to match closely, as operationally defined and measured by an instrument validated in the context of school-based intervention, there is always the possibility for conceptual mismatch. However, since the ORIC measure does not cover any of the potential mechanisms thought to contribute to ORC, this was a worthwhile risk. One future direction for the present work would be to operationalize and test other contextual factors using the Weiner model (2009). Other future directions might be to apply this model to other types of organizational changes in schools, different applications of applied behavior analysis in the public-school setting, or to examine the adoption of ABA classrooms in private or specialized schools. In the next chapter, I will explore the theoretical mechanisms that are thought to contribute to the other constituent factor of ORC (i.e., change efficacy). These will include the same contextual factors previously discussed but will introduce three new mediators thought to comprise the informational assessment component of Weiner's (2009) model. These include task demands (i.e., an understanding of the necessary steps to successfully implement an intervention), resource perceptions (i.e., stakeholder perceptions of the requisite resources for successful implementation), and situational factors (i.e., the perception of situational factors favorable to implementation). 74 CHAPTER 4: CONTEXTUAL FACTORS, INFORMATIONAL ASSESSMENT, & CHANGE EFFICACY Contextual Factors and Change Efficacy LITERATURE REVIEW The second theoretical pathway through which contextual factors could influence ORC involves an informational assessment of the task demands, resource perceptions, and situational factors involved in the organizational change (Weiner, 2009). Since contextual factors and change efficacy have been thoroughly discussed in previous chapters, I will not revisit them in the same depth here. However, it is important to note that the mechanisms by which contextual factors could affect the informational assessment and subsequently affect change efficacy could play out in a variety of ways. For example, contextual factors (e.g., perceived congruence between a school’s mission and the implementation of special education initiatives) could lead to a more favorable assessment of the requirements to adopt an ABA classroom, and thus enhance participants’ ratings of change efficacy. Alternatively, strong bonds between parents and educators could lead to the perception that adequate resources exist to get an ABA classroom off the ground and thus contribute to a stronger sense of efficacy. Finally, a school climate that is supportive of evidence-based initiatives could lead to the perception of favorable situational factors that subsequently promote change efficacy. Contextual factors and change efficacy will be operationalized and measured in the same way as they have been previously. This chapter will focus on the literature relevant to the informational assessment component of Weiner’s (2009) model, delve into how this construct might be operationalized and measured in the present context, and outline the methods used to assess the relationship between possible contextual factors and change efficacy, mediated by 75 informational assessment. This chapter will conclude with results of this assessment of the relationships between key study variables and an interpretation of findings. Informational Assessment According to Helfrich et al. (2018), “Informational assessment refers to organizational members’ perceptions that the resources available to implement the change (human, financial, material, and informational) are sufficient to the demand” (p. 3). This appraisal requires organizational stakeholders to “acquire, share, assimilate, and integrate information” in order to ensure that task demands are well understood, necessary materials are available, and the change can be implemented, given the current situation (Weiner, 2009, p. 4). In the present context, school principals and ISD administrators most likely serve as local hubs of information, given the nascent status of ABA classrooms in Michigan schools and the lack of clarity about ABA as an overarching framework upon which various interventions are based. Thus, it is important for these leaders to positively appraise knowledge of task demands, requisite resources, and situational factors to ensure effective implementation. If organizational stakeholders share this favorable assessment, then they will likely be confident in the face of impending change and efficacy will be high (Weiner, 2009). Operationalizing Informational Assessment In the present context, informational assessment is the appraisal that schools and districts have the requisite knowledge of the task demands involved in adopting an ABA classroom, the resources necessary to get these classrooms off the ground, and that the situational factors will permit this change to occur. As in the case of change valence in the prior chapter, the positionality of informational assessment as a mediator dictates that this positive assessment will be highly context dependent. For example, if organizational stakeholders have some awareness 76 of the task demands involved in setting up an ABA classroom, if their school or district allocates ample funding to special education initiatives, and if the situational factors are favorable (e.g., open-minded staff, strong leadership in favor of the change, a mandate to align their instructional approach with empirically-validated best practices), then the assessment will likely be positive and perceived efficacy will be strong. URP-IR Understanding Subscale In a similar manner to how various constructs in the Weiner (2009) model were mapped onto URP-IR subscales in the previous chapter, each of the components of the informational assessment construct outlined in Figure 5 were assigned a corresponding subscale. First, the URP-IR understanding subscale was used to assess the task demands component of the informational assessment (Briesch et al., 2013; Weiner, 2009). This subscale is meant to assess the extent to which stakeholders understand how to implement a given intervention and a high score indicates substantial knowledge of the procedures involved and how to implement them (Briesch et al., 2013). In this way, the URP-IR understanding subscale provides a straightforward operationalization of task demand perceptions in the educational context. URP-IR Feasibility Subscale The resource perceptions component of informational assessment was assessed using the URP-IR feasibility subscale (Briesch et al., 2013; Weiner, 2009). This subscale assesses the perceived feasibility of a given intervention, and high scores indicate that stakeholders feel that a given intervention is possible to implement, given current conditions (Briesch et al., 2013). The items contained in the feasibility subscale are expected to capture the various types of resource demands that the adoption of ABA classrooms would entail in an educational setting (e.g., 77 materials, time, etc.), and thus provides a useful tool to assess resource perceptions in the educational domain. URP-IR System Support Subscale Finally, the situational factors component of informational assessment was assessed using the URP-IR system support subscale (Briesch et al., 2013; Weiner, 2009). The system support subscale is meant to assess the need for external support to implement a given intervention, with higher scores indicating that help from other adults will be necessary for success (Briesch et al., 2013). Obviously, schools are unique among organizational settings due to the presence of both adults and children, and the necessity of adults to lean on each other for support to implement changes for the students’ benefit. Though situational factors are perhaps the most ambiguous component of the informational assessment construct, the system support subscale helps to operationalize this concept for the present context, since it gauges the need for professional development, consultation, and additional resources which will all likely be vital to successfully adopt ABA classrooms (Briesch et al., 2013). METHOD This section will describe the methodology used to address Research Question (3) What is the relationship between possible contextual factors and change efficacy? I hypothesize that informational assessment will mediate the relationship between contextual factors and change efficacy. More specifically, positive endorsement of contextual factors (i.e., system climate and family-school collaboration) will be associated with increases in participants’ informational assessment (i.e., understanding, feasibility, and system support), which in turn will be associated with higher levels of change efficacy (Hypothesis 3a-f). 78 Figure 5 Relationship Between Possible Contextual Factors and Change Efficacy Note: Figure modified from Weiner (2009). Constructs of interest are presented in bold font. Corresponding measurement instruments are presented in blue (Briesch et al., 2013; Shea et al., 2014). Sample and Data Collection This study relied on the same sample, sampling approach, and data collection protocols as those outlined in Chapters 2 and 3. This involved the same analytic sample of 346 complete cases, comprised of 293 LEA principals and 53 ISD administrators from public schools across the state of Michigan. It focused on the ORIC change efficacy subscale, which was previously discussed in Chapter 2 (Shea et al., 2014), as well as the URP-IR system climate and family- school subscales to measure possible contextual factors, and the URP-IR understanding, feasibility, and system support subscales to measure the task demands, resource perceptions, and situational factors of informational assessment, respectively (Briesch, 2013). Measures URP-IR Understanding Subscale The understanding subscale was used to assess the task demands component of informational assessment. It is composed of 3 items on a Likert-type scale from (1) ‘strongly 79 disagree’ to (6) ‘strongly agree.’ It demonstrated an adequate level of internal consistency reliability in the present study (α = .97). Participants who score high on this scale believe that they understand the necessary actions that facilitate successful intervention (Briesch et al., 2013). It contains items such as “I understand the procedures of this intervention” (Briesch et al., 2013). A mean composite score of the constituent items on the URP-IR understanding subscale was calculated and used in subsequent analyses. URP-IR Feasibility Subscale The feasibility subscale was used to assess the resource perception component of informational assessment. It is composed of 6 items on a Likert-type scale from (1) ‘strongly disagree’ to (6) ‘strongly agree.’ It demonstrated an adequate level of internal consistency reliability in the present study (α = .78). Participants who score high on this scale endorse the practical utility of a particular intervention, given the current scenario (Briesch et al., 2013). It contains items such as “The total time required to implement the intervention procedures would be manageable” (Briesch et al., 2013). A mean composite score of the constituent items on the URP-IR feasibility subscale was calculated and used in subsequent analyses. URP-IR System Support Subscale The system support subscale was used to assess the situational factors component of informational assessment. It is composed of 3 items on a Likert-type scale from (1) ‘strongly disagree’ to (6) ‘strongly agree.’ It demonstrated adequate internal consistency reliability in the present study (α = .91). Participants who score high on this scale endorse the need for external support for successful implementation (Briesch et al., 2013). It contains items such as “I would require consultative support to implement this intervention” (Briesch et al., 2013). A mean 80 composite score of the constituent items on the URP-IR system support subscale was calculated and used in subsequent analyses. ORIC Instrument Change efficacy was measured using the ORIC change efficacy subscale (Shea et al., 2014). Similar to change commitment in Chapter 3, change efficacy was treated as a latent variable in the present study. A marker indicator approach (i.e., reference variable method, fixed marker scaling) was used to set the factor loading of the first indicator on the ORIC change efficacy subscale to 1.0 and to allow for the factor variance to be scaled to the first indicator (Kline, 2016; Klopp & Klößner, 2021). This allows for the interpretation of regression coefficients when the latent change efficacy factor is included as the outcome variable. A visual of the model can be found in Figure 5. For descriptives such as Pearson correlations, a mean composite score of the ORIC change efficacy subscale was calculated and used. Control Variables In addition to the above measures, just as in Chapter 3, the survey also included demographic covariates such as length of time in current placement and position, years of educational experience, age, gender, race, educational level, and years since last degree. These were included in the model to control for potential effects on the outcome. Categorical covariates were binarized and recoded for inclusion in regression models in the same fashion as Chapter 3. The Evidence-Based Practice Attitude Scale (EBPAS) was also included as a covariate to control for participants’ general stance toward evidence-based practices (Aarons, 2004). The EBPAS demonstrated adequate internal consistency reliability in the present study (α = .82). 81 Normality As in Chapters 2 and 3, Shapiro-Wilk tests were conducted to examine univariate normality for each relevant model variable (i.e., task demands, resource perceptions, and situational factors). Similar to Chapters 2 and 3, each of the Shapiro-Wilk tests of key study variables yielded p-values < .05, indicating a significant departure from the univariate normal distribution. However, just as noted previously, skewness and kurtosis values were rather small, ranging from -0.22 to 0.65 for skewness and -0.63 to 0.17 for kurtosis. Therefore, non-normality is not expected to be a problem in the present analyses. Data Analysis Descriptive statistics such as means and standard deviations were calculated for the ORIC change efficacy subscale and each of the relevant URP-IR subscales, as well as Pearson correlations between each of the continuous variables. These can be found in Table 4. As in Chapters 2 and 3, missing data was not a concern due to the use of only complete cases (N = 346). Predictors (i.e., contextual factors operationalized using the URP-IR system climate and family-school collaboration subscales) and mediators (i.e., task demands operationalized by the URP-IR understanding subscale, resource perceptions operationalized by the URP-IR feasibility subscale, and situational factors operationalized by the URP-IR system support subscale) were mean-centered prior to analysis. The two exogenous variables (i.e., system climate and family- school collaboration) were treated as correlated in the structural equation model to account for a statistically significant correlation between the variables and to account for the fact that they represent subscales of the same overall URP-IR measure. Similarly, covariances between each of 82 the mediators were estimated in the model since they comprise the related elements of the informational assessment from Weiner's (2009) theory of ORC. Various covariates were included (e.g., length of time in current placement and position, years of educational experience, age, gender, race, educational level, and years since last degree, and the EBPAS measure; Aarons, 2004) to account for various external factors that could otherwise affect participant endorsement of change efficacy. Just as in Chapter 3, data was analyzed using a structural equation modeling (SEM) approach with mediated paths using the lavaan package for R and maximum likelihood (ML) estimation (R Core Team, 2017; Rosseel, 2012). SEM with mediated paths is the appropriate approach to use regarding Hypothesis 3 because it simultaneously assesses potential mechanisms by which one or more predictors (i.e., possible contextual factors) are associated with outcomes (i.e., change efficacy) directly and indirectly through a mediating variable (i.e., task demands, resource perceptions, and situational factors). It also accommodates the use of latent and non- latent variables within the same model and provides estimates of overall fit in addition to estimates of direct and indirect effects. As such, SEM with mediated paths offers a straightforward way to test the hypothesized pathways in Weiner’s (2009) model of ORC. The present analysis relied on raw data to estimate variances and covariances among study variables using ML estimation which can be found at https://osf.io/ub254/. Like Chapter 3, SEM analysis unfolded in several steps. First, I interpreted the overall model goodness-of-fit using several indices provided by the lavaan package in R (e.g., χ2, CFI, TLI, RMSEA). I then obtained individual standardized and unstandardized regression coefficients for each model component along with percentile bootstrap confidence intervals from 1,000 bootstrapped resamples. Next, I interpreted the relative magnitude and statistical 83 significance of these coefficients. Finally, I assessed the percentile bootstrap confidence intervals for the indirect effects. RESULTS Means, standard deviations, and Pearson correlations for continuous study variables can be found in Table 4. Across most key study variables (e.g., task demands, resource perceptions, and change efficacy), average endorsement was slightly above the midpoint of the relevant scale. For system support, average endorsement was considerably lower than the midpoint of the scale, indicating that respondents were generally confident that they could implement the adoption of ABA classrooms with their available resources and personnel (i.e., without much need for external support). Correlations generally align with expectations, given the variables presented and the context. For example, there were particularly strong positive correlations between system climate and resource perceptions (r = .56, p < .001), and between task demands and resource perceptions (r = .59, p < .001). There were also particularly strong positive correlations between system climate and change efficacy (r = .61, p < .001), and between resource perceptions and change efficacy (r = .50, p < .001). There was a statistically significant negative correlation observed between the need for family-school collaboration and system support (r = -.23, p < .001), but a statistically significant positive correlation between system support and task demands (r = .17, p = .001). 84 Table 4 Chapter 4 Means, Standard Deviations, and Correlations Between Continuous Variables Variable 1. Age M SD 1 2 3 4 5 6 7 8 9 10 11 47.55 7.05 2. Years Since Degree 12.19 7.70 .57** 3. Years at Placement 12.81 9.31 .35** .39** 4. Years in Role 6.03 4.98 .34** .41** .55** 5. Years as Educator 21.86 6.51 .78** .63** .40** .39** 6. EBPAS 4.12 0.45 -.04 -.02 .07 .06 .02 7. System Climate 4.72 0.80 .06 .07 .09 .15** .12* .36** 8. Family-School Collaboration 5.13 0.79 -.03 .04 -.02 -.01 .02 .23** .25** 9. Task Demands 3.56 1.26 -.02 -.05 -.01 -.01 -.01 .03 .34** .07 10. Resource Perceptions 3.79 0.71 .05 .04 .06 .05 .06 .18** .56** .19** .59** 11. Situational Factors 1.88 0.80 -.00 -.00 -.06 -.04 .01 -.21** -.08 -.23** .17** .00 12. Change Efficacy 3.73 0.81 .14** .13* .08 .15** .15** .29** .61** .21** .26** .50** -.09 Note. * indicates p < .05. ** indicates p < .01. 85 SEM with Mediated Paths Structural equation modeling with mediated paths was used to examine the mechanisms by which contextual factors (i.e., system climate and family-school collaboration) might be associated with change efficacy, both directly and indirectly through an informational assessment (i.e., task demands, resource perceptions, and situational factors). Various fit indices reflected reasonably good overall fit (χ2 = 196.46, df = 79, p < .001, RMSEA = 0.07, SRMR = 0.06, CFI = 0.94, TLI = 0.89) with estimated values falling within or near conventional cutoffs (e.g., Hu & Bentler, 1999; MacCallum, Brown, & Sugawara, 1996). Though the TLI is slightly lower than expected, I look to guidance from Niemand and Mai (2018) who advocate for flexibility in cutoff values, rather than a rigid adherence to arbitrary cutoffs for fit indices. Taken together, the results indicate a reasonably close fit between the model and the data. Similar to Chapter 3, the outcome variable (i.e., change efficacy) was treated as a latent factor and the marker indicator approach was used. Therefore, it should be noted that unstandardized regression coefficients pertinent to change efficacy are scaled to the first indicator (i.e., a single unit shift in the predictor amounts to an estimated shift in change efficacy based upon the scale of its first indicator). Unstandardized regression coefficients are presented in Figure 6. It should be noted that Figure 6a and 6b represent parts of the same structural equation model, where exogenous variables (i.e., system climate and family-school collaboration) were treated as correlated just as in Chapter 3. The model components were split in Figures 6a and 6b for the sake of clarity. Each regression controls for the effects of age, gender, race, years since degree, years in current school or district, years in current role, years as an educator, highest degree, and attitudes toward evidence-based practices as measured by the EBPAS instrument (Aarons, 2004). 86 Looking at the model in Figure 6, only one of the components of information assessment had a significant relationship with change efficacy. Specifically, the direct relationship between resource perceptions and change efficacy was statistically significant and positive, b = 0.28, p < .001. However, the relationships between task demands and change efficacy (b = -0.02, p = .52), as well as situational factors and change efficacy (b = -0.02, p = .62) were non-significant. The direct relationship of system climate and change efficacy was statistically significant and positive, b = .39, p < .001, as was the direct relationship of system climate and task demands, b = 0.6, p < .001, and system climate and resource perceptions, b = 0.51, p < .001. The direct relationship of system climate and situational factors was not significant, b = 0.03, p = .67. The overall indirect relationship of system climate and change efficacy, mediated by resource perceptions was found to be positive and statistically significant, b = 0.14, p < .001, 95% CI [0.07, 0.22], as indicated by percentile bootstrap confidence intervals that do not cross zero. The overall indirect relationship of system climate and change efficacy, mediated by task demands was not statistically significant, b = -0.01, p =.52, 95% CI [-0.05, 0.03], nor was the indirect relationship of system climate and change efficacy mediated by situational factors, b = -0.001, p =.87, 95% CI [-0.01, 0.01]. The direct relationship of family-school collaboration and change efficacy was non- significant, b = 0.05, p = .18. A statistically significant negative association between family- school collaboration and situational factors was observed, b = -0.2, p = .001. However, there were non-significant direct relationships observed between family school collaboration and task demands (b = 0.00, p = .97), as well as family-school collaboration and resource perceptions (b = 0.05, p = .29). Standardized regression coefficients can be found in Table 6, Appendix E. The overall indirect relationship of family-school collaboration and change efficacy, mediated by task 87 demands was non-significant, b = 0.00, p = .99, 95% CI [-0.01, 0.01], as indicated by percentile bootstrap confidence intervals that cross zero. The overall indirect relationship of family-school collaboration and change efficacy, mediated by resource perceptions was similarly non- significant, b = 0.01, p = .31, 95% CI [-0.01, 0.04], as was the indirect relationship of family- school collaboration and change efficacy, mediated by situational factors, b = 0.01, p = .65, 95% CI [-0.02, 0.03]. Taken together, results from the present study provide partial support for Hypothesis 3, specifically H3b: resource perceptions mediate the relationship between system climate and change efficacy. 88 Figure 6 Parameter Estimates for Direct Effects on Change Efficacy Note. Figure 6a and 6b represent parts of the same structural equation model in which a covariance was estimated between system climate and family-school collaboration. Regression coefficients are unstandardized. The relationship between mediators is expressed as a correlation. Dashed lines represent non-significant paths. ** indicates p <.01, *** indicates p < .001 DISCUSSION The present study examined the relationship between contextual factors (i.e., system climate and family-school collaboration) and change efficacy, and potential mechanisms by which contextual factors might indirectly relate to change commitment through an informational 89 assessment which includes an understanding of task demands, resource perceptions, and situational factors. Weiner's (2009) theory of ORC posits that contextual factors likely have an indirect association with change efficacy. For example, it is plausible that contextual factors (e.g., alignment between a school's organizational climate and the adoption of ABA classrooms) could lead to an increased understanding of what is required for implementation, perceived access to the resources required to get these classrooms up and running, and a positive endorsement of the situational factors that would be supportive of ABA classrooms, which in turn could lead to enhanced efficacy. A more thorough understanding of potential mechanisms by which Michigan educators might become more able to implement ABA classrooms is important because it speaks to the pragmatic considerations required for stakeholder empowerment. The present study seeks to examine respondents' understanding of what task demands are required, whether the change is feasible given available resources, whether situational factors would permit the adoption of ABA classrooms, and how each of these considerations might contribute to the relationship between contextual factors and change efficacy. These findings are expected to provide a roadmap for administrators, policymakers, and stakeholders to focus resources on the logistical concerns that will likely play a key role in the adoption of ABA classrooms in Michigan. The analyses in Chapter 4 began with an examination of descriptive statistics. Descriptives in the present study suggest an average tendency for educators to positively endorse an understanding of the task demands necessary for ABA classroom adoption, a perception that they generally would have the resources available to institute this change, and that in general, they would be able to enact this change, based on ratings of change efficacy. It should be noted 90 that endorsement was just slightly above the midpoint of the 6-point scale for task demands and resource perceptions. In the case of situational factors (i.e., system support), average responding fell considerably lower than the midpoint of the scale which suggests that respondents do not believe that the adoption of ABA classrooms will require a lot of external support or outside consultation. This level of responding might have been partially attributable to the wording of the prompt (see Appendix C, p. 134-135) which stated that during the hypothetical shift toward ABA classrooms, appropriate training for educators would be provided, as would consultation with behavioral experts, and that none of the material resources would be paid for by the school's normal budget (i.e., additional funds would be provided). It is possible that some respondents answered the system support items as though little additional training, consultation, and resources would be needed above and beyond what was already included in the prompt, rather than answering more generally about whether the types of support included in the prompt would be necessary in their particular circumstance. The wording of the prompt was intended to mitigate respondents' concerns about a potential drain on their extant resources that might come along with this organizational change. However, it might have also contributed to the perception that they would have everything they need to implement an ABA classroom in their school or district. Another possibility might be attributable to the overall lack of ABA integration, which could cause respondents to be less familiar with the necessary requirements for specialized classroom adoption and thus assume that they would be able to implement this change with little outside support. One final possibility could be an issue with the operationalization of the situational factors variable, which is discussed in more depth as a potential limitation below. 91 Correlational analyses suggest a particularly strong positive relationship between system climate and resource perceptions, which could provide evidence for the connection between a supportive organizational climate and stakeholders' perception that they have what is required to successfully implement ABA classrooms. Another particularly strong positive relationship was found between system climate and change efficacy, which suggests that this supportive organizational climate is also associated with stakeholders feeling able to enact change more directly. A similarly strong positive association was found between task demands and resource perceptions, which suggests that stakeholders who understand what is required to implement an ABA classroom are also more likely to endorse having the required resources. Finally, a noteworthy positive relationship between resource perceptions and change efficacy was also found, which suggests that the perception of having the required resources is associated with stakeholders feeling able to change. Looking ahead from these findings, evidence of acceptable model fit lends support for the inclusion of Chapter 4 study variables and the relationships between them. However, similar to Chapter 3, a discrepancy between the exogenous contextual factors of system climate and family-school collaboration emerged. Significant positive direct relationships between system climate and task demands, resource perceptions, and change efficacy were observed, as well as a significant indirect relationship between system climate and change efficacy mediated by resource perceptions. In contrast, only one significant direct relationship was pertinent to family-school collaboration, and it was in the negative direction, in addition to no significant indirect relationships pertinent to family-school collaboration. Overall, system climate could play a meaningful role in the lead up to Michigan educators feeling able to implement organizational change in the form of ABA classroom 92 adoption. Findings suggest that stronger endorsement of alignment between school/district climate and ABA classroom adoption is associated with stronger endorsement of the required resources for implementation and also stronger endorsement of efficacy to undertake this change. Although only one mediational pathway was supported, it makes intuitive sense that the pathway through resource perceptions (i.e., feasibility) plays a significant role in determining stakeholder efficacy to implement change. After all, the key component of the informational assessment is whether stakeholders feel able to marshal the necessary resources to implement change (Helfrich et al., 2018). Since the resource perceptions construct taps into resource availability directly, it is perhaps not surprising that it was found to mediate the relationship between system climate and change efficacy. Taken together, the results of the present study provide partial support for H3, specifically H3b, which holds that positive endorsement of contextual factors (i.e., system climate) is associated with increases in participants' informational assessment (e.g., resource perceptions), which in turn is associated with higher levels of change efficacy. Similar to Chapter 3, family-school collaboration played a less meaningful role in the mechanisms thought to contribute to enhanced efficacy to adopt ABA classrooms. Family-school collaboration did not have a significant direct relationship with change efficacy, and none of the mediated pathways involving family-school collaboration as an endogenous variable were found to be significant. The only significant relationship involving family-school collaboration was a negative direct association with situational factors. This suggests that participants who endorse higher levels of need for family-school partnership to successfully implement ABA classrooms also endorse less of a need for additional external support to get these classrooms off the ground. Conversely, low endorsement of the need for home-school collaboration was associated with an enhanced need for outside professional help, professional development, and additional resources. 93 It is possible that administrators who strongly value more formal support like professional consultation are less likely to see the value of informal, community-centered supports like family-school collaboration. It is also possible that respondents were focused on the initial thrust to get the ABA classrooms up and running which might be associated with more parental input (i.e., advocacy, support of the school board, etc.) and less focus on professional consultation which might be associated with challenges that might arise later (i.e., on a more ad- hoc basis). Finally, it is possible that despite educators acknowledging the importance of family- school collaboration, as evidenced by high endorsement, these personal relationships have little bearing on the informational assessment or change efficacy that educators experience because internal school/district dynamics are more central to educators' perceptions of the necessary information, resources, and efficacy for implementation. In any case, just as in Chapter 3, findings suggest that family-school collaboration does not seem to play a meaningful role in shaping Michigan educators' change efficacy regarding the adoption of ABA classrooms. Implications for Practice These findings are noteworthy for a few reasons. First, they reinforce the importance of system climate as a key component of organizational readiness to adopt ABA classrooms as previously discussed in Chapter 3. Secondly, they highlight the importance of resource perceptions (i.e., feasibility) as a critical mechanism by which system climate is associated with change efficacy. Alignment between a school/district's mission and a given intervention could lead to educators' perceptions that they have what it takes to implement the intervention and thus lead to enhanced efficacy to follow through with the change. Since system climate seems to contribute directly to change efficacy and indirectly through resource perceptions, it might be necessary for organizational changemakers to focus on both the system climate as a whole and 94 the resource perceptions that constituents attribute to ABA classrooms when trying to implement this shift. For example, if a district administrator focuses resources on building a district climate that is aligned with evidence-based practices, they should probably also (1) make sure that staff have what they need to follow through on evidence-based practices, and (2) ensure that staff are fully aware of the resources at their disposal. In this case, the administrator will have aligned the mission of the district to the adoption of ABA classrooms in a way that also enhances the material resources and the perception that staff have what they need for successful implementation. The lack of support for family-school collaboration playing a role in task demands, resource perceptions, and change efficacy is also noteworthy. Similar to the findings in Chapter 3, these findings for the present study point to the potential for limited input from parents and a small role in the actual implementation of ABA classrooms. While most educators view family- school collaboration as important in general, they do not necessarily perceive it as related to their efficacy to adopt ABA classrooms. One potential takeaway for organizational changemakers is to be aware of the potentially negative relationship between family-school collaboration and situational factors. Positive endorsement of the importance of family input and collaboration with schools does not necessarily translate to an openness to receive outside help writ large. Despite low endorsement of the perceived need for outside support for implementation, the highly specialized nature of ABA classrooms and lack of credentialed behavioral staff in schools will likely necessitate some outside expertise. So, changemakers should be discerning if they bring in outside consultative help for implementation and might have to help manage the relationships between families, teachers, and outside consultants when trying to get ABA classrooms off the ground. 95 Limitations and Future Directions Limitations and future directions pertinent across dissertation studies will be discussed in Chapter 5. One limitation specific to the present study is the potential misalignment of measures for informational assessment and Weiner's (2009) model. URP-IR subscales (i.e., understanding, feasibility, and system support) were used to operationalize Weiner's (2009) constructs (i.e., task demands, resource perceptions, and situational factors), and despite a close match in terms of definition and operationalization, it is always possible that these subscales do not exactly tap into the theorized constructs. One potential future direction for the present research would be to expand and validate a new version of the ORIC measure (Shea et al., 2014), which would include the constructs that are theorized to contribute to change commitment and change efficacy. This would allow for better alignment between the theoretical constructs and the measures, as well as a more comprehensive way to test Weiner's (2009) model. The next and final chapter will sum up findings from Chapters 1-4. More specifically, it will discuss the aims and significance of the present dissertation, offer implications for research and practice, and discuss overall limitations and future directions, before concluding with some final thoughts. 96 CHAPTER 5: CONCLUSION THE PRESENT DISSERTATION This dissertation examined organizational readiness for change (ORC) as it relates to Michigan educators' potential adoption of specialized applied behavior analysis (ABA) classrooms in Michigan public schools. The positioning of ORC as a theoretical precursor to the successful implementation of an intervention, as well as the lack of access to ABA classrooms for students with special needs in Michigan public schools makes this work particularly relevant, especially for stakeholders who aspire to leverage behavioral science to improve educational equity in these settings. In addition to exploring the ORC construct as it pertains to this potential organizational change, I also closely examined the possible mechanisms thought to contribute to ORC as put forth in Weiner's (2009) model. Dissertation Aims The were several key aims of the present dissertation. Aims for Chapter 1 included reviewing the literature relevant to the efficacy of ABA in the educational setting and examining potential barriers to integration of ABA unique to Michigan public schools. Chapter 2 described Weiner's (2009) theory of ORC which provided the framework for the present dissertation, reviewed the challenges of operationalization and measurement of ORC, and presented the ORIC measure as a potential way to overcome these barriers (Shea et al., 2014). Chapter 2 then went on to examine the proposed two-factor structure of ORC (i.e., change commitment and change efficacy) which set the stage for later chapters that differentiated these two factors as unique outcome variables. The aim of Chapter 3 was to assess the potential mechanisms by which contextual factors (i.e., system climate and family-school collaboration) might be associated with change commitment directly and indirectly through change valence. The aim of Chapter 4 was to 97 assess the possible mechanisms by which contextual factors might be associated with change efficacy directly and indirectly through an informational assessment of the resources necessary to adopt ABA classrooms. In this fifth and final chapter, I will integrate findings across the previous chapters and focus on the significance of this research, implications for future research, implications for future practice, and overall limitations and future directions. Significance This research is significant for several reasons. First, there has been a paucity of research on ORC in educational settings. The present work addresses a gap in the literature and deepens our understanding of how ORC takes shape in this unique context. Second, this research identifies mechanisms by which educators might adopt evidence-based practices in general, with potential applicability beyond ABA. Third, this research assessed readiness with an eye toward aligning current practice in Michigan public schools with the needs of special education students. If the adoption of ABA classrooms were to take place, this could present a more equitable option for students with disabilities, helping them keep pace with general education peers. Therefore, this change is expected to have a direct and positive impact on students with special needs and their families. Finally, this research is significant due to its translational nature, its focus on stakeholders and their needs (i.e., students, families, educators, practitioners, and policymakers), and steps toward more thorough integration of behavioral science in Michigan public schools. Regarding the translational component, this dissertation represents actionable research for practitioners whose partnership will be critical to get this intervention off the ground. The identification of mechanisms associated with ORC present key points of leverage for credentialed behavioral practitioners (i.e., BCBAs) to partner with schools, get more of a foothold for behavioral science 98 in educational policies and practices, and perhaps even take the more concrete step of implementing specialized ABA classrooms that run with appropriate supervision. For example, practitioners who are partnering with schools for the first time might be contracted to work on one student's behavioral challenges. However, this research suggests that it might also be useful to pay attention to the climate of the school as a whole. Practitioners could attend to larger systemic factors, such the school's organizational stance toward behavioral intervention when drafting a behavior plan to meet one student's behavioral needs. Attending to these larger systemic factors is expected to help interventions be more successful, regardless of the level of implementation (i.e., individual, classroom, or school-wide). Given the unique language of behavior analysis, I also made efforts to translate the constructs and processes in the present dissertation to be more meaningful and useful for behavioral practitioners. This includes defining ORC in terms of private events, thinking of the processes that predict ORC in terms of settings events and motivating operations, as well as thinking about the potentially reinforcing consequences of successful implementation of specialized ABA classrooms (e.g., meeting students' needs, teachers receiving the appropriate resources for behavior management, administrators placing students in the least restrictive environment). IMPLICATIONS FOR RESEARCH The findings of this dissertation present several implications for future research. One such implication discussed in detail within Chapter 2 is the applicability of the two-factor structure of ORC to the educational field, where it likely functions in a similar way to other previously studied fields (e.g., healthcare). Given the importance of context in the study of organizational change readiness (i.e., idiosyncrasies of the proposed change itself, unique 99 characteristics of stakeholders and settings, etc.), the applicability of this two-factor structure to the educational sphere is encouraging. From these findings, one might infer that there is some continuity to the structure of ORC that holds, despite its application to seemingly disparate organizational changes in very different fields (e.g., healthcare, education, etc.). This presents an opportunity to integrate what is known about the ORC construct across fields and apply it further to organizational changes in other domains as well. Building off support for the two-factor structure of ORC, another implication for future research from Chapters 3 and 4 is the idea that certain components of the Weiner (2009) model (e.g., system climate, change valence, and resource perceptions) might play more of a role than others (e.g., family-school collaboration, task demands, and situational factors) in stakeholders' readiness for change. However, it is not clear whether these differences could be attributable to the proposed organizational change itself, the choice of which variables might constitute the contextual factors from the Weiner model, and/or how the components of the model that fall outside the scope of the ORIC instrument are conceptualized and measured. In the case of the present dissertation, the proposed organizational change (i.e., ABA classroom adoption) had unique qualities which might make certain factors related to ORC particularly salient while de-emphasizing others. For example, the contextual factor of system climate seemed to play an important role, while the contextual factor of family-school collaboration seemed less crucial. It is possible that the particular characteristics of the proposed change itself might have contributed to respondents' prioritization of one contextual factor over another. For example, ABA classrooms exist within the confines of the school and will likely not require a lot of direct input from parents. A different initiative, such as a parent-mediated homework intervention, would likely require more correspondence with parents to be successful, 100 and might draw more heavily on parent-school collaboration. In this way, future research could test whether the role of contextual factors vary by the nature of the proposed change. It is also likely that the specific choices that researchers make regarding which contextual factors to include will play a role in future ORC research. The incorporation and treatment of contextual factors is an important conceptual contribution of the present dissertation. Weiner's (2009) model leaves the contextual factors vague, which is likely necessary to accommodate a multitude of different organizational changes in different contexts but presents an important gap that needs to be filled. To test the model in a particular context, contextual factors need to be defined, operationalized, and measured. In the present research, I was able to overcome this deficit using metrics from the implementation science literature (i.e., URP-IR subscales). The operationalization and measurement of an otherwise vague construct, using validated scales specifically developed for school-based intervention, presents a path forward for future research which seeks to incorporate contextual factors for the study of ORC in educational settings. Furthermore, this mapping of implementation science metrics onto organizational constructs represents a merging of different fields and constitutes a conceptual contribution in and of itself, which might lead to other useful points of intersection between implementation science and organizational research. Future research that tests Weiner's (2009) model of ORC will likely need to make similar decisions and choose contextual factors that are relevant to the particular organizational context and proposed change put forth. Given the impact that these choices could have on the assessment of the model, these choices will require careful consideration. A related consideration is the operationalization and measurement of other components (i.e., change valence, informational assessment) of the Weiner (2009) model that are perhaps less 101 vague than the contextual factors, but are also not presently incorporated into the ORIC measure (Shea et al., 2014). Future research might be able to expand upon and validate a new iteration of the ORIC measure that includes these components. In any case, future research will have to incorporate theoretically relevant contextual factors and consider the operationalization and measurement of other key variables from measures that are validated in relevant settings. Given the specific findings from Chapter 3 in the present dissertation that show support for the association of system climate with change commitment both directly and indirectly through change valence, future research might build on these findings and assess whether this is a pathway by which change commitment could be increased in the lead up to an organizational change. For example, if a particular researcher wanted to develop and test an intervention to increase change commitment before the introduction of an organizational change, they might want to focus on the alignment of the organizational climate with the change itself, as well as stakeholder perceptions of whether the change is necessary, beneficial, and worthwhile (i.e., positive change valence) as an empirically derived starting point. A similar implication for future research can be drawn from the Chapter 4 findings relevant to the direct association between system climate and change efficacy, and indirect association by way of resource perceptions. In the case of future research that might seek to develop and test interventions to improve efficacy in the lead up to an organizational change, the present research indicates that a potentially useful area of focus would be on the climate of the organizational system itself, as well as stakeholder perceptions that they have the required resources to implement change. Given the resource expenditures likely necessary for the development and testing of such an intervention, the present dissertation provides useful 102 information about where the most meaningful areas of focus might be in relation to organizational readiness for change. These implications for future research from the present dissertation point to the clarity and utility of conceptualizing ORC in terms of its two-factor structure (i.e., change commitment and change efficacy) and emphasize the mechanisms that were found to be most salient for readiness for change. These findings contribute to our understanding of ORC in a novel organizational context with a potentially transformative change (i.e., ABA classroom adoption in Michigan public schools). By testing this organizational theory and connecting it with the wider literature on educational interventions, implementation science, and applied behavior analysis, this research contributes to the knowledge base in each area as well as to the integration of concepts across areas for the benefit of students with special needs. In the following section, I will discuss implications for practice from the present findings. IMPLICATIONS FOR PRACTICE There are several implications of this research that will likely be useful for practitioners who seek to more thoroughly integrate behavioral science in schools. The first is the importance of focusing on the role of both stakeholder commitment and efficacy in organizational change. Given the evidence that suggests the dual contribution of each of these factors to ORC overall, organizational changemakers would do well to focus on both as they try to spur increased readiness for change. In practice, a school principal or administrator who seeks to get staff motivated, engaged, and ready for change should focus on building the perception that the change is necessary, beneficial, and worthwhile in addition to ensuring that staff have everything that is needed to be successful. 103 Another key implication of this research is that system climate could matter for both change commitment and change efficacy. If this is the case, administrators could devote resources to align their school or district climate to the evidence-based intervention that they seek to implement. This could be achieved broadly, by touting evidence-based practices as a central feature of school climate in general, or in more targeted ways which identify the alignment between school climate and specific interventions that administrators are trying to instantiate. In the case of ABA classrooms, this might involve highlighting useful practices that are derived from ABA research that educators are likely already using (e.g., positive praise, token reinforcement systems, visual cues for students who need them), and framing ABA classrooms as an extension of these practices that aligns with the organizational mission. If the climate of the school is aligned with the evidence base, then staff will likely start from a posture of enhanced commitment and efficacy to implement evidence-based change more readily. In line with the potential importance of system climate's direct relationship with the constituent factors of ORC, system climate also potentially sets the tone for enhanced stakeholder commitment through change valence. Efforts toward improving school climate to align with evidence-based practices could connect the dots between climate, valence, and commitment for stakeholders. For example, in the specific case of ABA classrooms where administrators begin with efforts to promote a climate of evidence-based practice for the benefit of students with special needs, they might also emphasize the aspects of an ABA classroom that make it necessary, beneficial, and worthwhile. Administrators could focus on the strong empirical basis of ABA, its alignment with the school's mission, the ways that ABA classrooms could improve outcomes for students (and teachers), and why these outcomes necessitate and deserve sustained commitment. In this way, administrators who use and promote language 104 around alignment between school climate, the positive aspects of ABA classrooms (i.e., valence), and stakeholder commitment will draw on the associations between these concepts to maximize readiness to implement change. Furthermore, given the lack of causal inference that can be drawn from the present findings, it should be noted that change valence could play a key role in stakeholder commitment, independent of system climate. If this were the case, it would be critical to ensure that stakeholders perceive the adoption of ABA classrooms as necessary, beneficial, and worthwhile, regardless of school climate. Though not ideal, if valence were the key concern and the immediate stakeholders endorsed positive regard for the adoption of ABA classrooms, they might still achieve positive outcomes at the classroom level without a very supportive school climate in the background. Additionally, system climate is also potentially associated with change efficacy through resource perceptions. Once administrators have fostered a school or district climate that is supportive of evidence-based practices generally and ABA classrooms specifically, it will likely be useful to align that climate with the availability of resources to support ABA classroom adoption. This will likely involve getting stakeholders what they need, but also making sure that they perceive the availability of all necessary resources. For example, an administrator who seeks to implement an ABA classroom at their school might invest in software for classroom data collection. These software packages often come replete with training modules for ABA program implementation. However, if educators are not aware of these features, they might feel as though they have the means to collect data but lack the training to execute on the ABA-based pedagogical techniques that data collection was meant to capture. In this case, the administrator would do well to make classroom staff aware of all the resources at their disposal and perhaps 105 make efforts to ensure that staff fully understand the pedagogical techniques of an ABA classroom and the proper way to take data in various contexts. This would likely be easier with the help of a credentialed professional in ABA whom staff could also look to as a resource in getting the specialized ABA classroom off the ground. In this way, a school climate that is aligned with evidence-based practices would ensure that staff have what they need to implement those practices successfully. Furthermore, the staff would have full awareness of the resources at their disposal and feel empowered to implement change. On the subject of resource perceptions, it will likely also be critical to ensure that stakeholders are fully aware of the resource requirements for successful ABA classroom adoption. The present findings suggest that respondents feel confident to implement this change without much external support. However, the ABA model requires regular supervision from credentialed professionals. Given the present lack of integration of ABA into Michigan public schools, it seems likely that most of this supervision would come from outside practitioners. ABA classrooms will also require quite a few other elements to be successful. These include, but are not limited to physical space, staff training for behavior management and skill acquisition, investment and training in new assessment protocols, data collection software, classroom materials, etc. It is critical that stakeholders are fully aware of what is required so that every effort is made to marshal the necessary resources, make staff aware of what is available, and ensure that resources are utilized to the fullest. The next section describes limitations and future directions that apply across all previous chapters. OVERALL LIMITATIONS AND FUTURE DIRECTIONS Overall limitations for this dissertation include the lack of generalizability of the findings beyond Michigan public educators, an inability to account for different levels of analysis, the 106 complicating factors of data collection during the COVID-19 pandemic, the relegation of the present research to only a portion of Weiner's (2009) model, the fact that a significant portion of respondents had no prior knowledge of ABA, the potential mismatch between theoretical constructs and those captured by the measures, and the use of cross-sectional data which prohibits causal conclusions. First, it is important to note that the findings presented here only pertain to a representative sample of Michigan principals and school administrators. The Michigan public education system is highly unique due to its structure (i.e., the utilization of intermediate school districts for special education and other related services), legislative landscape, stance toward applied behavior analysis, etc. As such, it is not clear whether the findings presented here would generalize to other locations with different circumstances. Given the lack of generalizability, one future direction for research would be to expand this work to other states in the Midwest, other areas of the country, and perhaps even with a representative sample of U.S. educators to assess whether findings might be applicable to the wider population. A related future direction for this research might involve a comparison of the present findings to a state that has already achieved robust adoption of ABA classrooms to ascertain similarities and differences in the mechanisms thought to contribute to ORC. A related limitation of the present research is an inability to account for different levels of analysis (e.g., individual school, district, county-level ISD, etc.) where participants are positioned. A more thorough examination at different levels was not possible for a few reasons. First, in this study, I aimed to collect data from representative samples of principals and ISD administrators through the use of simple random sampling. This sampling design provides representative data across the state of Michigan but does not provide enough principals and ISD administrators clustered within county to conduct a multi-level analysis. A randomized cluster 107 sample that involves recruiting a certain number of ISD administrators from each ISD and a certain number of principals from a given district was not logistically or economically feasible. Second, in quite a few cases, I found that an ISD administrator or district-level superintendent might also function as a principal for one or more schools. This typically occurred in less populous districts where one administrator could feasibly execute more than one role. This means that a multilevel analysis would not be possible without a totally different sampling strategy which could exclude less populous districts that do not have enough employees in a given role to comprise a suitable group. A related consequence of the present sampling approach is that system climate, which might ordinarily be assessed as a group or organization-level attribute, was assessed as individuals' perception of climate instead. A group-level measure would be ideal, given that the climate for successful implementation is typically measured using a "collection of multi-dimensional perceptual data from many expected innovation users within an organization" and "implies a high degree of within-group agreement in climate perceptions" (Weiner et al., 2011, p. 1). However, future research might overcome the challenges endemic to the structure of the Michigan public school system with a multi-level approach. Issues with data collection and other concerns during the COVID-19 pandemic were also a potential limitation of the present research. Many educators anecdotally reported feeling overwhelmed by pandemic-related responsibilities (e.g., the shift to/from online learning when schools were closed and then re-opened, managing public health mandates for masking, testing, contact tracing, etc.). Even though educators were encouraged to think of a time after the COVID-19 pandemic when answering questions about ABA classroom adoption, they were likely still affected to some degree by the additional burden of the pandemic. It is unclear whether they would have responded to questions about implementation of a new initiative in the 108 same way at a time when they were not faced with these pressing issues. From a logistical perspective, the pandemic seemed to drastically increase churn within the sample, such that many educators left their position and had to be replaced in the sample by their successor. One future direction for research would be to try to replicate these findings now that the pandemic is no longer such a concern. It could be important to assess how new educational initiatives might be perceived (and potentially de-prioritized) when acute and pervasive threats to public health take precedent. It seems likely that these threats would consume resources and bandwidth that could otherwise be devoted to new program implementation. Another limitation of the present research is that I was not able to test Weiner's (2009) model in its entirety. The theoretical model goes a bit further than the present dissertation to suggest the potential effect that ORC has on change-related effort and subsequent implementation effectiveness. It will be critical to explore the effect of organizational readiness on the effort put forth by stakeholders to make implementation a success. This might be achieved by future research testing Weiner's (2009) model in its entirety. This larger project might also involve piloting an educational intervention to increase ORC with educators prior to the implementation of a new initiative, and a comparison to a control group. This type of research would assess whether it is possible to increase ORC in the lead up to organizational change, and whether this increase is associated with change-related effort and success of implementation. An additional potential limitation of the present dissertation is that only 80.3% of respondents had prior awareness of ABA before taking the survey. It is possible that prior awareness of ABA could color responding, such that response patterns for participants with prior awareness might differ significantly than the wider sample. However, sensitivity analyses 109 indicate that conclusions would not change if the analyses in Chapters 2-4 only focused on the subgroup of participants with prior awareness of ABA (See Appendix F). Another limitation that was discussed more specifically in previous chapters but affects this research overall is a potential mismatch between the theoretical constructs and the ones captured by the measures. For example, I set out to measure system climate as a contextual factor, but I was contending with the vague nature of what counts as a contextual factor in Weiner's (2009) model and the fact that climate could not be measured at the group or organizational level. Thus, it is possible that the system climate variable in the present study is more akin to Rogers' (2003) concept of compatibility, or the extent to which potential adopters perceive congruence between an innovation and the “existing values, past experiences, and needs” within their schools or districts (p. 15). Future work might go further to isolate and measure the contextual factors that are expected to play the largest role in a given scenario with a specific organizational change. Finally, it is important to note that the present research relies on cross-sectional data, so causal conclusions cannot be drawn, and results should be interpreted with this in mind. This is certainly a limitation of the present work, but future research might employ longitudinal data to overcome this limitation, and perhaps even time waves of data collection to capture information about organizational readiness for change before, during, and after the change occurs. This type of approach would likely get closer to causal inference than the present research and provide more conclusive evidence regarding the mechanisms that predict ORC. CONCLUSION The present dissertation examined organizational readiness to adopt specialized ABA classrooms in Michigan public schools. In doing so, I was able to examine Weiner's (2009) 110 theory of ORC in a novel context with an important potential organizational change that is expected to benefit many public-school students with special needs. The results of the present dissertation suggest that ORC functions in the educational domain similarly to other domains (e.g., healthcare) where it has been studied in the past, such that change commitment and change efficacy are the constituent factors. This research also suggests that certain variables (i.e., system climate, change valence, and resource perceptions) play a meaningful role in the mechanisms associated with change commitment and change efficacy. This research is perhaps the first to examine ORC relevant to ABA intervention. It is expected to contribute to the literature in developmental and organizational psychology, as well as ABA and implementation science, particularly within the realm of school-based intervention. However, I hope that the main contribution of this research will be to help illuminate why ABA treatment has not been as widely adopted in Michigan public schools as it has elsewhere, identify potential barriers to comprehensive integration of ABA treatment in schools, and provide insight into how educators, parents, advocates, and policymakers might leverage behavioral science to improve outcomes for students and their families. 111 REFERENCES Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental health services research, 6(2), 61-74. https://doi.org/10.1023/B:MHSR.0000024351.12294.65 Abdinnour-Helm, S., Lengnick-Hall, M. L., & Lengnick-Hall, C. A. (2003). Pre-implementation attitudes and organizational readiness for implementing an enterprise resource planning system. European Journal of Operational Research, 146(2), 258-273. https://doi.org/10.1016/S0377-2217(02)00548-9 Anderson, S. R. (2000, March 9). Clinical practice guidelines: Methodology, findings, and implications [Conference session]. The Association for Science in Autism Treatment Conference. New York, NY, United States. Armenakis, A. A., Harris, S. G., & Mossholder, K. W. (1993). Creating readiness for organizational change. Human Relations, 46(6), 681-703. https://doi.org/10.1177/001872679304600601 Bandura, A. (1997). Self-efficacy: The exercise of control. W.H. Freeman and Company. Bijou, S. W., & Baer, D. M. (1961). Child development I: systematic and empirical theory. Prentice-Hall. Bohman, T. M., Kulkarni, S., Waters, V., Spence, R. T., Murphy-Smith, M., & McQueen, K. (2008). Assessing health care organizations’ ability to implement screening, brief intervention, and referral to treatment. Journal of Addiction Medicine, 2(3), 151-157. https://doi.org/10.1097/ADM.0b013e3181800ae5 Bouckenooghe, D. (2010). Positioning change recipients’ attitudes toward change in the organizational change literature. Journal of Applied Behavioral Science, 46(4), 500-531. https://doi.org/10.1177/0021886310367944 Briesch, A. M., Chafouleas, S. M., Neugebauer, S. R., & Riley-Tillman, T. C. (2013). Assessing influences on intervention implementation: Revision of the Usage Rating Profile- Intervention. Journal of School Psychology, 51(1), 81-96. https://doi.org/10.1016/j.jsp.2012.08.006 Brown, T. A., & Moore, M. T. (2012). Confirmatory factor analysis. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 361–379). The Guilford Press. Cherry, B. (2011). Assessing organizational readiness for electronic health record adoption in long-term care facilities. Journal of Gerontological Nursing, 37(10), 14-19. https://doi. org/10.3928/00989134-20110831-06 112 Chonko, L. B., Jones, E., Roberts, J. A., & Dubinsky, A. J. (2002). The role of environmental turbulence, readiness for change, and salesperson learning in the success of sales force change. Journal of Personal Selling & Sales Management, 22(4), 227-245. https://doi.org/10.1080/08853134.2002.10754311 Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Pearson Education, Inc. The Council of Autism Service Providers [CASP]. (2020). Applied behavior analysis treatment of autism spectrum disorder: Practice guidelines for healthcare funders and managers (2nd ed.). https://casproviders.org/wp-content/uploads/2020/03/ABA-ASD-Practice- Guidelines.pdf Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16– 29. https://doi.org/10.1037/1082-989X.1.1.16 Eby, L. T., Adams, D. M., Russell, J. E., & Gaby, S. H. (2000). Perceptions of organizational readiness for change: Factors related to employees' reactions to the implementation of team-based selling. Human Relations, 53(3), 419-442. https://doi.org/10.1177/0018726700533006 Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2002). Intensive behavioral treatment at school for 4-to 7-year-old children with autism: A 1-year comparison controlled study. Behavior Modification, 26(1), 49-68. https://doi.org/10.1177/0145445502026001004 Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2007). Outcome for children with autism who began intensive behavioral treatment between ages 4 and 7: A comparison controlled study. Behavior Modification, 31(3), 264-278. https://doi.org/10.1177/0145445506291396 Ferreri, S., & Bolt, S. (2011). Educating Michigan's students with autism spectrum disorder (ASD): An initial exploration of programming "The ASD Michigan Project." Education Policy Center at Michigan State University. https://files.eric.ed.gov/fulltext/ED525238.pdf Foxx, R. M. (2008). Applied behavior analysis treatment of autism: The state of the art. Child and Adolescent Psychiatric Clinics of North America, 17(4), 821-834. https://doi.org/10.1016/j.chc.2008.06.007 Gagnon, M. P., Attieh, R., El Kebir Ghandour, F. L., Ouimet, M., Estabrooks, C. A., & Grimshaw, J. (2014). A systematic review of instruments to assess organizational readiness for knowledge translation in health care. PloS One, 9(12). https://doi.org/10.1371/journal.pone.0114338 113 Gist, M.E., & Mitchell, T.R. (1992). Self-efficacy - A theoretical analysis of its determinants and malleability. Academy of Management Review. 17, 183-211. https://doi.org/10.5465/amr.1992.4279530 Grindle, C. F., Hastings, R. P., Saville, M., Hughes, J. C., Huxley, K., Kovshoff, H., ... & Remington, B. (2012). Outcomes of a behavioral education model for children with autism in a mainstream school setting. Behavior Modification, 36(3), 298-319. https://doi.org/10.1177/0145445512441199 Hamilton, S., McLaren, S., & Mulhall, A. (2007). Assessing organisational readiness for change: use of diagnostic analysis prior to the implementation of a multidisciplinary assessment for acute stroke care. Implementation Science, 2(1), 1-11. https://doi.org/10.1186/1748-5908-2-21 Hamilton, A. B., Cohen, A. N., & Young, A. S. (2010). Organizational readiness in specialty mental health care. Journal of General Internal Medicine, 25(1), 27-31. https://doi.org/10.1007/s11606-009-1133-3 Hannon, P. A., Helfrich, C. D., Chan, K. G., Allen, C. L., Hammerback, K., Kohn, M. J., ... & Harris, J. R. (2017). Development and pilot test of the workplace readiness questionnaire, a theory-based instrument to measure small workplaces’ readiness to implement wellness programs. American Journal of Health Promotion, 31(1), 67-75. https://doi.org/10.4278/ajhp.141204-QUAN-604 Helfrich, C. D., Kohn, M. J., Stapleton, A., Allen, C. L., Hammerback, K. E., Chan, K. G., ... & Hannon, P. A. (2018). Readiness to change over time: change commitment and change efficacy in a workplace health-promotion trial. Frontiers in Public Health, 6, 110. https://doi.org/10.3389/fpubh.2018.00110 Holt, D. T., Armenakis, A. A., Harris, S. G., & Feild, H. S. (2007a). Toward a comprehensive definition of readiness for change: A review of research and instrumentation. In W.A. Pasmore and R.W. Woodman (Eds.) Research in organizational change and development (pp. 289-336). JAI Press, Inc. Holt, D. T., Armenakis, A. A., Feild, H. S., & Harris, S. G. (2007b). Readiness for organizational change: The systematic development of a scale. The Journal of Applied Behavioral Science, 43(2), 232-255. https://doi.org/10.1177/0021886306295295 Howard, J. S., Sparkman, C. R., Cohen, H. G., Green, G., & Stanislaw, H. (2005). A comparison of intensive behavior analytic and eclectic treatments for young children with autism. Research in Developmental Disabilities, 26(4), 359-383. https://doi.org/10.1016/j.ridd.2004.09.005 Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118 114 Ingersoll, G. L., Kirsch, J. C., Merk, S. E., & Lightfoot, J. (2000). Relationship of organizational culture and readiness for change to employee commitment to the organization. The Journal of Nursing Administration, 30(1), 11-20. https://doi.org/10.1097/00005110-200001000-00004 Jackson, D. L., Gillaspy Jr, J. A., & Purc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychological Methods, 14(1), 6. https://doi.org/10.1037/a0014694 Jones, R. A., Jimmieson, N. L., & Griffiths, A. (2005). The impact of organizational culture and reshaping capabilities on change implementation success: The mediating role of readiness for change. Journal of Management Studies, 42(2), 361-386. https://doi.org/10.1111/j.1467-6486.2005.00500.x Kantor, J. R. (1959). Interbehavioral psychology: The metasystem. In J. R. Kantor, Interbehavioral psychology: A sample of scientific system construction (2nd rev. ed., pp. 69–74). Principia Press. https://doi.org /10.1037/13165-006 Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology. New York: Appleton- Century-Crofts. Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44(3), 486-507. https://doi.org/10.1177/0049124114543 Kline, R. B. (2016). Principles and practice of structural equation modeling (Fourth ed.). The Guilford Press. Klopp, E. & Klößner, S. (2021). The impact of the choice of scaling methods on the properties and interpretation of parameter estimates in structural equation models with latent variables. Structural Equation Modeling: A Multidisciplinary Journal, 28(2), 182-206. https://doi.org /10.1080/10705511.2020.1796673 Lewin, K. (1947). Frontiers in group dynamics: concept, method and reality in social science; Social equilibria and social change. Human Relations, 1(1), 5-41. https://doi.org/10.1177/001872674700100103 Lewin, K. (1951). Field theory in social science: Selected theoretical papers (Edited by Dorwin Cartwright.). Harpers. Lindig, A., Hahlweg, P., Christalle, E., & Scholl, I. (2020). Translation and psychometric evaluation of the German version of the Organisational Readiness for Implementing Change measure (ORIC): A cross-sectional study. BMJ Open, 10(6). https://doi.org /10.1136/bmjopen-2019-034380 115 MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130. https://doi.org/10.1037/1082-989X.1.2.130 Maenner, M. J., Shaw, K. A., Bakian, A. V., Bilder, D. A., Durkin, M. S., Esler, A., ... & Cogswell, M. E. (2021). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2018. MMWR Surveillance Summaries, 70(11), 1. https://doi.org/10.15585/mmwr.ss7011a1 McGarrell, M., Healy, O., Leader, G., O’Connor, J., & Kenny, N. (2009). Six reports of children with autism spectrum disorder following intensive behavioral intervention using the Preschool Inventory of Repertoires for Kindergarten (PIRK®). Research in Autism Spectrum Disorders, 3(3), 767-782. https://doi.org/10.1016/j.rasd.2009.02.006 McGee, H. M., & Crowley-Koch, B. J. (2021). Performance assessment of organizations. Journal of Organizational Behavior Management, 41(3), 255-285. https://doi.org/10.1080/01608061.2021.1909687 Michigan Department of Health and Human Services (MDHHS). (2019). Michigan medicaid applied behavior analysis update. https://www.michigan.gov//media/Project/Websites/autism/Reference- Guides/Michigan_Medicaid_Autism_Program_Update.pdf?rev=a1ce4765145c44178ef1f ad0ed1d11b9. Michael, J. (2007). Motivating Operations. In J. O. Cooper, T. E. Heron, & W. L. Heward (Eds.), Applied behavior analysis (2nd ed., pp. 374–391). Pearson. Michigan Association of Administrators of Special Education (MAASE). (2017). Individualized family service plan and individualized education program considerations for students with ASD receiving insurance-based treatment/intervention. http://maase.pbworks.com/w/file/fetch/123664986/ASD%20in%20IFSP%20and%20IEP %20Process%20.pdf Michigan Association of Administrators of Special Education (MAASE). (2021). ABA & Michigan schools: More is more. https://www.maase.org/wp- content/uploads/2021/03/ABA_3.2021-1.pdf Michigan Department of Education. (2017). MI school data, 2017-2018 special education counts [Data set]. MI School Data. https://www.mischooldata.org/Other2/DataFiles/StudentCounts/HistoricalSpecialEducati onEnrollment.aspx Michigan Department of Education. (2021). MiMTSS Technical Support Catalog Version 2.0. https://mimtsstac.org/sites/default/files/Documents/About/MiMTSS%20TA%20Catalog %202021%20-%20Interactive%207-28.pdf 116 Michigan Department of Education. (2022). Family matters: Applied behavior analysis fact sheet. https://www.michigan.gov//media/Project/Websites/mde/specialeducation/familymatters/ FM1/ABA_FactSheet.pdf?rev=b4dbc0c8c0954ce7ac162de3829a72f3 Michigan Department of Education. (n.d.-a). Special education programs data portraits overview: Disability. https://www.mischooldata.org/special-education-data-portraits- disability/ Michigan Department of Education. (n.d.-b). Quick Facts. https://www.mischooldata.org/quick-facts/ Michigan Department of Education. (n.d.-c). Staffing count. https://www.mischooldata.org/staffing-count/ Michigan Department of Education. (n.d.-d). MI School data compare tool. https://www.mischooldata.org/compare-tool/ Miake-Lye, I. M., Delevan, D. M., Ganz, D. A., Mittman, B. S., & Finley, E. P. (2020). Unpacking organizational readiness for change: an updated systematic review and content analysis of assessments. BMC Health Services Research, 20(1), 1-13. https://doi.org/10.1186/s12913-020-4926-z Moshagen, M., & Erdfelder, E. (2016). A new strategy for testing structural equation models. Structural Equation Modeling, 23, 54–60. https://doi.org/10.1080/10705511.2014.950896 Niemand, T., & Mai, R. (2018). Flexible cutoff values for fit indices in the evaluation of structural equation models. Journal of the Academy of Marketing Science, 46, 1148- 1172. http://doi.org /10.1007/s11747-018-0602-9 Nosik, M. R., & Carr, J. E. (2015). On the distinction between the motivating operation and setting event concepts. The Behavior Analyst, 38(2), 219-223. http://doi.org /10.1007/s40614-015-0042-5 Neal, Z.P., Neal, J.W., & Piteo, A. (2020). Call me maybe: Using incentives and follow-ups to increase principals’ survey response rates. Journal of Research on Education Effectiveness, 13(4), 784-793. https://doi.org/10.1080/19345747.2020.1772423 Partington, W. J. (2010). ABLLS-R: The assessment of basic language and learning skills. Behavior Analyst, Inc. Prochaska, J. O., & DiClemente, C. C. (1983). Stages and processes of self-change of smoking— Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51(3), 390-395. https://doi.org/10.1037/0022-006X.51.3.390 117 Rafferty, A. E., Jimmieson, N. L., & Armenakis, A. A. (2013). Change readiness: A multilevel review. Journal of Management, 39(1), 110-135. https://doi.org/10.1177/0149206312457417 R Core Team (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/ Rogers, E. M. (2003). Diffusion of Innovations (5th ed.). Free Press. Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02 Ruest, M., Léonard, G., Thomas, A., Desrosiers, J., & Guay, M. (2019). French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Services Research, 19, 1-14. https://doi.org/10.1186/s12913-019-4361-1 Schein, E. H. (1990). Organizational culture. American Psychologist, 45(2), 109–119. https://doi.org/10.1037/0003-066X.45.2.109 Shea, C. M., Jacobs, S. R., Esserman, D. A., Bruce, K., & Weiner, B. J. (2014). Organizational readiness for implementing change: a psychometric assessment of a new measure. Implementation Science, 9(1), 7. https://doi.org/10.1186/1748-5908-9-7 Skinner, B. F. (1938). The behavior of organisms: an experimental analysis. Appleton-Century-Crofts. Skinner, B. F. (1945). The operational analysis of psychological terms. Psychological Review, 52(5), 270-277. https://doi.org/10.1037/h0062535 State of Michigan. (2022, May 26). Autism Legislation. Michigan.gov. https://www.michigan.gov/autism/insurance Steinbrenner, J. R., Hume, K., Odom, S. L., Morin, K. L., Nowell, S. W., Tomaszewski, B., Szendrey, S., McIntyre, N. S., Yücesoy-Özkan, S., & Savage, M. N. (2020). Evidence- based practices for children, youth, and young adults with autism. The University of North Carolina at Chapel Hill, Frank Porter Graham Child Development Institute, National Clearinghouse on Autism Evidence and Practice Review Team. https://files.eric.ed.gov/fulltext/ED609029.pdf Storkholm, M. H., Mazzocato, P., Tessma, M. K., & Savage, C. (2018). Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementation Science, 13, 1-7. https://doi.org/10.1186/s13012-018-0769-y 118 Sugai, G., Horner, R. H., Dunlap, G. Hieneman, M., Lewis, T. J., Nelson, C. M., Scott, T., Liaupsin, C., Sailor, W., Turnbull, A. P., Turnbull, H. R., III, Wickham, D. Reuf, M., & Wilcox, B. (2000). Applying positive behavioral support and functional behavioral assessment in schools. Journal of Positive Behavioral Interventions, 2, 131-143. https://doi.org/10.1177/109830070000200302 Sundberg, M. L. (2008). VB-MAPP: Verbal behavior milestones assessment and placement program. A language and social skills assessment program for children with autism or other developmental disabilities: Guide. Mark Sundberg. Swindle, T., Whiteside-Mansell, L., & Johnson, D. (2018). Organizational readiness for implementing a nutrition curriculum in early care and education. Journal of Nutrition Education and Behavior, 50(7), S78. https://doi.org/10.1016/j.jneb.2018.04.129 Twyman, J. S. (2014). Behavior analysis in education. In F. K. McSweeney & E. S. Murphy (Eds.), The Wiley Blackwell handbook of operant and classical conditioning (pp. 533– 558). Wiley Blackwell. https://doi.org/10.1002/9781118468135.ch21 U.S. Department of Health and Human Services (1999). Mental health: A report of the surgeon general. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National Institutes of Mental Health. https://www.google.com/books/edition/Mental_Health/S3NHAAAAMAAJ?hl=en&gbpv =0 Weiner, B. J., Amick, H., & Lee, S. Y. D. (2008). Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Medical Care Research and Review, 65(4), 379-436. https://doi.org /10.1177/1077558708317802 Weiner, B. J. (2009). A theory of organizational readiness for change. Implementation Science, 4(1), 67. https://doi.org/10.1186/1748-5908-4-67 Weiner, B. J., Belden, C. M., Bergmire, D. M., & Johnston, M. (2011). The meaning and measurement of implementation climate. Implementation Science, 6, 1-12. https://doi.org/10.1186/1748-5908-6-78 Weiner, B. J., Clary, A. S., Klaman, S. L., Turner, K., & Alishahi-Tabriz, A. (2020). Organizational readiness for change: what we know, what we think we know, and what we need to know. In: Albers, B., Shlonsky, A., Mildon, R. (Eds.) Implementation Science 3.0 (pp. 101-144). Springer, Cham. https://doi.org/10.1007/978-3-030-03874-8_5 119 Wilczynski, S., Green, G., Ricciardi, J., Boyd, B., Hume, A., Ladd, M., & Rue, H. (2009). National standards report: The national standards project: Addressing the need for evidence-based practice guidelines for autism spectrum disorders. National Autism Center, Randolph (MA). https://mn.gov/mnddc/asd-employment/pdf/09-NSR-NAC.pdf Zachor, D. A., Ben-Itzchak, E., Rabinovich, A. L., & Lahat, E. (2007). Change in autism core symptoms with intervention. Research in Autism Spectrum Disorders, 1(4), 304-317. https://doi.org/10.1016/j.rasd.2006.12.001 120 APPENDIX A: IRB APPROVAL EXEMPT DETERMINATION Revised Common Rule October 28, 2020 To: Jennifer Renee Watling Neal Re: MSU Study ID: STUDY00005237 Principal Investigator: Jennifer Renee Watling Neal Category: Exempt 2ii Exempt Determination Date: 10/28/2020 Limited IRB Review: Not Required. Title: Change for the Better: Assessing Organizational Readiness for ABA Classrooms in Michigan Public Schools This study has been determined to be exempt under 45 CFR 46.104(d) 2ii. Principal Investigator (PI) Responsibilities: The PI assumes the responsibilities for the protection of human subjects in this study as outlined in Human Research Protection Program (HRPP) Manual Section 8-1, Exemptions. Institutional restrictions to in-person human subject research activities conducted by MSU employees, MSU students, or agents of MSU are in place, but MSU is phasing in human research that has the potential for in-person interactions with participants, using a Tier approach. Restrictions to in- person interactions with human research participants by MSU employees, MSU students, or agents of MSU are in place until the activity is permitted under a Tier and a Human Research Plan for a Safe Return is approved. Visit http://hrpp.msu.edu/COVID-19/index.html for the restrictions, Tiers, forms, and the process. Continuing Review: Exempt studies do not need to be renewed. Modifications: In general, investigators are not required to submit changes to the Michigan State University (MSU) Institutional Review Board (IRB) once a research study is designated as exempt as long as those changes do not affect the exempt category or criteria for exempt determination (changing from exempt status to expedited or full review, changing exempt category) or that may substantially change the focus of the research study such as a change in hypothesis or study design. See HRPP Manual Section 8-1, Exemptions, for examples. If the study is modified to add additional sites for the research, please note that you may not begin the research at those sites until you receive the appropriate approvals/permissions from the sites. Please contact the HRPP office if you have any questions about whether a change must be submitted for IRB review and approval. New Funding: If new external funding is obtained for an active study that had been determined exempt, a new initial IRB submission will be required, with limited exceptions. If you are unsure 121 if a new initial IRB submission is required, contact the HRPP office. IRB review of the new submission must be completed before new funds can be spent on human research activities, as the new funding source may have additional or different requirements. Reportable Events: If issues should arise during the conduct of the research, such as unanticipated problems that may involve risks to subjects or others, or any problem that may increase the risk to the human subjects and change the category of review, notify the IRB office promptly. Any complaints from participants that may change the level of review from exempt to expedited or full review must be reported to the IRB. Please report new information through the study’s workspace and contact the IRB office with any urgent events. Please visit the Human Research Protection Program (HRPP) website to obtain more information, including reporting timelines. Personnel Changes: After determination of the exempt status, the PI is responsible for maintaining records of personnel changes and appropriate training. The PI is not required to notify the IRB of personnel changes on exempt research. However, he or she may wish to submit personnel changes to the IRB for recordkeeping purposes (e.g. communication with the Graduate School) and may submit such requests by submitting a Modification request. If there is a change in PI, the new PI must confirm acceptance of the PI Assurance form and the previous PI must submit the Supplemental Form to Change the Principal Investigator with the Modification request (available at hrpp.msu.edu). Closure: Investigators are not required to notify the IRB when the research study can be closed. However, the PI can choose to notify the IRB when the study can be closed and is especially recommended when the PI leaves the university. Closure indicates that research activities with human subjects are no longer ongoing, have stopped, and are complete. Human research activities are complete when investigators are no longer obtaining information or biospecimens about a living person through interaction or intervention with the individual, obtaining identifiable private information or identifiable biospecimens about a living person, and/or using, studying, analyzing, or generating identifiable private information or identifiable biospecimens about a living person. For More Information: See HRPP Manual, including Section 8-1, Exemptions (available at hrpp.msu.edu). Contact Information: If we can be of further assistance or if you have questions, please contact us at 517-355-2180 or via email at IRB@msu.edu. Please visit hrpp.msu.edu to access the HRPP Manual, templates, etc. Exemption Category. The full regulatory text from 45 CFR 46.104(d) for the exempt research categories is included below. 1234 Exempt 1. Research, conducted in established or commonly accepted educational settings, that specifically involves normal educational practices that are not likely to adversely impact students' opportunity to learn required educational content or the assessment of educators who provide instruction. This includes most research on regular and special education instructional strategies, and research on the effectiveness of or the comparison among instructional techniques, curricula, or classroom management methods. 122 Exempt 2. Research that only includes interactions involving educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures, or observation of public behavior (including visual or auditory recording) if at least one of the following criteria is met: The information obtained is recorded by the investigator in such a manner that (i) the identity of the human subjects cannot readily be ascertained, directly or through identifiers linked to the subjects; Any disclosure of the human subjects' responses outside the research would not (ii) reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, educational advancement, or reputation; or The information obtained is recorded by the investigator in such a manner that (iii) the identity of the human subjects can readily be ascertained, directly or through identifiers linked to the subjects, and an IRB conducts a limited IRB review to make the determination required by 45 CFR 46.111(a)(7). Exempt 3. (i) Research involving benign behavioral interventions in conjunction with the collection of information from an adult subject through verbal or written responses (including data entry) or audiovisual recording if the subject prospectively agrees to the intervention and information collection and at least one of the following criteria is met: The information obtained is recorded by the investigator in such a manner that (A) the identity of the human subjects cannot readily be ascertained, directly or through identifiers linked to the subjects; Any disclosure of the human subjects' responses outside the research would not (B) reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, educational advancement, or reputation; or The information obtained is recorded by the investigator in such a manner that (C) the identity of the human subjects can readily be ascertained, directly or through identifiers linked to the subjects, and an IRB conducts a limited IRB review to make the determination required by 45 CFR 46.111(a)(7). (ii) For the purpose of this provision, benign behavioral interventions are brief in duration, harmless, painless, not physically invasive, not likely to have a significant adverse lasting impact on the subjects, and the investigator has no reason to think the subjects will find the interventions offensive or embarrassing. Provided all such criteria are met, examples of such benign behavioral interventions would include having the subjects play an online game, having them solve puzzles under various noise conditions, or having them decide how to allocate a nominal amount of received cash between themselves and someone else. (iii) If the research involves deceiving the subjects regarding the nature or purposes of the research, this exemption is not applicable unless the subject authorizes the deception through a prospective agreement to participate in research in circumstances in which the subject is informed that he or she will be unaware of or misled regarding the nature or purposes of the research. 123 Exempt 4. Secondary research for which consent is not required: Secondary research uses of identifiable private information or identifiable biospecimens, if at least one of the following criteria is met: The identifiable private information or identifiable biospecimens are publicly (i) available; (ii) Information, which may include information about biospecimens, is recorded by the investigator in such a manner that the identity of the human subjects cannot readily be ascertained directly or through identifiers linked to the subjects, the investigator does not contact the subjects, and the investigator will not re-identify subjects; The research involves only information collection and analysis involving the (iii) investigator's use of identifiable health information when that use is regulated under 45 CFR parts 160 and 164, subparts A and E, for the purposes of ``health care operations'' or ``research'' as those terms are defined at 45 CFR 164.501 or for ``public health activities and purposes'' as described under 45 CFR 164.512(b); or The research is conducted by, or on behalf of, a Federal department or agency (iv) using government-generated or government-collected information obtained for non- research activities, if the research generates identifiable private information that is or will be maintained on information technology that is subject to and in compliance with section 208(b) of the E-Government Act of 2002, 44 U.S.C. 3501 note, if all of the identifiable private information collected, used, or generated as part of the activity will be maintained in systems of records subject to the Privacy Act of 1974, 5 U.S.C. 552a, and, if applicable, the information used in the research was collected subject to the Paperwork Reduction Act of 1995, 44 U.S.C. 3501 et seq. Exempt 5. Research and demonstration projects that are conducted or supported by a Federal department or agency, or otherwise subject to the approval of department or agency heads (or the approval of the heads of bureaus or other subordinate agencies that have been delegated authority to conduct the research and demonstration projects), and that are designed to study, evaluate, improve, or otherwise examine public benefit or service programs, including procedures for obtaining benefits or services under those programs, possible changes in or alternatives to those programs or procedures, or possible changes in methods or levels of payment for benefits or services under those programs. Such projects include, but are not limited to, internal studies by Federal employees, and studies under contracts or consulting arrangements, cooperative agreements, or grants. Exempt projects also include waivers of otherwise mandatory requirements using authorities such as sections 1115 and 1115A of the Social Security Act, as amended. (i) Each Federal department or agency conducting or supporting the research and demonstration projects must establish, on a publicly accessible Federal Web site or in such other manner as the department or agency head may determine, a list of the research and demonstration projects that the Federal department or agency conducts or supports under this provision. The research or demonstration project must be published on this list prior to commencing the research involving human subjects. Exempt 6. Taste and food quality evaluation and consumer acceptance studies: (i) If wholesome foods without additives are consumed, or (ii) If a food is consumed that contains a food ingredient at or below the level and for a use found to be safe, or agricultural chemical or 124 environmental contaminant at or below the level found to be safe, by the Food and Drug Administration or approved by the Environmental Protection Agency or the Food Safety and Inspection Service of the U.S. Department of Agriculture. Exempt 7. Storage or maintenance for secondary research for which broad consent is required: Storage or maintenance of identifiable private information or identifiable biospecimens for potential secondary research use if an IRB conducts a limited IRB review and makes the determinations required by 45 CFR 46.111(a)(8). Exempt 8. Secondary research for which broad consent is required: Research involving the use of identifiable private information or identifiable biospecimens for secondary research use, if the following criteria are met: (i) Broad consent for the storage, maintenance, and secondary research use of the identifiable private information or identifiable biospecimens was obtained in accordance with 45 CFR 46.116(a)(1) through (4), (a)(6), and (d); Documentation of informed consent or waiver of documentation of consent was (ii) obtained in accordance with 45 CFR 46.117; (iii) An IRB conducts a limited IRB review and makes the determination required by 45 CFR 46.111(a)(7) and makes the determination that the research to be conducted is within the scope of the broad consent referenced in paragraph (d)(8)(i) of this section; and (iv) The investigator does not include returning individual research results to subjects as part of the study plan. This provision does not prevent an investigator from abiding by any legal requirements to return individual research results. 1Exempt categories (1), (2), (3), (4), (5), (7), and (8) cannot be applied to activities that are FDA regulated. Each of the exemptions at this section may be applied to research subject to subpart B (Additional 2 Protections for Pregnant Women, Human Fetuses and Neonates Involved in Research) if the conditions of the exemption are met. The exemptions at this section do not apply to research subject to subpart C (Additional Protections for 3 Research Involving Prisoners), except for research aimed at involving a broader subject population that only incidentally includes prisoners. 4 Exemptions (1), (4), (5), (6), (7), and (8) of this section may be applied to research subject to subpart D (Additional Protections for Children Involved as Subjects in Research) if the conditions of the exemption are met. Exempt (2)(i) and (ii) only may apply to research subject to subpart D involving educational tests or the observation of public behavior when the investigator(s) do not participate in the activities being observed. Exempt (2)(iii) may not be applied to research subject to subpart D. 125 APPENDIX B: RECRUITMENT EMAIL Hello ______, We are reaching out to public school principals and administrators in the state of Michigan to learn more about their opinions related to evidence-based practices in the classroom. Your thoughts matter to us and we really value your time. The survey below will only take a few minutes and to say thank you, we will send a $10 Amazon e-gift card your way once the survey is completed. We hope you will take the time to help us learn more about your schools and what might be done to improve educational outcomes for the students in your area. Thank you so much for your help and we hope to hear from you soon. Please click the link below to take the survey and do not hesitate to let me know if there are any questions about the survey, your participation, or our work here in the Michigan State Department of Psychology. Best, Brian Brutzman, M.A. BCBA Ph.D. Student Social/Personality Psychology Michigan State University 126 APPENDIX C: SURVEY INSTRUMENT FOR ISD ADMINISTRATORS ABA in Michigan Schools - For ISD Admin Start of Block: Screening ID_Screen This survey is intended for ${m://FirstName} ${m://LastName} at ${m://ExternalDataReference}. Are you the person listed above? o Yes (1) o No (2) End of Block: Screening Start of Block: Consent Display This Question: If This survey is intended for ${m://FirstName} ${m://LastName} at ${m://ExternalDataReference}. ... = Yes Consent Consent ABA in Michigan Schools Survey What is the purpose of this study? The purpose of this study is to assess Michigan public school educators’ awareness of applied behavior analysis (ABA) and its application in classrooms for students with autism and other developmental disabilities. A second aim of this study is to assess Michigan public school educators’ potential readiness to adopt specialized applied behavior analytic classrooms in their schools, intermediate school districts, or educational service agencies. This study is being conducted by researchers at Michigan State University and is funded by the Michigan State University College of Social Sciences. What am I being asked to do? The survey includes questions about your level of awareness of applied behavior analysis and its successful application in classroom settings, potential readiness to adopt specialized ABA classrooms, and some basic questions about you and your work history. It will take between 20 – 30 minutes to complete this survey. Your individual answers are confidential and will only be seen by members of the research team. They will not be shared with others in your school district or state administrators, and will not be used to evaluate your job performance. Your participation is voluntary, and you can stop the survey or skip questions at any time if you feel uncomfortable. 127 What will I receive if I complete this survey? You will receive an Amazon.com e-gift card in the amount of $10 for your participation if you are eligible for this study and complete the survey. Additionally, we will share anonymous findings from the study with you, other participating educators, and Michigan educational stakeholders. What are the potential risks and benefits of my participation? The main risk of completing this online survey is the possibility that certain questions might make you uncomfortable. You will not directly benefit from your participation in this study but your answers may provide insight into how we can improve Michigan educators’ awareness of and experiences applying ABA. Who can I contact if I have questions? If you have any questions about the study, you can contact Brian Brutzman at brutzman@msu.edu or Dr. Jennifer Watling Neal at jneal@msu.edu. Are you willing to participate? By completing this survey, you are voluntarily agreeing to participate in this research study. o Yes (1) o No (2) End of Block: Consent Start of Block: ABA Awareness ASD_Y/N Are there any students who attend your intermediate school district or educational service agency that meet criteria for autism spectrum disorder (ASD)? o Yes (1) o No (2) Display This Question: If Are there any students who attend your intermediate school district or educational service agency... = Yes 128 ASD_NUM Approximately how many students who attend your intermediate school district or educational service agency meet criteria for autism spectrum disorder (ASD)? (Please enter a whole number in the box below) ________________________________________________________________ Display This Question: If Are there any students who attend your intermediate school district or educational service agency... = Yes ASD_Percent Approximately what percentage of your entire intermediate school district or educational service agency's student population meets criteria for autism spectrum disorder (ASD)? (Please enter a whole number in the box below without a "%" or the word "percent.") ________________________________________________________________ ABA_Aware_1 A notable treatment approach for people with autism spectrum disorder is called applied behavior analysis (ABA). ABA has become widely accepted among healthcare professionals and used in many schools and treatment clinics. ABA encourages positive behaviors and discourages negative behaviors to improve a variety of skills. The child’s progress is tracked and measured (source: https://www.cdc.gov/ncbddd/autism/treatment.html). Applied behavior analytic services are typically delivered by a board certified behavior analyst (BCBA) with or without the assistance of a board certified assistant behavior analyst (BCaBA) or a registered behavior technician (RBT). Prior to this survey, have you ever heard of applied behavior analysis (ABA)? o Yes (1) o No (2) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes 129 ABA_Aware_2 Which contexts for ABA are you aware of, either in Michigan or elsewhere? (Please select all that apply) In-home therapy for children on the autism spectrum (1) Clinic-based therapy for children on the autism spectrum (2) School-based ABA consulting for students with challenging behavior (3) Specialized ABA classrooms in public schools for students with autism and other developmental disabilities (4) ▢ ▢ ▢ ▢ ▢ Specialized private schools that utilize ABA for curriculum development, skill acquisition, and behavior management (5) ▢ Other (6) Display This Question: If Which contexts for ABA are you aware of, either in Michigan or elsewhere? (Please select all that... = Other ABA_Aware_3 Please describe the other context(s) for ABA that you are aware of. ________________________________________________________________ Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes 130 MI_Licensure To the best of your knowledge, does the state of Michigan currently offer licensure for professionals in the field of applied behavior analysis? o Yes (1) o No (2) o Not Sure (3) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes likelihood_to_seek If ANY student in your intermediate school district or educational service agency was experiencing a behavioral problem (either as part of a formal autism diagnosis or not), how likely would you be to seek out the services of a credentialed professional in applied behavior analysis? o Extremely likely (1) o Somewhat likely (2) o Neither likely nor unlikely (3) o Somewhat unlikely (4) o Extremely unlikely (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes 131 ease_to_seek If a student required the services of a credentialed professional in applied behavior analysis, and the intermediate school district or educational service agency was involved in procuring those services, how easy would it be to obtain those services? o Extremely easy (1) o Somewhat easy (2) o Neither easy nor difficult (3) o Somewhat difficult (4) o Extremely difficult (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes Consult_exp In your role as an educator or administrator, have you ever employed or consulted with a credentialed professional in ABA (e.g., in IEP meetings, to draft a behavior plan, or to complete a functional behavior assessment), such as a board certified behavior analyst (BCBA) or a board certified behavior analyst-doctoral level (BCBA-D)? o Yes (1) o No (2) o Not Sure (3) Display This Question: If In your role as an educator or administrator, have you ever employed or consulted with a credenti... = Yes 132 Consult_sat How satisfied were you with the services that the ABA professional provided? o Extremely satisfied (1) o Somewhat satisfied (2) o Neither satisfied nor dissatisfied (3) o Somewhat dissatisfied (4) o Extremely dissatisfied (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes MI_Aware Are you aware of any public school, intermediate school district, or educational service agency within the state of Michigan that utilizes specialized, applied behavior analytic classrooms to meet the needs of students with autism or other developmental disabilities? (This type of classroom is typically led by a teacher with advanced behavioral training, staffed with paraprofessionals who also receive additional training, and may or may not retain the services of a professional behavioral consultant.) o Yes (1) o No (2) Display This Question: If Are you aware of any public school, intermediate school district, or educational service agency w... = Yes MI_Aware_Loc Please enter the name of the Michigan School District(s) (if known) that utilizes one or more specialized ABA classrooms. ________________________________________________________________ 133 Display This Question: If Are you aware of any public school, intermediate school district, or educational service agency w... = Yes MI_Aware_Eff In your estimation, how effective are these classroom(s) in meeting the needs of special education students? o Extremely effective (1) o Very effective (2) o Moderately effective (3) o Slightly effective (4) o Not effective at all (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes Out_Aware Are you aware of any public school, intermediate school district, or educational service agency outside the state of Michigan that utilizes specialized, applied behavior analytic classrooms to meet the needs of students with autism or other developmental disabilities? (This type of classroom is typically led by a teacher with advanced behavioral training, staffed with paraprofessionals who also receive additional training, and may or may not retain the services of a professional behavioral consultant.) o Yes (1) o No (2) Display This Question: If Are you aware of any public school, intermediate school district, or educational service agency o... = Yes 134 Out_Aware_Eff In your estimation, how effective are these classroom(s) in meeting the needs of special education students? o Extremely effective (1) o Very effective (2) o Moderately effective (3) o Slightly effective (4) o Not effective at all (5) Pilot_Att If the state of Michigan asked your intermediate school district or educational service agency to pilot a specialized, applied behavior analytic classroom to meet the needs of students with special needs, and all necessary resources and support were provided, how would you feel about this change? o Extremely positive (1) o Somewhat positive (2) o Neither positive nor negative (3) o Somewhat negative (4) o Extremely negative (5) End of Block: ABA Awareness Start of Block: EBPAS 135 EBPAS For the next set of questions, please rate the extent to which the following statements describe you. Please fill in all answer choices as best you can. Not at all (1) To a Slight Extent (2) To a Moderate Extent (3) To a Great Extent (4) To a Very Great Extent (5) I like to use new types of interventions to help my students. (1) I am willing to try new types of interventions even if I have to follow an instructional manual. (2) I know better than academic researchers how to provide instruction for my students. (3) I am willing to use new and different types of interventions developed by researchers. (4) Research-based interventions are not useful in a practical sense. (5) Teaching experience is more important than using manualized interventions. (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 136 I would not use manualized interventions. (7) I would try a new intervention even if it were very different from what I am used to doing. (8) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was intuitively appealing? (9) If you received training in an intervention that was new to you, how likely would you be to adopt it if it “made sense” to you? (10) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was required by your supervisor? (11) o o o o o o o o o o o o o o o o o o o o o o o o o 137 If you received training in an intervention that was new to you, how likely would you be to adopt it if it was required by your ISD/ESA? (12) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was required by your state? (13) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was being used by colleagues who were happy with it? (14) If you received training in an intervention that was new to you, how likely would you be to adopt it if you felt you had enough training to use it correctly? (15) End of Block: EBPAS Start of Block: ORIC o o o o o o o o o o o o o o o o o o o o 138 ORIC For the next several questions, please think of the following scenario: Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your intermediate school district or educational service agency is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your intermediate school district or educational service agency. Please fill in all answer choices as best you can. With this scenario in mind, would you say that: Disagree (1) Somewhat Disagree (2) Neither Agree nor Disagree (3) Somewhat Agree (4) Agree (5) Teachers, administrators, and staff who work in my ISD/ESA would be committed to implementing this change. (1) Teachers, administrators, and staff who work in my ISD/ESA would want to implement this change. (2) Teachers, administrators, and staff who work in my ISD/ESA would be determined to implement this change. (3) o o o o o o o o o o o o o o o 139 Teachers, administrators, and staff who work in my ISD/ESA would be motivated to implement this change. (4) Teachers, administrators, and staff who work in my ISD/ESA would feel confident that they can handle the challenges that might arise in implementing this change. (5) Teachers, administrators, and staff who work in my ISD/ESA would feel confident that they can keep track of progress in implementing this change. (6) Teachers, administrators, and staff who work in my ISD/ESA would feel confident that they can coordinate tasks so that implementation goes smoothly. (7) o o o o o o o o o o o o o o o o o o o o 140 Teachers, administrators, and staff who work in my ISD/ESA would feel confident that the organization can support people as they adjust to this change. (8) Teachers, administrators, and staff who work in my ISD/ESA would feel confident that they can manage the politics of implementing this change. (9) o o o o o o o o o o End of Block: ORIC Start of Block: Possible Contextual Factors/ URPI-R System Climate + Family-School Context/URPI For the next several questions, please CONTINUE to think of the following scenario: *For the purposes of this survey, the term "intervention" refers to the establishment of an applied behavior analytic classroom. Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your intermediate school district or educational service agency is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your intermediate school district or educational service agency. Please fill in all answer choices as best you can. 141 With this scenario in mind, would you say that: Strongly Disagree (1) Disagree (2) Somewhat Disagree (3) Somewhat Agree (4) Agree (5) Strongly Agree (6) Use of this intervention would be consistent with the mission of my ISD/ESA (1) My administration would be supportive of my use of this intervention (2) These intervention procedures are consistent with the way things are done in my system (3) My work environment is conducive to implementation of an intervention like this one (4) Implementation of this intervention is well matched to what is expected in my job (5) Parental collaboration is required in order to use this intervention (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 142 A positive home–school relationship is needed to implement this intervention (7) Regular home– school communication is needed to implement intervention procedures (8) o o o o o o o o o o o o End of Block: Possible Contextual Factors/ URPI-R System Climate + Family-School Start of Block: Change Valence/ URPI-R Acceptability Valence/Accept For the next several questions, please CONTINUE to think of the following scenario: *For the purposes of this survey, the term "intervention" refers to the establishment of an applied behavior analytic classroom. Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your intermediate school district or educational service agency is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your intermediate school district or educational service agency. Please fill in all answer choices as best you can. With this scenario in mind, would you say that: Strongly Disagree (1) Disagree (2) Somewhat Disagree (3) Somewhat Agree (4) Agree (5) Strongly Agree (6) 143 This intervention is a good way to handle students' behavior problems (1) I would implement this intervention with a good deal of enthusiasm (2) This intervention would not be disruptive to other students (3) The intervention procedures easily fit in with my current practices (4) I would not be interested in implementing this intervention (5) I would have positive attitudes about implementing this intervention (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 144 The intervention is a fair way to handle students' behavior problems (7) This intervention is an effective choice for addressing a variety of problems (8) I would be committed to carrying out this intervention (9) o o o o o o o o o o o o o o o o o o End of Block: Change Valence/ URPI-R Acceptability Start of Block: Informational Assessment/ URPI-R Understanding + Feasibility + System Support Info_Ass/Und_Feas For the next several questions, please CONTINUE to think of the following scenario: *For the purposes of this survey, the term "intervention" refers to the establishment of an applied behavior analytic classroom. Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your intermediate school district or educational service agency is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your intermediate school district or educational service agency. Please fill in all answer choices as best you can. 145 With this scenario in mind, would you say that: Strongly Disagree (1) Disagree (2) Somewhat Disagree (3) Somewhat Agree (4) Agree (5) Strongly Agree (6) I am knowledgeable about the intervention procedures (1) I understand the procedures of this intervention (2) I understand how to use this intervention (3) The total time required to implement the intervention procedures would be manageable (4) Material resources needed for this intervention are reasonable (5) This intervention is too complex to carry out accurately (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 146 I would be able to allocate my time to implement this intervention (7) The amount of time required for record keeping would be reasonable (8) Preparation of materials needed for this intervention would be minimal (9) I would require additional professional development in order to implement this intervention (10) I would need consultative support to implement this intervention (11) I would need additional resources to carry out this intervention (12) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o End of Block: Informational Assessment/ URPI-R Understanding + Feasibility + System Support Start of Block: Demographics 147 years_school The last few questions ask about you and your work experience. When did you begin working in your current district? ▼ 2020 (1) ... 1951 (70) years_principal When did you start in your current position? ▼ 2020 (1) ... 1951 (70) district_num In how many different districts, including this one, have you worked in the past? ________________________________________________________________ years_educator How many years of experience as an educator do you have? ________________________________________________________________ birth_year When were you born? ▼ 2001 (1) ... 1920 (82) 148 gender How do you describe your gender? o Male (1) o Female (2) o Non-Binary (3) race How do you describe your race or ethnic background (mark all that apply)? ▢ ▢ ▢ ▢ ▢ ▢ White (1) Black or African American (2) American Indian or Alaska Native (3) Asian (4) Native Hawaiian or Pacific Islander (5) Other (6) latinx Do you identify as Hispanic or Latino/Latina/LatinX? o Yes (1) o No (2) 149 highest_degree What is your highest degree? o High school graduate (1) o 2 year degree (2) o 4 year degree (3) o Master's Degree (4) o Doctorate (Ed.D., Ph.D.) (5) degree_year When did you complete your most recent degree? ▼ 2020 (1) ... 1951 (70) End of Block: Demographics Start of Block: Closing open_end Please write anything else you would like us to know about evidence-based interventions for special education at your intermediate school district or educational service agency. ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ 150 email In order to receive your Amazon electronic gift card, please enter the email address where you would like it to be sent. It should arrive within two weeks of the completion of this survey. Please contact Brian Brutzman at brutzman@msu.edu of there is any issue with receiving your compensation. ________________________________________________________________ End of Block: Closing 151 APPENDIX D: SURVEY INSTRUMENT FOR SCHOOL PRINCIPALS ABA in Michigan Schools - For LEA Principals Start of Block: Screening ID_Screen This survey is intended for ${m://FirstName} ${m://LastName} at ${e://Field/ExternalDataReference}. Are you the person listed above? o Yes (1) o No (2) End of Block: Screening Start of Block: Consent Display This Question: If This survey is intended for ${m://FirstName} ${m://LastName} at ... = Yes Consent Consent ABA in Michigan Schools Survey What is the purpose of this study? The purpose of this study is to assess Michigan public school educators’ awareness of applied behavior analysis (ABA) and its application in classrooms for students with autism and other developmental disabilities. A second aim of this study is to assess Michigan public school educators’ potential readiness to adopt specialized applied behavior analytic classrooms in their schools. This study is being conducted by researchers at Michigan State University and is funded by the Michigan State University College of Social Sciences. What am I being asked to do? The survey includes questions about your level of awareness of applied behavior analysis and its successful application in classroom settings, potential readiness to adopt specialized ABA classrooms, and some basic questions about you and your work history. It will take between 20 – 30 minutes to complete this survey. Your individual answers are confidential and will only be seen by members of the research team. They will not be shared with others in your school district or state administrators, and will not be used to evaluate your job performance. Your participation is voluntary, and you can stop the survey or skip questions at any time if you feel uncomfortable. 152 What will I receive if I complete this survey? You will receive an Amazon.com e-gift card in the amount of $10 for your participation if you are eligible for this study and complete the survey. Additionally, we will share anonymous findings from the study with you, other participating educators, and Michigan educational stakeholders. What are the potential risks and benefits of my participation? The main risk of completing this online survey is the possibility that certain questions might make you uncomfortable. You will not directly benefit from your participation in this study but your answers may provide insight into how we can improve Michigan educators’ awareness of and experiences applying ABA in classrooms. Who can I contact if I have questions? If you have any questions about the study, you can contact Brian Brutzman at brutzman@msu.edu or Dr. Jennifer Watling Neal at jneal@msu.edu. Are you willing to participate? By completing this survey, you are voluntarily agreeing to participate in this research study. o Yes (1) o No (2) End of Block: Consent Start of Block: ABA Awareness ASD_Y/N Are there any students who attend your school that meet criteria for autism spectrum disorder (ASD)? o Yes (1) o No (2) Display This Question: If Are there any students who attend your school that meet criteria for autism spectrum disorder (ASD)? = Yes ASD_NUM Approximately how many students who attend your school meet criteria for autism spectrum disorder (ASD)? (Please enter a whole number in the box below) ________________________________________________________________ 153 Display This Question: If Are there any students who attend your school that meet criteria for autism spectrum disorder (ASD)? = Yes ASD_Percent Approximately what percentage of your entire school's student population meets criteria for autism spectrum disorder (ASD)? (Please enter a whole number in the box below without a "%" or the word "percent.") ________________________________________________________________ ABA_Aware_1 A notable treatment approach for people with autism spectrum disorder is called applied behavior analysis (ABA). ABA has become widely accepted among healthcare professionals and used in many schools and treatment clinics. ABA encourages positive behaviors and discourages negative behaviors to improve a variety of skills. The child’s progress is tracked and measured (source: https://www.cdc.gov/ncbddd/autism/treatment.html). Applied behavior analytic services are typically delivered by a board certified behavior analyst (BCBA) with or without the assistance of a board certified assistant behavior analyst (BCaBA) or a registered behavior technician (RBT). Prior to this survey, have you ever heard of applied behavior analysis (ABA)? o Yes (1) o No (2) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes 154 ABA_Aware_2 Which contexts for ABA are you aware of, either in Michigan or elsewhere? (Please select all that apply) In-home therapy for children on the autism spectrum (1) Clinic-based therapy for children on the autism spectrum (2) School-based ABA consulting for students with challenging behavior (3) Specialized ABA classrooms in public schools for students with autism and other developmental disabilities (4) ▢ ▢ ▢ ▢ ▢ Specialized private schools that utilize ABA for curriculum development, skill acquisition, and behavior management (5) ▢ Other (6) Display This Question: If Which contexts for ABA are you aware of, either in Michigan or elsewhere? (Please select all that... = Other ABA_Aware_3 Please describe the other context(s) for ABA that you are aware of. ________________________________________________________________ Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes 155 MI_Licensure To the best of your knowledge, does the state of Michigan currently offer licensure for professionals in the field of applied behavior analysis? o Yes (1) o No (2) o Not Sure (3) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes likelihood_to_seek If ANY student in your school was experiencing a behavioral problem (either as part of a formal autism diagnosis or not), how likely would you be to seek out the services of a credentialed professional in applied behavior analysis? o Extremely likely (1) o Somewhat likely (2) o Neither likely nor unlikely (3) o Somewhat unlikely (4) o Extremely unlikely (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes 156 ease_to_seek If a student required the services of a credentialed professional in applied behavior analysis, and the school was involved in procuring those services, how easy would it be to obtain those services? o Extremely easy (1) o Somewhat easy (2) o Neither easy nor difficult (3) o Somewhat difficult (4) o Extremely difficult (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes Consult_exp In your role as an educator or school administrator, have you ever employed or consulted with a credentialed professional in ABA (e.g., in IEP meetings, to draft a behavior plan, or to complete a functional behavior assessment), such as a board certified behavior analyst (BCBA) or a board certified behavior analyst-doctoral level (BCBA-D)? o Yes (1) o No (2) o Not Sure (3) Display This Question: If In your role as an educator or school administrator, have you ever employed or consulted with a c... = Yes 157 Consult_sat How satisfied were you with the services that the ABA professional provided? o Extremely satisfied (1) o Somewhat satisfied (2) o Neither satisfied nor dissatisfied (3) o Somewhat dissatisfied (4) o Extremely dissatisfied (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes MI_Aware Are you aware of any public school within the state of Michigan that utilizes specialized, applied behavior analytic classrooms to meet the needs of students on the autism spectrum or other developmental disabilities? (This type of classroom is typically led by a teacher with advanced behavioral training, staffed with paraprofessionals who also receive additional training, and may or may not retain the services of a professional behavioral consultant.) o Yes (1) o No (2) Display This Question: If Are you aware of any public school within the state of Michigan that utilizes specialized, applie... = Yes MI_Aware_Loc Please enter the name of the Michigan School District(s) (if known) that utilizes one or more specialized ABA classrooms. ________________________________________________________________ 158 Display This Question: If Are you aware of any public school within the state of Michigan that utilizes specialized, applie... = Yes MI_Aware_Eff In your estimation, how effective are these classroom(s) in meeting the needs of special education students? o Extremely effective (1) o Very effective (2) o Moderately effective (3) o Slightly effective (4) o Not effective at all (5) Display This Question: If A notable treatment approach for people with autism spectrum disorder is called applied behavior... = Yes Out_Aware Are you aware of any public school outside the state of Michigan that utilizes specialized, applied behavior analytic classrooms to meet the needs of students on the autism spectrum or other developmental disabilities? (This type of classroom is typically led by a teacher with advanced behavioral training, staffed with paraprofessionals who also receive additional training, and may or may not retain the services of a professional behavioral consultant.) o Yes (1) o No (2) Display This Question: If Are you aware of any public school outside the state of Michigan that utilizes specialized, appli... = Yes 159 Out_Aware_Eff In your estimation, how effective are these classroom(s) in meeting the needs of special education students? o Extremely effective (1) o Very effective (2) o Moderately effective (3) o Slightly effective (4) o Not effective at all (5) Pilot_Att If the state of Michigan asked your school to pilot a specialized, applied behavior analytic classroom to meet the needs of students with special needs, and all necessary resources and support were provided, how would you feel about this change? o Extremely positive (1) o Somewhat positive (2) o Neither positive nor negative (3) o Somewhat negative (4) o Extremely negative (5) End of Block: ABA Awareness Start of Block: EBPAS EBPAS For the next set of questions, please rate the extent to which the following statements describe you. 160 Please fill in all answer choices as best you can. Not at all (1) To a Slight Extent (2) To a Moderate Extent (3) To a Great Extent (4) To a Very Great Extent (5) I like to use new types of interventions to help my students. (1) I am willing to try new types of interventions even if I have to follow an instructional manual. (2) I know better than academic researchers how to provide instruction for my students. (3) I am willing to use new and different types of interventions developed by researchers. (4) Research-based interventions are not useful in a practical sense. (5) Teaching experience is more important than using manualized interventions. (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 161 I would not use manualized interventions. (7) I would try a new intervention even if it were very different from what I am used to doing. (8) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was intuitively appealing? (9) If you received training in an intervention that was new to you, how likely would you be to adopt it if it “made sense” to you? (10) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was required by your supervisor? (11) o o o o o o o o o o o o o o o o o o o o o o o o o 162 If you received training in an intervention that was new to you, how likely would you be to adopt it if it was required by your district? (12) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was required by your state? (13) If you received training in an intervention that was new to you, how likely would you be to adopt it if it was being used by colleagues who were happy with it? (14) If you received training in an intervention that was new to you, how likely would you be to adopt it if you felt you had enough training to use it correctly? (15) o o o o o o o o o o o o o o o o o o o o 163 ORIC For the next several questions, please think of the following scenario: Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your school is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your school. Please fill in all answer choices as best you can. With this scenario in mind, would you say that: 164 Disagree (1) Somewhat Disagree (2) Neither Agree nor Disagree (3) Somewhat Agree (4) Agree (5) o o o o o o o o o o o o o o o o o o o o Teachers, administrators, and staff who work in my school would be committed to implementing this change. (1) Teachers, administrators, and staff who work in my school would want to implement this change. (2) Teachers, administrators, and staff who work in my school would be determined to implement this change. (3) Teachers, administrators, and staff who work in my school would be motivated to implement this change. (4) 165 Teachers, administrators, and staff who work in my school would feel confident that they can handle the challenges that might arise in implementing this change. (5) Teachers, administrators, and staff who work in my school would feel confident that they can keep track of progress in implementing this change. (6) Teachers, administrators, and staff who work in my school would feel confident that they can coordinate tasks so that implementation goes smoothly. (7) Teachers, administrators, and staff who work in my school would feel confident that the organization can support people as they adjust to this change. (8) o o o o o o o o o o o o o o o o o o o o 166 Teachers, administrators, and staff who work in my school would feel confident that they can manage the politics of implementing this change. (9) o o o o o Context/URPI For the next several questions, please CONTINUE to think of the following scenario: *For the purposes of this survey, the term "intervention" refers to the establishment of an applied behavior analytic classroom. Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your school is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your school. Please fill in all answer choices as best you can. With this scenario in mind, would you say that: 167 Strongly Disagree (1) Disagree (2) Somewhat Disagree (3) Somewhat Agree (4) Agree (5) Strongly Agree (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o Use of this intervention would be consistent with the mission of my school (1) My administrator would be supportive of my use of this intervention (2) These intervention procedures are consistent with the way things are done in my system (3) My work environment is conducive to implementation of an intervention like this one (4) Implementation of this intervention is well matched to what is expected in my job (5) Parental collaboration is required in order to use this intervention (6) 168 A positive home–school relationship is needed to implement this intervention (7) Regular home– school communication is needed to implement intervention procedures (8) o o o o o o o o o o o o End of Block: Possible Contextual Factors/ URPI-R System Climate + Family-School Start of Block: Change Valence/ URPI-R Acceptability Valence/Accept For the next several questions, please CONTINUE to think of the following scenario: *For the purposes of this survey, the term "intervention" refers to the establishment of an applied behavior analytic classroom. Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your school is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your school. Please fill in all answer choices as best you can. With this scenario in mind, would you say that: 169 Strongly Disagree (1) Disagree (2) Somewhat Disagree (3) Somewhat Agree (4) Agree (5) Strongly Agree (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o This intervention is a good way to handle students' behavior problems (1) I would implement this intervention with a good deal of enthusiasm (2) This intervention would not be disruptive to other students (3) The intervention procedures easily fit in with my current practices (4) I would not be interested in implementing this intervention (5) 170 I would have positive attitudes about implementing this intervention (6) The intervention is a fair way to handle students' behavior problems (7) This intervention is an effective choice for addressing a variety of problems (8) I would be committed to carrying out this intervention (9) o o o o o o o o o o o o o o o o o o o o o o o o End of Block: Change Valence/ URPI-R Acceptability Start of Block: Informational Assessment/ URPI-R Understanding + Feasibility + System Support Info_Ass/Und_Feas For the next several questions, please CONTINUE to think of the following scenario: *For the purposes of this survey, the term "intervention" refers to the establishment of an applied behavior analytic classroom. Suppose that under typical circumstances (i.e. a future time when the COVID-19 pandemic is no longer a threat to community health), your school is asked to establish a new, specialized applied behavior analytic classroom for students with autism spectrum disorder. This classroom would utilize behavioral science to help students acquire and retain skills, as well as minimize problematic behavior. Appropriate training for teaching staff would be provided, as would the consultation of outside behavioral experts. 171 Please note: None of the training, consultation, or material resources for this hypothetical new classroom will be paid for by the school's normal budget. Assume that costs for all of these services and materials will be covered by additional funds provided to your school. Please fill in all answer choices as best you can. With this scenario in mind, would you say that: Strongly Disagree (1) Disagree (2) Somewhat Disagree (3) Somewhat Agree (4) Agree (5) Strongly Agree (6) I am knowledgeable about the intervention procedures (1) I understand the procedures of this intervention (2) I understand how to use this intervention (3) The total time required to implement the intervention procedures would be manageable (4) Material resources needed for this intervention are reasonable (5) This intervention is too complex to carry out accurately (6) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o 172 I would be able to allocate my time to implement this intervention (7) The amount of time required for record keeping would be reasonable (8) Preparation of materials needed for this intervention would be minimal (9) I would require additional professional development in order to implement this intervention (10) I would need consultative support to implement this intervention (11) I would need additional resources to carry out this intervention (12) o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o End of Block: Informational Assessment/ URPI-R Understanding + Feasibility + System Support Start of Block: Demographics 173 years_school The last few questions ask about you and your work experience. When did you begin working in your current district? ▼ 2020 (1) ... 1951 (70) years_principal When did you start in your current position? ▼ 2020 (1) ... 1951 (70) district_num In how many different districts, including this one, have you worked in the past? ________________________________________________________________ years_educator How many years of experience as an educator do you have? ________________________________________________________________ birth_year When were you born? ▼ 2001 (1) ... 1920 (82) 174 gender How do you describe your gender? o Male (1) o Female (2) o Non-Binary (3) race How do you describe your race or ethnic background (mark all that apply)? ▢ ▢ ▢ ▢ ▢ ▢ White (1) Black or African American (2) American Indian or Alaska Native (3) Asian (4) Native Hawaiian or Pacific Islander (5) Other (6) latinx Do you identify as Hispanic or Latino/Latina/LatinX? o Yes (1) o No (2) 175 highest_degree What is your highest degree? o High school graduate (1) o 2 year degree (2) o 4 year degree (3) o Master's Degree (4) o Doctorate (Ed.D., Ph.D.) (5) degree_year When did you complete your most recent degree? ▼ 2020 (1) ... 1951 (70) End of Block: Demographics Start of Block: Closing open_end Please write anything else you would like us to know about evidence-based interventions for special education at your school. ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ 176 email In order to receive your Amazon electronic gift card, please enter the email address where you would like it to be sent. It should arrive within two weeks of the completion of this survey. Please contact Brian Brutzman at brutzman@msu.edu of there is any issue with receiving your compensation. ________________________________________________________________ End of Block: Closing 177 APPENDIX E: STANDARDIZED REGRESSION COEFFICIENTS Table 5 Chapter 3 Standardized Regression Coefficients System Climate -> Change Valence Change Valence -> Change Commitment System Climate -> Change Commitment Family-School Collab -> Change Valence Family-School Collab -> Change Commitment Estimate SE 0.04 0.07 0.07 0.04 0.04 0.76 0.37 0.34 0.05 0.03 p-value 0.00 0.00 0.00 0.18 0.57 Table 6 Chapter 4 Standardized Regression Coefficients Resource Perceptions -> Change Efficacy Task Demands -> Change Efficacy Situational Factors -> Change Efficacy System Climate -> Change Efficacy System Climate -> Task Demands System Climate -> Resource Perceptions System Climate -> Situational Factors Family-School Collab -> Change Efficacy Family-School Collab -> Task Demands Family-School Collab -> Resource Perceptions Family-School Collab -> Situational Factors Estimate SE 0.07 0.04 0.05 0.06 0.11 0.05 0.06 0.04 0.09 0.05 0.06 0.28 -0.04 -0.03 0.44 0.38 0.57 0.03 0.06 0.00 0.06 -0.2 p-value 0.00 0.51 0.62 0.00 0.00 0.00 0.67 0.18 0.97 0.29 0.00 178 APPENDIX F: SENSITIVITY ANALYSIS Table 7 Results of Sensitivity Analysis for Chapter 2 CFA Fit Indices Chi Square RMSEA SRMR CFI TLI Factor Loadings Commitment -> ORIC Item 1 Commitment -> ORIC Item 2 Commitment -> ORIC Item 3 Commitment -> ORIC Item 4 Efficacy -> ORIC Item 5 Efficacy -> ORIC Item 6 Efficacy -> ORIC Item 7 Efficacy -> ORIC Item 8 Efficacy -> ORIC Item 9 Whole Sample Prior Knowledge of ABA 118.859 0.102 0.045 0.965 0.952 0.679 0.809 0.836 0.787 0.755 0.763 0.793 0.786 0.806 100.61 0.102 0.048 0.966 0.953 0.652 0.766 0.8 0.771 0.732 0.758 0.785 0.8 0.844 Note. Given the potential lack of statistical power in subsamples, the parameter estimates above are meant to be compared with each other and not interpreted in terms of statistical significance. 179 Table 8 Results of Sensitivity Analysis for Chapter 3 SEM Fit Indices Chi Square RMSEA SRMR CFI TLI Direct Effects Climate -> Valence Collaboration -> Valence Valence -> Commitment Collaboration -> Commitment Climate -> Commitment Indirect Effects Climate -> Valence -> Commitment Collaboration -> Valence -> Commitment Whole Sample Prior Knowledge of ABA 119.646 0.057 0.068 0.964 0.946 0.688 0.05 0.337 0.022 0.275 0.231 0.017 112.467 0.06 0.07 0.961 0.941 0.722 0.019 0.238 0.027 0.362 0.172 0.004 Note. Given the potential lack of statistical power in subsamples, the parameter estimates above are meant to be compared with each other and not interpreted in terms of statistical significance. All regression coefficients are unstandardized in the table above. 180 Table 9 Results of Sensitivity Analysis for Chapter 4 SEM Fit Indices Chi Square RMSEA SRMR CFI TLI Direct Effects Climate -> Task Demands Collaboration -> Task Demands Climate -> Resource Perceptions Collaboration -> Resource Perceptions Climate -> Situational Factors Collaboration -> Situational Factors Task Demands -> Efficacy Resource Perceptions -> Efficacy Situational Factors -> Efficacy Collaboration -> Efficacy Climate -> Efficacy Indirect Effects Climate -> Task Demands -> Efficacy Climate -> Resource Perceptions -> Efficacy Climate -> Situational Factors -> Efficacy Collaboration -> Task Demands -> Efficacy Collaboration -> Resource Perceptions -> Efficacy Collaboration -> Situational Factors -> Efficacy Whole Sample Prior Knowledge of ABA 196.458 186.03 0.066 0.061 0.936 0.891 0.601 0.003 0.509 0.05 0.027 -0.204 -0.023 0.276 -0.024 0.054 0.392 -0.014 0.141 -0.001 0 0.014 0.005 0.07 0.062 0.929 0.878 0.524 -0.007 0.516 -0.003 0.014 -0.169 0 0.213 -0.047 0.095 0.395 0 0.11 -0.001 0 -0.001 0.008 Note. Given the potential lack of statistical power in subsamples, the parameter estimates above are meant to be compared with each other and not interpreted in terms of statistical significance. All regression coefficients are unstandardized in the table above. 181