A RANDOMIZED PILOT STUDY OF THE INCREDIBLE YEARS SELF-ADMINISTERED TEACHER CLASSROOM MANAGEMENT PROGRAM DURING COVID-19 By Rachel Korest A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of School Psychology –– Doctor of Philosophy 2021 ABSTRACT A RANDOMIZED PILOT STUDY OF THE INCREDIBLE YEARS SELF-ADMINISTERED TEACHER CLASSROOM MANAGEMENT PROGRAM DURING COVID-19 By Rachel Korest A pilot randomized control trial (RCT) was conducted to examine the fidelity, effectiveness, and acceptability of the Incredible Years Self-Administered Teacher Classroom Management (SATCM) program with coaching (n = 13) compared to a Book + Activity comparison group (n = 11) for 24 teachers who were identified as “at-risk” for experiencing classroom management challenges. Six remote treatments sessions were delivered biweekly over 12-weeks (~36 hours) during the COVID-19 pandemic. Throughout the process, nine participants (28%) dropped out of the study (SATCM: n = 5, Book +Activity: n = 4). Results from the research did not indicate a significant difference between groups; however, they did unexpectedly reveal similarly higher levels of improvements within fidelity, effectiveness, and acceptability for both an expensive SATCM program with coaching compared to a more cost-effective Book + Activity comparison group. Additionally, post-test interviews with SATCM group teachers indicated improvements in classroom behavior and a preference of the SATCM program over other programs. This small RCT pilot study is the first to exclusively examine the SATCM program with coaching compared to a Book + Activity comparison group for teachers who presented as at-risk for poor classroom management. Findings provide initial evidence that self-administered interventions that are systematic, sustained over time, and require active learning components may be an effective way to improve teacher outcomes. Additionally, findings emphasize the need to have accessible self-administered professional development TCM trainings as a part of a cost- effective tiered approach to support teachers who struggle with effective TCM strategies. ACKNOWLEDGEMENTS I am very thankful for the support I have received throughout my dissertation process. First, I am very grateful for the support from my advisor, Dr. John Carlson, for pushing me to develop and carry out this rigorous dissertation project and for helping me become the scholar and researcher I am today. Additionally, Dr. Carlson was instrumental in helping me seek funding for my dissertation through necessary funding sources including Michigan State University’s Dissertation Completion Fellowship. Second, I am very appreciative of the feedback and support from my committee members, Dr. Courtenay Barrett, Dr. Ignacio Acevedo, and Dr. Martin Volker. Their perspectives and ideas were invaluable for improving the quality of my dissertation. This project also would not be possible without the support of Holly Tiret, my Incredible Years coach. She was essential in providing materials to use within my dissertation, recruitment efforts, and coaching all of my participants within my research study. I am also thankful to my research assistants, Briana Williams and Maddy Esterer (School Psychology Doctoral students at Michigan State University), for helping the project run so smoothly. I could not have done this project without them. Finally, I would like to thank my family and friends for being an important part of the dissertation process. I would like to extend a special shoutout to my fiancé, Harry Jadun and my family for listening, providing encouragement, and helping with material drop offs. I would also like to thank my cohort mates, Nicole, Lake, Emma, and Sally, for their continuous moral support and advice to help me through this PhD journey. iii TABLE OF CONTENTS LIST OF TABLES ........................................................................................................................vii LIST OF FIGURES ........................................................................................................................ ix CHAPTER 1 .................................................................................................................................... 1 INTRODUCTION ........................................................................................................................... 1 Incredible Years Teacher Classroom Management Program .............................................. 5 Pilot Studies ....................................................................................................................... 15 Current Study..................................................................................................................... 16 CHAPTER 2 .................................................................................................................................. 18 LITERATURE REVIEW .............................................................................................................. 18 Teacher Classroom Management Defined ........................................................................ 18 Teachers At-Risk for Poor TCM Skills ............................................................................. 19 Lack of training. .................................................................................................... 20 Lack of support for at-risk teachers ....................................................................... 21 The Role of Teachers in Early Childhood Development .................................................. 22 Attachment theory. ................................................................................................ 22 Social interaction learning theory. ......................................................................... 23 Evidence-Based Strategies to Improve TCM .................................................................... 24 Positive student-teacher relationships. .................................................................. 25 Praise and incentives. ............................................................................................ 26 Rules, expectations, and setting limits. ................................................................. 27 Social-emotional learning ...................................................................................... 27 Home-school collaboration. .................................................................................. 29 Individualized behavioral plans. ............................................................................ 29 Evidence-based training components to support TCM skills ............................................ 30 Manualized EBIs and group training ..................................................................... 31 Coaching ................................................................................................................ 31 Modeling................................................................................................................ 34 Promoting Implementation Success of Treatment Programs ............................................ 35 Disruptive Innovations ...................................................................................................... 37 Self-administered treatment approaches................................................................ 37 The Incredible Years ......................................................................................................... 39 The IY-TCM program. .......................................................................................... 40 IY-TCM program components. ............................................................................. 40 Promoting fidelity and acceptability of the IY-TCM program. ............................ 41 Research on the IY-TCM. ..................................................................................... 42 Barriers in implementation of the IY-TCM group training program. ................... 47 The Self-Administered IY-TCM Program ........................................................................ 48 Research on the Self-Administered IY Programs ............................................................. 52 iv Treatment implementation fidelity and treatment adherence of the SAPT Programs. ............................................................................................................... 52 Effectiveness of the SAPT program. ..................................................................... 54 Acceptability of the SAPT program. ..................................................................... 57 The self-administered IY-TCM (SATCM) program. ............................................ 59 Pilot Study to Assess Fidelity, Effectiveness, and Acceptability ...................................... 61 Rationale for a feasibility study. ............................................................................ 61 Designing a pilot study. ......................................................................................... 61 Randomized Experimental Pilot Designs .......................................................................... 63 Effect size calculations .......................................................................................... 64 Research Questions and Hypotheses ................................................................................. 65 Question 1a. ........................................................................................................... 66 Question 1b. ........................................................................................................... 66 Question 2a. ........................................................................................................... 67 Question 2b. ........................................................................................................... 68 Question 3a. ........................................................................................................... 69 Question 3b. ........................................................................................................... 70 CHAPTER 3 .................................................................................................................................. 76 METHODS .................................................................................................................................... 76 Participants ........................................................................................................................ 76 Attrition. ................................................................................................................ 81 Measures ............................................................................................................................ 82 Demographics questionnaire. ................................................................................ 82 Treatment fidelity. ................................................................................................. 82 Treatment effectiveness. ........................................................................................ 84 Treatment acceptability. ........................................................................................ 86 Procedure ........................................................................................................................... 91 Recruitment. .......................................................................................................... 91 Project personnel and training. .............................................................................. 92 Treatment phases. .................................................................................................. 94 Data Analyses .................................................................................................................. 101 Question 1a-b....................................................................................................... 101 Question 2a-b....................................................................................................... 101 Question 3a. ......................................................................................................... 102 Question 3b. ......................................................................................................... 103 Effect size calculations. ....................................................................................... 103 CHAPTER 4 ................................................................................................................................ 105 RESULTS .................................................................................................................................... 105 Research Questions and Hypotheses ............................................................................... 105 Question 1a. ......................................................................................................... 105 Question 1b. ......................................................................................................... 106 Question 2a. ......................................................................................................... 107 Question 2b. ......................................................................................................... 109 Question 3a. ......................................................................................................... 111 v Question 3b. ......................................................................................................... 112 CHAPTER 5 ................................................................................................................................ 118 DISCUSSION ............................................................................................................................. 118 Adherence to Treatment Sessions ................................................................................... 118 Coaching Adherence and Inter-Rater Reliability ............................................................ 120 Effectiveness of Teacher Classroom Management Strategies ......................................... 121 Acceptability between the SATCM and Book +Activity Group..................................... 125 Acceptability of the SATCM Intervention ...................................................................... 127 DVD acceptability. .............................................................................................. 129 Limitations ....................................................................................................................... 130 Treatment dosage................................................................................................. 130 Attrition. .............................................................................................................. 131 Teacher-collected data. ........................................................................................ 132 Coaching fidelity reliability. ................................................................................ 132 Randomization. .................................................................................................... 133 Generalizability of results.................................................................................... 133 Small sample size. ............................................................................................... 133 Research analysis................................................................................................. 134 Implications for Research ................................................................................................ 134 Implications for Practice.................................................................................................. 137 Conclusion ....................................................................................................................... 139 APPENDICES ............................................................................................................................. 141 APPENDIX A: Inclusion And Exclusion Criteria .......................................................... 142 APPENDIX B: Cut-Off Score ......................................................................................... 143 APPENDIX C: Demographics Questionnaire ................................................................. 144 APPENDIX D: Example Of Teacher Self-Monitoring Checklist ................................... 146 APPENDIX E: Example Of Teacher-Coach Meeting Form: Self-Monitoring Checklist ......................................................................................................................................... 149 APPENDIX F: Example Of Teacher-Coach Meeting Form: Observation Checklist ..... 151 APPENDIX G: Teacher Strategies Questionnaire .......................................................... 152 APPENDIX H: Treatment Evaluation Questionnaire (Teq) – Teacher Form ................ 154 APPENDIX I: Teacher Workshop Satisfaction Questionnaire ....................................... 156 APPENDIX J: Example Of The Teacher Video Evaluation Questionnaire .................... 159 APPENDIX K: Final Teacher Interview Questionnaire (Treatment Group Only) ......... 160 APPENDIX L: Study Recruitment Flyer ........................................................................ 161 APPENDIX M: Consent Forms ..................................................................................... 162 REFERENCES ............................................................................................................................ 170 vi LIST OF TABLES Table 1. Evidence-Based Classroom Management Strategies for Early Childhood Teachers ....... 1 Table 2. Summary of Fidelity, Effectiveness, and Acceptability Outcomes on the Incredible Years Self-Administered Programs ......................................................................................................... 12 Table 3. Summary of Published Studies on the Acceptability of IY-TCM Group Training........... 45 Table 4. Comparison Between IY-TCM Group Training, SATCM Training, and Book + Activity Only ............................................................................................................................................... 49 Table 5. Research Questions, Hypotheses, Measures, and Data-Analysis ................................... 73 Table 6. Demographics of Analyzed Sample ................................................................................. 80 Table 7. Assessments Completed During Each Phase .................................................................. 96 Table 8. Assigned Chapters for the Book + Activity and SATCM Group ................................... 100 Table 9. Means, Standard Deviations, and Percentages of Fidelity scores for the SATCM Treatment Group and Book +Activity Comparison Group ......................................................... 106 Table 10. Average Self-Reported Coaching Scores .................................................................... 107 Table 11. Teacher Perceptions of Classroom Management Strategies ...................................... 108 Table 12. Descriptive Statistics of Teacher Classroom Management Strategy Perceptions and Frequency-of-Use ........................................................................................................................ 109 Table 13. ANCOVA Results of Teacher Frequency-of-Use Scores ............................................. 110 Table 14. Teacher Acceptability Ratings of Treatment across SATCM Treatment and Book + Activity Comparison groups ........................................................................................................ 111 Table 15. Average Teacher Acceptability Ratings across SATCM Treatment and Book + Activity Comparison groups ..................................................................................................................... 112 Table 16. Average Values of Acceptability Scores ...................................................................... 113 Table 17. Average Acceptability Ratings of SATCM Strategies.................................................. 113 Table 18. Average Acceptability Ratings of DVDs ..................................................................... 114 vii Table 19. Teacher Interview Themes of Acceptability ................................................................ 115 viii LIST OF FIGURES Figure 1. The Teaching Pyramid. .................................................................................................. 25 Figure 2. Flow Diagram of Participants Through the Trial ......................................................... 77 ix CHAPTER 1 INTRODUCTION Early childhood education teachers (preschool and kindergarten) often cite inadequate training in managing disruptive classroom behavior (Dicke, Elling, Schmeck, & Leutner, 2015). An inability to effectively manage classroom behavior may elevate levels of teacher stress, mental health problems, burnout, and issues with teacher retention (Flower, McKenna, & Haring, 2017; Webster-Stratton, 2012). With limited training in teacher classroom management (TCM), teachers may rely on poor teaching strategies such as yelling and harsh criticism, or engage in coercive interactions with children (Patterson, 1982; Webster-Stratton, 2012). Such strategies can exacerbate behavioral problems in the classroom and lead to poor social-emotional, behavioral, and academic outcomes for students (Jones & Jones, 2015). To promote successful behavior and disrupt this learned pattern of negative interactions with children, Patterson (1982) outlined the importance of training teachers to reshape their behavior to reduce disruptive behavior while also promoting socially competent behaviors. Several positive TCM strategies have been identified in the literature to improve classroom behavior (see Table 1). Interventions that utilize these strategies to optimize management of classroom behavior and promote positive interactions with students in early childhood is essential and has the potential to enhance teacher well-being (Ford et al., 2019). Table 1. Evidence-Based Classroom Management Strategies for Early Childhood Teachers Evidence-Based TCM Strategy Definition/Examples of Components Research Outcomes Behavior Teachers provide child with verbal praise • Reinforces positive rates of specific praise to increase desired behavior (e.g., good job behavior and helps with putting away your toys) building student relationships (Allday et al., 2012; Jones & Jones, 2015) 1 Table 1 (cont’d) Coaching With coaching, teachers use descriptive • Expands vocabulary through comments to highlight specific academic modeling and naming of skills (e.g., “You have two yellow objects (Smith, & Jones, 2011) blocks”), social skills (e.g., “I’m going to • Improves language and school be your friend and share with you”), readiness (Webster-Stratton, persistence (e.g., “Diana is really 2012) concentrating on her work”), emotion • Reduces aggressive behavior identification (e.g., “You look really and improves school readiness excited to finish that project. Your whole (Durlak, Weissburg, Dymnicki, body is bouncing!”) Taylor, & Schellinger, 2011) Collecting Teachers are taught how to choose a target • Results in fewer misbehaviors data/creating behavior, collect data on what happens and improves student individual before and after the behavior, create a achievement (Reinke & behavioral plans hypothesis about why the behavior is Herman, 2002) occurring, and then create a behavior plan to improve the behavior Home-school Teacher establishes positive relationship • Improve academic performance collaboration with parents (e.g., notes home, phone and prevents behavior calls), uses a family-center approach, and problems (Jones & Jones, understands family’s cultural backgrounds 2015) to promote students’ social-emotional, and academic needs Incentives Teacher gives child an attention-based • Increases positive behavior in (e.g., lunch with teacher, special helper) or preschool children and reduces tangible reward (e.g., stickers) for undesired behavior (Doll, performing desired behavior McLaughlin, & Barretto, 2013) Problem-solving Teaching children alternative thinking • Increases skills in emotion and emotion- strategies to solve problems and direct knowledge and emotion regulation instruction of how to calm down recognition (Domitrovich, Cortes, & Greenberg, 2007), and lowers levels of aggression (Bierman et al., 2008) Positive teacher- Teachers use warmth, empathy, play, and • Reduces classroom behavior student positive attention to improve classroom problems (Marzano, Marzano, relationships behavior & Pickering, 2003; Pianta, Hamre, & Allen, 2012) 2 Table 1 (cont’d) Rules, Teachers set up clear, age-appropriate • Reduces likelihood of expectations, rules, expectations of behavior, and behavioral problems in the and limit setting organize the classroom to prevent classroom (Jones & Jones, behavioral problems from occurring 2015; Simonsen & Fairbanks, Briesch, Myers, Sugai, 2008) • Improves executive functioning skills (Ursache, Blaire, & Raver, 2012) Currently, in-service training, often called professional development training, is used to address TCM skill deficits in early education teachers (Egert, Fukkink, & Eckhardt, 2018). Traditionally, schools have focused on passive approaches (i.e., reading or 3-hour workshops) that focus exclusively on content. However, these have been found to be ineffective in creating long-standing behavior change as teachers are less likely to generalize skills if they are not practiced in the context of their own classroom or practiced these skills over a sustained period of time (Darling-Hammond, Hyler, & Gardner, 2017). Researchers indicate that in-service professional development is most effective when the training involves active learning (e.g., teachers have opportunities to practice skills, observe other teachers, receive feedback, and reflect), coherence (i.e., the PD content and activities are linked to teacher beliefs/goals to connect to the “big picture,” content-focused (i.e., focused on learning new skill to use with students through readings or lectures), collaborative participation (i.e., work with peers, mentors, coach or group to learn and engage with material), and sustained over time (i.e., 10-12 workshops or at least 20 hours over a semester or year; DeSimone & Garet, 2015; Wei, Darling-Hammond, & Adamson, 2010). One solution to address the TCM skill gap is to implement in-service professional development training using a manualized evidence-based intervention (EBI). Manualized approaches are a linear series of sessions with a fixed order which clearly list the procedures and 3 treatment goals to address the problem during each session (Marchette & Weisz, 2017). Evidence-based interventions are those that demonstrates positive outcomes in at least two well- designed studies (Mihalic & Elliot, 2015). A common way to deliver information is using a group format where a group leader instructs teachers on evidence-based TCM strategies. This can enhance teacher training as teachers can have group discussions and model skills to enhance learning. EBIs that focus on training teachers in evidence-based classroom management strategies have been found to decrease aggression and reduce behavioral problems in the classroom (Durlak et al., 2011). Furthermore, these EBIs have been found to (a) improve children’s peer relationships and other social-emotional skills, (b) increase teachers’ use of positive classroom management strategies, and (c) decrease teachers’ use of negative classroom management strategies (Webster-Stratton, Reid, & Stoolmiller, 2008; Nye, Mellendez-Torres, & Gardner, 2018). Although PD is helpful to improve teacher learning, PD has been found to be more effective when delivered with a coaching component than PD workshops alone (Kraft, Blezar, & Hogan, 2018). Bandura’s social cognitive theory (2001) argues that underlying beliefs and cognitive perceptions are essential for behavior change to occur. Specifically, using a coaching component emphasizing collaborative discussion has been found to be an effective way to create belief change to improve teacher skills (Cook et al., 2015). Kraft, Blezar, and Hogan (2018) describe coaching as an expert working with a teacher in a manner that is individualized (one-on one), intensive (e.g., meet at least biweekly), sustained (throughout school semester or school year), focused, and context-specific (i.e., within the context of the classroom). Coaching has been found to be an effective strategy to help teachers improve implementation fidelity of an intervention, generalize skills learned in PD training sessions, and help differentiate instruction 4 for teachers with various backgrounds and concerns (Driscoll, Wang, Mashburn, & Pianta, 2011; Reinke, Newcomer, Marchese, and Lewis, 2015; Stormont, Herman, & Newcomer, 2014). Additionally, coaching can be an effective strategy to help change teacher misconceptions and beliefs of strategies (a significant barrier to implementation success; Forman et al., 2009) through collaborative discussion about teacher practices (Darling-Hammond, Hyler, & Gardner, 2017). Incredible Years Teacher Classroom Management Program Several systematic reviews of evidence-based classroom programs identify IY-TCM (Webster-Stratton, 2001) as the most effective treatment to improve classroom disruptive behavior and train teacher behavior (Bierman & Motamodi, 2015; Whear et al., 2013). The IY- TCM program is part of the Incredible Years series which includes a teacher, parent, and child, behavioral training program (Webster-Stratton, 2012). The IY-TCM program focuses on shaping caregiver behavior to change child behavior. The program is founded in theoretical learning and behavioral models such as social interaction learning theory (i.e., coercive cycles of interactions between caregivers and children reinforce behavioral difficulties in children; Patterson, 1982) and attachment theory (i.e., emphasizes the importance of positive relationships between adults and children; Bowlby, 1982). The program also emphasizes social learning theory (i.e., humans learn best by observing and imitating the actions of others; Bandura, 1977) and social constructivist learning theory (i.e., humans learn best through interactions with one another; Vygotsky, 1978) as a philosophy of training caregivers. Two meta-analyses have been conducted which support the effects of the IY-TCM program to improve TCM skills. Specifically, Nye, Mellendez-Torres, and Gardner (2018) conducted a multilevel meta-analysis and found a moderate to large improvement on positive (g 5 =0.73) and negative (g =0.49) teacher strategies, a small improvement on children’s prosocial skills (g =0.12) and a small to negligible improvement on child conduct problems (g =0.05). Korest and Carlson (2021) conducted a multivariate meta-analysis and found comparable results in both positive (g = 0.70) and negative (g = 0.50) TCM strategies. Additionally, the IY-TCM program has been successfully implemented in multiple contexts (e.g., preschool, elementary, Head Start [Raver et al., 2008]), with a variety of populations (e.g., parapros [Baker- Henningham &Walker], after school care workers [Hicks-Hoste, Carlson, & Tiret, 2015], elementary school teachers [Ford, 2019]), and has demonstrated effectiveness across the world (Korest & Carlson, 2019). Despite the research supporting this program, the group training format demonstrates several potential barriers when implementing IY-TCM in the school context. First, the program requires a certified group facilitator to lead training programs, which can be difficult for schools if there is not a certified trainer in the area. Second, the program is lengthy, requiring 7-hour monthly training sessions over the course of 6 months with homework assignments completed in between (Webster-Stratton, 2001). Third, teachers may not have the flexibility in their schedules (i.e., substitute teachers need to be hired) to attend training or may have to take personal time (i.e., weekends or after school) to attend additional training sessions. Fourth, the out-of-pocket costs for IY-TCM training can be up to $1730-2100 per day if the school does not already have a certified trainer (http://www.incredibleyears.com/workshop-info/hosting-workshop/). Finally, the group training program may be paced too quickly for teachers that are particularly new to implementing TCM strategies (i.e., first year teachers, paraprofessionals), which may necessitate more training on a particular topic (Webster-Stratton, Reinke, Herman, & Newcomer, 2011). 6 Thus, barriers in IY-TCM implementation may make it impossible to reach populations that are most in need of TCM training and support (Forman et al., 2013). To improve the implementation success of interventions to populations in need, researchers highlight several helpful necessary treatment components. Witt and Elliot (1985) indicate there is a sequential and reciprocal relationship between treatment acceptability, treatment integrity, treatment use, and treatment effectiveness. Acceptability refers to whether stakeholders perceive an intervention as fair, reasonable, and appropriate (Witt & Elliot, 1985). Specifically, acceptability may be examining a participants satisfaction with specific aspects of a treatment such as content, procedures, ease, delivery, time, outcomes, and perceived usefulness (Eckert & Hintze, 2000; Proctor et al., 2011), while treatment integrity (also called fidelity) incorporates treatment adherence (e.g., “Did they follow the procedures?”), participant responsiveness, and treatment implementation (e.g., “Was the treatment implemented as intended?”; Forman et al., 2013). If a treatment or intervention is considered acceptable among participants, participants are more likely to implement the procedures of the treatment with integrity, which increases the likelihood of positive change in behavior, which in turn should increase the acceptability of participants implementing the intervention (Eckert & Hintze, 2000). A manualized treatment approach should have higher integrity, effectiveness, and acceptability because of its specific directions and easy implementation. Although the IY-TCM group training program includes a manualized treatment approach, a self-administered version may also address pragmatic barriers (i.e., buy-in, time, cost of treatment) associated with the group training, thus improving implementation success. To increase the accessibility of EBIs to all school personnel in need of training, Rotheram-Borus, Swendeman, and Chorpita (2012) discuss disruptive innovations, which use 7 novel delivery formats (i.e., coaching, brief, self-directed, media) to improve implementation and dissemination. Specifically, disruptive innovations are thought to improve successful implementation of EBIs because of their reduced cost and higher accessibility to the general public. One type of disruptive innovation recommended is self-directed or self-administered delivery formats (Rotheram-Borus, Swendeman, & Chorpita, 2012). Self-directed interventions are supported by adult learning theories which indicate that adults prefer to take personal control over their own learning (Louws, Meirink, van Veen, van Driel, 2017). Self-administered treatments are often delivered via bibliotherapy (i.e., self-help books or instructional manuals) or digital/multimedia materials (i.e., audio or computer-delivered information). Although self-help books typically do not have a set of required guidelines, there are several self-administered evidence-based treatments that align with most PD criteria and include a systematic approach (i.e., action plans, goals, practice activities, and readings) that focus on changing adult behavior to change child behavior (however most of these focus on parent behavior change). Additionally, the use of video modeling (e.g., DVDs which provide examples of TCM strategies can be an easy and cost-effective delivery method to learn TCM skills to enhance training (DeSimone & Garet, 2015; Powell). Self-administered interventions also have potential to address implementation issues related to access to expert trainers as self-administered interventions do not require contact between an expert and trainee (McCyntire & Neece, 2016) and can be used by teachers, paraprofessionals, or other health care or school professionals (Rotheram-Borus et al., 2012). Because of their systematic, easy, and accessible format, this can be an inexpensive, convenient, and flexible approach for teachers who lack time to attend long professional development seminars and who have few resources available in the school (Reinke, Stormont, Herman, & Newcomer, 2014). Furthermore, teachers that self-select an intervention based on 8 their needs might find the intervention more engaging and thus be more likely to participate in an intervention (Louws et al., 2017). Although self-administered programs can be useful to reduce treatment barriers, this format may not address teachers’ complaints on the lack of support in the school system (Forman et al., 2013; Ingersoll, 2005) or address important PD components such as feedback or collaboration. Thus, an additional disruptive innovation to improve successful implementation of an EBI is coaching (Reinke, Stormont, Webster-Stratton, Newcomer & Herman, 2012; Webster- Stratton et al., 2011). Coaches are implementation agents (i.e., experienced teachers, school psychologists, school counselors) that help promote successful implementation of EBIs and promote treatment fidelity (Rotheram-Borus et al., 2012). In the schools, coaches can utilize evidence-based training components to enhance teacher training in TCM strategies such as providing feedback, modeling interventions, helping teachers set goals, and clarifying information on how to successfully deliver positive TCM strategies (Reinke et al., 2012). Although increased use of coaching has been demonstrated in the school context, coaching can be costly. Thus, new trends of utilizing remote or web-mediated coaching to support trainers have been used to reduce cost and need of several expert coaches (Powell, Diamond, Burchinal, & Koehler, 2010). Research on self-administered interventions with multiple sessions (10-12) have demonstrated similar levels of effectiveness compared to group evidence-based interventions (e.g., Webster-Stratton, Kolpacoff, & Hollinsworth, 1988). However, self-administered interventions with coaching may increase the integrity, perceptions of usefulness, and acceptability of the program components, which can serve as implementation barriers for interventions. Researchers suggest that adding a coaching component may also help the program 9 perform more similarly to a group training format (Taylor et al., 2008; Shernoff & Kratochwill, 2007; Webster-Stratton, 1990). For teachers who may lack knowledge and skills in TCM such as paraprofessionals, after-school care workers, new teachers, daycare workers, and teachers who are struggling with TCM, Reinke, Stormont, Webster-Stratton, Newcomer, and Herman (2012) suggest that teachers may need to spend more time learning specific strategies than teachers with more training. Thus, self-administered interventions that are systematic, sustained over time that require active learning may benefit less experienced school caregivers as they can learn the material at their own pace. Additionally, collaborating with a coach may help teachers more successfully generalize skills, address teacher beliefs, and change teacher behavior within the classroom context. To address the barriers of the group training program, the IY-TCM program developer created a self-administered format for caregivers (i.e., parents and teachers; Webster-Stratton, 2001). Unlike the group training research, only one study to date has evaluated the self- administered IY-TCM program (SATCM) with coaching (Shernoff & Kratochwill, 2007; See Table 2); thus, most of the research supporting the self-administered format comes from the self- administered Incredible Years Parent Training program (SAPT), a behavioral parent training program for children ages 3-8 (Blueprints for Health Youth Development, 2019; Mihalic & Elliot, 2015). Although several studies have examined the SAPT program, the research on the fidelity, effectiveness, and acceptability of the self-administered formats is varied (see Table 2). For instance, only five studies examined treatment adherence with most studies demonstrating wide ranges of fidelity (i.e., Taylor et al., 2008). Most studies (n = 8) included effectiveness measures using observations, goal attainment scales, or self-report measures; however, only three studies 10 found improvements in positive caregiver techniques, and several did not include data within the report. Seven studies included acceptability measures including the acceptability of the intervention, perceived effectiveness to improve child behaviors, and time taken to improve child behaviors at post treatment and most participants rated the program as having adequate acceptability (i.e., ratings at or above the midpoint scale; Shernoff & Kratochwill, 2007). 11 Table 2. Summary of Fidelity, Effectiveness, and Acceptability Outcomes on the Incredible Years Self-Administered Programs Author Program Description of Research Fidelity Effectiveness Acceptability Kratochwill, PT& Randomly assigned 69% 75% of parents and 95% of Reported high scores on Elliot, Loitz, TTb participants to a self- compliance teachers demonstrated all three TEQ subscales for Sladeczek, & administered TT and PT with treatment progress towards goals both teachers and parents Carlson (2003) program with conjoint program using based on a goal attainment consultation, a manual checklist scale. Repeated measures only treatment, or a analysis for observation of control group caregiver behaviors and self-report use of strategies were not significant Osburn (2009) PT Repeated AB design Declined over Parent reported increase in High treatment comparing externalizing four sessions setting rules and acceptability score (M = and internalizing from 78% to expectations, use of praise, 96.89 on TEQ-Parent; behavior, acceptability, 68% using and use of time outs compared to midpoint and integrity checklist instead of harsh discipline score of 73.5) Ogg & Carlson PT Conducted AB design Varied Parents Practices Interview High treatment (2009) for (N = 5) parents between 0- indicates parents improved acceptability on TEQ- examining perceived 93% for with positive parenting Parent (M = 4.6/5 per effectiveness, workbooks (23% change), clear item) acceptability, and completed and expectations (11% change) integrity with children 92-100% for and monitoring (7% diagnosed with ADHD videos watched change) Shernoff & TTb Compared TT only to Did not TT with consultation TT with consultation had Kratochwill TT with consultation measure group had higher higher acceptability than (2007) using single case design confidence ratings than TT TT only on TEI-SF (no only group data reported) Stewart & PT Measured acceptability Did not Did not measure Significant difference in Carlson (2010) of PT videos collecting measure acceptability between data bi-weekly for 8 video series 2 (Reducing weeks inappropriate behaviors) and 3 (supporting your child’s education; d= 0.39) 12 Table 2 (cont’d) Taylor et al. PTb Measured participation, 76% of All families made at least 87% of the participants (2008) goal setting, and participants 50% progress on at least reported they felt ‘‘very satisfaction of the completed one goal positive’’ or ‘‘positive’’ computer-based PT more than half about the program; 93% program with coaching of the program would recommend the and 63% of program to a friend or participants relative; 76% felt completed “confident” or “very 100% of the confident” in managing program current child behavior components problems Walcott, PT Four parents of children Ranged from 2/4 parents found Did not measure Carlson, & with ADHD using single 79%-96% for improvements in setting Beamon (2009) case ABAB design skills practices clear expectations, and 40%-79% increased use of praise, for manuals and “other positive completed reinforcement techniques” Webster-Stratton PT Randomly assigned to Did not SA reduced mother CSR indicated SA not as et al. (1988) parent GT, GT+VM, measure behavior compared to high as GT and GT+VM SA, and WLC group control; but similar (No specific data reported) findings between SA and GT on CSR b Webster-Stratton PT Randomly assigned to Did not SA and SA+ Coaching Did not measure (1990) SA + coaching, SA only measure improved compared to or WLC WLC (No data reported) *Note: GT=Group training; SA=Self-administered; WLC=Wait list control; TEQ = Treatment Evaluation Questionnaire; CSR=Consumer Satisfaction Report; TEI = Treatment Evaluation Inventory-Short Form a Only article that exclusively looks at SATCM program. b Includes coaching in treatment 13 An additional gap in the literature is with the coaching component. Although there is some evidence to support coaching, the type of coaching delivered to caregivers has varied. For instance, some researchers used a behavioral consultation model or conjoint behavioral consultation model to support caregivers (Kratochwill et al., 2003; Shernoff & Kratochwill, 2007), while other researchers used therapists who were certified in the IY training (Taylor et al., 2008, Webster-Stratton, 1990). Additionally, dosage has differed for coaching sessions. For instance, Webster-Stratton (1990) only added two additional hours of coaching support compared to the self-administered only group, making it difficult to conclude if coaching really improved teacher skills. A final gap in the literature is the primary focus on children at-risk for conduct problems instead of teachers at-risk for challenges with classroom behavior management. Research indicates that teachers at-risk for poor TCM skills such as paraprofessionals, daycare center workers, first year teachers, pre-service teachers, and teachers who receive limited training in TCM may demonstrate larger improvements in the IY-TCM program than teachers who already demonstrate knowledge in TCM training (Baker-Henningham and Walker, 2018). For example, Hutchings, Martin-Forbes, Daley, and Williams (2013) examined elementary and preschool head teachers with an average of nine years of experience and found relatively small improvement in the use of both positive (d = 0.17) and negative TCM skills (d = 0.03). Baker-Henningham and Walker (2018) on the other hand, found that in a sample of paraprofessionals in Jamaica with only a high school education, there was a large improvement in positive TCM strategies (d= 3.35) and a significant reduction in negative TCM strategies (d=1.35). These results parallel treatment outcomes found when examining the effects of the IY-TCM group training program on high-risk children. For example, Hutchings and colleagues (2013), Hickey and colleagues 14 (2017), and Baker-Henningham, Scott, Jones, and Walker (2012) found that children identified as at-risk for conduct problems through a cut-score demonstrated substantially more improvements than typically developing children. Thus, it may be more beneficial for researchers and school personnel to identify teachers who are at-risk as TCM strategies are seen as the mechanism of change to improve classroom disruptions. Pilot Studies Sheridan (2014) recommends consideration of a progressive 10-step intervention trajectory model to identify the most appropriate stage of research development when examining an intervention. Following the identification of an issue (i.e., TCM problems) and strategies to address the issue (i.e., SATCM training program), Sheridan indicates an intervention should attempt to test for feasibility in a pilot study. Bowen and colleagues (2009) explain specific research designs that are most appropriate when assessing feasibility in a pilot study. Feasibility studies are an overarching term which encompasses the term pilot study (Eldridge et al., 2016). Bowen and colleagues (2009) define a feasibility study as “any kind of study that can help investigators prepare for full-scale research leading to intervention” (p. 453) when limited published research is available on an intervention program. Additionally, feasibility studies are used to determine if the intervention is appropriate when there is uncertainty about the future of large-N, randomized control trials (RCT) and when attempting to figure out if the intervention can work in a specific setting (Bowen et al., 2009; Eldridge et al., 2016). Bowen and colleagues (2009) proposed a protocol for identifying potential study designs based on eight areas of focus and three intervention development phases. For this study, the areas of focus that are most aligned with the current study include implementation (i.e., integrity or the extent to which the intervention was executed as planned), limited efficacy 15 testing (i.e., the exploration of whether the intended effects of the intervention occurred and the consideration of intervention effectiveness in a future study with more controlled conditions [e.g., RCT]), and acceptability (i.e., participant or other stakeholder reactions to the intervention). If there is some evidence that an intervention might be efficacious under ideal conditions (e.g., Webster-Stratton, 1990) compared to other conditions, Bowen and colleagues (2009) suggest conducting a pre-post design, small-scale RCT experimental design with a comparison group, or combining research designs to their areas of focus. Additionally, because a pilot study is attempting to find out if this intervention could work in a school setting, adding in a qualitative component such as an exit interview, may help to provide a deeper understanding of implementation strategies, barriers, and facilitators of the intervention programs (Shoonenboom & Johnson, 2017). Although the IY-TCM program (i.e., group format) has been extensively tested and has demonstrated positive outcomes, (Nye et al., 2018) and there is some initial efficacy to support the SAPT program, (i.e., Webster-Stratton et al., 1988; Webster-Stratton, 1990) only one published study to date (i.e., Shernoff & Kratochwill, 2007) has examined the SATCM program as a standalone in a self-administered format. Additionally, Shernoff and Kratochwill (2007) used a problem-solving consultation model, primarily focused on child outcomes. Thus, based on Sheridan’s model (Sheridan, 2014) and Bowen’s recommendations (Bowen et al., 2009), a pilot study was implemented to assess the fidelity, effectiveness, and acceptability of a self- administered intervention with coaching for teachers at-risk of classroom management issues. Current Study This study examined the fidelity, effectiveness, and acceptability of the SATCM with a coaching component using a pre-post pilot RCT design with a Book + Activity comparison 16 group. The treatment program was implemented in a 12-week, 6-session program for teachers identified as needing additional classroom behavior management support based on a TCM questionnaire during the COVID-19 pandemic. Thirteen participants were randomized to receive the SATCM treatment and 11 participants were randomized to the Book + Activity comparison group using assigned chapters in the Incredible Years: Nurturing Children’s Social, Emotional, and Academic Competence (Webster-Stratton, 2012), a book typically assigned to the IY-TCM program and directly aligns with the self-administered program. The SATCM coach met with SATCM treatment group participants every two weeks to create/review goals, discuss miscommunication about program components and strategies, model strategies, provide feedback, and answer teacher questions with SATCM treatment group teachers. Teachers within the Book + Activity comparison group received reminders to complete surveys from Qualtrics XM, a computer survey system, and reminders from two research assistants (RAs) to fill out reflection questionnaires and activity worksheets. Qualtrics reminders and RAs served to create a manualized approach for the Book + Activity comparison group. 17 CHAPTER 2 LITERATURE REVIEW In order to further specify the need and importance of the current study and its research questions, hypotheses, and study design, a thorough literature review was conducted on (a) TCM definitions, (b) teachers at risk for poor TCM and potential causes for poor TCM, (c) the role of teachers in child development, (d) evidence-based TCM strategies and components (e) common implementation barriers, disruptive innovations, and self-administered treatment programs, (f) IY-TCM group training program description, research, and barriers, (e) the SATCM program and research, and (f) randomized pilot study to assess the SATCM program. Teacher Classroom Management Defined TCM is broadly defined as the process a teacher takes to create an environment that produces and maintains appropriate behaviors to enhance academic engagement and improve social-emotional learning (Evertson & Weinstein, 2006). Researchers indicate TCM is considered a precondition for learning as effective teaching cannot occur in disruptive classrooms (Jones & Jones, 2015; Marzano et al., 2003). Although most preservice programs require teachers to demonstrate competencies in TCM strategies (e.g., Council for the Accreditation of Educator Preparation, 2018) and have knowledge on a broad range of tools to support student behavior, teachers report handling disruptive behavioral problems in the classroom as the most commonly cited problem (Korpershoek, Harms, de Boer, van Kuijk, & Doolaard, 2016). Negative classroom management practices are typically conceptualized as practices which include reactive “low rates of praise, harsh discipline, negative teacher-student-parent relationships, failure to focus on the social-emotional curriculum, and low emphasis on home- 18 school collaboration…” (Webster-Stratton, 2012, p. 40). Teachers may use reactive strategies focused on discipline when dealing with problem behaviors such as using punishment for disruptive students, name calling, frequent negative commands, calling out a student in front of the class, sending students into the hallway, in-house suspension, sending negative notes home to a parent and engaging in coercive interactions with students (Korpershoek et al., 2016). Research examining the use of negative TCM strategies have demonstrated profound impacts on teacher’s well-being and teacher retention (McCann & Johannessen, 2004; Jones & Jones, 2015). Specifically, several studies report challenging student behavior as the most stressful part of their professional lives (Jazaar, Lambert, & O'Donnell, 2007; Lambert, McCarthy, O'Donnell, & Wang, 2009). Ingersoll (2005) examined the rate of first year teacher departures and found that of those teachers who left the profession, 44% cited disruptive behavior problems as the reason for leaving. Additionally, teachers with poor TCM skills have significantly higher rates of burn out and teacher stress, which increases the likelihood of teachers using poor TCM skills as well as leaving the profession (McCann & Johannessen, 2004). Teachers At-Risk for Poor TCM Skills The literature identifies, preschool teachers (Webster-Stratton, 2001), first year teachers (Ingersoll & Smith, 2003), pre-service teachers (O’Neill & Stephenson, 2012), paraprofessionals (also called paraeducators and assistant teachers; Sosinsky & Gilliam, 2011), afterschool care workers (Hicks-Hoste et al., 2015), and teachers who self-identify as needing support in TCM strategies (Woodcock & Reupert, 2012) as being particularly at-risk for poor TCM skills. One of the most significant reasons this group is at a higher risk for poor TCM is due to lack of training 19 and/or experience (Maxwell, Lim, & Early, 2006), and lack of support in positive TCM strategies (Ingersoll & Smith, 2003). Lack of training. Maxwell, Lim, and Early (2006) discuss that programs with rigorous pre-service training typically includes didactic training and practicum experiences, while in- service training typically includes professional development. Despite the importance of training professionals to use TCM strategies, several pre-service programs do not include TCM courses. For instance, Frank Porter Graham Child Development Institute at the University of North Carolina at Chapel Hill conducted a nationwide study on higher education learning programs and found that of programs offering bachelor’s and master’s degree programs in education, only 13% offered coursework in classroom and behavioral management. The researchers found that with little training available, teachers were often left to their own resources and experiences to find how to manage classroom behavior (Maxwell et al., 2006). O’Neill and Stephenson (2012) conducted a study with final-year student teachers in Australia and examined their feelings of preparedness, familiarity, and confidence in using TCM strategies. The researchers compared students who completed focused coursework units on classroom management in teacher training programs compared to students who had not completed the coursework. They found that focused coursework increased feelings of preparedness, familiarity, and confidence in using TCM among student teachers. However, the student teachers reported that they were confident in using only half of the strategies they were familiar with, and that they did not feel fully prepared to manage the misbehavior of students. With a lack of confidence in using preventative TCM strategies, teachers tend to be more reliant on reactive strategies (Woodcock & Reupert, 2012), which can exacerbate classroom disruptions (Korpershoek et al., 2016; Webster-Stratton, 2012). 20 Although there is a lack of training reported for pre-service teachers, even less training is provided for other school and daycare professionals (Hicks-Hoste et al., 2015; Ratcliff, Jones, Vaden, Sheen, & Hunt, 2011; Sosinsky & Gilliam, 2011). Training paraprofessionals and after- school care workers are often an overlooked, but essential part of the early education school team (Mahoney, Levine, & Hinga, 2010; Sosinsky & Gilliam, 2011). Although paraprofessionals often have the most contact with delivering interventions, they often have the least amount of pre- service training requirements (Sosinsky & Gilliam, 2011). States vary in credentials for paraprofessionals as some districts only require a high school degree while others require an Associate degree in early child development (Sosinsky & Gilliam, 2011). Afterschool care workers may obtain even less training and fewer credentials (Hicks-Hoste et al., 2015; Ratcliff, et al., 2011). In the United States, nearly 30% of youth spend 3-5 days of the week in after- school care programs, yet few afterschool programs require workers to obtain teacher degrees or formal credentials for childcare (Mahoney, Levine, & Hinga, 2010). Additionally, afterschool workers typically do not receive more than 2-3 days of on-the-job training from their employer (Ratcliff et al., 2011) Lack of support for at-risk teachers. Poor teacher classroom management training can also be attributed to lack of support in the school environment (Forman et al., 2013; Ingersoll, 2005). Specifically, first year teachers report feeling overwhelmed and unqualified in supporting children with disruptive behaviors and cite lack of support and coaching (Ingersoll & Smith, 2003). Previous research indicates that teachers often want meaningful feedback on their teaching; however, they seldom receive feedback (Ingersoll & Smith, 2003). Although some schools assign a mentor to support first year teachers, research indicates mentors only reduce 21 attrition rates by one percentage point (Smith & Ingersoll, 2004). Thus, we need to understand how to best support teachers to improve TCM skills (Reinke et al., 2014). The Role of Teachers in Early Childhood Development Before understanding how to train teachers in proactive TCM strategies, it is essential to understand how teacher’s interactions with children affect the onset of disruptive behaviors in the classroom. Specifically, there are two theories essential to the understanding of how TCM strategies influence child behavior: attachment theory (Ainsworth, Blehar, & Water, 1978; Bowlby, 1982) and social interaction learning theory (Patterson, 1982). Attachment theory. One foundational theory in understanding the conceptualization of teacher classroom management and its effects on classroom disruptive behavior is attachment theory. Attachment theory is defined as the ability to develop warm relationships and bonds with others. When a child develops a warm relationship with an adult, they use this internalized model of a relationship to promote the foundation of self-regulation skills, which are important in promoting self-control, patience, and school readiness (Ainsworth et al., 1978). Attachment theorists such as Bowlby (1982) believe a secure attachment (e.g., caused by consistency in teacher responsiveness and teacher warmth) to an adult early in childhood is crucial for positive child development, while insecure (e.g., caused by low teacher warmth) and disorganized attachment (e.g., caused by inconsistency in teacher response) can lead to the development of psychopathology in children and thus, disruptive classroom behavior (Cohn, 1990). Research supports attachment theory and how it can lead to both positive student behavior and negative disruptions in the classroom. Specifically, children who develop positive relationship with their teachers characterized by warmth and positive interactions are more likely to demonstrate prosocial skills, use less aggressive behavior towards others (Cohn, 1990), and more likely to 22 follow teacher directives (Jones & Jones, 2015). In contrast, young children who develop poor relationships with their teachers have lower self-esteem, use more aggressive, hostile behaviors in school and with their peers, and have an increased likelihood of school dropout (Fearon, Bakermans-Kranenburg, Van IJzendoorn, Lapsley, & Roisman, 2010). Social interaction learning theory. Patterson’s (1982) social interaction learning theory provides another conceptual framework to understand how patterns of teacher-child interactions over time can affect classroom behavioral outcomes. This theory indicates that when teachers have positive or neutral interactions with children, this improves children’s prosocial behavior and reduces classroom disruptions. However, teachers who engage with children in early childhood using coercive interactions can lead to a pattern of disruptive behavioral problems (Webster-Stratton, 2012). Specifically, the coercive interactions occur because the child displayed an aversive behavior (e.g., whining, non-compliance, tantrum). This results in the teacher reacting to the behavior and the child counter-reacting in a back-and-forth interaction. The interaction typically ends with the teacher giving in (i.e., providing access to something the child wanted) which reinforces the child’s behavior to repeat this interaction again or the child giving in which reinforces the teacher to use this interaction again (or intensify the reaction) to reduce behavior (Patterson, 1982). Teachers who engage in this coercive interaction can exacerbate and maintain a child’s aggressive behavior (Reinke & Herman, 2002). Additionally, children who learn this interaction often continue this pattern with other adults and peers outside the classroom (Reinke & Herman, 2002; Webster-Stratton, 2012). School-age children between the ages of 3-8 are rapidly developing social-emotional skills, especially skills involved with regulating their emotions (i.e., inhibiting aggressive behavior). Thus, they tend to have a higher incident rate of tantrums and aggression in the 23 classroom and more heavily rely on adults to regulate their behavior (Bierman & Motamodi, 2015). To promote successful behavior and disrupt this learned pattern of negative interactions in children, Patterson (1982) outlined the importance of training caregivers (i.e., parents and teachers) to reshape their behavior to increase positive interactions with their students and reduce coercive interactions. Research indicates that caregivers who learned how to respond correctly to child aversive behaviors predict healthy development of social-emotional skills and long-term positive outcomes (Patterson, 1982; Webster-Stratton, 2001). On the other hand, caregivers who respond to a child’s inappropriate behavior using harsh or negative practices in early childhood predict the development of conduct problems in children (Odgers et al., 2008) and the child’s antisocial behaviors tend to elicit more harsh and coercive reactions from caregivers (Patterson, 1982). This behavior can also lead to long-term consequences for children. For instance, coercive interactions at school can lead to negative outcomes for children such as peer rejection, negative school reputations, and academic failure, which may increase their antisocial behavior (Webster- Stratton, 2001). Evidence-Based Strategies to Improve TCM As Patterson (1982) indicated, reshaping teacher behaviors to promote positive TCM strategies in the classroom is imperative to improve both teacher competence and reduce classroom disruptive behavior. Thus, it is also important to understand which strategies are the most effective to improve classroom disruptive behavior. Researchers define evidence-based TCM strategies as liberally applying preventative strategies such as displaying warmth, having positive interactions with students, explaining and providing consistent rules and structured routines, using high rates of praise, setting limits on disruptive behavior, promoting social-emotional learning strategies, and promoting home-school 24 collaboration (Jones & Jones, 2015; Korpershoek et al., 2016; Webster-Stratton, 2012). One way to conceptualize the importance of preventative teaching strategies is through the Teaching Pyramid (Webster-Stratton, 2012; see Figure 1). The Teaching Pyramid suggests that teachers should emphasize skills in the base of the triangle (i.e., creating relationships, proactive teaching strategies, and praise) that work to prevent behavioral problems from occurring and use the strategies on the top of the triangle sparingly (losing rewards, timeouts). Going with this analogy, if teachers start with the top of the pyramid (i.e., discipline strategies), the pyramid will topple as teachers have not developed foundational TCM skills to develop a trusting relationship with the student. The following paragraphs outline the most important classroom management strategies identified in the literature. Figure 1. The Teaching Pyramid. Reprinted from The Incredible Years Inc. by C. Webster- Stratton, 2020, www.incredibleyears.com. Copyright 2012 by Carolyn Webster-Stratton. Reprinted with permission. Positive student-teacher relationships. Creating positive relationships with students is considered the most basic foundational skill to promote positive outcomes for children (Webster- Stratton, 2012). Positive relationships with students are often characterized by warmth, positive 25 interactions, and positive attention (Jones & Jones, 2015; Korpershoek et al., 2016). Teachers may promote positive relationships through empathy, listening to a child’s story, playing with children during play time or recess, or asking a child about their weekend (Webster-Stratton, 2012). Employing relationship-building strategies demonstrates significant effects on classroom behavior. Specifically, a meta-analysis conducted by Marzano, Marzano, and Pickering (2003) found that students who had a positive relationship with their teachers had 31% less behavioral problems than students who did not have positive relationships with teachers. Additionally, a study by Copeland-Mitchell, Denham, and DeMulder (1997) found that positive student-teacher relationships tend to compensate for impairments in parent-child relationships. Praise and incentives. Another foundational evidence-based strategy is the use of praise and incentives. Behavioral researchers indicate that behavior specific praise that explicitly describes a behavior (e.g., “Good job sitting in your seat”) can increase a desired behavior in the classroom (e.g., sitting in seat; Jones & Jones, 2015; Webster-Stratton, 2012). Research supports the use of behavior specific praise to improve classroom behavior (Allday et al., 2012). For instance, Allday and colleagues (2012) conducted a single-case design examining the use of behavior specific praise with children diagnosed with emotional behavioral disturbance. They found at post-treatment teachers reduced their own use of corrective statements and students increased their on-task behavior. Additionally, teachers can provide incentives to increase the desired behavior through tangible rewards of the students’ preference (e.g., a teacher may pass out stickers for students sitting in their seat to increase on-task behavior; McGoey & DuPaul, 2000). McGoey and DuPaul (2000) conducted a study using reinforcers for preschool students with Attention Deficit Hyperactivity Disorder (ADHD). Within the study, preschool teachers were trained to provide 26 students stickers and place stickers on a classroom wide sticker chart when they were “caught” being on-task. After a certain number of stickers, they were rewarded with a bigger sticker. This intervention technique has been cited in literature reviews as significantly improving disruptive and off-task behavior in the classroom (Doll et al., 2013). Rules, expectations, and setting limits. The third essential evidence-based strategy for preventing behavior disruptions involves creating clear rules, expectations, and setting limits (Korpershoek et al., 2016). Researchers indicate that teachers should employ three to five positively stated rules to prevent behavioral problems (Jones & Jones, 2015; Sprick, 2009). Specifically, these rules should be displayed at eye level and should be frequently taught and rehearsed within the classroom setting to prevent disruptive behavior (Sprick, 2009). Additionally, teachers should clearly state the expectations of the day through a daily schedule as this increases predictability of what students should expect each day (Jones & Jones, 2015). Finally, teachers should clearly state consequences of rule violations so students have a clear understanding of what will occur if students do not listen to behavior and role play these expectations to ensure an understanding (Webster-Stratton, 2012). Utilizing these strategies have been found to reduce the likelihood of behavioral problems in the classroom (Simonsen et al., 2008) and promote executive functioning and language development skills which has been linked to improved self-regulation (Ursache, et al., 2012). Social-emotional learning. Including components of social-emotional learning (SEL) strategies such as providing direct instruction in problem solving skills, training students to use emotion regulation strategies, and using coaching to scaffold academic and behavior skills, are essential in promoting positive child outcomes and reducing disruptive classroom behavior (Bierman & Motamodi, 2015; Domitrovich et al., 2007; Webster-Stratton, 2001). 27 Problem-solving involves presenting students with a problem scenario and identifying alternative solutions to address the problem (Bierman & Motamodi, 2015). Teaching problem solving skills have demonstrated improved performance in academic and social-emotional success (Bierman et al., 2008; Bierman & Motamodi, 2015). Specifically, Bierman and colleagues (2008) found that when including a problem-solving component into the Head Start REDI program, it reduced children’s aggression. Training kids how to regulate their emotions also helps to improve child behavior (Webster-Stratton, 2012). Specifically, direct instruction using role-playing or puppets to practice how to control emotions has been found to significantly reduce aggressive behavior (Williams, Bywater, Lane, Williams, & Hutchings, 2019). Finally, coaching involves training teachers to use descriptive comments to highlight specific learning skills (Webster-Stratton, 2012). Webster-Stratton (2012) describes four different types of coaching to improve child outcomes: (a) academic coaching which involves describing objects to improve language (e.g., “You have two yellow blocks”), social skill coaching which involves modeling social interactions to improve social skills (e.g., “I’m going to be your friend and share with you”), persistence coaching which helps children build resilience (e.g., “Diana is really concentrating on her work”), and emotion coaching, which involves labeling children’s feelings and emotions (e.g., “you look really excited to finish that project. Your whole body is bouncing!”). Coaching can expand vocabulary through modeling and naming of objects (Whitehurst et al., 1999; Smith & Jones, 2011), promote cooperative learning and increase skills in emotion knowledge and emotion recognition (Domitrovich et al., 2007), and improve self-regulation (Wilson et al., 2014). Research demonstrates positive effects on both teachers and children when using programs which include SEL (Durlak et al., 2011). Specifically, an RCT by Domitrovich, Cortes, 28 and Greenburg (2007) found that after implementing two preventative behavioral interventions which emphasized SEL, teachers’ beliefs and perceptions regarding burnout, self-efficacy, and social-emotional competence increased significantly more in the group that contained a social- emotional learning component. Furthermore, Durlak, Weissburg, Dymnicki, Taylor, and Schellinger (2011) conducted a meta-analysis of EBIs focused on TCM strategies that emphasize SEL. They found that compared to a control group, children demonstrated improvements in social-emotional skills, attitudes and behavior, and a gain of 11 percentile points in academic achievement. Home-school collaboration. Home-school collaboration involves establishing positive relationship with parents (e.g., notes home, phone calls), having a family-center approach, and understanding family cultural backgrounds to promote students’ social-emotional and academic needs (Jones & Jones, 2015). Teacher’s often report feeling insecure in their interpersonal skills with families (Webster-Stratton, 2012). However, training in home-school collaboration has been found to improve teacher confidence, improve child academic performance, promote consistency in behavior across home-school contexts, and prevent behavioral problems in the classroom (Jones & Jones, 2015). Individualized behavioral plans. A final strategy to promote positive classroom management is learning how to create behavior plans for students with more severe behavior. A survey study conducted by Flower, McKenna, and Haring (2017) found that most pre-service classroom management courses focused on how to use basic classroom management strategies (e.g., praise, developing rules, child-teacher relationships); however, few focused on how to reduce behavior that cannot be exclusively addressed by basic classroom management strategies. Thus, educating teachers on how to collect data, understand the functions of behavior, and how 29 to develop an individualized behavioral plan for children is essential to reduce disruptive behavior (Webster-Stratton, 2012). Research supports this approach as teachers who have been trained to develop behavior plans for students were more likely to have fewer misbehaviors within the classroom (Reinke & Herman, 2002; Reinke et al., 2014). Evidence-based training components to support TCM skills In order to change teacher behavior and support skill development, schools often provide in-service PD training to support teacher learning. PD training can range from teachers being assigned readings, attending single workshops, attending multiple session workshops, or providing on the job community practice (Stewart, 2014). Traditionally, schools have focused on passive approaches (i.e., reading or 3-hour workshops) that focus exclusively on content; however, these have been found to be ineffective in creating long-standing behavior change as teachers are more likely to go back to their own procedures if the skills are not practiced in their own setting and taught over a longer period of time (Desimone & Garet, 2015). To improve the effectiveness of PD, a growing research base has identified critical features to enhance teacher skills. These effective PD trainings are typically content-focused, include active learning approaches, are coherent, have a sustained duration, and include collective participation (DeSimone & Garet, 2015; Wei, Darling-Hammond, & Adamson, 2010. Additionally, several successful PD programs have a strong theoretical focus on changing teacher behavior to enhance student skills (Darling-Hammond et al., 2010). Research indicates that programs that incorporate these features find substantial gains in student achievement. For instance, May, Sirinidis, Gray and Goldsworthy (2016) conducted a study on the Reading Recovery program, which includes intensive PD for teachers that incorporates all critical features and found children performed nearly three times as well as students in the national average. 30 Manualized EBIs and group training. One method that may address the effective components in PD is manualized EBIs. Manualized EBIs can be beneficial for in-service PD training for teachers at-risk. Specifically, EBI protocols can include several sessions that outline goals and procedures (i.e., content-focused), activities that allow a teacher to practice skills both within treatment sessions and in the classroom setting (i.e., active learning), can be continuous and sustained overtime (i.e, scheduled sessions), and theoretically driven (e.g., emphasis Patterson’s social interaction theory; Kendall & Frank, 2018). One common approach to implementing manualized EBIs is within a group training format (Darling-Hamond et al., 2017; Desimone & Garet, 2015). Group formats were developed to allow several participants (i.e., parents or teachers) to receive treatment all at once, which can make treatment delivery more efficient and effective for a therapist or school psychologist (Taylor et al., 2008). Group training formats have been found to be beneficial because they can promote discussion among participants to help change misconceptions and beliefs (Webster-Stratton, 2012). Participants can model skills and behavior and practice role playing with others. In addition, peers can serve as a support system as participants are all experiencing the same problem (Rotheram-Borus et al., 2012; Webster-Stratton, 2012). Although group training can be beneficial, limitations also exist. For instance, typically group training sessions are not conducted unless there is a full session of people identified as needing support (Rotheram-Borus et al., 2012). Additionally, group training can be inflexible and place barriers on participants that do not have flexibility with their schedule or need more differentiated instruction based on personal experiences (Desimone & Garet, 2015; Rotheram-Borus et al., 2012). Coaching. Another PD model that contains several of these critical features includes teacher-coaching models. Coaching is defined as a supplemental PD practice where an expert 31 observes and provides performance feedback to teachers to help them change behavior and close skill gaps (Kraft et al., 2018; Reinke et al., 2012). Within the PD literature, coaching stems from the theory of action which indicates student behaviors will not improve without changes in teacher knowledge, skills, or practice (Kraft et al., 2018). Within the theory of action, Kraft and colleagues (2018) indicate training sessions or workshops have been found to be successful in increasing content knowledge of strategies while coaching has been directly tied to changing the behavior of teachers such as their increased implementation success of teaching practice or increased ability to identify teaching strategies to address a behavior problem. However, teacher workshops on their own can be seen as insufficient to address the complexity of issues that occur in the classroom (Desimone & Garet, 2015). Therefore, coaches are seen as an essential component to teacher behavior change within the theory of action and are often combined with group training or teacher workshops to enhance skill development (Kraft et al., 2018). Coaching also stems from a social constructivist learning theory (Vygotsky, 1978) which indicates the best learning occurs through creating relationships and interactions with others. Coaching utilizes several research-supported PD strategies that includes creating action plans, goal setting, reviewing skills, role-playing situations to practice skills, and providing performance feedback to ensure implementation fidelity (Kraft et al., 2018; Reinke et al., 2012). To enhance the effectiveness of PD and create teacher behavior change, Kraft, Blazer, and Hogan (2015) recommend coaching is individualized, intensive, sustained, context specific, and focused to support teacher learning. PD programs that include coaching components demonstrate effect sizes of 0.49 standard deviations on instructional practice (Kraft et al., 2018). Researchers also indicate coaches should use a collaborative approach and frequent praise to promote teacher acceptability and promote changes in behavior (Taylor et al., 2008). 32 Coaches are typically seen as skilled professionals that have a high level of expertise in TCM (Reinke et al., 2012). However, it is important for the coach to understand the teacher has a level of expertise as well (Raver et al., 2008). For instance, when coaches use a more collaborative approach, teachers feel more supported and are more likely to implement strategies as intended (Erchul & Martens, 2010). Additionally, praising teachers frequently for their efforts is crucial in building rapport and encouraging behavior change (Taylor et al., 2008). Coaches should also emphasize providing performance feedback during coaching sessions, which can help teachers generalize and maintain intervention implementation (Reinke et al., 2012). Delivering only one training session for teachers typically does not promote long-term success for intervention implementation (Garcia & Weiss, 2019; Kraft et al., 2018). To promote success, coaches should use performance feedback by directly observing a specific skill in an applied setting (e.g., using behavior specific praise) and providing feedback to the teacher on ways to ensure growth (Garcia & Weiss, 2019; Reinke et al., 2012). An extensive amount of research supports coaching. Specifically, Driscoll, Wang, Mashburn, and Pianta (2011) found teachers were 13 times more likely to implement an intervention with fidelity when provided with a coach to support implementation. Additionally, Stormont, Reinke, Newcomer, Marchese, and Lewis (2015) conducted a literature review and found that 83% of interventions that included a coaching component yielded positive results. Furthermore, teachers who receive coaching are more likely to maintain newly learned skills, implement an intervention with high fidelity, and report high levels of self-efficacy (Forman et al., 2013). Finally, Reinke, Stormont, Herman, and Newcomer, (2014) found that more performance feedback delivered to teachers resulted in higher levels of implementation of evidence-based teacher strategies over time. 33 Modeling. One specific aspect of coaching that is particularly important to highlight includes modeling. Bandura’s social learning theory (2001) indicates humans learn best by observing and imitating the actions of others. In terms of learning TCM skills, teachers may utilize modeling by observing a peer in the classroom, within a PD session, or serving as an intern in a classroom to practice teacher skills. However, in-service teachers at risk of poor TCM may need more time to observe and implement evidence-based TCM strategies than what are provided in the school context (Webster-Stratton et al., 2011). Thus, teachers may use video modeling to enhance teacher skills. Video modeling allows teachers to view several different types of TCM strategies to increase knowledge on teaching practices (Webster-Stratton, 1990). Video modeling can also be tailored to meet teachers’ unique needs. For example, video modeling can demonstrate both positive and negative TCM strategies, and teachers working with students from diverse cognitive abilities, developmental levels, behavioral skills, and cultural backgrounds (Webster-Stratton, 1990; 2012). An extensive amount of research supports the use of video modeling (Menting, de Castro, & Matthys, 2013; Nye et al., 2018). A study provided by Webster-Stratton (1988) found that when adding a video modeling component to a traditional parent management training intervention, video modeling enhanced parent skills compared to only using a group training program. Video modeling can also be tailored to reflect cultural backgrounds of teachers in different countries and ethnic backgrounds. For example, Baker-Henningham and Walker (2018) adapted a teacher training program using a video modeling component to reflect the cultural background of preschool teachers in a Jamaican context. The researchers found a significant increase in teacher use of positive strategies and reduction in negative TCM strategies. 34 Promoting Implementation Success of Treatment Programs A recent push in educational policy initiatives such as the Every Student Succeeds Act of 2015 (ESSA; P.L. 114-95), support the implementation of EBIs that promote successful classroom management. However, not many schools successfully administer EBIs due to challenges in the implementation process (Kratochwill & Shernoff, 2004). For instance, Webster-Stratton, Reinke, Herman, and Newcomer (2011) found even though teachers are often the main implementers of EBIs, they reported a lack of coaching and support when it comes to using interventions in the school context. Additionally, Forman and colleagues (2013) found one of the biggest barriers to implementation success, includes teacher buy-in and acceptability of evidence-based TCM strategies. Additionally, teachers and school personnel report time, resources, and cost as barriers to implementing interventions successfully. Thus, when considering EBIs, stakeholders should evaluate factors related to implementation success (Forman et al., 2013). Witt and Elliot’s model of treatment acceptability (1985) indicates acceptability, fidelity, use of treatment, and effectiveness have a reciprocal relationship and should be considered when determining if an intervention will be successfully implemented. Although effectiveness can help determine if the treatment worked to change the desired behavior, integrity and acceptability are essential in considering if the intervention is transportable in the school context. Research literature supports this conceptualization of successful interventions. In terms of treatment fidelity, adherence to treatment protocols (for teachers and trainers) and participant engagement with the materials (i.e., completing the recommended dosage of training) can all have significant effects on treatment success. Durlak and Dupre (2008) found that treatments 35 implemented with 60-80% fidelity have a high chance of improving effectiveness outcomes than interventions with lower percentages of fidelity. In terms of acceptability, Proctor and colleagues (2011) discuss that acceptability is often influenced by several different factors. Specifically, it can be influenced by the intervention format, contents of the program, the perceived effectiveness in improving behavior, time taken to plan the intervention, time taken to improve behavior, and the ease of delivering the intervention. Therefore, Proctor and colleagues indicate it is important to examine these different components as they can either reduce or increase implementation success and usability of an intervention. For instance, State, Harrison, Kern, and Lewis (2017) examined the feasibility and acceptability of interventions designed for emotional behavioral difficulties in high school students for 336 teachers. Specifically, they rated specific intervention based on priority, feasibility, and acceptability before implementation and after implementation. The researchers found that interventions that were rated as the most acceptable required the least amount of time and interventions that were time-consuming were rated as the least acceptable. Additionally, teachers cited lack of time, perceived lack of effectiveness, and poor environmental fit as reasons for lack of feasibility with a specific intervention. Another important factor related to improving EBI interventions is teacher beliefs of an intervention. Bandura’s social cognitive theory (2001) argues that underlying beliefs and cognitive perceptions are essential for behavior change to occur. When behavior change occurs this directly related to the effectiveness of an intervention. Research indicates that belief change can occur through professional development training from collaborative group discussions or through coaching. Specifically, a research study by Cook and colleagues (2015) found that training teachers in TCM strategies resulted in a significant increase in teacher perceptions of 36 EBPs from pre to post intervention. Additionally, these beliefs predicted successful implementation of the EBI strategies. Disruptive Innovations Self-administered treatment approaches. To address barriers for treatment in traditional group design, Rotheram-Borus and colleagues (2012) recommend disruptive innovations such as self-administered programs. Self-administered programs are programs that can be delivered individually without the need for an expensive certified group trainer. They are often seen as a flexible treatment approach as they can be delivered based on the client’s own treatment time and individualized to meet the client needs. Typically, self-administered treatments can be delivered in a bibliotherapy type format that may include a treatment manual with specific outline procedures, goals, and activities for participants to practice, readings to increase knowledge regarding behavior, reflection components. Additionally, self-administered treatments can be delivered through multimedia platforms (e.g., audio or video) to model positive behavior (Rotheram-Borus et al., 2012) and create a more interactive and engaging experience. Additionally, they may be designed to use exclusively as a self-administered treatment program, or they may be adapted from traditional manualized group training programs (Elgar & McGrath, 2003). Although most of the self-administered treatments are self-conducted, some treatment programs require an individual to meet with an expert such as a coach to discuss program components (Harwood & L’Abate, 2010). The most common types of self- administered formats discussed in the literature include bibliotherapy and multimedia interventions. Bibliotherapy. Bibliotherapy interventions are typically defined as self-help books or instructional manuals and designed for children, parents, or teachers to improve skills and 37 behavior (Harwood & L’Abate, 2010). Bibliotherapy can be a significant way to reach a majority of people to transport research into practice. For instance, Rotheram-Borus and colleagues (2012) indicated that in 2011, 13.5 million bought self-help books at a higher rate. Additionally, Harwood and L’Abate, (2010) indicated that using a self-help book that motivates and outlines a step-by step change format could have a large influence on a mental health treatment, reaching as high as 15% of the population. Because of increased ease of use, participants also may demonstrate evidence of improved acceptability and fidelity of implementation (Rotheram-Borus et al., 2012). For teachers, using a manualized step-by-step bibliotherapy manual may allow teachers autonomy to deliver an intervention on their own time and receive more direct instruction on new teaching practices (Louws et al., 2017). Multimedia. Multimedia assisted self-administered programs can also be administered to a parent or teacher to improve their own behavior. Because modeling is an important component for effective intervention success, multimedia techniques (i.e., DVDs, audio) can be a useful intervention delivery format to enhance evidence-based treatment strategies (Elgar & McGrath, 2003; Webster-Stratton, 2012). Elgar and McGrath (2003) indicated that despite their growth for potential, few multimedia products have been developed to improve parent, teacher, and child outcomes. Webster-Stratton, Kolpacoff, and Hollinsworth (1988) used parent training videos to train parents how to reshape their behavior and to improve their child’s behavioral difficulties. Results of this study found that parents who used the parent training videos were more effective in reducing externalizing behaviors of their children as parents in traditional group training without video modeling format. Despite the increased use of self-help books and manuals, these methods may be missing important PD components such as modeling and feedback which have 38 found to be most helpful in changing teacher beliefs and thus more long-term behavior change. Thus, adding a coaching component may help to address this issue. Coaching. Coaching can be an effective way to support teacher learning and implementation of training skills (Reinke et al., 2014). To address expense issues and lack of expert coaches, some research has highlighted the use of web-mediated coaching or a remote coaching as an effective approach to enhance teacher learning. For instance, Powell, Diamond, Burchinal, & Koehler (2010) examined remote coaching where educators shared video clips and coaches provided written feedback, video links, and other materials to support 16 coaching sessions over a semester. They found significant gains in student skill development compared to those in a control group. Research indicates that pairing coaching with a self-administered program can result in improved implementation and acceptability of program components (Reinke et al., 2014; Shernoff & Kratochwill, 2007; Taylor et al., 2008; Webster-Stratton, 1990). For instance, Shernoff and Kratochwill (2007) reported higher levels of teacher treatment acceptability following the self-administered Incredible Years Teacher Classroom Management training program (SATCM) with consultation compared to teachers in a self-administered only group based on rating scales and qualitative interviews. Similarly, Webster-Stratton (1990) found that parents who received additional supervision from a self-administered parent training intervention had higher acceptability ratings than those who received the self-administered only program. The Incredible Years One evidence-based manualized intervention that includes evidence-based teacher training strategies, coaching, and multimedia assisted material is the Incredible Years Teacher Classroom Management program (IY-TCM). The IY series is comprised of three interlocking 39 empirically supported programs targeting children, parents, and teachers (Webster-Stratton, 2001). The aim of this series is to prevent, reduce, and treat behavioral problems and promote social-emotional, behavioral, and academic success for young children. Although first viewed as a multi-systemic program to address child disruptive behaviors, recently, the IY-TCM program has been studies as a stand-alone program (Korest & Carlson, 2020). The IY-TCM program. The key mechanism of the IY-TCM program is training teachers in positive TCM strategies to reduce classroom behavioral problems (Webster-Stratton, 2001). The IY-TCM program series is rooted in both social learning interaction theory (Patterson, 1982), attachment theory (Bowlby, 1982), and social learning theory (Bandura, 1985). In the group format, 1-2 certified and trained group facilitators lead teachers in group discussions and activities. Within this group setting, teachers have opportunities to role-play, receive feedback from facilitators, self-reflect, view video scenes of teachers working with children, and engage in group discussions on ways to problem-solve and develop ideas for reinforcing children’s behaviors (Webster-Stratton, 2012). Furthermore, teachers have opportunities to practice and reinforce newly learned skills via handouts, assigned activities and readings they received during the training session (Webster-Stratton et al., 2008). IY-TCM program components. The IY-TCM program developers recommend delivering the program in monthly, 5-6, full day (7-hour) sessions led by a trained group facilitator. In between sessions, a coach is recommended to visit teachers and provide feedback and support on skill implementation (Webster-Stratton et al., 2011). The facilitator follows a step-by-step manual that includes six lessons focused on research-based teacher classroom management practices:1) building positive relationships with students and the proactive teacher, 2) teacher attention, coaching, encouragement, and praise, 3) motivating students through 40 incentives, 4) decreasing inappropriate behavior—ignoring and redirecting, 5) decreasing inappropriate behavior-follow through with consequences, and 6) emotional regulation, social skills, and problem-solving training (Webster-Stratton, 2012). To train teachers, the developers use the Teaching Pyramid as an analogy for TCM practices to help teachers conceptualize TCM training strategies (see Figure 1). Additionally, the pyramid serves as a roadmap for IY-TCM program delivery (Webster-Stratton et al., 2011). Specifically, Webster-Stratton (2012) explains the bottom of the pyramid (i.e., Session 1), should be applied first and the most liberally to prevent unwanted child behavior and improve positive child outcomes and discipline strategies should be performed last. Promoting fidelity and acceptability of the IY-TCM program. Webster-Stratton and colleagues (2011) indicate the fidelity and acceptability of the program is emphasized as an important component of treatment effectiveness. Specifically, they conceptualize treatment fidelity as threefold: (a) treatment adherence (participants adhere to recommended sequence and dosage), (b) skill level of trained interventionist in using IY-TCM program components, and (c) implementing the program for whom the program was designed. To adhere to fidelity, the IY- TCM program is evaluted through a six-step model developed by Webster-Stratton (2001), which includes session adherence protocols, session process checklists, final client evaluations, certification, and clinician evaluations of supervision. Additionally, Webster Stratton and colleagues (2011) indicate acceptability is inherent to achieving high fidelity. Thus, they also incorporate, attendance, low dropout rates, and client satisfaction based on a teacher questionnaire as a measure of treatment fidelity. Webster-Stratton and colleagues (2011) explain the IY-TCM program uses a principle- driven framework which allows this program to be replicable and to measure fidelity. However, 41 the researchers also explain that by using a principle-driven framework, the program can be tailored to meet the needs for a particular group. The developers define their principal-driven framework as having a program structure (i.e., big ideas, DVDs, and book) and that flexible implementation is created through reciprocal interaction between a trained facilitator with extensive knowledge on the program and the cultural context and background experience of the teachers. For example, new teachers, as well as teachers in a Head Start preschool classroom, paraeducators, or afterschool care workers, may have less experience and knowledge in classroom management training than seasoned teachers and general elementary school teachers (Reinke et al., 2014; Webster-Stratton et al., 2011). As a result, the program, allows the facilitator to slow down the pace, provide more support, and spend more time on certain content areas where teachers need additional practice. Additionally, teachers set the goals of what they want the lessons to focus on with the group leader at the beginning of the sessions, which helps promote acceptability and buy-in of the program (Webster-Stratton et al., 2011). Research on the IY-TCM. The IY program has recently been examined as a stand-alone program to promote TCM skills as a mechanism of change for child behavior (Korest & Carlson, 2020). Specifically, there have been over 30 studies conducted (Korest and Carlson, 2020) and it is identified as Possibly Efficacious on the Blueprints Program (Center for the Study and Prevention of Violence, 2018). Several articles have examined the fidelity of the group training program through quality of the trainer (i.e., a certified IY-TCM training) or dosage of treatment (i.e., attendance of the group training). Outcomes of treatment adherence ranged from 58% to 95% (Baker-Henningham & Walker, 2018; Ford et al., 2019; Hickey et al., 2017; Hutchings et al., 2013; Murray, Rabiner, 42 & Carrig, 2014, Raver et al., 2008), indicating below acceptable fidelity to high fidelity (i.e., above 80%; Durlak & Dupre, 2008). Two meta-analysis examined the effectiveness of the IY-TCM group training program. Nye, Mellendez-Torres, and Gardner (2018) conducted a multilevel meta-analysis with both quantitative and qualitative results to examine the current state of research for nine RCT studies. They found a moderate to large effect on positive (g = 0.73) and negative (g = 0.49) teacher strategies, a small effect on child prosocial skills (g = 0.12), and a small to negligible effect on child conduct problems (g = 0.05). Korest and Carlson (2020) conducted a multivariate meta-analysis examining the current state of evidence of the IY-TCM program. Specifically, they evaluated 16 studies on teacher and child outcomes examining both observation and self-report measures. The results of this study indicated a moderate effect on positive teaching strategies (i.e., praise, clear rules and expectations; g = 0.70) and negative teaching strategies (i.e., harshness, unclear expectations, demands; g = 0.50). Additionally, child prosocial skills (e.g., social skills, friendship skills; g = 0.19 - 0.21) and child externalizing behavior (e.g., conduct problems, hyperactivity; g = 0.14 - .16) had a small effect. Four published studies have examined acceptability of the group training program using a rating scale or qualitative interviews evaluating teacher’s satisfaction of the program overall, acceptability of program contents, strategies used, acceptability of the group leader, and evaluation of the video series (Fergusson, Horwood, and Stanley, 2013; Hicks-Hoste et al., 2015; Hutchings et al., 2011; McGilloway et al., 2011). From the data provided, acceptability ratings were high (i.e., over 90% rated positive or very positive for content) or adequate (had a score at or above the midpoint on a Likert scale), for the strategies, group leader, and satisfaction of the 43 program overall. Even though most articles reported using the Teacher Workshop Satisfaction Questionnaire (TWSQ), researchers used the TWSQ differently to summarize acceptability scores. For instance, Fergusson, Horwood and Stanley (2013) reported on all individual items on the survey and summarized by percentage of participant ratings. Hutchings and colleagues (2007) summarized all four subscales through mean overall ratings. McGilloway and colleagues (2011) and Hicks-Hoste, Carlson, and Tiret (2015) only reported specific items from one subscale. None of the articles reported data on the acceptability of the video content. 44 Table 3. Summary of Published Studies on the Acceptability of IY-TCM Group Training Study Study Design Measure/Analysis Acceptability Outcomes Fergusson et al. Pre-post design. IY-TCM • TWSQ measured at post- Over 90% rated as positive (rating of 5-6) or (2013) program implemented to treatment very positive (rating of 7) for all four subscales. 297 teachers in New • Likert scale from 1-7 (very Specific Items Zealand negative to very positive) • 68.8% were optimistic and 26.6% were “very • Summarized outcomes of all optimistic” on their expectations for good items based on percentage of results from the workshop participant ratings • 39.8% would “recommend” and 55.5% would “strongly recommend” the program • 37.5% “liked” the trainer and 60.9% liked the trainer “very much” Hutchings et al. Pilot study control and • TWSQ measured at post-test • usefulness of the program rated very useful (2007) treatment group. • Likert scale from 1-5 (not (M = 4.5), Measured acceptability useful to very useful) • confidence in delivering the program rated of IY-TCM with 20 • Found mean average of each as very easy (M = 4.6), preschool teachers in four subscales in TWSQ • ease of putting the program into practice treatment group in Wales rated as very easy (M = 4.6) • use of strategies to improve home-school links rated as neutral to satisfied (M = 3.5). McGilloway et Small RCT design. • Measured teacher • All teachers found the program to be al. (2011) Evaluated 22 preschool acceptability at post treatment “appropriate” or “very appropriate” teachers (11 in IY-TCM) using an adapted TWSQ and • 64% of teachers said they would from 11 schools in qualitative information. No “recommend” the program to another teacher Ireland. information about the tool or • All teachers reported they felt “more how it was adapted was confident” in managing behavior problems reported. • 73% of teacher rated their overall impressions of training as “positive” or “very positive” 45 Table 3 (cont’d) Hicks-Hoste et Pre-post design. • TWSQ post treatment • 27% reported “liking” the training (score of al. (2015) Implemented adapted IY- • Reported outcomes of 5-6), and 72% reported “liking very much” TCM with 18 afterschool individual items of the (rating of 7) Americore workers in questionnaire • 55.6 % reported feeling “optimistic” and Michigan 38.9 % reported feeling “very optimistic” that the workshop will improve outcomes • 52.9% felt teaching techniques to change child behaviors were “appropriate” and 41.2 % of participants felt it was “very appropriate” • When asked if they would recommend to another trainer, 44% would “recommend” and 55.6% would “strongly recommend” 46 Additionally, research indicates success implementing the program with different at-risk populations of teachers. For instance, Hicks-Hoste and colleagues (2015) examined the IY-TCM group training program for after-school care workers (N = 18) in an elementary school using a pre-post pilot design. Their findings indicated that the IY-TCM program improved perceptions of positive classroom management strategies, confidence in managing future behavior problems, and resulted in a high level of satisfaction with the program and certified trainer. Baker- Henningham and Walker (2018) examined the IY-TCM group training program with paraprofessionals in Jamaica. Their findings also demonstrated significant improvements in positive classroom management strategies. Barriers in implementation of the IY-TCM group training program. There may be some implementation barriers when attempting to disseminate IY-TCM on a widespread scale. Even though the IY-TCM program has demonstrated positive outcomes, a lot of these studies had grant money and university resources to implement the treatment with fidelity (Ford et al., 2019; Hickey et al., 2017; Hutchings et al., 2013). Thus, it is difficult to tell if these programs would work equally well outside of well-funded treatment studies. Although the IY-TCM programs is founded in theory and evidence-based teacher training strategies, the group training is time consuming and costly, which may limit the integrity and acceptability of administrators to implement these programs in the schools (Forman et al., 2013). For instance, IY-TCM requires time intensive training (42 hours for teachers, and 3 days for a group leader to become trained), expensive program materials ($1,425 for the IY-TCM program), and lengthy monthly sessions (7 hours per session). Thus, many schools may lack the resources to implement this program and may lack qualified professionals to become a trained group leader. Even if the schools did have the resources to provide this training, schools would need to employ substitute teachers to allow 47 teachers to take time off or request teachers use their weekends to become trained. An additional barrier in the program is that it does not address the needs of teachers that may need more time on additional sections of the program. For instance, novice teachers or paraprofessionals that are new to TCM may need to spend more time implementing the training to improve skills (Webster-Stratton et al., 2011). For instance, Baker-Henningham and Walker (2018) found that when extending the group training program sessions from six to eight sessions, for paraprofessionals in Jamaican preschools, this significantly increased positive TCM strategies. Furthermore, novice teachers may need more consistent feedback and coaching to implement newly developed skills (Reinke et al., 2014). For instance, adding a coaching component to the group training program has been found to improve the implementation of TCM skills (Raver et al., 2008). The Self-Administered IY-TCM Program To address the barriers associated with the group training program, the IY program developers created a more flexible self-administered format for both the parent and teacher training programs (Webster-Stratton, 2012). Originally, the developers designed the self- administered program as an alternative to use when teachers were unable to attend the group training programs or to use as supplementary practice for teachers that were struggling to learn the concepts of the IY-TCM program (Webster-Stratton, 2009). However, to address the barriers of expensive group training, the self-administered program has been used for teachers that do not have time or resources available to attend group training sessions. The SATCM program is comparable to the group training program in terms of the manualized approach and treatment goals (see Table 4). 48 Table 4. Comparison Between IY-TCM Group Training, SATCM Training, and Book + Activity Only IY Series IY-TCM Self-Administered Program IY-TCM Group Training Format Format IY-TCM Book + Activities Program Lessons Workshop 1a: Building positive Manual 1: Building positive Session 1: Building positive relationships relationships relationships Workshop 1b: Preventing problem Manual 2: Preventing problem Session 2: Preventing problem behaviors behaviors behaviors Workshop 2: The importance of teacher Manual 3: The importance of Session 3: The importance of attention, coaching, and praise teacher attention, coaching, and teacher attention, coaching, and Workshop 3: Motivating children praise praise through incentives Manual 4: Motivating children Session 4: Motivating children Workshop 4: Ignoring and redirecting through incentives through incentives Workshop 5: Follow through with Manual 5: Ignoring and Session 5: Ignoring and consequences redirecting redirecting Workshop 6: Emotional regulation, Manual 6: Follow through with Session 6: Follow through with social skills, and problem solving consequences consequences Manual 7: Emotional regulation, Session 7: Emotional regulation, social skills, and problem solving social skills, and problem solving Program 42 hours, 6 full day group teacher Flexible. Recommended at least Flexible. Teacher can read and Timeline training sessions led by one-two trained 45 minutes to read training complete activities at their own facilitators held monthly across 6 materials, and have at least 1-2 pace months weeks to practice skills before moving on to next lesson Program Delivery Certified group leader (3-day training) Implemented by teacher Implemented by teacher individually individually 49 Table 4 (cont’d) Content • Delivered in monthly group • Delivered in self- • Delivered in Incredible Years Delivery/Program sessions by group trainer administered manuals Teacher Book and via online Components • Group Session Components: • Treatment Session activities from session 1-7 on o Check-in/questions from Components the Incredible Years Website previous session o Watch assigned • Treatment Session o Instruction from Group DVDs in manual Components leader on new content o Discussion questions/ o Read assigned o Watch DVDs as group self-reflections to readings o Have large group discussion promote critical o Complete activities o Small group discussion thinking o Have a discussion o Practice modeling skills o Have a discussion with peer to promote o Create goals with peer to promote collaboration • Homework assignments: Take 3-4 collaboration (optional) weeks to practice goals, complete (optional) o Create goals reading assignments, and complete o Create goals • Homework Assignments: homework assignments for next • Homework Assignments: Practice goals, homework session Practice goals, complete assignments for next session reading assignments, and complete homework assignments for next session • Check-in with coach after learning and practicing skills (optional) Program Group Leaders Group leaders: None • Book: $27.95 Costs/Other costs • Hire on-site group leader = $1650- • Worksheets: Free on website 2000/day plus airfare, lodging, and Trained Coach (optional): $690 travel for group leader training not • Group Leader Training = $690 not including airfare, lodging, travel, including airfare, lodging, travel, meals+ $200 minimum for group meals + $200 minimum for leader certification certification 50 Table 4 (cont’d) Materials Materials • Book: ($27.95)/teacher • Self-administered manual set: • Program materials (DVD, manual): $80 $1,425 • Book: $27.95 • Worksheets: Free on website • DVD set: $1,425 • Worksheets: Free on website Other: • Substitute teachers or overtime pay Other: • Space to hold training • No additional costs • Refreshments or food • A 7-hour time period to hold training 51 Specifically, the mechanism of change is still focused on training teachers to implement TCM strategies to improve disruptive classroom behavior. However, instead of using the group leader to deliver the treatment sessions, the self-administered program includes seven teacher manuals to help teachers learn the program content. Each treatment manual outlines treatment goals, DVDs to watch, book chapters to read, and homework assignments/activities to practice. To address the absence of group discussion, the developers included discussion questions for the teacher to answer after watching the treatment videos. Within manual and homework activities, they also encourage teachers to have a discussion with a peer after completing a video or completing discussion questions within an activity handout to enhance their knowledge on the topic (Webster-Stratton, 2009). Research on the Self-Administered IY Programs Exploration of the fidelity, effectiveness, and the acceptability of the SATCM program with coaching as a treatment for at-risk teachers compared to a Book + Activity comparison group has yet to be documented in the literature. Thus, most of the research used to support the SATCM program comes from the self-administered Incredible Years Parent Training program (SAPT). The studies below outline the fidelity, effectiveness, and acceptability research available on the SAPT with and without coaching and outlines two studies examining the SATCM program. Treatment implementation fidelity and treatment adherence of the SAPT Programs. Research on the treatment fidelity of the SAPT program with coaching and without coaching are inconsistent. Webster-Stratton and colleagues (1988) compared different formats of the Incredible Years parenting training with a group training only, a group training with video modeling, a self-administered video modeling, and a wait-list control group. To allow access to 52 treatment materials and compare findings to the group training program, parents in the self- administered group were required to come to the treatment clinic to watch the videos and complete the assignments. Fidelity was assessed through treatment dosage (i.e., attendance), quality of therapist service delivery, and therapist adherence to treatment. Results of fidelity revealed the parents in the self-administered training sessions had comparable attendance sessions (M = 10.1 for mothers in group training and M = 9.1 for mothers in SA training). To ensure quality of service delivery, therapists were required to engage in supervision, feedback, and training throughout the study. Additionally, the researchers required therapists to follow a treatment manual and keep detailed notes of each session. However, this data was not recorded. Webster-Stratton’s (1990) study stemmed from Webster-Stratton and colleagues (1988) study findings. Webster-Stratton (1990) conducted an RCT comparing the SAPT program (N = 17 mothers, 10 fathers), SAPT program with coaching (N = 16 mothers, 9 fathers), and a wait-list control group (N = 14 mother, 9 fathers). Participants received treatment in a clinic-based setting. Similar to Webster-Stratton and colleagues (1988), participants attendance records were recorded to measure fidelity of treatment dosage. Specifically, in the SAPT coaching group 14 mothers and 8 fathers had 100% attendance ratings with two families who dropped out. For the SAPT without coaching group, mothers attended all 10 sessions and 8 out of 10 fathers attended all 10 sessions. Additionally, certification of therapist training and quality of service delivery were used to ensure treatment fidelity. Specifically, the therapist had training with over 10 years of experience delivering treatment and was told to keep detailed notes of each session. Taylor and colleagues (2009) used a self-administered computer-based program with coaching to support 90 families. Coaches were used to mimic group leaders in the group training program, address parent problems with completing assignment, and assist with problems in 53 training techniques. A parenting forum was also used to mimic the collaborative approach of the group training program. Additionally, a reminder system was used to support parents to complete their homework and surveys. Similarly to Webster-Stratton and colleagues’ (1988) study, Taylor and colleagues (2008) measured quality of therapist service delivery by providing feedback to coaches to ensure fidelity of delivery; however, no data was reported. Taylor and colleagues did, however, measure participant adherence in the training components as a form of fidelity using a self-monitoring checklist. Findings indicated that 76% of participants completed more than half of the program and 63% of participants completed 100% of the program components. Walcott, Carlson, and Beamon (2009) conducted a single-case ABAB design comparing four parents with children diagnosed with ADHD. Similarly, to Taylor and colleagues (2008), they focused on parental adherence by assessing the percentage of treatment manuals parents completed as well as the skills actually practiced by parents. They found that the percentage of manuals parents completed ranged from 40%-79% and parents treatment fidelity scores for skills practiced ranged from 79%-96%. Ogg and Carlson (2009) also evaluated treatment adherence of the program components using an integrity checklist. Specifically, they evaluated the percentage of video tutorials completed and the percentage of workbooks completed. Ogg and Carlson found that most of the participants completed 92% or more of the video tutorials indicating a high level of integrity. The researchers reported that participants demonstrated a more difficult time with workbook completion which ranged from 0-93%. Osburn (2009) measured treatment fidelity using a self-report questionnaire. They found that treatment fidelity was relatively high, but it decreased over the four treatment sessions from 78% to 68%. Effectiveness of the SAPT program. The outcomes of these studies provide some evidence on the effectiveness of the training program; however, it is unclear if coaching adds to 54 the effectiveness of the program. Although effect sizes were not calculated from this study, Webster-Stratton et al., (1988) found observation scores indicated greater parent training improvements in the group training with coaching group (pre M = 20.8, SD = 15.3; post M = 13.8; SD = 11.9) than the group training only, SAPT group, and control group. However, they indicated the SAPT group means were only slightly lower (pre M = 16.5, SD = 9.6; post M = 12.8; SD = 0.2), which the researchers posited might make the SAPT program more cost- effective. Additionally, Webster-Stratton and colleagues (1988) found improvements in the SAPT program compared to a control group. Specifically, observations of mother behavior revealed mothers in the SAPT group were less critical than mothers in the control group at post treatment. Self-report questionnaire data revealed mothers in the SAPT group reported improved positive interactions with their child. No specific data was reported for either of the observation or self-report outcomes. Although SAPT was found to have comparable results with the group training program, Webster-Stratton (1990) wanted to see if adding a coaching component would improve parent outcomes while maintaining the feasibility of the self-administered format. Specifically, the researcher hypothesized that adding a coach would mimic the effects of the group leader in the group training program and allow parents to seek consultation to improve skill generalization. Webster-Stratton (1990) found evidence for both the SAPT program with coaching and without coaching, based on the Dyadic Parent-Child Interaction Coding System (DPICS; Robinson & Eyeburg, 1981), which involves observing parent and child behavior. Although no data was reported, Webster-Stratton (1990) stated that when comparing the SAPT with and without coaching, both treatments reduced negative parenting behavior compared to a control group. When comparing the two groups, the only significant difference found on parenting behaviors 55 was that mothers in the coaching group demonstrated significantly fewer commands after treatment (Pre: M = 23.93, SD = 21.9; Post: M = 13.78, SD = 7.5) than the self-administered only group (Pre: M = 23.44, SD = 22.3; Post: M = 20.62, SD = 17.3). No effect sizes were reported on these measures. Webster-Stratton noted coaching may have only slightly added to the effectiveness of the outcomes due to only adding two additional hours of coaching to the program. Taylor and colleagues (2008) measured goal setting as their outcome measure of effectiveness using a goal setting scale, which measured the frequency of current behavior, expected behavior goals, and achieved behavior goals. Based on their reporting, families made at least 50% of progress on at least one goal. In contrast, Walcott and colleagues (2009) results indicated 2 out of 4 parents improved in only one positive reinforcement technique (increased .5 or more on the 7-point scale) based on the LIFT Positive Parenting Interview (available at incredibleyears.com). However, it is important to note that Walcott and colleagues examined children between the ages of 7-12 which may have affected their results as research indicates that parenting interventions (and more specifically the Incredible Years program) are most effective for children between the ages of 3-8 and program strategies should be modified to meet the developmental needs of older children (Webster-Stratton, 1990). Osburn (2009) conducted a single-case repeated AB design examining the acceptability, fidelity, and effectiveness of treatment on parent skills and children’s externalizing and internalizing behaviors. Using a self-report measure, Osburn found a statistical difference from pre to post in parents increased use of setting rules and expectations, use of praise, use of time outs, and knowledge on their child’s friends based on the Parent Practices Interview. No effect sizes were reported for this study. 56 Ogg and Carlson (2009) conducted a single case AB pre-post design over 10 weeks examining the perceived effectiveness, integrity, and acceptability of the SAPT program for children with ADHD. Using the LIFT Parent Practices Interview (www.incredibleyears.com), Ogg and Carlson (2009) found parents reported a decrease in most negative parenting practices and an increase in positive parenting practices. Specifically, there was an 18% change in mean scores from baseline to treatment in harsh discipline, a 10% change in inconsistent discipline, a 23% change in positive parenting, 11% change in clear expectations, and a 7% change in monitoring child behaviors. However, there was not a decrease in appropriate discipline used by parents. No effect sizes were reported for parenting practices. Acceptability of the SAPT program. Research indicates adequate acceptability for both the SAPT program both with coaching and without coaching with some research to support coaching may have slightly higher acceptability scores than the self-administered only program. Webster-Stratton and colleagues (1988) measured acceptability of parents’ perceptions of child behavior improvement, format treatment difficulty (e.g., videos), treatment usefulness, and overall difficulty of parenting skills taught. Ratings were gathered from the Consumer Satisfaction Rating Scale (CSR; Forehand & McMahon, 1981), a Likert scale from 1-7 (very negative to very positive). Ratings indicated that the implementation and perceptions of child improvements were not as high as the group training or group training plus video modeling treatment. No specific acceptability data was reported in this study. Webster-Stratton (1990) also examined participant acceptability using the CSR. Acceptability ratings indicated parents in the treatment groups had “high scores” on the CSR; however, no specific data was reported in this paper. 57 Ogg and Carlson (2009) measured content acceptability, acceptability of procedures, perceived effectiveness, and acceptability of time-taken to improve behavior. Content acceptability was measured through the Video Evaluation Questionnaire-Parent Form (VEQ-P; Webster-Stratton, 2001) provided in the Incredible Years program materials, a Likert scale from 1-5 (unhelpful to very helpful) which evaluates the videos included in the program. Results of the study revealed that the VEQ-P had an average rating of 4.2 for “promoting positive behaviors” and 4.3 (i.e., helpful to very helpful) for “reducing negative behaviors” while “supporting your child’s education” was rated as a 3.2 (i.e., neutral to helpful). The researchers also evaluated treatment acceptability using the Treatment Evaluation Questionnaire-Parents (TEQ-P; Kelley, Heffer, Gresham, & Elliot, 1989) which includes a Likert Scale ranged from 1-6 (strongly disagree to strongly agree), and examines the acceptability, effectiveness, and time- taken to improve behavior. On the TEQ-P; the average parent rating was a 4.6 (overall score~96.6), which was considered an adequate rating of acceptability. Osburn (2009) also measured acceptability using the TEQ-P (Kelley et al., 1989). The researchers defined adequate acceptability as a score above the midpoint of the total score on the TEQ-P (i.e., 73.5). Based on the treatment score of four parents, the median treatment score was 96.6 which surpassed the midpoint treatment scale score and indicated adequate treatment acceptability. Stewart and Carlson (2010) also measured the acceptability of the Incredible Years training video series using the VEQ-P (Webster-Stratton, 2001) and the TEQ-P (Kelley et al., 1989). Specifically, they collected parent acceptability ratings biweekly for 8 weeks with 30 parents. Mean scores of 12 or more on the VEQ-P form and mean scores of 73.5 or more on the TEQ-P was identified as the midpoint of acceptability scores indicating adequate acceptability. 58 Results of this study indicated the VEQ-P had an aggregate mean score of 15 (SD = 3.5) and the TEQ-P scores indicated an aggregate mean score of 100.73 (SD = 11.69) which are both well above the midpoint indicating adequate acceptability. Taylor and colleagues (2008) used items from the Parent Satisfaction Questionnaire (www.incredibleyears.com), which includes a Likert scale from 1-7 (very negative to very positive), to examine overall treatment satisfaction. Based on the subscales reported, 89% of the participants felt positive or very positive about the program, 93% would recommend the program, 76% felt confident about managing of child behavior problems, and 80% felt confident about managing future behavioral problems. The self-administered IY-TCM (SATCM) program. Two studies to date address the SATCM program. Kratochwill, Elliot, Loitz, Sladeczek, and Carlson (2003) conducted an RCT examining conjoint SAPT and SATCM with added conjoint behavioral consultation. Specifically, the researchers randomly assigned Head Start teachers and parents to a videotape modeling treatment group, a manual only group (included a manual for parents on how to address externalizing and internalizing behavior problems), and a no treatment control group. Fidelity was measured through a self-report checklist of completed treatment activities. Based on their findings, teachers and parents reported 69% compliance with treatment activities. Effectiveness was measured through goal attainment, observations, and self-report. Results concerning the effectiveness of the treatment indicated 75% of parents in the in SAPT group (compared to 75% of parents in the manual only group) and 95% of teachers in the SATCM group (compared to 60% of teachers in the manual only group) demonstrated progress towards goals. However, no significant group effects were found. Acceptability was measured using three subscales from the TEQ adapted measure for parents and teachers. Results of the study revealed 59 both parents and teachers in the videotape group reported high rates of acceptability (Teachers: M = 71; score range = 14 to 82; Parents: M = 55; score range = 11 to 66), effectiveness (Teachers: M = 33; score range = 8 to 48; Parents: M = 36; score range = 8 to 48), and amount of time for improvement (Teachers: M = 8; score range = 2 to 12; Parents: M = 9; score range = 2 to 12). However, these scores were the same or only varied slightly from the manual only treatment group. Additionally, social validity of consultation was recorded for caregivers using the Consultation Services Questionnaire and found both teachers and parents rated the consultation as helpful. However, no significant differences between groups were reported. Shernoff and Kratochwill (2007) was the only study identified that exclusively examined the SATCM program. Specifically, these researchers randomly assigned eight teachers to a self- administered teacher training group or the self-administered program plus behavioral consultation group to compare the fidelity, effectiveness, and acceptability. Fidelity was not reported for this intervention. Treatment effectiveness/treatment acceptability was measured using a self-report questionnaire using the Teacher Strategies Questionnaire (Webster-Stratton, 2011). Specifically, they used the Teacher Confidence subscales, which uses a Likert scale ranging from 1 to 7 (not confident to very confident) and the Proactive Teacher Strategies subscales 1 to 5 (rarely/never to very often). Researchers reported that participants in the SATCM with consultation group had higher confidence ratings than the SATCM only group. Specifically, the researchers found that the coaching group felt neutral/not sure (M = 4.0; SD = 1.15) at baseline and felt confident to very confident (M = 6.6; SD = 0.48) at post-intervention compared to the SATCM only group who felt neutral to somewhat confident at baseline (M = 4.6, SD = 1.38) to somewhat confident to confident at post intervention (M = 5.5; SD = 0.58). These findings were statistically significant and demonstrated a medium effect size (!! = .64). 60 For Proactive Teaching Strategies, the researchers found that teachers in the SATCM with coaching group (M = 3.7; SD = 0.14) had increased proactive strategies more than SATCM only teachers (M = 3.4; SD = 0.19). These results were statistically significant and demonstrated a medium effect size (!! = .59). To measure acceptability, Shernoff and Kratochwill (2007) used the Treatment Evaluation Interview (TEI; Kazdin, 1980), which measures overall procedural acceptability, effectiveness, and ethical acceptability. The TEI has participants rate items using a Likert scale from 1 to 5, with higher scores indicating higher acceptability and a midpoint score of 27. Results indicated that both groups of teachers found the program highly acceptable (i.e., greater than the midpoint score). However, teachers in the consultation group were slightly higher ranging from 35-42 compared to the self-administered only group which ranged from 27-35. They found the between group comparison of post mean scores was statistically significant and demonstrated a medium effect (!! = .49). Pilot Study to Assess Fidelity, Effectiveness, and Acceptability Rationale for a feasibility study. Based on the initial success of the IY self-administered interventions for parents and limited research published on teachers, Sheridan (2014) would indicate the next step of the research trajectory is conducting a pilot study to examine the feasibility of an intervention in a school setting. Specifically, Sheridan (2014) suggests that examining components of feasibility (i.e., fidelity and acceptability of an intervention) are the first steps before evaluating a treatment with intensity and precision. Designing a pilot study. Attempts to identify appropriate research design standards that align with feasibility studies suggested a gap in the current literature (i.e., an overarching term that includes pilot studies; Bowen et al., 2009; Eldridge et al., 2016). However, Bowen and 61 colleagues (2009) have developed a set of guidelines for researchers which address how to design a pilot study that aims to assess interventions. First, they recommend having a strong rationale for developing a pilot study which emphasizes limited or no previous research published on the current topic. Next, they suggest researchers narrow an area of focus. Specifically, they outline eight areas of focus: acceptability, demand, implementation, practicality, adaptation, integration, expansion, and limited efficacy testing. Because this study attempted to understand how an intervention with limited research can work with a new population (e.g., teachers at-risk) using a novel delivery format (self-administered), the best fitting areas of focus for this study included acceptability, implementation, and limited efficacy testing. Finally, Bowen and colleagues recommend using a research design that aligns with the area of focus based on the three intervention developmental phases: Can it work? (i.e., is there some evidence that the intervention might work), Does it work? (i.e., is there some initial evidence of efficacy suggesting that the intervention might work compared to other practices), and Will it work? (i.e., will the intervention be effective in real-life contexts, settings, and cultures/populations that might adopt the intervention in practice). Because there is some initial support from the parent training program (i.e., Webster-Stratton [1990] conducted an RCT examining the SAPT with coaching and without coaching), Bowen and colleagues would recommend utilizing a small-randomized experimental design or pre-post design to evaluate treatment outcomes for the proposed study. Bowen and colleagues suggest researchers can be creative and combine these research designs to evaluate their areas of focus and use a mixed- methods approach. Additionally, researchers recommend adding in a qualitative component such as an exit interview which may help to provide a deeper understanding of implementation 62 strategies, barriers and facilitators of the intervention programs (Shoonenboom & Johnson, 2017). Randomized Experimental Pilot Designs Bowen and colleagues (2009) indicated that experimental designs are often underutilized in the feasibility literature, but compared to cohort designs, they can be a cost-effective and time sensitive way to determine if an intervention could work and is feasible before implementing a full-scale design. Additionally, through random assignment, pre-post analysis, and including a comparison group, this design significantly improves internal validity—ruling out a number of alternative explanations for significant results. Small-scale experimental designs with comparison groups that examine the fidelity, effectiveness, and acceptability of an intervention are often described as randomized pilot studies (Eldridge et al., 2016). Eldridge and colleagues (2016) describe a randomized pilot study as a type of feasibility study that is piloting the trial process to see if an intervention will work on a larger scale. No set guidelines are currently available for randomized pilot designs, thus, there are a variety of suggestions regarding sample size and effect size estimations. Whitehead, Julious, Cooper, and Campbell (2016) discuss the aims of pilot studies differ from a main or full trial. For instance, pilot trials are often performed to see if an intervention is feasible to conduct before implementing full trials. A pilot study can also be used to estimate the effect size to plan for a full-scale trial; therefore, calculating sample size through a power analysis is not necessary. Regardless, researchers should determine a justification for the size of their sample. Some researchers recommend using rules of thumb to determine their sample size (Julious, 2004). For instance, Julious (2004) recommends a sample size of 12 for each group for a pilot study. Julious indicates that sample size justification for a pilot study should be 63 determined based on feasibility or precision about the means and variances. In terms of feasibility, he explained that a sample size of 12 is an even number and a round number, making it easier to create equal sample sizes or block sizes. In terms of a justification for precision about the means and variances, Julious assessed the gain in precision of each unit increase in a finite sample to determine at what sample size are gains less pronounced. To assess a gain in precision, he used the right side of a confidence interval expression which was estimated using unit variance and a two-sided 95% confidence interval. Specifically, he found marked gains in precision and reduced variance when adding participants to a sample, but when he reached 12, he noticed slower gains. Despite this rationale for small sample size, there are limitations to this approach such as increased risks for type I and II errors. Thus, Lancaster, Dodd, and Williamson (2004) state that results of a pilot study should be treated as preliminary and interpreted with caution. Effect size calculations. Effect size can be a useful way to help readers understand the magnitude of differences found between treatment groups, standardize data, compare effects to previous research, provide practical significance effects of an intervention, and to calculate sample size for a main trial (Sullivan, 2012; Ferguson, 2010). As mentioned previously, only calculating statistical significance in a small-n trial could result in an increased risk of type I or type II errors as a result of the underpowered study (Sullivan, 2012). Thus, only reporting a p- value is not enough. Because the effect size is independent of sample size this helps provide researchers a deeper understanding into the research results as statistical significance does not predict effect size and has a truer magnitude of effect (Sullivan, 2012; Ferguson, 2009). For instance, if a potentially meaningful effect size is detected in a study, but the results were not 64 statistically significant, this could suggest that statistical power was a possible or likely issue and could be addressed by planning a larger, fully powered RCT. Research Questions and Hypotheses Using a pre-post randomized experimental design with a Book + Activity comparison group, the purpose of the current research was to compare the treatment fidelity, effectiveness, and acceptability of the SATCM treatment group to a Book + Activity comparison group. Because the program was self-administered, the participants had the flexibility to view the materials at home or school context on their own time. This method proved especially helpful during school closures and social distancing requirements associated with COVID-19. To reflect the group training components of the IY-TCM group training program, participants received a coach to support skill implementation bi-weekly for 30 minutes to provide feedback and support. Participants received training over a 12-week time period and had approximately two weeks to view each manual and/or program materials to practice their newly learned skills in each session. For the Book + Activity comparison group, the Incredible Years: Nurturing Children’s Social, Emotional, and Academic Competence (Webster-Stratton, 2012) was implemented similarly to using a self-administered only treatment. The program contents in the book directly align with the contents of the self-administered program and the author states the book can be useful as a standalone guide for teachers. Additionally, this book was considerably more affordable than the self-administered program materials (e.g., $27.95/book versus $1,425 for DVDs + $80 for self- administered manuals). Additionally, research assistants sent Book +Activity participants activities from the self-administered manual (found on the Incredible Years website) to engage in skill development and complete self-reflection questionnaires to align with the SATCM treatment group more closely. The Book + Activity comparison group did not have access to 65 DVDs or a coach. Instead, Qualtrics XM [survey tool] and two research assistants were used to provide reminders to complete surveys and read chapter contents to mirror a more manualized treatment approach. Participants for this study included 13 participants in the SATCM treatment group and 11 participants in the Book + Activity comparison group that met the inclusion criteria for needing support in TCM based on a screener. Question 1a. Was there a difference in treatment adherence between the SATCM treatment group and the Book + Activity comparison group as measured by an adapted Self- Monitoring Checklist? (Webster-Stratton, 2001)? The adherence of treatment fidelity in intensive and self-administered interventions is not well-documented in the literature. Taylor et al. (2008) conducted a treatment on the SAPT program and found that 76% of participants (N = 68) completed more than half of the program and 66% (N = 59) completed 100% of the program. It was hypothesized that the self- administered program plus coaching would have higher treatment adherence than Book +Activity group. Even though the Book + Activity comparison group received check-ins to support treatment adherence through an online reminder system using Qualtrics XM [survey tool], it was hypothesized that the face-to-face coach meetings would help motivate treatment adherence (Taylor et al., 2008; Webster-Stratton et al., 2011). Overall mean scores of 60% were considered adequate via the Teacher Self-Monitoring Checklist (Durlak & Dupre, 2008). Question 1b. Did the IY coach adhere to treatment principles of the SATCM program when supporting teachers biweekly as measured by an adapted IY-TCM coaching checklist (Webster-Stratton, 2001) at post-treatment? How often did raters (IY Coach and primary investigator) agree on items as measured by an adapted IY-TCM observation checklist during SATCM treatment sessions? 66 The adherence of coaching fidelity in intensive and self-administered interventions is not well-documented in the literature. IY-TCM program developers recommend coaches must first be trained as group leaders and have experience implementing the IY-TCM program with fidelity before becoming coaches. The IY-TCM group training program indicates high fidelity implementation from group leaders (>80%; Hickey et al., 2017; Leckey et al., 2016; Murray, 2017) when observed by IY trainers. There is no documentation in the literature regarding inter- rater agreement percentages between coaches and observers. Because the coach in this study was a previous group leader and has 20 years of experience implementing the program, it was hypothesized that the coach would implement the SATCM with coaching program with adequate fidelity. Overall mean scores of 60% based on the self-report IY-TCM coaching measure checklist indicated adequate fidelity (Durlak & Dupre, 2008). Additionally, because of the coach’s experiences, it was hypothesized there would be high inter-rater agreement between the observer and the coach. Inter-rater agreement percentage scores of 75% or higher will considered acceptable and inter-rater agreement scores of 90% or higher (Hartmann, 1977; Stemler, 2004). Question 2a. Does participating in the 12-week SATCM treatment group result in a self- reported increase in teachers’ perceptions of positive classroom management strategies as well as a decrease in negative classroom management strategies (i.e., Confidence in Managing Classroom Behavior, Total Positive Strategies, Inappropriate Strategies) compared to the Book + Activity comparison group (post-test differences controlling for pre-test) as measured by the TSQ? Research on the use of perceptions of TCM skills indicate positive findings for the group training program, but less research has reported perceptions of caregivers for the self- 67 administered programs. Group training studies using the TSQ found teachers’ perceptions of positive TCM strategies significantly increased following training (Carlson et al., 2011, Webster- Stratton et al., 2001) and teachers rated the TCM strategies as more useful compared to a control group (Hickey et al., 2017). Shernoff and Kratochwill (2007) also reported statistically significantly higher confidence rating in the consultation group, however, due to the small sample size in their study, they recommended interpreting this significance with caution. Additionally, Webster-Stratton (1990) found through qualitative interviews that parents in the SAPT with coaching group thought the therapist consultation supported their understanding of the program contents. Because the coaching component is supposed to emulate the group training program, it was hypothesized that the coaching group would demonstrate greater increases in perceived usefulness of the IY-TCM strategies and higher confidence than those in the Book + Activity comparison group. Using Shernoff and Kratochwill’s (2007) formula, adequate ratings of acceptability on the TSQ included anything at or above the midpoint. Overall scores above 3 for the positive subscales and 4 on the confidence rating subscales were considered adequate treatment effectiveness scores. Question 2b. Does participating in the 12-week SATCM treatment group result in a self- reported increase in teachers’ frequency of use of positive classroom management (i.e., strategies (i.e., Total Positive Strategies) as well as a decrease in negative classroom management strategies (i.e., Inappropriate Strategies) compared to a Book + Activity comparison group (post-test differences controlling for pre-test) as measured by the TSQ? The research has demonstrated positive results regarding the SATCM program with consultation. Webster-Stratton (1990) compared the SAPT group to an SAPT group with coaching group and found a significant difference with mother’s use of no-opportunity 68 commands favoring the SAPT with coaching group. Shernoff and Kratochwill (2007) reported significantly greater mean increases on the TSQ proactive teacher strategies subscale (M = 3.7) at post-treatment for participants who received the SATCM program with consultation compared to the SATCM only program (M = 3.4). Hickey et al. (2017) indicated a significant difference between participants in a wait-list control group versus participants in the IY-TCM group training. Given the weekly coaching, modeling, and feedback components provided by the SATCM training format (DeSimone & Garet, 2015; Wei, Darling-Hammond, & Adamson, 2010), it was hypothesized that the SATCM treatment group would demonstrate greater increases in using positive TCM strategies and greater decreases in negative TCM strategies (when controlling for pre-test differences) as measured by the TSQ than the Book + Activity comparison group. Using Shernoff and Kratochwill’s (2007) formula, adequate ratings of acceptability on the TSQ included anything at or above the midpoint. Overall scores above 3 for the positive subscales were considered adequate treatment effectiveness. Question 3a. Was there a difference in treatment acceptability scores between the SATCM treatment group and the Book + Activity comparison group as measured by the TEQ-T (i.e., intervention acceptability, effectiveness, and time to implement strategies)? It was hypothesized the SATCM treatment group would have higher levels of acceptability of treatment compared to the Book +Activity comparison group as indicated by the Treatment Evaluation Questionnaire for Teachers (TEQ-T; Kratochwill et al., 2003). Overall mean scores greater than 72.5 (i.e., the midpoint) were considered adequate levels of treatment acceptability on the TEQ-T. Shernoff and Kratochwill (2007) reported both teachers who received consultation and those that only received the SATCM group were well above the midpoint for treatment acceptability as measured on the TEI (midpoint score = 27). Additionally, 69 teachers in the consultation group were slightly higher ranging between 35-42 compared to the self-administered only group which ranged between 27-35 and found a significant difference between treatment groups. Using Shernoff and Kratochwill’s (2007) formula, adequate ratings of acceptability on the TEQ-T included anything at or above 3.5 for individual items and 73.5 for the overall scores. For subscale scores, adequate ratings of acceptability included anything at or above 38.5, 28, or 7 for acceptability, effectiveness, and time required respectively (Kratochwill et al., 2003). Question 3b. Does the SATCM Treatment group demonstrate similar levels of treatment acceptability based on previous IY research as measured by the Teacher Video Evaluation Questionnaire (i.e., DVD acceptability), the Teacher Workshop Satisfaction Questionnaire (i.e., coach, strategies, and techniques used), and teacher interviews (i.e., treatment and implementation barriers/facilitators, changes noticed in behavior)? It was hypothesized the SATCM treatment group would have similar levels of acceptability of treatment as indicated by the (a) Teacher Workshop Satisfaction Questionnaire (TWSQ; Webster-Stratton, 2001), (b), Teacher Video Evaluation Questionnaire (VEQ-T; Webster-Stratton, 2001), and (c) teacher interviews to previous research studies. Overall mean scores greater than the midpoint (i.e., M = 3.5) based on a Likert scale from 1-7 (neutral to above average) on the TWSQ were considered adequate levels of acceptability for the program content, coach, strategies, and techniques. Additionally, overall mean scores greater than the midpoint (i.e., 3) on the VEQ-T were considered adequate levels of video acceptability. Research on the group training program using the TWSQ measure indicates high acceptability. Group training studies revealed that 80-90% of participants rated teaching techniques, strategies used to teach the program, and overall program contents as useful or very 70 useful (i.e., 5-7 on rating scale; Fergusson et al., 2009; Hicks-Hoste et al., 2015). Additionally, Taylor et al. (2008) found that when implementing the IYPT program with coaching 93% of the parents reported they would recommend or highly recommend the treatment program to a friend (i.e., 6-7), 80% of the parents rated themselves as confident in managing current programs (i.e., 6), and 76% of the parents rated themselves as confident in managing future problems (i.e., 6-7). VEQ data is limited. The SAPT study by Ogg & Carlson (2009) revealed that the VEQ-P (a Likert scale from 1-unhelpful to 5-very helpful) had an average rating of 4.2 (i.e., helpful) for DVD 1: Promoting positive behaviors and 4.3 (i.e., helpful) for DVD 2 for Reducing negative behaviors while DVD 3: Supporting your child’s education was rated as a 3.2 (i.e., neutral). Similarly, Webster-Stratton and colleagues (1988) found that parents who received additional supervision had higher acceptability ratings than those who received the self-administered only program; however, no specific scores were reported. During teacher interviews, it was hypothesized teachers in the SATCM treatment group would have similar acceptability levels to the group training or other self-administered programs with coaching as receiving the SATCM treatment group addresses barriers associated with traditional group training programs. Qualitative interviews from other studies indicate that at teachers reported higher levels of treatment acceptability with consultation compared to teachers in the SATCM only group at post-treatment (Webster-Stratton et al., 1988). Furthermore, when providing feedback and support to help caregivers implement an intervention, caregivers reported more positive acceptability outcomes (Taylor et al., 2008). Finally, Hutchings and colleagues (2011) conducted qualitative interview with teachers who received IYTCM group training and cited 95% of teachers shared an improvement in their classroom behavior, 52% of teachers shared an improvement in their own behavior, 91% felt better equipped to handle target 71 children, and 33% felt calmer or less stressed. Interview results were reviewed and analyzed for general acceptability themes related to the TWSQ and TEQ subscales (Effectiveness, Acceptability) as well as identified treatment barriers that intensive and self-administered interventions attempt to circumvent. Qualitative analyses were informal as this data was meant to enhance the quantitative data of participant perspectives to inform future researchers. 72 Table 5. Research Questions, Hypotheses, Measures, and Data-Analysis Research Question Hypotheses Measure Data Analysis Question 1a: Was there a difference in • The self-administered • Teacher Self- • Descriptive treatment adherence between the SATCM program plus coaching may Monitoring analysis (average of treatment group and the Book + Activity have higher treatment Checklist (e.g., bi-weekly scores) comparison group as measured by an adherence than the Book + completed adapted Self-Monitoring Checklist? Activity comparison group homework as teachers had a coach to assignment, hold them accountable for completed session, their progress and receive practiced skills, feedback on their time to complete implementation skills. session) Question 1b: Did the IY coach adhere to • The coach will implement • Teacher-coach • Descriptive treatment principles of the SATCM program SATCM coaching with meeting form: Self- analysis (average of when supporting teachers biweekly as adequate fidelity (i.e., monitoring biweekly scores) measured by an adapted IY-TCM coaching >60%). checklist • Self-report checklist at post treatment? How often did • Inter-rater agreement will • Teacher-coach checklist (i.e., raters (IY Coach and primary investigator) be acceptable (i.e., > 75%) meeting form: >60%). agree on items as measured by an adapted observation • Inter-rater IY-TCM observation checklist during checklist agreement SATCM treatment sessions? percentage (acceptable: >75%, high: >90%) Question 2a: Does participating in the 12- • The SATCM treatment • Teacher Strategies • ANCOVA using week SATCM treatment group result in a group will demonstrate Questionnaire FIML in Lavaan’s self-reported increase in teachers’ higher perceived usefulness (TSQ) package in R studio perceptions of positive classroom ratings of positive TCM management strategies as well as a decrease strategies and lower in negative classroom management strategies perceived usefulness ratings (i.e., Confidence in Managing Classroom in negative TCM strategies Behavior, Total Positive Strategies, compared to the Book + Activity comparison group. 73 Table 5 (cont’d) Inappropriate Strategies) compared to the Book +Activity comparison group (post-test differences controlling for pre-test) as measured by the TSQ? Question 2b: Does participating in the 12- • The SATCM treatment • Teacher Strategies • ANCOVA using week SATCM treatment group result in a group will demonstrate Questionnaire FIML in Lavaan’s self-reported increase in teachers’ frequency higher frequency of use (TSQ) package in R studio of use of positive classroom management ratings for positive TCM strategies as well as a decrease in negative strategies and lower classroom management strategies (i.e., Total frequency of use ratings for Positive Strategies, Inappropriate Strategies) negative TCM strategies compared to a Book + Activity comparison compared to the Book + group (post-test differences controlling for Activity comparison group. pre-test) as measured by the TSQ? Question 3a: Was there a difference in • The SATCM treatment will • Treatment • Descriptive treatment acceptability scores between the have higher acceptability Evaluation analysis SATCM treatment group and the Book + scores based on the TEQ-T Questionnaire • Independent t-test Activity comparison group as measured by due to an added coaching (TEQ) to compare groups the TEQ-T (i.e., intervention acceptability, component compared to the effectiveness, and time to implement Book +Activity comparison strategies)? group. Question 3b: Does the SATCM treatment • The SATCM treatment • Teacher Workshop • Self-report: group demonstrate similar levels of group will demonstrate Satisfaction descriptive analysis treatment acceptability compared to previous comparable effects to Questionnaire (mean average of research on the IY-TCM program as previous research. (TWSQ) subtest scores at measured by the Teacher Workshop • Teacher-Video post-test) Satisfaction Questionnaire (i.e., Evaluation (VEQ- • VEQ-T: descriptive acceptability, helpfulness of coach, strategies T) Questionnaire analysis (mean to teach skills, and techniques used), Teacher • Teacher Interview average of Video Evaluation Questionnaire (i.e., DVD biweekly scores) 74 Table 5 (cont’d) acceptability), and teacher interviews (i.e., • Teacher Interviews: acceptability, effectiveness, treatment Informal review of barriers)? responses 75 CHAPTER 3 METHODS Participants Upon approval from MSU-IRB, participants were recruited across the state of Michigan between May 2020 until October 2020 (see Figure 2). Out of 96 participants recruited, 47 participants were eligible through study recruitment efforts described in the procedures and were sent an email by the primary investigator (PI) to sign consent forms. Forty-nine potential participants did not meet specific inclusion criteria as they presented with high levels of classroom management strategies or did not have face-to-face contact with children, three participants declined to participate, and 12 participants did not respond to their invitations. Thirty-two participants of the 47 eligible participants signed consent forms and were randomly assigned to the SATCM treatment or Book + Activity comparison groups resulting in 17 participants assigned to the SATCM treatment group and 15 assigned to the Book +Activity comparison group. Two participants from the Book +Activity comparison group dropped out before initial baseline data was collected, four participants dropped out of the SATCM treatment group before completing session 1, four dropped out of the Book +Activity comparison group before completing session 1, and one participant dropped out of the SATCM treatment group after completing session 1. Twelve participants completed the SATCM treatment group, and 11 participants completed the Book + Activity comparison group. 76 Figure 2. Flow Diagram of Participants Through the Trial For mean descriptive analyses for the fidelity checklists and demographics section, 24 participants were analyzed for this study (SATCM treatment: n = 13, Book + Activity comparison: n =11; see Table 6). For ANCOVA analysis, all participants who completed a pre- test survey were included within the analysis (n =30) using FIML estimation with intent-to-treat 77 analysis. For the independent samples t-test, 22 participants were analyzed for this study (SATCM treatment: n =12, Book + Activity comparison: n = 10) as only participants who completed the TEQ scale at post-test could be evaluated. The sample reflected teachers who were considered essential daycare center workers during the start of the COVID-19 pandemic. These teachers provided in-person daycare services throughout the state of Michigan and worked within home daycares, daycare centers, and preschools. See Appendix A for inclusion criteria. Of the 24 participants who began the training, 3 (12%) were teacher’s assistants, 4 (16%) were daycare center lead teachers, 8 (33%) were preschool teachers, and 10 (42%) were home daycare teachers. One participant (3%) had a high school education or GED, 10 (42%) had some college experience, 4 (16%) had associate degrees, 7 (29%) had college degrees, and two (8%) had post-college degrees. Six (25%) participants reported teaching between 1-5 years, 3 (12%) reported teaching between 6-10 years, six (25%) reported teaching between 11-20 years, and 9 (37%) reported they had been teaching for more than 20 years. Majority of participants identified as female (97%), and the one male included in the group was randomly assigned to the Book + Activity comparison group. The average mean age of participants was 45.8 years (SD = 12.17), with more participants (n =15, 62%) above the age of 40. Of the 24 participants, 18 were white/Caucasian (75%), followed by four Black/African Americans (16%) and two Asian/Pacific Islanders (8%). Participants in the SATCM treatment group spent more time on average completing the training per session (M = 5.4 hours, SD = 1.9) than the Book + Activity comparison group (M = 3.5 hours, SD = 1.6). Additionally, teachers from both groups most frequently completed the training within 12 weeks (calculated by subtracting the date they received materials from the date they completed their final post TSQ survey), with the SATCM treatment group ranging from 11-19 weeks and the Book + Activity comparison group ranging 78 from 10-23 weeks. The one participant within the Book + Activity group who completed the program within 23 weeks had frequent difficulties with COVID-19, which resulted in multiple daycare shutdowns. Teachers recruited into the study represented a specific subset of teachers with lower levels of classroom management training. The TSQ was adapted and used as a screening measure to operationalize inclusion criteria for participants in the study. The Total Positive Strategies frequency-of-use scale (i.e., Coaching, Praise and Incentives Subscale, Proactive Strategies subscale, and Social and Emotional Teaching Strategies subscale) was used as the screener for teachers. The researcher chose the Total Positive Strategies subscale because of greater reliability in comparison to the Inappropriate Strategies subscale (Carlson, Tiret, Bender, & Benson, 2011; Webster-Stratton, 2001) and because the measure aligns best with the goals of the program (i.e., improving positive TCM strategies). The cut score was developed based on generally high absolute ratings on the Total Positive Strategies frequency-of-use subscale. Specifically, if teachers provided a score greater than or equal to 92 on the Total Positive Strategies frequency-of-use-composite score, which indicates the respondent on average circled a 4 (i.e., “often”), then the teacher was not considered eligible for the research study (see Appendix B). Due to COVID-19 pandemic, a specific focus was spent on recruiting essential daycare workers and workers that had in-person contact with students as many schools were completely virtual. Participants were excluded from this study if they: (a) worked in the school system or caregiving facility less than 3 days per week, (b) received a score of 92 or higher on the Teacher Strategies Questionnaire (TSQ; Webster-Stratton, 2001) Total Positive Strategies composite score, or (c) Had previously attended the IY-TCM training program, (d) did not currently have 79 face-to-face contact with students (because this study required in-person practice to implement skills). Although the purpose of a randomized controlled trial is used to ensure similarities between groups, demographics information was initially analyzed to ensure the groups were not significantly different at pretest because of the small sample size. Specifically, a t-test was used to compare pre-test scores between SATCM treatment and the Book + Activity comparison groups. Analysis revealed there was not a significant difference in mean treatment screening score between groups (t30 = .602, p = .504). Based on the initial pre-screening scores using the Teacher Strategies Total Positive Frequency-of-use Scale, the SATCM treatment group had an average mean TSQ score of 64.6 (SD = 11.8) and the Book + Activity comparison group had an average pre-screening score of 69.4 (SD = 13.02). The average treatment score for the SATCM treatment group participants was 4.8 points lower than Book +Activity comparison group participants. These scores indicate teachers selected an average of 2-sometimes or 3-half the time on a five-point Likert scale for frequently using positive classroom management strategies within their classroom setting before beginning treatment. This represents a much lower score on the TSQ positive frequency-of-use score than previous studies using this measure at baseline (M = 3.6 [~82 raw score] Hickey et al., 2017) and slightly higher than the Fergusson and colleagues (2009) study (M = 2.6 [~60 raw score]). Table 6. Demographics of Analyzed Sample SATCM Group (n) Book +Activity Group (n) Sample size 13 11 Average Pre-Screening Score 64.6 (2.80, sometimes- 69.4 (3.0, half the time) half the time) Average age (min-max) 49 (19-62) 40 (25-54) Total hours (min-max) 32 (18-62) 21 (10-44) 80 Table 6 (cont’d) Total hours/Session 5.4 (SD = 1.9) 3.5 (SD = 1.6) Student Count (min-max) 12 (2-27) 14 (8-23) Race Asian/Pacific Islander 1 1 Black/African American 2 2 White/Caucasian 10 8 Level of Education Highschool/GED 1 0 Some College 6 4 Associates Degree 2 2 Bachelor’s degree 3 4 Post College Degree 1 1 Experience (years teaching) 1-5 years 3 3 6-10 years 0 3 11-20 years 5 1 More than 20 years 5 4 Role Lead Preschool teacher 2 6 Lead daycare center teacher 2 2 Lead home daycare teacher 8 2 Teacher’s Assistant/Teacher Aide 1 2 Note. Average pre-screening scores were collected using the Teacher Strategies Questionnaire (TSQ) positive frequency-of-use scale. This scale includes 23 items and ranges from 1 (rarely/never) to 5(very often). Attrition. Throughout the process, 72% (23 participants) were able to successfully complete the program and nine participants (28%) dropped out. Two participants signed the consent form for the Book + Activity group but did not complete the pre-test. Four participants dropped out of the SATCM treatment group and two participants dropped out of the Book + Activity comparison group after completing the pre-test but before treatment started. One participant dropped out of the SATCM treatment group after completing session 1. The most cited reasons for dropping out was due to family conflict (n = 2) and busy schedules (n = 3) for home daycare center workers due to new COVID-19 protocols. Other reasons included health problems (n = 1), lack of interest because they had been pushed to sign up by an employer (n = 1), and no response after multiple attempts to contact after completing their pre-screening survey 81 (n = 1). One participant dropped out in the middle of the second treatment session within the SATCM treatment group due to a death in the family. Measures Demographics questionnaire. The Demographics data was collected at the pre- intervention time point using a researcher-developed questionnaire. Data included on the demographics form was adapted from previous research studies conducting research on the IY- TCM program (e.g., Baker-Henningham & Walker, 2018; see Appendix C). Items on the caregiver demographic questionnaire included items such as the age, race/ethnicity, gender, number of years teaching, number of years teaching current grade, level of education, current role (paraeducator, assistant teacher, lead teacher, afterschool care worker, preservice teacher, daycare center), type of school, (e.g., private, public, Head Start, daycare, charter), and whether they previously attended an Incredible Years TCM training. Treatment fidelity. Treatment fidelity is when a treatment is implemented as intended (Forman et al., 2013). If a treatment is not implemented as intended or the participant does not adhere to important aspects of the treatment protocol, there is a lower chance that an intervention will create behavior change (Durlak & Dupre, 2008). The current study examined a novel delivery format of IY-TCM through a self-administered multimedia intervention using coaching. Thus, ensuring the intervention is carried out as intended was a critical component of this study. Self-monitoring checklists have been found to be successful at ensuring high treatment fidelity of teachers and parents implementing interventions (Proctor et al., 2011). To assess if the treatment was being carried out as intended, self-monitoring checklists were used for both teachers and the IY coach. 82 Teacher self-monitoring checklist. Treatment fidelity for the SATCM treatment group and Book + Activity comparison group was measured after the completion of each training session using an adapted version of the Incredible Years (IY) teacher self-monitoring checklist to determine treatment integrity percentages. The IY teacher self-monitoring checklist was created by the program developers and is included in the program curriculum (see Appendix D for the SATCM checklist and Book +Activity checklist). The checklist includes a list of homework assignments, DVDs to review (for SATCM program), goals to complete for the following weeks, and readings to complete for the following session. The original self-monitoring checklist does not include the time to complete the session, thus, this checklist included a space for the teachers to record the time to complete the training session. Treatment fidelity percentages were calculated by dividing the sum number of Yes scores by the overall opportunities for Yes scores. Percentage scores were averaged at the end of treatment to describe average treatment fidelity rates. For this study, scores of 60% were considered high treatment integrity (Durlak & Dupre, 2008). The average time taken to complete the program for both the SATCM treatment group and the Book + Activity comparison group were calculated by finding the mean hours recorded. Teacher-coach meeting form self-monitoring checklist. Treatment fidelity for the Incredible Years coach was measured via the Teacher-Coach Meeting form as a self-monitoring checklist after the completion of each treatment session with a teacher (see Appendix E). This form was created in conjunction with the IY-TCM program and is included in the training curriculum materials at the end of the self-administered program book (Webster-Stratton, 2009) and TCM website (http://www.incredibleyears.com/for-researchers/measures/). The Checklist included nine items: (a) review strategies related to the previous workshop (b) ask the teacher to determine goals for the meeting (c) review teacher completed workshop handouts (d) review and 83 problem-solve behavioral plans for students (e) discuss progress on reading chapter assignments (f) discuss plans to involve student’s family in education; (g) summarize teacher’s strengths and goals, and (h) create a plan to review further vignettes. Additionally, the coach was asked two additional questions not included in the original measure. The follow-up questions included the average length of the session and the most frequently used coaching strategy used during coaching sessions (e.g., feedback, roleplay, modeling, praise). Percentage scores were averaged at the end of treatment to describe average treatment fidelity rates. For this study, scores of 60% were considered adequate treatment integrity (Durlak & Dupre, 2008). Teacher-coach meeting form observation checklist. Treatment fidelity of how well the coach implemented the IY training was measured via an adapted version of the teacher-coach meeting form via an observation checklist (http://www.incredibleyears.com/for- researchers/measures/; see Appendix F). The checklist included the same items as listed on the Teacher-coach meeting form: Self-monitoring checklist. The PI observed the IY coach working with a teacher on two selected training sessions from a Zoom recording for the first participant during session 3 and session 6 as recommended by the IY-TCM group training program. For this study, scores of 60% or higher were considered adequate treatment integrity (Durlak & Dupre, 2008). Treatment effectiveness. Treatment effectiveness determines if an intervention caused a change in behavior and resulted in positive predicted outcomes (Bowen et al., 2009). Typically, effectiveness is measured by effect size estimates, change scores, or changes in mean scores at post-test when controlling for pre-test (Gliner, Morgan, & Harmon, 2003). The rationale for measuring treatment effectiveness for the current study was to see if coaching improved outcomes of the SATCM treatment group more than the Book + Activity group. For the current 84 study, effectiveness was measured by a pre-post self-report measure. Observations had to be removed from the measures as COVID-19 safety protocols did not permit in-person contact with teachers. Teacher Strategies Questionnaire (TSQ; Webster-Stratton, 2001). The Teacher Strategies Questionnaire (TSQ) was used to measure participants’ (1) self-reported frequency of positive and negative classroom management strategies (herein after frequency-of-use measure) and (2) perceived usefulness of positive and negative TCM strategies (herein after perceived usefulness measure) for managing classroom behavior at pre- and post-intervention (see Appendix G). The TSQ was created in conjunction with the Incredible Years program, is included in the IY-TCM curriculum and can be found on the IY website (http://www.incredibleyears.com/for-researchers/measures/). The TSQ includes 59 items, four subsections: (a) Managing Classroom Behavior, (b) Specific Teaching Techniques, (c) Working with Parents, and (d) Planning and Support, and seven subscales: (a) Confidence in Managing Classroom Behaviors, (b) Praise and Incentives, (c) Proactive Strategies, (d) Limit-Setting Strategies, (e) Inappropriate Strategies, (f) Positive Approaches with Parents (g) and Planning and Support. The respondents rate their behavior based on a 7-point Likert scale for Confidence in Managing Classroom Behaviors (1 =Very Unconfident; 7 = Very Confident) and the rest of the subscales include a 5-point Likert scale (1= Rarely/Never; 5 =Very Often). The praise and incentives, proactive strategies, and social and emotional teaching strategies subscales create the Total Positive Strategies composite score. Additionally, the inappropriate strategies subscale creates the Inappropriate Strategies composite score. Within the Teaching Techniques subsection, teachers answer 38 items for both the frequency-of-use measure and perceived 85 usefulness measure simultaneously and receive separate Total Positive Strategies composite scores and Inappropriate Strategies composite scores for each measure. Previous research on this measure (Webster-Stratton, 2001) indicates acceptable (i.e,.70 ≤ α < .80) to good internal consistency reliability (i.e., .80 ≤ α < .90) for the Confidence in Managing Classroom Behavior scale (α =. 94), the Total Positive Strategies frequency-of-use scale (α = .79), the Total Positive Strategies perceptions-of-usefulness subscale (α =. 70), the Inappropriate Strategies frequency-of-use subscale (α = .77), the Inappropriate Strategies perceptions-of-usefulness subscale (α = .84), and the Positive Approaches with Parents subscale (α =. 78). Independent researchers (Carlson, Bender, Tiret, & Benson, 2011) have found similar results to Webster-Stratton (2001) on the TSQ; however, they reported considerably weaker internal consistency reliability on the Inappropriate Strategies perceptions-of-usefulness subscale (α = .54). Thus, scores collected via this subscale should be analyzed with caution. Treatment acceptability. Rationale for measuring teacher treatment acceptability was to determine if program formats could be easily implemented in the school system. Research indicates treatment acceptability is an important factor in determining the transportability of an intervention (Witt & Elliot, 1985). A lack of treatment acceptability such as lack of teacher buy- in on program content or structure can serve as an implementation barrier for EBIs. Witt and Elliot’s (1985) conceptual model of treatment acceptability indicate there is a sequential and reciprocal relationship between treatment acceptability, treatment integrity, and treatment effectiveness. However, they find that if an intervention is implemented with high integrity, it has a higher likelihood of increasing effectiveness of an intervention, which in turn increases treatment acceptability of those implementing the intervention of using positive strategies taught within the intervention (Witt & Elliot, 1985). The current intervention study examined whether 86 the self-administered format, teaching techniques, videos, and coaching are acceptable to teachers to enhance knowledge on TCM. Thus, examining the teacher’s acceptability of the self- administered IY-TCM formats is a critical component in the research study. Treatment Evaluation Questionnaire (TEQ-T; Kelley, Heffer, Gresham, & Elliot, 1989). Overall treatment acceptability, acceptability of effectiveness, and time to improve treatment were measured at post treatment with both the SATCM treatment group and Book + Activity comparison group using the Treatment Evaluation Questionnaire-Teacher Form (TEQ- T; Kelley et al., 1989; see Appendix H). The TEQ-T is a 21-item questionnaire which includes three subscales: Acceptability (11 items), Effectiveness (8 items), and Amount of Time Required (2 items). Participants were asked to rate their experiences on a Likert scale from 1 – 6 (strongly disagree to strongly agree) with higher scores indicating higher acceptability (Kratochwill et al., 2003). Overall scores range from 21 to 126 with a midpoint score of 73.5. Using Shernoff and Kratochwill’s (2007) formula, adequate ratings of acceptability on the TEQ-T included anything at or above 3.5 for individual items and 73.5 for the overall scores. For subscale scores, adequate ratings of acceptability included anything at or above 38.5, 28, or 7 for acceptability, effectiveness, and time required respectively (Kratochwill et al., 2003). The TEQ-T is typically used with parents to assess acceptability. For the current study, a teacher version of the TEQ was created for use with teachers by replacing the word “parents” with “teacher” to reflect teacher acceptability (e.g., “Most teachers would find this intervention appropriate for behavior problems” instead of “Most parents would find this intervention appropriate for behavior problems”). The TEQ-parent form was adapted from the Treatment Evaluation Inventory- short form (TEI-SF; Kazdin, 1980), a measure which demonstrates acceptable internal reliability (" = .85; Kelley et al., 1989), through factor analytic procedures. 87 Previous research demonstrates the TEQ-P can differentiate acceptable from non-acceptable treatments (Kazdin, 1980). The TEQ-Parent form (TEQ-P), has been used several times to evaluate acceptability for the IY parent training program (Kratochwill et al., 2003; Ogg & Carlson, 2009; Osburn, 2009; Stewart & Carlson, 2010). This measure of acceptability also differs from the TWSQ as it examines intervention acceptability, rather than overall satisfaction of the treatment, strategies, or content. Thus, this measure was used to compare findings of the SATCM treatment program to the Book +Activity comparison group to add to the current literature. Teacher Workshop Satisfaction Questionnaire. Content acceptability, acceptability of strategies, acceptability of the coach, and acceptability of the evidence-based TCM techniques of the SATCM program with coaching was measured at post-treatment with the SATCM treatment group only using an adapted version of the Teacher Workshop Satisfaction Questionnaire (TWSQ, Webster-Stratton, 2001; see Appendix I). The questionnaire was created in conjunction with the IY-TCM program and is included in the program curriculum. The TWSQ has five subscales: (a) overall experience of the program (8 items), (b) usefulness of different learning methods (8 items), (c) usefulness of teaching techniques taught (9 items), (d) evaluation of workshop leader (4 items), and (e) overall program evaluation (4 short-response items). In the first four sections, items are rated using a 7-point Likert type scale, ranging from 1 (e.g., Extremely Not Helpful/ Extremely Useless) to 7 (e.g., Extremely Helpful/ Extremely Useful). Additionally, a score of “4” is considered “neutral.” For the overall program evaluation there are four open-ended questions. These questions include: (a) “What part of the program was most helpful to you?” (b) “What did you like most about the program?” (c) “What did you like least about the program?” and (d) “How could the program have been improved to help you more?” 88 For this study, only section b, c and d were included as the other sections of the questionnaire were covered by other acceptability questionnaires within this study. To adapt these subscales to fit with the current study, all items of the “usefulness of different learning methods” subscale were eliminated except for four items where the wording was changed to fit language used for the self-administered manual. Additionally, one item was removed from the workshop leader section because it related to facilitating group discussions which are not applicable in the self-administered version. For the other items in the workshop leader section, the word “group leader” was replaced with the word “coach” to reflect the treatment received by the teachers. Murray, Rabiner, Kuhn, Pan, and Sabet (2018) indicate strong internal consistency reliability on the scales: Overall Satisfaction (α = 0.82), Usefulness (α = 0.85), and Strategies (α = 0.83). Subscales of this tool have been used in several previous group training studies to measure acceptability of the treatment program (Fergusson et al., 2009; Murray et al., 2018). Thus, utilizing this tool for the current study was helpful to compare acceptability scores of the self-administered treatment program to the group training program. Shernoff and Kratochwill’s (2007) formula when measuring adequate acceptability scores includes anything at or above the midpoint. Using this formula, average scores of 3.5 or higher (i.e., neutral or better) on the subscales were considered adequate acceptability ratings. Incredible Years Video Evaluation Questionnaire-Teacher Form. The acceptability of the SATCM treatment videos was measured after each treatment session to evaluate acceptability of the video content using the Video Evaluation Questionnaire Teacher form (VEQ-T) Webster- Stratton, 2001; see Appendix J). The VEQ-T is a 4-item questionnaire which measures the content and perceived helpfulness of the DVDs provided in the treatment. Participants rated their 89 perceived helpfulness of the videos on a Likert scale from 1 to 5 (Not Helpful to Very Helpful) with higher scores indicating higher acceptability. Overall scores range from 4 to 20. The questionnaire was created in conjunction with the parent training program and is included in the program curriculum; however, this item has been adapted for use with the SATCM program. To adapt the measure, the wording was changed in item four from “my child” to “my students.” Finally, seven forms were created instead of only four forms used for the parent training program to reflect the seven DVDs included within the program. Stewart and Carlson (2010) previously examined the internal consistency reliability of the VEQ parent form and found high levels of reliability (" = .89). Additionally, previous researchers have used the VEQ parent form when examining the SAPT study (Stewart & Carlson, 2010; Taylor et al., 2008), thus, this was a useful tool to compare findings to the SATCM treatment. Using Shernoff and Kratochwill’s (2007) formula, average scores of 3 or higher on individual items and a total score of 12 or higher indicated adequate reliability. Teacher interview. Acceptability of the SATCM program was measured via a teacher interview conducted at post-treatment to assess perceived barriers and acceptability of treatment. The PI conducted teacher interviews immediately following the last treatment session of the SATCM treatment program. The interview questions were derived to align with the TWSQ and TEQ subscales and common implementation barriers identified in the literature that self- administered program formats attempt to mitigate. Interview questions probed at the overall acceptability of the SATCM intervention (i.e., “Would you recommend this program for other teachers?”) and explored whether the program was feasible and acceptable to implement in the school/daycare setting (e.g., What challenges or barriers did you face when trying to complete the assignments [e.g., time, scheduling?]). An overview of the topics and questions are included 90 in Appendix K. Reliability and validity were not analyzed for these open-ended questions; however, the interview included research-developed questions that align with the training program. Specifically, these research-developed questions were created based on (a) previous researcher-developed interview questions (e.g., Thompson, 2018), (b) the TWSQ and TEQ, which demonstrate high internal consistency reliability and, (c) common implementation barriers identified in the literature (e.g., Rotheram-Borus et al., 2012). The PI recorded responses over Zoom and coded for themes. Responses were reviewed to inform future research and practice efforts. Procedure Recruitment. Target population of participants were childcare providers who worked with children between the age of 3-8 (preschool to third grade) and resided in the mid and western Michigan areas. Due to limited in-person schooling because of the COVID-19 pandemic, most participants were recruited from essential preschools, daycare centers, and home daycares. To find access to these workers, the PI contacted the state of Michigan to receive a list of essential daycare workers and used the MiRegistry website which included whether daycare centers and preschools were open or not due to COVID-19. Additionally, social media platforms such as Instagram and Facebook were used to recruit participants. Interestingly, this strategy yielded fewer results than sending direct emails to essential daycare workers or advertising over MiRegistry. Recruitment flyers and consent forms (see Appendix L & M) outlined program goals, professional development, and monetary incentives for participating in the research study. Specifically, each flyer and consent form indicated participants had the opportunity to receive between 18-36 credit hours on MiRegistry, a certificate of completion, and $50 gift cards. Credit 91 hours reflected an average of 3 hours per session for the Book + Activity comparison group participants and 6 hours per session for SATCM treatment group participants. This aligned with the amount of time previous participants spent on the SATCM program (e.g., Shernoff & Kratochwill, 2007) and based on an estimated time spent watching the DVDs reflected on the IY DVDs. The rationale for including compensation was to incentivize participants to attend and complete programs/measures, especially if they were placed in the Book +Activity comparison group. Both recruitment flyers and consent forms indicated that the length of the program would take approximately 12 weeks. For teachers in the SATCM treatment group, both forms specified that the program would take at least 3 hours per week and an additional 30-minute biweekly check-in with a certified coach. For teachers in the Book + Activity comparison group, the consent form specified an average of 1.5 hours per week. Project personnel and training. Project personnel included (a) the PI, (b) the Incredible Years Certified Coach, and (c) and two school psychology graduate students. The PI was the primary researcher and was responsible for designing the study, recruiting participants, obtaining materials and incentives for participants, locating and organizing data collection tools, handling data collection, training school psychology graduate students in using data collection tools, and analyzing data. The PI also obtained IY group leader training with the program developers prior to the start of this research study. The certified IY-TCM Coach was responsible for coaching participants throughout the session using evidence-based coaching strategies (i.e., action plans, feedback, modeling, role play, rehearsal of strategies, and supporting with barriers to treatment) that is individualized, intensive, sustained, context specific, and focused (Kraft et al., 2018). Additionally, the coach was available to teachers if they had any questions in between treatment sessions. To obtain 92 certification as an IY-TCM Coach, the coach must: (a) have a graduate degree in counseling, psychology, social work or psychiatry, (b) complete a three-day, 6-hour, intensive group training workshop with the program developers, (c) conduct two IY-TCM training workshops, and (d) submit materials (e.g., a two-hour video recording of the workshop, participant evaluations, and training checklists) to be reviewed by the program developers for evaluation, feedback, and approval for certification. Once a certified group leader, the coach must conduct at least six group training sessions, attend a 2-day coach training session, and receive positive evaluations from other group leaders or mentors based on two coaching DVDs submitted. The IY-TCM coach for this study has obtained strong credentials to provide coaching to participants. Specifically, the IY-TCM coach for this research study is already a certified IY group trainer. She began the certification process in 2009 and was officially certified for group training in 2011. Prior to the current study, she led eight IY-TCM group training sessions between 2009-2020, which meets the minimum requirement of the coach certification. The IY group leader also has her Bachelor of Arts degree in Child Development and a Master of Arts degree in Guidance and Development. Furthermore, the IY group leader has had over 35 years of experience working in early childhood education as a preschool teacher, family childcare provider, Montessori infant and toddler teacher, and preschool director. Currently, she works as a Family and Consumers Science Educator at Michigan State University Extension where she specializes in the areas of health research and social-emotional health for families and caregivers. Her role includes teaching university students, leading caregiver workshops, and training sessions at research conferences and to caregivers in the community. Although this IY-TCM coach has not implemented the self-administered program as an IY coach, her extensive training 93 in parent behavior training and experience implementing the IY group training workshop positioned her well to complete these coaching services. Two school psychology doctoral students were recruited as research assistants (RAs) to assist the PI with data quality and reminders. Specifically, the researcher trained the RAs on how to use the Qualtrics system to send reminders to participants who did not complete their surveys. Additionally, RAs sent participants new activity sheets, fidelity sheets that explained what tasks to complete for the next session, and survey links for participants to complete on the session most recently completed, every two weeks. Each RA was assigned to a group to keep track of participants. They also provided the PI a weekly summary and updated an excel spread sheet to keep track of completed participant surveys. Treatment phases. Pre-intervention. Before enrollment, teachers were asked to complete screening measures and the demographics questionnaire to determine if they met the inclusionary criteria for this study. Based on scores from the TSQ Total Positive Strategies component score, teachers were notified if they were a part of the research study in an email. After signing a consent form, teachers were randomly assigned to a treatment group and assigned a code to ensure anonymity. Treatment randomization. Kazdin (2017) indicates that grouping subjects into blocks or sets can help avoid unequal group sizes in randomization. To ensure equal group sizes, he recommends each set include the same number of subjects as the number of groups in the experiment. For this study, it was proposed that the PI would randomize two participants at a time to ensure equal group sizes. To address time constraints, the PI randomized each set of participants upon arrival of the program using the Sealed Envelope website (https://www.sealedenvelope.com/simple-randomiser/v1/lists), a website that allows block 94 randomization lists. The PI continued to recruit participants until the proposed goal of 24 participants was met or until funding availability was reached. If there were only two participants at a time, these participants were randomized and placed within their randomized treatment group. If only one participant was available, the participant was randomized into either the SATCM treatment group or Book + Activity comparison group, while the other slot remained empty. The empty slot was filled by the next participant to sign the consent form (e.g., a participant was randomized into the SATCM treatment group slot so the next participant to sign up was placed in the Book + Activity comparison group). Towards the end of recruitment, there were two groups of seven participants that signed consent forms after a recruitment email was sent in July and August. To ensure 24 participants were met in time, the PI randomized the first group of seven participants that arrived in July all together, where four participants went into the SATCM treatment group, and three participants were randomly assigned to the Book + Activity comparison group. Due to a documenting error made by the PI, the second group of seven participants who arrived in August were also randomized all together. In other words, instead of choosing the first participant to sign the consent form within the August group to fill the empty treatment slot in the July group to make an even grouping of eight, the PI randomized all seven participants from the August grouping at once. As a result, four participants were randomly assigned into the SATCM treatment group and three participants were randomly assigned to the Book + Activity comparison group. This resulted in uneven participants across the two groups creating 17 in the SATCM treatment group and 15 in the Book +Activity comparison group. Intervention phase. During the intervention phase, there were two groups of teachers receiving treatment: 1) The SATCM treatment group and 2) a Book + Activity comparison 95 group. The SATCM treatment group participants were asked to complete six training sessions, while the Book +Activity comparison group were asked to read from a pre-made schedule outlining chapters from the Incredible Years: Nurturing Children’s Social, Emotional, and Academic Competence with assigned readings and activities. Upon enrollment, teachers were provided a welcome letter and phone call to discuss procedures and expectations. See Table 7 for the survey schedule for the SATCM treatment group and Book + Activity comparison group. Table 7. Assessments Completed During Each Phase Phase SATCM Group Book +Activity Group Coach/PI for SATCM Coaches Pre-test 1. Demographic 1. Demographic Questionnaire Questionnaire 2. TSQ screener 2. TSQ screener Baseline 1. TSQ 1. TSQ Treatment 1. Session Treatment 1. Session Treatment 1. Adherence Adherence Adherence checklists checklists for checklists for for Book +Activity coaches- self- SATCM group (7x) group (7x) reflection form (6 per participant) 2. VEQ-T (7x) 2. Adherence checklists for coaches - observation form (2x-completed by PI) Post-Test 1. TSQ 1. TSQ 2. TEQ-T 2. TEQ-T 3. TWSQ 4. Interview Self-administered teacher classroom management. The SATCM program included six lessons, held over 12 weeks, and was administered individually by the participant either at home or school. Because the SATCM program is intended to be a more flexible treatment approach, there was no prescribed number of hours to complete the program for the current study. However, the treatment developers recommend teachers spend at least 45 minutes in a quiet 96 space to read program materials and were only provided 2 weeks to complete the materials. During the first week of the sessions, teachers were asked to complete the SATCM session workbooks, watch the videos, and read the assigned book chapters. During the second week of the session, teachers were asked to practice implementing their newly learned skills, meet with their coach, and complete their required surveys to be eligible for their 36 credit hours and $50 gift certificate. The curriculum of the SATCM training program mirrors the IYTCM group training program. The six program workbooks include: 1) building positive relationships with students and the proactive teacher, 2) teacher attention, coaching, encouragement, and praise, 3) motivating students through incentives, 4) decreasing inappropriate behavior—ignoring and redirecting, 5) decreasing inappropriate behavior-follow through with consequences, and 6) emotional regulation, social skills, and problem-solving training (Webster-Stratton, 2001). Additionally, a program workbook or treatment manual was included for each session (except lesson one which includes two manuals) to explain the process and required assignments for that treatment session. Each program workbook includes assigned reading material from the Incredible Years: Nurturing Children’s Social, Emotional, and Academic Competence (Webster- Stratton, 2012), assigned DVDs to view with guiding questions to promote critical thinking, and activities throughout the manual to practice goal setting, individual behavioral plans, and other effective strategies. On the first day of the session, the PI provided written and verbal instructions on how to fill out the treatment measures and activities. To ensure treatment fidelity, the PI mailed SATCM manuals and DVDs directly to the teacher in a sequential order to ensure the participants practiced their skills before moving onto the next session. They were also provided pre-paid 97 return envelopes to return the materials after two weeks. Additionally, teachers were emailed survey links and next session materials at the end of each session. The coaching sessions were delivered by a certified group trainer for six sessions, once every two weeks (i.e., every session) for 12 participants. These sessions were held at the end of the second week of the SATCM session to allow the teacher to practice skills and ask the coach any clarifying questions. Upon enrollment, the coach called teachers to schedule their six, 30 minutes coaching sessions and provided Zoom version 4.6.2 [computer software] links, a secure video conferencing application that protects against third party software, to ensure privacy and confidentiality. The purpose of these coaching sessions was to allow the teacher an opportunity to collaborate with a coach to create action plans, goals, reflect on and review materials, role- playing unclear situations to practice skills, and receive performance feedback (Reinke et al., 2015) in a tailored, individualized intensive format (Kraft et al., 2018) to enhance TCM skills without attending a group training session. To ensure teachers implemented the treatment program with fidelity every 2 weeks, teachers filled out links provided in their email from Qualtrics so they could earn their gift certificate and 36 credit hours. If teachers did not complete session activities by the time of the coaching sessions, the coach worked with teachers on potential barriers hindering treatment and how to find success for the following session. To ensure coaching fidelity, the coach filled out the Adherence Checklist for Coaches Self-Reflection form (Webster-Stratton, 2012) measured after each treatment session through a Qualtrics survey link. The coach also used this sheet to record any questions from the teacher to follow up at the next session. Additionally, the PI observed two treatment sessions (i.e., session 3 and session 6) using the Adherence Checklist for Coaches-Observation form from a recorded Zoom session to ensure inter-rater reliability for two 98 of the coaching sessions. Observing session 3 and session 6 is recommended in the IYTCM group training program. To measure teacher acceptability of the video content, teachers filled out the Teacher Video Evaluation Questionnaire (VEQ-T; Webster-Stratton, 2001) for each treatment session after watching their assigned videos. Book + activity comparison group. Teachers randomly assigned to the Book + Activity comparison condition were provided with a schedule of selected chapters from the Incredible Years: Nurturing Children’s Social, Emotional, and Academic Competence (Webster-Stratton, 2012) and activities. This book was chosen as it directly aligns with the SATCM program contents and can be used as a standalone resource for teachers; however, the book does not include the DVDs or the SATCM manual. To more closely mirror SATCM, teachers completed activities provided in the SATCM manual, as they are freely provided on the IY website and completed self-reflection questionnaires following each chapter provided in the book. Within the Book + Activity comparison group, the teachers did not receive any consultation or follow-up conversations. Qualtrics XM [survey tool] and RAs helped to mirror the SATCM treatment program to serve as a more manualized approach as teachers received reminders to complete their assigned reading and complete their surveys. The Book + Activity comparison group was considered the more cost-effective treatment as the book (and survey materials) are available online within the IY website at http://www.incredibleyears.com/products/products.asp and thus could be accessed by any teacher if they desired. Teachers were assigned new readings and activities every two weeks to align with the SATCM treatment group. Similar to the SATCM treatment group, the Book + Activity comparison group was provided with a list of instructions by the PI before beginning treatment. Upon assignment to this comparison condition, teachers were provided with a book delivered via mail and a schedule for 99 selected chapters (see Table 8) which align with the IY-TCM program curriculum. They also had a list of instructions and activities listed within their welcome letter. The Book + Activity comparison group also was required to fill out a self-monitoring checklist at the end of each session for the homework and practice activities, self-reflection form, and book chapters. Teachers logged how long it took them to get through the chapters and assignments to compare to the SATCM treatment group. Additionally, teachers were required to turn in their self- monitoring form and complete the reading assignments to receive their $50 gift certificate and 18 credit hours. Table 8. Assigned Chapters for the Book + Activity and SATCM Group Schedule (readings Book + Activity SATCM Assigned assigned every 2 weeks) Assigned Chapters Chapters Session 1 Chapter 1, Chapter 2, Chapter 1, Chapter 2, Chapter 3, Chapter 14 Chapter 3, Chapter 14 Session 2 Chapter 4, Chapter 5 Chapter 4, Chapter 5 Session 3 Chapter 6 Chapter 6 Session 4 Chapter 7, Chapter 15 Chapter 7, Chapter 15 Session 5 Chapter 8, Chapter 9, Chapter 8, Chapter 9, Chapter 10 Chapter 10 Session 6 Chapter 11, Chapter 12, Chapter 11, Chapter 12, Chapter 13 Chapter 13 Post-intervention phase. Upon completion of the 12-week training sessions, teachers from both the SATCM treatment group and the Book + Activity comparison group filled out the TSQ and TEQ-T, while only the SATCM treatment group completed the TWSQ and an informal interview on the perceived barriers such as feasibility, time, and resources. The informal interview was completed by the PI via zoom and recorded. Audio recordings were coded for themes and no names or other identifying information were recorded. 100 Data Analyses Question 1a-b. Treatment adherence was analyzed based on the self-monitoring form completed by teachers, the teacher-coach self-monitoring form completed by the coach, and the teacher-coach observation form conducted by the PI. For the teacher and coach self-administered forms, teachers/coach responses were coded as Yes/No. Based on the average of the six sessions, the percent of weekly adherence to biweekly fidelity checks were calculated. Biweekly and average percentages were then compared to the 60% standard to determine if teachers and the IY coach could implement the intervention formats with adequate fidelity (Durlak & Dupre, 2008). Inter-rater agreement percentages were assessed as a separate question to see if there were acceptable levels of agreement between ratings from the IY coach and the PI. To measure inter-rater reliability, the PI observed two coaching sessions (session 3 and session 6) from zoom recorded sessions using an observation checklist identical to the Coach’s self-report fidelity checklist and compared observation ratings to the coach’s fidelity ratings. Inter-rater agreement percentages were calculated by finding the total number of agreed ratings/ (total number of disagreed + agreed ratings). Inter-rater agreement percentages of 75% or higher was considered an acceptable level of inter-rater agreement and 90% or higher was considered high levels of inter-rater agreement (Hartmann, 1977; Stemler, 2004). Question 2a-b. Average scores on the TSQ teachers’ frequency of use of positive/negative classroom management strategies, perceptions of positive/negative classroom management strategies as measured on five TSQ subscales (i.e., Confidence in Managing Classroom Behavior, Total Positive Strategies, Inappropriate Strategies, Positive Approaches). These average scores were analyzed using an Analysis of Covariance (ANCOVA). Specifically, the TSQ pre-test score was the (covariate), the post-test score was the dependent variable and the 101 treatment group (SATCM treatment group, Book + Activity comparison group) was the independent variable (SATCM coded as 1, Book +Activity coded as 0). Running an ANCOVA can be a more powerful approach than the use of other common pre-post comparison group analyses such as the use of gain scores. According to Gliner and colleagues (2003) adjusting for the pre-test score within an ANCOVA model can be a more accurate approach to assess post score differences as this approach can help ensure that post-score differences are due to the treatment rather than variation between groups at pre-test. Additionally, adjusting for the pre-test score can account for variation of post-test means that can occur because of variation in the pre- test groups. To address missing data due to dropouts or missing items on surveys, missing variables were imputed using a full information maximum likelihood (FIML) estimation in Lavaan’s package in R Studio (Rosseel, 2012). FIML can be a more powerful approach to address missing data as it can include the entire data set, while common methods such as listwise deletion in SPSS can remove large sets of data if survey items are missing (Buhi, Goodson, & Neilands, 2007). Question 3a. For the TEQ, an overall score and subtest scores were summarized at post treatment. Individual items of 3.5 or higher and overall scores of 73.5 or higher was considered adequate levels of treatment acceptability on the TEQ (Shernoff & Kratochwill, 2007). Subtest scores were analyzed by Acceptability (items 1-11), Effectiveness (items 12, 14, 15, 17-21), and Amount of Time (items 13 and 16). Midpoint score at or above 38.5, 28, and 7, represented adequate Acceptability, Effectiveness, and Amount of Time respectively. To determine group differences in teacher acceptability, the PI compared the scores between the SATCM treatment group and the Book +Activity comparison group using an independent t-test. 102 Question 3b. Descriptive analysis of treatment acceptability was conducted with the TWSQ, VEQ-T, and qualitative interviews. To measure SATCM program acceptability, the mean average of each subscale on the four subscale scores was summarized at post-treatment from the TWSQ. Scores of 3.5 or higher for each category were considered adequate levels of treatment acceptability (Shernoff & Kratochwill, 2007). Individual items on the overall acceptability subscale of the TWSQ and strategies subscale were reported to compare to previous research outcomes. To evaluate video acceptability, participant ratings per video session were averaged using the VEQ-T. Scores were analyzed bi-weekly and summarized by video session. Scores greater than 3 on individual items or scores greater than 12 on the overall scale were considered adequate treatment acceptability (Shernoff & Kratochwill, 2007). To analyze barriers, acceptability, and effectiveness of treatment, post-treatment teacher interviews were coded for themes related to acceptability of the self-administered treatment program. Effect size calculations. The effect size for the acceptability analyses were calculated ⎛ !! " !" ⎞ % using Hedge’s g, = ⎜ ⎟ +1 − & (( )( )"+., where M represents the means in " ($ %!)'(! )($! %!)'("" ! " $ ! $! )$" %" ⎝ ⎠ the two treatment groups, n represents the sample size in each group, SD is the standard deviation in each group (Hedges & Olkin, 1985, p. 110.) For effectiveness analyses, Cohen’s d was calculated using the ANCOVA t with , ((!) ) (" ) independent groups, / = 0 1 2, where R is the correlation between outcome and (! ) (" (-." / " ) covariate. Hedges’ g was calculated by multiplying Cohen’s d by J, the correction formula, % +1 − & (( . to correct for any upward bias and create an unbiased estimate. ! )(" "0)". 103 Hedge’s g can be a useful analysis if you have different sample sizes between groups as you can weight each group’s standard deviation by the sample size. Additionally, this equation can correct for upward bias of samples smaller than 50 by multiplying the equation by a correction formula. The following are the suggested standards for interpreting Hedge’s g: small = 0.2; medium = 0.5; large = 0.8 (Cohen, 1988) 104 CHAPTER 4 RESULTS Research Questions and Hypotheses Question 1a. Was there a difference in treatment adherence between the SATCM treatment group and the Book + Activity comparison group as measured by an adapted Self- Monitoring Checklist? Based on average fidelity ratings collected at each biweekly session, all participants (n = 23; 100%) had fidelity ratings over 90% as measured by an adapted self-monitoring checklist (see Table 9) which met the criteria of adequate adherence defined by Durlak & Dupre (i.e., > 60%). Specifically, the SATCM treatment group demonstrated an average fidelity rating of 96% and the Book + Activity comparison group had an average fidelity rating of 94%. No statistical analyses were completed between groups as fidelity checklists were not equivalent. Common items not completed on the checklist in the SATCM treatment group included difficulties completing the chapters due to lack of time and collaborating with parents due to curriculum restrictions or factors outside of the teachers’ control (e.g., students moving classrooms). Most common items not completed in the Book +Activity comparison group included barriers to creating behavior plans due to interruptions from COVID-19 and difficulties collaborating with parents due to curriculum restrictions. 105 Table 9. Means, Standard Deviations, and Percentages of Fidelity scores for the SATCM Treatment Group and Book +Activity Comparison Group SATCM Book +Activity Session n % M (SD) Min- n % M (SD) Min- complete Max complete Max Session 1 13 96% 19.1(1.8) 14-20 11 80% 15.4 (1.2) 15-19 Session 2 12 96% 9.6 (0.7) 8-10 11 97% 6.8(0.6) 5-7 Session 3 12 98% 8.8 (0.6) 7-9 11 96% 6.7 (0.5) 6-7 Session 4 12 96% 8.6 (.50) 8-9 11 98% 5.9 (0.3) 5-6 Session 5 12 93% 10.3(1.3) 7-11 11 100% 8 (0.0) 8-8 Session 6 12 97% 10.6(0.5) 10-11 11 98% 7.9 (0.3) 7-8 Total 96% 94% Question 1b. Did the IY coach adhere to treatment principles of the SATCM program when supporting teachers biweekly as measured by an adapted self-report IY-TCM coaching checklist at post treatment? How often did raters (IY Coach and primary investigator) agree on items as measured by an adapted IY-TCM observation checklist during SATCM treatment sessions? Yes, based on biweekly self-report measures, the IY Coach was able to adhere to the coaching checklist with adequate fidelity at 81% (see Table 10) which met the criteria of adequate adherence defined by Durlak & Dupre (i.e., > 60%). Common items not completed on the coaching checklist included reviewing further vignettes with teacher and including parents within behavior plans. Inter-rater reliability was calculated for the first coaching participant with the intent to provide feedback and support the coach. Inter-rater agreement percentages were calculated by finding the total number of agreed ratings/ (total number of disagreed + agreed ratings). The primary investigator observed session 3 and session 6 from a zoom recorded session for participant 1, as recommended by the developers. Ratings were compared to the IY coach’s self- reoprt fidelity checklist for session 3 and session 6. Results indicated acceptable levels of agreement for session 3 (83%) and session 6 (75%) and met criteria for Hartmann (1977) and 106 Stemler’s (2004) rule of thumb for acceptable levels of agreement for inter-rater agreement percentages (i.e., 75% to 90%) Common items observed by the PI that were not completed by the coach during sessions included reviewing vignettes with the teacher, involving plans with parents, and reviewing activity worksheets. Common reported barriers to completing the checklist included lack of time (i.e., teachers were on a shortened break such as nap time), teachers no longer had access to DVDs to discuss the vignettes with the coach, and teachers reporting limited access to parents due to COVID-19. Table 10. Average Self-Reported Coaching Scores IY Coach Session n % complete M (SD) Min-Max Session 1 13 70% 16.69 (2.52) 15-24 Session 2 11 81% 12.90 (1.22) 12-16 Session 3 12 89% 12.42(0.69) 12-14 Session 4 11 63% 13.90(2.70) 13-22 Session 5 12 89% 12.50 (0.80) 12-14 Session 6 12 93% 11.25 (.452) 11-12 Total 81% Note. Mean fidelity scores are based on number of items selected yes (1) or no (0) by the coach. Question 2a. Does participating in the 12-week SATCM program result in a self- reported increase in teachers’ perceptions of positive classroom management strategies as well as a decrease in perceived negative classroom management strategies (i.e., Confidence in Managing Classroom Behavior, Total Positive Strategies, Inappropriate Strategies) compared to the Book + Activity comparison group when adjusting for pre-test differences as measured by the TSQ? An ANCOVA analysis was run in RStudio using FIML estimation to impute missing variables and to examine the effectiveness of the SATCM treatment and Book + Activity comparison groups while adjusting for pre-test scores. Confidence in managing classroom 107 behavior, perceived usefulness of positive strategies, and perceived usefulness of negative strategies did not demonstrate a significant difference between the SATCM treatment group and Book + Activity comparison group at post-test when adjusting for pre-test scores (see Table 11). Raw mean scores demonstrated a positive upward trend for both the SATCM treatment group and the Book +Activity comparison group for usefulness of positive strategies (see Table 12). When examining within group pre-post differences within the descriptive statistics, it appears that post-test means increased by 20.1 points for the SATCM treatment group and 9.6 points for the Book + Activity comparison group for usefulness of using positive TCM strategies. Use of negative TCM strategies stayed the same or improved for both groups. Specifically, the SATCM group increased by 0.2 points at post-test and the Book + Activity comparison group decreased 1.3 points at post-test. Effect size estimates for between groups ANCOVA were calculated from the ANCOVA t with independent groups using Cohen’s d formula multiplied by the correction formula to derive Hedges’ g estimates (Hedges & Olkin, 1985, p. 110). Table 11. Teacher Perceptions of Classroom Management Strategies CI95% for Estimate Est. SE t p Lower Upper 30 g [CI95%] Confidence teacher skills ANCOVA Pretest 0.23 0.14 1.69 .090 -0.04 0.50 Covariate ANCOVA Posttest 1.16 0.92 1.25 .210 -0.65 2.96 0.14 0.40 [-0.23 –1.03] Usefulness of positive TCM strategies ANCOVA Pretest 0.43* 0.19 2.32 .020 0.07 0.80 Covariate ANCOVA Posttest 8.94 5.00 1.79 .074 -0.85 18.73 0.27 0.53 [-0.06 –1.12] 108 Table 11 (cont’d) Usefulness of Negative TCM strategies ANCOVA Pretest 0.33 0.24 1.41 .159 -0.13 0.79 Covariate ANCOVA Posttest 2.15 3.52 0.61 .542 -4.76 9.06 0.10 0.20 [-0.44 – 0.84] Note. Hedges’ g was calculated using the t-value from ANCOVA posttest. ANCOVA posttest represents the between group estimates of the treatment groups when controlling for pretest measures. ANCOVA pretest estimates represent the correlation between pretest covariate on posttest outcome variable. *= p <.05, **, p<.001. Table 12. Descriptive Statistics of Teacher Classroom Management Strategy Perceptions and Frequency-of-Use Book + Activity Assigned Group Comparison SATCM Treatment a Time n M SD n M SD Confidence pre 13 12.6 3.3 17 12.2 3.6 post 11 17.6 2.7 13 18.7 2.4 Total Positive Strategies-Useful pre 12 72.1 13.3 15 69.0 14.3 post 11 81.7 16.0 12 89.1 11.8 Inappropriate Strategies-Useful pre 12 16.5 7.9 15 17.1 7.6 post 11 15.2 9.8 12 17.3 8.6 Total Positive Strategies - Frequency pre 13 66.9 10.7 17 63.6 12.5 post 11 81.8 14.2 12 86.0 12.5 Inappropriate Strategies- Frequency pre 13 14.6 3.5 17 15.5 5.2 post 11 12.8 2.7 12 14.0 7.1 Note. Confidence scale ranged from 1-7 (total points: 7-21; Midpoint >4 [12]), Total Positive Strategies scale ranged from 1-5 (total points: 23-115; Midpoint >3 [>69])), Inappropriate perceptions scale ranged from 1-5 (total points: 9-45; Midpoint <3, [27]) a = post mean indicates unadjusted mean Question 2b. Does participating in the 12-week SATCM program result in a self- reported increase in teachers’ frequency of use of positive classroom management strategies as well as a decrease in frequency of use of negative classroom management strategies (i.e., Total 109 Positive Strategies, Inappropriate Strategies) over time (post-test differences controlling for pre- test) as measured by the TSQ? Results from the ANCOVA analysis did not demonstrate a significant difference between the SATCM treatment group and Book + Activity comparison group at post-test when adjusting for pre-test scores for the frequency-of-use of positive TCM strategies or frequency-of-use of negative TCM strategies (see Table 13). Similar to the perceptions of usefulness subscales, there was a positive trend in pre and post scores for both the SATCM treatment group and the Book +Activity comparison group (see Table 12). Additionally, when examining within group pre-post differences within descriptive statistics, it appears that scores increased by 23 points for the SATCM treatment group and 15 points for the Book + Activity comparison group at post-test for frequency of using positive TCM strategies. Use of negative TCM strategies decreased on average 1.5 points over time for the SATCM treatment group and 1.2 points over time for the Book + Activity comparison group. Table 13. ANCOVA Results of Teacher Frequency-of-Use Scores CI95% for Estimate Est. SE t p Lower Upper 30 G [CI95%] Positive TCM Frequency ANCOVA Pretest 0.63** 0.1 3.28 .001 0.25 1.01 Cov 9 ANCOVA Posttest 6.65 4.4 1.49 .136 -2.08 15.38 0.34 0.42 6 [-0.14 –0.98] Negative TCM Frequency ANCOVA Pretest 0.20 0.3 0.60 .556 -0.46 0.85 Cov 3 ANCOVA Posttest 1.28 2.1 0.59 .557 -2.98 5.54 0.05 0.20 7 [-0.46 – 0.86] Note. Hedges’ g was calculated using the t-value from ANCOVA posttest. ANCOVA posttest represents the between group estimates of the treatment groups when controlling for pretest 110 Table 13 (cont’d) measures. ANCOVA pretest estimates represents the correlation between pretest covariate on posttest outcome variable. *= p <.05, **, p<.001. Question 3a. Does the SATCM group demonstrate similar levels of treatment acceptability compared to the Book +Activity group at post treatment as measured by the TEQ- T? For the TEQ, treatment acceptability was considered adequate if participants had overall scores of 73.5, Acceptability greater than 38.5, Effectiveness greater than 28, and Amount of Time greater than 7. Based on post-test ratings, both the SATCM treatment group and Book + Activity comparison group demonstrated adequate acceptability. Independent samples t-tests were run with a 95% confidence interval to assess mean treatment acceptability differences between the SATCM treatment group and the Book + Activity comparison group (see Table 14). Based on the data collected, Levene’s tests did not indicate significant differences in variances between groups. Additionally, there were no significant mean differences between the SATCM treatment group and Book + Activity comparison group on the Overall score, Acceptability subscale, Effectiveness subscale, and Amount of Time subscale. Hedges’ g estimates were calculated using means and standard deviations in Cohen’s d multiplied by a correction factor (Hedges & Olkin, 1985, p. 110). Table 14. Teacher Acceptability Ratings of Treatment across SATCM Treatment and Book + Activity comparison groups CI95% t df p Mean Diff. SE Diff. Lower Upper g Overall score -1.70 20 0.10 -5.78 3.39 -12.86 1.31 -0.64 Acceptability -1.60 20 0.13 -2.22 1.38 -5.07 0.67 -0.62 Effectiveness -1.70 20 0.11 -3.37 2.04 -7.42 0.69 -0.64 Time -0.50 20 0.66 -0.22 0.48 -1.21 0.78 -0.26 111 Table 14 (cont’d) Note. Negative Hedges g’ estimates represent improvement in the Book + Activity group. Table 15. Average Teacher Acceptability Ratings across SATCM Treatment and Book + Activity Comparison groups Book + Activity SATCM N M SD N M SD Overall score 10 105.7 4.6 12 99.9 9.9 Acceptability 10 58.7 3.1 12 56.5 3.3 Effectiveness 10 37.7 2.7 12 34.3 5.9 Time 10 9.3 1.2 12 9.9 1.2 Note. Scores based on Treatment Evaluation Questionnaire (range from 1-6). Question 3b. Does the SATCM group demonstrate similar levels of treatment acceptability compared to previous research on the IY-TCM program as measured by the Teacher Workshop Satisfaction Questionnaire (i.e., acceptability, helpfulness of coach, strategies to teach skills, and techniques used), Teacher Video Evaluation Questionnaire (i.e., DVD acceptability), and teacher interviews (i.e., length of treatment, treatment barriers)? TWSQ. TWSQ measures were rated on a scale from 1 (not useful) to 7 (very useful) for three subscales 1) Materials, 2) Strategies, and 3) Coaching for 12 participants in the SATCM treatment group (see Table 15 & Table 16). Scores rated above the midpoint scale of 3.5 were considered highly acceptable. All three measures were rated as 5-somewhat useful to 6-useful indicating that the intervention had high acceptability. Within the Materials subtest, the average rating on the self-administered manual (M = 6.0, SD = 0.73) was slightly higher than the Incredible Years book (M = 5.92, SD = 1.2), demonstration of skills in the DVDs (M = 5.9, SD = .99), and the activities recommended in the treatment manual (M = 5.9, SD = .74). Within the Coaching subtests, participants rated the coach’s preparation and overall satisfaction as superior (M = 6.83, SD = .58) and the coach’s interest and concern in the participant and their student as superior (M = 6.75, SD = .62). This data is comparable to other group training studies who found that 80-90% of participants rated teaching techniques, strategies used to teach the program, and 112 the coach as useful or very useful (i.e., 5-7 on the TWSQ rating scale; Fergusson et al., 2013; Hicks-Hoste et al., 2015). Within the Strategies subscale, the highest rated item was the social-emotional coaching strategy where four participants (33.3%) found the strategy 6-useful and eight participants (66.7%) found the strategy 7-very useful. The lowest rated strategy was the time out/calm down strategy where two participants (16%) rated this strategy as 3-neutral, nine (75%) participants rated this strategy as 5-somewhat useful to 6-useful, and only one participant (8%) rated this strategy as 7-very useful. Fergusson et al. (2013) found similar ratings for social-emotional coaching and timeout/calm down strategies. Fergusson and colleagues found 6.8% of participants rated timeout as neutral, 66.2% found it somewhat useful to useful, and 26.1 % of participants rate the strategy as very useful (26.1%). Table 16. Average Values of Acceptability Scores Acceptability n Min Max M SD Coaching 12 5.0 7.0 6.8 0.6 Strategies 12 5.0 7.0 6.1 0.6 Materials 12 4.75 7.0 5.9 0.7 Note. Scores greater than 3.5 are considered moderately acceptable Table 17. Average Acceptability Ratings of SATCM Strategies IY-TCM Strategies n Min Max M SD Child-Directed Play 12 5 7 6.2 0.7 Academic and Persistence Coaching 12 4 7 6.3 1.1 Social and Emotional Coaching 12 6 7 6.7 0.5 Praise/Encouragement 12 5 7 6.3 0.9 Incentives to Motivate Children 12 4 7 5.9 1.1 Ignoring 12 4 7 5.5 1.3 Good Commands/Clear and Respective Limit Setting 12 5 7 6.3 0.8 Time Out/Calm Down Place 12 4 7 5.3 0.9 Loss of Privileges, Logical Consequences 12 4 7 6.4 1.0 Redirects/Distraction/Prompting Alternative Responses 12 4 7 6.0 1.0 Problem-Solving Training 12 4 7 6.3 0.9 Note. Scores greater than 3.5 are considered adequate acceptability 113 VEQ-T. Participants rated DVDs on a scale from 1 to 5 (not helpful to very helpful). Overall scores on the DVD sessions were rated with adequate acceptability (≥ 12) except for DVD 1: Building positive relationships (see Table 17), indicating participants on average found the DVD to be 2-somewhat helpful to 3-neutral. DVD 4: Motivating children through incentives had the highest mean score with descriptive scores indicating participants found the DVD 3- neutral to 4-helpful. These ratings were slightly lower than previous research on the VEQ-parent from Stewart and Carlson’s (2010) study (M = 14.44 - 15.73). Table 18. Average Acceptability Ratings of DVDs DVD N Min-Max M SD DVD 1: Building positive relationships 13 5-16 11.8 2.8 DVD 2: Preventing problem behaviors 13 4-16 12.0 3.4 DVD 3: The importance of teacher attention, coaching, and praise 12 8-16 12.1 2.5 DVD 4: Motivating children through incentives 11 10-16 13.5 2.2 DVD 5: Ignoring and redirecting 12 8-16 12.8 2.5 DVD 6: Follow through with consequences 12 8-16 12.1 2.3 DVD 7: Emotional regulation, social skills, and problem solving 12 8-16 12.8 2.5 Note. Scores 1 to 5 (not helpful to very helpful) with higher scores indicating higher acceptability. Scores ≥3 (neutral) were considered adequate acceptability. Overall scores ranged from 4 to 20. Teacher Interviews. Teacher interviews were organized by common themes found in the IY literature, the TEQ-T, and TWSQ scales. Specifically, themes were summarized by effectiveness, acceptability, treatment barriers/facilitators, and recommendations. Teacher interviews were conducted with 12 teachers from the SATCM treatment group at post-treatment by the PI (see Table 18 for a summary of interview findings) via zoom. A list of questions was pre-determined at the start of the interview and were categorized by each identified theme (see appendix (see Appendix K). Interview responses were then summarized in an excel table and were then quantified based on similarities/frequency of answers reported. Qualitative analyses 114 were informal as this data was meant to enhance the quantitative data of participant perspectives to inform future researchers. Themes summarized within this literature are comparable to other qualitative findings reported in Hutchings and colleagues (2013). For instance, within the current study, when participants asked what changes occurred within their classroom after implementing the SATCM treatment program, 10 (83%) teachers noted an improvement in classroom behavior, seven teachers (52%) felt that the program added “more tools to their toolbox” to address disruptive behavior or learned how to implement the strategy in a new, creative way, and all teachers noted an improvement within their own behavior (e.g., more patient, calm). This is comparable to Hutchings and colleagues who cited 95% of teachers found an improvement in their classroom behavior, 52% of teachers found an improvement in their own behavior, 91% felt better equipped to handle target children, and 33% felt calmer or less stressed. Table 19. Teacher Interview Themes of Acceptability Domain Themes Effectiveness Improvement of behavior (classroom, Reduced behavior difficulties in classroom and target, teacher) target student (n =10) Increased number of teacher strategies used (n = 7) Improved patience/calmness in teacher (n = 4) Increased confidence in teachers (n = 2) Preference More effective/preferred over other programs (n = 10) Usefulness of TCM Strategies Social-emotional tools (e.g., emotion chart, emotional coaching, emotion thermometer) (n = 4) Incentives/reward charts (n = 3) Redirection (n = 3) Other (n = 3) Praise (n = 2), Puppets (n = 2) 115 Table 19 (cont’d) Coaching Effectiveness Brainstormed ideas/developed behavior plans for target students/ created modifications (n = 7) Pushed new perspectives (n = 2) Supported with implementation (n = 2) Helped with accountability (n = 1) Acceptability Willingness to Use Strategies “Everything”(n = 5) Reward program/incentives (n = 3) Social-Emotional Strategies (n = 3) Relationship building (n = 2) Praise (n = 2) Other (building connections with parents, behavior plan) (n = 2) Accessibility Accessible to other teachers (n = 11) Ease Ease of implementation for teachers (n = 12) Barriers/Facilitators Implementation Barriers Implementation of skills (n = 5) Time (n = 4) Engagement with materials (e.g., reading chapters, DVDs) (n = 4) No challenges (n = 2) Most Helpful Aspect of Program DVDs (n = 9) Other (i.e., social-emotion lesson, behavior plan, manuals, handouts) (n = 4) Book (n = 3) Coaching (n = 3) Aspect of Program Participant Liked Most Training Tools (e.g., Handouts, DVDs, manual questions) (n = 6) Changed behavior (“it worked!”) (n = 2) Coaching (n = 2) New strategies (e.g., relationship building) (n = 1) Relatable (n = 1) Flexible (n = 1) Easy/simple to learn and implement (n = 1) Positive emphasis (n = 1) Aspect of Program Participant Liked DVD (n = 4) (e.g., boring, technology challenge) Least Repetitive across materials (n = 3) Timeout Feasibility (n = 3) Negative teacher examples (n = 2) 116 Table 19 (cont’d) Chapter reading (n = 2) Time Constraints (n = 2) Incentives (n = 1) Program Improvement Recommendations Observations with feedback from coach (n = 4) Teacher support group (n = 3) Updated videos/technology (n = 2) Audiobook (n = 2) More time for each session (n = 2) 117 CHAPTER 5 DISCUSSION This small RCT pilot study is the first to exclusively examine the fidelity, effectiveness, and acceptability of the Incredible Years SATCM treatment program with a coaching component (Webster-Stratton, 2009) compared to a Book + Activity comparison group for teachers who presented as at-risk for poor classroom management. The current study extends previous literature (Shernoff & Kratochwill, 2007) by examining “at-risk” teachers identified through a cut-off score as previous literature indicates that at-risk teachers may need more time to independently learn the material and may benefit from learning at their own pace (Louws et al., 2017). Findings from the current study appear to show that novice/at-risk teachers can benefit from the use of self-administered professional tools to improve their perceptions and use of classroom management skills. Additionally, findings highlight the potential need to have accessible self-administered classroom management PD trainings as a part of a cost-effective tiered approach to support teachers who struggle with effective classroom management strategies. Adherence to Treatment Sessions High rates of treatment fidelity were found in those who completed the SATCM treatment group (96%) and those who completed the Book + Activity comparison group (94%). Both groups demonstrated greater than 90% fidelity across sessions indicating a very high level of treatment implementation compared to desired rates (Durlak and Dupre, 2008). The high rates of implementing the self-administered procedures as intended found in this study were considerably more impressive than the moderate fidelity ratings (i.e., 69%; 68-78%) reported within other IY self-administered studies (Kratochwill et al., 2003; Osburn, 2009) and were 118 significantly higher or in-line with other group training studies (i.e., 58%-95%, Baker- Henningham & Walker, 2018; Ford et al., 2019; Hickey et al., 2017; Hutchings et al., 2013; Murray, Rabiner, & Carrig, 2014, Raver et al., 2008). Additionally, previous research demonstrates higher treatment fidelity for some components of the program (e.g., 92-100% for videos watched; 79-96% of skills practiced; Ogg & Carlson, 2009) than for others (40-79% manuals completed; Walcott et al., 2009). In the present study, all treatment components were in the “high fidelity” range, indicating that these components were feasible for teachers to complete in the context of their own home. High fidelity scores highlight promising data for self-administered interventions as disruptive innovations are intended to reduce implementation barriers that can occur in the group training programs (i.e., time commitment, length of treatment, buy-in) and increase access for novice and high-need teachers (Forman et al. 2009; Louws et al., 2017). Specifically, high-risk teachers (i.e., with self-identified classroom management skill needs) identified in the present study (i.e., daycare workers, preschool teachers, and paraprofessionals), with several noted barriers (e.g., time, COVID-19 related stressors), successfully displayed superb adherence of session protocols following a manualized evidence-based treatment approach. This also supports evidence that providing clear expectations and reminders within a manualized treatment approach may promote higher fidelity of an intervention (Marchette & Weisz, 2017) and may be more important than a coach to support implementation success. Although there was high fidelity and adherence to program components for participants who completed the training, 28% of participants recruited dropped out before starting the program. As a result, this could have affected the external validity of treatment outcomes as participants who stayed in the study were highly motivated to improve classroom behavior and 119 were not discouraged by the time commitment and COVID-19 barriers presented. Because of the higher drop-out rate, these outcomes should be interpreted with caution as these results may only generalize to those teachers who are highly motivated to make changes in their classroom management practices. Coaching Adherence and Inter-Rater Reliability Similarly to high adherence ratings for self-report measures, the IY coach was able to implement the treatment with greater than 60% fidelity (Durlak & Dupre, 2008) based on average self-report ratings of 81% and adequate inter-rater reliability ratings (Session 3: 83%, Session 6: 75%). These findings are slightly lower or in-line with previous IY-TCM group training program research that indicates adequate fidelity implementation from group leaders (>80%; Hickey et al., 2017; Leckey et al., 2016; Murray, 2017). The IY coach and the PI demonstrated acceptable levels of inter-rater agreement percentages (75% to 90%; Hartmann, 1977; Stemler, 2004) for both session ratings (Session 3: 83%, Session 6: 75%). Interestingly, there was lower agreement on session 6 between the IY Coach and the PI. Disagreed items included “review self-reflection form” “review further video vignettes,” and “create ideas to implement strategies with children’s parents.” Although inter- rater agreement levels were still in the adequate range (>75%), one reason to explain the lower reliability scores were the result of less systematic evaluation of the coach due to time constraints. According to Domitrovich (2008) programs are more likely to be implemented with high fidelity when support systems are in-place and when trainers are systematically evaluated. The current IY coach received extensive four-day training from the developers and was required to provide two recordings of training sessions to receive feedback and become a certified IY Coach. Although it was the PIs intention to provide systematic evaluation throughout the study, 120 due to time constraints, systematic evaluation was provided at the end of the study. To provide training, the PI met with the coach before and after session 1 to discuss procedures for delivering materials and reviewed Zoom observations for session 3 and session 6 after the completion of the study. Durlak and Dupre (2009) indicate that having an outside observer provide feedback on the fidelity of an intervention can increase overall fidelity and implementation success. Thus, less systematic evaluation could have led to more disagreed upon item responses from the fidelity checklist. Overall, fidelity data provided in the present study suggests initial evidence for the ease of adhering to procedures provided in self-administered intervention programs—a common treatment barrier to implementation success. This also adds evidence within the PD literature that using self-administered manualized interventions as disruptive innovations may be a helpful way to address the barrier of access to PD training programs for teachers. Effectiveness of Teacher Classroom Management Strategies The hypothesis that there would be a significant difference in treatment outcomes between groups –specifically that the SATCM treatment group would demonstrate an increase in confidence, a reduction in perceived usefulness and frequency of use of negative TCM strategies, and an increase in perceived usefulness and frequency of use of positive TCM strategies compared to the Book + Activity comparison group—was not supported within this pilot study. However, means within both groups improved similarly over time for all three dependent variables at post-treatment. These results are somewhat consistent with other IY literature comparing treatment groups. For instance, Hickey et al. (2017) completed an ANCOVA analysis using the TSQ to compare the IY-TCM group training program to a wait-list control group and found a difference 121 in all treatment groups except for perceived usefulness of negative TCM strategies. Webster- Stratton (1990) compared an SAPT group to an SAPT group with coaching and found a significant difference with mother’s use of no-opportunity commands favoring the SAPT with coaching group; otherwise, both groups performed similarly in changing parent behaviors. Shernoff and Kratochwill (2007) examined between and within group differences using change scores and reported statistically significantly greater mean increases on the TSQ proactive teacher strategies subscale (M =3.7) at post-treatment for participants who received the SATCM program with consultation compared to the SATCM only program (M = 3.4). However, Shernoff and Kratochwill’s study included a small sample size and did not control for pre-test differences indicating that some of the post-test differences could be due to error. One explanation for the similarities in both groups includes the similarity of evidence- based manualized PD components within each group. Researchers indicate that in-service professional development is most effective when the training involves the following components: 1) active learning, 2) coherence, 3) content-focused, 4) collaborative participation, and 5) sustained over time (DeSimone & Garet, 2015; Wei, Darling-Hammond, & Adamson, 2010). Because the SATCM treatment group and the Book + Activity comparison group performed similarly, this may suggest that the effectiveness of the Book + Activity comparison group as a standalone treatment may include enough components to increase comparable behavior change to the more expensive SATCM treatment group. For instance, both groups received evidence-based TCM approaches (e.g., building relationships, promoting positive behaviors through attention and reinforcements, and reducing negative behaviors through ignoring, timeouts) through differing methods (i.e., video, coach and/or book) and were able to participate in evidence-based PD strategies that included some components of active learning 122 (i.e., practiced skill implementation each session, received feedback from coach, or observed other teachers through the DVDs), coherence (i.e., learned strategies linked to beliefs/goals by writing goals and reflections for each session), content-focused (i.e., learned how to implement skills with students from a book, DVDs, or coach), collaboration (i.e., collaborated with peers in daycare center or coach), and sustained over time (i.e., 6 sessions over 12 weeks). Although both groups did not have a formalized group to collaborate and discuss opinions as indicated in the group training program, both groups shared informally and via qualitative interviews that they worked with peers within their classroom or daycare center when completing assignment. Thus, it might be enough for participants to collaborate with their peers, rather than hosting a formalized training program so that participants can have a collaborative experience. Additionally, previous research has discussed that bibliotherapy and self-help books can be an effective way to change people’s behavior (Louws et al., 2017; Rotheram-Borus et al., 2012). Because the Incredible Years Teacher book contains comparable information to the information presented in the DVDs and manuals, and because both groups were presented in a systematic format, it could be that the DVDs and coaching components were not necessary to outperform the Book +Activity group. Another explanation for similarities in improvement may be that both groups contained “high-risk” teachers. Within the current study, the average treatment score on the pre-screening assessment before beginning treatment was 67 (i.e., 2-sometimes or 3-half the time on a five- point Likert scale for frequently using positive classroom management strategies). The current study’s baseline score was much lower than the baseline score for Hickey et al. (2017) IY-TCM group training study (M = 3.6 [~82 raw score]) who assessed teachers in Ireland that taught Junior and Senior infant classes and had higher levels of education and experience. Additionally, 123 the current study saw greater gains at post-treatment with an average post-treatment score of 83.9 for both groups, representing a 16-point increase at post-treatment, compared to Hickey and colleagues (2017) who had a post-score of 92, representing a 10-point increase at post-treatment. The current study results parallel previous research indicating that at-risk teachers (e.g., paraprofessionals, limited education/training) show greater improvements in utilizing TCM strategies and reducing negative classroom management strategies (e.g., d= 3.35; Baker- Henningham & Walker, 2018) than teachers who have higher levels of education and experience (e.g., d = 0.17; Hutchings, Martin-Forbes, Daley, and Williams, 2013). Thus, it may be more beneficial to identify teachers at-risk to improve classroom behavior as teachers are seen as the primary mechanism of change to improve child behavior. A final explanation of why these studies performed so similarly could be that the inferential statistics testing was affected by the study’s small sample size resulting in a type II error. This is evidenced by the wide confidence intervals demonstrated in the results which indicates that the SATCM with coaching should not be ruled out as a possibility for being a more effective treatment for teachers struggling with TCM difficulties. Additionally, several previous researchers support the use of coaching to promote collaborative discussions as an effective tool to change beliefs (Forman, 2009; Reinke et al., 2012) and that adding a coach may help teachers improve implementation fidelity of an intervention, generalize skills in PD, help teachers with negative self-talk and beliefs about student behavior, and help teachers who are especially at risk for poor TCM strategies (Driscoll, Wang, Mashburn, & Pianta, 2011; Reinke et al., 2015; Stormont et al., 2014). Thus, this study should be replicated on a larger scale to determine if coaching can be an effective treatment for struggling teachers. 124 In sum, although this study did not capture the behavioral outcomes demonstrated in pervious literature on the use of coaching, the hypothesized antecedents (i.e., perceptions of use and frequency of use) did change for both groups within the current study, providing evidence that this study should be replicated on a larger scale. Additionally, these results emphasize the potential utility of a self-administered intervention as a disruptive innovation for PD to improve TCM strategies. When compared to an interactive coach, participants demonstrated similar changes in perceptions of usefulness and frequency of using positive TCM strategies occur regardless of group membership indicating a solitary, interactive, cost-effective PD such as book reading with activities that contain clear expectations for active learning may be another helpful PD approach to promote positive classroom management strategies. Thus, it may be helpful for school administrators to conceptualize these findings as a potential tiered approach for PD for teachers struggling with TCM—in other words, less costly training materials for teachers who are identified as needing support and more intensive supports for those who fail to respond to solitary/active learning approaches. Acceptability between the SATCM and Book +Activity Group Analysis using the TEQ-T which examined overall acceptability, effectiveness, and time required, indicated that acceptability between the SATCM treatment group and the Book +Activity comparison group did not significantly differ (see Table 14). This is somewhat consistent with previous literature. For instance, Shernoff and Kratochwill (2007) found teachers in the consultation group had slightly higher acceptability scores ranging from 35-42 compared to the self-administered only group which ranged from 27-35 and results were statistically significant (p<.05, 50 =.49). Shernoff et al. (2003) reported that although they found high mean scores on the Acceptability (M=71, SD=8), Effectiveness (M=33, SD=7), and Time subscales 125 (M=8, SD=2), they did not find any significant difference between their manual only group and video + manual group. One rationale that could explain similar acceptability scores rather than higher acceptability scores in the SATCM treatment group, was the number of tasks required of the SATCM group versus the Book + Activity comparison group. On average, SATCM participants had to complete double the hours of work (6 hours versus 3 hours per week) compared to the Book +Activity participants. Additionally, SATCM participants described in a qualitative interview that the requirement to read the book, watch the DVDs, and practice the activities seemed repetitive at times as similar information was presented across delivery methods. Some SATCM participants also shared that they felt they did not have enough time to complete all of the activities and practice their skills, while Book + Activity comparison group participants never indicated difficulties with time. Forman (2009) indicates that time can be a significant barrier to implementation success for EBIs. Researchers Harwood and L’Abate, (2010) indicated that using a self-help book that motivates and outlines a step-by step change format can be an easy way to transport research into practice. For teachers in the Book + Activity comparison group, using a manualized step-by-step bibliotherapy book with activities to practice skills may have been less time consuming, been easier to implement, provided teachers more autonomy to deliver an intervention on their own time, and helped teachers receive more direct instruction on new teaching practices (Louws et al., 2017). Despite no significant difference between groups, both groups were greater than the midway (>73.5) and demonstrated adequate to high acceptability (Book: M = 105, SD = 4.6; M = 99.9, SD = 9.9). These results mirror other IY SAPT studies (e.g., M=100.5, SD = 11.69; Stewart & Carlson, 2010; Shernoff & Kratochwill, 2007) indicating teachers generally found the 126 treatments to be fair, appropriate, and effective interventions for increasing teacher classroom management skills and reducing disruptive behavior. Equally high scores in both groups may be because sequential and reciprocal interactions between fidelity and effectiveness lead to higher acceptability. For instance, the research emphasizes that if something is perceived as effective, it is more likely to promote overall acceptability of an intervention as well as indicate a higher likelihood of transportability of either treatment group into real world contexts as acceptability is associated with increased likelihood of intervention adoption and compliance (Witt & Elliot, 1985). Additionally, previous IY-TCM research finds that implementing a program with high- risk teachers improved perceptions of positive classroom management strategies, confidence in managing future behavior problems, and resulted in a higher level of satisfaction with the program and certified trainer (Hicks-Hoste et al., 2015). Acceptability of the SATCM Intervention Proctor and colleagues (2011) discuss that acceptability is often influenced by the intervention format, contents of the program, perceived effectiveness in improving behavior, and strategies taught within the program. Thus, this study examined the acceptability of the intervention format, contents, materials, strategies, and coaching to ensure these aspects of the training did not serve as implementation barriers. As hypothesized, analysis of the SATCM materials, strategies, and coaching, all signified high acceptability ratings. These results mirror high content acceptability ratings (Materials: M=5.9; Strategies: M=6.1; Coaching: M = 6.8) found in other studies (TWSQ materials, strategies, and coaching scores: 5-7, Fergusson, 2013). This finding was expected as previous research indicates that interventions that include both positive and negative strategies are perceived as more acceptable than interventions lacking these characteristics (Cowan & Sheridan, 2003). Additionally, previous research indicates programs 127 that are manualized, self-administered, and include a coaching component are more acceptable as they allow teachers autonomy to deliver an intervention on their own time and receive more direct instruction on new teaching practices (Louws et al., 2017). Interestingly, participants rated timeout as the least acceptable strategy compared to other evidence-based strategies where two participants (16%) rated this strategy as neutral, nine (75%) participants rated this strategy as 5-somewhat useful to 6-useful, and only one participant (8%) rated this strategy as 7-very useful. Qualitative data also indicated a dissatisfaction with timeout as many of the participants both in daycare centers and home daycares indicated that timeouts were very inefficient in their classroom and difficult to implement, especially when they were the only teacher in the classroom. Participant ratings compare similarly to Fergusson et al. (2013) who indicated that 6.8% of participants rated timeout as neutral, 66.2% found it somewhat useful to useful, but found more participants rate the strategy as very useful (26.1%). However, in Fergusson et al.’s study, participants rated child-directed play rather than timeout as the least acceptable strategy. Social-emotional skills, on the other hand, were considered the most acceptable strategy where four participants (33.3%) found the strategy 6-useful and eight participants (66.7%) found the strategy 7-very useful. This also compares similarly to Fergusson et al.’s study, however, more participants rated social-emotional strategies as somewhat useful to useful (62.9%) and 35.8% of participants rated the strategy as very useful. Given the era of promoting SEL within classrooms, all teachers appeared to be motivated to learn about techniques that improved SEL skills for both teachers and students. Explanations for lower ratings of the timeout strategy and higher ratings on SEL strategies aligns with previous research which indicates that the best PD is coherent (i.e., the PD content and activities are linked to teacher beliefs/goals to connect to the “big picture,”; 128 DeSimone & Garet, 2015; Wei, Darling-Hammond, & Adamson, 2010). Additionally, teachers want PD programs that can help them learn skills that align with their classroom goals and help them deliver skills, techniques, and strategies that address their students’ individual needs quickly (Matherson & Windle, 2017). Thus, based on the research and ratings of this study, it seems that the use of timeout in the classroom did not align as well with teachers’ beliefs and was difficult to implement without support while SEL learning fit with teacher beliefs, daycare center goals, and were possibly easier to implement. DVD acceptability. This was the first SATCM study and first IY-TCM study to examine the content acceptability of the DVDs. In line with our hypotheses, ratings on the VEQ-T (M=11.8-13.5) were considered moderately acceptable for all DVDs (>12) except for DVD 1: Proactive Strategies (M=11.8). Although these ratings are slightly lower than previous research on the VEQ-parent from Stewart and Carlson’s (2010) study (M=14.44-15.73), these findings were expected as the ease of consuming information through multimedia delivery methods have been suggested to improve acceptability of evidence-based programs (Rotheram-Borus et al., 2012). Additionally, Rotheram-Borus et al., (2012) discuss disruptive innovations, which use novel delivery formats such as media platforms help to improve implementation and dissemination. Interestingly, lower scores were indicated on the Proactive Strategies DVD. This is surprising especially because building relationships is considered fundamental in creating a warm environment for students to learn (Schonert-Reichl, 2017). One rationale that may explain lower ratings is the amount of time needed to view the DVDs. For instance, research indicates that time can be a substantial barrier to implementation success and pacing assignments too quickly can lead to poorer treatment outcomes (Webster-Stratton et al., 2011). Within the current 129 training program, the average length of each DVD was around 60 minutes and instructions required participants to pause the DVDs to reflect and answer questions, extending the amount of time needed to watch. This also may explain why the first DVD rating was so low. Because participants were required to watch two DVDs within the first two weeks to mirror the group training program, participants may have found this overwhelming, thus lowering the acceptability score of the first rated item. Despite the lower scores on the first DVD, the adequate acceptability ratings on the DVDs are promising and indicate the potential use of the SATCM program as a disruptive intervention to improve implementation success for daycare center workers as higher acceptability has been linked to improved rates of fidelity and effectiveness. Additionally, using DVDs has the potential to address implementation issues related to access to expert trainers as watching DVDs do not require contact between an expert and trainee (McCyntire & Neece, 2016) that are not always available for daycare center providers. Limitations The present study is limited by (a) treatment dosage (b) teacher-collected data, (c) coaching fidelity, (d) randomization, (e) generalizability of results, and (f) research analysis. Treatment dosage. In the current training, the program length was modified to reduce recruitment barriers and to allow enough time for all participants to receive the intervention within the school year due to limited materials. Specifically, participants were given materials at 2-week intervals and received a 30-minute coaching session at each treatment session. Although participants had similar levels of training compared to the group training program (6 hours/session = 36 hours compared to 7 hours/session =42) and met the criteria for best practice components of PD (10-12 sessions, Desimone & Garet, 2015), the reduced time frame likely 130 limited participants opportunities to practice implementing positive classroom management training skills before their coaching session. Thus, this may have reduced appropriate connections between positive classroom management strategies and negative classroom management strategies as well as reduced perceptions of usefulness of positive TCM strategies. Future research should examine the differences between training intensity and duration between trainings to see if this increases acceptability, implementation success, and use of positive TCM skills, especially for teachers at risk who might require more intense training. Additionally, future researchers may want to include a separate session for manual 1 rather than attempting to mirror the group training session as participants noted it was difficult to complete both sessions within the two-week time frame. Attrition. The current study had a 28% attrition rate of participants with most dropping out before starting the intervention. Previous research indicates that participants are most likely to drop out of an intervention due to time, motivation, and job turn-over (Long, Dubois, and Faley 2009). Within the current study, participants appeared to drop out prior to starting the study due to perceived time barriers, family barriers, or unwillingness to complete the program. Additionally, participants faced complexities created by the COVID-19 pandemic such as illness, state requirements for smaller classrooms sizes affecting income, and strict cleaning requirements which created additional time and motivation barriers. As a result, the participant sample included teachers who were highly motivated to complete the study and were able to face barriers created by the pandemic. Because of the high dropout rates, this may indicate more difficulties with potential treatment implementation success in real-world contexts for teachers and more specifically, motivating teachers to get started with training programs. Future research should collect more specific data on drop-out reasons and participant characteristics to determine 131 which type of teachers are most likely to drop out to address potential barriers for completing the training programs. Teacher-collected data. Another limitation is that this study relied on self-report analysis for both teacher-rated effectiveness and teacher-rated integrity instead of multiple forms of data collection (e.g., observations). Due to restrictions from COVID-19, all observation measures had to be removed from the present study to ensure safety of RAs and teachers. As self-report data is notoriously erroneous and tends to have an upward bias in teacher outcomes due to social desirability, having a secondary data source using observations is important to create a more accurate assessment of improvement (Kazdin, 2017). Additionally, changes in the SATCM treatment group might have been more prominent through behavioral observations as opposed to self-report measures. For instance, other studies utilizing behavioral observations measurements have found coaching to be an effective strategy to help teachers improve implementation fidelity of an intervention, generalize skills learned in PD training sessions, and help differentiate instruction for teachers with various backgrounds and concerns (Driscoll et al., 2011; Reinke, Newcomer, Marchese, and Lewis, 2015; Stormont, Herman, & Newcomer, 2014). Thus, including an observational component in future studies is essential to examine behavior changes between the two groups. Coaching fidelity reliability. Although it was the PIs intention to provide systematic evaluation throughout the study, due to time constraints, systematic evaluation was provided at the end of the study. To provide training, the PI met with the coach before and after session 1 to discuss procedures for delivering materials and reviewed Zoom observations for session 3 and session 6 after the completion of the study. This presents an additional limitation as not all participants may have received all of the intended treatment components within the coaching 132 session which could have impacted effectiveness outcomes for the SATCM treatment group. Thus, these results should be interpreted with caution. Randomization. Another limitation is that this study did not use true block design to ensure equal group sizes due to time constraints. Specifically, within the current study, participants were blocked into sets of two or sets of seven which created unequal group sizes. Kazdin (2017) indicates that grouping subjects into blocks or sets can help avoid unequal group sizes in randomization without violating random assignment. Kazdin also reports that equal groups can help improve power of statistical tests and convenience when conducting statistical analyses. Additionally, equal group sizes and sets can make it easier for future researchers to replicate the current study. Future research should include a block design to ensure equal number of participants, improve convenience and statistical power, and improve replicability of future study findings. Small sample size. Although this was a pilot study, the small sample size likely underpowered the results of this analysis leading to risk of Type II error. For instance, the wide confidence intervals within the effectiveness questions indicate that the SATCM treatment groups should not be ruled out as a possibility, as small sample sizes could be creating increased error variance in the outcomes. Thus, the outcomes of this study should be interpreted with caution as the small sample size makes it difficult to conclude that these two treatments performed similarly. Generalizability of results. A majority of the participants were daycare center workers and home daycare workers who tend to have a different experience than general classroom teachers when applying classroom management principals, thus limiting the generalizability of the results. For instance, many of the home daycare workers had an average class size between 3- 133 9 students which is significantly smaller than the average classroom for elementary school. Unfortunately, due to COVID, most classroom teachers were engaged in virtual teaching for the duration of this study. This presents an additional limitation as these results cannot translate to all teacher classroom experiences. Thus, these results should be interpreted with caution. Research analysis. A final limitation for this study was within the data analysis. One of the strengths of an ANCOVA model is that it holds the pre-test variable constant to ensure any changes in the post-test are not due to differences between the pre-test scores of the assigned groups (Gliner et al., 2003). However, the downside of this approach is that ANCOVA does not address change over time. As the mean scores within SATCM treatment group and Book + Activity comparison group appeared to improve over time, analysis of pre-post changes may be helpful to determine the effectiveness of these interventions on TCM skills improvement. Implications for Research This study’s small pre-post comparison group pilot RCT design with qualitative interviews addresses Bowen et al.’s (2009) recommendation when designing a feasibility study. Specifically, when there is some initial research to support a program (i.e., Shernoff & Kratochwill, 2007; Webster-Stratton [1990]), with a narrow focus on fidelity, effectiveness, and acceptability of an intervention, Bowen recommends the use of a pilot small-scale RCT experimental design with a comparison group to find out if the intervention could work. Although research findings indicated similar outcomes between groups, the findings add to current professional development TCM literature and self-administered literature as only one study previously examined the SATCM study to a comparison group. Findings from this study highlight the potential for future research. 134 Sheridan (2014) recommends consideration of a progressive 10-step intervention trajectory model to identify the most appropriate stage of research development when examining an intervention. Following Sheridan’s intervention direction, Step 1 focused on identification of an issue (i.e., TCM problems) and Step 2 focused on strategies to address the issue (i.e., SATCM training program). The present study fulfilled Step 3, conducting a pilot study to assess the feasibility of the intervention through examining the fidelity, effectiveness, and acceptability of the training program. Due to the outcome of these findings, the next step of the research can include Step 4, evaluating the SATCM study with increased methodological rigor to get a better understanding of the components of the intervention. To fulfill step 4, study replication should be implemented with increased methodological rigor with an observation component on a larger training scale to get a better understanding of the treatment components of these interventions as this may demonstrate larger training effects between groups (Sullivan, 2012). Thus, larger sample sizes may indicate significant differences between the results. The current study adds to the TCM literature that identifying teachers through a cut-score may be beneficial to see greater improvements at post-treatment. Future research may want to include a measure to screen and identify at-risk teachers at the beginning of the school year to demonstrate which teachers improve. Additionally, future research may want to examine different levels of at-risk teachers (e.g., moderate, severe) to compare differences in responses to the Book + Activity comparison group to the SATCM treatment group. Finally, future research should include an observation component to see if at-risk teachers have greater improvements with an additional coaching component than teachers without a coaching component. The current research also adds to the PD literature self-administered programs may be enough to increase TCM skills. Previous research emphasizes the importance of a collaborative 135 component for the most effective PD (DeSimone & Garet, 2015; Wei, Darling-Hammond, & Adamson, 2010) and qualitative interviews indicated a preference for having a support group or online blog to discuss difficulties with other teachers. Because of the small sample size in this study which increases the potential or type II errors, future studies should replicate this study on a larger scale to determine if these two treatments were both equally effective. Additionally, due to the feedback researchers may want to replicate this study by adding in a more intensive collaborative component with other participants to assess if this increases success for at-risk teachers. Some suggestions for adding a collaborative component include pairing participants with a buddy at pre-treatment or including a message board with guided discussion questions (e.g., Taylor et al., 2008). Additionally, researchers should consider comparing the acceptability, fidelity, and effectiveness of the SATCM training compared to the IY-TCM group training study and a Book + Activity comparison condition. This could help uncover if these specific training formats may inform transportability of PD in real world settings and if a self-administered format performs equally well to the more expensive group training format. Additionally, to address limitations with the current study, it would be helpful to include both a self-report measure and a behavioral observation component to see if coaching support behavioral changes above and beyond a Book + Activity comparison group. Researchers should also closely examine the qualitative feedback presented within the study. For instance, many participants found the DVDs to be clunky and difficult to administer. Since implementing the training study, the IY-TCM program developers have created an online learning platform to replace DVDs which may help to improve acceptability and accessibility of the DVDs. Although DVDs are helpful for research administration purposes, online learning may 136 be more accessible to most child caregivers. For instance, of the 17 participants initially enrolled in the treatment study, six needed a DVD player mailed to them to view materials. Online modules may also increase acceptability ratings thus indicating higher transportability. Future research examining the SATCM study should examine the acceptability of the video modules in the online streaming format to see if this improves acceptability. Implications for Practice Initial research has demonstrated the promising role self-administered interventions have for building behavioral management skills and social-emotional competence in both parent, teacher, and child behavior (Kratochwill et al., 2003, Shernoff & Kratochwill, 2007, Taylor et al., 2008). The role of this study was unique as it was the first to examine the fidelity, acceptability, and effectiveness of the SATCM program with a comparison group for high-risk daycare center workers, home daycare owners, and preschool teachers within a global pandemic. Additionally, it served to provide new modes of delivery for professional development training to address disruptive classroom behavior during a global pandemic. This study informs initial implications for practice. First, this study adds to the current literature that evidence-based TCM interventions that utilize praise, positive relationships, limit setting, incentives, ignoring, and parent-teacher collaboration (Doll et al., 2013; Jones & Jones, 2015; Pianta et al., 2012; Webster-Stratton, 2012) have the potential to promote prosocial behavior in children both short and long-term outcomes and enhance teacher skills and well-being (Ford et al., 2019). Improvements of TCM skills in self-report measures and qualitative narratives were demonstrated in both the SATCM treatment group and the Book +Activity comparison group. Improvements in teacher behavior for both treatment groups emphasize important steps in optimizing management of classroom behavior 137 and promoting positive interactions with students in early childhood within a self-administered format. Second, this study adds to the current literature as high fidelity (e.g., >80%), effectiveness, and adequate acceptability (e.g., greater than the midpoint), were demonstrated for both groups. This a positive finding given that interventions tend to have a sequential and reciprocal relationships between treatment acceptability, fidelity, and effectiveness indicating that if an intervention is more highly acceptable and easy to carry out, they are more likely to be implemented successfully (Witt & Elliot, 1985). At a time when interventions are moving to a virtual format, this disruptive intervention has the potential to increase accessibility of PD training and increase the likelihood of longer-term participation for individuals in hopes to prevent future behavioral difficulties in the convenience of their own home. Third, the current study findings appear to show that novice/at-risk teachers can benefit from the use of self-administered professional tools to improve their perceptions and use of classroom management skills. Given the initial success of a more cost-effective Book + Activity treatment option and the improvement in less time (i.e., 32 total hours vs 21 total hours), schools or daycare programs may be more inclined to use the Book +Activity intervention as a first line of treatment for teachers at-risk to promote TCM skills. One consideration for practice within the school system is utilizing school psychologists as a cost-effective way to implement coaching strategies for teachers who need more support in the school system. Preschool and daycare centers are often connected to a school psychologist (a professional who is trained in evidence-based TCM strategies) who can aid in the implementation of evidence-based materials. To save time and money, schools may consider identifying teachers at the beginning of the school year though a screener and providing 138 identified teachers with the Book + Activity format only. For teachers with difficulties implementing the program, schools can then utilize the school psychologist to provide additional support. For home daycare workers, accessing a coach may be more difficult. Thus, it might be more cost-effective, convenient, and beneficial for those coaches to start with the Book + Activity format only and access activities on the IY website—especially as this format performed similarly to the SATCM treatment group within the current study. For teachers who are seeking additional support, it may be beneficial for the IY developers to create a list of remote web-mediated online coaches that home daycare workers can access and sign up for an affordable IY coach. Currently, the IY website has online consultation to address program implementation for coaches to run online groups; however, no coaches are available to access for individually run home daycare programs. Continued exploration of the SATCM program in applied setting will help to uncover which components are essential in implementing this program in real-world contexts to address teacher’s individual needs. Conclusion Twenty-four teachers identified as at-risk participated in the small pilot RCT study examining the fidelity, effectiveness, and acceptability of the of the SATCM treatment group that included a coaching component compared to a Book +Activity comparison group. As a feasibility pilot study with a qualitative component (Bowen et al., 2009), the purpose of this research was to examine the initial fidelity, effectiveness, and acceptability to determine if this study should be replicated on a larger scale. Results from the research did not indicate a significant difference between groups; however, they did unexpectedly reveal similarly higher levels of improvements within fidelity, effectiveness, and acceptability for both an expensive SATCM program with coaching compared to a more cost-effective Book + Activity comparison 139 group. Ultimately, these findings will help lead to the increase in accessibility of affordable, self- paced treatments to address previous implementation barriers within the PD research for teachers. Additionally, these findings indicate that effective PD approaches with or without coaching may promote TCM strategies of at-risk teachers. This may lead to the development and use of more professional development training programs as a part of a cost-effective tiered approach to support at-risk teachers with effective classroom management strategies. In conclusion, this study adds to the self-administered intervention literature by specifically examining fidelity, effectiveness, and acceptability of the SATCM program with coaching compared to a Book +Activity group with an at-risk group of teachers. This study was also the first to conduct a small randomized controlled trial, first to compare the effectiveness and acceptability between groups, and first to examine content acceptability of the SATCM program using the VEQ-T. The limitations from this study should be evaluated and corrected in future research to provide support for future implementation. 140 APPENDICES 141 APPENDIX A Inclusion and Exclusion Criteria Instructions for Research Assistants: Please check all those that apply. If all criteria are not met, then the teacher is not eligible for participation in this research. Inclusions: ________ Works at least 3 days a week ________ Work as a preschool teacher/elementary school teacher, paraeducator/parapro, assistant teacher/teachers aid, afterschool care worker, childcare worker, intern/practicum student, first year teacher ________ Works with children 3-8 years old ________ Works in-person during COVID-19 pandemic Exclusions: _______Rated above 92 on Teacher Strategies Questionnaire cut-off measure _______Has received the IY-TCM program before 142 APPENDIX B Cut-off Score 143 APPENDIX C Demographics Questionnaire Contact Information: (This will only be used to Type of School allow researchers to inform you if you have met • Private eligibility for the research study): • Public • Charter Name: • Head Start/Great Start Readiness Email: Program (GSRP) • Daycare Center • Other: Gender Identity: Number of Years Teaching • Woman, female or feminine • 1-5 years teaching • Man, male, or masculine • 6-10 years • Gender non-conforming, gender queer, or • 11-20 years gender questioning • Transgender woman, female or feminine • More than 20 years • Transgender man, male or masculine • Two-spirit • I prefer not to answer Age: _______ Number of Year Teaching Current Grade • 1-5 years teaching • 6-10 years • 11-20 years • More than 20 years Race/Ethnicity: Current Role • Latino/Hispanic • Lead Elementary School Teacher • Black/African American • Lead Preschool Teacher • White/Caucasian • Lead Daycare Center Teacher • Native American • Paraeducator/Paraprofessional • Asian/Pacific Islander • Teachers’ Assistant/Teacher’s Aide • Two or more races • Before/After School Care Worker • Some other race, ethnicity or origin • College Intern/Practicum Student (Early education or elementary ed) • Other Level of Education: Have you received the Incredible Years Teacher • Highschool or GED Classroom Management Training before? • Some College • Yes • Associate degree • No • College Graduate • Post-College Degree Grade Level of Students Do you work in the school at least 3 days per • Preschool (3-5 years) week? • Developmental Kindergarten • Yes 144 • Kindergarten • No • First grade • Second grade • Third grade Do you currently have direct, in-person contact with Do you plan on staying in your current position students? for the next 12 weeks? • Yes • Yes • No • No 145 APPENDIX D Example of Teacher Self-Monitoring Checklist Session 1 Checklist: Treatment Group Directions: Please complete this survey fidelity checklist to the best of your ability after completing your coaching session and your self-administered workbook to be able to earn your gift card and PD credits at end of treatment. Circle yes if you completed ALL the items in a section. If you did not complete the activity, please select “no” and leave a comment so we can understand potential barriers and how to improve the program. Session #1 Checklist: Building Relationships with Students and Proactive Teaching Intervention Task Completed? Comments Part 1: Building Relationships with Students Read and followed directions, and reflected on discussion Yes No questions provided in manual Read Chapters 1, 2, & 14 from Incredible Teachers book Yes No Watched DVD Vignettes Yes No DVD 1: (Vignettes 1-30, 24 minutes) Completed Brainstorm/Buzz Worksheet activities from Yes No Workbook Part: • Promoting a Sense of Responsibility • Changing Negative Reputations • Building Relationships with Students • Building Relationships with Parents • Goal Setting • Thinking Like a Scientist: Overcoming Obstacles • Thinking Like a Scientist: Goal Setting • Record Sheet: Special Connections Read Section on Ideas for Building Positive Relationships Yes No Read section on Bullying Yes No Created plans/goals for Suggested Activities Yes No Wrote future goals for increasing positive relationships for the Yes No week: Completed IY Self-Reflection Forms from Session 1 Yes No Part 2: Proactive Strategies Read and followed directions, and reflected on discussion Yes No questions provided in manual DVD 2: Watched Vignettes (1-57, 7 supplemental vignettes 52 Yes No minutes) 146 Read Chapter 3 from Incredible Teachers book Yes No Completed Brainstorm/Buzz Worksheet activities from Yes No Workbook Part • Rewriting Commands • Classroom Schedule • Classroom Rules • Nonverbal Cues • Environment • Making Learning Fun • Rewrite Commands Completed IY Self-Reflection Forms from session 2 Yes No Created a behavior plan for a student (inattentive, impulsive, Yes No hyperactive or disruptive) using the Preventing Problems- Proactive Teacher Behavior Plan Worksheet by 1) selecting negative behavior, 2) creating a hypothesis for why child behavior was occurring using Functional Assessment worksheet, 3) identifying positive opposite behaviors, and 4) identifying positive relationship strategies/proactive strategies to support student Began behavior plan for 1 student Yes No Created plans/goals for Suggested Activities Yes No Created future goals for increasing proactive strategies based Yes No on self-reflection questionnaire Practiced new strategies throughout the week Yes No Met with Coach Yes No Time to complete self-administered program materials: Number of Tasks Completed (Yes) = ____/20 Session 1 Checklist: Book + Activity comparison group Session #1: Building Relationships with Students and Proactive Teaching Intervention Task Completed? Comments: Part 1: Building Relationships with Students Read Chapters 1, 2, & 14 from Incredible Teachers book Yes No Completed IY Self-Reflection Forms from session 1 Yes No Completed Brainstorm/Buzz Worksheet activities from Yes No Workbook Part: • Promoting a Sense of Responsibility • Changing Negative Reputations • Building Relationships with Students • Building Relationships with Parents • Goal Setting 147 • Thinking Like a Scientist Handout: Overcoming Obstacles • Thinking Like a Scientist Handout: Goal Setting • Record Sheet: Special Connections Read Section on Ideas for Building Positive Relationships Yes No Read section on Bullying Yes No Created plans/goals for Suggested Activities Yes No Wrote future goals for increasing positive relationships for the Yes No week Part 2: Proactive Strategies Read Chapter 3 from Incredible Teachers book Yes No Completed IY Self-Reflection Forms from session 2 Yes No Completed Brainstorm/Buzz Worksheet activities from Yes No Workbook Part • Rewriting Commands • Classroom Schedule • Classroom Rules • Nonverbal Cues • Environment Created a behavior plan for a student (inattentive, impulsive, Yes No hyperactive or disruptive) using the Preventing Problems- Proactive Teacher Behavior Plan Worksheet by 1) selecting negative behavior, 2) creating a hypothesis for why child behavior was occurring using Functional Assessment worksheet, 3) identifying positive opposite behaviors, and 4) identifying positive relationship strategies/proactive strategies to support student Began behavior plan for 1 student Yes No Created plans/goals for Suggested Activities Yes No Created future goals for increasing proactive strategies based Yes No on self-reflection questionnaire Practiced new strategies throughout the week Yes No Time to complete self-administered program materials: Number of Tasks Completed (Yes) = ____/15 148 APPENDIX E Example of Teacher-Coach Meeting form: self-monitoring checklist Coaching self-monitoring checklist Session 1 Example Directions: You will complete this self-monitoring form after each coaching session to measure fidelity and help you keep track of teacher progress. Circle Yes or No after completing your coaching session to indicate whether each coaching component was completed. Coaching Components Items Complete 1. Reviewed self-reflection form on Positive Yes No Relationships with Children Review 2. Provide feedback to teacher from first video Yes No recording observation 3. Reviewed the workshop handouts for Positive Yes No Relationships (i.e., brainstorm activities, Thinking Like a Scientist Handout) 4. Reviewed creating an individual behavioral plan Yes No 5. Helped the teacher pick strategies to promote Yes No positive relationships with their students 6. Set goals with the teacher to build relationships Yes No with the parents of students in their classroom 7. Practiced, rehearsed, reviewed opportunities to Yes No promote students sense of responsibility in the classroom 8. Reviewed self-reflection form on Proactive Yes No Teaching Strategies 9. Reviewed workshop handouts/homework Yes No assignments from Proactive Teaching Strategies Manual (e.g., goals to change a child’s negative reputation in the classroom, non-verbal hand signals, making a special connection, brainstorm/buzz worksheets) 10. Practiced, rehearsed, modeled proactive strategies Yes No (e.g., when-then commands, transition strategies, nonverbal signals) 11. Discussed challenges/successes in implementing Yes No strategies throughout the week 12. Discussed progress on chapter readings, workshop Yes No assignments, and any questions 13. Discussed plans to involve student’s family in Yes No education 149 14. Help the teacher praise themselves for all their Yes No work 15. Create a plan to review further vignettes Yes No 16. Create future goals Yes No Other Coaching Session Information: Average time of session: Most frequently used strategy during the session (e.g., role playing, modeling, praise): Notes: Number of Tasks Completed (Yes) = ____/16 150 APPENDIX F Example of Teacher-Coach Meeting form: Observation checklist Coaching Observation Checklist (Completed by researcher) Session 3: Motivating Children Through Incentives Directions: Research assistants should observe the first enrolled teacher in the session. Circle Yes or No after observing the coaching session and indicate whether each coaching component was completed. Coaching Components Items Complete 1. Reviewed/discussed self-reflection form using Praise and Yes No Incentives 2. Reviewed workshop handouts (e.g., brainstorm handouts, record Yes No sheets) 3. Teacher-Coach reviewed and implementation of behavior plans – Yes No targeting positive opposites to praise and setting up reward systems 4. Reviewed incentives and targeted behaviors to be sure they are Yes No developmentally appropriate (using functional assessment checklist) 5. Reviewed and practiced praise statements with teacher Yes No 6. Reviewed and practiced coaching strategies (emotion, academic, Yes No persistence) based on the development of the child selected for behavior plan. Set goals on coaching strategies. 7. Discussed progress on chapter readings, workshop assignments, Yes No and any questions 8. Discussed plans to involve students’ parents in teacher’s incentive Yes No system 9. Summarized teachers’ strengths ((i.e., reinforce the teacher’s Yes No accomplishments regarding prior goals) and goals 10. Created a plan to review further vignettes Yes No 11. Discussed rewards/pleasurable things the teacher can give Yes No themselves in order to sustain energy from students 12. Reviewed goals for praise and incentive strategies Yes No Other Coaching Session Information: Average time of session: Most frequently used strategy during the session (e.g., role playing, modeling, praise): Number of Tasks Completed (Yes) = ____/12 151 APPENDIX G Teacher Strategies Questionnaire 152 153 APPENDIX H Treatment Evaluation Questionnaire (TEQ) – Teacher Form Treatment Evaluation Questionnaire (TEQ) – Teacher Form Directions: You recently completed an intervention in a research study on treatment approach to child disruptive behaviors. Please evaluate the intervention by circling the number which best describes your agreement or disagreement with each statement regarding the target child you created a behavioral plan for in your class. Please answer each question. Strongly Slightly Slightly Strongly Disagree Agree Disagree Disagree Agree Agree 1. This was an acceptable intervention for 1 2 3 4 5 6 my student’s problem behavior. 2. Most teachers would find this intervention appropriate for behavior 1 2 3 4 5 6 problems. 3. The intervention was effective in changing the student’s problem 1 2 3 4 5 6 behavior. 4. I would suggest the use of this 1 2 3 4 5 6 intervention to other teachers. 5. The behavior problems of the student were severe enough to warrant use of 1 2 3 4 5 6 this intervention. 6. Most teachers would find this intervention suitable for the behavior 1 2 3 4 5 6 problems described. 7. The intervention did not result in 1 2 3 4 5 6 negative side effects for my student. 8. The intervention would be appropriate 1 2 3 4 5 6 for a variety of children. 9. The intervention was a fair way to 1 2 3 4 5 6 handle my student’s problem behavior. 10. I liked the procedure used in the 1 2 3 4 5 6 intervention. 11. The intervention was a good way to handle my student’s behavior 1 2 3 4 5 6 problems. 12. Overall, the intervention was beneficial 1 2 3 4 5 6 for my student. 13. The intervention quickly improved my 1 2 3 4 5 6 student’s behavior. 14. The intervention produced a lasting 1 2 3 4 5 6 improvement in my student’s behavior. 15. The intervention improved my student’s behavior to the point that it 1 2 3 4 5 6 would not noticeably deviate from other well-behaved children’s behavior. 154 16. Soon after starting the intervention, I noticed a positive change in my 1 2 3 4 5 6 student’s problem behavior. 17. I believe my student’s behavior will remain at an improved level even after 1 2 3 4 5 6 the intervention is discontinued. 18. Using the intervention not only improved my student’s behavior in the 1 2 3 4 5 6 classroom, but also in other settings. 19. When comparing the disruptive student in my class compared with a well- behaved peer before and after use of 1 2 3 4 5 6 the intervention, my student and peer’s behavior was more alike after using the intervention. 20. The intervention produced enough improvement in the student’s behavior, 1 2 3 4 5 6 so the behaviors are no longer a problem. 21. Other behaviors related to the problem behavior also were improved by the 1 2 3 4 5 6 intervention. 155 APPENDIX I Teacher Workshop Satisfaction Questionnaire Incredible Years® Teacher Workshop Satisfaction Questionnaire Participant’s Name ________________________________ Date______________________________ The following questionnaire is part of our evaluation of the workshop that you have received. It is important that you answer as honestly as possible. The information obtained will help us to evaluate and continually improve the program we offer. Your cooperation is greatly appreciated. All responses will be strictly confidential. Usefulness In this section, we would like you to indicate how useful you find each of the following types of strategies used in these workshops. Please circle the response that most clearly de- scribes your opinion. 1. Information presented in the self-administered manual was 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 2. Demonstration of skills through the use of video vignettes was 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 3. Suggested classroom activity assignments were 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 4. Book – Incredible Teachers was 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 156 B Specific Teaching Techniques Usefulness In this section, we would like you to indicate how useful each of the following techniques is for teaching students. Please circle the response that most accurately describes the usefulness of the technique. 1. Child-Directed Play 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 2. Academic and Persistence Coaching 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 3. Social and Emotional Coaching 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 4. Praise/Encouragement 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 5. Incentives to motivate children 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 6. Ignoring 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 7. Good Commands/Clear and respective limit setting 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 8. Time Out/Calm Down Place 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 157 9. Loss of Privileges, Logical Consequences 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 10. Redirects/Distraction/Prompting alternative responses 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful 11. Problem-Solving Training 1-Extremely 2-Useless 3-Slightly 4-Neutral 5-Somewhat 6- Useful 7-Very Useful useless useless useful C. Evaluation of Workshop Coach In this section we would like you to express your opinions about your coach. Please circle the response to each question that best describes how you feel. 1. The coach’s preparation was 1-Very poor 2-Poor 3-Below 4-Average 5-Above 6- Superior 7-Excellent Average Average 2. At this point, I feel that the coach was 1-Very poor 2-Poor 3-Below 4-Average 5-Above 6- Superior 7-Excellent Average Average 3. Concerning the coach’s interest and concern in me and my students, it was 1-Very poor 2-Poor 3-Below 4-Average 5-Above 6- Superior 7-Excellent Average Average Thank you! 158 APPENDIX J Example of the Teacher Video Evaluation Questionnaire TEACHER VIDEO EVALUATION QUESTIONNAIRE Video One: Building Relationships, Using Praise Directions: Please evaluate the video examples used in session 1 by circling the number which best describes your agreement or disagreement with each statement 1. The content of these videos were: 1-not helpful 2- somewhat helpful 3-neutral 4-helpful 5-very helpful 2. I feel the video examples were: 1-not helpful 2- somewhat helpful 3-neutral 4-helpful 5-very helpful 3. In terms of changing my own behavior, the techniques presented will be: 1-not helpful 2- somewhat helpful 3-neutral 4-helpful 5-very helpful 4. In terms of changing my students’ behavior, the techniques will be: 1-not helpful 2- somewhat helpful 3-neutral 4-helpful 5-very helpful 159 APPENDIX K Final Teacher Interview Questionnaire (Treatment group only) Effectiveness 1. How has this intervention changed or not changed child behaviors within your classroom? Target student? 2. How has this intervention changed or not changed your own behavior as a teacher? 3. How did this program compare to other strategies you have used to support teacher classroom management practices used in the classroom (e.g., consultation with a school psychologist)? 4. How did you use the evidence-based training strategies within the classroom? 5. What strategies were the most useful to you? 6. How was your experience working with a coach to implement strategies from the program? Acceptability 1. What strategies taught would you be willing to use after this program is complete? 2. Do you think this program is an accessible treatment for teachers struggling with disruptive classroom behavior? 3. Do you think this program could be easily implemented with other teachers? Barriers/Facilitators 1. How did you schedule out your time to read, watch the videos, and practice the homework assignments? 2. What challenges or barriers did you face when trying to complete the assignments (e.g., time, scheduling?) 3. What part of the program was most helpful to you? 4. What did you like most about the program? 5. What did you like least about the program? 6. How could the program have been improved to help you more? 160 APPENDIX L Study Recruitment Flyer 161 APPENDIX M Consent Forms Screening Consent form Script Thank you for visiting the survey link regarding the Incredible Years Teacher Classroom Management training! This survey includes a few questions in order to determine whether you may be eligible for the research. In this study, we are trying to learn if teachers believe an evidence-based self-administered training program designed to address disruptive child behavior problems with the addition of a certified coach can improve teacher skills and is an acceptable program to support disruptive classroom behavior compared to a Book +Activity training group. If you would like to continue the screening, please answer and submit the questions below. The screening will take 5 minutes. The survey includes questions which ask about your teaching background, age, and current use of teacher training strategies. You do not have to answer any questions you do not wish to answer or are uncomfortable answering, and you may stop at any time. Your participation in the screening is voluntary. Your answers will be confidential. No one will know your answers except for the research team. Once you have submitted your survey, you will be contacted by one of our researchers whether you have met the qualifications of our research study. If you do not meet the qualifications of the research study, your information will be destroyed, and you will be provided with resources to support disruptive classroom behavior. Alternately, if you qualify for the research study, you will be provided a consent form on whether you would like to participate. If you provide consent, your survey data will be deidentified and kept in a secure location in our research lab until the completion of the study. Survey Questions Name: Type of School • Private • Public • Charter • Head Start • Other: Gender Identity: Number of Years Teaching • Male • 1-5 years teaching • Female • 6-10 years • 11-20 years • More than 20 years Age: _______ Number of Year Teaching Current Grade • 1-5 years teaching • 6-10 years • 11-20 years • More than 20 years Race/Ethnicity: Current Role 162 • Latino/Hispanic • Lead Elementary School Teacher • Black/African American • Lead Preschool Teacher • White/Caucasian • Lead Daycare Center Teacher • Native American • Paraeducator/Paraprofessional • Asian/Pacific Islander • Teachers’ Assistant/Teacher’s Aide • Two or more races • Before/After School Care Worker • Some other race, ethnicity • College Intern/Practicum Student (Early education or elementary or origin ed) • Other Level of Education: Have you received the Incredible Years Teacher Classroom Management • Highschool or GED Training before? • Some College • Yes • Associate degree • No • College Graduate • Post-College Degree Grade Level of Students Do you work in the school at least 3 days per week? • Preschool (3-5 years) • Yes • Developmental • No Kindergarten • Kindergarten • First grade • Second grade • Third grade 163 Thank you for answering the screening questions. You will be contacted as soon as possible about your eligibility in the screening process. If you have concerns or questions about this study, such as scientific issues, how to do any part of it, or to report an injury, please contact the Rachel Korest through email (korestra@msu.edu) or by phone (517-898-3937). You may also contact Dr. John Carlson through email (carlsoj@msu.edu) or by phone (517-432-4856). If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at 4000 Collins Rd, Suite 136, Lansing, MI 48910. Thank you again for your willingness to answer our questions. 164 Follow up Email Script if Participants are Eligible/Ineligible for Research Study Eligible Research Study Script Thank you for participating in the screening process for the study examining the Incredible Years Teacher Training Program! You have been selected as a participant for research study. If you are interested in being a part of the research study, please click on the survey link to read the consent document which includes an overview and description of your role in the research study. Ineligible for Research Study Script delivered over email: Thank you for participating in the screening process for the study examining the Incredible Years Teacher Training program. Unfortunately, you will not eligible to be a participant for the research study as you did not meet the inclusion criteria required. To ensure privacy and confidentiality, your screening data will be destroyed. For resources to support difficulties with disruptive child behaviors, please examine the following sites: • National Center on Intensive Interventions: https://intensiveintervention.org/ Provide free resources and print outs on behavioral strategies and step by step instructions how to correct student misbehavior and promote positive behavior learning. • Intervention Central: https://www.interventioncentral.org/home Includes free resources such as videos, handouts, and intervention, tools, blogs and suggestions for both academic and behavioral problems for students of all ages. • EBI Missouri: http://ebi.missouri.edu/?page_id=402 Includes evidence-based academic and behavioral interventions, directions, videos, and handouts related to the type of behavior the student is displaying • PBIS for the Classroom: https://www.pbis.org/resource-type/materials Includes presentations and resources on how to support classroom management 165 Minimal Risk Consent Form BRIEF SUMMARY You are being asked to participate in a research study for a dissertation that will investigate a teacher classroom management training program with a coaching component. If you volunteer for this study, you will be randomly assigned to one of two possible groups. Your participation in this study will take approximately 1.5 hours per week if assigned to the Book + Activity comparison group and 3 hours per week if assigned to the treatment group. Each group will complete 6 sessions over the course of 12 weeks (i.e., two weeks per session). Teachers will spend approximately two weeks with each session materials. Teachers in the treatment group will exchange these materials for new materials every two weeks. If assigned to the treatment group, you will complete the Incredible Years Self-Administered Teacher training program (a program which includes a set of 7 DVDs to model skills, a set of 7 self-administered manuals that address different teacher classroom management topics and homework activities, and the Incredible Teachers book) and meet with a coach biweekly for 30 minutes. The coach is a certified Incredible Years group leader through Michigan State University, has delivered several Incredible Years group training sessions, and has over 35 years of experience in the field of early education. If assigned to the Book + Activity comparison group, you will have assigned reading material from the Incredible Years series and assigned activities to complete from the self-administered manual. If assigned to either group, you will be asked to complete a minimum of three self-report surveys; one survey will be completed before and after treatment and the other two surveys will be completed after each session (6 total checklists for each survey that align with each treatment session). You will also be asked to be observed in your classroom at pre and post treatment to examine teaching strategies for a duration of 20 minutes total for each observation. If assigned to the treatment group, you will be asked to complete an additional 4-item questionnaire to be completed after each session (7 total questionnaires for the 7 DVDs), two additional short surveys at post treatment, and a 20-minute interview over Zoom with a research assistant. There are no foreseeable risks to the intervention groups. Benefits include an opportunity to receive training from an evidence-based program. Teachers will be not be required to pay for any training materials and will be compensated for their time with a $50 gift card upon completion of the study. PURPOSE OF RESEARCH The purpose of this study is to learn if teachers believe a self-administered training program with the addition of a certified coach can be implemented with fidelity (accuracy), is effective at improving teachers’ perceptions of evidence-based teaching classroom management strategies, and increases frequency of use of teacher skills. For teachers assigned to the treatment group, a final interview will be used to examine whether teachers find the program contents acceptable and if any barriers exist when trying to implement the program. WHAT YOU WILL BE ASKED TO DO 166 If you volunteer for this study, you will be randomly assigned to one of two possible groups: A Treatment Group or a Book + Activity comparison group. Each group will last a duration of 12 weeks. Both groups will include six treatment sessions that will each last approximately two weeks each. Treatment Group: Upon consent of the study, participants in the treatment group will be assigned to a D2L page that will include directions for each session, links to surveys, and places to upload completed activities. Those assigned to the treatment group will receive the self- administered Incredible Years Teacher Classroom Management program. This program is a six- session program that will be completed over a 12-week time period (i.e., two weeks for each session). Materials within this study will include 7 DVDs to instruct and model skills, 7 self- administered manuals that include activities and guiding questions to support critical thinking, assigned activities to complete throughout the week (i.e., new skills learned from each session, developing a behavioral intervention plan for a challenging student), reading chapters from the Incredible Teachers book that align with self-administered manuals and DVDs, and a coach to support skill development, answer questions, provide feedback, model skills from the IY program, and support your creation of a behavioral intervention plan with a challenging student from your classroom. Upon consent, teachers will be provided with the first set of DVDs (DVD 1 & 2), the first treatment manual for session 1 (Manuals 1 & 2), and the Incredible Teachers book. For both the DVDs and the treatment manual, teachers will be given approximately two weeks to complete and review materials. After two weeks, a research assistant will swap the previous session materials for a new set of materials. Teachers will meet with their assigned coach biweekly (or once per treatment session) for a duration of 30 minutes. You will also be allowed to reach out to the coach throughout the week for additional questions. Once assigned to a group, a research assistant will reach out over email to choose a time for the researcher to deliver materials and have the first meeting with the IY Coach. Book + Activity comparison group Upon consent, teachers in the Book + Activity comparison group will be assigned to a D2L page where they will find instructions for each session, links to surveys, and places to upload completed activities. The Book + Activity comparison group will not receive the manuals or coaching support but will be provided with reading material from the Incredible Years program on classroom management strategies from the Incredible Teachers: Nurturing Children’s Social Emotional Competence, receive a reading schedule that corresponds with the reading schedule of the Intervention group, and receive activities to complete from the self-administered manual. Teachers will not be assigned a new set of materials each week and instead the D2L page will unlock a new schedule for reading chapters and assigned activities once every two weeks. Data Collection Procedures: Your involvement in this study will also include completing data collection procedures. All participants will complete a rating scale and have 20-minute observations conducted in the classroom before the intervention begins and after the intervention is completed. During the observation, research assistants will email participants for the best time to come observe teachers for a 20-minute class period during the school week. Additionally, research assistants may video record participants within the classroom setting to help aid in data 167 collection accuracy. Participants will also complete an 8-12 item fidelity checklist after each session in order to receive credit for gift cards. These survey links will be provided within the assigned D2L page. If assigned to the treatment group, you will complete one additional checklist on the program videos after each session, two additional self-report measures at the end of treatment, and one final 20-minute interview to understand your thoughts on the treatment program. No information will be collected directly from the students within the classroom. Participation in this research study is completely voluntary. You may choose not to participate at all, or you may refuse to answer certain questions or discontinue your participation at any time. HOW CLASSROOM INFORMATION WILL BE KEPT CONFIDENTIAL Each teacher will be assigned an ID number which will be used in place of names in order to maintain confidentiality. The only time that names will be directly tied to the data will be during the pre-test screening phase to know if teachers meet the inclusion criteria for the study. All rating forms and data will be kept in a password protected file on the computer and only the researchers will have access to this password protected document. Individual names and identifying information will not be used in any research reports. To protect confidentiality during the video chat session with the coach or PI during the final interview, Zoom version 4.6.2 [computer software], a HIPPA compliant, secure website that protects against third party software will be used during video chat sessions. The Zoom session will be password protected and participants will be required to show identification before starting Zoom conversations to ensure confidentiality. For observations that use a recording device, observers will upload files and create password protected files. Immediately after coding the data, video recordings and identifying information will be deleted from computer files. BENEFITS THAT MAY OCCUR IF YOU CHOOSE TO PARTICIPATE IN THE STUDY Teachers included within the two intervention groups will receive the benefit of receiving an evidence-based intervention to improve classroom management strategies which may lead to potential improvements in positive classroom atmosphere, teacher-student relationships, peer relationships between students, positive relationships with children’s parents, and improved child behavior COMPENSATION All materials will be provided to the teacher free of charge. All teachers that participate in this study will be provided with $50 gift cards which will be delivered at the end of their participation in the data collection procedures and research study. Gift cards will be provided only if teachers successfully complete their fidelity checklist to the D2L page. QUESTIONS OR CONCERNS ABOUT THIS STUDY If you have any concerns or questions regarding this study, you may contact the researcher Rachel Korest through email (korestra@msu.edu) or by phone (517-898-3937). You may also contact Dr. John Carlson through email (carlsoj@msu.edu) or by phone (517-432-4856). If you have any questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University Human Research Protection 168 Program by phone at 517-355-2180, email irb@msu.edu, or by regular mail 408 West Circle Drive Room 207 Olds Hall, MSU, East Lansing, MI 48824. YOUR RIGHTS TO PARTICIPATE, SAY NO, OR WITHDRAW Your participation in this research is voluntary and you will not be penalized or lose benefits if you refuse to participate or decide to stop. However, if you choose to end your participation you may not be eligible for receiving money for the gift cards. Please read and complete the consent form in the next section in order to provide your consent for participation. Informed Consent Form If provided a copy of this consent form through mail, please mail or email and scan this form back to the researcher Rachel Korest (email: korestra@msu.edu, regular mail: Erickson Hall Building, 620 Farm Lane Room 435, East Lansing, MI 48824). Your signature on this form indicates that you consent to participate in this research study, understand the chance of being assigned to any of the two groups, and agree to participate in data collection procedures. Teachers Information: Teacher Name: (Please Print):_____________________ Contact Information: _________________________________________________________________ School Mailing Address ________________________________________________________________ Home Phone Cell Phone or Work Phone ________________________________________________________________ Email Address (primary mode of communication used for this study) *I agree to allow audiotaping/videotaping of the interview. Yes No Initials____________ *Videos will be kept in a password protected confidential file and will be deleted upon completion of study. Digital Signature of Teacher: ______________________ Date: _________________ 169 REFERENCES 170 REFERENCES Ainsworth, M. D. S., Blehar, M. C., Waters, E., & Wall, S. (1978). Patterns of attachment: A psychological study of the strange situation. Lawrence Erlbaum. Allday, Hinkson-Lee, Hudson, Neilsen-Gatti, Kleinke, & Russel (2012). Training general educators to increase behavior-specific praise: Effects on students with EBD. Behavioral Disorders, 37, 87-98. Baker-Henningham, H., Scott, S., Jones, K., & Walker, S. (2012). Reducing child conduct problems and promoting social skills in a middle-income country: Cluster randomised controlled trial. The British Journal of Psychiatry, 201, 101-108. doi: 10.1192/bjp.bp.111.096834. Baker-Henningham, H., & Walker, S. (2018). Effect of transporting an evidence-based, violence prevention intervention to Jamaican preschools on teacher and class-wide child behaviour: a cluster randomised trial. Global Mental Health: https://doi.org/10.1017/gmh.2017.29 Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychological Review, 84, 191–215. https://doi.org/10.1037/0033-295X.84.2.191 Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1-26. https://doi.org/10.1146/annurev.psych.52.1.1 Bierman, K. L., Domitrovich, C. E., Nix, R. L., Gest, S. D., Welsh, J. A., Greenberg, M. T., ... & Gill, S. (2008). Promoting academic and social‐emotional school readiness: The Head Start REDI program. Child Development, 79, 1802-1817. https://doi.org/10.1111/j.1467- 8624.2008.01227.x Bierman, K. L., & Motamedi, M. (2015). Social and emotional learning programs for preschool children. In J. Durlak, C. Domitrovich, R.P. Weissburg, and T. Gullotta (Eds.) The Handbook of Social and Emotional Learning: Research and Practice (pp.135-150). New York: Guilford. Blueprints for Healthy Youth Development (2019). Blueprints for healthy youth development. Retrieved from www.blueprintprograms.com Bowen, D. J., Kreuter, M., Spring, B., Cofta-Woerpel, L., Linnan, L., Weiner, D., ... & Fernandez, M. (2009). How we design feasibility studies. American Journal of Preventive Medicine, 36, 452-457. https://doi.org/10.1016/j.amepre.2009.02.002 Bowlby, J. (1982). Attachment and loss: Retrospect and prospect. The American Journal of Orthopsychiatry, 52, 664-678. https://doi.org/10.1111/j.1939-0025.1982.tb01456.x 171 Buhi, E. R., Goodson, P., & Neilands, T. B. (2007). Structural equation modeling: A primer for health behavior researchers. American Journal of Health Behavior, 31(1), 74- 85. https://doi.org/10.5993/AJHB.31.1.8 Carlson, J. S., Tiret, H. B., Bender, S. L., & Benson, L. (2011). The influence of group training in the Incredible Years Teacher Classroom Management Program on preschool teachers' classroom management strategies. Journal of Applied School Psychology, 27, 134-154. Carlson, K. D., & Schmidt, F. L. (1999). Impact of experimental design on effect size: Findings from the research literature on training. Journal of applied psychology, 84, 851-862. Center for the Study and Prevention of Violence (2018). Blueprints for healthy youth development. Retrieved from http://www.blueprintsprograms.com/ Cohen, J. (1988). Statistical power analysis for the behavioral sciences. 2nd Edition Hillsdale, NJ: Lawrence Erlbaum. Cohn, D. A. (1990). Child-mother attachment of six-year-olds and social competence at school. Child Development, 61(1), 152-162. https://doi.org/10.1111/j.1467- 8624.1990.tb02768.x Cook, C. R., Lyon, A. R., Kubergovic, D., Wright, D. B., & Zhang, Y. (2015). A supportive beliefs intervention to facilitate the implementation of evidence-based practices within a multi-tiered system of supports. School Mental Health, 7(1), 49-60. https:// doi/10.1007/s12310-014-9139-3 Copeland-Mitchell, J., Denham, S. A., & DeMulder, E. K. (1997). Q-sort assessment of child- teacher attachment relationships and social competence in the preschool. Early Education and Development, 8, 27–39. https://doi.org/10.1207/s15566935eed0801_3 Council for the Accreditation of Educator Preparation. (2018). CAEP 2018 K-6 elementary teacher preparation standards [initial licensure programs]. Retrieved from http://caepnet.org/standards/introduction Cowan, R. J., & Sheridan, S. M. (2003). Investigating the acceptability of behavioral interventions in applied conjoint behavioral consultation: Moving from analog conditions to naturalistic settings. School Psychology Quarterly, 18(1), 1– 21. https://doi.org/10.1521/scpq.18.1.1.20877 Darling-Hammond, L., Hyler, M. E., Gardner, M. (2017). Effective Teacher Professional Development. Palo Alto, CA: Learning Policy Institute. Desimone, L. M., & Garet, M. S. (2015). Best practices in teachers’ professional development in the United States. Psychology, Society and Education, 7, 252-263. 172 Dicke, T., Elling, J., Schmeck, A., & Leutner, D. (2015). Reducing reality shock: The effects of classroom management skills training on beginning teachers. Teaching and Teacher Education, 48, 1-12. https://doi.org/10.1016/j.tate.2015.01.013 Doll, C., McLaughlin, T. F., & Barretto, A. (2013). The token economy: A recent review and evaluation. International Journal of Basic and Applied Science, 2(1), 131-149. Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. The Journal of Primary Prevention, 28, 67-91. https://doi.org/10.1007/s10935-007-0081-0 Driscoll, K.C., Wang, L., Mashburn, A.J., & Pianta, R.C. (2011). Fostering supportive teacher– child relationships: Intervention implementation in a state-funded preschool program, Early Education and Development, 22, 593- 619, doi: 10.1080/10409289.2010.502015 Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school- based universal interventions. Child Development, 82(1), 405-432. doi: 10.1111/j.1467- 8624.2010.01564.x. Eckert, T. L., & Hintze, J. M. (2000). Behavioral conceptions and applications of acceptability: Issues related to service delivery and research methodology. School Psychology Quarterly, 15, 123. https://doi/10.1037/h0088782 Egert, F., Fukkink, R. G., & Eckhardt, A. G. (2018). Impact of in-service professional development programs for early childhood teachers on quality ratings and child outcomes: A meta-analysis. Review of Educational Research, 88, 401-433. https://doi.org/10.3102/0034654317751918 Eldridge, S. M., Lancaster, G. A., Campbell, M. J., Thabane, L., Hopewell, S., Coleman, C. L., & Bond, C. M. (2016). Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PloS one, 11, doi:10.13710150205 Elgar, F. J., & McGrath, P. J. (2003). Self‐administered psychosocial treatments for children and families. Journal of Clinical Psychology, 59, 321-339. https://doi.org/10.1002/jclp.10132 Erchul, W.P., & Martens, B.K. (2010). School Consultation: Conceptual and Empirical Bases of Practice. Springer Science and Business Media. 173 Evertson, C. M. & Weinstein, C. S. (Eds.) (2006). Handbook of Classroom Management: Research, Practice, and Contemporary Issues. Mahwah, NJ: Larence Erlbaum Associates, Inc. Every Student Succeeds Act, Pub. L. No. 114-195, 20 U.S.C. 1 6301 et seq. (2015). Fearon, R. P., Bakermans-Kranenburg, M. J., Van IJzendoorn, M. H., Lapsley, A. M., & Roisman, G. I. (2010). The significance of insecure attachment and disorganization in the development of children’s externalizing behavior: A meta-analytic study. Child Development, 81, 435-456. https://doi.org/10.1111/j.1467-8624.2009.01405.x Ferguson, C. J. (2016). An effect size primer: A guide for clinicians and researchers. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research (p. 301–310). American Psychological Association. https://doi.org/10.1037/14805-020 Fergusson, D. M., Horwood, L. J., & Stanley, L. (2013). A preliminary evaluation of the Incredible Years teacher programme. New Zealand Journal of Psychology, 42, 51-56. Flower, A., McKenna, J. W., & Haring, C. D. (2017). Behavior and classroom management: Are teacher preparation programs really preparing our teachers? Preventing School Failure: Alternative Education for Children and Youth, 61, 163-169. https://doi.org/10.1080/1045988X.2016.1231109 Food and Drug Administration (FDA). (2002). Guidance for industry: Food-effect bioavailability and fed bioequivalence studies. Retrieved http://www.fda.gov/cder/guidance/5194fnl.pdf Ford, T., Hayes, R., Byford, S., Edwards, V., Fletcher, M., Logan, S., ... & Ganguli, P. (2019). The effectiveness and cost-effectiveness of the Incredible Years Teacher Classroom Management programme in primary school children: results of the STARS cluster randomised controlled trial. Psychological Medicine, 49, 828-842. https://doi.org/10.1017/S0033291718001484 Forehand, R. L., & McMahon, R. J. (1981). Helping the Non-Compliant Child: A Clinician's Guide to Parent Training. New York: Guilford Press. Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy, L. A., Rosenfield, S. A ., . . . Stoiber, K. C. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77-100. doi: 10.1037/spq0000019 Garcia, E., & Weiss, E. (2019). The role of early career supports, continuous professional development, and learning communities in the teacher shortage. Economic Policy Institute. Retrieved from: https://www.epi.org/publication/teacher-shortage-professional- development-and-learning-communities/ 174 Gliner, J. A., Morgan, G. A., & Harmon, R. J. (2003). Pretest-posttest comparison group designs: Analysis and interpretation. Journal of the American Academy of Child & Adolescent Psychiatry, 42, 500-503. https://doi.org/10.1097/01.CHI.0000046809.95464.BE Hartmann, D. P. (1977). Considerations in the choice of interobserver reliability measures. Journal of Applied Behavior Analysis, 10, 103–116. Harwood, T. M., & L’Abate, L. (2010). Distance writing: Helping without seeing participants. In Self-Help in Mental Health (pp. 47-58). Springer, New York, NY. Hedges, L.V. and I. Olkin (1985), Statistical Methods for Meta-Analysis. London: Academic Press. Hickey, G., McGilloway, S., Hyland, L., Leckey, Y., Kelly, P., Bywater, T., … O’Neill, D. (2017). Exploring the effects of a universal classroom management training programme on teacher and child behaviour: A group randomised controlled trial and cost analysis. Journal of Early Childhood Research, 15, 174-194. https://doi.org/10.1177%2F1476718X15579747 Hicks-Hoste, T. B., Carlson, J. S., & Tiret, H. B. (2015). A pilot study exploring after-school care providers’ response to the Incredible Years Classroom Management program. Contemporary School Psychology, 19, 173-183. doi:10.1007/s40688-014-0036- 4. Hoyt, W.T., & Del Re, A. C. (2018). Effect size calculation in meta-analyses of psychotherapy outcome research. Psychotherapy Research, 28, 379- 388, doi: 10.1080/10503307.2017.1405171 Hutchings, J., Daley, D., Jones, K., Martin, P., Bywater, T., & Gwyn, R. (2007). Early results from developing and researching the Webster-Stratton Incredible Years Teacher Classroom Management Training Programme in North West Wales. Journal of Children's Services, 2, 15-26. https://doi.org/10.1108/17466660200700023 Hutchings, J., Martin-Forbes, P., Daley, D., & Williams, M.E. (2013). A randomized controlled trial of the impact of a teacher classroom management program on the classroom behavior of children with and without behavior problems. Journal of School Psychology, 51, 571-585. https://doi.org/10.1016/j.jsp.2013.08.001 Ingersoll, R. M. (2005). The problem of underqualified teachers: A sociological perspective. Sociology of Education, 78, 175-178. https://doi.org/10.1177/003804070507800206. Ingersoll, R. M., & Smith, T. M. (2003). The wrong solution to the teacher shortage. Educational Leadership, 60, 30-33. Retrieved from https://repository.upenn.edu/gse_pubs/126 Jazaar, M., Lambert, R. G., & O'Donnell, M. (2007). An investigation of elementary teacher 175 stress to guide educational administrators in curbing the early career departure of elementary school teachers. In L. K. Lemasters, & R. Papa (Eds.), At the Tipping Point: Navigating The Course for the Preparation of Educational Administrators (pp. 59 -72). Lancaster, PA: DEStech Publications, Inc. Jones, V., & Jones, L. (2015). Comprehensive Classroom Management: Creating Communities of Support and Solving Problems. Boston, MA: Pearson Education. Julious, S. A. (2004). Sample sizes for clinical trials with normal data. Statistics in Medicine, 23, 1921-1986. https://doi.org/10.1002/sim.1783 Kazdin, A. E. (1980). Acceptability of alternative treatments for deviant child behavior. Journal of Applied Behavior Analysis, 13, 259-273. https://doi.org/10.1901/jaba.1980.13-259 Kazdin. A.E. (2017). Research design in clinical psychology (5th ed.). Boston, MA: Pearson. Kelley, M. L., Heffer, R. W., Gresham, F. M., & Elliott, S. N. (1989). Development of a modified treatment evaluation inventory. Journal of Psychopathology and Behavioral Assessment, 11, 235-247. doi:10.1007/bf00960495 Kendall, P. C., & Frank, H. E. (2018). Implementing evidence-based treatment protocols: Flexibility within fidelity. Clinical psychology: A publication of the Division of Clinical Psychology of the American Psychological Association, 25, e12271. https://doi.org/10.1111/cpsp.12271 Korest, R., & Carlson, J.S., (2020). A meta-analysis on the current state of evidence of the Incredible Years Teacher Classroom Management Program. An unpublished manuscript. Korpershoek, H., Harms, T., de Boer, H., van Kuijk, M., & Doolaard, S. (2016). A meta-analysis of the effects of classroom management strategies and classroom management programs on students’ academic, behavioral, emotional, and motivational outcomes. Review of Educational Research, 86, 643-680. https://doi.org/10.3102/0034654315626799 Kraft, M. A., Blazar, D., & Hogan, D. (2018). The Effect of Teacher Coaching on Instruction and Achievement: A Meta-Analysis of the Causal Evidence. Review of Educational Research, 88, 547–588. https://doi.org/10.3102/0034654318759268 Kratochwill, T. R., Elliott, S. N., Loitz, P. A., Sladeczek, I., & Carlson, J. S. (2003). Conjoint consultation using self-administered manual and videotape parent-teacher training: Effects on children’s behavioral difficulties. School Psychology Quarterly, 18, 269–302. https://psycnet.apa.org/doi/10.1521/scpq.18.3.269.22574 Kratochwill, T. R., & Shernoff, E. S. (2004). Evidence-based practice: Promoting evidence- based interventions in school psychology. School Psychology Review, 33, 34-48. Lakens D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a 176 practical primer for t-tests and ANOVAs. Frontiers in psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863 Lambert, R. G., McCarthy, C, O'Donnell, M., & Wang, C (2009). Measuring elementary teacher stress and coping in the classroom: Validity evidence for the classroom appraisal of resources and demands. Psychology in the Schools, 46, 973-988. https://doi.org/10.1002/pits.20438 Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004). Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10, 307- 312. https://doi.org/10.1111/j..2002.384.doc.x Louws, M. L., Meirink, J. A., van Veen, K., & van Driel, J. H. (2017). Teachers' self-directed learning and teaching experience: What, how, and why teachers want to learn. Teaching and teacher education, 66, 171-183. https://doi.org/10.1016/j.tate.2017.04.004 Mahoney, J. L., Levine, M. D., & Hinga, B. (2010). The development of after-school program educators through university-community partnerships. Applied Developmental Science, 14, 89-105. Marchette, L. K., & Weisz, J. R. (2017). Practitioner Review: Empirical evolution of youth psychotherapy toward transdiagnostic approaches. Journal of Child Psychology and Psychiatry, 58, 970-984. https://doi.org/10.1111/jcpp.12747 Martens, B. K., Witt, J. C., Elliott, S. N., & Darveaux, D. X. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191. Marzano, R. J., Marzano, J. S., & Pickering, D. J. (2003). Classroom management that works. Alexandra, VA: ASCD. Retrieved from: http://www.ascd.org/publications/books/103027.aspx Matherson, L., & Windle, T. M. (2017). What do teachers want from their professional development? Four emerging themes. Delta Kappa Gamma Bulletin, 83, 28. https://doi.org/10.1080/0305764X.2013.860083 Maxwell, K. L., Lim, C-I., & Early, D. M. (2006). Early childhood teacher preparation programs in the United States: National report. Chapel Hill: University of North Carolina, FPG Child Development Institute. May, H., Sirinides, P., Gray, A., & Goldsworthy, H. (2016). Reading Recovery: An Evaluation of the Four-Year i3 Scale-Up. Philadelphia, PA: Consortium for Policy Research in Education. http://repository.upenn.edu/ cgi/viewcontent.cgi?article=1089&context=cpre_researchreports (accessed 11/17/2016). McCann, T. M., & Johannessen, L. R. (2004). Why do new teachers cry? The Clearing 177 House, 77, 138–146. https://doi.org/10.3200/TCHS.77.4.138-145. McGilloway, S., Hyland, L., Mháille, G. N., Lodge, A., O’Neill, D., Kelly, P., ... & Donnelly, M. (2011). Positive classrooms, Positive children. Dublin: Archways. McGoey, K. E., & DuPaul, G. J. (2000). Token reinforcement and response cost procedures: Reducing the disruptive behavior of preschool children with attention- deficit/hyperactivity disorder. School Psychology Quarterly, 15, 330. https://psycnet.apa.org/doi/10.1037/h0088790 Menting, A. T., de Castro, B. O., & Matthys, W. (2013). Effectiveness of the Incredible Years parent training to modify disruptive and prosocial child behavior: A meta-analytic review. Clinical Psychology Review, 33, 901-913. https://doi.org/10.1016/j.cpr.2013.07.006 Mihalic, S. F., & Elliott, D. S. (2015). Evidence-based programs registry: Blueprints for healthy youth development. Evaluation and Program Planning, 48, 124-131. https://doi.org/10.1016/j.evalprogplan.2014.08.004 Murray, D. W., Rabiner, D. L., & Carrig, M. M. (2014). Grade level effects of the Incredible Years Teacher Training Program on emotion regulation and attention. Paper presented at the Society for Research on Education Effectiveness, Washington D.C. Murray, D. W., Rabiner, D. L., Kuhn, L., Pan, Y., & Sabet, R. F. (2018). Investigating teacher and student effects of the Incredible Years Classroom Management Program in early elementary school. Journal of school psychology, 67, 119-133. https://doi.org/10.1016/j.jsp.2017.10.004 Nye, E., Melendez‐Torres, G. J., & Gardner, F. (2018). Mixed methods systematic review on effectiveness and experiences of the Incredible Years Teacher Classroom Management programme. Review of Education, 7, 631-669. https://doi.org/10.1002/rev3.3145 O'Neill, S. C., & Stephenson, J. (2014). Evidence-based classroom and behaviour management content in Australian pre-service primary teachers' coursework: Wherefore art thou?. Australian Journal of Teacher Education, 39, 1-22. doi: 10.14221/ajte.2014v39n4.4 Odgers, C. L., Moffitt, T. E., Broadbent, J. M., Dickson, N., Hancox, R. J., Harrington, H., ...Caspi, A. (2008). Female and male antisocial trajectories: From childhood origins to adult outcomes. Development and Psychopathology, 20, 673-716. https://doi.org/10.1017/S0954579408000333 Ogg, J. A., & Carlson, J. S. (2009). The self-administered Incredible Years Parent Training Program: Perceived effectiveness, acceptability, and integrity with children exhibiting symptoms of attention-deficit/hyperactivity disorder. Journal of Evidence-Based 178 Practices for Schools, 10, 143–166. Retrieved from: https://psycnet.apa.org/record/2009- 23638-003 Osburn, J. L. (2013). An investigation of the effectiveness and transportability of the incredible years self-administered parent training program with an at-risk head start sample (Unpublished Dissertation). Available from Dissertations & Theses @ CIC Institutions; ProQuest Dissertations & Theses Global. (1438884831). Retrieved from http://ezproxy.msu.edu. Patterson G.R. (1982). Coercive Family Process. Eugene, OR: Castalia. Pianta, R. C., Hamre, B. K., & Allen, J. P. (2012). Teacher-student relationships and engagement: Conceptualizing, measuring, and improving the capacity of classroom interactions. In Handbook of research on student engagement (pp. 365-386). Springer, Boston, MA. Powell, D. R., Diamond, K. E., Burchinal, M. R., & Koehler, M. J. (2010). Effects of an early literacy professional development intervention on Head Start teachers and children. Journal of Educational Psychology, 102, 299–312. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., ... & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65-76. doi: 10.1007/s10488-010-0319-7 Qualtrics XM [survey tool]. Provo, Utah, USA. https://www.qualtrics.com/ Ratcliff, N. J., Jones, C. R., Vaden, S. R., Sheen, H., & Hunt, G. H. (2011). Paraprofessionals in early childhood classrooms: An examination of duties and expectations. Early Years, 31, 163-179. https://doi.org/10.1080/09575146.2011.576333. Raver, C. C., Jones, S. M., Li-Grining, C. P., Metzger, M., Smallwood, K., & Sardin, L. (2008). Improving preschool classroom processes: Preliminary findings from a randomized trial implemented in Head Start settings. Early Childhood Research Quarterly, 23, 10-26. https://dx.doi.org/10.1016j.ecresq.2007.09.001 Reinke, W. M., & Herman, K. C. (2002). Creating school environments that deter antisocial behaviors in youth. Psychology in the Schools, 39, 549-560. doi:10.1002/pits.10048 Reinke, W. M., Stormont, M., Herman, K. C., & Newcomer, L. (2014). Using coaching to support teacher implementation of classroom-based interventions. Journal of Behavioral Education, 23(1), 150–167. https://doi.org/10.1007/s10864-013-9186-0 Reinke, W. M., Stormont, M., Herman, K. C., Wachsmuth, S., & Newcomer, L. (2015). The 179 Brief Classroom Interaction Observation–Revised: An observation system to inform and increase teacher use of universal classroom management practices. Journal of Positive Behavior Interventions, 17, 159-169. doi: 10.1177/1098300715570640 Reinke, W. M., Stormont, M., Herman, K. C., Wang, Z., Newcomer, L., & King, K. (2014). Use of coaching and behavior support planning for students with disruptive behavior within a universal classroom management program. Journal of Emotional and Behavioral Disorders, 22, 74-82. https://doi.org/10.1177/1063426613519820 Reinke, W.M., Stormont, M., Webster-Stratton, C., Newcomer, L., & Herman, K.C. (2012). The Incredible Years teacher classroom management program: Using coaching to support generalization to real‐world classroom settings. Psychology in the Schools, 49, 416-428. DOI: 10.1002/pits.21608. Robinson, E. A., &Eyberg, S. M. (1981).The dyadic parent-child interaction coding system: Standardization and validation. Journal of Consulting and Clinical Psychology, 49, 245-250. Rosseel Y. (2012). Lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48, 1–36. http://www.jstatsoft.org/v48/i02/. Rotheram-Borus, M. J., Swendeman, D., & Chorpita, B. F. (2012). Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist, 67, 463476. doi: 10.1037/a002818 Schonert-Reichl, K. (2017). Social and Emotional Learning and Teachers. The Future of Children, 27(1), 137-155. Retrieved February 14, 2021, from http://www.jstor.org/stable/44219025 Schoonenboom, J., & Johnson, R. B. (2017). How to Construct a Mixed Methods Research Design. Kolner Zeitschrift fur Soziologie und Sozialpsychologie, 69, 107–131. https://doi.org/10.1007/s11577-017-0454-1 Sheridan, S. M. (2014). Single-case designs and large-N studies: The best of both worlds. In T. R. Kratochwill & J. R. Levin (Eds.), Single-Case Intervention Research: Methodological and Statistical Advances (pp. 299-308). Washington, DC: American Psychological Association. Shernoff, E. S., & Kratochwill, T. R. (2007). Transporting an evidence-based classroom management program for preschoolers with disruptive behavior problems to a school: An analysis of implementation, outcomes, and contextual variables. School Psychology Quarterly, 22, 449–472. https://doi.org/10.1037/1045-3830.22.3.449 Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based practices in classroom management: Considerations for research to practice. Education and Treatment of Children, 31, 351-380. Retrieved from www.jstor.org/stable/42899983 180 Smith, T. M., & Ingersoll, R. M. (2004). What are the effects of induction and mentoring on beginning teacher turnover?. American Educational Research Journal, 41, 681-714. https://doi.org/10.3102/00028312041003681 Smith, L. B., & Jones, S. S. (2011). Symbolic play connects to language through visual object recognition. Developmental Science, 14, 1142-1149. https://doi.org/10.1111/j.1467- 7687.2011.01065.x Sosinsky, L. S., & Gilliam, W. S. (2011). Assistant teachers in prekindergarten programs: What roles do lead teachers feel assistants play in classroom management and teaching?. Early Education & Development, 22, 676-706. https://doi.org/10.1080/10409289.2010.497432 Sprick R. (2009). CHAMPS: A Proactive & Positive Approach to Classroom Management. Eugene, OR: Pacific Northwest Publishing. State, T. M., Harrison, J. R., Kern, L., & Lewis, T. J. (2017). Feasibility and acceptability of classroom-based interventions for students with emotional/behavioral challenges at the high school level. Journal of Positive Behavior Interventions, 19(1), 26-36. https://psycnet.apa.org/doi/10.1177/1098300716648459 Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research & Evaluation, 9. http://pareonline.net/getvn.asp?v=9&n=4 Stewart, C. (2014). Transforming professional development to professional learning. Journal of Adult Education, 43, 28-33. Stewart, L. S., & Carlson, J. S. (2010). Investigating parental acceptability of the Incredible Years self-administered parent training program for children presenting externalizing behavior problems. Journal of Applied School Psychology, 26, 162-175. https://doi.org/10.1080/15377901003709369 Stormont, M., Reinke, W. M., Newcomer, L., Marchese, D., & Lewis, C. (2015). Coaching teachers’ use of social behavior interventions to improve children’s outcomes: A review of the literature. Journal of Positive Behavior Interventions, 17, 69-82. https://doi.org/10.1177/1098300714550657 Streiner, D. L. (2005). Finding our way: an introduction to path analysis. The Canadian Journal of Psychiatry, 50, 115-122. https://doi.org/10.1177/070674370505000207 Sullivan, G.M., & Feinn, R (2012. Using effect size—or why the p value is not enough. Journal of Graduate Medical Education, 4, 279-282. https://doi.org/10.4300/JGME-D-1200156.1 Taylor, T. K., Webster‐Stratton, C., Feil, E. G., Broadbent, B., Widdop, C. S., & Severson, H. H. 181 (2008). Computer‐based intervention with coaching: An example using the Incredible Years program. Cognitive Behaviour Therapy, 37, 233-246. https://doi.org/10.1080/16506070802364511 Thompson, B. (2018). A self-administered parent training intervention for building social- emotional competence among low-income preschoolers: A non-concurrent multiple- baseline design across three cases. (Unpublished dissertation). Retrieved from: PsycINFO. (2126641109; 2018-48576-100). https://d.lib.msu.edu/etd/19310 Ursache, A., Blair, C., & Raver, C. C. (2012). The promotion of self‐regulation as a means of enhancing school readiness and early achievement in children at risk for school failure. Child Development Perspectives, 6, 122-128. https://doi.org/10.1111/j.1750- 8606.2011.00209.x Vygotsky, L.S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press. Walcott, C. M., Carlson, J. S., & Beamon, H. L. (2009). Effectiveness of a self- administered training program for parents of children with ADHD. School Psychology Forum: Research in Practice, 3, 44-62. Webster-Stratton, C. (1990). Enhancing the effectiveness of self-administered videotape parent training for families with conduct-problem children. Journal of Abnormal Child Psychology, 18, 479-492. https://doi.org/10.1007/BF00911103 Webster-Stratton, C. (2001). The Incredible Years: Parents, teachers, and children training series. Residential Treatment for Children & Youth, 18, 31-45. https://doi.org/10.1300/J007v18n03_04 Webster-Stratton, C. (2009). Teacher’s Self-Administered Manual. Seattle, WA: Incredible Years Inc. Webster-Stratton C. (2012). Incredible Teachers: Nurturing Children’s Social, Emotional, and Academic Competence. Seattle, WA: Incredible Years Inc. Webster-Stratton, C., Kolpacoff, M., & Hollinsworth, T. (1988). Self-administered videotape therapy for families with conduct-problem children: Comparison with two cost-effective treatments and a control group. Journal of Consulting and Clinical Psychology, 56, 558. https://doi.org/10.1037//0022-006x.56.4.558 Webster-Stratton, C., Reid, M. J., & Hammond, M. (2004). Treating children with early-onset conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child and Adolescent Psychology, 33(1), 105-124. https://doi.org/10.1207/S15374424JCCP3301_11 182 Webster‐Stratton, C., Reid, M., & Stoolmiller, M. (2008). Preventing conduct problems and improving school readiness: Evaluation of the Incredible Years teacher and child training programs in high‐risk schools. Journal of Child Psychology and Psychiatry, 49, 471-488. Webster-Stratton, C., Reinke, W. M., Herman, K. C., & Newcomer, L. L. (2011). The Incredible Years Teacher Classroom Management Training: The methods and principles that support fidelity of training delivery. School Psychology Review, 40, 509-529. Retrieved from: http://www.incredibleyears.com/wp-content/uploads/the-incredible-years-teacher- training-methods_11.pdf Wei, R. C., Darling-Hammond, L., & Adamson, F. (2010). Professional development in the United States: Trends and challenges (Vol. 28). Dallas, TX: National Staff Development Council. Whear, R., Thompson‐Coon, J., Boddy, K., Ford, T., Racey, D., & Stein, K. (2013). The effect of teacher‐led interventions on social and emotional behaviour in primary school children: A systematic review. British Educational Research Journal, 39, 383-420. https://doi.org/10.1080/01411926.2011.650680 Whitehead, A. L., Julious, S. A., Cooper, C. L., & Campbell, M. J. (2016). Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Statistical Methods in Medical Research, 25, 1057-1073. https://doi.org/10.1177/0962280215588241 Williams, M. E., Bywater, T., Lane, E., Williams, N. C., & Hutchings, J. (2019). Building social and emotional competence in school children: A randomised controlled trial. Psychology, 10, 107-121. Witt, J. C., & Elliott, S. N. (1985). Acceptability of classroom intervention strategies. In T. R. Kratochwill (Eds.), Advances in School Psychology (Vol. 4, pp. 251-288). Hillsdale, NJ: Erlbaum. Woodcock, S., & Reupert, A. (2012). A cross-sectional study of student teachers’ behaviour management strategies throughout their training years. The Australian Educational Researcher, 39, 159-172. doi:10.1007/s13384-012-0056-x Zoom Video Communications (version 4.6.2) [computer software]. San Jose, CA: Zoom Video Communications Inc. 183