THE EFFECTS OF THE INCREDIBLE YEARS TEACHER CLASSROOM MANAGEMENT GROUP TRAINING ON TEACHER STRATEGY USE, CLASSROOM ENVIRONMENT, RELATIONSHIPS, AND STUDENT INTERNALIZING BEHAVIOR By Erin Patricia Rappuhn A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of School Psychology – Doctor of Philosophy 2014 ABSTRACT THE EFFECTS OF THE INCREDIBLE YEARS TEACHER CLASSROOM MANAGEMENT GROUP TRAINING ON TEACHER STRATEGY USE, CLASSROOM ENVIRONMENT, RELATIONSHIPS, AND STUDENT INTERNALIZING BEHAVIOR By Erin Patricia Rappuhn This study examined the effects of the Incredible Years Teacher Classroom Management (IY-TCM) Training Program on teacher, classroom, and student level variables for preschool classrooms. Dependent measures included teacher-reported classroom management strategies, classroom atmosphere, teacher-student relationships, peer interactions, student social skills and problem behaviors. Thirty-one teachers were randomly assigned to the IY-TCM group or a bibliotherapy/reading comparison group. The total number of students included within the study was 443 students. Two target students per teacher demonstrating risk for internalizing and externalizing behavior problems were selected for additional data collection. Pretest and posttest measures were collected for all student participants through teacher ratings. Data collection for other teacher, classroom variables, and student variables were collected at pretest, midpoint, and post-test. Two- and three-level hierarchical linear modeling was used in order to analyze the nested data. Results indicated significant group differences in favor of the IY-TCM group for several teacher classroom management strategies over time, as well as significant increases in classroom management strategies over time. Social skills ratings for classroom students also significantly improved over time. Comparison students were found to have significantly lower conflict scores and higher total quality scores for the relationship between the teacher and student compared to the identified target students over time. Potential implications of these findings for teacher student relationships are discussed. The effectiveness of the IY-TCM and bibliotherapy group interventions for influencing teacher, classroom, and student level variables are discussed in addition to its potential use as a comprehensive prevention program. Keywords: Incredible Years, teacher classroom management, early childhood prevention ACKNOWLEDGEMENTS I would like to acknowledge the grant funding sources which provided invaluable support for the completion of this dissertation study, including the MSU Graduate Student Research Enhancement Award, the MSU Graduate School Dissertation Completion Fellowship Award, the MSU College of Education Distance Dissertation Fellowship, and several CEPSE departmental funding opportunities. I would like to thank Dr. John Carlson, my research advisor and dissertation chair, for his continued guidance over the years and his unwavering support and encouragement throughout the development and implementation of this dissertation project. I would also like to thank my dissertation committee members, Dr. Evelyn Oka, Dr. Cary Roseth, and Dr. Holly Brophy-Herb, for their guidance and feedback for this project. I would like to thank Holly Tiret for her contributions to this project and for all of the time and effort she put towards coordinating the intervention groups. I would like to acknowledge Amber, Kaitlin, Katie, and Ali for all of their hard work and dedication. Most importantly, I would like to extend my gratitude and love to my husband Michael and my family, who have supported me throughout my educational endeavors and have been there for me through the difficult times and the successes. I would not be where I am today without their unconditional love and guidance. iv TABLE OF CONTENTS LIST OF TABLES………………………………………………………………………….…..viii LIST OF FIGURES………………………………………………………………………………xi KEY TO ABBREVIATIONS……………………………………………………………………xii CHAPTER 1: INTRODUCTION.…………………………………………………………….......1 Theoretical Orientation……………………………………………………………………........4 Purpose of Current Study…………………………………………………………………….....5 CHAPTER 2: LITERATURE REVIEW……………….……………………………………........9 Theoretical and Conceptual Orientation…………...………………………………………......9 Psychopathology…………………………...………………………………………………....14 Early Intervention and Prevention for Social-Emotional Behaviors………………...……….14 Externalizing and Internalizing Behaviors…………………………………...…………….....16 Risk Factors for Internalizing Disorders……………………………………...……………....18 Anxiety Disorders……………………………………………………………...…………......20 Generalized Anxiety Disorder. ……………………………………………………….....23 Separation Anxiety Disorder. ………………………………………………………...….24 Social Phobia (Social Anxiety Disorder). ……………………………………………….25 Interventions for Anxiety………………………………………...………………………..….26 Depressive Disorders……………………………...……………………………………….....30 Major Depressive Disorder. …………………………………………………………......33 Interventions for Depression…………………………………………...……………………..34 Development of Social Competence……………...…………………………………………..36 Development of Social Skills……………………………………...……………………….....37 Interventions to Address Social Skill Development………………...……………………......38 Important Relationships for the Child……………………………...………………………....44 Peer Relationships. ………………………………………………………………………44 Adult-Child Relationships. ……………………………………………………………...47 The Incredible Years Series…………………………...…………………………………..….53 Incredible Years Teacher Classroom Management (IY-TCM) Training Program…….…….54 Application of the Incredible Years Series to Internalizing Symptoms……………...……....62 Research Questions and Hypotheses………………………………………………...……….65 CHAPTER 3: METHOD…………………………………….…………………………….…….73 Design………………………………...……………………………………………………....73 Participants…………………………………...……………………………………………….81 Measures………………………………………………...…………………………………....82 Teacher Strategies Questionnaire (TSQ). ……………………………………………….83 Classroom Atmosphere Measure. ……………………………………………………….85 Student Teacher Relationship Scale (STRS). …………………………………………...86 v Direct Behavior Ratings (DBR). ………………………………………………………...90 Behavior Assessment System for Children, Second Edition (BASC-2). ………………..95 Teacher Workshop Satisfaction Questionnaire. …………………………………………97 Professional Development Log of Hours. ……………………………………………….98 Procedures……………………………………………...………………………………...…...99 Recruitment. ……………………………………………………………………………..99 Pretest data collection phase. ……………………………………………………………99 Intervention phase. ……………………………………………………………………..102 Random assignment to two conditions................................................................102 IY-TCM Group Training Intervention Condition. ……………………………..103 Bibliotherapy comparison condition. …………………………………………..105 Midpoint phase. ………………………………………………………………………...107 Post-test phase. …………………………………………………………………………107 Data Analysis…………………………………………...………………………………..….107 Question One. …...……………………………………………………………………..111 Questions Two and Three. ………………………...…………………………………...111 Questions Four (a-c). ……………………………..……………………………………115 Questions Four (d-h). ……………………..……………………………………………116 Question Five. …………………...……………………………………………………..119 Missing Data. …………………………………………………………………………..120 CHAPTER 4: RESULTS…………………………………………….…………………….…...122 Question One. ………………………...…………………………………………………….122 Total professional development hours for the study. ..…………………………………122 Question Two. …………………………………...……………………………………...…..123 TSQ Summary Scale. …………………………………………………………………..123 Frequency of positive strategy use. …………………………………………………….128 Perception of usefulness of positive strategies. ………………………...……………...129 Frequency of inappropriate strategy use. ………………………………………………130 Perception of usefulness of inappropriate strategies. …………………………………..130 Frequency of planning and support strategies. ………………………………………...131 Confidence in managing classroom behavior. …………………………………………131 Frequency of positive approaches with parents strategy use. ………………………….132 TSQ Subscales of the Positive Strategies Scale. ……………………………………….132 Frequency of limit setting strategy use. ………………………………………………..137 Perception of usefulness of limit setting strategies. ……………………………………137 Frequency of social emotional strategy use. …………………………………………...138 Perception of usefulness of social emotional strategies. ……………………………….138 Frequency of proactive strategy use. …………………………………………………..139 Perception of usefulness of proactive strategies. ………………………………………140 Frequency of coaching strategy use. …………………………………………………...140 Perception of usefulness of coaching strategies. ……………………………………….141 Summary of teacher level variable outcomes. …………………………………………142 Question Three. ……………………………………………...……………………………...143 Classroom atmosphere measure. ……………………………………………………….151 STRS conflict score. …………………………………………………………………...151 vi STRS closeness score. …………………………………………………………………152 STRS dependency score. ………………………………………………………………152 STRS total score. ………………………………………………………………………153 DBR teacher-student interaction. ………………………………………………………154 Summary of classroom level variable outcomes. ……………………………………...155 Question Four (4a, 4b, 4c). ………………………...………………………………….........155 Internalizing scores for all classroom students. ………………………………………..160 Externalizing scores for all classroom students. ……………………………………….160 Social skills scores for all classroom students. ………………………………………...161 Summary of outcomes for student level variables for all students. ……………………162 Question Four (4d, 4e, 4f, 4g, 4h). ………………...…………………………………..........163 Target student internalizing scores (teacher ratings). ………………………………….173 Target student internalizing scores (parent ratings). …………………………………...173 Target student externalizing scores (teacher ratings). ………………………………….174 Target student externalizing scores (parent ratings). …………………………………..175 Target student social skills scores (teacher ratings). …………………………………...175 Target student social skills scores (parent ratings). ……………………………………176 Target student DBR positive social behavior scores. ………………………………….177 Target student DBR peer interaction scores. …………………………………………..177 Summary of outcomes for target student variables. ……………………………………178 Question Five. …..……………………………………...…………………………………...179 Overall program acceptability. …………………………………………………………179 Acceptability of strategies. ……………………………………………………………..180 CHAPTER 5: DISCUSSION………………………………………………….…..……….…...181 Treatment procedural integrity. ………………………………………….…………….182 Teacher level outcomes. ………………………………………………….…………….183 Classroom level outcomes. ………………………………………………….…………187 Student level outcomes for all classroom students. …………………………………....189 Target student outcomes. ……………………………………….……………………...191 Acceptability of treatment. …………………………………………….……………….194 Results in light of the procedural integrity. ……………………………………………195 Overall results and implications. ……………………………………………………….197 Limitations….………………………...……………………………………………………..198 Future Directions for Research….………………...…….…………………………………..201 APPENDICES………………………………………………………………………………….204 Appendix A. Professional Development Log of Hours…………………………………….205 Appendix B. Figure 2 Recruitment Flyer……….………….…………………..…………..211 Appendix C. Teacher Consent Form…………...……………………………..…………....212 Appendix D. Parent Waiver of Consent Form……………………………...……………....215 REFERENCES………………………………………………………………………………....218 vii LIST OF TABLES Table 1 DSM-IV-TR Classification of Externalizing and Internalizing Disorders…….. 16 Table 2 Research Questions, Assessment Procedures, and Data Analyses ……………. 74 Table 3 Data Collection Timeline……………………………………………………… 79 Table 4 Descriptive Statistics for Teacher and Student Participants…………………… 82 Table 5 Bibliotherapy Schedule for Assigned Readings……………………………….. 106 Table 6 Hierarchical Linear Modeling Nesting Structure……………………………… 109 Table 7 Study Variables and Codes…………………………………………………….. 110 Table 8 Missing Data per Teacher and Student Level Variables………………………. 121 Table 9 Results of t-test for professional development (PD) log of hours……………... 122 Table 10 Descriptive Data for Teacher Classroom Management Strategies and Classroom Atmosphere Ratings at Three Time Points………………………... 124 Table 11 Frequency of Positive Strategy Use…………………………………………… 125 Table 12 Perception of usefulness of positive strategy………………………………….. 125 Table 13 Frequency of inappropriate strategy use………………………………………. 126 Table 14 Perception of usefulness of inappropriate strategy…………………………….. 126 Table 15 Frequency of planning and support strategy…………………………………... 127 Table 16 Confidence in managing classroom behavior…………………………………. 127 Table 17 Frequency of positive approaches with parents strategy use………………….. 128 Table 18 Frequency of limit-setting strategy use………………………………………... 133 Table 19 Perception of usefulness of limit-setting strategies……………………………. 133 Table 20 Frequency of social-emotional strategy use…………………………………… 134 Table 21 Perception of usefulness of social-emotional strategies……………………….. 134 viii 116 Table 22 Frequency of proactive strategy use…………………………………………… 135 Table 23 Perception of usefulness of proactive strategy………………………………… 135 Table 24 Frequency of coaching strategy use…………………………………………… 136 Table 25 Perception of usefulness of coaching strategy………………………………… 136 Table 26 Descriptive Data for Target Student Observation Data for Teacher-Student Interactions, Peer Interactions, and Positive Social Behavior at Three Time Points…………………………………………………………………………... 144 Table 27 Descriptive Data for Student-Teacher Relationship Scale Ratings at Three Time Points…………………………………………………………………….. 144 Table 28 Classroom atmosphere measure……………………………………………….. 145 Table 29 STRS conflict score……………………………………………………………. 146 Table 30 STRS closeness score………………………………………………………….. 147 Table 31 STRS dependency score……………………………………………………….. 148 Table 32 STRS total score……………………………………………………………….. 149 Table 33 DBR teacher-student interaction ……………………………………………… 150 Table 34 Descriptive Data for All Classroom Students’ Teacher-Rated Externalizing Behaviors, Internalizing Behaviors, and Social Skills at Pretest and Posttest… 156 Table 35 Internalizing scores for all classroom students………………………………… 157 Table 36 Externalizing scores for all classroom students……………………………….. 158 Table 37 Social skills scores for all classroom students…………………………………. 159 Table 38 Number of Classroom Students within the Clinically Significant and At-Risk Range at Pretest and Posttest…………………………………………………... 163 Table 39 Descriptive Data for Target Student Teacher-Rated Externalizing Behaviors, Internalizing Behaviors, and Social Skills at Three Time Points……………… 164 Table 40 Target student internalizing scores (teacher ratings)…………………………... 165 Table 41 Target student internalizing scores (parent ratings)…………………………… 166 ix Table 42 Target student externalizing scores (teacher ratings)………………………….. 167 Table 43 Target student externalizing scores (parent ratings)…………………………… 168 Table 44 Target student social skills scores (teacher ratings)…………………………… 169 Table 45 Target student social skills scores (parent ratings)…………………………….. 170 Table 46 Target student DBR positive social behavior scores…………………………... 171 Table 47 Target student DBR peer interaction scores…………………………………… 172 Table 48 Number of Target Students with Clinically Significant or At-Risk Scores at Pretest and Posttest…………………………………………………………….. 179 Table 49 Results of t-test for overall program acceptability…………………………….. 180 Table 50 Results of t-test for acceptability of strategies………………………………… 180 x LIST OF FIGURES Figure 1 Conceptual Model…………………………………………………………………13 Figure 2 Recruitment Flyer…………………………………………………………………211 xi KEY TO ABBREVIATIONS APA American Psychiatric Association BASC-2 Behavior Assessment System for Children, Second Edition BASC-2 Behavior Assessment System for Children, Second Edition CBT Cognitive Behavior Therapy CD Conduct Disorder DBR Direct Behavior Rating DSM-IV TR Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision HLM Hierarchical Linear Modeling ICC Intra-class correlation IY Incredible Years IY-TCM Incredible Years Teacher Classroom Management MDD Major Depressive Disorder ODD Oppositional Defiant Disorder PQA Program Quality Assessment PRS Parent Rating Scale RTI Response to Intervention SPSS Statistical Package for the Social Sciences STRS Student-Teacher Relationship Scale TRS Teacher Rating Scale TSQ Teacher Strategy Questionnaire TWSQ Teacher Workshop Satisfaction Questionnaire xii CHAPTER 1: INTRODUCTION There is a rising awareness of the importance of addressing the mental health needs of youth, especially as efforts and initiatives are created stressing the need for the provision of prevention and intervention services for a variety of academic, social-emotional, and mental health needs (Mash & Barkley, 2006). While the prevalence of diagnosable mental health disorders in adults have been well recognized as being high, with some studies finding lifetime prevalence rates around 27% (Wittchen, Nelson, & Lachner, 1998) and approximately 5-7% of adults having a serious mental health problem in a year (New Freedom Commission on Mental Health, 2003), the prevalence rates of mental health disorders in childhood and adolescent populations are also alarming. Research indicates that approximately one in five children in the United States develop difficulties with their mental health functioning (Mash & Barkley, 2006). Children often present with significant symptoms of externalizing and internalizing behavior problems, with some Head Start prevalence rates demonstrating that 16-30% of children presented with externalizing behavior problems and 7-31% presented with symptoms of internalizing behavior problems (Qi & Kaiser, 2003). Another study with 336 Head Start children within a primarily low-income, African American population identified that 17.3% presented with externalizing behavior problems within the “problem range” and 11% presented with internalizing behaviors within the “problem range” (B. Anthony, L. Anthony, Morrel, & Acosta, 2005). With this awareness of the prevalence of mental health problems within childhood and adolescent populations, federal and local attention has been directed towards finding ways to provide preventative and early intervention services for these problems before they become 1 severe enough to require intensive, and possibly expensive, remediation (Weist & Paternite, 2006; Merrell, 2008). In fact, President Bush commissioned a working group of policy leaders to initiate a plan and recommendations for how to strengthen the current provision of mental health care services in the United States (New Freedom Commission on Mental Health, 2003). This commission determined that existing services for mental health disorders were insufficient and often initiated too late, rather than early enough to address the symptoms before developing into a more significant problem. In addition, the commission pointed to the need to focus on resilience factors in order to attempt to alleviate problems. There has been a growing emphasis placed on shifting services from remediation efforts towards preventative efforts, and many agencies have been devoted to providing research and support for the use of mental health and social-emotional programs within schools in order to build prosocial skills, coping strategies, academic skills, and alleviate early symptoms that may lead to later problems (e.g., Collaborative for Academic, Social, and Emotional Learning, 2011). A shift towards a public health model of service delivery within the community and schools has provided awareness of the need for both prevention, early intervention, and intensive remediation services within a three-tiered model in order to build these necessary basic skills and provide more intensive intervention when these universal services do not prove sufficient (Merrell & Buchanan, 2006). With these efforts in mind, continued assessment and early intervention is warranted for early childhood populations. As the awareness of the prevalence of problem behaviors in early childhood populations increases, such as within toddler and preschool populations, it is important to keep in mind that externalizing problem behaviors often receive more attention and are more easily identified than internalizing behavior problems, as overt symptoms and behaviors are more recognizable than those internalizing signs that are often unfamiliar to adults (Merrell, 2 2008; Luby et al., 2004). However, anxiety and mood disorders have been identified as two of the most prevalent childhood mental health disorders (U.S. Department of Health and Human Services, 1999), calling attention to the need for early identification and intervention for these problem areas that may often go untreated. If left untreated, individuals with internalizing symptoms or behaviors are at a heightened risk for a variety of difficulties, some of which include the continued presence and increasing severity of problem behaviors leading to a diagnosable disorder later on (Campbell, 1995), difficulties with relationships, a lack of social support, and difficulties in social contexts (Albano, Marten, Holt, Heimberg, & Barlow, 1995; Caspi, 2000). In addition, these individuals may experience rejection or isolation, low selfesteem, and difficulties with social skill development and social competence (Henricsson & Rydell, 2004; Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005; Rubin, Coplan, & Bowker, 2009), as well as poor academic outcomes (Normandeau & Guay, 1998). In addition to developing a better understanding of the prevalence of problematic symptoms and the effects of interventions to address these problems, it would also be important to understand the factors that may coincide with or contribute to the risk for developing these problem behaviors, as well as the factors that may protect children from this risk. Peer acceptance and peer relationships have been found to relate to an individual’s social competence, which may serve as a protective factor for the development of problem behaviors and future relationships (Rubin, Bukowski, & Laursen, 2009). This is especially important as it has been found that perceived peer rejection and loneliness are often reported by individuals experiencing anxiety and depression (Hutcherson & Epkins, 2009). Additionally, significant adult-child relationships (e.g., mother-child and teacher-child) within a child’s life can serve to positively or negatively influence a child’s social-emotional and behavioral development (Baker, Grant, & 3 Morlock, 2008; Lillas & Turnbull, 2009). Positive adult-child relationships including appropriate attachment, closeness, and interactions consisting of warmth and nurturance have been associated with better social-emotional and academic adjustment for children, while negative adult-child relationships consisting of overdependence and conflict are associated with more negative outcomes (Hamre & Pianta, 2001; Lillas & Turnbull, 2009; O’Connor & McCartney, 2006). This research demonstrates the important influence of peer and adult relationships on a child’s development, and how the quality of these relationships may serve as risk or protective factors for future social-emotional, behavioral, and academic development. Theoretical Orientation In order to examine and better understand the interplay between the child’s temperamental characteristics, behaviors, the classroom environment, teacher-child interactions, and peer relationships, the conceptual framework for this study was grounded within three models. The first framework in which this work was examined is the Ecological Systems Model (Bronfenbrenner, 1979), which proposes that a child’s development and behavior exist within a reciprocal interaction between the child and various levels of environmental influences. Throughout the various levels of environmental influences, including the microsystem, mesosystem, exosystem, and macrosystem, the levels in which this study is most grounded are within the microsystem and mesosystem. Within these levels, the reciprocal interaction between the immediate environment in which the child exists, such as the school and the home environment, play a role in shaping and influencing the child’s behavior and skill development (Bronfenbrenner, 1979). In a similar fashion, Sameroff’s transactional model also helped to explain the examination of this interaction between child and important individuals in the child’s life, such as a parent or a teacher (Sameroff & Fiese, 2000). This theoretical model discusses 4 how the child’s temperament and behavior can influence the adult’s reaction to the child, which in turn may create a negative adult-child relationship and influence future interactions. These two theoretical orientations highlight the importance of the environment and adult-child interactions in the development and behavior of a child. The third model used within this study was the Incredible Years Logic Model, which highlights the mechanism of change addressed within the IY Teacher Classroom Management Program for targeted training for teachers in positive classroom management strategies in order to influence change in the teachers’ behaviors, the classroom environment, and student behaviors and social competence. These three models provide support for interventions in which the focus is changing adult-child interactions, classroom management strategies, and the environmental context in which the child exists in order to influence change in the child’s behavioral development and social competence. Purpose of the Current Study The purpose of the current research study was to examine the use of the Incredible Years Teacher Classroom Management (IY-TCM) Group Training Program with preschool teachers in order to influence change in teacher practices, the classroom environment, and the behaviors and social competency of children at risk for externalizing and internalizing behavior problems. The Incredible Years (IY) Program, including the Parent Training program, the Dina Dinosaur Child Training and Classroom Curriculum programs, and the Teacher Classroom Management Training program, have been well researched and supported throughout the literature (WebsterStratton & Reid, 2010; Webster-Stratton, Reid, & Stoolmiller, 2008). Specifically, these IY programs have been found to be globally effective in reducing the severity of externalizing symptoms, specifically conduct disorder (CD) and oppositional defiant disorder (ODD; WebsterStratton & Reid, 2010; Webster-Stratton, Reid, & Hammond, 2001a; Webster-Stratton, Reid, & 5 Hammond, 2004). In addition, these programs have been found to have significant effects on positive teacher and parenting practices, classroom atmosphere, children’s social competence (Webster-Stratton, Reid, & Hammond, 2001a; Webster-Stratton, Reid, & Hammond, 2001b; Shernoff & Kratochwill, 2007), and secondary improvements associated with internalizing symptoms (i.e., anxiety and depression; Ogg & Carlson, 2009; Barrera et al., 2002). Recent efforts have examined the use of the IY programs in intervening with children that are presenting with symptoms of internalizing behaviors (Herman, Borden, Reinke, & WebsterStratton, 2011; Webster-Stratton & Herman, 2008). One randomized controlled trial (RCT) explored outcomes of five treatment groups receiving one or more IY program training versions (Herman, Borden, Reinke, & Webster-Stratton, 2011), and another RCT addressed this area by exploring the application of the parent training program to the treatment of internalizing symptoms (Webster-Stratton & Herman, 2008). The results of these studies indicated that children within the intervention groups demonstrated significantly greater reductions in severity and presence of internalizing symptoms at post-test compared to the wait-list control groups. A limitation of these two studies, however, is that both included samples of children that had already been referred to the University of Washington Parenting Clinic for conduct problems; therefore, these results are not necessarily generalizable to populations of children experiencing internalizing behaviors without clinically significant CD behaviors. However, these results provide promising evidence for the utility of the IY training programs and curriculum for influencing both externalizing and internalizing behaviors for children. In addition to the study by Herman, Borden, Reinke, and Webster-Stratton (2011), another study conducted by Snyder and colleagues (2011) served as a key study upon which the current study was built. Snyder and colleagues (2011) examined the use of the IY Teacher 6 Training Program with Head Start teachers in order to examine changes in teachers’ behavior and peers’ behavior towards target students by influencing teachers’ classroom management skills. This study demonstrated improvements in teachers’ positive behaviors and interactions, increases in positive behaviors exhibited by target students, as well as decreases in rejection, dislike, and ignoring behaviors of the peers towards target students with high levels of conduct problems and low levels of conduct problems who may have been previously excluded.. The current research study examined the outcomes associated with the IY-TCM program on the teacher, classroom, and student level. Specifically, this study investigated whether the IYTCM group training program would have an influence on the teachers’ use and perceptions of positive behavior and classroom management strategies as the teacher level outcomes. The IYTCM program suggests that the changes in the teaching strategies are the primary mechanism influencing other variables. The teacher-child relationships, classroom atmosphere, student social skills, internalizing behaviors, externalizing behaviors, and peer interactions were also examined as the classroom and student level outcomes associated with this IY-TCM training program. Some research studies have examined the influence of the IY-TCM program in these areas, including the examination of peer liking and ignoring behaviors towards students with behavior problems (Snyder et al., 2011), teachers’ use of positive classroom management strategies in the classroom (Carlson, Tiret, Bender, & Benson, 2011; Hutchings et al., 2007), positive classroom climate as measured by the Classroom Assessment Scoring System (CLASS) and the Early Childhood Environment Rating Scale – Revised (ECERS-R; Raver et al., 2008), and child social competence (Webster-Stratton, Reid, & Stoolmiller, 2008; Shernoff & Kratochwill, 2007). The current study aimed to add to the existing literature on the IY-TCM through continued exploration of these variables and the application of this early childhood 7 program to the area of internalizing behaviors in order to provide additional research regarding the use of this program as a comprehensive prevention and early intervention program for children within the schools. 8 CHAPTER 2: LITERATURE REVIEW In order to address conceptualization for this study, the following sections describe 1) the theoretical orientation for the current study, 2) psychopathology, 3) externalizing and internalizing disorders, 4) diagnostic criteria, etiology, and risk factors of internalizing disorders such as anxiety disorders, 5) interventions for anxiety, 6) diagnostic criteria, etiology, and risk factors of internalizing disorders such as depressive disorders, 7) interventions for depression, 8) childhood social competence and social skills, 9) peer relationships, 10) adult-child relationships, 11) the Incredible Years Training Program, and 12) the Incredible Years Teacher Classroom Management Training Program. Although the purpose of the current study was to influence outcomes through a top-down approach from the teacher level to the classroom, peer, and individual student level, the following review proceeds from the student level to the peer and classroom level and ends with outcomes at the teacher level. This order is presented in order to provide the rationale and need for interventions to change student behavior for internalizing symptoms, and then present the environmental factors and levels that serve to influence child behavior. These environmental factors and levels are the focus of the intervention. A review of these constructs and literature fields provides a framework for this study and explains the rationale for examining the relationship between the selected variables and the child’s development of social competence and reduction of internalizing behaviors. Theoretical and Conceptual Orientation This research study was grounded in three models of child development, including Sameroff’s Transactional Model (Sameroff & Fiese, 2000), Bronfenbrenner’s Ecological Model (1979), and the Incredible Years Teacher Program Logic Model. 9 Within the Transactional Model, the interaction between the child’s temperament or individual characteristics and the way that the environment responds to the child is an essential element in understanding the development of the child (Sameroff & Fiese, 2000). For example, if the child presents with a withdrawn or defiant temperament, then the environment and the individuals within the environment (e.g., the parent) may respond to this temperament with anxiety, frustration, agitation, or a lack of warmth in their relationship. The resulting poor adultchild relationship may continue to contribute to the presenting problem and influence the child’s future interactions with other adults or peers. This can serve to further complicate and add to the behavior problem, highlighting the interaction between the environment and the child in shaping the child’s development. The child is viewed as the resulting outcome of the influence of individuals and environment across multiple levels and the interaction between each of these levels (Lillas & Turnbull, 2009). One of the foundational principles within this model is the idea that interventions focusing on changing the child’s environment as well as the child’s behavior will allow for the strengthening of competencies and resilience (Sameroff & Fiese, 2000). Similarly, one of the major tenets of Bronfenbrenner’s Ecological Systems Theory is that a child’s development is the result of reciprocal interactions between the child and the environment (Bronfenbrenner, 1979). According to this systems theory, a child’s development is significantly influenced by a variety of levels and systems in their environment, which serve to influence development in varying direct and indirect ways. The first level of the ecological model surrounding the child is the microsystem, which includes the most direct relationships and experiences that the individual has with the environment and other individuals, such as the child’s parents, peers, teachers, and classroom. The next system is the mesosystem, which involves the interaction between environments and individuals within the microsystem, such as 10 the bidirectional relationship between the home environment and the school environment (Bronfenbrenner, 1986). These interactions can have a positive or negative influence on a child’s development and feeling of stability. Additionally, the exosystem and the macrosystem levels within this model involve more indirect environmental influences on a child, such as the community, school system, cultural beliefs, and the economy (Bronfenbrenner, 1979). The interaction of most importance in the current study is the interaction between the various contexts within the microsystem; this interplay between individuals and environments within a child’s life can serve to influence behavior and environmental responses to these behaviors through the interaction between the teacher, parents, peers, the classroom environment, and the child. This study also followed the Incredible Years Teacher Program Logic Model outlined within the Incredible Years Program. This Logic Model highlights the mechanism of change addressed within the IY Teacher Classroom Management Program, which includes a model in which intervention is targeted at training teachers in order to influence change in the teachers’ classroom management strategies, the classroom environment, and student behaviors and social competence. As outlined within this Logic Model, the IY-TCM program includes the goals of promoting positive classroom management strategies, positive relationships between teachers and students, and strategies for teaching social and emotional skills to students. Also included within the model, the short term outcomes that have been demonstrated throughout the research as resulting directly after the completion of this training program include improvements in positive classroom management strategies and coaching of student social and emotional skills, improvements in student social competence, problem solving skills, interactions with teachers and peers, and decreases in negative behaviors. This Logic Model highlights the immediate 11 teacher, classroom, and student outcomes associated with completion of this IY-TCM program. In addition, the IY Teaching Pyramid within the IY-TCM program outlines this mechanism of change through its focus within the beginning training sessions on enhancing positive teaching strategies, building positive relationships, and teaching cooperation, problem solving skills, and social skills (Webster-Stratton, 2012). Therefore, the primary focus within the first few months of training is on the promotion of positive relationships, behaviors, and skills of teachers and students within the classroom. As the IY-TCM program continues, training focuses on moving up the pyramid to discuss discipline strategies and ideas for handling misbehavior, which the program describes should be used sparingly and will be needed less often when positive behaviors and skills are promoted. In alignment with these models, this study examined the teacher, classroom, and student outcomes associated with the IY-TCM program, which suggests that the training program addresses changes at the teacher level which influence changes on the classroom level and child level. This sequence is exhibited within the conceptual model presented in Figure 1, which highlights the various outcome levels that were measured as a result of this training program. 12 Figure 1 Conceptual Model 13 Psychopathology Mental health disorders are common among adolescents and children, with approximately 20% of children in the United States exhibiting deficits or difficulties in their mental health and social-emotional functioning (Mash & Barkley, 2006). The development of a mental health disorder in the childhood or adolescent period can often be attributed to a mismatch and negative interaction between the child and the child’s environment (U.S. Department of Health and Human Services, 1999). Although one-fifth of children have a diagnosable disorder, very few of these children receive the services and treatment that they need, especially children within minority populations or those within low socioeconomic populations who may be at the greatest risk for developing serious mental health disorders (Hoagwood & Johnson, 2003). If left untreated, these childhood difficulties can lead to more severe distress and malfunctioning as the child ages and develops and can lead to consequences for society as a whole, including increased costs for intensive mental health services as adults, the costs associated with later involvement in the criminal justice system, as well as negative influences to societal functioning (Mash & Barkley, 2006). The prevalence of diagnosable disorders in preschool populations is alarming, with rates around 21% in a community sample of preschool children aged two to five years old (Lavigne et al., 1996). The importance of intervening early and allocating financial resources towards the development of prevention programs has become a primary focus, especially due to the fact that the severity of these disorders continue to rise as the child enters adolescence and adulthood (Mash & Barkley, 2006). Early Intervention and Prevention for Social-Emotional Behaviors Although there has been an increased awareness in the community as to the importance of addressing social-emotional and behavior problems early, there is still a lag in the necessary 14 identification and intervention for these developmental areas (Lillas & Turnbull, 2009). Schools have begun the process of integrating a public health model approach of service delivery into the educational system, with the goal of targeting efforts toward early intervention and prevention programs in order to attempt to meet the academic, behavioral, and social-emotional needs of the majority of students before more serious problems arise (Merrell, 2008; Merrell & Buchanan, 2006). Through the Response to Intervention (RTI) movement within educational fields, a focus on primary prevention and early intervention efforts within a multi-tiered model including the provision of universal, targeted, and intensive services has mostly been targeted towards addressing academic problem areas such as reading and math (Merrell, 2008; Reschley, 2008). However, many authors point to the importance of utilizing a public health framework and an RTI approach in order to address behavioral and social-emotional domains as well (Hunter, 2003: Merrell, 2008). Intervention and prevention efforts directed towards early childhood populations are essential in order to remediate risk before the development of more severe mental health disorders in adulthood, especially with the current knowledge as to the severity and level of impairment that continued exposure to negative mental health status can produce (Barrett & Turner, 2004; United States Department of Health and Human Services, 1999). Schools can be an effective place in which to provide these preventative services, especially due to the fact that many mental health services to which children have access are currently being provided in the schools (Doll & Yoon, 2010). In addition, schools provide a system in which services can be provided in a more accessible environment to populations of children that may not be able to seek outside services due to a number of potential barriers (Dwyer & Van Buren, 2010). 15 Externalizing and Internalizing Behaviors The classification of behavior problems has been identified as existing along a continuum of two categories: Internalizing and Externalizing Behaviors (Albano, Chorpita, & Barlow, 2003; Merrell, 2008). Table 1 depicts the categorization of psychopathology into externalizing and internalizing dimensions based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Text Revision (DSM-IV-TR; as cited by Merrell, 2008). Table 1 DSM-IV-TR Classification of Externalizing and Internalizing Disorders Externalizing Disorders Attention Deficit Hyperactivity Disorder Conduct Disorder Oppositional Defiant Disorder Internalizing Disorders Mood Disorders Anxiety Disorders Somatoform Disorders Externalizing behaviors are more likely to be identified and to receive intervention or treatment than internalizing disorders (Merrell, 2008), with some research identifying the prevalence of treatment received for children with externalizing behavior problems within a lowincome population as being two times the prevalence of treatment received for internalizing behaviors (Thompson, 2005). Prevalence rates of externalizing and internalizing behavior problems have been identified as a serious concern for children ranging from infant and preschool populations through adolescence (Briggs-Gowan, Carter, Skuban, & McCue Horwitz, 2001; Qi & Kaiser, 2003). One study investigating the prevalence of behavior problems for infants between one and two years of age found that approximately 7% of the two-year-old infants presented with Internalizing Behaviors within the subclinical or clinical range on the Child Behavior Checklist (CBCL), while almost 10% of the infants were within these ranges for 16 the Externalizing problems scale (Briggs-Gowan, Carter, Skuban, & McCue Horwitz, 2001). In addition, results indicated that approximately 12 to 16% of the two-year-old children demonstrated serious social-emotional and behavioral problems, and approximately one-third of these children demonstrated delays in their social-emotional competence. In a review of the research regarding preschool psychiatric disorders, Egger and Angold (2006) indicated that prevalence rates of behavior problems for preschool children ranged from 7 to 25% across various studies. Additionally, in a sample of Head Start preschoolers within a low SES population, prevalence rates for externalizing behaviors were identified as ranging from 16 to 30%, with internalizing behaviors not far behind with rates between 7 to 31% (Qi & Kaiser, 2003). These data indicate that social-emotional and behavioral problems are occurring more frequently than one may assume within early childhood populations, with high prevalence rates for both externalizing and internalizing behavior difficulties in infant and preschool populations. These data lend support for the focus of the proposed study on prevention for externalizing and internalizing problems within early childhood education settings. Internalizing symptoms are psychological characteristics that can be more difficult to identify than externalizing behaviors because they include more covert symptoms that are not as easily observed as overt behaviors (Merrell, 2008). The four categories typically identified within the internalizing domain include anxiety, social withdrawal, depression, and somatic complaints (Merrell, 2008). One difficulty in identifying and treating internalizing behaviors can be attributed to the fact that the symptoms related to depression, inhibition or withdrawal, anxiety, and somatic complaints often occur together and are comorbid, therefore making identification and diagnosis a more challenging task (Merrell, 2008). As highlighted in Table 1, three diagnostic categories within the DSM-IV-TR that fall within this internalizing disorder area 17 include Anxiety Disorders, Mood Disorders, and Somatoform Disorders (American Psychiatric Association, 2000; Merrell, 2008). Although depression and anxiety are typically the two major domains associated with internalizing problems, another important and related symptom domain is social withdrawal (Merrell, 2008). Social withdrawal is often associated with social isolation and inhibition attributed to a deficit in or lack of social skills (Merrell, 2008). A fourth domain within the area of internalizing disorders is the presentation of somatic complaints, which is highly correlated with anxiety, depression, and social withdrawal. Somatic complaints include feelings of sickness or physical illness, such as feeling dizzy, which are associated with a psychological component instead of a medical cause (Merrell, 2008). Risk Factors for Internalizing Disorders Research on factors associated with a higher risk for developing internalizing disorders such as depression and anxiety have identified a number of variables. One of these identified risk factors for the development of internalizing behavior problems or disorders includes birth complications, such as complications associated with prenatal exposure to alcohol, drugs, or low birth weight (U.S. Department of Health and Human Services, 1999). Biological and hereditary factors have also been linked to the potential for developing social-emotional difficulties in childhood and adolescence. A family history or current level of psychopathology, depression, or anxiety has been closely linked to childhood internalizing disorders (Luby et al., 2003; Egger & Angold, 2006; Lavigne et al., 1998). Environmental factors, including traumatic or stressful life events, and neighborhood factors including low socioeconomic status (SES), have been identified as potential risk factors for the development of these social-emotional issues, especially in preschool populations (Egger & Angold, 2006; Luby, Si, Belden, Tandon, & Spitznagel, 2009; Mesman & Koot, 2001; U.S. Department of Health and Human Services, 18 1999). Similarly, temperament and predisposition for particular behaviors may serve as risk factors, including early signs of pessimism, self-blaming, anhedonia, difficult temperament, shyness, behavioral inhibition, negative affect (e.g. sadness, fear, frustration, and anger), and withdrawal from social situations (Egger & Angold, 2006; Luby et al., 2003; U.S. Department of Health and Human Services, 1999). Family relationships lacking closeness or involving negative and harsh parenting, insecure attachment relationships, and poor parent-child relationships have also been identified as risk factors (Lavigne et al., 1998; Campbell, 1995; Mesman & Koot, 2001). Three additional factors that have been associated with the development of anxiety and depressive disorders include academic failure, lack of social skills, and difficulty with peer relationships (Herman & Ostrander, 2007). These early risk factors are especially important to consider when examining early intervention and prevention efforts, as the early development of risk has been associated with continued difficulties throughout childhood, adolescence, and adulthood (Campbell, 1995). Behavior problems in preschool and early childhood have often been demonstrated to continue well into the elementary years (Campbell, 1995). In one longitudinal study, children who were observed to demonstrate inhibited behavior at age three were found to be more likely to demonstrate behaviors of depression, difficulties in relationships with others, and a lack of social support at age 21 than those children not categorized as being inhibited, which stresses the lifetime effect that these behavioral dimensions can have on a person’s development (Caspi, 2000). Additionally, other research has identified that children with internalizing symptoms in preschool were three times more likely to have diagnoses of internalizing disorders eight years later during preadolescence (Mesman & Koot, 2001). These data highlight the importance of efforts to intervene for these behaviors before they become more severe and lasting. 19 Anxiety Disorders Among the various mental health disorders of childhood, one of the most prevalent emotional or behavioral disorders that children tend to exhibit includes symptoms of withdrawal and anxiety (Kauffman, 1997). In fact, the prevalence of anxiety disorders in childhood populations has been reported to range from approximately 10-20% (Kendall, Furr, & Podell, 2010), and it has been identified as being the most prevalent category of mental health disorder for this population (U.S. Department of Health and Human Services, 1999). Most individuals feel some level of anxiety as a response to a threat of danger or pain, as this is a natural response that serves an important function in daily life (Kauffman, 1997; Perry, 1998). However, individuals who experience a clinical anxiety disorder have a significantly higher level of stress related to these anxiety symptoms, including a drastic increase in the frequency, intensity, severity, and duration of the anxiety which impairs the ability to function normally (APA, 2000; Perry, 1998). While the lifetime prevalence of anxiety disorders in childhood within epidemiological studies has been reported as falling between 6 and 15%, some researchers suggest that approximately 2-5% of children may experience continuous anxiety symptoms (Kauffman, 1997; Chorpita & Southam-Gerow, 2006). Vasa and Pine (2004) suggest that this number may be higher, with their review of research indicating that estimates may be closer to 20% of adolescents and children having experienced some anxiety disorder during the course of their lives. However, the presence of anxiety symptoms deserves more attention, as there is a high comorbidity between anxiety disorders and other disorders such as depression, conduct disorder (CD), attention deficit hyperactivity disorder (ADHD), disruptive behavior disorders, and other 20 related anxiety disorders, ranging from 60 to 84% (Chorpita & Southam-Gerow, 2006; Vasa & Pine, 2004). Although little attention has previously been directed towards the early identification and intervention efforts for early signs of anxiety, this area has begun to receive more attention as the field learns more about the benefits of preventative efforts and the development of serious disorders later on (Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005). There is much evidence to highlight the fact that children with anxiety disorders are at a heightened risk for the later development of more serious emotional, social, and behavioral difficulties later in life (Albano, Chorpita, & Barlow, 2003). Although anxiety is typically not diagnosed until middle to late childhood or preadolescence, early onset and risk factors associated with the development of later anxiety disorders have been recognized, typically developing out of a pattern of childhood shyness and inhibition during social situations (APA, 2000; Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005). One reason that early intervention and prevention efforts for internalizing disorders and symptomology in early childhood populations has received less attention until recently is due to the difficulty in recognizing these early signs (Merrell, 2008). Identified risk factors and significant predictors for the development of later anxiety disorders include withdrawn behavior, behavioral inhibition, and shyness (Fox, Henderson, Marshall, Nicholas, & Ghera, 2005; Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005). Behavioral inhibition is a pattern of behavior in which an individual demonstrates withdrawal and avoidance of circumstances and environments that are not familiar (Hirshfeld-Becker, Biederman, & Rosenbaum, 2004). This trait tends to be relatively stable, as research has demonstrated that the majority of infants demonstrating high levels of behavioral inhibition continue to demonstrate this trait when reassessed at seven years 21 of age. Research has demonstrated that infants and toddlers that have demonstrated a pattern of shyness and behavioral inhibition were more likely to develop an anxiety disorder when entering into adolescence, with studies reporting that approximately 42% of shy children may develop anxiety disorders later on (Beidel, Morris, & Turner, 2004; Hirshfeld-Becker, Biederman, & Rosenbaum, 2004). In addition, research has indicated that adolescents with a diagnosed anxiety disorder were two to three times more likely to have an anxiety disorder in adulthood, highlighting the persistent and lifelong course of this disorder (Pine, Cohen, Gurley, Brook, & Ma, 1998). In this study, those adolescents with social phobia were highly likely to have this same diagnosis in adulthood. This highlights the importance of prevention programs that target the early signs of risk and symptomology associated with the development of anxiety disorders and the improvement of social competence. While much of the research on the treatment of childhood and adolescent anxiety disorders has focused on the targeted small group or intensive individual treatment of the disorder, reports indicate that many individuals with anxiety do not seek treatment and that many who do receive treatment often continue to demonstrate significant symptoms of anxiety after treatment (Barrett & Turner, 2004). Children and adolescents with anxiety disorders have been found to be more likely to have negative outcomes across various academic and functioning areas of life. For example, individuals with social anxiety have been found to also experience more difficulty in academic performance, demonstrate difficulties with unemployment, and have a lower quality of life (Normandeau & Guay, 1998; Wittchen & Fehm, 2003). If the level of anxiety increases to the point where there is impairment, individuals may refuse to engage in social activities, refuse to go to school, isolate themselves from others, or develop symptoms of depression (Beidel, Morris, & Turner, 2004). In addition, adolescents that have been diagnosed 22 with social anxiety and social phobia often continue to exhibit these dysfunctional symptoms throughout the lifespan, contributing to continued difficulty with relationships and disruption during important life events (Albano, Marten, Holt, Heimberg, & Barlow, 1995). However, some authors suggest that individuals that develop anxiety during adolescence or adulthood may have less severe consequences, as these individuals are more likely to benefit from treatment and recover from this anxiety; on the other hand, children who have developed anxiety disorders at a younger age may not benefit as much from treatment unless intervention occurs early on (Beidel, Morris & Turner, 2004). According to the DSM-IV, based on a categorical system of classification, there are nine categories of anxiety disorders (APA, 2000). These diagnostic categories include panic disorder, separation anxiety disorder (SAD), generalized anxiety disorder (GAD), social phobia (also known as social anxiety), agoraphobia, specific phobia, obsessive compulsive disorder (OCD), selective mutism, posttraumatic stress disorder (PTSD), and acute stress disorder (Albano, Chorpita, & Barlow, 2003; APA, 2000; Merrell, 2008). Three of the more common childhood anxiety disorders, including GAD, SAD, and social anxiety, are discussed in more detail in the following sections. Generalized Anxiety Disorder. Generalized Anxiety Disorder (GAD) involves excessive and clinically significant worry across various situations that are outside of an appropriate and typical response to such an event, including concerns regarding family, friends, school, social events, or other life situations (Chorpita & Southam-Gerow, 2006). Symptoms of GAD often include restlessness, muscle tension, irritability, fatigue, and difficulty sleeping (APA, 2000). Individuals with GAD have difficulty controlling their excessive worry and concern, and these symptoms are present most days for at least six months (APA, 2000). 23 Individuals with GAD often present with excessive muscle tension, shakiness, and somatic complaints or symptoms of sweating and nausea. The presentation of GAD in children and adolescents may differ from that of adults. Children and adolescents with GAD often focus their excessive worry and anxiety around school-related events, are constantly seeking approval and worried about their performance, or may worry about unlikely natural disasters (APA, 2000). The lifetime prevalence rates for GAD are about 5% (APA, 2000). Individuals with anxiety in early childhood or with GAD in childhood or adolescence often continue to experience these symptoms throughout the lifespan. Separation Anxiety Disorder. Separation Anxiety Disorder (SAD) is characterized by excessive worry and distress when separating from home or from an adult figure (e.g., parent; APA, 2000). These symptoms must be developmentally inappropriate for the individual’s age, and they often consist of excessive anxiety related to worrying about something bad happening to that attachment figure (APA, 2000; Chorpita & Southam-Gerow, 2006). This disorder is typically first diagnosed in childhood, with an early onset around preschool, and the symptoms must last for at least four weeks and develop before 18 years old (APA, 2000). This disorder sometimes begins after a stressful life event. Children with SAD are often so consumed with their worry and preoccupation with this anxiety that they frequently need to call home and they may constantly worry about the attachment figure being involved in an accident. These children often have difficulty going to sleep and present with physical somatic complaints when they are anticipating being separated from the adult, including nausea, vomiting, or stomachaches (APA, 2000). Children with SAD often exhibit social withdrawal and sadness when apart from their families (e.g., at school). Prevalence rates for SAD are approximately 4% for young children and adolescents, although this rate decreases as the individual enters adulthood (APA, 2000). 24 Social Phobia (Social Anxiety Disorder). Social Phobia, also known as Social Anxiety Disorder, is characterized by persistent and clinically significant fear or anxiety towards social contexts and contexts in which the individual may fear judgment based on their performance (APA, 2000). Typically, the feared situation or context is usually avoided as much as possible. Individuals with Social Phobia often fear being negatively evaluated by others, lack necessary assertiveness, have poor social skills, and have low self-esteem (APA, 2000). The presentation of symptoms may be different for children than adults. Children experiencing these symptoms of distress typically demonstrate shyness and timidity, engage in avoidance of group play or observation of play, demonstrate inhibition during social interactions, and maintain close proximity to an adult or attachment figure. This anxiety often is demonstrated in the following observable behaviors: crying, freezing, excessive tantrum throwing, staying on the side during social events or near a familiar adult, and avoidance of situations in which there may be unfamiliar individuals present (APA, 2000; U.S. Department of Health and Human Services, 1999). In some cases, children may demonstrate a refusal or inability to speak in social situations. For Social Phobia to be diagnosed in childhood, the anxiety must occur during interactions with peers as well as adults, and the symptoms must be present for six months. The lifetime prevalence of social phobia ranges from 3 to 13% in epidemiological and community studies, with about 10 to 20% of anxiety disorders in outpatient clinics including a diagnosis of social phobia (APA, 2000). Children that exhibit symptoms that may put them at risk for developing social anxiety often have a limited number of social interactions, deficits in their social skill development, a lack of coping skills and strategies, low self-esteem, difficulty with peer relationships including feelings of isolation and being the target of bullying, and difficulties in school, including refusal 25 to go to school and difficulty with skill development (Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005; Rubin, Coplan, & Bowker, 2009). If intervention for the anxiety symptoms does not begin until the child advances in age, this allows more time for these early signs of anxiety to accumulate into greater negative influences, including further peer rejection and isolation, and a decrease in self esteem across domains (Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005). A lack of or delay in the development of social skills and a tendency to avoid social interactions are important factors that may be associated with or lead to the development of social anxiety symptoms (Beidel, Morris, & Turner, 2004). Researchers have postulated that if children receive intervention and treatment for risk behaviors such as shyness, withdrawn behaviors, and anxiety at an earlier age, then there is a greater likelihood for the development of social competence and adaptive functioning in both social and academic areas, which can lead to a decrease in the likelihood of future psychopathology (Wettig, Coleman, & Geider, 2011). Interventions for Anxiety The most commonly utilized treatment methods for anxiety disorders in childhood and adolescence that have received the strongest empirical support in the literature include cognitive behavior therapy (CBT) and its specific components (e.g., systematic desensitization, contingency management, exposure therapy, cognitive restructuring, social skills training, modeling), and psychopharmacological treatment (e.g., Selective Serotonin Reuptake Inhibitors) (Roblek & Piacentini, 2005; Segool & Carlson, 2008; U.S. Department of Health and Human Services, 1999). In one meta-analytic review of research utilizing CBT interventions and/or psychopharmacological treatment for children with social anxiety disorders, results indicated that CBT and SSRI treatments both demonstrated large decreases in the social anxiety symptoms that children were displaying from pretest to post-treatment assessment, indicating that both 26 treatment approaches reduced the anxiety behaviors for these children (Segool & Carlson, 2008). However, results from the SSRI treatments indicated significantly greater decreases in the social anxiety and greater improvement in functioning of children within these studies. In terms of measures of social competence, CBT demonstrated greater increases in social functioning than did the SSRI treatments (Segool & Carlson, 2008). One such study utilizing a CBT treatment for individuals between the ages of seven and sixteen referred to a clinic for excessive anxiety examined the differences in change outcomes between a CBT condition with parental involvement and a CBT condition with limited parental involvement (Silverman, Kurtines, Jaccard, & Pina, 2009). The CBT condition consisted of treatment in which the participants were taught how to use behavioral and cognitive strategies when the youth were exposed to anxiety-provoking situations. This treatment consisted of 50 minute individual sessions with the therapist and 10 minute sessions with the parent and therapist. The CBT condition with parental involvement consisted of the same CBT strategies, but the parent was actively included in the full 60 minute session. Results indicated that both the CBT and the CBT with parental involvement conditions demonstrated statistically significant improvements in the levels of anxiety from pretreatment to posttreatment for mother and selfratings of the child’s anxiety, indicating that both groups demonstrated a reduction in their symptom severity. For the child’s self-ratings, these effects continued to show improvement at the 12 month follow-up, whereas the mothers’ ratings indicated maintenance of the effects but not a significant improvement. These results indicated that parental involvement in the treatment did not significantly influence the child’s reduction in anxiety symptoms following CBT (Silverman, Kurtines, Jaccard, & Pina, 2009). 27 Bernstein, Layne, Egan, and Tennison (2005) conducted a study in which they compared different interventions for anxiety used within the schools. Sixty-one children between the ages of seven and eleven years old demonstrating symptoms related to separation anxiety disorder, generalized anxiety disorder, and social phobia were assigned to one of the three groups. One treatment group received nine weeks of group CBT, the second group received nine weeks of group CBT combined with parent training, and the third group did not receive treatment. The CBT conditions consisted of groups of eight to nine children, and the treatment used was the FRIENDS Program (Bernstein, Layne, Egan, & Tennison, 2005). Results indicated that children within the two CBT treatment groups demonstrated significantly greater improvement in their anxiety symptoms after the treatment compared to the control group. In addition, the CBT combined with parental training indicated significantly greater improvements than the CBT condition alone on some measures, such as the Clinical Global Impressions scale and the MASC filled out by parents (Bernstein, Layne, Egan, & Tennison, 2005). These results, although contradicting Silverman and colleagues (2009) work which highlighted no difference between conditions with or without parental involvement, continue to add to the literature supporting CBT as an effective treatment approach for anxiety disorders for children. Another frequently utilized intervention for young childhood populations with anxiety disorders is the Coping Cat Program. This treatment program includes the structure and principles of CBT and has been applied to children between the ages of seven and thirteen presenting with primary diagnoses of social phobia, separation anxiety, or generalized anxiety disorder (Kendall, Furr, & Podell, 2010). This treatment has been found to be a probably efficacious treatment program, and many studies have demonstrated the positive effects of this treatment in reducing anxiety symptoms through teacher report, parent report, self-report, and 28 observations, as well as a decrease in the number of participants meeting criteria for an anxiety disorder from pretreatment to posttreatment (Kendall, Furr, & Podell, 2010). Although some authors suggest that the use of CBT treatment and cognitive approaches with children under the age of six may be inappropriate (Schoff D’Eramo & Franics, 2004), others have argued that components of CBT can be adapted and applied to these early childhood populations (Hirshfeld-Becker, Micco, Mazursky, Bruett, & Henin, 2011). These authors provide a review of research that has been conducted with younger populations utilizing adapted individual components of CBT. In this review, Hirshfeld-Becker and colleagues (2011) highlight the adaptability of coping skills training, modeling, exposure, and changing cognitive distortions through the use of developmentally appropriate pictures, visual displays, examples, slower-paced exposure procedures, and specific emphasis on relaxation strategies and parental involvement in the treatment. This review reinforces the fact that while young children do not have the higher level of cognition that older children and adolescents possess, these children are still able to use problem solving strategies, make predictions, and identify their feelings (Hirshfeld-Becker et al., 2011). Amidst the debate over the use of CBT in early childhood populations, other intervention methods have been utilized that take these developmental factors into consideration. Wettig, Coleman and Geider (2011) conducted two studies in which the use of Theraplay as a therapeutic intervention for children ages two to six years old was used in order to target shyness and social anxiety/withdrawn behaviors. The first study was conducted in Germany with 22 children in which a highly trained therapist led the Theraplay treatment, whereas the second study focused on generalizability of the intervention and included 167 children at different clinical centers with different therapists. Both studies included a comparison control group, and all children had dual 29 diagnoses of a language disorder and shyness or social anxiety (Wettig, Coleman, & Geider, 2011). Intervention duration was relatively short for most participants, including an average of 18 total sessions. Posttreatment assessment indicated that both of the experimental groups demonstrated improvements in shyness, timidity, and mistrust/suspicion. Children within the experimental groups demonstrated significant improvements at posttreatment in the following areas: shyness/timidity, attention, poor cooperation, overadapting/conforming, social withdrawal, and mistrust. Although significant improvements were found within this study, several of these symptoms were still below those of the control group (Wettig, Coleman, & Geider, 2011). These results were maintained at the two-year follow-up, and the shyness level scores for the experimental groups were equal to that of the control group. Improvements in communication skills were also measured, suggesting that a decrease in symptoms of social anxiety may relate to an increase in the demonstration of verbal communication skills. The authors highlight the fact that these results are promising for the effectiveness of short-duration interventions for social anxiety (Wettig, Coleman, & Geider, 2011). Depressive Disorders Although previously believed to develop in adolescence or adulthood, the field has come to the realization that mood disorders and depressive symptoms can be present at much younger ages (e.g., as early as three to six years of age; Stark et al., 2006). The prevalence of mood disorders in childhood and adolescent individuals has been identified as being around 6%, making this disorder one of the most prevalent disorders of childhood (U.S. Department of Health and Human Services, 1999). According to the DSM-IV-TR, mood disorders are typically divided into the two broad categories of depressive disorders and bipolar disorders (APA, 2000; 30 Stark et al., 2006). Depressive Disorders are further classified into several categorical areas, such as Major Depressive Disorder (MDD), Dysthymic Disorder (DD), and Depressive Disorder not otherwise specified (DDNOS; Merrell, 2008; Stark et al., 2006). Lifetime prevalence rates for Major Depressive Disorders are higher than the other depressive disorders (APA, 2000). Based on a review of various epidemiological, clinical, and community studies, prevalence rates for depressive disorders typically range from 1 to 3% for school-age children, although some rates have been estimated as high as 5 to 6% (Merrell, 2008; Stark et al., 2006). Prevalence rates continue to steadily increase through adolescence (approximately 3%) and adulthood (MDD: 10-25% in women, 5-12% in men) (APA, 2000; Stark et al., 2006). According to some prevalence studies, the prevalence of depressive disorders within pediatric populations was greater than 8% (Emslie & Mayes, 2001), with some reviews indicating percentages that could range from 0 to 2% in children up to 9% in adolescents (Delate, Gelenberg, Simmons, & Motheral, 2004; Egger & Angold, 2006). Women are twice as likely as men to present with a depressive disorder in adolescence and adulthood, whereas within childhood populations the prevalence is approximately equal (APA, 2000). Although the average age of onset for depressive disorders is around 20 years of age, it is suggested that individuals born more recently are at greater risk for developing this disorder at younger ages (APA, 2000). The presentation of depressive symptoms may be different for children and adolescents than for adults. Within adult populations, the presence of depression may result in the person feeling and acting sad (APA, 2000). However, children and adolescents with symptoms of depression typically demonstrate behaviors such as intense irritability, continuous anger, becoming easily frustrated over small issues, or reacting to situations with intense anger. 31 Children are more likely to show signs of irritability, guilt, somatic complaints, social withdrawal, and a decreased ability to enjoy interesting activities, whereas adults and adolescents are much more likely to show their symptoms through motor retardation, delusions, and hypersomnia (APA, 2000; Luby, 2009). Some researchers have argued that children in preschool populations meet criteria for depression, but fail to meet the duration criteria for symptoms; these researchers have also argued that the duration criteria should not be used when diagnosing children (Luby, Si, Belden, Tandon, & Spitznagel, 2009). Reports indicate that children and adolescents with depressive disorders frequently present with a comorbid disorder; in fact, approximately two-thirds of children with Major Depressive Disorder have been identified as having another disorder as well (U.S. Department of Health and Human Services, 1999). Other disorders that frequently co-occur with depressive disorders include anxiety disorders, disruptive behavior disorders, and eating disorders (APA, 2000). It is especially important within preschool populations to consider comorbidity, as approximately 25-30% of children with one disorder present with other comorbid disorders, and these children demonstrate increased impairment in functioning (Egger & Angold, 2006). As internalizing disorders are often comorbid with behavioral disorders, these externalizing behaviors often receive more attention and intervention than the more covert emotional disorders. An important consideration within child development in relation to internalizing symptoms is the development of a healthy self-concept (Merrell, 2008). A positive and supportive classroom environment, in addition to enhanced and positive teacher-student relationships, can have a great influence on a child’s positive self-evaluation, which is an area in which children with depression are often lacking (Herman & Ostrander, 2007). 32 Due to the relatively recent increase in awareness and knowledge regarding the presence of depressive symptoms and disorders in young children, with rates ranging across studies from 1.5 to approximately 8% (Emslie & Mayes, 2001; Stark et al., 2006), there is a greater need for increased public attention towards the early identification and intervention for depressive symptoms and disorders during the developmental stage of preschool (Luby et al., 2004). In a review of research regarding preschool psychiatric disorders, Egger and Angold (2006) discuss how some studies have identified that 93% of preschoolers meeting criteria for a depressive disorder presented with significant impairments, indicating that this disorder is an important area to consider in preschool populations. Unfortunately, early risk and symptoms of depression in preschool and early childhood populations are often left undetected, due in part to the adult’s unfamiliarity with the signs of risk or lack of recognition or reporting of changes in the child’s behavior (Luby et al., 2004; Luby, 2009). In one of the first longitudinal studies regarding preschool depressive disorders, Luby and colleagues (2009) determined that children presenting with diagnostic symptoms of MDD were approximately 11 times more likely than typical children to continue to present with symptoms of depression at one and two year follow-up assessment periods. These data provide evidence for the severity and recurring nature of depressive disorders from early childhood through school age populations if left untreated. Major Depressive Disorder. Major Depressive Disorder (MDD) is a mood disorder that is identified by the presence of at least one major depressive episode (APA, 2000). MDD is diagnosed when depressive symptoms or a loss of interest in most activities occurs for more than a two week period and when the symptoms continue for almost every day. Usually these symptoms involve a loss of interest and pleasure in activities and hobbies that the individual has previously enjoyed. Other areas that are affected include the individual’s appetite, somatic 33 complaints, disturbance in sleeping patterns, psychomotor changes (e.g., restlessness, agitation, or retardation of speech and movement), decreased energy level and increased tiredness, an increase in feelings of worthlessness and guilt, inability to concentrate or make decisions, suicidal ideation or attempts, as well as symptoms that negatively and significantly impact the individual’s social, academic, or occupational functioning (APA, 2000). The occurrence of MDD can stem from a stressful life event, such as the death of a loved one or a significant struggle in a person’s life (APA, 2000). Interventions for Depression Within the literature, there are two treatment approaches for depression that have received the most support for decreasing symptoms and improving functioning for individuals: cognitive behavior therapy (CBT) and psychopharmacological treatment (Stark et al., 2006; U.S. Department of Health and Human Services, 1999). There has been little research conducted in order to test the efficacy and safety of the use of prescription medications for children and adolescents (U.S. Department of Health and Human Services, 1999). However, of the medications prescribed to children for depression, the use of the selective serotonin reuptake inhibitor (SSRI) named fluoxetine has received the most support in terms of its positive results with this population (U.S. Department of Health and Human Services, 1999). A meta-analysis was conducted by Weisz, McCarty, and Valeri (2006) in order to examine effect sizes and utility of psychotherapeutic interventions with school-aged children. The authors highlight an increased need to provide an effective alternative to psychopharmacological treatment for childhood depression, due in large part to the lack of research demonstrating reliable safety and efficacy data, as well as the potential side effects. Although previous meta-analytic studies reported high effect sizes for psychotherapeutic 34 interventions for youth (0.97-1.27), these authors applied more strict guidelines and found only moderate effect sizes (0.34) for these treatments in school-age populations. CBT was the most commonly used method within studies, with 31 of the 44 studies involving this treatment approach. Findings indicated that CBT approaches did not differ in effect size from noncognitive treatments (e.g., relaxation techniques), that more studies focused on adolescent populations than childhood populations, and that similar effect sizes were demonstrated for decreasing symptoms of depression and anxiety. The authors assert that this last finding may point to the utility of similar intervention methods for depression and anxiety symptoms (Weisz, McCarty, & Valeri, 2006). In a meta-analytic review of prevention programs for depression, 32 prevention programs were identified including such methods as CBT, psychoeducation, and interpersonal therapy (Stice, Shaw, Bohon, Marti, & Rohde, 2009). This review identified that 41% of the prevention studies demonstrated significant decreases in depressive symptoms for participants and 13% demonstrated decreases in likelihood for developing depressive symptoms later on. Effect sizes across studies were small, at approximately 0.14. Across the studies, the average age of participants was between 10 and 19 years of age (Stice et al., 2009). Results indicated that effect sizes were significantly greater for prevention studies focusing on high risk groups of individuals (r = 0.23) than for universal programs (r = 0.04), and that interventions that required less treatment time, included homework assignments, and were conducted with older adolescents demonstrated greater effect sizes. No significant differences were found between the different intervention techniques and methods (Stice et al., 2009). Although universal programs for depression were found to be less effective than targeted interventions, it is important to keep in 35 mind that participants ranged from middle childhood to late adolescence rather than early childhood populations, who may benefit most from universal prevention programs. Due to the fact that the recognition of depression in preschool populations is a relatively new revelation, there have been few intervention studies completed with this population (Luby, Lenze, & Tillman, 2012). These authors developed a pilot RCT study to examine the use of an adapted psychotherapeutic intervention with three to seven year old preschoolers called the Parent-Child Interaction Therapy – Emotion Development (PCIT-ED) intervention. This intervention consisted of in vivo practice and coaching methods in order to improve positive parent-child relationships, increase nurturance and appropriate commands used in parenting, and improve child emotional control (Luby, Lenze, & Tillman, 2012). This intervention was compared to a control pscyhoeducational intervention. Results indicated that participants in the PCIT-ED and the control groups demonstrated significant decreases in depressive symptoms and severity, improvement in internalizing scores, and improvements in impairment. Significant improvements in emotional development and executive functioning were found for the PCIT-ED group but not for the control group. Development of Social Competence Social competence has been defined as a broad term including domains related to behavioral, cognitive, and emotional functioning that is necessary to engage in social interactions (Merrell, 2008). This concept is often related to the subcategories of social skills, adaptive behavior, and peer relationships (Merrell, 2008). Other researchers have defined the concept of social competence as consisting of three dimensions: aggression, social withdrawal, and prosocial behaviors (Howes, 2000). In order to develop social competence as a preschool child, the behaviors that one must be able to demonstrate include “communication abilities, emotion 36 knowledge, self-regulation, access to a repertoire of appropriate and effective social strategies, and a sense of self-efficacy in social situations” (Rose-Krasnor & Denham, 2009, p. 162). In addition, these authors include problem solving in social situations and positive interactions with others as important dynamics of social competence in predicting later competence. Young children that demonstrate more difficulty and less skill in social interactions often are the targets of bullying, peer isolation and rejection, loneliness, school dropout, lower academic performance, and significant difficulties in creating and maintaining friendships (Rubin, Bowker, & Kennedy, 2009). Children with internalizing and externalizing behavior problems have been found to have lower social competence scores, especially in the areas of prosocial behaviors and initiating interactions with others in social situations for those with internalizing problems (Henricsson & Rydell, 2004). These authors postulate that while some children with internalizing problems may be able to explain the appropriate prosocial behaviors that they should be using, they fail to behave in this manner due to their tendency to avoid peer social interactions (Henricsson & Rydell, 2004). In addition, a child’s own fears or maladaptive emotions within a social situation can lead to a decrease in the child’s ability to effectively assess and respond to the social interaction in a competent way (Rose-Krasnor & Denham, 2009). Socially-withdrawn children are even more likely to remove themselves from social interactions and simply watch the interactions from the sideline (Rubin, Bowker, & Kennedy, 2009). Development of Social Skills There is much variability within the research literature as to the definition of social skills (Merrell, 2008). While one group of authors has defined social skills as the process of “identifying emotions from social cues, goal setting, perspective taking, interpersonal problem solving, conflict resolution, and decision making” (Durlak, Weissberg, Dymnicki, Taylor, & 37 Schellinger, 2011, p. 410), others have broadly defined this term as “specific behaviors, that when initiated, lead to desirable social outcomes for the person initiating them” (Merrell, 2008, p. 381). In an effort to create a similar behavioral classification system as is used for problem behaviors (e.g., externalizing and internalizing behaviors), Caldarella and Merrell (1997) proposed a classification system of social skill constructs from the literature. These social skill areas are grouped into five dimensions: peer relations, self management, academic skills, compliance, and assertion. Peer relations relates to a child’s ability to interact with peers in a prosocial manner, self management includes the ability to control behaviors and feelings, and academic skills refer to the social skills necessary to be successful in the classroom. Additionally, compliance relates to the child’s ability to conform to and follow rules and expectations, while the assertion domain refers to skills related to initiating interactions with others (Caldarella & Merrell, 1997). A child’s inability to effectively interact with others or use appropriate social skills can lead to the development of depressive symptoms due to a lack of positive attention and reinforcement from peers (Hokanson & Rubert, 1991). Social skills have been found to play an important role in the development of the child and their ability to form close relationships with others (U.S. Department of Health and Human Services, 1999). However, if a child demonstrates difficulties in social skill development, they are more likely to be isolated or rejected by peers and demonstrate withdrawn behaviors during social situations (U.S. Department of Health and Human Services, 1999). Interventions to Address Social Skill Development The literature and research on social anxiety, social withdrawal, and behavioral inhibition make references to children’s lack of social skill development and knowledge (Morris & March, 38 2004). Morris (2004) points to the reciprocal influence and transactional relationship between these two constructs, as children demonstrating behaviors of social avoidance or withdrawal in social situations typically isolate themselves from others and avoid these social interactions, which prevents them from acquiring the necessary social skills. This can lead to an increase in and maintenance of the child’s level of anxiety and avoidance due to the lack of skills. As mentioned previously, some researchers suggest that intervention methods including cognitive behavioral therapy (CBT) may not be appropriate for children that are six years of age or younger, as the focus of such interventions involves cognitive techniques that may be developmentally inappropriate at this age (Schoff D’Éramo & Francis, 2004). Therefore, there is uncertainty as to the most effective method of early intervention and prevention efforts for young children exhibiting anxious, inhibited, and withdrawn behaviors. However, it has been suggested that interventions incorporating social skills training and development may serve as an appropriate method of treatment delivery for children exhibiting these symptoms of anxiety (Beidel, Morris, & Turner, 2004). Social learning theory also points to the importance of teaching social skills at an early age in order to prevent later social difficulties, as children who have not been given instruction in developing positive social skills and interactions with their peers, or children who have experienced failure and criticism for previous social interactions, may develop more anxious and withdrawn behaviors as a result of these experiences (Kauffman, 1997). Effective social skills interventions or curriculum should focus on the development of social-emotional competence and social skills while also reducing maladaptive aggressive or withdrawn behaviors (Masten & Coatsworth, 1998; Domitrovich, Cortes, & Greenberg, 2007). The development of social competence within childhood populations can be achieved through 39 methods of intervening at the individual skill level or through changes to the child’s environment in order to support the development of competence within social domains (e.g., the classroom) (Masten & Coatsworth, 1998). An area that plays an important role in the development of social competence includes appropriate self-regulation skills, as children that are less skilled in managing their inappropriate and negative emotions may experience anxiety, aggressive behaviors, or social-emotional distress (Masten & Coatsworth, 1998). With this in mind, several research studies have examined the use of interventions for anxiety in combination with social skills training (Beidel, Morris, & Turner, 2004). Spence, Donovan, & Brechman-Toussaint (2000) conducted a study in which children with social phobia between the ages of 7 and 14 were assigned to a CBT treatment including a social skills training portion, a CBT/social skills and parent involvement condition, or a wait list group. Treatment groups received treatment after referral to the Kids Coping Project in the Behaviour Research and Therapy Centre in Australia. Results of this study indicated that both treatment conditions resulted in fewer children meeting diagnostic criteria for social phobia, as well as fewer symptoms related to generalized anxiety. In addition, the children included within the CBT and social skills treatment group demonstrated significant improvements in social skills functioning based on parental ratings (Spence, Donovan, & Brechman-Toussaint, 2000). The authors highlight the association between the improvement in social skills and reduction in anxiety symptoms, pointing to the importance of improved social skills in the treatment of social anxiety. Another study utilizing an intervention in which treatment methods were combined in order to increase social skills and reduce symptoms of social anxiety included the use of the Social Effectiveness Therapy for Children (SET-C) for individuals with social phobia between the ages of 8 and 12 referred to the Anxiety Prevention and Treatment Center of the Medical 40 University of South Carolina (Beidel, Turner, & Morris, 2000). Children were randomly assigned to the SET-C group or a Testbusters Intervention Control Group. The SET-C intervention focuses on reducing social anxiety, increasing social functioning and social skills, and increasing the amount of time and engagement in social activities with others through the use of social skills training, psychoeducation, and opportunities to practice these skills during peer play situations. Results indicated clinically and statistically significant improvements in the following areas: social phobia symptoms, symptoms of generalized anxiety, social skills, reduction in avoidance behavior and more competence during social interactions (Beidel, Turner, & Morris, 2000). In addition, 67% of the children included in the SET-C group did not meet criteria for a clinical diagnosis of social phobia after treatment. These results were maintained at the six month follow-up. In order to demonstrate the relationship between interventions for children with symptoms of anxiety and the child’s development of social competence, one study examined the effects of a home and parent-based intervention for 43 preschoolers in Montreal exhibiting scores of at least one standard deviation above the mean on the anxious-withdrawn scale of the Social Competence and Behavior Evaluation (SCBE) assessment (LaFreniere & Capuano, 1997). Participants within the intervention group received a six month intervention including home visits, parent education, parent-child play and interaction training, and guidance and support. Results indicated that differences on the anxious-withdrawn symptoms between the control and intervention groups were marginally significant at post-test. However, children within the intervention group demonstrated a significant difference and improvement on social competence ratings in comparison to the control group (LaFreniere & Capuano, 1997). These results indicate 41 that this intervention not only influenced anxious-withdrawn behaviors for this preschool population, but also had an effect on their levels of social competence in the classroom. Many studies have not only supported the importance of interventions for social competence in relation to anxiety, but have also supported the importance of early preventative efforts within this domain. In a meta-analytic review of effect sizes and moderators related to class-wide social skills interventions in preschool and kindergarten through twelfth grade, January, Casey, and Paulson (2011) highlight the critical importance of early intervention for social skill development. The review of research in this area indicated larger effect sizes for outcomes in studies in which social skills interventions were conducted in Kindergarten and preschool programs (d = 0.55). In comparison, smaller effect sizes were found for the early elementary (d = 0.12) and middle school (d = 0.19) grades (January, Case, & Paulson, 2011). While some early interventions have been targeted at small group instruction for children at-risk, other studies have examined the use of universal interventions delivered as a socialemotional curriculum to the entire classroom. Domitrovich, Cortes, and Greenberg (2007) conducted a three-year randomized control study in which the goal was to determine the effectiveness of an adapted version of the Promoting Alternative Thinking Strategies curriculum (PATHS) for Head Start preschool children in improving children’s social-emotional skills and development. The sample included 246 children in Head Start programs in Pennsylvania, and the universal social-emotional curriculum, consisting of thirty lessons delivered one time per week, was implemented by classroom teachers to all students. Pre- and post-test data were gathered for the children through assessment of the child’s social and emotional skills, parent ratings, and teacher ratings. Results indicated that children in the intervention groups demonstrated better “emotion vocabulary” and were better able to properly identify feelings than 42 the control group (Domitrovich, Cortes, & Greenberg, 2007). Teacher ratings indicated that students in the intervention group demonstrated more social-emotional and interpersonal skills, cooperation, emotional understanding, and were rated as less anxious and less withdrawn than the control group at the post-test. In addition, parents of the children in the control groups rated their children as being more skilled in their social interactions at post-test than the control group. Another widely recognized preschool intervention program emphasizing the promotion of social skills development is the Incredible Years Dinosaur School Program (Webster-Stratton, 2012; Domitrovich, Cortes, & Greenberg, 2007). Although the Incredible Years Program is most known for its focus on early intervention for conduct problems in young children in preschool and early elementary grades, the applicability of the social skills curriculum for improving internalizing symptoms has recently become an interest within the field (Herman, Borden, Reinke, & Webster-Stratton, 2011). This interest has arisen out of the wide range of social skills that the Incredible Years Program targets, including self-control, problem solving skills, emotional awareness, and the building of friendship skills, which are all skills which can apply to skill development for both externalizing and internalizing problems. Researchers suggest that the components of the Incredible Years programs allow the therapist an opportunity to target other internalizing symptoms as well, such as social withdrawal and social skill deficits (Webster-Stratton & Herman, 2008). In addition, previous research conducted with the Incredible Years intervention program have indicated the predictive effect of comorbid anxiety or depressive symptoms on significant improvements for children with conduct problems (Beauchaine, Webster-Stratton, & Reid, 2005), suggesting that the program may also address these internalizing symptoms. These authors suggest that children that are exhibiting high levels 43 of anxiety may respond well to the use of social attention and rewards in the child training program, which may lead to greater improvements in behavior and functioning. Important Relationships for the Child Peer Relationships. Peer relationships are often directly related to the development of social competence and social skills, as these allow appropriate skills in initiating and maintaining positive peer outcomes within social situations (Merrell, 2008). Peer relationships are important throughout the lifespan, as the amount of time that children spend with their peers throughout the day is substantial and these relationships provide opportunities for the child to develop friendship, companionship, support systems, and social skills (Rubin, Bukowski, & Laursen, 2009; Hay, Caplan, & Nash, 2009). While some researchers argue that mother-child relationships serve as a means to influence later peer development, others argue that these two relationships develop concurrently (Hay, Caplan, & Nash, 2009). Regardless, the importance of peer relationships in addition to parent-child relationships in influencing a child’s adjustment has been well supported, especially in relation to the influence that peer rejection and peer acceptance can play in a child’s development and future ability to form relationships (Rubin, Bukowski, & Laursen, 2009). The preschool period is an especially important time for the development and growth of peer interactions and relationships, especially as children begin to communicate and play with others in order to create friendships (Fabes, Martin, & Hanish, 2009; Hay, Caplan, & Nash, 2009; Coplan & Arbeau, 2009). As this is likely the most amount of time that a young child has spent with this many children at a time, opportunities to learn and grow throughout play interactions can help to provide children with opportunities to develop and gain skills in engaging in healthy and appropriate peer interactions (Fabes, Martin, & Hanish, 2009; Coplan & 44 Arbeau, 2009). Research on child development and peer interactions have found that children can develop friendships with others through experiences in which they engage in prosocial play and conflicts, and children at early ages have also been able to demonstrate a higher degree of liking and friendliness towards one individual more than another in the selection of their friends (Coplan & Ardeau, 2009; Hay, Caplan, & Nash, 2009). These social interactions will be more likely to lead to further opportunities to engage in play with these peers, which provide more opportunities for the children to develop their skills (Hay, Caplan, & Nash, 2009). Although a young child may initially engage in more “nonsocial” types of play (e.g., exclusively observing others’ play or playing alone), it is expected that the child will progress to more active and advanced forms of play before or while entering preschool age (Coplan & Arbeau, 2009, p. 144). Continuation of this type of observational play, rather than development of more engaged play, may leader to shyness and risk for internalizing behaviors. Behaviors that are learned during this preschool period that will help to facilitate effective peer interactions in elementary grades may include such behaviors as being able to selfregulate behavior and engage in appropriate displays of cooperativeness, while aggressiveness and withdrawn behaviors are less likely to lead to successful peer interactions later on (Fabes, Martin, & Hanish, 2009). In addition to these temperamental variables, other factors that have been found to influence the development of positive or negative peer relationships include genetic predisposition to behaviors (e.g., aggression), gender, language development, cognitive abilities, family composition and relationships, and previous experience in relationships (Hay, Caplan, & Nash, 2009). Hutcherson and Epkins (2009) discuss the literature base examining the fact that individuals with anxiety and depression tend to have overly negative views of their peer 45 acceptance and friendships, which is often associated with an increased level of loneliness. In order to examine this further, Hutcherson and Epkins (2009) conducted a study in which the self and parental reports of peer acceptance, rejection, and loneliness of 100 girls in fourth through sixth grade were examined. Children rated their levels of anxiety, depression, loneliness, peer acceptance, parental perception, and social support, while parents rated their child’s social acceptance and their own parental behavior. Ratings of loneliness were highly related to low peer acceptance and were significantly related to social anxiety and depression. High levels of social anxiety and depression were negatively correlated with peer support and were significantly related to their social acceptance (Hutcherson & Epkins, 2009). This study highlights the importance of considering peer acceptance and friendship variables in the understanding of depression and anxiety within adolescent populations. These authors hypothesize that peer rejection that occurs earlier in childhood would have similarly detrimental and negative effects. Although it is generally accepted that individuals with clinical levels of anxiety or depression experience difficulties in the development of their social skills and relationships with others, there is not as much research examining the influence that the presence of lower levels of anxiety and depression, that put a child at risk but do not meet diagnostic criteria for a disorder, may have on peer friendships and the quality of these relationships (Rose et al., 2011). To test this hypothesis, Rose and colleagues (2011) conducted two studies in which they examined the current and future number and quality of friendships for individuals in third, fifth, seventh, and ninth grade in relation to symptoms of anxiety and depression with a nonclinical sample of children. Peer nominations were used in which experimenters utilized a rating system in which students identified their top three friends and circled their best friends in order to determine reciprocal friendships. The results of this study suggested that depression was predictive of 46 having a lower number of friends and lower quality friendships. However, anxiety was a predictor of a greater number of friendships and higher quality friendships, but a higher chance of having conflict in these relationships (Rose et al., 2011). These results were similar for the predictive nature of anxiety and depression for future friendships at the second time interval. The authors suggest that individuals with generalized anxiety or subclinical anxiety may be better able to maintain friendships than those individuals with clinical levels of anxiety or social phobias (Rose et al., 2011). Other studies have supported the importance of peer relationships as a protective factor for the development of internalizing problems as well, finding that difficulty in developing peer relationships, anxious behaviors, and excessive loneliness were associated with poor social-emotional adjustment, a cycle of poor interactions and rejection, and increased likelihood of internalizing problems later on (Ladd & Troop-Gordon, 2003). Adult-Child Relationships. Some researchers have argued that many of the behavior problems that are apparent in preschoolers or young children may be better attributed to either the negative relationships between the adult and the child or a lack of fit with their environment, rather than attributing these problems to the child themselves (Egger & Angold, 2006). This leads to the hypothesis that by changing the relationships between the child and important adults in their lives (e.g., teachers, caregivers), as well as changing peer interactions and the classroom environment, it may be possible to alter the child’s behavior and negative developmental course. To be able to influence change in adult-child interactions, it is first important to gain an understanding of the important principles within the development of such relationships. Within the neurorelational framework for considering parent-child relationships proposed by Lillas and Turnbull (2009), the authors describe six important social-emotional developmental milestones within the field in the development of a foundation for social-emotional health within these 47 relationships, including: “1) attention and regulation, 2) mutual engagement and attachment, 3) purposeful two-way communication, 4) complex gestures and problem solving, 5) use of symbols to express thoughts and feelings, and 6) bridging emotional themes” (p. 73-74). Each of these developmental milestones play an important role in creating a healthy parent-child relationship, as well as developing healthy future relationships with others (e.g., teachers, peers) (Lillas & Turnbull, 2009). For example, a positive attachment to the parent can allow for the child to develop the ability to feel confident in cooperating and interacting with other individuals in future interactions. Additionally, the third milestone involving “purposeful two-way communication” can set the stage for the infant to learn how to interact with another person and demonstrate appropriate ways for initiating communication or play (Lillas & Turnbull, 2009). If the parent and child do not experience positive interactions within this developmental milestone, the child may demonstrate difficulty in appropriately initiating play or engaging in interactions with others. Some possible negative outcomes associated with difficult parent-child interactions within the other social-emotional milestones include the child demonstrating anxious behaviors during play, the child presenting with a lack of enjoyment while playing with others, or the child demonstrating a lack of confidence during play resulting in avoidance or withdrawn behaviors (Lillas & Turnbull, 2009). The parent-child relationship is an important relationship in which to help the child to develop their emotions and learn how to develop relationships with others (Lillas & Turnbull, 2009). In addition, a parent’s ability to provide an appropriate balance between responding to their child and creating a directive and predictable environment within this relationship can provide a child with the foundation to be able to develop an understanding of the self and gain better control over their emotions and stress. If a child has a history of safe, reliable, and 48 positive relationships, then the child will learn that they can rely on these similar experiences in the development of their relationships with others (e.g., teachers, peers) and will be more likely “to develop emotional balance and to learn to thrive within primary relationships” (Lillas & Turnbull, 2009, p. 257). While the parent-child relationship is fundamental during the early years, the importance of the teacher-child relationship has been identified in a variety of studies as well (Baker, Grant, & Morlock, 2008; O’Connor & McCartney, 2006; Zhang & Sun, 2011). The role of the preschool teacher is especially important because this figure is often one of the most important adult figures in the child’s life other than the parents, and this teacher often serves the role of providing both academic and social-emotional support in order to help to ease the transition into school (Dobbs & Arnold, 2009). Research has identified the importance of the quality of the teacher-child relationship in shaping the social-emotional, academic, and intellectual development of children, highlighting the fact that a positive teacher-child relationship can serve as a protective factor or mediator between a poor maternal attachment and potential negative outcomes associated with this poor relationship (O’Connor & McCartney, 2006; Zhang & Sun, 2011). Based on a review of the literature, O’Connor and McCartney (2006) highlight the various factors that have been found to play a role in affecting teacher-child relationships. These include maternal attachment, the quality of previous teacher-child relationships, the amount of time spent in school or daycare in order to build positive relationships, the teachers’ level of education, the teachers’ level of experience, the child’s gender or race/ethnicity, child behavior problems, reading level, and family variables such as socioeconomic status or level of parental education. Across three time points within this study, children with insecure or avoidant attachment relationships with their mothers were found to have lower quality relationships with 49 teachers in childcare, kindergarten, and first grade. Higher quality relationships were noted for girls than boys, and lower relationships were noted between teachers and children with behavior problems (O’Conner & McCartney, 2006). In addition, Hamre and Pianta (2001) highlight the fact that poor teacher-child relationships in the early schools years, defined as those involving high levels of overdependence and conflict, were associated with negative academic and behavioral outcomes in early elementary school grades and continued to predict negative behavioral outcomes through eighth grade. These results were mediated by the child’s early school outcomes, and the results were especially negative for those students demonstrating early and significant behavior problems (Hamre & Pianta, 2001). This demonstrates the long-term effects that this early adult-child interaction can have on a child’s developmental trajectory. A positive teacher-child relationship appears to be especially important for children demonstrating externalizing and internalizing behavior problems, as these children are at greater risk for academic and social-emotional difficulties (Baker, Grant, & Morlock, 2008). Baker, Grant, and Morlock (2008) conducted a study with 423 children between kindergarten and fifth grade in order to determine the relationship between externalizing problems, internalizing problems, closeness and conflict in the teacher-student relationship, and school adjustment. Results indicated that children who had relationships with their teachers built around warmth and trust demonstrated better school adjustment, whereas relationships involving conflict were associated with worse school adjustment. Relationships defined by closeness and warmth served as a protective factor for children with externalizing behavior problems, and these children demonstrated higher reading achievement than those with conflicting relationships (Baker, Grant, & Morlock, 2008). This is similar to the findings by Hamre and Pianta (2001) 50 demonstrating that children with early behavior problems and positive teacher-child relationships had better academic and behavioral outcomes later on than those with a negative teacher-child relationship. Conflict within the teacher-child relationship appeared to have an especially negative effect on children with internalizing behavior problems, and these children demonstrated difficulty with work habits and adaptability (Baker, Grant, & Morlock, 2008). In another study examining the teacher-child relationship, Henricsson and Rydell (2004) found that children with internalizing behaviors had relationships with teachers that consisted of more dependence, more conflicts, and lower levels of closeness than children without internalizing behavior problems. These children demonstrated lower social competence scores, especially in the area of social initiative, which the authors attribute to the fact that they demonstrated less “social visibility” (Henricsson & Rydell, 2004, p. 133). Interestingly, children with internalizing behaviors that did demonstrate higher social initiative scores also had higher scores for conflict in the teacher-child relationship. Zhang and Sun (2011) highlight the importance of examining the directional influence of the child’s behavior and the teacher-child relationship in order to understand whether these variables represent a reciprocal interaction. In examining the internalizing and externalizing behavior problems of Chinese preschoolers in relation to the teacher-child relationship (e.g., closeness and conflict), these researchers found that a bidirectional relationship existed between the child’s externalizing behaviors and a teacher-child relationship consisting of negative conflict. In other words, the presence of child externalizing behaviors at the beginning of the preschool year was predictive of a teacher-child relationship defined by conflict at the end of the preschool year, and a relationship filled with conflict at the beginning of preschool predicted higher externalizing behavior problems for children at the end of preschool. On the other hand, 51 the relationship between child internalizing behaviors and teacher-child conflict was represented by a unidirectional relationship, indicating that the presence of internalizing behavior problems at the beginning of preschool were predictive of teacher-child relationship conflict by the end of the preschool year (Zhang & Sun, 2011). These findings highlight the importance of the teacherchild relationship in influencing the child’s externalizing and internalizing behaviors in preschool. This reciprocal interaction is especially important to be aware of due to the fact that research has shown that a teacher’s perception of a child can greatly influence how they behave towards that child and how they view that child’s future behaviors (Dobbs & Arnold, 2009). This potential bias could serve to exacerbate the negative relationship cycle between the teacher and child. Social competence and the presence of externalizing and internalizing behavior problems may be influenced by more variables than just the teacher-child relationship, as the socialemotional climate of the early childhood classroom can affect the child’s development as well (Howes, 2000). In a longitudinal study examining the predictive power of preschool variables in determining second grade social competence with peers, Howes (2000) determined that social competence in second grade was predicted by a number of variables including preschool behavior problem ratings, preschool teacher-child relationship, second grade teacher-child relationship, and the preschool social-emotional climate. In terms of children demonstrating risk or presence of internalizing symptoms, the author found that children in second grade with socially withdrawn behaviors tended to have low behavior problem ratings in preschool and were in a preschool classroom that had many behavior problems and a poor social-emotional climate. In addition, these students were engaged in teacher-child relationships defined by a lack of closeness and high levels of conflict (Howes, 2000). 52 The Incredible Years Program is one training series that includes two training approaches focused on changing adult behavior in order to facilitate changes in the adult-child relationship, the social-emotional climate of the classroom, and the child behavioral functioning, as well as one training approach focused on intervening directly at the child-level. These training series are explored in further detail within the following sections. The Incredible Years Series The Incredible Years (IY) Program is an evidence-based social-emotional prevention curriculum consisting of three different training programs aimed at addressing children’s behavior problems and functioning across multiple settings (Webster-Stratton & Reid, 2010). The three programs consist of the Dinosaur Social Skills and Problem-Solving Child Curriculum Program, the Parent Group Training Series, and the Teacher Classroom Management Training Program (Webster-Stratton, 2012). The goals of this program include the reduction of disruptive behavior problems, the development of appropriate problem solving and social skills, and the promotion of social competence in childhood populations (Webster-Stratton, 2008). The Incredible Years Program has been well researched, has demonstrated potentially positive effects according to the What Works Clearinghouse (Institute of Education Sciences, n.d.), and has been rated as an effective and model prevention and early intervention program by the Office of Juvenile Justice and Delinquency Prevention Blueprint Program (U.S. Department of Justice Office of Justice Programs, n.d.). The research has consistently demonstrated that the IY series demonstrates efficacy in improving children’s social competence and reducing the severity of symptoms related to disruptive behaviors including CD and ODD (Webster-Stratton, Reid, & Hammond, 2001a; Webster-Stratton, Reid, & Hammond, 2004). In addition to positive changes in child behaviors, research utilizing the parenting and teacher training programs in 53 combination with the other program series have demonstrated significant improvements in parents’ use of positive parenting practices and discipline strategies, teachers’ use of positive classroom management strategies, the positive social-emotional climate of the classroom, and improvements in adult-child interactions across both contexts (Reid, Webster-Stratton, & Beauchaine, 2001; Webster-Stratton, Reid, & Hammond, 2001a; Webster-Stratton, Reid, & Hammond, 2001b; Webster-Stratton, Reid, & Hammond, 2004; Webster-Stratton, Reid, & Stoolmiller, 2008). Incredible Years Teacher Classroom Management (IY-TCM) Training Program The Incredible Years Teacher Classroom Management (IY-TCM) Program is one component of the Incredible Years series that is offered in a group format for teachers (WebsterStratton & Herman, 2010). This program focuses on learning effective classroom management strategies in order to change the relationships and interactions that teachers have with children in their classroom by learning ways to handle misbehavior, improving teacher-child positive interactions and relationships, and modeling and promoting the use of appropriate problem solving skills, friendship skills, and social skills between the children in the classroom. This program also includes the goal of helping children to increase their problem solving skills in order to deal with peer rejection (Webster-Stratton, Reid, & Hammond, 2001b). In addition, the program aims to strengthen the collaboration and communication between home and school environments (Webster-Stratton & Herman, 2010). The IY-TCM Program is based on the idea that making changes in a teacher’s use of strategies for classroom management and promoting the development of prosocial behaviors and skills in the classroom will have short and long term effects on the classroom environment as well as the academic and behavioral development of the students in the class (Webster-Stratton, 54 2012). As described within the IY Logic model, the IY-TCM program proposes that group training focused on changing the teacher’s behavior and use of strategies, classroom management skills, relationships and interactions with students, and use of modeling and coaching of students’ problem-solving and prosocial skills will affect change in the classroom context and improve the students’ social-emotional and behavioral functioning. This link between teachers’ behavior and strategy use and child outcomes has been demonstrated in various studies (e.g., Shernoff & Kratochwill, 2007), including procedures utilizing consultation models in which teachers are instructed in how to implement intervention strategies in order to change child behaviors (Dunson III, Hughes, & Jackson, 1994). One study examining the effect of preschool teacher-child relationships on the behavioral and academic outcomes of Portuguese students one year later found that more positive teacher-student relationships and emotional, organizational, and instructional classroom environments were associated with greater academic outcomes for children (Cadima, Leal, & Burchinal, 2010). In addition, negative teacher-child relationships have been associated with a continued pattern of negative interactions with teachers, poor behavioral outcomes, and poor academic outcomes (Hamre & Pianta, 2001; Sutherland, Lewis-Palmer, Stichter, & Morgan, 2008). Higher teacher effectiveness in delivering instructional material in Kindergarten has also been linked with greater student social and behavioral skills (Jennings & DiPrete, 2010). Sutherland and colleagues (2008) suggest that interventions should be focused on not only changing student behavior but also targeting teacher behaviors in order to affect change for more students, specifically in the areas of classroom management, attention, and positive reinforcement. Several studies have been conducted that have included the IY teacher training component, either in isolation or in combination with the other IY child and parent training 55 programs. Three randomized controlled trials (RCTs) have been conducted with the IY teacher training programs by the program developers in combination with other IY training components (Reinke, Stormont, Webster-Stratton, Newcomer, & Herman, 2012). Webster-Stratton, Reid, and Hammond (2004) included the IY teacher training program as an additional component along with the IY Child and Parent Training Programs, which have received much empirical examination and support within the research. The authors highlighted the importance of adding a teacher training component to these interventions in order to increase the effectiveness of the treatment modalities, as children’s behavior problems typically exist across multiple contexts and many of these behaviors occur within the school settings in which children spend most of their time and in which there may be poor teacher-student relationships (Webster-Stratton, Reid, & Hammond, 2004). This study examined the outcomes of various treatment combinations with 159 children between the ages of four and eight years old diagnosed with Oppositional Defiant Disorder (ODD) and seeking treatment at the University of Washington Parenting Clinic. The children and their families were randomly assigned to one of six different conditions, including: parent training, child training, parent training and teacher training, child training and teacher training, parent and child and teacher training, or a wait list group. Results indicated that the treatment models including the teacher training components demonstrated significant improvements in teachers’ behavior towards and interactions with children, a significant reduction in children’s negative externalizing behaviors at school, and improvements in the teachers’ classroom management strategies (Webster-Stratton, Reid, & Hammond, 2004). In addition, the child training and the child and teacher training models also demonstrated significant effects across contexts in strengthening and improving parent-child interactions, suggesting that these treatments may transport across settings (Webster-Stratton, 56 Reid, & Hammond, 2004). Improvements in teacher and peer relationships were also found for those teachers participating in the training. Another study examined a combined prevention treatment utilizing the Incredible Years Parent and Teacher Training Programs with 272 mothers and 61 teachers of children in Head Start programs (Webster-Stratton, Reid, & Hammond, 2001a). In this study, parents and teachers of the Head Start children were randomly assigned to the IY treatment condition (parent and teacher training) or a control condition. Significant differences were found between treatment and control conditions for reductions in negative parenting, improvements in positive parenting, improvements in parent-teacher relationships, reductions in conduct problems at school and at home, and more effective teacher classroom management strategies. Webster-Stratton, Reid, and Stoolmiller (2008) conducted a study in which they examined the use of the IY Teacher Classroom Management Program (IY-TCM) and the Dina Dinosaur Child Social, Emotional, and Problem-Solving Curriculum as a universal prevention program for children within Head Start, Kindergarten, and first grade in high poverty areas. Teachers were provided training in how to implement the Dinosaur School Program in their classrooms, as well training in effective classroom management strategies in order to enhance prosocial behaviors, reduce conduct problems, and improve emotional awareness and problem solving. Results indicated that teachers that received the teacher training and implemented the Dinosaur School in biweekly lessons in their classrooms demonstrated improvements in their use of positive management strategies, showed significant improvements in their affectionate behaviors and use of social and emotional teaching strategies, and demonstrated significant reductions in their harsh, critical, and inconsistent behaviors. Students of teachers that received the training demonstrated increased social competence, emotional regulation, number of positive 57 feelings identified, and a greater reduction in conduct and behavior problems than those in the control condition. In addition, results indicated a more positive classroom atmosphere at posttreatment for the intervention classroom than the control classrooms, with a strong effect size of 1.03 (Webster-Stratton, Reid, & Stoolmiller, 2008). A fourth study examining the use of IY parent training and an adapted version of the IYTCM program as a consultation model of service delivery was implemented with teachers and parents of 103 Head Start preschoolers in order to address disruptive behavior problems (Williford & Shelton, 2008). Teachers within the intervention group attended one group training session and then at least four months of weekly individual consultation with the group leader, with consultation sessions focused around the teachers’ specific needs. Parents within the intervention group attended ten weeks of group training. Results indicated that teachers within the intervention group reported more use of effective teaching strategies, and parents within the intervention group reported more effective parenting skills. Teachers within the intervention group reported that the disruptive behaviors of their students remained consistent across the year, whereas the comparison group demonstrated more disruptive behaviors (Williford & Shelton, 2008). However, when examining whether children indicated a change of at least 1 SD in one behavior area, teachers within the experimental group reported that 55% of the children improved compared to only 30% of the comparison group. In addition to these four studies, six other studies have been conducted by independent researchers exploring the IY teacher training program (Reinke, Stormont, Webster-Stratton, Newcomer, & Herman, 2012). Hutchings and colleagues (2007) explored the implementation of the IY-TCM program in North West Wales with 23 teachers receiving one day of training per month over a five month period. In the first part of the study assessing teacher satisfaction with 58 the IY-TCM program, results showed that teachers indicated satisfaction with the program and found that it was easy to implement. Qualitative interviews with the teachers demonstrated that the teachers found the group training to be valuable because they were able to share ideas and strategies with other teachers and were able to problem solve together. Teachers also reported that their students were more confident as a result of this training, paid more attention in class, were more responsible, and were more considerate of others. As part of the second study, observations indicated that teachers used significantly more clear and direct commands and allowed more time for compliance. In addition, children demonstrated significantly more positive behaviors, including positive language and compliance (Hutchings et al., 2007). Raver and colleagues (2008) investigated the use of IY-TCM program in combination with weekly one-on-one consultation with 94 teachers in Head Start programs with the goal of increasing the positive emotional climate of classrooms, decreasing disruptive behaviors, and increasing school readiness. Results showed that significant differences were found between intervention teachers and control teachers in building a positive emotional climate (d = 0.89), with intervention teachers demonstrating more positive behavior and enthusiasm, paying more attention to their students, and reducing their use of harsh and negative statements. In addition, teachers within the intervention group demonstrated marginal improvements in teacher sensitivity (d = 0.53) and classroom management techniques used (d = 0.52). Another study examined the use of the IY-TCM training with monthly consultation sessions in combination with the implementation of the Dina Dinosaur Classroom Curriculum in five preschool classrooms in Jamaica (Baker-Henningham, Walker, Powell, & Meeks Gardner, 2009). Teachers within the intervention classrooms attended seven full day training sessions once per month. In addition, the intervention groups received 14 classroom lessons using the IY 59 child social-emotional curriculum. Significant differences were found between the intervention and control groups favoring the TCM and child curriculum programs, including a significant increase in positive teacher behaviors, frequency with which social and emotional skills were reinforced, and reductions in negative teacher behaviors. In addition, students within the intervention classrooms demonstrated more appropriate behaviors at posttreatment than control groups (Baker-Henningham, Walker, Powell, & Meeks Gardner, 2009). Similarly, Shernoff and Kratochwill (2007) examined the use of consultation in combination with the TCM program with eight teachers randomly assigned to conditions in order to assess outcomes related to classroom management skills, teacher acceptability, reduction in child disruptive behaviors in the classroom, and improvements in social competence. These researchers examined the use of the IY self-administered teacher training model over five weeks in isolation compared to the self-administered training with 45 to 60 minute consultation sessions. Results indicated that students of teachers engaging in the self-administered plus consultation conditions exhibited significant increases in social competence compared to students in the other condition (Shernoff & Kratochwill, 2007). Students within the selfadministration plus consultation classrooms were more likely to demonstrate significant positive scores on the Reliable Change Index (RCI) for social competence compared to the other condition, indicating that these students demonstrated significant improvements from screening to post-intervention measures. Students within both conditions demonstrated decreases in disruptive externalizing behaviors, with no significant differences between conditions. Carlson, Tiret, Bender, and Benson (2011) investigated changes in teacher perceptions and frequency of use of classroom management strategies in relation to the IY-TCM program with 24 preschool teachers in a low-income area in Michigan. Teachers participated in eight 60 group training sessions, lasting four hours one night per week over a span of eight to ten weeks, and data was collected at baseline, posttreatment, and 16 week follow-up. Results demonstrated significant differences from pretest to post-test for teacher-reported use of positive strategies, usefulness of these positive strategies, as well as perceptions and use of proactive classroom management strategies (Carlson, Tiret, Bender, & Benson, 2011). This study promoted the usefulness and acceptability of this evidence based intervention for teachers, as well as the promotion of a group training method that was a better fit with realistic daily schedules. Snyder and colleagues (2011) further examined the effectiveness of the IY teacher training program through a RCT with 28 Head Start teachers assigned to either a regular (nonIY) teacher training or a brief adaptation of the IY teacher training program. The adapted IY teacher training program consisted of five group training sessions lasting three hours each, a focus on the Acceptance and Commitment (ACT) framework in order to increase teachers’ involvement in developing skills, and three classroom consultation sessions per teacher for 45 minutes. This study examined the influence of changing teachers’ classroom management skills in order to change the teachers’ behavior and peers’ behaviors towards target children. Each teacher selected five target children to focus on, three of which demonstrated high conduct problems (high CD) and two of which demonstrated low conduct problems (low CD). Results of this study indicated marginally significant differences between the IY and nonIY groups for increases in teachers’ and target students’ positive behaviors (Snyder et al., 2011). Significant differences were found between the groups in the negative behaviors of the target students, peers, and teachers, indicating a decrease in this area for the IY group and an increase in negative behaviors for the non-IY group. The results also indicated that the classrooms participating in the teacher training demonstrated decreases in the peers’ negative behaviors and 61 dislike towards the target students with high CD, as well as decreases in the peers’ ignoring behavior towards target students with low CD (Snyder et al., 2011). This finding is encouraging, as it highlights the relationship between the teacher training and an improved classroom environment, and the results indicated less social isolation and rejection for children with high CD and children with low CD who may have previously been ignored. Application of the Incredible Years Series to Internalizing Symptoms While the Incredible Years Child, Parent, and Teacher Training Programs have been well-researched and have consistently demonstrated positive effects for parenting skills, teaching skills, adult-child relationships, and child behaviors, these programs have primarily targeted children with conduct and externalizing behavior problems (Herman, Borden, Reinke, & Webster-Stratton, 2011; Webster-Stratton, 2008). However, attention has begun to shift towards identifying outcomes related to internalizing behaviors when using the Incredible Years training programs. Previous research has led some to hypothesize that individuals with disruptive behaviors and internalizing behaviors may respond similarly to similar intervention techniques and methods, as risk factors are often related (Herman, Borden, Reinke, & Webster-Stratton, 2011). More specifically, some authors suggest that the Incredible Years Parent, Teacher, and Child training programs contain many of the intervention methods that other interventions for internalizing behaviors contain, addressing important risk factors for internalizing behaviors such as unstable environments, inappropriate and negative adult-child relationships, and poor social skill development (Herman, Borden, Reinke, & Webster-Stratton, 2011). In addition, the Incredible Years Program addresses a variety of skills domains that are applicable to both externalizing and internalizing problems, including problem solving skills, emotional awareness skills, and friendship skills. 62 Researchers suggest that the components of the Incredible Years programs allow the therapist an opportunity to target other internalizing symptoms as well, such as social isolation, social withdrawal, and social skill deficits (Webster-Stratton & Herman, 2008). Previous research conducted with the Incredible Years intervention program have indicated that comorbidity of anxiety or depressive symptoms with conduct problems may be predictive of significant improvement outcomes (Beauchaine, Webster-Stratton, & Reid, 2005), suggesting that the program may also address these internalizing symptoms. These authors suggest that children that are exhibiting high levels of anxiety may respond well to the use of social attention and rewards in the CT program, which may lead to greater improvements in behavior and functioning. Several research studies have also examined the effect of the IY programs on reductions in internalizing symptoms as a secondary outcome to externalizing symptoms. For example, in one study examining the use of the Self-Administered Incredible Years Parent Training Program for parents of six to nine-year-old children with Attention-Deficit Hyperactivity Disorder (ADHD), Ogg and Carlson (2009) found that parents’ perception of child internalizing symptoms, included as a secondary outcome, significantly improved from pretest to post-test. On the other hand, parents did not report a significant decrease in the primary outcome symptoms of ADHD. The IY Child Training Series has also demonstrated promising results for internalizing symptoms. For example, Barrera and colleagues (2002) examined the use of the IY Parenting Program, the Dina Dinosaur Social Skills Program, and an additional social skills intervention for children in kindergarten through third grade exhibiting problems with aggression or reading. The results of this study indicated positive improvements in observed externalizing problems and ratings of internalizing behaviors of the children. These research studies provide 63 support for the need for further exploration of each of the IY training series for internalizing symptoms as a primary outcome and focus for early intervention and prevention efforts. Herman, Borden, Reinke, and Webster-Stratton (2011) examined the influence that the Incredible Years Programs can have on the internalizing behaviors of children by closely and exclusively examining the secondary data relating to internalizing behaviors gathered from previous data collection procedures. These authors examined the various Incredible Years intervention programs and their influence on building social skills and improving internalizing symptoms. This study included 159 families of four to eight-year-old children that were referred to the University of Washington Parenting Clinic for issues relating to conduct problems and who met criteria for a DSM diagnosis of Oppositional Defiant Disorder (ODD), were in preschool through second grade, and whose parents reported at least ten behavior problems for the child. Families were randomly assigned to one of six conditions: Child Treatment (CT) only, CT plus Teacher Training (TT), CT plus TT plus Parent Training (PT), PT only, PT and TT, or a wait list group. The CT groups received the Dinosaur School intervention during two hour sessions for 18 to 19 weeks (Herman et al., 2011). Teachers within the teacher training (TT) treatment groups attended four full day training sessions (32 hours total), while parents within the PT groups attended group training sessions for weekly two hour sessions for 22 to 24 total weeks. The internalizing scale of the Child Behavior Checklist (CBCL) and the Anxious/Depressed subscale were used in order to collect maternal ratings of symptoms at preand post-test (Herman et al., 2011). Results at post-test indicated that children in the treatment groups demonstrated significantly lower ratings on internalizing symptoms than the control group (d = .44), with children presenting with higher internalizing behaviors at baseline demonstrating significantly 64 greater improvement (Herman et al., 2011). The CT + PT + TT combined treatment group (d = .64) demonstrated internalizing scores that were six points lower than the control group and four points lower than the CT only or PT only groups (ds = .41 and .42). The PT and TT combined group demonstrated that the intervention had a moderate effect (d = .44), but the difference was not statistically significant from the control group. When examining the Reliable Change Index (RCI) of 13 points, 24% of the children in the treatment groups showed an improvement in their internalizing symptoms, and 39% of the children with high internalizing baseline scores showed improvements at the post-test (Herman et al., 2011). All of the treatment groups maintained their improved scores at the 12 month follow-up. This study provides promising results for the influence of the Incredible Years programs on the reduction of internalizing symptoms (Herman et al., 2011; Webster-Stratton & Herman, 2008). These authors support the continued exploration of the use of the Incredible Years program as a targeted group intervention for children that are demonstrating risk for developing anxiety or depressive symptoms, with the goal of serving as a way to prevent serious future psychopathology (Herman et al., 2011; Webster-Stratton & Herman, 2008). Research Questions and Hypotheses The research questions examined within this study were as follows: Question 1: Will there be a difference in the procedural integrity of treatment adherence between the two groups, as measured through the teacher-reported log of hours spent on professional development activities for classroom management strategies? This variable was examined first within this study due to the importance of determining that the treatment groups were carried out as intended prior to discussion of the results related to those groups. It was hypothesized that teachers within the IY-TCM group would spend more 65 time engaged in professional development activities that include the IY-TCM training and outside planning for the use of IY-TCM classroom management strategies than the teachers in the other bibliotherapy condition. Question 2: Will there be a difference in the mean scores over time for the teachers’ use and perceptions of classroom management strategies (i.e., Confidence in Managing Classroom Behavior, Total Positive Strategies, Inappropriate Strategies, Planning and Support Strategies, and Positive Approaches with Parents) for the teachers across the two group conditions from pretest to the midpoint to post-test? Research has demonstrated positive results regarding the teachers’ positive classroom management strategies from pretest to post-test after receiving the IY-TCM training (e.g., Webster-Stratton, Reid, & Hammond, 2004). Carlson, Tiret, Bender, and Benson (2011) found significant improvements in teachers’ reported use and perceptions of positive classroom management strategies from pretest to post-test, however, no significant differences were found for inappropriate strategies and positive approaches with parents. However, other studies have found group differences for IY-TCM groups and comparison groups for positive classroom management strategies and decreases in negative teaching strategies (Webster-Stratton, Reid, & Hammond, 2001a), while Shernoff and Kratochwill (2007) found that teachers reported increases in their confidence in managing classroom behaviors. It was hypothesized that this study would also find results indicating that teachers within the IY-TCM group would demonstrate significant increases in their use and perceptions of usefulness of the positive classroom management strategies, significant increases in their confidence in managing classroom behavior, and significant decreases in the use and perceptions of inappropriate classroom management strategies. In addition, it was hypothesized that teachers within the IY-TCM group would 66 demonstrate significant increases in the use and perceptions of planning and support strategies and positive approaches with parents. Research regarding the use of bibliotherapy as a treatment has indicated limited support and mixed results (Gavigan, Kurtts, & Mimms, 2010). However, articles focusing on the use of bibliotherapy with teachers indicate that this method can help teachers become aware of and reflect on their personal professional behaviors and techniques (Morawski, 1997). This method has also been described as a way to provide teachers with content knowledge, ways to solve problems, and a chance to reflect on how they view their teaching and alter their beliefs (Wilson & Thornton, 2007). For the current study, it was hypothesized that the teachers within the bibliotherapy group would demonstrate some improvements in their use of classroom management strategies, but that this would not be as significant as the IY-TCM group due to the lack of group discussion, video vignettes, role playing, and modeling, which are key components of change used within the IY-TCM program (Webster-Stratton & Reid, 2010). However, it was hypothesized that bibliotherapy teachers would demonstrate greater improvements in their perceptions of usefulness of the positive classroom management strategies and perceptions of inappropriate classroom management strategies. Question 3: Will there be a difference in the mean scores over time for the classroom atmosphere, percentage of time in which teachers are involved in positive interactions with target students based on Direct Behavior Rating (DBR) data, and the teacher-student relationships on the STRS for teachers across the two group conditions from pretest to the midpoint to post-test? Raver and colleagues (2008) found that classrooms in which teachers participated in the IY-TCM training and consultation showed significantly higher positive emotional climate from 67 pretest to post-test than teachers within the control group. It was hypothesized that classrooms assigned to the IY-TCM group would demonstrate significant improvements in classroom atmosphere from pretest to the midpoint to post-test. It was also hypothesized that the bibliotherapy group would demonstrate small improvements in classroom atmosphere, but not significant differences. Snyder and colleagues (2011) found that teachers included within the teacher training group demonstrated improvements in teachers’ positive behaviors and decreases in their negative behaviors towards target students. It was hypothesized that there would be significantly higher scores and improvements in the percentage of time that target students were involved in positive interactions with teachers and improvements in the teacher-student relationships for the IY-TCM group, and that the bibliotherapy group would demonstrate small improvements in teacher interactions and teacher-student relationships, but not significant differences. Question 4: Will there be a difference in the mean scores for the behavior of the entire class of students and the target students in particular across the two group conditions over time? Question 4a: Will there be a difference in the mean scores over time for the internalizing scores for all classroom students through teacher ratings from pretest to post-test across the two group conditions? Several research studies with the IY programs have demonstrated improvements in secondary internalizing symptoms for children (e.g., Barrera et al., 2002; Ogg & Carlson, 2009). It was hypothesized that there would be significant group differences, with the students within the IY-TCM groups demonstrated significant improvements in internalizing scores from pretest to post-test, and that the students within the bibliotherapy group would demonstrate some improvements but not reaching significance. 68 Question 4b: Will there be a difference in the mean scores over time for the externalizing scores for all classroom students through teacher ratings from pretest to post-test across the two group conditions? Research regarding the IY programs have consistently demonstrated decreases in child externalizing behaviors and problems from pretest to post-test after receiving these evidencebased parent, teacher, and/or child training programs (e.g., Webster-Stratton, Reid, & Hammond, 2004; Webster-Stratton, Reid, & Stoolmiller, 2008). It was hypothesized that there would be significant group differences in favor of the IY-TCM group, with students within the IY-TCM group demonstrating reductions in externalizing scores from pretest to post-test, and the students within the bibliotherapy group demonstrating small but nonsignificant reductions in externalizing scores. Question 4c: Will there be a difference in the mean scores over time for the social skills scores on teacher ratings for all classroom students from pretest to post-test across the two group conditions? Studies utilizing the IY programs have demonstrated improvements in children’s socialemotional competence and social skills for those included within the IY conditions. Shernoff and Kratochwill (2007) found that students demonstrated significant improvements in social competence from pretest to post-test. Webster-Stratton, Reid, and Stoolmiller (2008) found that children within the IY-TCM group demonstrated improvements in social competence and emotional knowledge. It was hypothesized that students within the IY-TCM classrooms would demonstrate significant increases in teacher ratings of their social skills from pretest to post-test, whereas students within the bibliotherapy group would demonstrate slight, but nonsignificant, improvements. 69 Question 4d: Will there be a difference in the mean scores over time for the target students’ internalizing scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions? In addition to findings of improvements in secondary internalizing outcomes in several studies (e.g., Ogg & Carlson, 2009), Herman, Borden, Reinke, and Webster-Stratton (2011) found that children with higher internalizing scores at pretest demonstrated significantly greater improvements in their internalizing symptoms over time. It was hypothesized that target students within the IY-TCM groups would demonstrate reductions in internalizing scores from pretest to the midpoint to post-test that are significantly difference from the bibliotherapy group, and that target students within the bibliotherapy group would demonstrate small improvements. Question 4e: Will there be a difference in the mean scores over time for the target students’ externalizing scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions? Studies conducted with the IY programs have consistently indicated decreases in student externalizing behavior problems from pretest to post-test (e.g., Webster-Stratton, Reid, & Hammond, 2004) and specifically for target students demonstrating high and clinically significant externalizing behaviors at pretest (Shernoff & Kratochwill, 2007). It was hypothesized that target students within the IY-TCM group would demonstrate significantly different scores and reductions in externalizing behaviors, whereas those within the bibliotherapy group would demonstrate minimal reductions from pretest to midpoint to post-test. Question 4f: Will there be a difference in the mean scores over time for the target students’ social skill scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions? 70 As mentioned above, research conducted with the IY programs have demonstrated significant improvements in social competence scores for students and the emotional knowledge of the classroom students (Shernoff & Kratochwill, 2007; Webster-Stratton, Reid, & Stoolmiller, 2008). It was hypothesized that target students within the IY-TCM classrooms would demonstrate significant differences from the other group and would show improvements in teacher and parent ratings of their social skills from pretest to the midpoint to post-test. It was also hypothesized that target students within the bibliotherapy group would demonstrate small improvements, but that these improvements would not reach significance. Question 4g: Will there be a difference in the mean scores over time for the percentage of time that target students demonstrate positive social behaviors across the two group conditions from pretest to the midpoint to post-test, as measured by the DBR observations? Snyder and colleagues (2011) found increases in positive behaviors demonstrated by target students from pretest to post-test for children within the teacher training groups, however this was not found for the comparison group. Additionally, Hutchings and colleagues (2007) found that children demonstrated improvements in their positive behaviors from pretest to posttest. It was hypothesized that target students within the IY-TCM group would demonstrate significant improvements in their positive social behaviors from pretest to the midpoint to posttest. It was also hypothesized that target students within the bibliotherapy group would demonstrate improvements, but these would not be significant. Question 4h: Will there be a difference in the mean scores for the percentage of time in which target students are involved in positive interactions with peers from pretest to the midpoint to post-test across the two group conditions? 71 Snyder and colleagues (2011) indicated that students within the IY-TCM group engaged in more prosocial behaviors towards peers and demonstrated decreases in peers’ negative behaviors, rejection, and ignoring towards target students with high conduct problems and low conduct problems. It was hypothesized that target students within the IY-TCM group would demonstrate significant improvements in their peer interactions, whereas the target students within the bibliotherapy group would demonstrate nonsignificant improvements. Question 5: Will there be a difference in the ratings of acceptability of the treatment between the two conditions (IY-TCM and bibliotherapy) at post-test? Shernoff and Kratochwill (2007) found that teachers within the IY video modeling and consultation group rated the program significantly higher than the IY video modeling group, suggesting that the more intensive and involved training may have influenced the rating of acceptability. Research related to treatment acceptability and treatment outcomes have supported the important role that acceptability can play in relation to effectiveness of the treatment, treatment integrity, and behavioral change (Eckert & Hintze, 2000; Kazdin, 2000). It was hypothesized that teachers within the IY-TCM group would report significantly higher acceptability ratings than the teachers in the bibliotherapy comparison group. 72 CHAPTER 3: METHODS Design For the current research study, a two-group experimental design was used in which teachers and their students were randomly assigned to either an IY-TCM treatment group or a bibliotherapy/reading comparison group. The IY-TCM treatment group received the IY-TCM group training intervention and the bibliotherapy comparison group received IY reading materials. Tables 2 and 3 provide a visual display for the sequence of data collection procedures, research questions, measures, and data analyses. This two group design was used in order to provide a comparison between the active IYTCM treatment group receiving the evidence-based intervention as intended and a comparison group. The bibliotherapy comparison group served as a less intensive comparison group due to the fact that they received the Incredible Teachers: Nurturing Children’s Social, Emotional, and Academic Competence book, which can be ordered online by any individual, but did not engage in the important components of change highlighted within the IY program, such as group discussion, video vignettes, role playing, or modeling. In addition, this condition served as a comparison condition in which the amount of time the teachers spent on professional development or active learning related to classroom management strategies was measured and controlled, therefore better controlling for confounding variables of increased attention and time engaged in learning between the conditions. 73 Table 2 Research Questions, Assessment Procedures, and Data Analyses Question Measures and Constructs Question 1: Will there be a difference in the procedural integrity of treatment adherence between the two groups, as measured through the teacherreported log of hours spent on professional development activities for classroom management strategies? Question 2: Will there be a difference in the mean scores over time for the teachers’ use and perceptions of classroom management strategies (i.e., Confidence in Managing Classroom Behavior, Total Positive Strategies, Inappropriate Strategies, Planning and Support Strategies, and Positive Approaches with Parents) for the teachers across the two group conditions from pretest to the midpoint to post-test? Professional Development Log (time engaged in PD related to classroom management strategies) Teacher Strategies Questionnaire (TSQ) Five summary scales (Total Positive Strategy Frequency of Use Score, Total Positive Strategy Perception of Usefulness Score, Inappropriate Strategies Frequency of Use Score, Inappropriate Strategies Perception of Usefulness Score, Planning and Support Strategy Score, Confidence in Managing Classroom Behavior Score, Positive Approaches with Parents Score) Four subscales (Limit Setting Strategy Frequency of Use, Limit Setting Strategy Perception of Usefulness, SocialEmotional Strategy Frequency of Use, Social-Emotional Strategy Perception of Usefulness, Proactive Strategy Frequency of Use, Proactive Strategy Perception of 74 Treatment Phase Post-test Scores Used Data Analysis Pretest, midpoint, posttest Total summary Two-level scale and subscale hierarchical scores linear model Total raw score of Independent hours (training samples t-test and reading) Table 2 (cont’d) Question 3: Will there be a difference in the mean scores over time for the classroom atmosphere, percentage of time in which teachers are involved in positive interactions with target students based on Direct Behavior Rating (DBR) data, and the teacher-student relationships on the STRS for teachers across the two group conditions from pretest to the midpoint to post-test? Question 4: Will there be a difference in the mean scores for the behavior of the entire class of students and the target students in particular across the two group conditions over time? Usefulness, Coaching Strategy Frequency of Use, Coaching Strategy Perception of Usefulness) Pretest, Classroom Atmosphere Measure (disruptive behavior, transitions, rulemidpoint, postfollowing, cooperation, communication, test problem solving, expression of appropriate feelings, interest/enthusiasm/involvement, on-task behaviors, focus on individual needs, supportive of student efforts) Direct Behavior Ratings (DBR) – Teacher-Student Interactions (teacher directed praise, recognition, and support towards child) Pretest, midpoint, posttest Mean Raw Score Two-level hierarchical linear model Mean raw score Three-level hierarchical linear model Pretest, Student-Teacher Relationship Scale midpoint, post(STRS) (Conflict, Closeness, Dependency, and test Total relationship quality between student and teacher) Conflict Raw Score, Closeness Raw Score, Dependency Raw Score, STRS Total Score Pretest, midpoint (for target students), post-test Internalizing composite TScore, externalizing composite TScore, social skills subscale T- BASC-2 TRS-P (internalizing behaviors: anxiety, depression, somatization, externalizing behaviors: hyperactivity, aggression, conduct problems, social skills: social adaptation and interactions such as complimenting 75 Three-level hierarchical linear model (all classroom students) Table 2 (cont’d) Question 4a: Will there be a difference in the mean scores over time for the internalizing scores for all classroom students through teacher ratings from pretest to post-test across the two group conditions? Question 4b: Will there be a difference in the mean scores over time for the externalizing scores for all classroom students through teacher ratings from pretest to post-test across the two group conditions? Question 4c: Will there be a difference in the mean scores over time for the social skills scores on teacher ratings for all classroom students from pretest to post-test across the two group conditions? others, assisting others, encourages, helps others) Score BASC-2 PRS-P (internalizing behaviors, externalizing behaviors, social skills) Pretest, midpoint, posttest Internalizing composite TScore, externalizing composite TScore, social skills subscale TScore Direct Behavior Ratings (DBR) – Peer Interactions (positive and inviting peer interactions) Pretest, midpoint, posttest Mean Raw Score Direct Behavior Ratings (DBR) – Positive Social Behaviors (engaging in play or work with others, cooperative participation, negotiating and accepting suggestions during play) Pretest, midpoint, posttest Mean Raw Score Question 4d: Will there be a difference in the mean scores over time for the target students’ internalizing scores through parent and teacher ratings from pretest to the midpoint to post-test across the Three-level hierarchical linear model (target students) 76 Table 2 (cont’d) two group conditions? Question 4e: Will there be a difference in the mean scores over time for the target students’ externalizing scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions? Question 4f: Will there be a difference in the mean scores over time for the target students’ social skill scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions? Question 4g: Will there be a difference in the mean scores over time for the percentage of time that target students demonstrate positive social behaviors across the two group conditions from pretest to the midpoint to post-test, as measured by the DBR observations? 77 Table 2 (cont’d) Question 4h: Will there be a difference in the mean scores for the percentage of time in which target students are involved in positive interactions with peers from pretest to the midpoint to post-test across the two group conditions? Question 5: Will there be a difference in the ratings of acceptability of the treatment between the two conditions (IY-TCM and bibliotherapy) at post-test? Teacher Workshop Satisfaction Questionnaire (usefulness of specific strategies, overall acceptability of the program) 78 Post-test Total subscale raw scores Independent samples t-test Table 3 Data Collection Timeline Pretest All classrooms Teacher Strategy Questionnaire BASC-2, Teacher Rating Scale BASC-2, Parent Rating Scale (target students) County 1 Only Classroom Atmosphere Measure Direct Behavior Ratings (target students) Student Teacher Relationship Scale (selected students) Midpoint All classrooms Teacher Strategy Questionnaire BASC-2, Teacher Rating Scale (target students only) BASC-2, Parent Rating Scale (target students) County 1 Only Classroom Atmosphere Measure Direct Behavior Ratings (target students) Student Teacher Relationship Scale (selected students) Post-test All classrooms Teacher Strategy Questionnaire BASC-2, Teacher Rating Scale BASC-2, Parent Rating Scale (target students) Teacher Workshop Satisfaction Questionnaire Professional Development Log of Hours County 1 Only Classroom Atmosphere Measure Direct Behavior Ratings (target students) Student Teacher Relationship Scale (selected students) 79 The study design included a pretest-midpoint-posttest design, in which data was collected through teacher and parent rating scales and direct observations in order to measure whether there would be a change in the dependent variables. This study examined changes in teacher classroom management practices, classroom atmosphere, relationships, and student behaviors from the pretest data collection period to the post-test data collection period in the spring at the end of the school year. The intervention phase was designed to be conducted over a six month period (e.g., November through April). This timeline for data collection and intervention implementation was selected in accordance with previous research conducted using the IY-TCM training program, in which the intervention was delivered across periods ranging from approximately ten weeks (e.g., Carlson, Tiret, Bender, & Benson, 2011) to three or four months (e.g., Snyder et al., 2011; Williford & Shelton, 2008) to six months (e.g., Webster-Stratton, Reid, & Hammond, 2004), which is the format recommended by the program developers (WebsterStratton, 2012). The research on the IY-TCM program have found changes in teacher strategy use, classroom climate, relationships, and student social competence and behaviors within timeframes similar to this proposed study, typically collecting pretest data in the fall of the school year and post-test data in the spring of that school year ranging from a total six to nine month period (e.g., Raver et al., 2008; Shernoff & Kratochwill, 2007; Snyder et al., 2011), with some of the studies including a one year follow-up period (e.g., Webster-Stratton, Reid, & Hammond, 2004; Williford & Shelton, 2008). Therefore, it was expected that this research study would find changes in teacher, classroom, and student variables within this timeframe as well. Due to actual recruitment schedules, participants were recruited from two county areas according to different timelines. Therefore, one IY-TCM group and bibliotherapy group were established from the first county and an additional IY-TCM group and bibliotherapy group were 80 established for the second county. The data collection procedures and structure of the interventions remained consistent across the two groups; however the length of time during which the intervention occurred varied between the two counties. For the first county, the intervention was conducted between the end of November to beginning of May (e.g., approximately six months). For the second county, the intervention was conducted between the beginning of March to the end of May (e.g., approximately three months). Due to these differences, analyses considered the converged groups as well as the county differences. Participants The participants in this study included 31 preschool teachers divided into two group formats: the IY-TCM group (N = 16) and the bibliotherapy group (N = 15). The program developers recommend that teacher group training sessions should typically include up to 20 individuals in order to facilitate adequate group discussion (Webster-Stratton, 2012). However, for this study the number of teachers within the two groups was less than 20, with individual county training groups of less than 10. The teachers included within this study were primarily female (female = 30, male = 1) and were primarily from Great Start Readiness Programs (GSRP= 26, other = 5). In addition, participants also included the preschool students within the teachers’ classrooms ranging from three to five years of age. Parents of students within these classrooms were provided with the opportunity to remove their child from the data collection procedures, if desired. Before the pretest data collection period, 18 children were removed from the study due to their parents returning the form. In addition, based on notifications from teachers about students who had moved or missing data for individual students in a classroom, it was determined that 14 students dropped from the study. However, due to missing data procedures, 81 the dropped students were still able to be included in the HLM data analyses. In total, 443 students were included in data collection. Each teacher identified two target students demonstrating risk for internalizing and/or externalizing behavior problems to serve as target students for additional data collection based on pretest BASC-2 composite scores. The total number of target students within this study was 62 students. Table 4 includes the descriptive data for the participants within this study, including the total number of teacher participants within each group format, the types of programs the teachers were from, the mean age of teacher participants, and the number of students. Table 4 Descriptive Statistics for Teacher and Student Participants Variables Training Bibliotherapy Group Group Teachers (N) 16 15 Teacher Age M (SD) 38.15 (9.77) 38.53 (10.63) Gender Female 16 14 Male 0 1 Preschool Program GSRP 15 11 Private 1 4 Total Students (N) 233 210 Target Students (N) 32 30 Target Student Age (M) 4.58 4.59 Gender Female 16 15 Male 16 15 Measures The dependent variables for this study included analysis of variables at the teacher, classroom, and student level. At the teacher and classroom level, dependent variables measured included the teachers’ strategy use and perception of usefulness of the IY-TCM classroom management techniques, classroom atmosphere, student-teacher relationships, and teacherstudent positive interactions. At the individual student level, dependent variables included 82 positive peer interactions, positive social behaviors, internalizing behaviors, externalizing behaviors, and social skills. In addition, teacher acceptability of the treatment program was assessed through the Teacher Workshop Satisfaction Questionnaire and procedural treatment integrity was assessed through the teacher’s self-reported hours within the Professional Development Log of Hours. Teacher Strategies Questionnaire (TSQ). One of the objectives of the IY-TCM program is to provide teachers with the classroom management skills to be able to effectively manage the behavior of their students. The program developers assert that research data conducted with the IY-TCM program indicate that not only did children demonstrate differences in their problem behaviors and prosocial behaviors after the teacher training intervention, but that teachers’ level of confidence in using effective classroom management strategies increased after the intervention (Webster-Stratton, 2008). This study examined the teachers’ use of the IY strategies, perceptions of the usefulness of these strategies in the classroom, and their comfort in using the classroom management strategies at the pretest, midpoint, and post-test data collection phases. In order to measure these constructs, teachers in the two groups completed the Teacher Strategies Questionnaire (The Incredible Years, Inc., 2012). The Teacher Strategies Questionnaire was used within this study in order to determine teachers’ use and perceptions of the classroom management strategies because it is a measure that has been consistently used throughout various research studies including the IY-TCM program and in order to facilitate comparisons and maintain consistency in the evaluation of this construct (e.g., Carlson, Tiret, Bender, & Benson, 2011; Shernoff & Kratochwill. 2007; Williford & Shelton, 2008). The Revised Teacher Strategies Questionnaire (TSQ; The Incredible Years, Inc., 2012) is a 59-item questionnaire measuring the teacher’s self-reported use of strategies within the TCM 83 curriculum, perceptions of the usefulness of these teaching strategies in managing the classroom, confidence in using these strategies within the classroom, use of planning and support strategies taught within the TCM curriculum, and usefulness of these strategies in collaborating between home and school. Teachers complete likert-type ratings for the various items, either rating the item on a seven point scale (very unconfident, unconfident, somewhat unconfident, neutral, somewhat confident, confident, very confident), a six point scale (never, 1 time per year, 2-3 times per year, once a month, once a week, daily), or on a five point scale (rarely/never, sometimes, half the time, often, very often). The TSQ includes five summary scales, including Confidence in Managing Classroom Behavior, Total Positive Strategies, Inappropriate Strategies, Planning and Support Strategies, and Positive Approaches with Parents. The Total Positive Strategies Frequency of Use Scale and Perception of Strategy Usefulness Scale includes 28 items. In addition, this measure includes four subscales within the Total Positive Strategies summary scale, including a proactive strategies subscale (8 items), a coaching, praise and incentives subscale (8 items), a social and emotional teaching strategies subscale (7 items), and a limit-setting strategies subscale (5 items). The Inappropriate Strategies Scale includes nine items, the Confidence in Managing Classroom Behavior Scale includes three items, the Positive Approaches with Parents Scale includes eleven items, and the Planning and Support Scale includes eight items. The frequency and perception of usefulness of each of the five summary scales, in addition to the four additional subscales, were included in analyses within this study. The subscales were included in the analyses in order to determine more specific information pertaining to the changes in positive strategies and in order to align with previous IY-TCM research utilizing the TSQ measure, which included one or more of these subscales in their analyses (e.g., Carlson et al., 2011; Shernoff & Kratochwill, 2007; Williford & Shelton, 2008). 84 The program developers indicate on the Incredible Years Website that the psychometric properties of the revised version (2012) of the TSQ have not yet been analyzed (The Incredible Years, Inc., 2012). However, the internal consistency coefficients reported by the program developers for the older version of the TSQ indicate that the majority of subscales have adequate to good internal consistency (The Incredible Years, Inc., 2012). The older version of the TSQ (2001) includes fewer items on the Confidence in Managing Classroom Behavior Scale (2 items), the Total Positive Strategies Scale (18 items), Praise and Incentives subscale (6 items), and more items on the Positive Approaches with Parent Scale (17 items). The older TSQ does not include a Social and Emotional Teaching Strategies subscale or a Planning and Support Scale. The program developers report that the internal consistency coefficients at pretest for the TSQ scales are as follows: Confidence in Managing Classroom Behavior Scale = .94, Total Positive Strategies Frequency of Use Scale = .79, Inappropriate Strategies Frequency of Use Scale = .77, Total Positive Strategies Perception of Usefulness Scale = .70, Inappropriate Strategies Perception of Usefulness Scale = .82, Positive Approaches with Parents Scale = .78. Classroom Atmosphere Measure. This measure was used in order to assess the classroom climate for each classroom at the pretest, midpoint, and post-test phases in order to determine changes in the climate and atmosphere of the classroom context as the teachers engaged in the IY-TCM program or the bibliotherapy (reading) comparison group. It has been found within the literature that the social-emotional climate of the preschool classroom and the relationship between the child and the teacher were predictive of the level of social competence of children when entering early elementary school grades (Howes, 2000), which contribute to the interest in measuring this construct within this proposed study. 85 The Classroom Atmosphere Measure is a 10-item questionnaire developed by Fast Track and the Conduct Problems Prevention Research Group and used by program developers on the Incredible Years website for research purposes. This measure is completed by an observer at the end of an observation period and is used to provide an overall rating of student behavior and teacher interactions with students in order to produce a rating of the quality of the classroom atmosphere. The internal consistency for this scale has been reported to be .94 by the IY program developers and the interrater reliability coefficients have been reported between .55-.70 (Incredible Years, Inc., 2012; Webster-Stratton, Reid, & Hammond, 2004). The Classroom Atmosphere Measure consists of 10 items within three categories of rated observable behavior, including three items under the category of “disruptive behavior and compliance,” three items under the category of “cooperation, communication, and problem solving,” and four items under the category of “classroom, interest level, focus, responsiveness.” Each item is rated on a six point scale (very high, moderately high, average, moderately low, very low, and unable to code). Behavioral descriptions are provided for each rating option in order to provide further explanation of the behavior to be rated. After completing the 10-item questionnaire, the average score is obtained from each of the items and used as a measure of the classroom atmosphere, with a lower number (e.g., 1) representing a higher quality classroom atmosphere and a higher number (e.g., 5) representing a lower quality classroom atmosphere. Within this study, this measure was completed after each of the observation periods conducted by the research assistants, totaling four ratings per classroom. The mean of these four ratings created a Classroom Atmosphere score for each classroom. Student Teacher Relationship Scale (STRS). The Student Teacher Relationship Scale (STRS: Pianta, 2001) was completed at the pretest, midpoint, and post-test phases as a measure 86 of the teacher’s rating of the quality of the relationship between the teacher and students. The teacher completed forms for five selected students, including the two target students and three comparison students, in order to compare the relationship quality and any changes in the dynamics of the relationship between the teacher and these students. The three comparison students were selected based on pretest BASC-2 scores. When student scores were ranked from highest internalizing and externalizing risk scores to lowest scores, the comparison students were selected from the middle to bottom of the list and consisted of students with no scores within the risk ranges. The STRS is a 28-item self-report scale completed by teachers of children between preschool and third grade in order to obtain the teacher’s rating of the teacher-student relationship. This scale includes items comprising three subscales which assess important components of the relationship dyad and one overall scale score which assesses the overall quality of the relationship, including the Conflict Subscale, Closeness Subscale, Dependency Subscale, and the Total Scale score (Pianta, 2001). The Conflict Subscale includes 12 items that assess relationships defined by negativity and strain. The Closeness Subscale includes 11 items that measure dimensions of relationships that include caring, warmth and appropriate communication. The Dependency Subscale includes 5 items that measure the level of dependence and overreliance between a student and teacher. The Total Scale score assesses the overall quality of the relationship, with lower scores indicating a lower quality relationship. Scores on the Total Scale score can range from 28 to 140. The teacher is asked to consider their relationship with an individual student and the student’s behavior towards the teacher by rating items on a five-point likert scale (1: definitely does not apply, 2: does not really apply, 3: neutral/not sure, 4: applies somewhat, 5: definitely 87 applies). This scale typically takes between five and ten minutes to complete for one student. Scoring of this scale provides raw scores for subscales and the total scale which are converted to percentiles using a normative table. Subscale and scale scores for the teacher-student relationship are considered to be at-risk if the scores are at or above the 75th percentile for the Conflict (scores of 28 and above) or Dependency (scores of 12 and above) subscales or at or below the 25th percentile (scores of 40 or below) for the Closeness subscale or Total Scale score. The scale creators suggest that this scale can be used as a way to identify maladaptive or difficult relationships early on and to evaluate changes in the relationship as the result of intervention efforts. The STRS was normed with 1535 children between preschool and third grade and 275 teachers. The test-retest reliability presented in the STRS manual indicate adequate reliability within a four week period, with coefficients of .88 (Closeness), .92 (Conflict), .76 (Dependency), and .89 (Total). Internal consistency of the STRS is high for the Closeness (.86), Conflict (.92), and Total (.89) scores, but lower for the Dependency scores (.64). Research studies summarized within the STRS manual demonstrate the correlation between the STRS ratings and current and future academic skills, behaviors, and social skills (Pianta, 2001), supporting the concurrent and predictive validity of this measure. For example, the STRS ratings for kindergarten students correlated with behavior problem and social competence scores on the Teacher-Child Rating Scale of Classroom Adjustment, with correlations with behavior problems of .65 (Conflict), -.53 (Closeness), .29 (Dependency), and -.72 (Total), and correlations with social competence of -.60 (Conflict), .52 (Closeness), -.28 (Dependency), and .67 (Total). In addition, these correlations were similar to those between Kindergarten STRS ratings and first grade ratings of behavior problems and social competency in the expected directions. Concurrent validity between STRS 88 ratings and ratings of interactions and behaviors with peers indicated moderate correlations, and conflict scores in Kindergarten were found to significantly relate to academic outcomes in later grades (Pianta, 2001). In addition, the manual indicates that the STRS has adequate discriminant validity, as correlations between the STRS and other measures of teacher-rated behaviors and social competence indicate that the STRS does not overlap with these measures and explains an important amount of the variance for later outcomes in academic and social-emotional functioning. The STRS has been widely used throughout the literature to assess the teacher-student relationship construct in order to intervene for problem areas (Pianta, 2001; Webb & NeuharthPritchett, 2011), and several studies have also examined the psychometric properties further. For example, Doumen and colleagues (2009) found support for the convergent and discriminant validity of the Closeness and Conflict scales with other measures of these constructs collected from peer, outside observers, and the child. Less clear support was found for the validity of the Dependency scale; however the researchers highlight the need for more research and attention for this scale. Webb & Neuharth-Pritchett (2011) examined the factor structure of the STRS and differences in ratings for European American and African American children. This study found that there were factor loading problems for two items which affected the psychometric properties of the scale; however, when these two items were removed the factor structure of the scale was strong. These authors highlight the fact that one of the problem items is included within the Dependency subscale, which likely influences the lower psychometric properties of this subscale. In addition, this study found that ratings were sensitive to ethnicity and cultural interpretations, indicating that these results should be considered within the cultural context and differences should be attended to. Additionally, the STRS manual suggests that data collected 89 from teachers within the normative group indicate that teachers may report relationships to be slightly more positive (Pianta, 2001). Therefore, in order to address these potential biases presented by this research, this self-report rating scale was used in combination with an objective observational method (i.e., Direct Behavior Ratings) of teacher-student interactions in order to provide another objective source of data. The raw scores for each of the five students, including the two at-risk target students and the three comparison students, were used in data analyses in order to analyze changes in the student-teacher relationships for target students whom the teachers were directing attention towards as well as comparison students who were not identified as being at-risk for internalizing behaviors at the baseline phase. Direct Behavior Ratings (DBR). Direct Behavior Ratings (DBR) are a systematic observational assessment tool developed in order to allow for greater efficiency in collecting progress monitoring data and observations of a student’s behavior within the natural setting of the classroom (Chafouleas, Riley-Tillman, & Christ, 2009; University of Connecticut, 2010). Although most commonly examined with teachers, DBRs can be utilized for classroom observations conducted by any individual wishing to monitor behavior, such as school personnel collecting intervention data or parents (University of Connecticut, 2010). Additionally, students can be taught to collect self-monitoring DBR data for their own behaviors. Rating scale developers report that the DBR system is a method that combines the strengths of two well-utilized behavioral data collection methods, including systematic direct observation and behavioral rating scales (Chafouleas, Riley-Tillman, & Christ, 2009; Christ, Riley-Tillman, Chafouleas, & Jaffery, 2011). Rather than collecting data during each specified time interval (e.g., every 15 seconds), the DBR system requires the teacher or observer to define a specific observational period (e.g., 15 minutes) and to pay attention to the defined behaviors 90 provided on the DBR rating form (e.g., Disruptive Behavior, Respectful Behavior) during that observational period. Immediately after the specified observational period, the observer completes the DBR rating form in order to rate the child’s behavior across the specific observation period. The DBR system is not restricted to one set form, but instead the intervals and form can be adapted to fit the needs of the observation based on the behavior being observed, the rating intervals needed (i.e., percentage of time, scaling), and the use of a single item DBR versus a multi-item DBR (Chafouleas, Riley-Tillman, & Christ, 2009; University of Connecticut, 2010). Several studies have found that DBR outcomes are more reliable when behaviors are defined on a broad level (e.g., academically engaged), rather than with more specific definitions (e.g., raising hand), and that positive or negative wording of items depends on the behavior being defined (Riley-Tillman, Chafouleas, Christ, Briesch, & LeBel, 2009) In addition, studies have suggested that limited training is needed in order for teachers to collect accurate data using DBRs, as indicated by no significant difference in ratings between teachers in three training conditions (i.e., no training, indirect training, direct training) for disruptive behavior and less accurate ratings for the direct training group as compared to the other two groups when rating academic engagement (LeBel, Kilgus, Briesch, & Chafouleas, 2010). The research examining the use of DBR procedures in behavioral data collection has indicated positive results for the utility and psychometrics of this method. Chafouleas, Christ, Riley-Tillman, Briesch and Chanese (2007) examined the psychometric properties of the DBR through the application of generalizability theory (GT) in measuring the social behaviors of preschoolers. These authors found that approximately 57-58% of the variance in DBR ratings could be attributed to the rater and the target child. The results of the study indicated that ratings 91 across raters were not consistent, but that ratings within person were more consistent and a greater amount of the variance could be attributed to the target child and the setting/time of day. This finding supports the importance of maintaining consistency in the person collecting DBR observational data. In addition, reliability coefficients of the DBRs were above .70 when teachers completed seven ratings during a four to seven day period, and coefficients of .80 to .90 or higher were found when teachers had collected 10 DBRs, indicating that a greater number of DBRs increases the reliability of ratings (Chafouleas, Christ, Riley-Tillman, Briesch & Chanese, 2007). Research has indicated that the DBR method produces behavioral rating results consistent and similar to other forms of data collection. Chafouleas, McDougal, Riley-Tillman, Panahon, and Hilt (2005) found that approximately 82-87% of behavioral ratings from direct observations by trained observers and DBR-type ratings from teachers were within one point of each other, with none of the ratings exceeding a two point difference, and significant correlations found between the ratings of off-task behavior. In addition, Chafouleas and colleagues (2010) found that DBR data collected by a head teacher were consistent and detected change in behavior in a similar fashion with the data collected by the outside research assistants, whereas DBRs collected from a consultant teacher were less reliable and less consistent with the ratings of the head teacher and research assistants. This provides support for the consistency between DBR ratings collected by a lead teacher and observational data collected by outside observers with more experience and training (Chafouleas et al., 2010). Additionally, DBRs have been found to have large criterion-related validity coefficients when compared to systematic direct observation raw scores and standard scores for ratings of academic engagement and disruptive behavior (Christ, Riley-Tillman, Chafouleas, & Jaffery, 92 2011), were found to be significantly correlated to systematic direct observation ratings for ontask and disruptive behavior (Riley-Tillman, Chafouleas, Sassu, Chanese, & Glazer, 2008), and were moderately to strongly correlated at the beginning of the school year with the Social Skills Rating System (SSRS) (Chafouleas, Kilgus, & Hernandez, 2009). Research examining the use of DBRs in measuring change as a result of interventions has found that DBRs are sensitive to change for both positive and negative behaviors observed, and that the ratings of behavior and change were significantly correlated with the systematic direct observational (SDO) data collected by trained outside observers (Chafouleas, Sanetti, Kilgus, & Maggin, 2012). These findings provide support for the validity of the DBR observational method in comparison to another widely used observational method (e.g., SDO). DBRs were used in this study to measure the target students’ positive social behavior, teachers’ interactions with students, and interactions between peers and target students. A multiitem DBR form was used for the following three broadly defined constructs: positive social behavior, positive interactions with peers, and positive interactions with teachers. The definition for positive social behavior was adapted from a similar construct (“interacts cooperatively”) used by Chafouleas, Christ, Riley-Tillman, Briesch, and Chanese (2007), including “entering a work or play situation, participating cooperatively, and negotiating by making and accepting suggestions for play scenarios” (p. 68). Positive peer interactions towards target child was defined by slightly adapting a definition from the behavioral definition used by Webster-Stratton within the Independent Observation of Children in Classroom measure (see http://www.incredibleyears.com/Measures/em.asp) for “peer initiation/target child positive response” behaviors: “peers interact in a verbal or nonverbal manner towards the target child that is positive and inviting.” The definition for positive teacher interactions towards the target child 93 was slightly adapted from a similar construct (“positive social teacher behavior towards child”) used by Snyder and colleagues (2011), including “the teacher provides praise, recognition, and supportive verbal or physical actions towards the child” (p. 338). In accordance with recent studies utilizing DBRs (e.g., Christ, Riley-Tillman, Chafouleas, & Jaffery, 2011), the DBR for each of the three behaviors was rated on a line that is divided into 10 sections. The DBRs included three labels across the line, including ratings of 0%, 50%, and 100%, as well as ratings of never, sometimes, and always in these same labels (Christ, RileyTillman, Chafouleas, & Jaffery, 2011; Chafouleas, Kilgus, & Hernandez, 2009). The multi-item DBR included the positive social behaviors, positive peer interactions towards target child definitions, and positive teacher interactions towards the target child definition, which were rated by the graduate research assistants at the pretest, midpoint, and post-test phases. As mentioned previously, although DBRs have been most commonly examined when utilized by teachers within classrooms, this observational method is an efficient data collection method that can be used to collect behavioral data by a variety of school personnel (University of Connecticut, 2010). In addition, the research supports the consistency of DBR data collection with SDO data collection, and DBRs can allow the observer to pay attention to multiple behaviors through a less strenuous procedure. Therefore, the DBR data was collected by the research assistants in order to provide efficient data collection procedures of target student behavior within each of the classrooms. After each observation period, the trained graduate research assistants rated the child’s demonstration of the defined behavior according to how often it occurred represented as a percentage. While serving as a way to directly measure student behavior and interactions with peers and teachers, the DBR observations used within this study also served as a way to triangulate data from multiple sources collected within this study, such as self-reported teacher 94 strategy use and teacher-rated student behavior and teacher-student relationships. These DBR observational measures served as a way to determine whether results or outcomes reported on these rating scale measures for teacher and classroom level variables could also be directly captured during observations within the classroom in order to support this data. Behavior Assessment System for Children, Second Edition (BASC-2). The BASC-2 Teacher Rating Scale Preschool Version (TRS-P) and BASC-2 Parent Rating Scale Preschool Version (PRS-P) measures were used in this study as a pretest, midpoint, and post-test measure of child externalizing behavior, internalizing behavior, and social skills. The BASC-2 rating scales have been used previously in one study examining the use of the IY-TCM program (i.e. Williford & Shelton, 2008) in assessing externalizing behaviors, and have been used in various other studies throughout intervention research with preschool and school age children (e.g., Jurecska, Hamilton, & Peterson, 2011; McIntosh, Campbell, Carter, & Dickey, 2009; Schultz, Richardson, Barber, & Wilcox, 2011). The BASC-2 TRS-P (Reynolds & Kamphaus, 2004) is a 100-item rating scale that is completed by the teacher in order to provide data regarding an individual student’s problem behaviors and adaptive behaviors. The TRS-P is used for children between two and five years of age. Teachers are expected to consider the student’s behavior over the last several months and to rate each item on a four point scale regarding the frequency of the behavior, including never occurs, sometimes occurs, often occurs, or almost always occurs. The TRS-P includes items for three composites, including Externalizing Problems, Internalizing Problems, and Adaptive Behaviors, as well as one broad composite of Behavior Symptoms Index (BSI). This rating scale consists of items that comprise 11 scales, including eight clinical scales (hyperactivity, aggression, anxiety, depression, somatization, attention problems, atypicality, withdrawal) and 95 three adaptive scales (adaptability, social skills, functional communication) (Tan, 2007). The TRS includes three validity checks, including the F index (indicating an overly negative response pattern), response pattern index (indicating that the rater was not paying attention to the descriptions and content of the items), and consistency index (indicating that the rater was not rating similar items in a similar manner). The BASC-2 PRS-P (Reynolds & Kamphaus, 2004) is a 134-item rating scale completed by parents in order to provide information regarding their child’s problem behaviors and adaptive behaviors at home. This rating scale is also used with children between ages two and five. In completing the rating scale, parents consider a four point scale for each item in describing their child’s behavior ranging from never to almost always. The PRS-P measures behaviors across three composites, including Externalizing Problems, Internalizing Problems, and Adaptive Behaviors, as well as the Behavioral Symptoms Index (BSI) (Reynolds & Kamphaus, 2004; Tan, 2007). It has 11 of the same scales as listed above for the TRS-P. However, one additional adaptive scale (activities of daily living) is included within the PRS-P, bringing the total scales included within this measure to 12. The PRS also includes the three validity checks described previously. Scoring of the BASC-2 TRS and PRS produces two types of scores: T scores (mean of 50 and standard deviation of 10) and percentiles (Reynolds & Kamphaus, 2004). The BASC-2 rating scales can be hand scored or scored using a computer software program. These scores can be compared to a General Norms Sample or a Clinical Norms Sample. For the purposes of the current study, the General Norms Sample were used in order to compare the students within the classroom to the general population. Within the Clinical Scales, a score of 70 and above is classified as being within the Clinically Significant Range, and a score of 60-69 is within the At- 96 Risk Range. Therefore, a higher score on the Clinical Scales indicates higher impairment. Within the Adaptive Scales, a score 30 and below is classified as being within the Clinically Significant Range, and a score of 31-40 is within the At-Risk Range. Therefore, a lower score on the Adaptive Scales indicates higher impairment. The BASC-2 PRS and TRS rating scales have been found to have adequate psychometric properties (Reynolds & Kamphaus, 2004; Tan, 2007). The PRS was normed with a sample of 4,800 children, and the TRS was normed with a sample of 4,650 children; both scales were representative of the 2001 Current Population Survey (Tan, 2007). The internal consistency coefficients for the TRS were above .90 for the Externalizing Problems composite (mid .90s), Adaptive Skills composite (low to mid .90s), and the Behavioral Symptoms Index (mid .90s), and the coefficients were in the high .80s to low .90s for the Internalizing Problems composite. For the individual preschool scales, the median coefficient was approximately .84 (Reynolds & Kamphaus, 2004). Internal consistency coefficients for the PRS range from above .90 for some areas (Behavior Symptoms Index, Adaptive Skills) to the mid .80s to mid .90s for other areas (Externalizing Problems and Internalizing Problems). Test-retest reliability coefficients for the TRS range from the mid .80s to low .90s, and the PRS are around the low .80s to the low .90s. The median test-retest reliability for the individual PRS preschool scale is .77 and .82 for the TRS. The median interrater reliabilities for the TRS preschool scales are .65 and .74 for the PRS. The construct validity for the BASC-2 scales are moderate to high, and the criterionrelated validity scores are high for the TRS when compared to the ASEBA, CTRS-R, and BASC, and moderate to high for the PRS when compared to other parent rating scales (Tan, 2007). Teacher Workshop Satisfaction Questionnaire. The Teacher Workshop Satisfaction Questionnaire is a 37-item questionnaire used in order to evaluate teacher acceptability of the 97 Incredible Years Teacher Training Program at the end of treatment. This questionnaire includes items that assess the teachers’ ratings of the usefulness of the program in changing their behavior or their students’ behavior, usefulness of the format of the training program, usefulness of specific IY classroom management strategies covered, aspects of the program that were most and least helpful, and a rating of the group leader. This questionnaire was used within this study in order to measure teacher acceptability of the program content for the IY-TCM group and bibliotherapy (reading) comparison group. The questionnaire was modified for the bibliotherapy comparison group to exclude two sections related to the presentation of the content during the workshop and the group leader, as they were not exposed to these group features. However, sections related to the teachers’ ratings of the usefulness of the program in changing their behavior or their students’ behavior, usefulness of specific IY classroom management strategies covered, and aspects of the program that were most and least helpful were used for both groups in order to measure acceptability of the treatment. The psychometric properties of this measure have not been examined in the literature at this time. Professional Development Log of Hours. Teachers within each of the two conditions completed the Professional Development Log of Hours throughout the study and turned it in at the end of the study (Appendix A). This Professional Development Log of Hours asked teachers to keep track of the number of hours that they spent reading about the new classroom management strategies, practicing these strategies, or attending training sessions. The form also asked the teachers to list the ways that they applied these strategies (if applicable). This measure served as a self-monitoring system of the number of hours spent on professional development for each teacher and served as a measure of the procedural integrity and intensity of the treatment for each condition. 98 Procedures Prior to beginning recruitment or data collection procedures, approval was obtained for this study from the Michigan State University Institutional Review Board (IRB), and the study was determined to have exempt status (IRB# x13-594e/ APP# i043822). Table 3 outlines the measures that were collected at the pretest, midpoint, and post-test phases for this study for the teachers within each county. Recruitment. Teacher participants were recruited primarily from two counties within Michigan through the dissemination of flyers and consent forms (Appendix B and C) related to the study and the teacher training program to preschool programs. Flyers described the features of the IY-TCM program, program length, and program format, and outlined the incentives that would be provided to teachers for participating. All teachers who participated in the study received two $25 gift cards, one of which was provided at the midpoint time period and the other at the end of the study. In addition to the gift cards, teachers within the reading group were able to earn 15 State Continuing Education Clock Hours (SCECHs) for completing the reading assignments and the teachers within the IY-TCM group were able to earn 30 SCECHs for attending the six training sessions. Pretest data collection phase. After the recruitment phase, teachers were assigned to groups using a fixed number random assignment procedure with a random number generator. Teachers within each county were assigned to one of two groups: the IY-TCM training group or a bibliotherapy (reading) comparison group. Each county consisted of one IY-TCM group and one reading group, for a total of four groups. Each of these group conditions are explained in more detail below. Consent forms were sent home to the parents of each of the children within the classrooms included within this study, and parents were instructed to return the form if they 99 wanted to remove their child from the study data collection procedures (Appendix D). However, the teachers still participated in the study even if they had students who were removed from data collection. Due to the different recruitment timelines for the two counties, the county groups had a different timeline for completing the reading assignments and the training sessions. County one participated in the intervention over a six month period, and County two participated in the intervention over a three month period. Before the beginning of the intervention, teachers were asked to complete several data collection procedures. First, all teachers were asked to complete the Teacher Strategies Questionnaire during the pretest phase of the study in order to indicate their use and perceptions of a variety of classroom management strategies, as well as their collaboration with parents. Teachers were then asked to fill out the BASC-2 forms for each of the preschool students in their class. These rating forms were scored by the researchers, and each teacher was provided with a list of four to six students with the highest risk scores for internalizing and/or externalizing scores in order to select two target students to focus on throughout the study. This procedure was utilized in accordance with best practices in primary prevention and early screening methods within a Response to Intervention (RTI) framework in order to determine the level of risk for students’ behaviors at the initial screening point within the problem solving process in order to inform interventions (Ikeeda, Neesen, & Witt, 2008; Huberty, 2008). This method also allowed teachers and researchers to determine which students were in most need for more directed and targeted interventions. Based on the data presented, teachers chose one student in their class with a higher risk on the Internalizing Problems domain (e.g., within the At-Risk or Clinically Significant Ranges) and one student who had higher scores for both the Internalizing and 100 Externalizing scales (within the At-Risk or Clinically Significant Ranges), indicating comorbidity. If no students in the class fit those range classifications, students with the highest scores in those domain areas were included for the teacher to select. The two students selected from each classroom were included in additional data collection procedures for each of the teachers. Parents of the target students were asked to complete the BASC-2 PRS forms in order to provide additional data regarding the students’ internalizing behaviors, externalizing behaviors, and social skills. Parents of these target students were informed that they would be provided with a $20 gift card incentive after the end of the study for completing the data collection forms. The above procedures were the same for all teachers within the study. However, additional data collection occurred within County one (N = 14) in order to gather more in-depth observation data and teacher-student relationship data for this subset of the participants. These procedures were used only for this subset due to constraints in timelines and resources. For these additional procedures, graduate research assistants who were blind to the study conditions to which teachers were assigned completed Direct Behavior Rating (DBR) observations for each target student for two fifteen minute time periods, one period during an unstructured period and one period during a structured period in the classroom. This method is an adaptation from behavioral observation methods that have been used across IY studies in the past (e.g., Shernoff & Kratochwill, 2007; Snyder et al., 2011). The graduate research assistants underwent training with videotape recordings of students within classrooms in order to achieve an acceptable reliability level in using the DBR observational measures and the Classroom Atmosphere Measure. The graduate research assistants participated in nine hours of video training in order to establish the observation procedures and consistent ratings. In addition, a percentage of the 101 observations for each data collection phase required two research assistants to observe the same classroom at the same time in order to obtain a reliability check. Eight out of 14 classrooms were compared for the pretest, and four out of 14 for each of the midpoint and posttest periods. The reliability during the three data collection observations were 96%, 95%, and 96% agreement. The DBR observational scales defining the positive teacher interactions towards the target children, the positive peer interactions, and the positive social behavior of the target children were filled out by the trained observers after observing the behaviors and interactions in the classroom. At the end of each of the observation periods, they completed the Classroom Atmosphere Measure, leading to four Classroom Atmosphere Measure ratings per classroom (two observations per two students). Teachers within County one were also asked to complete STRS forms for five students selected by the researchers, including the two target students and three other students with internalizing scores within the average to low risk range. The teachers were asked to complete these ratings in order to provide data regarding the student-teacher relationships between teachers and target students and between the teachers and a sample of comparison students. Intervention phase. After the pretest data collection procedures were completed, the intervention phase of the study began. The IY-TCM group participated in six training sessions, and the bibliotherapy comparison group was given IY reading materials and provided with a schedule for completing reading assignments. Teachers within both conditions were asked to keep a log of the amount of time spent on reading, practicing, training, and professional development activities related to classroom management strategies per week in order to calculate the amount of time spent by these teachers on the Professional Development Log of Hours. Random assignment to two conditions 102 IY-TCM Group Training Intervention Condition. The IY-TCM program was delivered by a certified IY group leader to the teachers within the group training intervention condition across six full-day workshops (six hours each for training content, one hour for breaks; 36 total content hours), which is the recommended training format by program developers (WebsterStratton, 2012; Webster-Stratton, Reinke, Herman, & Newcomer, 2011). The IY-TCM program can be adapted to fit the needs of the given group of teachers and children (Webster-Stratton & Reid, 2010). With this in mind, the IY-TCM training program for this study encouraged teachers to not only consider the needs of their most disruptive students in the classroom, but to also focus on two target students demonstrating higher risk for internalizing problems, as indicated by the pre-test data collection procedures. The certified group leaders for the Incredible Years programs are required to complete an intensive certification process in order to ensure the appropriate training and fidelity of the intervention process. The program developers require this certification process for group leaders in order to use the Incredible Years program within research studies, as this certification supports the comparison between published literature on the IY program and group training outcomes for groups led by these certified leaders (Webster-Stratton, 2008). The certification process includes attendance and completion of an approved training workshop led by program developers, conducting two teacher training workshops including videotaping sessions and completing all self and teacher evaluation forms of the quality of the training sessions, submitting and receiving feedback from a certified trainer/mentor on these videotapes and evaluations, and application for official certification (Webster-Stratton, 2008). In addition to this certification process completed by the certified IY trainer, the integrity of the IY-TCM training sessions was measured for this study by having the IY group leader 103 videotape half of the training sessions (e.g., six sessions) and complete two integrity checklists for each session included within the IY-TCM training manual. For these six videotaped sessions, a graduate research assistant also provided a fidelity check for four of the sessions by individually completing the same two integrity checklists based on the content provided within the video recordings. For the four sessions coded by the graduate research assistant, the reliability between the trainer and coder were as follows: 82%, 76%, 68%, and 82%. The items for which there were disagreements in ratings were related to modifications made by the trainer or items that did not apply to that session, such as the method of providing the agenda, the discussion of homework after the video recording was ended, writing versus paraphrasing the discussion points, working in pairs versus triads, and showing two vignettes in a row. Analysis of the differences in ratings revealed that these were minor and did not indicate that content or key components were missing from instruction. The IY-TCM curriculum is divided into the following workshop days targeting specific skills: 1) building positive relationships with students, 2) preventing behavior problems: the proactive teacher, 3) the importance of teacher attention, coaching, encouragement, and praise, 4) motivating children through incentives, and 5) decreasing inappropriate behaviors (WebsterStratton, 2011). The goals of the IY-TCM program, as outlined by the program developers, are to provide a group training format for teachers and group leaders to work collaboratively through discussions of video vignettes involving various teacher-student interactions and classroom situations, role playing activities, discussion of important concepts, and feedback from the group during practice activities (Webster-Stratton & Reid, 2010). Within this Classroom Management Curriculum, teachers learn strategies and skills for handling various student behaviors in the classroom, promoting and helping children to develop 104 appropriate social skills, friendships, and problem solving skills when working with others across a variety of contexts, developing positive relationships between teachers and students, working with parents, creating behavior plans to address student behaviors, and appropriate and effective discipline strategies (Webster-Stratton, 2011; Webster-Stratton & Herman, 2010). Not only do the teachers within the group learn evidence-based and effective classroom management strategies, but they also learn how to foster the social, emotional, and academic skills of all of their students (Webster-Stratton, 2012). By targeting student behavior early on during the early childhood and elementary school years, the IY program developers highlight the importance for providing prevention and early intervention efforts to increase student resilience and decrease risk for maladaptive behavioral functioning through direct child intervention (Child Training Series), teaching of parenting skills (Parent Training Series), or teaching of effective classroom management strategies (Teacher Training Series; Webster-Stratton & Reid, 2003). Bibliotherapy comparison condition. The teachers randomly assigned to the bibliotherapy comparison condition were not included within the IY-TCM group training, but were provided with a book to read on their own. The use of bibliotherapy as an intervention method has been examined throughout the literature, however some authors note that the research on this therapeutic technique is still limited and presents with mixed results (Gavigan, Kurtts, & Mimms, 2010). Various forms of bibliotherapy have been utilized that differ in setting and process, including those techniques used within a clinical therapeutic setting, techniques used within schools and libraries to provide individuals with materials to help them through difficult or stressful situations (Gavigan, Kurtts, & Mimms, 2010), bibliotherapy used within group therapy along with discussions and role playing (e.g., Iaquinta & Hipsky, 2006; Shechtman, 2000; Strobel, 2011), and bibliotherapy provided through books or workbooks for 105 self-study in combination with a brief description of the process or telephone check-ins (e.g., Ackerson, Scogin, McKendree-Smith, & Lyman, 1998; Kilfedder et al., 2010). Within the current study, bibliotherapy was used as a comparison condition in which teachers were provided with a book and a schedule for selected chapters to read without followup phone calls or consultation. The book that was provided to the bibliotherapy condition was the Incredible Teachers: Nurturing Children’s Social, Emotional, and Academic Competence book, authored by Webster-Stratton (2012). This group condition served as typical treatment due to the fact that any teacher is able to access the book on their own if they wish, through the Incredible Years website at http://www.incredibleyears.com/products/products.asp. Teachers within this condition were provided with the book and a schedule for selected chapters to read across the intervention that aligned with the order in which the topics were covered across the IY-TCM curriculum (see Table 5). The minimum number of hours that this group was expected to spend on the content was 15 hours in order to complete the reading assignments and earn the SCECHs. In addition, this condition served as a comparison condition in which the amount of time the teachers spent on professional development or active learning was measured between the IY-TCM and bibliotherapy comparison condition. In this manner, differences between the results for these two conditions would unlikely be attributed to increased attention and time spent by the teachers between these two conditions, and would be more likely to be attributed to the IY-TCM group training format received by the intervention group. Table 5 Bibliotherapy Schedule for Assigned Readings Schedule Assigned Chapters Month 1 Chapter 1, Chapter 2, Chapter 3, Chapter 14 Month 2 Chapter 4, Chapter 5 Month 3 Chapter 6 Month 4 Chapter 7, Chapter 15 Month 5 Chapter 8, Chapter 9, Chapter 10 Month 6 Chapter 11, Chapter 12, Chapter 13 106 Midpoint phase. After the third training session (out of six sessions), midpoint data collection procedures were completed in order to collect data regarding changes in teacher level variables, classroom variables, and student variables. All of the teachers completed the TSQ measure and the BASC-2 TRS forms for the two target students. Parents also completed the BASC-2 PRS for the target children. For the classrooms within the first county, the teachers completed the STRS forms for the five students selected at the pretest phase, and the graduate research assistants completed observations using the DBR measures (teacher-student interactions, peer interactions, positive social behavior) for the target students and the Classroom Atmosphere Measure. Post-test phase. Within one month of the last IY-TCM group training session, post-test data was collected in the same fashion as during the pretest phase. Teachers completed the TSQ and completed BASC-2 TRS rating forms for all students in their classroom. Parents completed BASC-2 PRS forms for the selected target children. Teachers within the first county also completed the STRS for the selected students. The graduate research assistants completed DBR observational ratings for the positive teacher interactions towards the target children, target student positive social behaviors, and positive peer interactions towards target student, as well as the Classroom Atmosphere Measure for the classrooms within the first county. In addition to these measures, all of the teachers completed the Teacher Workshop Satisfaction Questionnaire in order to provide information regarding teacher ratings of acceptability of the overall program and the specific strategies. In addition, teachers submitted the Professional Development Log of Hours. Data Analysis 107 In order to address research questions two through four, hierarchical linear modeling (HLM) was used in order to analyze the differences between the two groups on teacher, classroom, and student level variables, similar to the statistical methodology used within the IY studies conducted by Williford and Shelton (2008) and Webster-Stratton, Reid, and Stoolmiller (2008). HLM is an appropriate statistical method to use within educational research when outcomes are being assessed for classroom level or student level variables which are naturally nested within an organization or system (Peugh, 2010). This level of analysis accounts for the possible influence of the system on the individual outcomes, whereas much of the current research within education utilizing single-level analyses may ignore these important nested effects (Raudenbush & Bryk, 1986). These multilevel analyses allow for the analysis of nested data, such as classrooms/teachers within schools, students within classrooms/teachers, and multiple data points within students (Peugh, 2010). This nesting effect is important to examine because individuals from similar organizations will likely show some relationship to each other, which would violate an assumption of independence for single-level analyses (McCoach, 2010). However, the assumption of independence is not needed within HLM research. HLM allows for the investigation of the amount of variance that can be attributed to nested level variables or to other predictors, such as gender or socioeconomic status. HLM allows for the analysis of both within-group variation and between-group variation due to these nested structures (Raudenbush & Bryk, 1986). Within HLM analyses, outcomes of the level one within-model analyses are used within the level two between-model analyses in order to explain the outcomes (Lee and Bryk, 1989). For this study, there were three potential levels of nested data, including time points nested within students or classrooms/teachers, and students nested within 108 classrooms/teachers. These nested levels are displayed in Table 6, and the codes for each HLM variable are in Table 7. Table 6 Hierarchical Linear Modeling Nesting Structure Classroom/Teacher Outcomes All Student Outcomes Classroom/Teacher Repeated Measures Classroom/Teacher Students Repeated measures 109 Target Student Outcomes Classroom/Teacher Students Repeated measures Table 7 Study Variables and Codes Variable Description Teacher Level Variables Frequency of Positive Strategy Use Perception of Usefulness of Positive Strategy Frequency of Limit Setting Use Perception of Usefulness of Limit Setting Frequency of Social-Emotional Use Perception of Usefulness of Social-Emotional Frequency of Proactive Strategy Use Perception of Usefulness of Proactive Strategy Frequency of Coaching Use Perception of Usefulness of Coaching Frequency of Inappropriate Strategy Use Perception of Usefulness of Inappropriate Strategy Planning and Support Strategy Confidence in Managing Classroom Behaviors Positive Approaches with Parents Variable Code PSTR_FREQ PSTR_USE LIMSET_FREQ LIMSET_USE SOCE_FREQ SOCE_USE PROACT_FREQ PROACT_USE COACH_FREQ COACH_USE INAPP_FREQ INAPP_USE PLAN CONFID POSAPP Classroom Level Variables Classroom Atmosphere Measure Total STRS Score STRS Conflict STRS Closeness STRS Dependency DBR Teacher-Student Interaction CLASSATMOS STRSTOT CONF CLOS DEP TSINT Student Level Variables BASC-2 Teacher Rating - Externalizing Composite Score BASC-2 Teacher Rating - Internalizing Composite Score BASC-2 Teacher Rating - Social Skills Score BASC-2 Parent Rating - Externalizing Composite Score BASC-2 Parent Rating - Internalizing Composite Score BASC-2 Parent Rating - Social Skills Score DBR Positive Social Behavior DBR Peer Interactions EXT INT SS PEXT PINT PSS SOCIAL PEER Dummy Variable Codes Group Assignment County Group by County Target Student versus Non-target Student Target Student: at-risk for internalizing Target Student: at-risk for internalizing and externalizing Linear pattern over time Nonlinear trend over time GROUP COUNTY GRPXCOUNTY TARGNOT RISK_INT RISK_BOTH TIME TIME2 110 Question One. The first research question examined whether the treatment groups were carried out as intended prior to examining the outcomes of these treatment groups. In order to determine differences in the procedural integrity of treatment, as measured by teacher-reported log of hours spent on professional development for classroom management strategies at post-test, an independent samples t-test was used in order to determine significant differences between the two groups. Questions Two and Three. For the teacher level variables and the first classroom level variable (classroom atmosphere) assessed in the second and third research questions, the repeated measures were nested within classrooms/teachers. Two-level longitudinal random intercept models (Raudenbush & Bryk, 2002) were used in order to determine whether the linear growth trajectories across the two groups from baseline to midpoint to post-test were different and to determine the amount of variance that could be accounted for by the classroom/teacher level variables and the treatment condition to which they were assigned. This two-level model consisted of waves of data (three data points including pretest, midpoint, and post-test) at level one, nested within classrooms/teachers at level two. The other classroom variables (DBR teacher-student interaction and STRS variables) were analyzed within a three-level longitudinal random intercept model in order to account for the repeated measures nested within student level data for these observations and rating scales, which were nested within the teachers/classrooms. The linear patterns across the three time points was also examined in order to determine whether there were significant increases over time and whether this pattern was linear or quadratic. Within the two-level models, the first level of analysis included the classroom/teacher level variables across each of the time points, representing repeated measures. Within this level, the linear pattern between the pretest and posttest time periods were measured (TIME), and a 111 variable was entered in order to determine whether the increase was linear or non-linear (TIME2). Each variable was entered separately as an outcome variable. The second level of analysis included other classroom/teacher variables selected specifically for that outcome variable, such as Total Positive Strategy Frequency of Use Score (PSTR_FREQ), Total Positive Strategy Perception of Usefulness Score (PSTR_USE), Frequency of Limit-Setting Strategy Use (LIMSET_FREQ), Perception of Usefulness of Limit-Setting Strategies (LIMSET_USE), Frequency of Social-Emotional Strategy Use (SOCE_FREQ), Perception of Usefulness of Social-Emotional Strategies (SOCE_USE), Frequency of Proactive Strategy Use (PROACT_FREQ), Perception of Usefulness of Proactive Strategies (PROACT_USE), Frequency of Coaching/Praise/Incentives Strategy Use (COACH/FREQ), Perception of Usefulness of Coaching/Praise/Incentive Strategies (COACH_USE), Inappropriate Strategies Frequency of Use Score (INNAP_FREQ), Inappropriate Strategies Perception of Usefulness Score (INAPP_USE), Planning and Support Strategy Score (PLAN), Confidence in Managing Classroom Behavior Score (CONFID), Positive Approaches with Parents Score (POSAP), and classroom atmosphere measure observation ratings (CLASSATMOS). The group assignment was set as a dummy variable included at the second level (bibliotherapy = 0, IY-TCM = 1) in order to analyze the group classification as a fixed effect predictor. The county from which the teacher came was also included as a fixed effect dummy variable predictor (county 1 = 1, county 2 = 0) in order to account for any differences between the two recruitment groups, as these two groups participated in the study for different lengths of time. Intra-class correlations were calculated for all variables to examine the models (Raudenbush & Bryk, 2002). Within the three-level models, the first level of analysis included the variables across each of the time points, representing repeated measures (TIME, TIME2). The second level of 112 analysis included other student/classroom level variables selected as predictors within the model, such as STRS conflict score (CONF), STRS closeness score (CLOS), STRS dependency score (DEP), STRS total score (STRSTOT), and DBR teacher-student interactions (TSINT). Dummy variables were also included for the target student classification (TARGNOT; 0 = target student, 1 = not target student) and the identification of risk for internalizing symptoms (RISK_INT; 1 = internalizing only target, 0 = not) or both externalizing and internalizing symptoms (RISK_BOTH; 1 = both internalizing and externalizing target, 0 = not) for the target students. The third level of analysis included the teacher level variables, group dummy variable, and county dummy variable. Within each model, time variables and dummy variables were entered as uncentered variables, and continuous variables were centered around the grand mean. One sample equation for the two-level models for question two is included below, along with two sample equations for the two-level and three level models for question three: Question Two: Two-level model Level-1 Model PSTR_FREti = π0i + π1i*(TIMEti) + π2i*(TIME2ti) + eti Where π0i is the intercept, π1i- π2i are the multiple observations over time, and eti is the residual term. Level-2 Model π0i = β00 + β01*(COUNTYi) + β02*(GROUPi) + β03*(PSTR_USEi) + β04*(INAPP_FREQi) + β05*(INAPP_USEi) + r0i π1i = β10 π2i = β20 Where β00 is the intercept, β01- β05 are the teacher level predictors, and r0i is the residual term for teacher differences. 113 Question Three: Two-level model Level-1 Model CLASSATMOSti = π0i + π1i*(TIMEti) + π2i*(TIME2ti) + eti Where π0i is the intercept, π1i- π2i are the multiple observations over time, and eti is the residual term. Level-2 Model π0i = β00 + β01*(GROUPi) + β02*(PSTR_FREQi) + β03*(INAPP_FREQi) + r0i π1i = β10 π2i = β20 Where β00 is the intercept, β01- β03 are the teacher level predictors, and r0i is the residual term for teacher differences. Question Two: Three-level model Level-1 Model CONFtij = π0ij + π1ij*(TIMEtij) + π2ij*(TIME2tij) + etij Where π0ji is the intercept, π1ij- π2ji are the multiple observations over time, and etij is the residual term. Level-2 Model π0ij = β00j + β01j*(TARGNOTij) + β02j*(CLOSij) + β03j*(DEPij) + r0ij π1ij = β10j π2ij = β20j Where β00 is the intercept, β01j- β03j are the student level predictors, and r0i is the residual term for teacher/classroom differences. Level-3 Model 114 β00j = γ000 + γ001(GROUPj) + γ002(PSTR_FREQj) + γ003(INAPP_FREQj) + u00j β01j = γ010 β02j = γ020 β03j = γ030 β10j = γ100 β20j = γ200 Where γ000 is the intercept, γ001- γ003 are the teacher level predictors, and u00j is the residual term for teacher differences. Question Four (a-c). For the fourth research questions pertaining to changes in teacher ratings of student internalizing scores, teacher ratings of student externalizing scores, and teacher ratings of student social skills for all classroom students across the two groups, a three-level random intercept model with random effects was used in order to determine pretest to post-test differences on these student level variables and the amount of variance that could be accounted for by the student level variables, classroom/teacher level variables, and treatment condition to which they were assigned. The first level of analysis consisted of student internalizing scores on teacher ratings, student externalizing scores on teacher ratings, and student social skills scores on teacher ratings as outcome variables over time as repeated measures (TIME). The second level of analysis included the other student level variables not designated as the outcome variable. Predictors on this level serving as random effects included the other variables not designated as outcome variables. This model was run for each separate student outcome variable. The third level of analysis consisted of teacher level variables, the group classification dummy variable (0 = bibliotherapy, 1 = IY-TCM), and the county dummy variable (1 = county 1, 0 = county 2). Within each model, time variables and dummy variables were entered as 115 uncentered variables, and continuous variables were centered around the grand mean. One example model equation is included below: Level-1 Model INTtij = π0ij + π1ij*(TIMEtij) + etij Where π0ij is the intercept, π1ij is the multiple observations over time, and etij is the residual term. Level-2 Model π0ij = β00j + β01j*(EXTij) + β02j*(SSij) + r0ij π1ij = β10j Where β00j is the intercept, β01j- β02j are the student level predictors, and r0ij is the residual term. Level-3 Model β00j = γ000 + γ001(COUNTYj) + γ002(GROUPj) + γ003(PSTR_FREQj) + γ004(INAPP_FREQj) + u00j β01j = γ010 β02j = γ020 β10j = γ100 Where γ000 is the intercept, γ001- γ004 are the teacher level predictors, and u00j is the residual term for teacher differences. Questions Four (d-h). In order to address these subparts of the fourth research question pertaining to the identified target students, a three-level longitudinal random intercept model with random effects was used in order to determine whether the individual linear growth trajectories for the target students within the two groups from pretest to midpoint to post-test on student level variables were different and to determine the amount of variance that could be 116 accounted for by the student level variables, classroom/teacher level variables, and treatment condition to which they are assigned. This three-level model consisted of waves of data (three data points including pretest, midpoint, and post-test) at level one, nested within students on level two, nested within classrooms/teachers at level three. The linear patterns across the three time points was also examined in order to determine whether there were significant increases over time and whether this pattern was linear or quadratic. The first level of analysis included target student internalizing scores on parent and teacher ratings, target student externalizing scores on parent and teacher ratings, target student social skill scores on parent and teacher ratings, target student DBR peer interaction scores, and target student DBR positive social behavior scores as outcome variables across the repeated measure time periods (TIME, TIME2). Each variable was entered separately as an outcome variable. The second level of analysis included the other student level predictors, such as internalizing scores on teacher and parent ratings, externalizing scores on teacher and parent ratings, social skills on teacher and parent ratings, target student DBR peer interaction scores, and target student DBR positive social behavior scores. Dummy variables for the target student identification of risk for internalizing symptoms (RISK_INT; 1 = internalizing only target, 0 = not) or both externalizing and internalizing symptoms (RISK_BOTH; 1 = both internalizing and externalizing target, 0 = not) were also included at the second level. The third level of analysis consisted of classroom/teacher level variables, the group classification dummy variable (0 = bibliotherapy, 1 = IY-TCM), and the county dummy variable (1 = county 1, 0 = county 2). Within each model, time variables and dummy variables were entered as uncentered variables, and continuous variables were centered around the grand mean. One example model equation is included below: 117 Level-1 Model INTtij = π0ij + π1ij*(TIMEtij) + π2ij*(TIME2tij) + etij Where π0ij is the intercept, π1ij- π2ij are the multiple observations over time, and etij is the residual term. Level-2 Model π0ij = β00j + β01j*(RISK_INTij) + β02j*(RISK_BOTij) + β03j*(EXTij) + β04j*(SSij) + r0ij π1ij = β10j π2ij = β20j Where β00j is the intercept, β01j - β04j are the student level predictors, and r0ij is the residual term. Level-3 Model β00j = γ000 + γ001(COUNTYj) + γ002(GROUPj) + γ003(PSTR_FREQj) + γ004(INAPP_FREQj) + u00j β01j = γ010 β02j = γ020 β03j = γ030 β04j = γ040 β10j = γ100 β20j = γ200 Where γ000 is the intercept, γ001 - γ004 are the teacher level predictors, and u00j is the residual term for teacher differences. During analyses, six variables analyzed within research questions 4d-h were found to have a third level intra-class correlation that was below 0.10, indicating that there was a lack of variance among teachers for these variables. These variables included: 1) parent-rated target 118 student internalizing behaviors, 2) parent-rated target student externalizing behaviors, 3) parentrated target student social skills, 4) teacher-rated target student externalizing behaviors, 5) DBR positive social behaviors, and 6) DBR positive peer interactions. Further analysis and calculation of the design effect indicated that the cluster effect did not have much of an influence, therefore making it necessary to include the third level variables at the second level without the teacher ID variables used as predictors at the second level. Therefore, analysis of these variables involved a two-level longitudinal random intercept model. An example model equation is included below: Level-1 Model SOCIALti = π0i + π1i*(TIMEti) + π2i*(TIME2ti) + eti Where π0i is the intercept, π1i - π2i are the multiple observations over time, and eti is the residual term. Level-2 Model π0i = β00 + β01*(PEERi) + β02*(TSINTi) + β03*(RISK_INTi) + β04*(RISK_BOTi) + β05*(SSi) + β06*(STRSTOTi) + β07*(GROUPi) + β08*(PSTR_FREQi) + β09*(INAPP_FREQi) + r0i π1i = β10 π2i = β20 Where β00 is the intercept, β01- β09 are the combined teacher and student level predictors, and r0i is the residual term for student and teacher differences. Question Five. In order to determine differences in the acceptability ratings of teachers in the IY-TCM group and the bibliotherapy comparison group at the post-test rating period, an independent samples t-test was used in order to determine significant differences between the 119 means of the two groups on the ratings of usefulness of specific teaching strategies covered in the program and the ratings of the usefulness of the program in changing their behavior or their students’ behavior. Missing Data. As a result of this longitudinal data collection procedure involving teacher level and student level data across three time periods, missing data analyses were taken into consideration before data analyses were conducted. As shown in Table 8 below, the percentage of missing data across the study variables ranged from 0% to 19.67%. Generally, if 5% or less of cases within a given variable is missing, then it is acceptable to consider case deletion and not utilize statistical techniques to address the missing data (Graham, 2009). For the HLM analyses utilized for research questions two through four and the corresponding subparts, multiple imputation was used in order to address the missing data. Five imputations were run in order to better estimate the missing data and these imputations were entered into the HLM statistical program when running the two and three level models. For research questions one and five, case deletion was used for missing data. 120 Table 8 Missing Data per Teacher and Student Level Variables Variables N % missing data Time1 Time2 Time3 Teacher Strategy Questionnaire Frequency of Positive Strategy Use 31 0 0 6.45 Perception of Usefulness of Positive Strategy 31 3.23 0 9.68 Frequency of Limit Setting Use 31 0 0 6.45 Perception of Usefulness of Limit Setting 31 0 0 6.45 Frequency of Social-Emotional Use 31 0 0 3.23 Perception of Usefulness of Social-Emotional 31 0 0 6.45 Frequency of Proactive Strategy Use 31 0 0 3.23 Perception of Usefulness of Proactive Strategy 31 3.23 0 3.23 Frequency of Coaching Use 31 0 0 3.23 Perception of Usefulness of Coaching 31 0 0 3.23 Frequency of Inappropriate Strategy Use 31 0 0 3.23 Perception of Usefulness of Inappropriate Strategy 31 3.23 0 3.23 Planning and Support Strategy 31 3.23 0 3.23 Confidence in Managing Classroom Behaviors 31 3.23 19.35 12.90 Positive Approaches with Parents 31 0 0 3.23 Classroom Atmosphere Measure 14 0 0 0 Student Teacher Relationship Scale Total STRS Score 69 0 0 1.45 Conflict 69 0 0 1.45 Closeness 69 0 0 1.45 Dependency 69 0 0 1.45 Direct Behavior Rating Teacher-Student Interaction 28 0 10.71 10.71 Positive Social Behavior 28 0 10.71 10.71 Peer Interactions 28 0 10.71 10.71 All Student BASC-2 (Teacher Ratings) Externalizing Composite Score 443 0 7.00 Internalizing Composite Score 443 0 7.00 Social Skills Score 443 0 7.00 Target Student BASC-2 (Teacher Ratings) Externalizing Composite Score 62 0 3.23 8.06 Internalizing Composite Score 62 0 3.23 8.06 Social Skills Score 62 0 3.23 8.06 BASC-2 (Parent Ratings) Externalizing Composite Score 61 4.92 11.48 19.67 Internalizing Composite Score 61 4.92 11.48 19.67 Social Skills Score 61 4.92 11.48 19.67 121 CHAPTER 4: RESULTS Question One. To answer the first research question of “Will there be a difference in the procedural integrity of treatment adherence between the two groups, as measured through the teacher-reported log of hours spent on professional development activities for classroom management strategies?”, an independent samples t-test analysis was used in order to examine the differences between the two groups at the post-test rating. The Statistical Package for the Social Sciences (SPSS version 22) was used for this analysis. Total professional development hours for the study. Five cases were excluded from analysis of this variable due to not completing the form or being an outlier case due to not accurately following the instructions on the form for logging hours for this training. The Levene’s Test for Equality of Variance indicated a significance level of p= 0.067, indicating that the group variances were homogeneous (Table 9). The results of the analysis indicated that there was a statistically significant difference between the teacher-reported numbers of hours spent on professional development for this study (i.e., reading and/or training) between the IY training group (M= 44.89, SD = 13.77) and the reading group (M= 16.77, SD = 4.93) conditions; t (24)= -7.145, p= 0.000. These results suggest that the IY training group spent significantly more time engaged in professional development related to this training study than the reading group, as would be expected through the integrity of treatment adherence. Table 9 Results of t-test for professional development (PD) log of hours Group 95% CI IY-TCM Reading M SD n M SD n PD log of 44.89 13.77 12 16.77 4.93 14 -36.24, hours -19.99 122 t -7.145 df 24 Question Two. To answer the second research question of “Will there be a difference in the mean scores over time for the teachers’ use and perceptions of classroom management strategies for the teachers across the two group conditions from pretest to the midpoint to posttest?”, a two-level hierarchical model was used to analyze the multiple time points nested within the teacher level variables. The HLM 7 statistical software was used for these analyses, and the descriptive statistics for these teacher variables are included in Table 10 found within the appendices. The county dummy variable was included within the teacher level for these analyses and all of the following HLM analyses. The two level model was run for each of the teacher level variables within the TSQ. TSQ Summary Scales. HLM analysis tables for the TSQ Summary Scales are included in Tables 11-17. 123 Table 10 Descriptive Data for Teacher Classroom Management Strategies and Classroom Atmosphere Ratings at Three Time Points Variable IY-TCM N = 16 M (SD) Freq. - Positive Strategy Pretest 82.19 (10.21) Midpoint 98.50 (11.32) Posttest 99.58 (9.88) Perception - Positive Strategy Pretest 90.94 (14.32) Midpoint 105.94 (14.85) Posttest 109.14 (10.51) Freq. - Limit Setting Pretest 14.94 (2.67) Midpoint 18.81 (2.61) Posttest 18.84 (2.61) Perception - Limit Setting Pretest 15.19 (2.88) Midpoint 18.81 (2.88) Posttest 19.85 (2.17) Freq. - Social-emotional strategy Pretest 21.31 (5.53) Midpoint 25.63 (4.15) Posttest 25.75 (3.99) Perception - Social-emotional Pretest 25.19 (6.40) Midpoint 28.19 (4.65) Posttest 29.14 (4.31) Freq. - Proactive Strategy Pretest 26.69 (5.67) Midpoint 30.69 (3.93) Posttest 31.25 (3.21) Perception - Proactive Strategy Pretest 28.56 (6.04) Midpoint 32.75 (4.61) Posttest 34.63 (3.03) M = mean score, SD = standard deviation Reading N = 15 M (SD) Variable Freq. – Coach/Praise/Incentive Pretest Midpoint Posttest Perception–Coach/Praise/Incentive Pretest Midpoint Posttest Freq. – Inappropriate Strategy Pretest Midpoint Posttest Perception - Inappropriate Strategy Pretest Midpoint Posttest Planning and Support Strategy Pretest Midpoint Posttest Confidence in Managing Behavior Pretest Midpoint Posttest Positive Approaches with Parents Pretest Midpoint Posttest Classroom Atmosphere Pretest Midpoint Posttest 79.87 (13.76) 83.53 (13.22) 89.92 (14.46) 86.92 (13.50) 91.73 (14.14) 94.77 (11.96) 16.33 (3.24) 16.87 (4.49) 17.15 (2.73) 16.33 (1.76) 17.20 (3.28) 18.13 (2.07) 19.93 (6.45) 20.80 (5.27) 24.37 (7.22) 22.20 (6.97) 23.73 (4.33) 24.27 (6.31) 25.13 (6.01) 26.20 (4.23) 27.99 (3.72) 27.60 (5.42) 28.73 (5.11) 29.41 (4.56) 124 IY-TCM N = 16 M (SD) Reading N = 15 M (SD) 19.25 (4.34) 23.38 (5.81) 23.94 (5.82) 18.47 (3.58) 19.67 (5.14) 20.24 (4.78) 22.00 (5.03) 26.19 (6.86) 26.75 (4.81) 20.53 (3.80) 22.07 (6.44) 23.00 (5.86) 14.44 (2.61) 13.50 (2.34) 14.13 (2.90) 14.27 (2.40) 14.27 (2.69) 13.39 (2.41) 15.00 (4.69) 18.69 (7.39) 18.94 (7.73) 17.07 (5.50) 15.53 (4.41) 19.21 (6.78) 23.49 (8.06) 35.94 (6.79) 36.63 (6.02) 20.00 (9.03) 28.53 (10.51) 31.52 (8.69) 16.51 (2.36) 16.28 (3.99) 17.63 (4.21) 17.07 (1.44) 17.65 (1.06) 17.31 (2.09) 36.38 (4.43) 39.19 (3.58) 40.13 (3.98) 35.33 (8.58) 36.80 (7.84) 39.79 (8.29) 1.55 (0.10) 1.53 (0.21) 1.51 (0.25) 1.70 (0.16) 1.58 (0.23) 1.61 (0.23) Table 11 Frequency of Positive Strategy Use Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 63.3451 6.2495 10.136 25 <0.001 COUNTY, β01 -5.1812 3.1645 -1.637 25 0.114 GROUP, β02 7.2378 3.5610 2.033 25 0.053 PSTR_USE, β03 0.2991 0.1337 2.237 25 0.034 INAPP_FREQ, β04 0.9234 0.8556 1.079 25 0.291 INAPP_USE, β05 0.0163 0.4649 0.035 25 0.972 TIME slope, π1 INTRCPT2, β10 20.1408 6.3613 3.166 60 0.002 TIME2 slope, π2 INTRCPT2, β20 -3.3158 1.4839 -2.235 60 0.029 Final estimation of variance components Standard Variance d.f Random Effect χ2 p-value Deviation Component . INTRCPT1, r0 8.7678 76.8741 25 120.4241 <0.001 level-1, e 7.7604 60.2241 Table 12 Perception of usefulness of positive strategy Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 66.6746 6.5474 10.183 24 COUNTY, β01 0.5278 4.5033 0.117 24 GROUP, β02 10.4592 3.8125 2.743 24 GRPXCOUNTY, β03 -1.6924 6.4351 -0.263 24 PSTR_FREQ, β04 0.6226 0.1151 5.410 24 INAPP_FREQ, β05 -0.0378 0.7348 -0.051 24 INAPP_USE, β06 0.2320 0.2869 0.808 24 TIME slope, π1 INTRCPT2, β10 20.5083 7.1691 2.861 60 TIME2 slope, π2 INTRCPT2, β20 -3.4566 1.6906 -2.045 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 8.0303 64.4864 24 85.2915 <0.001 level-1, e 8.6827 75.3885 125 p-value <0.001 0.908 0.011 0.795 <0.001 0.959 0.427 0.006 0.045 Table 13 Frequency of inappropriate strategy use Standard Fixed Effect Coefficient error INTRCPT1, π0 INTRCPT2, β00 14.3263 1.7000 COUNTY, β01 1.7877 0.8066 GROUP, β02 1.1533 0.8734 GRPXCOUNTY, β03 -1.9388 1.0444 PSTR_FREQ, β04 -0.0040 0.0259 INAPP_USE, β05 0.1597 0.0726 TIME slope, π1 INTRCPT2, β10 -1.0687 1.8689 TIME2 slope, π2 INTRCPT2, β20 0.1949 0.4562 Final estimation of variance components Standard Variance Random Effect d.f. Deviation Component INTRCPT1, r0 1.2469 1.5547 25 level-1, e 2.1466 4.6079 t-ratio Approx. d.f. p-value 8.427 2.216 1.320 -1.856 -0.156 2.201 25 25 25 25 25 25 <0.001 0.036 0.199 0.075 0.877 0.037 -0.572 60 0.570 0.427 60 0.671 χ2 50.3109 p-value 0.002 Table 14 Perception of usefulness of inappropriate strategy Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 16.7919 4.0306 4.166 25 <0.001 COUNTY, β01 -3.1943 1.6759 -1.906 25 0.068 GROUP, β02 0.7950 2.7217 0.292 25 0.773 GRPXCOUNTY, β03 -0.5983 3.0985 -0.193 25 0.848 PSTR_FREQ, β04 0.0027 0.0883 0.030 25 0.976 INAPP_FREQ, β05 0.8783 0.2820 3.114 25 0.005 TIME slope, π1 INTRCPT2, β10 -0.0005 3.9785 -0.000 60 1.000 TIME2 slope, π2 INTRCPT2, β20 0.3892 1.0092 0.386 60 0.701 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 4.2924 18.4248 25 102.9314 <0.001 level-1, e 4.2061 17.6911 126 Table 15 Frequency of planning and support strategy Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 -0.9622 4.8032 -0.200 25 COUNTY, β01 3.1970 2.0916 1.528 25 GROUP, β02 7.0213 2.8690 2.447 25 GRPXCOUNTY, β03 -5.9187 3.4992 -1.691 25 PSTR_FREQ, β04 0.3419 0.0862 3.968 25 INAPP_FREQ, β05 0.4749 0.3467 1.370 25 TIME slope, π1 INTRCPT2, β10 23.6247 4.6186 5.115 60 TIME2 slope, π2 INTRCPT2, β20 -4.3682 1.0253 -4.260 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 4.8471 23.4942 25 87.2821 <0.001 level-1, e 5.3117 28.2140 Table 16 Confidence in managing classroom behavior Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 17.6468 1.8081 9.760 24 COUNTY, β01 -0.7327 0.7675 -0.955 24 GROUP, β02 -0.5416 0.8031 -0.674 24 GRPXCOUNTY, β03 0.0914 1.5771 0.058 24 PSTR_FREQ, β04 0.0041 0.0345 0.120 24 INAPP_FREQ, β05 -0.2177 0.2341 -0.930 24 PLAN, β06 0.0254 0.0397 0.642 24 TIME slope, π1 INTRCPT2, β10 -0.4896 1.9332 -0.253 60 TIME2 slope, π2 INTRCPT2, β20 0.2122 0.4447 0.477 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 2.0184 4.0740 24 85.5072 <0.001 level-1, e 2.1683 4.7014 127 p-value 0.843 0.139 0.022 0.103 <0.001 0.183 <0.001 <0.001 p-value <0.001 0.349 0.506 0.954 0.906 0.362 0.527 0.801 0.635 Table 17 Frequency of positive approaches with parents strategy use Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 34.3335 2.516550 13.643 24 <0.001 COUNTY, β01 -3.6190 3.291452 -1.100 24 0.282 GROUP, β02 1.0495 1.9180 0.547 24 0.589 GRPXCOUNTY, β03 0.6754 3.7896 0.178 24 0.860 PSTR_FREQ, β04 0.1831 0.0798 2.295 24 0.031 INAPP_FREQ, β05 -0.7591 0.4097 -1.853 24 0.076 PLAN, β06 -0.0308 0.1069 -0.288 24 0.776 TIME slope, π1 INTRCPT2, β10 2.6032 2.5585 1.017 60 0.313 TIME2 slope, π2 INTRCPT2, β20 -0.1473 0.6353 -0.232 60 0.817 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 5.1400 26.4195 24 212.8413 <0.001 level-1, e 3.1918 10.1873 Frequency of positive strategy use. The intraclass correlation for this model was 0.4697, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components for at level two as the following: standard deviation 9.8300; variance component 96.6283; degrees of freedom = 30; chi square = 109.4537; p <0.001. The conditional model accounted for an additional 20% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 10.4458; variance component 109.1157. The conditional model accounted for an additional 45% of the variance at level one. Results of the model indicated that the group assignment (coefficient 7.2378, p= 0.053) was a marginally significant predictor and the teacher reported perception of usefulness of the positive strategies (coefficient 0.2991, p= 0.034) was a significant predictor of the teacher reported frequency of positive strategy use (see Table 11). For the group assignment variable, 128 the ratings for the IY training group were marginally significant and increased in comparison to the reading group. In addition, the TIME (coefficient 20.1527, p= 0.002) and TIME2 (coefficient -3.3197, p= 0.029) variables were significant predictors, indicating that the scores significantly increased over time and followed a quadratic (nonlinear) pattern. When a group by county variable was added to the model, that predictor variable was nonsignificant (p= 0.608) and affected the other variables. Therefore, it was not included in the model. Perception of usefulness of positive strategies. The intraclass correlation for this model was 0.4935, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components for at level two as the following: standard deviation 10.8683; variance component 118.1192; degrees of freedom = 30; chi square = 117.2724; p <0.001. The conditional model accounted for an additional 45% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 11.0114; variance component 121.2503. The conditional model accounted for an additional 38% of the variance level one. Results of the model indicated that the group assignment (coefficient 10.4592, p= 0.011) and frequency of positive strategy use (coefficient 0.6226, p <0.001) were significant predictors of the teacher reported perception of usefulness of the positive strategies (see Table 12). For the group assignment variable, the ratings for the IY training group were significantly different and increased in comparison to the reading group. In addition, the TIME (coefficient 20.5083, p= 0.006) and TIME2 (coefficient -3.4566, p= 0.045) variables were significant predictors, indicating that the scores significantly increased over time and followed a quadratic (nonlinear) pattern. 129 Frequency of inappropriate strategy use. The intraclass correlation for this model was 0.2952, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components for at level two as the following: standard deviation 1.3815; variance component 1.9086; degrees of freedom = 30; chi square = 67.6900; p <0.001. The conditional model accounted for an additional 19% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 2.1348; variance component 4.5575. The conditional model did not account for additional variance at level one. Results of the model indicated that the county (coefficient 1.7877, p= 0.036) and perception of usefulness of inappropriate strategies (coefficient 0.1597, p= 0.037) were significant predictors of teacher reported frequency of Inappropriate Strategy Use over time (see Table 13). For the county variable, county one was significantly different and increased in comparison to the county two. The group assignment was not a significant predictor. Perception of usefulness of inappropriate strategies. The intraclass correlation for this model was 0.5142, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 4.5531; variance component 20.7304; degrees of freedom = 30; chi square = 124.8173; p <0.001. The conditional model accounted for an additional 11% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 4.4259; variance component 19.5883. The conditional model accounted for an additional 10% of the variance at level one. 130 Results of the model indicated that the frequency of Inappropriate Strategy use (coefficient 0.8783, p= 0.005) was a significant predictor of the teacher reported perception of usefulness of inappropriate strategies over time (see Table 14). Group assignment was not a significant predictor of this variable. Frequency of planning and support strategies. The intraclass correlation for this model was 0.3094, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 5.6610; variance component 32.0471; degrees of freedom = 30; chi square = 70.1855; p <0.001. The conditional model accounted for an additional 27% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 8.4578; variance component 71.5341. The conditional model accounted for an additional 61% of the variance at level one. Results of the model indicated that the group assignment (coefficient 7.0213, p= 0.022) and frequency of positive strategy use (coefficient 0.3419, p <0.001) were significant predictors of teacher reported frequency of planning and support strategies (see Table 15). For the group assignment variable, the ratings for the IY training group were significantly different and increased in comparison to the reading group. The TIME (coefficient 23.6247, p<0.001) and TIME2 (coefficient -4.3682, p<0.001) variables were significant predictors, indicating that the scores significantly increased over time and followed a quadratic (nonlinear) pattern. Confidence in managing classroom behavior. The intraclass correlation for this model was 0.4327, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the 131 following: standard deviation 1.8930; variance component 3.5833; degrees of freedom = 30; chi square = 97.7500; p <0.001. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 2.1677; variance component 4.6988. The conditional model did not account for additional variance. Results of the model indicated that there were no significant predictors for the teacher reported Confidence in Managing Classroom Behavior (see Table 16). Frequency of positive approaches with parents strategy use. The intraclass correlation for this model was 0.6784, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 5.4188; variance component 29.3634; degrees of freedom = 30; chi square = 220.3726; p <0.001. The conditional model accounted for an additional 10% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 3.7314; variance component 13.9230. The conditional model accounted for an additional 27% of the variance at level one. Results of the model indicated that the total frequency of positive strategy use (coefficient 0.1831, p= 0.031) was a significant predictor of teacher reported frequency of positive approaches with parents strategy use over time (see Table 17). The group assignment was not a significant predictor. TSQ Subscales of the Positive Strategies Scale. The HLM analysis tables for the TSQ Subscales of the Positive Strategies Scale are included in Tables 18-25. 132 Table 18 Frequency of limit-setting strategy use Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 10.1054 1.6644 6.071 23 COUNTY, β01 1.7368 1.3552 1.282 23 GROUP, β02 2.1110 1.3080 1.614 23 GRPXCOUNTY, β03 -2.8469 1.7543 -1.623 23 PSTR_FREQ, β04 0.0648 0.0325 1.996 23 LIMSET_USE, β05 0.2100 0.1696 1.238 23 INAPP_FREQ, β06 0.0008 0.1722 0.004 23 INAPP_USE, β07 0.0279 0.0916 0.305 23 TIME slope, π1 INTRCPT2, β10 5.4234 2.0355 2.664 60 TIME2 slope, π2 INTRCPT2, β20 -1.0551 0.4910 -2.149 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 2.2928 5.2570 23 97.1792 <0.001 level-1, e 2.2096 4.8821 p-value <0.001 0.213 0.120 0.118 0.058 0.228 0.997 0.763 0.010 0.036 Table 19 Perception of usefulness of limit-setting strategies Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 11.9981 1.9485 6.158 23 <0.001 COUNTY, β01 -0.6339 0.8554 -0.741 23 0.466 GROUP, β02 0.7449 0.8273 0.900 23 0.377 GRPXCOUNTY, β03 0.3076 1.1360 0.271 23 0.789 PSTR_FREQ, β04 0.0573 0.0258 2.224 23 0.036 LIMSET_FREQ, β05 0.1918 0.0813 2.359 23 0.027 INAPP_FREQ, β06 0.0174 0.1218 0.143 23 0.888 INAPP_USE, β07 0.0116 0.0710 0.163 23 0.872 TIME slope, π1 INTRCPT2, β10 4.2042 2.2412 1.876 60 0.066 TIME2 slope, π2 INTRCPT2, β20 -0.6380 0.5445 -1.172 60 0.246 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 1.1504 1.3235 23 40.2583 0.014 level-1, e 2.3010 5.2946 133 Table 20 Frequency of social-emotional strategy use Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 16.0820 2.8054 5.733 23 COUNTY, β01 0.4957 1.3427 0.369 23 GROUP, β02 1.0437 1.3456 0.776 23 GRPXCOUNTY, β03 1.0416 1.9906 0.523 23 PSTR_FREQ, β04 0.2684 0.0743 3.613 23 SOCE_USE, β05 0.0196 0.1441 0.136 23 INAPP_FREQ, β06 -0.1460 0.3354 -0.435 23 INAPP_USE, β07 -0.1175 0.1189 -0.988 23 TIME slope, π1 INTRCPT2, β10 3.9752 3.0408 1.307 60 TIME2 slope, π2 INTRCPT2, β20 -0.4433 0.7456 -0.595 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 2.3531 5.5371 23 49.4622 0.001 level-1, e 3.8034 14.4658 Table 21 Perception of usefulness of social-emotional strategies Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 18.2431 2.6661 6.843 23 COUNTY, β01 -0.0377 1.6678 -0.023 23 GROUP, β02 3.6997 1.7287 2.140 23 GRPXCOUNTY, β03 -0.9815 2.6010 -0.377 23 PSTR_FREQ, β04 0.2361 0.1022 2.310 23 SOCE_FREQ, β05 0.0242 0.2586 0.093 23 INAPP_FREQ, β06 -0.4277 0.3179 -1.346 23 INAPP_USE, β07 -0.1382 0.1604 -0.862 23 TIME slope, π1 INTRCPT2, β10 4.6444 2.7596 1.683 60 TIME2 slope, π2 INTRCPT2, β20 -0.7847 0.6560 -1.196 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 3.4005 11.5635 23 88.4500 <0.001 level-1, e 3.4870 12.1593 134 p-value <0.001 0.715 0.446 0.606 0.001 0.893 0.667 0.333 0.196 0.554 p-value <0.001 0.982 0.043 0.709 0.030 0.926 0.192 0.398 0.098 0.236 Table 22 Frequency of proactive strategy use Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. For INTRCPT1, π0 INTRCPT2, β00 20.0245 2.4324 8.233 23 <0.001 COUNTY, β01 1.1691 1.0342 1.130 23 0.270 GROUP, β02 2.9300 1.2236 2.394 23 0.025 GRPXCOUNTY, β03 -0.6068 1.4214 -0.427 23 0.673 PSTR_FREQ, β04 0.2490 0.0343 7.264 23 <0.001 PROACT_USE, β05 -0.0178 0.0733 -0.243 23 0.810 INAPP_FREQ, β06 -0.2176 0.1706 -1.276 23 0.215 INAPP_USE, β07 0.1275 0.0975 1.307 23 0.204 For TIME slope, π1 INTRCPT2, β10 4.7508 2.3923 1.986 60 0.052 For TIME2 slope, π2 INTRCPT2, β20 -0.7234 0.5675 -1.275 60 0.207 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 0.7711 0.5946 23 25.6621 0.317 level-1, e 3.4724 12.0576 Table 23 Perception of usefulness of proactive strategy Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 22.5547 2.5382 8.886 23 COUNTY, β01 -0.5233 1.4766 -0.354 23 GROUP, β02 2.6989 1.3078 2.064 23 GRPXCOUNTY, β03 0.5772 2.0484 0.282 23 PSTR_FREQ, β04 0.0889 0.1069 0.832 23 PROACT_FREQ, β05 0.2473 0.2433 1.017 23 INAPP_FREQ, β06 -0.2037 0.3146 -0.648 23 INAPP_USE, β07 0.0993 0.1055 0.941 23 TIME slope, π1 INTRCPT2, β10 4.9446 2.5831 1.914 60 TIME2 slope, π2 INTRCPT2, β20 -0.7358 0.6095 -1.207 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 2.2887 5.2381 23 47.1501 0.002 level-1, e 3.8622 14.9168 135 p-value <0.001 0.726 0.051 0.781 0.414 0.320 0.524 0.356 0.060 0.232 Table 24 Frequency of coaching strategy use Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 12.4213 2.1260 5.843 23 <0.001 COUNTY, β01 1.3263 1.7989 0.737 23 0.468 GROUP, β02 2.6006 1.5987 1.627 23 0.117 GRPXCOUNTY, β03 -1.2597 2.3130 -0.545 23 0.591 PSTR_FREQ, β04 0.0779 0.0445 1.752 23 0.093 COACH_USE, β05 0.2075 0.1734 1.197 23 0.244 INAPP_FREQ, β06 0.7763 0.3043 2.551 23 0.018 INAPP_USE, β07 0.0061 0.1014 0.060 23 0.953 TIME slope, π1 INTRCPT2, β10 5.8955 2.2830 2.582 60 0.012 TIME2 slope, π2 INTRCPT2, β20 -1.0619 0.5357 -1.983 60 0.052 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 3.1227 9.7511 23 93.9458 <0.001 level-1, e 3.0778 9.4731 Table 25 Perception of usefulness of coaching strategy Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 14.2153 2.5887 5.491 23 COUNTY, β01 1.4334 1.5237 0.941 23 GROUP, β02 3.3779 1.5705 2.151 23 GRPXCOUNTY, β03 -1.4131 2.2591 -0.626 23 PSTR_FREQ, β04 0.0160 0.0595 0.269 23 COACH_FREQ, β05 0.7092 0.2046 3.467 23 INAPP_FREQ, β06 0.1946 0.3023 0.644 23 INAPP_USE, β07 0.2039 0.0984 2.072 23 TIME slope, π1 INTRCPT2, β10 6.1218 2.9427 2.080 60 TIME2 slope, π2 INTRCPT2, β20 -1.0729 0.7030 -1.526 60 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 2.8434 8.0848 23 71.3803 <0.001 level-1, e 3.3936 11.5166 136 p-value <0.001 0.357 0.042 0.538 0.790 0.002 0.526 0.050 0.042 0.132 Frequency of limit setting strategy use. The intraclass correlation for this model was 0.4203, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 2.1786; variance component 4.7461; degrees of freedom = 30; chi square = 95.1249; p <0.001. The conditional model did not account for additional variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 2.5584; variance component 6.5452. The conditional model accounted for an additional 25% of the variance at level one. Results of the model indicated that the TIME (coefficient 5.4234, p= 0.010) and TIME2 (coefficient -1.0551, p= 0.036) variables were significant predictors of teacher reported frequency of Limit-Setting strategy use over time (see Table 18), indicating that the scores significantly increased over time and followed a quadratic (nonlinear) pattern. The group assignment was not a significant predictor of this variable. Perception of usefulness of limit setting strategies. The intraclass correlation for this model was 0.0957, indicating marginal variance between teachers and a need for a two-level model due to the multiple time points. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 0.9195; variance component 0.8455; degrees of freedom = 30; chi square = 39.4261; p= 0.116. The conditional model did not account for additional variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 2.8266; variance component 7.9898. The conditional model accounted for an additional 34% of the variance at level one. 137 Results of the model indicated that the total frequency of positive strategy use (coefficient 0.0573, p= 0.036) and frequency of limit setting strategy use (coefficient 0.1918, p=0.027) were significant predictors of the teacher reported perception of usefulness of limit setting strategies (see Table 19). The group assignment was not a significant predictor. Frequency of social emotional strategy use. The intraclass correlation for this model was 0.4599, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 4.0128; variance component 16.1028; degrees of freedom = 30; chi square = 106.5307; p <0.001. The conditional model accounted for an additional 66% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 4.3490; variance component 18.9135. The conditional model accounted for an additional 24% of the variance at level two. Results of the model indicated that the total frequency of positive strategy use (coefficient 0.2684, p= 0.001) was a significant predictor of the teacher reported frequency of social emotional strategy use (see Table 20). The group assignment was not a significant predictor. Perception of usefulness of social emotional strategies. The intraclass correlation for this model was 0.6078, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 4.6980; variance component 22.0715; degrees of freedom = 30; chi square = 169.0349; p <0.001. The conditional model accounted for an additional 48% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 3.7740; variance 138 component 14.2433. The conditional model accounted for an additional 15% of the variance at level one. Results of the model indicated that the group assignment (coefficient 3.6997, p= 0.043) and total frequency of positive strategy use (coefficient 0.2361, p= 0.030) were significant predictors of the teacher reported perception of usefulness of Social Emotional strategies (see Table 21). For the group assignment variable, the ratings for the IY training group were significantly different and increased in comparison to the reading group. Frequency of proactive strategy use. The intraclass correlation for this model was 0.3906, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 3.1443; variance component 9.8868; degrees of freedom = 30; chi square = 87.5267; p <0.001. The conditional model accounted for an additional 94% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 3.9273; variance component 15.4234. The conditional model accounted for an additional 22% of the variance at level one. Results indicated that the group assignment (coefficient 2.9300, p= 0.025) and total frequency of positive strategy use (coefficient 0.2490, p<0.001) were significant predictors of teacher reported frequency of Proactive Strategy Use over time (see Table 22). For the group assignment variable, the ratings for the IY training group were significantly different and increased in comparison to the reading group. The TIME variable (coefficient 4.7508, p= 0.052) was a marginally significant predictor, indicated that the scores increased over time approaching significance, although this should be interpreted with caution. 139 Perception of usefulness of proactive strategies. The intraclass correlation for this model was 0.3696, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 3.3062; variance component 10.9312; degrees of freedom = 30; chi square = 82.6858; p <0.001. The conditional model accounted for an additional 52% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 4.3180; variance component 18.6454. The conditional model accounted for an additional 20% of the variance at level one. Results of the model indicated that the group assignment (coefficient 2.6989, p= 0.051) was a marginally significant predictor of the teacher reported perception of usefulness of Proactive Strategies over time (see Table 23). For the group assignment variable, the ratings for the IY training group were marginally significantly different and increased in comparison to the reading group, although this should be interpreted with caution. Frequency of coaching strategy use. The intraclass correlation for this model was 0.5690, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 4.0229; variance component 16.1837; degrees of freedom = 30; chi square = 148.7746; p <0.001. The conditional model accounted for an additional 40% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 3.5009; variance component 12.2562. The conditional model accounted for an additional 23% of the variance at level one. 140 Results indicated that frequency of inappropriate strategy use (coefficient 0.7763, p= 0.018) was a significant predictor of the frequency of coaching strategy use over time (see Table 24). The group assignment was not a significant predictor of this variable. The TIME (coefficient 5.8955, p= 0.012) and TIME2 (coefficient -1.0619, p= 0.052) variables were significant and marginally significant predictors, indicating that the scores significantly increased over time and generally followed a quadratic (nonlinear) pattern. Perception of usefulness of coaching strategies. The intraclass correlation for this model was 0.5763, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 4.4993; variance component 20.2440; degrees of freedom = 30; chi square = 152.3501; p <0.001. The conditional model accounted for an additional 60% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 3.8575; variance component 14.8806. The conditional model accounted for an additional 23% of the variance at level one. Results of the model indicated that the group assignment (coefficient 3.3779, p= 0.042), frequency of coaching strategy use (coefficient 0.7092, p= 0.002) and perception of usefulness of inappropriate strategies (coefficient 0.2039, p= 0.050) were significant predictors of teacher reported perception of usefulness of Coaching Strategies over time (see Table 25). For the group assignment variable, the ratings for the IY training group were significantly different and increased in comparison to the reading group. The TIME variable (coefficient 6.1218, p= 0.042) was also a significant predictor, indicating that the scores significantly increased over time. 141 Summary of teacher level variable outcomes. Overall, there were significant group differences between the two groups over time for the following variables: Perception of Usefulness of Positive Strategies, Perception of Usefulness of Social Emotional Strategies, Frequency of Proactive Strategy Use, Perception of Usefulness of Proactive Strategies, Perception of Usefulness of Coaching Strategies, and Frequency of Planning and Support Strategies. The results were marginally significant, and therefore interpreted with caution, for Frequency of Positive Strategy Use and Perception of Usefulness of the Proactive Strategies. For each of these variables, the scores for the IY training group increased over time in comparison to the reading group. There was a significant difference between counties for frequency of inappropriate strategy use, with county one being significantly different and increasing in comparison to the county two, indicating an increase in use of inappropriate strategies. It is unclear if this difference could be attributed to county differences at pretest, length of time of the intervention (county 1 = 6 months, county 2 = 3 months), or an additional factor. Variables that significantly increased over time from pretest to posttest were frequency of positive strategy use, perception of usefulness of positive strategies, frequency of limit setting strategy use, frequency of coaching strategy use, perception of usefulness of coaching strategies, and planning and support strategies. The increase over time was marginally significant, and therefore interpreted with caution, for the frequency of proactive strategy use. These patterns were nonlinear (quadratic) for the following variables: frequency of positive strategy use, perception of usefulness of positive strategies, frequency of limit setting strategy use, frequency of coaching strategy use, and planning and support strategies. 142 Question Three. To answer the third research question of “Will there be a difference in the mean scores over time for the classroom atmosphere, percentage of time in which teachers are involved in positive interactions with target students based on Direct Behavior Rating (DBR) data, and the teacher-student relationships on the STRS for teachers across the two group conditions from pretest to the midpoint to post-test?”, different models were needed for each variable. A two-level hierarchical model was used to analyze the multiple time points nested within the teacher level variables for the classroom atmosphere measure. Three-level hierarchical models were used for the DBR teacher-student interactions and the STRS variables, due to the fact that those measures were collected between teachers and target students, therefore necessitating a third level of student data. The HLM 7 statistical software was used for these analyses, and the descriptive statistics for these variables are included in Tables 10 (included earlier), 26, and 27. The HLM analyses are included in Tables 28-33 on the following pages. 143 Table 26 Descriptive Data for Target Student Observation Data for Teacher-Student Interactions, Peer Interactions, and Positive Social Behavior at Three Time Points IY-TCM Reading M (SD) M (SD) Teacher-Student Interaction Pretest 4.20 (2.50) 2.96 (2.96) Midpoint 3.49 (2.00) 3.05 (2.38) Posttest 3.97 (2.54) 3.44 (2.14) Peer Interaction Pretest 5.33 (1.46) 3.71 (2.26) Midpoint 5.11 (2.56) 4.73 (2.12) Posttest 5.47 (2.97) 5.83 (2.59) Positive Social Behavior Pretest 5.27 (1.84) 3.79 (2.21) Midpoint 5.14 (2.67) 5.43 (2.50) Posttest 5.88 (2.95) 6.32 (2.47) Table 27 Descriptive Data for Student-Teacher Relationship Scale Ratings at Three Time Points IY-TCM Reading N = 39 N = 30 M (SD) M (SD) Conflict Pretest 21.87 (9.58) 25.97 (7.65) Midpoint 20.95 (8.86) 25.33 (8.83) Posttest 20.10 (8.56) 25.21 (9.69) Closeness Pretest 42.80 (5.01) 41.87 (6.08) Midpoint 43.44 (6.79) 43.80 (5.74) Posttest 44.26 (6.63) 44.12 (6.38) Dependency Pretest 10.28 (3.24) 11.90 (3.67) Midpoint 10.23 (3.44) 11.70 (4.18) Posttest 9.72 (3.30) 11.39 (3.60) STRS Total Pretest 112.64 (13.16) 106.00 (10.18) Midpoint 114.26 (13.56) 108.77 (13.87) Posttest 116.44 (13.61) 109.46 (15.32) 144 Table 28 Classroom atmosphere measure Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 1.7765 0.1190 14.924 10 <0.001 GROUP, β01 -0.0813 0.0948 -0.857 10 0.411 PSTR_FREQ, β02 -0.0057 0.0064 -0.883 10 0.398 INAPP_FREQ, β03 -0.0022 0.0092 -0.239 10 0.816 TIME slope, π1 INTRCPT2, β10 -0.1486 0.1348 -1.103 26 0.280 TIME2 slope, π2 INTRCPT2, β20 0.0300 0.0332 0.905 26 0.374 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 0.1724 0.0297 10 71.2909 <0.001 level-1, e 0.1206 0.0146 145 Table 29 STRS conflict score Fixed Effect Coefficient Standard error t-ratio Approx. d.f. p-value INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 28.0913 2.6548 10.581 10 <0.001 GROUP, γ001 -2.3630 2.3618 -1.000 10 0.341 PSTR_FREQ, γ002 -0.2765 0.1645 -1.681 10 0.124 INAPP_FREQ, γ003 0.0685 0.2649 0.259 10 0.801 TARGNOT, β01 INTRCPT3, γ010 -3.3980 0.9190 -3.698 52 <0.001 CLOS, β02 INTRCPT3, γ020 -0.5114 0.1908 -2.681 52 0.010 DEP, β03 INTRCPT3, γ030 0.3577 0.2918 1.226 52 0.226 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 -1.1553 3.6980 -0.312 122 0.755 TIME2 slope, π2 INTRCPT2, β20 INTRCPT3, γ200 0.1194 0.9398 0.127 122 0.899 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 5.7841 33.4562 52 286.9249 <0.001 level-1, e 4.8702 23.7186 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 2.1671 4.6963 10 21.9630 0.015 146 Table 30 STRS closeness score Fixed Effect Coefficient Standard error t-ratio Approx. d.f. p-value INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 40.7782 3.3186 12.288 10 <0.001 GROUP, γ001 -0.2125 1.3894 -0.153 10 0.881 PSTR_FREQ, γ002 0.2731 0.1339 2.039 10 0.069 INAPP_FREQ, γ003 0.2640 0.2933 0.900 10 0.389 TARGNOT, β01 INTRCPT3, γ010 -0.1877 0.9640 -0.195 52 0.846 CONF, β02 INTRCPT3, γ020 -0.1538 0.0615 -2.500 52 0.016 DEP, β03 INTRCPT3, γ030 0.6756 0.1558 4.337 52 <0.001 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 2.0952 3.3006 0.635 122 0.527 TIME2 slope, π2 INTRCPT2, β20 INTRCPT3, γ200 -0.2974 0.8016 -0.371 122 0.711 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 2.7887 7.7768 52 133.3649 <0.001 level-1, e 4.0401 16.3223 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 1.8131 3.2871 10 31.1562 <0.001 147 Table 31 STRS dependency score Fixed Effect Coefficient Standard error t-ratio Approx. d.f. INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 11.3384 1.3230 8.570 10 GROUP, γ001 -0.9739 0.7769 -1.254 10 PSTR_FREQ, γ002 -0.0481 0.0588 -0.818 10 INAPP_FREQ, γ003 0.2207 0.1276 1.730 10 TARGNOT, β01 INTRCPT3, γ010 -0.0278 0.5766 -0.048 52 CONF, β02 INTRCPT3, γ020 0.1224 0.0432 2.835 52 CLOS, β03 INTRCPT3, γ030 0.1692 0.0637 2.658 52 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 0.3663 1.6943 0.216 122 TIME2 slope, π2 INTRCPT2, β20 INTRCPT3, γ200 -0.1608 0.4233 -0.380 122 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 1.7483 3.0564 52 146.4365 <0.001 level-1, e 2.3444 5.4962 Final estimation of level-3 variance components Standard Variance Random Effect Deviation Component INTRCPT1/INTRCPT2,u00 0.9098 0.8278 148 d.f. χ2 10 25.7191 p-value <0.001 0.238 0.432 0.114 0.962 0.007 0.010 0.829 0.705 p-value 0.004 Table 32 STRS total score Fixed Effect Coefficient Standard error t-ratio Approx. d.f. p-value INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 102.3534 4.8347 21.171 10 <0.001 GROUP, γ001 3.6664 3.4635 1.059 10 0.315 PSTR_FREQ, γ002 0.7025 0.2326 3.020 10 0.013 INAPP_FREQ, γ003 -0.0187 0.3588 -0.052 10 0.959 TARGNOT, β01 INTRCPT3, γ010 4.3092 1.5794 2.728 54 0.009 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 2.9251 6.5567 0.446 122 0.656 TIME2 slope, π2 INTRCPT2, β20 INTRCPT3, γ200 -0.2697 1.6481 -0.164 122 0.870 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 9.0309 81.5568 54 277.2067 <0.001 level-1, e 7.7873 60.6413 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 2.5047 6.2733 10 18.1993 0.051 149 Table 33 DBR teacher-student interaction Fixed Effect Coefficient Standard error t-ratio Approx. d.f. p-value INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 5.6993 1.9756 2.885 10 0.016 GROUP, γ001 1.6039 0.6971 2.301 10 0.044 PSTR_FREQ, γ002 -0.1734 0.0708 -2.448 10 0.034 INAPP_FREQ, γ003 0.2620 0.0961 2.727 10 0.021 RISK_INT, β01 INTRCPT3, γ010 -1.6028 0.9364 -1.712 9 0.121 RISK_BOT, β02 INTRCPT3, γ020 -2.4593 1.0666 -2.306 9 0.047 STRSTOT, β03 INTRCPT3, γ030 0.0136 0.0277 0.492 9 0.634 SOCIAL, β04 INTRCPT3, γ040 -0.2229 0.1154 -1.931 9 0.086 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 -1.8372 1.6435 -1.118 38 0.271 TIME2 slope, π2 INTRCPT2, β20 INTRCPT3, γ200 0.4698 0.3571 1.315 38 0.196 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 0.0677 0.0046 9 10.2935 0.327 level-1, e 1.9043 3.6262 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 0.9549 0.9118 10 34.3701 <0.001 150 Classroom atmosphere measure. The intraclass correlation for this model was 0.6404, indicating variance between teachers and a need for a two-level model. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 0.1612; variance component 0.0260; degrees of freedom = 13; chi square = 82.3086; p <0.001. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 0.1210; variance component 0.0146. The conditional model did not account for an additional variance at level one and level two. Results of the model indicated that there were no significant predictors of the classroom atmosphere ratings over time (see Table 28). STRS conflict score. The intraclass correlations for this model were 0.1601 (Level 3) and 0.5463 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 3.6312; variance component 13.1858; degrees of freedom 13; chi square 31.1447; p=0.004. The conditional model accounted for an additional 64% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 6.7087; variance component 45.0063; degrees of freedom 55; chi square 361.8958; p<.001. The conditional model accounted for an additional 26% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 4.9179; variance component 24.1856. The conditional model accounted for an additional 2% of the variance at level one. 151 Results of the model indicated that the target student identifier (coefficient -3.3980, p <0.001) was a significant predictor of conflict scores over time, with nontarget students being significantly different and having a lower score over time in comparison to the target students (see Table 29). The closeness score (coefficient -0.5114, p= 0.010) was also a significant predictor of conflict scores, signifying an inverse relationship. The group assignment was not a significant predictor for this variable. STRS closeness score. The intraclass correlations for this model were 0.2428 (Level 3) and 0.2990 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 3.0162; variance component 9.0976; degrees of freedom 13; chi square 50.6193; p<.001. The conditional model accounted for an additional 64% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 3.3472; variance component 11.2037; degrees of freedom 55; chi square 162.3170; p<.001. The conditional model accounted for an additional 31% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 4.1440; variance component 17.1730. The conditional model accounted for an additional 5% of the variance at level one. Results of the model indicated that conflict score (coefficient -0.1538, p= 0.016) and dependency score (coefficient 0.6756, p<0.001) were significant predictors of closeness scores over time (see Table 30). The group assignment was not a significant predictor for this variable. STRS dependency score. The intraclass correlations for this model were 0.3168 (Level 3) and 0.2525 (Level 2), indicating variance between teachers and students and a need for the three 152 level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 2.0264; variance component 4.1062; degrees of freedom 13; chi square 68.8307; p<0.001. The conditional model accounted for an additional 80% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 1.8090; variance component 3.2724; degrees of freedom 55; chi square 151.6076; p<.001. The conditional model accounted for an additional 7% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 2.3626; variance component 5.5819. The conditional model accounted for an additional 2% of the variance at level one. Results of the model indicated that conflict score (coefficient 0.1224, p= 0.007) and closeness score (coefficient 0.1692, p= 0.010) were significant predictors of dependency scores over time (see Table 31). The group assignment was not a significant predictor for this variable. STRS total score. The intraclass correlations for this model were 0.1942 (Level 3) and 0.4609 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 6.0065; variance component 36.0782; degrees of freedom 13; chi square 37.3643; p<.001. The conditional model accounted for an additional 83% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 9.2535; variance component 85.6275; degrees of freedom 55; chi square 275.7940; p<.001. The conditional model accounted for an additional 5% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the 153 following: standard deviation 8.0043; variance component 64.0680. The conditional model accounted for an additional 5% of the variance at level one. Results of the model indicated that total frequency of Positive Strategy Use score (coefficient 0.7025, p= 0.013), and the target student identifier (coefficient 4.3092, p= 0.009) were significant predictors of Total STRS scores over time (see Table 32). Nontarget students were significantly different and increased in scores over time in comparison to the target students. The group assignment was not a significant predictor for this variable. DBR teacher-student interaction. The intraclass correlations for this model were 0.3021 (Level 3) and 0.0706 (Level 2), indicating variance between teachers and a need for the three level model for the nested time periods. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 1.3643; variance component 1.8614; degrees of freedom 13; chi square 45.8422; p<0.001. The conditional model accounted for an additional 51% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 0.6597; variance component 0.4352; degrees of freedom 13; chi square 17.0163; p=0.198. The conditional model accounted for an additional 99% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 1.9657; variance component 3.8640. The conditional model accounted for an additional 6% of the variance at level one. Results of the model indicated that the group assignment (coefficient 1.6039, p= 0.044), frequency of positive strategy use (coefficient -0.1734, p= 0.034), frequency of inappropriate strategy use (coefficient 0.2620, p= 0.021), and target student risk for both internalizing and externalizing symptoms (coefficient -2.4593, p= 0.047) were significant predictors of the DBR 154 teacher-student interaction scores over time (see Table 33). For the group assignment variable, the IY training group was significantly different and had higher scores in comparison to the reading group. However, examination of the mean scores within Table 27 show that the mean scores for the IY-TCM group declined slightly over time, whereas the mean scores for the reading group increased. Despite this decline, the IY-TCM group remained a significant predictor in comparison to the other group. For the target student risk variable, students who were at risk for both internalizing and externalizing symptoms were significantly different and received teacher-student interaction scores that decreased over time in comparison to the other target students, indicating more negative teacher-student interactions. Summary of classroom level variable outcomes. There was a difference between the two groups over time for the DBR teacher-student interaction ratings. For this variable, the IY training group was significantly different in comparison to the reading group. Being a nontarget student was a significant predictor for the lower conflict scores over time in comparison to the target students, and higher scores for total STRS scores over time in comparison to the target students. Being a student who was at risk for both internalizing and externalizing symptoms was a significant predictor with lower scores for the teacher-student interaction scores in comparison to the other target students, indicating more negative teacher-student interactions. Question Four (4a, 4b, 4c). The first three subparts of research question four included “Will there be a difference in the mean scores for the behavior of the entire class of students and the target students in particular across the two group conditions over time?”, “Will there be a difference in the mean scores over time for the internalizing scores for all classroom students through teacher ratings from pretest to post-test across the two group conditions?”, “Will there be a difference in the mean scores over time for the externalizing scores for all classroom 155 students through teacher ratings from pretest to post-test across the two group conditions?”, and “Will there be a difference in the mean scores over time for the social skills scores on teacher ratings for all classroom students from pretest to post-test across the two group conditions?”. For these questions, three-level hierarchical models were used to analyze the multiple time points nested within the students nested within the teacher level variables. The HLM 7 statistical software was used for these analyses, and the descriptive statistics for these variables are included in Tables 34. The county and group dummy variables were included within the teacher level for these analyses. The HLM analyses are included in Tables 35-37 on the following pages. Table 34 Descriptive Data for All Classroom Students’ Teacher-Rated Externalizing Behaviors, Internalizing Behaviors, and Social Skills at Pretest and Posttest IY-TCM Reading M (SD) M (SD) Externalizing Behaviors Pretest 50.17 (10.19) 53.13 (11.02) Posttest 51.33 (10.27) 52.80 (11.01) Internalizing Behaviors Pretest 47.78 (9.01) 51.13 (11.06) Posttest 49.73 (10.07) 52.92 (11.18) Social Skills Pretest 47.58 (9.06) 48.84 (9.66) Posttest 49.74 (9.30) 52.34 (10.35) 156 Table 35 Internalizing scores for all classroom students Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 47.4320 3.0975 15.313 25 <0.001 COUNTY, γ001 6.2953 3.4184 1.842 25 0.077 GROUP, γ002 -1.3160 3.0974 -0.425 25 0.675 GRPXCOUNTY, γ003 -4.8097 4.6337 -1.038 25 0.309 PSTR_FREQ, γ004 0.0021 0.0982 0.022 25 0.983 INAPP_FREQ, γ005 0.4226 0.5130 0.824 25 0.418 EXT, β01 INTRCPT3, γ010 0.1213 0.0793 1.531 410 0.127 SS, β02 INTRCPT3, γ020 -0.1359 0.1410 -0.964 9 0.360 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 1.6499 1.1313 1.458 48 0.151 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 5.8138 33.8004 410 1318.1225 <0.001 level-1, e 5.8822 34.6005 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 5.1411 26.4310 25 276.2292 <0.001 157 Table 36 Externalizing scores for all classroom students Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 49.7191 2.2153 22.443 25 <0.001 COUNTY, γ001 5.9337 2.4851 2.388 25 0.025 GROUP, γ002 0.6408 2.6432 0.242 25 0.810 GRPXCOUNTY, γ003 -7.6851 3.7907 -2.027 25 0.053 PSTR_FREQ, γ004 0.0718 0.0860 0.834 25 0.412 INAPP_FREQ, γ005 0.0746 0.3895 0.192 24 0.850 INT, β01 INTRCPT3, γ010 0.1580 0.0900 1.755 410 0.080 SS, β02 INTRCPT3, γ020 -0.3051 0.1269 -2.405 8 0.043 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 0.6636 0.7938 0.836 19 0.414 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 7.6211 58.0813 410 2560.4471 <0.001 level-1, e 4.9170 24.1770 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 3.7730 14.2352 25 132.2942 <0.001 158 Table 37 Social skills scores for all classroom students Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 45.9531 1.3892 33.078 25 <0.001 COUNTY, γ001 3.6599 3.2312 1.133 25 0.268 GROUP, γ002 -0.9622 2.2014 -0.437 25 0.666 GRPXCOUNTY, γ003 -5.8335 3.9913 -1.462 25 0.156 PSTR_FREQ, γ004 0.2155 0.0684 3.151 25 0.004 INAPP_FREQ, γ005 -0.2184 0.2609 -0.837 25 0.410 EXT, β01 INTRCPT3, γ010 -0.2379 0.1179 -2.017 7 0.083 INT, β02 INTRCPT3, γ020 -0.0752 0.1247 -0.603 7 0.566 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 2.7913 0.6266 4.454 411 <0.001 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 5.6951 32.4338 410 1435.6247 <0.001 level-1, e 5.0959 25.9680 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 4.5970 21.1324 25 231.0212 <0.001 159 Internalizing scores for all classroom students. The intraclass correlations for this model were 0.3601 (Level 3) and 0.3214 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 6.3883; variance component 40.8107; degrees of freedom 30; chi square 386.4624; p<.001. The conditional model accounted for an additional 35% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 6.0355; variance component 36.4278; degrees of freedom 412; chi square 1377.0689; p<.001. The conditional model accounted for an additional 7% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 6.0086; variance component 36.1027. The conditional model accounted for an additional 4% of the variance at level one. Results of this model indicated that there were no significant predictors for internalizing scores for all classroom students over time (see Table 35). The group assignment was also not a significant predictor for this variable. Externalizing scores for all classroom students. The intraclass correlations for this model were 0.2022 (Level 3) and 0.5839 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 4.8136; variance component 23.1702; degrees of freedom 30; chi square 178.2801; p<.001. The conditional model accounted for an additional 39% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 8.1804; variance component 66.9189; degrees of freedom 412; chi square 2876.2705; 160 p<.001. The conditional model accounted for an additional 13% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 4.9515; variance component 24.5178. The conditional model accounted for an additional 1% of the variance at level one. Results of the model indicated that the county variable (coefficient 5.9337, p= 0.025) and student social skills score (coefficient -0.3051, p= 0.043) were significant predictors of externalizing scores for classroom students over time, and the group by county variable (coefficient -7.6851, p= 0.053) was a marginally significant predictor over time (see Table 36). For the county variable, county one was significantly different and had higher externalizing scores over time in comparison to county two, indicating higher behavior problems. For the group by county variable, the IY-TCM training group in county one was a marginally significant predictor of lower externalizing scores over time in comparison to the other county groups. The group assignment was not a significant predictor for this variable. Social skills scores for all classroom students. The intraclass correlations for this model were 0.2886 (Level 3) and 0.4014 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 5.2733; variance component 27.8080; degrees of freedom 30; chi square 253.1565; p<.001. The conditional model accounted for an additional 24% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 6.2188; variance component 38.6740; degrees of freedom 412; chi square 1472.5413; p<.001. The conditional model accounted for an additional 16% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at 161 level one as the following: standard deviation 5.4650; variance component 29.8666. The conditional model accounted for an additional 13% of the variance at level one. Results of the model indicated that total frequency of positive strategy use (coefficient 0.2155, p= 0.004) was a significant predictor of social skills scores for all students over time (see Table 37). The TIME variable (coefficient 2.7913, p<0.001) was also a significant predictor of social skill scores over time, indicating that social skills significantly increased between pretest and posttest. The group assignment was not a significant predictor for this variable. Summary of outcomes for student level variables for all students. Overall, the group variable alone was not a significant predictor over time for any of these variables. A county difference was found for externalizing scores, with county one being significantly different with higher scores over time in comparison to county two. The IY-TCM group in county one was a marginally significant predictor of lower externalizing scores (indicating improvement) over time in comparison to the other groups, although this should be interpreted with caution. Social skills were also found to significantly increase over time across students. Further analysis of mean social skills scores for all classroom students indicated that the number of students who were within the at-risk and clinically significant range for social skills decreased from 114 students at pretest to 68 students at posttest (see Table 38). This is a substantial decrease over time, indicating that the change over time was both statistically and practically significant. However, the number of students within the at-risk and clinically significant ranges increased for both externalizing and internalizing behaviors. 162 Table 38 Number of Classroom Students within the Clinically Significant and At-Risk Range at Pretest and Posttest Pretest Posttest Clinically At-Risk Total Clinically At-Risk Total Significant Significant Externalizing 32 53 85 37 54 91 Internalizing 22 37 59 29 66 95 Social Skills 3 111 114 5 63 68 Note: only students with complete data were included in this analysis Question Four (4d, 4e, 4f, 4g, 4h). The other five subparts of research question four included “Will there be a difference in the mean scores over time for the target students’ internalizing scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions?”, “Will there be a difference in the mean scores over time for the target students’ externalizing scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions?”, “Will there be a difference in the mean scores over time for the target students’ social skill scores through parent and teacher ratings from pretest to the midpoint to post-test across the two group conditions?”, “Will there be a difference in the mean scores over time for the percentage of time that target students demonstrate positive social behaviors from pretest to the midpoint to post-test, as measured by the DBR observations, across the two group conditions?”, and “Will there be a difference in the mean scores over time for the percentage of time in which target students are involved in positive interactions with peers from pretest to the midpoint to post-test across the two group conditions?” Three-level hierarchical models were used to analyze the multiple time points nested within the students nested within the teacher level variables. The HLM 7 statistical software was used for these analyses, and the descriptive statistics for these variables are included in Tables 26 (included earlier) and 39. The county and group dummy variables were 163 included within the teacher level for these analyses. HLM analyses are included in tables 40-47 on the following pages. Table 39 Descriptive Data for Target Student Externalizing Behaviors, Internalizing Behaviors, and Social Skills at Three Time Points IY-TCM Reading M (SD) M (SD) Externalizing Pretest 54.53 (10.92) 56.70 (10.20) Midpoint 54.48 (10.35) 54.74 (9.02) Posttest 54.55 (9.21) 53.92 (8.95) Internalizing Pretest 57.81 (8.82) 60.23 (11.54) Midpoint 56.66 (8.06) 59.58 (9.47) Posttest 57.33 (11.65) 59.05 (9.41) Social Skills Pretest 46.28 (7.77) 47.63 (9.46) Midpoint 47.70 (8.33) 50.69 (9.66) Posttest 50.35 (9.53) 53.30 (8.68) 164 Table 40 Target student internalizing scores (teacher ratings) Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 54.8749 4.7842 11.470 25 <0.001 COUNTY, γ001 0.3098 2.7639 0.112 25 0.912 GROUP, γ002 -1.4677 2.5387 -0.578 25 0.568 GRPXCOUNTY, γ003 -0.6697 3.6956 -0.181 25 0.858 PSTR_FREQ, γ004 -0.0852 0.0849 -1.003 25 0.325 INAPP_FREQ, γ005 0.0762 0.3971 0.192 25 0.849 RISK_INT, β01 INTRCPT3, γ010 9.2007 1.5504 5.934 27 <0.001 RISK_BOT, β02 INTRCPT3, γ020 8.6432 3.3160 2.607 27 0.015 EXT, β03 INTRCPT3, γ030 0.1116 0.1082 1.031 27 0.312 SS, β04 INTRCPT3, γ040 0.0515 0.1152 0.447 27 0.659 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 -2.3948 4.4654 -0.536 91 0.593 TIME2 slope, π2 INTRCPT2, β20 INTRCPT3, γ200 0.4859 1.1115 0.437 91 0.663 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 4.4144 19.4873 26 71.7074 <0.001 level-1, e 6.6447 44.1513 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 3.1019 9.6221 25 48.3443 0.004 165 Table 41 Target student internalizing scores (parent ratings) Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 54.6191 2.8261 19.326 51 <0.001 RISK_INT, β01 0.3364 2.0022 0.168 51 0.867 RISK_BOT, β02 -1.8720 2.7720 -0.675 51 0.503 INT, β03 -0.0543 0.0887 -0.613 51 0.543 PEXT, β04 0.5808 0.0868 6.690 51 <0.001 PSS, β05 0.1394 0.1071 1.301 51 0.199 GROUP, β06 -1.3860 2.2079 -0.628 51 0.533 COUNTY, β07 -1.7408 2.3974 -0.726 51 0.471 GRPXCOUNTY, β08 -0.0757 3.4908 -0.022 51 0.983 PSTR_FREQ, β09 -0.0588 0.0650 -0.906 51 0.369 INAPP_FREQ, β010 0.0979 0.2640 0.371 51 0.712 TIME slope, π1 INTRCPT2, β10 -2.7122 2.8937 -0.937 122 0.350 TIME2 slope, π2 INTRCPT2, β20 0.2652 0.7446 0.356 122 0.722 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 5.7092 32.5952 50 247.8224 <0.001 level-1, e 5.0606 25.6097 166 Table 42 Target student externalizing scores (teacher ratings) Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 57.8259 3.7181 15.552 52 <0.001 RISK_INT, β01 -4.4823 2.8458 -1.575 52 0.121 RISK_BOT, β02 6.9331 3.2806 2.113 52 0.039 INT, β03 -0.0205 0.1151 -0.178 52 0.859 SS, β04 -0.2226 0.1192 -1.869 52 0.067 GROUP, β05 1.6981 2.8353 0.599 52 0.552 COUNTY, β06 0.4077 2.3431 0.174 52 0.863 GRPXCOUNTY, β07 -5.1742 3.8394 -1.348 52 0.184 PSTR_FREQ, β08 0.0922 0.0882 1.045 52 0.301 INAPP_FREQ, β09 0.4326 0.3272 1.322 52 0.192 TIME slope, π1 INTRCPT2, β10 -2.0596 2.6272 -0.784 122 0.435 TIME2 slope, π2 INTRCPT2, β20 0.3529 0.6433 0.549 122 0.584 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 7.3329 53.7708 51 440.0078 <0.001 level-1, e 4.6463 21.5884 167 Table 43 Target student externalizing scores (parent ratings) Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 51.4774 3.5237 14.609 51 <0.001 RISK_INT, β01 2.6278 1.7838 1.473 51 0.147 RISK_BOT, β02 -1.4964 2.9661 -0.505 51 0.616 EXT, β03 0.4176 0.1266 3.298 51 0.002 PINT, β04 0.6136 0.0974 6.303 51 <0.001 PSS, β05 -0.0690 0.0884 -0.780 51 0.439 GROUP, β06 0.2189 2.1401 0.102 51 0.919 COUNTY, β07 0.3012 2.4335 0.124 51 0.902 GRPXCOUNTY, β08 1.6537 3.2681 0.506 51 0.615 PSTR_FREQ, β09 0.0720 0.0638 1.129 51 0.264 INAPP_FREQ, β010 -0.2734 0.3056 -0.895 51 0.375 TIME slope, π1 INTRCPT2, β10 -1.2696 3.6755 -0.345 122 0.730 TIME2 slope, π2 INTRCPT2, β20 -0.1801 0.9336 -0.193 101 0.847 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 5.1048 26.0586 50 186.2090 <0.001 level-1, e 5.4261 29.4426 168 Table 44 Target student social skills scores (teacher ratings) Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 INTRCPT3, γ000 48.9091 3.7104 13.182 25 <0.001 COUNTY, γ001 3.4129 2.8100 1.215 25 0.236 GROUP, γ002 1.4383 2.5849 0.556 25 0.583 GRPXCOUNTY, γ003 -10.4165 3.7447 -2.782 25 0.010 PSTR_FREQ, γ004 0.1554 0.0819 1.897 25 0.069 INAPP_FREQ, γ005 -0.4453 0.3475 -1.281 25 0.212 RISK_INT, β01 INTRCPT3, γ010 -6.8559 2.1632 -3.169 27 0.004 RISK_BOT, β02 INTRCPT3, γ020 -1.4026 3.2320 -0.434 27 0.668 EXT, β03 INTRCPT3, γ030 -0.3807 0.1540 -2.472 27 0.020 INT, β04 INTRCPT3, γ040 0.0356 0.0981 0.363 27 0.719 TIME slope, π1 INTRCPT2, β10 INTRCPT3, γ100 2.1108 2.9402 0.718 91 0.475 TIME2 slope, π2 INTRCPT2, β20 INTRCPT3, γ200 0.0694 0.6698 0.104 91 0.918 Final estimation of level-1 and level-2 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1,r0 5.2903 27.9876 26 147.5175 <0.001 level-1, e 4.7046 22.1335 Final estimation of level-3 variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1/INTRCPT2,u00 2.6725 7.1423 25 43.4763 0.012 169 Table 45 Target student social skills scores (parent ratings) Standard Approx. Fixed Effect Coefficient t-ratio error d.f. INTRCPT1, π0 INTRCPT2, β00 56.0174 3.4053 16.450 51 RISK_INT, β01 -2.1724 1.9717 -1.102 51 RISK_BOT, β02 -1.7799 2.1883 -0.813 51 SS, β03 0.1567 0.1215 1.290 51 PEXT, β04 -0.1681 0.0762 -2.206 51 PINT, β05 0.1153 0.1042 1.106 51 GROUP, β06 -2.4332 2.1861 -1.113 51 COUNTY, β07 -3.9740 2.7091 -1.467 51 GRPXCOUNTY, β08 3.3240 3.8567 0.862 51 PSTR_FREQ, β09 -0.1515 0.0664 -2.283 51 INAPP_FREQ, β010 -0.2721 0.3746 -0.727 51 TIME slope, π1 INTRCPT2, β10 -0.1545 3.3960 -0.045 122 TIME2 slope, π2 INTRCPT2, β20 0.2974 0.8322 0.357 122 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 6.1915 38.3351 50 250.3475 <0.001 level-1, e 5.3483 28.6042 170 p-value <0.001 0.276 0.420 0.203 0.032 0.274 0.271 0.149 0.393 0.027 0.471 0.964 0.721 Table 46 Target student DBR positive social behavior scores Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 4.7619 1.8871 2.523 17 0.022 PEER, β01 0.3916 0.1968 1.990 17 0.063 TSINT, β02 -0.0166 0.1001 -0.166 17 0.870 RISK_INT, β03 -0.0249 0.7014 -0.035 17 0.972 RISK_BOT, β04 -0.9572 0.8374 -1.143 17 0.269 SS, β05 0.0191 0.0289 0.662 17 0.517 STRSTOT, β06 0.0489 0.0262 1.866 17 0.079 GROUP, β07 -0.4208 0.6226 -0.676 17 0.508 PSTR_FREQ, β08 -0.0336 0.0476 -0.707 17 0.489 INAPP_FREQ, β09 -0.0597 0.0873 -0.684 17 0.503 TIME slope, π1 INTRCPT2, β10 0.3405 2.0774 0.164 52 0.870 TIME2 slope, π2 INTRCPT2, β20 0.0844 0.5219 0.162 52 0.872 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 0.3007 0.0904 17 17.3973 0.428 level-1, e 2.3099 5.3357 171 Table 47 Target student DBR peer interaction scores Standard Approx. Fixed Effect Coefficient t-ratio p-value error d.f. INTRCPT1, π0 INTRCPT2, β00 4.6072 1.9217 2.398 17 0.028 SOCIAL, β01 0.0754 0.1912 0.394 17 0.698 TSINT, β02 0.0536 0.1515 0.354 17 0.728 RISK_INT, β03 0.4619 0.8605 0.537 17 0.598 RISK_BOT, β04 -0.7922 1.0490 -0.755 17 0.460 SS, β05 -0.0138 0.0340 -0.405 17 0.691 STRSTOT, β06 0.0656 0.0303 2.166 17 0.045 GROUP, β07 0.1112 0.6851 0.162 17 0.873 PSTR_FREQ, β08 -0.0374 0.0708 -0.528 17 0.604 INAPP_FREQ, β09 -0.1044 0.0980 -1.065 17 0.302 TIME slope, π1 INTRCPT2, β10 -0.2111 2.1810 -0.097 52 0.923 TIME2 slope, π2 INTRCPT2, β20 0.1748 0.5588 0.313 52 0.756 Final estimation of variance components Standard Variance Random Effect d.f. χ2 p-value Deviation Component INTRCPT1, r0 1.0070 1.0141 17 27.2681 0.054 level-1, e 2.2229 4.9411 172 Target student internalizing scores (teacher ratings). The intraclass correlations for this model were 0.3970 (Level 3) and 0.1461 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 6.2158; variance component 38.6363; degrees of freedom 30; chi square 113.2070; p<0.001. The conditional model accounted for an additional 75% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 3.7707; variance component 14.2184; degrees of freedom 30; chi square 60.3323; p=0.001. The conditional model does not account for additional variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 6.6674; variance component 44.4543. The conditional model accounted for an additional 1% of the variance at level one. Results of the model indicated that identification as high risk for internalizing symptoms compared to other target students (coefficient 9.2007, p<0.001) and identification as high risk for both internalizing and externalizing symptoms compared to other target students (coefficient 8.6432, p= 0.015) were significant predictors of higher target student internalizing scores over time (see Table 40). The group assignment was not a significant predictor of this variable. Target student internalizing scores (parent ratings). The intraclass correlations for the three-level model were 0.0070 (Level 3) and 0.6904 (Level 2), indicating variance between students but a lack of variance between teachers. Therefore, the teacher variables were included on the second level and a two-level model was used. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 8.0890; variance component 65.4319; degrees of freedom 60; chi square 487.6119; p<0.001. The 173 conditional model accounted for an additional 50% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 5.2899; variance component 27.9833. The conditional model accounted for an additional 8% of the variance at level one. Results of this two-level model indicated that parent-rated student externalizing behavior (coefficient 0.5808, p<0.001) was a significant predictor of parent-rated student internalizing scores over time (see Table 41). The group assignment was not a significant predictor of this variable over time. Target student externalizing scores (teacher ratings). The intraclass correlations for the three-level model were 0.0006 (Level 3) and 0.7718 (Level 2), indicating variance between students but a lack of variance between teachers. Therefore, the teacher variables were included on the second level and a two-level model was used. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 8.6612; variance component 75.0170; degrees of freedom 60; chi square 690.9901; p<0.001. The conditional model accounted for an additional 28% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 4.6601; variance component 21.7168. The conditional model accounted for an additional 1% of the variance at level one. Results of this two-level model indicated that identification as being at-risk for both high internalizing and externalizing behaviors (coefficient 6.9331, p= 0.039) was a significant predictor of higher target student externalizing scores over time (see Table 42). The group assignment was not a significant predictor of this variable over time. 174 Target student externalizing scores (parent ratings). The intraclass correlations for the three-level model were 0.0021 (Level 3) and 0.6608 (Level 2), indicating variance between students but a lack of variance between teachers. Therefore, the teacher variables were included on the second level and a two-level model was used. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 8.1318; variance component 66.1259; degrees of freedom 60; chi square 426.1646; p<0.001. The conditional model accounted for an additional 61% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 5.7458; variance component 33.0143. The conditional model accounted for an additional 11% of the variance at level one. Results of this two-level model indicated that teacher-ratings of student externalizing behaviors (coefficient 0.4176, p= 0.002) and parent-ratings of student internalizing behaviors (coefficient 0.6136, p<0.001) were significant predictors of parent-ratings of target student externalizing scores over time (see Table 43). The group assignment was not a significant predictor of this variable over time. Target student social skills scores (teacher ratings). The intraclass correlations for this model were 0.1711 (Level 3) and 0.4931 (Level 2), indicating variance between teachers and students and a need for the three level model. Results of the null model indicated the final estimation of variance components at level three as the following: standard deviation 3.7649; variance component 14.1747; degrees of freedom 30; chi square 48.4129; p=0.018. The conditional model accounted for an additional 50% of the variance at level three. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 6.3907; variance component 40.8404; degrees of freedom 30; chi square 175 166.4988; p<.001. The conditional model accounted for an additional 31% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 5.2732; variance component 27.8064. The conditional model accounted for an additional 20% of the variance at level one. Results of this model indicated that the group by county variable (coefficient -10.4165, p= 0.010), identification as being at risk for internalizing symptoms (coefficient -6.8559, p= 0.004), and student externalizing scores (coefficient -0.3807, p= 0.020) were significant predictors of teacher-ratings of target student social skills over time (see Table 44). For the group by county variable, the IY-TCM group in county one was significantly different and had lower social skills scores over time in comparison to other groups in the two counties. Target student social skills scores (parent ratings). The intraclass correlations for the three-level model were 0.0536 (Level 3) and 0.5379 (Level 2), indicating variance between students but a lack of variance between teachers. Therefore, the teacher variables were included on the second level and a two-level model was used. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 6.5754; variance component 43.2354; degrees of freedom 60; chi square 324.5924; p<0.001. The conditional model accounted for an additional 11% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 5.4091; variance component 29.2582. The conditional model accounted for an additional 2% of the variance at level one. Results of this two-level model indicated that parent-ratings of student externalizing behaviors (coefficient -0.1681, p= 0.032) and teacher reported frequency of positive strategy use (coefficient -0.1515, p= 0.027) were significant predictors of parent-ratings of target student 176 social skill scores over time (see Table 45). The group assignment was not a significant predictor of this variable over time. Target student DBR positive social behavior scores. The intraclass correlations for the three-level model were 0.0004 (Level 3) and 0.1475 (Level 2), indicating variance between students but a lack of variance between teachers. Therefore, the teacher variables were included on the second level and a two-level model was used. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 1.0462; variance component 1.0945; degrees of freedom 26; chi square 41.1511; p=0.030. The conditional model accounted for an additional 92% of the variance at level two. Results of the null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 2.3788; variance component 5.6585. The conditional model accounted for an additional 6% of the variance at level one. Results of this two-level model indicated that there were no significant predictors of target student DBR positive social behavior scores over time (see Table 46). However, examination of the mean scores within Table 26 indicated that the reading group showed a greater improvement in their scores over time than the IY-TCM group. Target student DBR peer interaction scores. The interclass correlations for the threelevel model were 0.0174 (Level 3) and 0.1613 (Level 2), indicating variance between students but a lack of variance between teachers. Therefore, the teacher variables were included on the second level and a two-level model was used. Results of the null model indicated the final estimation of variance components at level two as the following: standard deviation 1.0951; variance component 1.1993; degrees of freedom 26; chi square 44.5548; p=0.013. The conditional model accounted for an additional 15% of the variance at level two. Results of the 177 null model indicated the final estimation of variance components for Intercept 1 at level one as the following: standard deviation 2.2410; variance component 5.0222. The conditional model accounted for an additional 2% of the variance at level one. Results of this two-level model indicated that the STRS total score (coefficient 0.0656, p= 0.045) was a significant predictor of target student DBR peer interaction scores over time (see Table 47). The group assignment was not a significant predictor of this variable. However, examination of the mean scores within Table 26 indicated that the reading group showed a greater improvement in their scores over time than the IY-TCM group. Summary of outcomes for target student variables. Overall, the group variable alone was not a significant predictor over time for any of these target student variables. The IY-TCM group in county one had significantly different teacher-rated social skills scores that were lower over time in comparison to the other groups in county one and county two. Further analysis was done of mean scores across time for these variables. Interestingly, the reading group demonstrated more improvement in DBR peer interactions and DBR positive behaviors than the IY-TCM group. Although the number of target students within the clinically significant or at-risk range for externalizing behaviors and internalizing behaviors at pretest and posttest remained fairly stable, the number of students within these ranges for social skills reduced by half (see Table 48). Further examination of the target student data indicated that 10 students who initially scored within the at-risk or clinically significant range for one or multiple behavior categories (e.g., externalizing, internalizing, social skills) were no longer within either range at posttest. These students were equally dispersed between the IY-TCM and reading group. 178 Table 48 Number of Target Students with Clinically Significant or At-Risk Scores at Pretest and Posttest Pretest Posttest Clinically At-Risk Total Clinically At-Risk Total Significant Significant Externalizing 5 12 17 5 10 15 Internalizing 10 14 24 10 15 25 Social Skills 0 16 16 0 7 7 Question Five. To answer the fifth research question of “Will there be a difference in the ratings of acceptability of the treatment between the two conditions (IY-TCM and bibliotherapy) at post-test?”, independent samples t-test analyses were run for each variable (Overall Program Acceptability, Acceptability of Strategies) in order to examine the differences between the two groups at the post-test rating. The Statistical Package for the Social Sciences (SPSS version 22) was used for these analyses. Overall program acceptability. Three cases were excluded from this analysis due to missing data for this section of the rating form. The Levene’s Test for Equality of Variance indicated a significance level of p= 0.987, indicating that the group variances were homogeneous (Table 49). The results of the analysis indicated that there was no statistically significant difference between the ratings of overall acceptability of the program between the IY training group (M= 46.03, SD = 3.55) and the reading group (M= 45.00, SD = 2.89) conditions; t (26) = 0.837, p= 0.410. The possible range for this variable was from a raw score of 8 to 56. These results suggest that the acceptability ratings of the overall program were similarly high between the two groups. 179 Table 49 Results of t-test for overall program acceptability Group IY-TCM Reading M SD n M SD Overall 46.03 3.55 15 45.00 2.89 Program Acceptability 95% CI n 13 -3.57, 1.51 t -0.837 df 26 Acceptability of strategies. The Levene’s Test for Equality of Variance indicated a significance level of p= 0.895, indicating that the group variances were homogeneous (Table 50). The results of this analysis indicated that there was no statistically significant difference between the ratings of acceptability of strategies taught within the program between the IY training group (M= 67.00, SD = 4.65) and the reading group (M= 64.00, SD = 5.22) conditions; t (29) = -1.692, p= 0.101. The possible range for this variable was from a raw score of 11 to 77. These results suggest that the strategy acceptability ratings were similarly high between the two groups. Table 50 Results of t-test for acceptability of strategies Group IY-TCM Reading M SD n M SD Acceptability 67.00 4.65 16 64.00 5.22 of Strategies 180 95% CI n 15 -6.63, 0.63 t -1.692 df 29 CHAPTER 5: DISCUSSION The importance of early intervention and prevention efforts within early childhood and school-age populations has been stressed throughout the literature, with discussion centering on the effectiveness of comprehensive and universal prevention supports versus more targeted and individualized intervention programs, especially for internalizing behaviors (Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005). School-based programs and interventions can be an effective method for reaching a wide range of students within the population and can serve as a place to provide more preventative supports (Doll & Yoon, 2010; Dwyer & Van Buren, 2010). The Incredible Years program is one such program geared towards providing early intervention and preventative supports for children, parents, and teachers in an effort to improve positive adult practices and relationships with children, child social competence, and problematic behaviors (Webster-Stratton, 2008; Webster-Stratton & Herman, 2008). The purpose of this study was to examine the use of the Incredible Years Teacher Classroom Management (IY-TCM) Group Training Program with preschool teachers in order to influence changes in teacher practices, the classroom environment, and the behaviors and social competency of children at risk for externalizing and internalizing behavior problems. Specifically, this study was intended to add to the existing literature base on the use of the IYTCM program through extending the variables that are explored within this program intervention analysis (e.g., internalizing behaviors) in order to explore the use of this program as a comprehensive prevention and early intervention program targeting teacher practices in order to extend outcomes to a broader scope of children within the classroom. The theoretical framework for this study suggested that the components of the Incredible Years Teacher Classroom Management (IY-TCM) Program work to directly influence teacher 181 perceptions and use of specific classroom management strategies for handling problem behaviors, which was hypothesized to then influence classroom variables and student variables, such as student behavior. In light of this, the second hypothesis for this study was that there would be significant differences between the two groups, in favor of the IY-TCM group, in terms of their frequency of use and perceptions of usefulness of the positive classroom management strategies, confidence in managing classroom behavior, and use of planning and support strategies and positive approaches with parents. Results of this study partially supported this hypothesis. Treatment procedural integrity. The results of this study indicated that the procedural integrity of the treatment groups was maintained throughout the study. Fidelity measures consisting of video recording and training checklists completed by the trainer and independently coded by a research assistant indicated that the training sessions were carried out as intended within the IY-TCM program manual, with minor adaptations made to meet the needs of the group. In addition, analyses of group differences between the number of hours spent on professional development related to this program, in terms of time spent in training, reading, and/or practicing the techniques, indicated that there were significant differences between the total hours that each group devoted to this intervention content (IY-TCM group M= 44.89, bibliotherapy group M= 16.77). These mean scores indicate that the participants, on average, spent more time than the minimum number of hours required to earn the SCECHs and meet the requirements of the group treatments (IY-TCM minimum = 36 hours, bibliotherapy minimum = 15 hours). This suggests that the procedural integrity of the treatment groups was maintained, according to the assignment requirements for the reading group and the training requirements for the IY-TCM group. These results also provide evidence that the training groups were carried out 182 as intended, and that the results for the teacher, classroom, and student level variables can be interpreted in light of this information. Teacher level outcomes. Within the current study, significant group differences over time were found in favor of the IY-TCM group for teacher perceptions of positive strategies (e.g., coaching), perception of social-emotional strategies (e.g., teaching social skills), frequency of use of proactive strategies (e.g., using problem solving strategies), perceptions of coaching/praise/incentives strategies (e.g., modeling positive behaviors), and frequency of planning and support strategies (e.g., self-reflection, stress management). Differences approaching significance in favor of the IY-TCM group were also found for frequency of positive strategy use and perception of usefulness of proactive strategies. These results support the theoretical framework for the IY-TCM program, which suggests that changes in teachers’ perceptions of the strategies precede changes in their frequency of use of the strategies, as is seen with several of the variables within this study. Significant increases over time for teachers, regardless of their group assignment, were found for frequency and perception of positive strategies, frequency of limit setting strategies (e.g., verbal redirection), planning and support strategies, and frequency and perceptions of coaching/praise/incentives strategies. Increases over time approaching significance were found for frequency of proactive strategies. Out of these variables demonstrating significant increases over time, the increase in scores followed a nonlinear (quadratic) pattern for teacher perceptions of positive strategies, perception of socialemotional strategies, and frequency of planning and support strategies, indicating that the degree of improvement was not the same between the three time points. These results are consistent with previous studies which found significant differences for teacher classroom management strategies as a result of the IY-TCM training program (Carlson, 183 Tiret, Bender, & Benson, 2011; Webster-Stratton, Reid, & Hammond, 2001a; Webster-Stratton, Reid, & Hammond, 2004; Webster-Stratton, Reid, & Stoolmiller, 2008). More specifically, Carlson, Tiret, Bender, and Benson (2011) found significant improvements in frequency of use and perceptions of positive strategies as well as frequency of use and perceptions of proactive strategies. Webster-Stratton, Reid, and Stoolmiller (2008) also found significant improvements in the use of social emotional teaching strategies, whereas the IY-TCM group in the current study only showed significant group differences in the perceptions of the social-emotional teaching strategies over time and there were no significant increases for this variable over time. However, analysis of mean scores within Table 10 did indicate that teachers overall across both groups demonstrated improvements in their use of the social-emotional strategies, although this change was not significant over time. Direct comparison to mean scores between the current study and previous IY-TCM studies, including the study conducted by Webster-Stratton, Reid, and Stoolmiller (2008), were not able to be made due to the fact that the Social Emotional subscale of the TSQ was not analyzed or a different measure of social emotional teaching was utilized within those studies. The results of the current study did not indicate significant decreases or group differences in the use or perceptions of inappropriate strategies over time (e.g., recognizing bad behavior), unlike the decreases found for other studies for these negative strategies (e.g., Webster-Stratton, Reid, & Stoolmiller, 2008). However, a study conducted by Carlson and colleagues (2011) also did not find significant decreases in these inappropriate strategy variables over time. These authors highlighted that the teachers began the study with lower scores for these variables overall (i.e., Frequency of Inappropriate Strategies pretest = 15.6, posttest = 15.3). For the current study, analysis of the mean scores for these variables within Table 10 indicated that the perceptions of 184 usefulness of the negative strategies increased for both groups, while the frequency of use of these strategies decreased slightly for both groups. Similar to the study conducted by Carlson and colleagues (2011), the mean scores for the frequency and perception of inappropriate strategies were relatively low for these groups at pretest and posttest (i.e., frequency at pretest = 14.44, posttest = 14.13). Therefore, significant decreases for these variables were less likely to occur than if the pretest scores were higher. It is interesting within the current study that significant group differences in favor of the IY-TCM group were found for the perceptions of coaching/praise/incentive strategies compared to the reading group, and that the frequency and perceptions of coaching/praise/incentives strategies significantly increased over time. This result is interesting due to the fact that there were discussions prior to the start of these interventions about the relevance of the praise and incentive strategies taught within the IY-TCM program for early childhood programs within Michigan currently using the Preschool Program Quality Assessment (PQA). The PQA is an observational rating assessment tool developed by High Scope Educational Research Foundation which has been used to determine ratings of the quality of programs according to seven indicators, including “Learning Environment, Daily Routine, Adult-Child Interaction, Curriculum Planning and Assessment, Parent Involvement and Family Services, Staff Qualifications and Staff Development, and Program Management” (Epstein, 2003, p. 3). This tool has been adopted by the Michigan Department of Education (n.d.), and it is required for Michigan Great Start Readiness Programs (GSRP; http://www.michigan.gov/documents/mde/GSRP-Overview_410757_7.pdf). Within the AdultChild Interaction indicator, there are specific items related to the use of praise/reinforcement and rewards/incentives which suggest that the use of the principles may constitute a lower score for 185 these items. Discussions with educators have indicated variability in the way that these items are interpreted, especially in light of other literature indicating the positive effects of such practices for student outcomes (e.g., Maag, 2001). However, the focus within the IY-TCM program on discussion of the positive and negative aspects of different strategies may have played a role in the changes in the perceptions and frequency of these strategies found within the study. On the other hand, these outcomes may be related to differences for specific items in this category, such as coaching strategies versus the use of incentives. However, this type of item-level analysis was not conducted for this study. Overall, as hypothesized, differences were found in favor of the IY-TCM group for several classroom management strategies over time. However, the bibliotherapy/reading group did also make improvements from pretest to posttest for each of the frequency and perception variables for positive classroom management strategies, as indicated by the mean scores within Table 10. Although the bibliotherapy group served as a comparison group which did participate in learning, this group also serves as a type of control group for the methodology or delivery of the IY content due to the fact that the content was the same between the groups with differences between the group training format (e.g., role playing, video vignettes, discussion) and the independent reading. Therefore, while it was found that the bibliotherapy control group did demonstrate improvements over time, the active group training group did show more significant outcomes related to the classroom management strategies. This indicates that the group training format, in comparison to a group which did not participate in this group training format, showed more significant outcomes over time for teacher classroom management strategies. It is important to note that the overall group means for the classroom management strategies for the teachers within this study started at relatively high levels and then indicated 186 substantial improvement, even if not all of the outcomes were significant. For example, several of the scores for the teachers within the current study were higher (e.g., Frequency of Proactive Strategies pretest = 26.9, posttest = 31.25; Frequency of Coaching/Praise/Incentives pretest = 19.25, posttest = 23.94; Perception of Coaching/Praise/Incentives pretest = 22.00, posttest = 26.75) in comparison to the pretest and/or posttest scores found within the study conducted by Carlson and colleagues (2011; e.g., Frequency of Proactive Strategies pretest = 22.9, posttest = 24.7; Coaching/Praise/Incentives pretest = 18.2, posttest = 21.5; Perception of Coaching/Praise/Incentives pretest = 18.0, posttest = 22.0). Additionally, when the frequency of proactive strategies group means are computed to an average score per item by dividing the score by the eight items (current study Proactive Strategies pretest = 3.33, posttest = 3.91), the results indicate slightly greater improvement than another IY-TCM study which utilized the TSQ proactive scale (Shernoff & Kratochwill, 2007; Proactive Strategies pretest = 3.3, posttest = 3.7). Classroom level outcomes. It was also hypothesized that the IY-TCM group would demonstrate significantly higher scores over time for classroom atmosphere ratings, positive teacher-student interactions, and ratings of teacher-student relationships in comparison to the bibliotherapy group. Although there were no significant predictors of classroom atmosphere ratings over time, examination of the mean scores showed that these ratings were high for each group (IY-TCM: pretest M= 1.55, posttest M= 1.51; Bibliotherapy: pretest M= 1.70, posttest M= 1.61), indicating a high rating of positive classroom atmosphere over time. There were significant differences between the two groups for the teacher-student interaction observational ratings over time, as measured by the DBR observations completed by research assistants in order to rate the positive interactions and support between the teacher and the student, with the IY-TCM group serving as a significant predictor of higher scores in comparison to the reading 187 group over time. While the IY-TCM group maintained higher scores for this variable over time than the reading group, the IY-TCM group mean score decreased from pretest to posttest (pretest M = 4.20, posttest M = 3.97), whereas the reading group mean scores increased (pretest M = 2.96, posttest M = 3.44). This is interesting to consider, as the IY-TCM group demonstrated significantly higher self-ratings of frequency of use of proactive strategies and perceptions of social-emotional, coaching, and total positive strategies, while observational data indicated an increase in group mean scores for positive relationships between teachers and students for the reading group and not for the IY-TCM group. These DBR observational measures were important sources of data in order to serve as a way to provide additional information regarding the use of these strategies within the classroom from a different, objective source, as well as to capture a snapshot of how these results may be observed at the classroom level or within the interactions between the individuals in the classroom. In addition, target students’ classification as being at-risk for internalizing and externalizing behavior problems was a significant predictor of lower teacher-student interaction scores over time. There were also significant differences between the target students and comparison students for the conflict score within the teacher-student relationship and the total rating of the relationship as measured by the STRS rating scales. For these variables, comparison students had lower conflict in the relationships over time and higher total relationship scores over time in comparison to the target students. These results align with prior research which indicates that children with more behavior problems in the classroom had more negative and conflictual relationships with their teachers (Henricsson & Rydell, 2004; Zhang & Sun, 2011). Previous IYTCM studies have not utilized a teacher self-rating for the quality and dynamics of the relationship between the teacher and the student. Instead, these IY-TCM studies have assessed 188 relationship and interaction variables through other measures, such as teacher-ratings of child behavior and social competence, teacher ratings of their classroom management strategies, or observational ratings of harsh/critical or warm/affectionate interactions. However, the selfratings of the quality of the teacher-student relationship (e.g., STRS ratings) within this study serves to support other research which highlights the relationship between student behavior problems and conflictual teacher-student relationships. In addition, this data may provide a valuable perspective for teachers to identify and reflect on the specific dynamics of their relationships with students presenting with externalizing behavior problems, internalizing behavior problems, or appropriate prosocial behavior on a regular basis. Given the heavy emphasis within the IY-TCM program for building positive relationships with students and improving student behavior through modeling and teaching strategies, it may be important for teachers to rate these relationships and reflect on how to overcome the potential conflictual relationships that have developed in order to achieve the desired changes in student behavior. This information should also be considered in light of the transactional model described as one of the theoretical frameworks in which this study was designed, it would be important and essential to gain a better understanding of the reciprocal nature between the way that the child’s behavior and adult’s reaction to this behavior serve to influence future relationships and interactions between these individuals. This variable should be further explored in the IY-TCM research in order to gain a better understanding of whether this piece of data may serve as a valuable tool for teachers to utilize in order to overcome these challenges. Student level outcomes for all classroom students. Although there were not significant group differences found between the two groups for the student level variables, there were still 189 noteworthy outcomes related to the internalizing behaviors, externalizing behaviors, and social skills for the classroom students and target students throughout this study. There were interesting differences found between the two counties for the externalizing behavior scores for all classroom students, with county one being significantly different and having higher externalizing scores over time (pretest M= 51.75, posttest M= 53.54) in comparison to county two (pretest M= 51.44, posttest M= 50.89), indicating worsening of behaviors. However, being within the IY-TCM group in county one (pretest M= 49.04, posttest M= 50.84) predicted lower externalizing scores over time in comparison to the other groups combined (pretest M= 52.45, posttest M= 52.44). This result may suggest that the length of time of the intervention (county 1 = six months, county 2 = three months) and data collection period may have played a role in influencing student externalizing behaviors. However, it is unclear if this difference can be attributed to the length of intervention or to the fact that county one had higher externalizing scores. A significant finding for the student level variables was related to classroom students’ social skills over time, regardless of the group assignment. These results indicated that social skills scores significantly increased over time across students. In addition, analysis of individual scores indicated that the number of students who were within the at-risk or clinically significant range for social skills decreased substantially from 114 students at the pretest period to 68 students at the posttest period. This is a promising outcome for this study, as it suggests that the improvement is likely not due to chance and that both group formats demonstrated an improvement in student social skills. However, the effects of schooling and maturation on student social skills cannot be ruled out due to a lack of a control group within this study. In addition, higher student social skills significantly predicted lower student externalizing scores 190 over time. These outcomes align with other research which identifies the relationship between social skills and later externalizing behaviors (Hymel, Rubin, Rowden, & LeMare, 1990). For this study, social skills were not a significant predictor of internalizing scores over time. Target student outcomes. Although no group differences or significant predictors were found for target student DBR positive social behaviors and peer interactions, examination of the mean scores showed that the reading group showed greater improvements in the observation scores for both variables. Overall, DBR mean observational data for positive social behaviors, peer interactions, and teacher-student interactions indicated that these ratings were relatively low across time (Table 26). Mean teacher-student interaction scores were less than five across the study (minimum 1, maximum 10), indicating a low rating of positive teacher-student interactions during these time periods. Mean peer interaction and positive social behaviors were slightly higher, but still indicated a lower number of positive scores, especially considering the impact that teacher interactions and peer interactions can have on student behavior outcomes (Baker, Grant, & Murlock, 2008; Hutcherson & Epkins, 2009; Rubin, Bukowski, & Laursen, 2009) and the focus of the IY-TCM program on increasing positive teacher-student relationships and coaching positive peer and social interactions (Webster-Stratton, Reid, & Hammond, 2001b). While these observation periods may not have captured daily changes occurring in the classrooms, these findings suggest that differences were found for teacher-rated perceptions and frequency of use, but that these changes may not have influenced observable classroom behavior yet. As mentioned previously, these DBR observational measures provide an important source of additional information in order to capture the changes to practices and interactions within the classroom in order to triangulate self-reported teacher strategy ratings. In addition, these measures could serve as a type of integrity measure for the ways in which teachers have made 191 changes within the classroom, as well as capture the transactional nature of the interactions and relationships between students and teachers, and how this may affect teacher practices. This latter point would be an important consideration for future research. No significant differences between groups or significant increases over time were found for target student internalizing behaviors, externalizing behaviors, and social skills. Target students with high risk for internalizing behavior problems were significantly different and had higher internalizing scores, higher externalizing scores, and lower social skills over time compared to other target students. Additionally, target students with high risk for both internalizing and externalizing behavior problems significantly predicted higher internalizing scores over time, but not for the other two variables. Further analysis of individual scores for these variables showed that the total number of target students within the at-risk and clinically significant ranges for externalizing and internalizing scores remained relatively stable over time, but the number of students within these risk categories for social skills scores decreased by half from pretest to posttest. In addition, 10 target students who were within these high risk ranges for one or multiple behaviors at pretest were no longer within these ranges for any of the behaviors at posttest. While this finding is not significant and was not found for all students, it does indicate that some students made improvement over the course of the school year. However, these changes may be due to chance and firm conclusions cannot be drawn at this time. Further discussion of the findings related to the internalizing, externalizing, and social skills scores for all classroom students and for target students is warranted, despite the limited findings for the externalizing and internalizing areas within this study. A primary focus for this study was to examine the use of the Incredible Years Teacher Classroom Management program 192 as a comprehensive Tier 1 intervention for teachers and students within the school setting. In particular, this study sought to examine the application of this program to the area of influencing student internalizing behavior problems within an early childhood population, as the potential effects of the IY programs on internalizing behaviors has been suggested within prior research (e.g., Barrera et al., 2002; Beauchaine, Webster-Stratton, & Reid, 2005; Herman, Borden, Reinke, & Webster-Stratton, 2011; Ogg & Carlson, 2009). Many of these studies examined the internalizing behaviors as a secondary outcome when the Child Program (Beauchaine, WebsterStratton, & Reid, 2005), self-administered IY Parent program (Ogg & Carlson, 2009), or a combination of the Child program, Parent program, and an additional social skills intervention were used (Barrera et al., 2002). However, these studies did not utilize the IY-TCM program. While the study conducted by Herman and colleagues (2011) did utilize the IY-TCM program, this teacher training program was not examined in isolation as it relates to internalizing behaviors. Instead, this training component was combined with both the Child and Parent programs, or just with the Parent program. Additionally, that study included treatments provided outside of the school setting and with children between the ages of four and eight years old. Taking these differences into consideration, in combination with the outcomes of this current study which did not find significant outcomes for internalizing behavior scores, it is unclear whether the teacher training program may take longer to demonstrate these effects on internalizing behaviors due to the indirect influence of the training for student level variables, or if an additional direct student intervention is an essential component for these outcomes. Future research should examine these factors more closely. Despite the lack of outcomes within the current study for internalizing behaviors, specific attention still should be given to the significant outcomes related to social skill development for 193 all classroom students across the study. This is important to consider due to the strong connection that has been made between a child’s lack of social competence and the child’s current or future problematic behaviors (Beidel, Morris, & Turner, 2004; Henricsson & Rydell, 2004; Hokanson & Rupert, 1991; Morris, 2004; Ladd &Troop-Gordon, 2003; Rubin, Bowker, & Kennedy, 2009). In fact, Morris (2004) highlights the importance of considering the bidirectional relationship that occurs between social skills, current or future social withdrawal, and potential internalizing problems. The significant outcomes within the current study related to social skills, while not immediately influencing child internalizing and externalizing scores, may serve as an early intervention effort affecting social skill development and potentially affecting future behavior improvement. In addition, while the IY-TCM program does focus on teaching social skills such as self-control, problem solving skills, emotional awareness, and the building of friendship skills, it is unclear if these skills have the potential to result in an immediate change to internalizing areas within a short period of time (e.g., up to six months). The lack of follow-up analyses included within the current study does not allow for further clarification of this point. Acceptability of treatment. Results related to the teacher participants’ ratings of acceptability for the overall program and the specific strategies taught within the content indicated that both groups rated their intervention as being equally acceptable. The results of the statistical analyses indicated that the group means were relatively similar, with no significant group differences found. Previous research suggests that perception of treatment acceptability is an important factor related to treatment outcomes and treatment adherence (e.g., Eckert & Hintze, 2000; Kazdin, 2000). The high levels of acceptability for the two groups within the current study were likely related to the improvements found for several of the classroom 194 management variables over time for both groups, as well as the adherence to the minimum number of hours required to complete the group intervention content (as described in the following paragraphs). This result is also interesting in light of the fact that the content presented within both groups was similar, but with important differences in the format of the intervention and the amount of time required for the intervention components between the two groups. Results in light of the procedural integrity. It is important to consider the outcomes related to teacher, classroom, and student level variables in light of the number of hours devoted to the intervention. While both groups of teachers demonstrated an increase in the mean group scores for many of the teacher classroom management variables, important differences were found for several variables in favor of the IY-TCM group in comparison to the bibliotherapy/reading group. Although the content was similar, these results suggest that the number of hours spent in active training and reading of the content, as well as the group format involving group discussions, role playing, and video vignettes, may have been influential to the group differences for these outcomes. Additionally, outcomes related to student social skills and externalizing behavior problems should also be considered in light of these group format and time variables. Results indicated that the mean student social skills increased significantly over time for classroom students, with the group assignment not serving as a significant predictor of this variable. This suggests that the similar content between the two groups may have been related to improvements in student social skills over the length of the study for both groups, despite the differences in treatment delivery and the significant differences in the amount of time devoted to the study. However, the lack of a control group within this study makes it uncertain whether these improvements can be attributed to the treatment groups, or whether students may 195 have demonstrated improvements in their social skills due to schooling or maturation. Further examination of these results is important when considering the dissemination of content as a Tier 1 intervention to a wide audience in order to address student social skills, as the results suggest that social skills outcomes were similar between the training and reading groups and that both formats may influence the development of student social skills. This would be an important consideration for future research. Group by county differences were found for the externalizing scores for classroom students and for the target student social skills scores, with assignment to the IY-TCM group in county one serving as a significant predictor of lower externalizing scores over time and lower target student social skills scores over time in comparison to the rest of the groups within the study. However, analysis of the mean scores showed that the IY-TCM group in county one had externalizing scores that slightly increased (pretest M= 49.04, posttest M= 50.84), although the mean scores were lower over time in comparison to the other groups combined. Additionally, the IY-TCM group in county one had mean target student social skills scores that increased over time (pretest M= 43.81, posttest M= 47.58), although these mean scores were lower over time in comparison to the other groups combined (pretest M = 48.02, posttest M = 53.24). Interestingly, the teachers within the county one IY-TCM group had a lower mean score for the number of hours spent engaged in the study material (M= 40.90) in comparison to the county two IY-TCM group (M= 48.88), which indicates that the group in county two spent more time reading about or practicing the concepts. It is also interesting to consider the limited significant differences found between the similar groups in both counties, as the span of time that these participants were able to practice and apply the skills varied between the two groups even though the intervention hours were the same. This is an area that should be examined further in future 196 research, as it is recommended that the IY-TCM training be conducted over six months and previous IY-TCM studies found reductions in externalizing behaviors and conduct problems for students within the training groups which more closely aligned with the recommended format, ranging between four months and six months (e.g., Williford & Shelton, 2008; Webster-Stratton, Reid, & Hammond, 2004). Overall results and implications. As was expected within the IY theoretical framework of this study, as well as the conceptual framework for this study displayed in Figure 1, significant outcomes were found first and foremost for the teacher classroom management strategies over time. As these were the direct focus and targets for the intervention groups, it was predicted that significant outcomes would be found for these variables. However, fewer significant outcomes related to significant group differences or significant changes over time were found at the classroom and student level, with the exception of the significant increase in student social skill scores over time. This data indicates that the top-down effect or influence was not found within the results for this study, as was conceptualized within the IY theoretical framework and within the conceptual framework for this study which highlighted the influence that the changes in teacher perceptions and strategy use would have on classroom variables, interactions, and student variables. This may suggest that the influence on the classroom and student level variables may take longer to demonstrate significant outcomes, or it may suggest that teacher use of positive practices were not as significant as self-reported or that the strategies may not have been utilized in the most effective and consistent manner in order to influence classroom and teacher outcomes. However, when considering these outcomes in light of the transactional model used as a theoretical framework for this study, the outcomes may also be related to the bidirectional and reciprocal interactions between teacher behaviors/reactions/practices and the students’ 197 behavior and temperament. Therefore, while significant outcomes may have been found for teacher reported perceptions and use of positive classroom management strategies, these practices were likely influenced by the transactional relationship of the students’ problem behaviors and the teachers’ relationships and interactions with that student. Therefore, it may be that the specific problem behaviors of the identified target students or the existing conflictual relationships between the teacher and target students influenced the teachers’ behaviors and strategy use in various ways. The teachers may have been able to increase their use of positive strategies for other students or for target students within specific situations, but these may not have been effectively captured within the observational behavior and interaction data based on the DBR measures or they may not have been utilized with enough consistency to influence student problematic behaviors. The theory behind the top-down effects of the IY model and the importance influence of the transactional and bidirectional relationship between individuals and variables should be examined further within future research. Limitations There were some limitations to this study that should be discussed when considering the study outcomes and future directions for research related to the Incredible Years programs. One limitation of this study was related to the different timelines for the two counties as a result of different recruitment schedules. Although differences between counties, groups, and a county by group variable were analyzed in order to determine whether there were differences between the counties, it is unclear whether more significant outcomes may have been found for teacher/classroom and student level variables if all of the teachers had received the intervention over a six month intervention period. Although the teachers following the six month timeline and the three month timeline spent the same number of hours fulfilling their group requirements 198 (i.e., reading or training hours), the six month groups were able to practice and apply the strategies over a longer period of time. In addition, observational data for the DBR teacherstudent interactions, peer interactions, and positive social behavior, as well as classroom atmosphere ratings, were conducted for just one county in order to determine outcomes for a subset of the population. Due to this, the sample size for these variables is smaller than for the other variables and the ability to generalize these findings is therefore limited. Another limitation of this study was related to the measures of integrity for the treatment groups. The data collected related to the IY-TCM trainer fidelity checklists and videotaped sessions, as well as the teachers’ adherence to the required hours to complete their group training requirements (i.e., reading or training attendance) as measured through the PD Log of Hours, served to highlight the procedural integrity of the study interventions, but did not capture the integrity with which the teachers carried out and practiced these principles within the classrooms. While procedural integrity is an important factor to consider, especially when ensuring that the interventions were carried out as intended, another essential component of the IY-TCM program is related to the teachers’ effective use of the principles taught within the content of the program. This information could be captured through such methods as observations within the classroom specifically targeting the teachers’ application and use of these strategies within the classroom to identify the integrity to which the teachers used and practiced the strategies that were taught within the training group or identified within the reading material. The current study did not include this type of integrity measures, therefore it cannot be determined whether the teachers were consistently practicing and applying these strategies with integrity. This would be an important factor to consider within future research. 199 Although randomization procedures were used within the current study, there were still differences between the two groups at pretest for some of the variables. This would not typically be expected with randomization procedures. Therefore, the lack of equivalency at pretest across the two groups after randomization to the groups is a limitation of this study and should be considered when discussing the outcomes. The internal validity of this study may be limited due to the lack of a control group. While significant outcomes were found for both groups or as differences between the groups, a control group was not included within this study to serve as a comparison of participants who did not participate in any additional intervention. While there were outcomes that were significant and therefore not likely due to chance, a comparison to a group of teachers and students who were not included in any intervention group would have been an important addition to this study. However, previous research which included comparison of the IY-TCM program to a notreatment control group has consistently found significant differences in favor of the IY-TCM treatment groups for teacher classroom management strategies, student behavior, and student social skills and social competence (e.g., Herman, Borden, Reinke, & Webster-Stratton, 2011; Raver et al., 2008; Webster-Stratton, Reid, & Hammond, 2001a; Webster-Stratton, Reid, & Hammond, 2004; Webster-Stratton, Reid, & Stoolmiller, 2008). Therefore, while the inclusion of a control group would have further contributed to clarifying the differences between the two active treatment conditions and a no-treatment control group related to these outcomes, the outcomes of this study should not be minimized. In addition, the differences between the intensity of the reading group and the IY-TCM group, as measured by the amount of time that was dedicated to the intervention format for that group, may have affected outcomes as well. Results indicated that the IY-TCM group reported 200 spending significantly more hours related to this study, as defined by training, reading, and/or practicing the material. Therefore, the outcomes may be related to the group training format of the IY-TCM group in comparison to the individual completion of content reading, or they may be related to the amount of time allocated to the intervention. There were also limitations to the external validity of this study. Due to the fact that participants were recruited from two county areas in Michigan, this sample is considered to be limited in its generalizability to participants in other counties within Michigan and to participants outside of Michigan. The majority of the teachers within this study (N = 26) were from GSRP classrooms, which also limits the generalizability to other types of early childhood classrooms. In addition, the specific challenges faced by these study participants in balancing the content of the IY-TCM program with the rating criteria of the PQA and other program assessments may limit the generalizability of these outcomes to the general population who may not have these same requirements or challenges. Future Directions for Research In terms of the current study, future research analyses could more closely examine the outcomes related to the differences between the IY-TCM group and bibliotherapy/reading group over time. Alterations to the HLM modeling procedure or different modeling procedures could be used in order to further examine the outcomes of this study. For example, the group variable could be added within level one of the HLM models and could be examined at time points two and three in order to more closely examine the group differences at these different time points and to examine the slope model in order to determine the differences between the time points. Such future research analyses could also examine whether there was variance between the growth rates for the teachers or students for the different variables examined within this study in 201 order to determine individual growth and potential differences in slope over time. Examination of the different growth rates or changes between pretest to midpoint and midpoint to posttest could also be analyzed in future analyses in order to examine any quadratic changes over time to further inform the theory behind when these variable changes for the IY program may occur. Although this study did not find significant differences for student internalizing outcomes, future research should continue to examine the effect that the various IY programs may have on the internalizing behaviors of children. Research including a greater number of participants, adherence to a six month or greater intervention timeline, and longitudinal followup data collection procedures may demonstrate promising outcomes for internalizing behavior outcomes, especially considering the significant results for student social skills over the period of the intervention for this study. With the established relationship between internalizing behaviors and social skills, this may result in promising findings in longitudinal or follow-up data collection. In addition, future research may want to examine differences in the effects on internalizing and externalizing behaviors between multiple treatment conditions, including a control group, bibliotherapy group, IY-TCM group, a student-centered Dina Dinosaur classroom intervention, and a combined IY-TCM and Dina Dinosaur group over time. This type of research may provide clarifying evidence as to the effects of various intervention supports with different levels of intensity using a longitudinal model with collection of long-term follow-up data, especially in terms of which intervention formats have an effect on teacher level variables, classroom level variables, and student level variables over time. This would also help to clarify whether the IY-TCM program may influence internalizing behaviors measured over a 202 longitudinal period, or whether a combined teacher training and child intervention program would be more effective in influencing these behavior areas. When considering the use of the IY programs within the school setting, future researchers may want to examine the use of these programs within a tiered intervention approach ranging from the least intrusive to the most intrusive intervention format. For example, future research could consider the use of the bibliotherapy treatment for all teachers, and a more specific IYTCM training group for those teachers who have students with risk for behavior problems or social skills deficits. Within this tiered approach, the Dina Dinosaur child program and/or the parent training programs could be utilized as a third tier intensive intervention for students indicating the most intensive needs based on initial data collection, or these programs could be used as a secondary intervention layer for students who do not respond to the IY-TCM program and are in need of additional intervention. This format may serve as an effective way to utilize a response to intervention format of mental health service delivery within the schools in order to meet the needs of a range of teachers and students. 203 APPENDICES 204 Appendix A. Professional Development Log of Hours Classroom Code Number: ________________ Professional Development Log Directions: Please complete this log for each of the six months in order to indicate the time spent in professional development activities related to classroom management strategies. Professional development activities may include: attending training, practicing strategies, or professional reading. Record the date and the number of hours spent on the professional development activities completed. In addition, please list how you applied these strategies below (if applicable). MONTH ONE Date Number of Hours Reading Number of Hours Practicing or (if applicable) Training (if applicable) Notes: Please list how you applied the concepts learned (if applicable) 205 MONTH TWO Date Number of Hours Reading Number of Hours Practicing or (if applicable) Training (if applicable) Notes: Please list how you applied the concepts learned (if applicable) 206 MONTH THREE Date Number of Hours Reading Number of Hours Practicing or (if applicable) Training (if applicable) Notes: Please list how you applied the concepts learned (if applicable) 207 MONTH FOUR Date Number of Hours Reading Number of Hours Practicing or (if applicable) Training (if applicable) Notes: Please list how you applied the concepts learned (if applicable) 208 MONTH FIVE Date Number of Hours Reading Number of Hours Practicing or (if applicable) Training (if applicable) Notes: Please list how you applied the concepts learned (if applicable) 209 MONTH SIX Date Number of Hours Reading Number of Hours Practicing or (if applicable) Training (if applicable) Notes: Please list how you applied the concepts learned (if applicable) 210 Appendix B. Figure 2 Recruitment Flyer 211 Appendix C. Teacher Consent Form Teacher Consent Form What is this study? You are being asked to participate in a research study that will investigate teacher classroom management strategies in relation to classroom variables and student variables. In this study, we are trying to learn more about the outcomes associated with involvement in teacher classroom management training in terms of teacher practices, classroom environment, peer relationships, and child social skills and behaviors. How will I be involved in the study? If you volunteer for this study, you will be randomly assigned to one of two possible groups. One group will receive the Incredible Years Teacher Classroom Management Training which will be led by a certified trainer and will include six full day training sessions. A second group will not receive the group training but will be provided with reading material on classroom management strategies and a schedule for readings to follow. Your involvement in this study will also include completing data collection procedures through completing rating scales about your behaviors and your students’ behaviors and social skills. No information will be collected directly from the students within the classroom. This research study will begin in February 2014 and end in May 2014 in order to include data collection before the study, during the intervention, and after the study. Participation in this research study is completely voluntary. You may choose not to participate at all, or you may refuse to answer certain questions or discontinue your participation at any time. How will be classroom information be kept confidential? Each child, teacher, and classroom will be assigned an ID number which will be used in the place of names in order to maintain confidentiality. The only time that names will be tied to the data will be during the pre-test data collection procedures when child ID scores will be provided to teachers in order for them to select students that are at-risk as target students to focus on. All rating forms and data will be kept in a locked file cabinet and only the researchers will have access to this cabinet. Individual names and identifying information will not be used in any research reports. What risks or benefits may occur if I choose to participate in the study? Teachers and children included within the two intervention groups will receive the benefit of receiving an evidence-based intervention to improve classroom management strategies which may lead to potential improvements in positive classroom atmosphere, teacher-student relationships, peer relationships, and child behavior. All teachers that participate in this study will be provided with a $50 gift card, divided into two $25 gift cards, for their participation in the data collection procedures and research study. The first $25 gift card will be provided to each teacher half way through the study at the midpoint data collection period. The second $25 gift card will be provided to each teacher at the end of the study by the end of May 2014. All teachers within the study will be able to receive State Continuing Education Clock Hours (SCECHs) for continuing education renewal requirements for completing the assigned requirements for their assigned group in addition to the gift cards. 212 What if I have questions or concerns about this study? If you have any concerns or questions regarding this study, you may contact the researcher Erin Rappuhn through email (moraner1@msu.edu), regular mail (Erickson Hall Building, 620 Farm Lane Room 435, East Lansing, MI 48824), or by phone (413-250-8163). You may also contact Dr. John Carlson through email (carlsoj@msu.edu), regular mail (Erickson Hall Building, 620 Farm Lane Room 431, East Lansing, MI, 48824), or by phone (517-432-4856). If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University Human Research Protection Program by phone at 517-355-2180, fax at 517-432-4503, email at irb@msu.edu, or by regular mail at 408 West Circle Drive Room 207 Olds Hall, MSU, East Lansing, MI 48824. Please read and complete the consent form on the next page in order to provide your consent to participation. 213 Informed Consent Form Please sign and mail or email this form back to the researcher Erin Rappuhn (email: moraner1@msu.edu, regular mail: Erickson Hall Building, 620 Farm Lane Room 435, East Lansing, MI 48824). Your signature on this form indicates that you consent to participating in this research study, understand the chance of being assigned to any of the two groups, and agree to participate in data collection procedures. Teacher Information: Teacher’s Name (please print):________________________________________ Teacher’s Age:___________ Teacher’s Gender (M/F):____________ Teacher’s School/Center: _________________________ School/Center Address: ______________________________________________ Number of Students in Classroom: _____________________________ Contact Information: ________________________________________________________________ School Mailing Address _____________________________ Home Phone _______________________________ Cell Phone or Work Phone ________________________________________________________________ Email Address (primary mode of communication used for this study) ___________________________________________ Signature of Teacher 214 _______________ Date Appendix D. Parent Waiver of Consent Form Parent Waiver of Consent Form This document contains information about a research study that your child’s teacher will be involved in related to their classroom management strategies. This document explains how data will be collected for students in the classroom in order to provide information about the outcomes of this teacher training. Your child will be included in data collection within your teacher’s classroom, but an ID number will be used in place of any names and the data will not be used to evaluate your child. However, if you object to your child’s involvement in data collection, you may choose to opt out of the study by filling out the attached form and returning it to your child’s teacher within two weeks. What is this study? You and your child are being asked to participate in a research study that your teacher has volunteered for that will investigate teacher classroom management strategies in relation to classroom variables and student variables. In this study, we are trying to learn more about the outcomes associated with involvement in teacher classroom management training in terms of teacher practices, classroom environment, peer relationships, and child social skills and behaviors. How will my child and I be involved in the study? If you agree to your child’s participation in this study, you are agreeing to the collection of data for your child before the start of the study, during the study, and after the study. The data collected for your child will gather information about behaviors, social skills, and relationships with others through rating scales completed by your child’s teacher, as well as the possibility of additional rating scales completed by yourself and observations of behaviors in the classroom conducted by research assistants. No data will be collected directly from the child and your child will not be directly involved in the intervention. You may be asked to complete rating scales regarding your child’s behavior and social skills during this study if your child is considered atrisk and selected as a target student to focus on. Your child’s involvement in the study may include classroom benefits of their teacher receiving training in classroom management techniques, if the teacher is randomly assigned to the treatment groups. If you do not wish for your child to be included in this study, you may fill out the attached form and return it to your child’s teacher within two weeks to remove your child from data collection. However, your child’s teacher will still participate in the study. This research study will begin in September 2013 and end in May 2014 in order to include data collection before the study, the six month intervention period, and data collection after the study. Participation in this research study is completely voluntary. You may choose not to participate at all, or you may refuse to answer certain questions or discontinue your participation at any time without affecting the treatment that your child receives. How will my child’s information be kept confidential? Each child’s information will be kept confidential. Children included within the study will be assigned an ID number, which will be used in the place of their name on all data collection forms as to maintain confidentiality. The only time that your child’s name will be tied to their data will be during the pre-test data collection procedures when child scores will be provided to teachers 215 in order for them to understand the baseline behavior scores of the children within their classroom and select students who are at-risk as target students to focus on. All rating forms and data will be kept in a locked file cabinet and only the researchers will have access to this cabinet. Individual child names and identifying information will not be used in any reports. What risks or benefits may occur if I choose to have my child participate in the study? Teachers and children have the opportunity to be assigned to one of two groups. Teachers and children included within the two intervention groups will receive the benefit of receiving an evidence-based intervention to improve classroom management strategies which may lead to potential improvements in positive classroom atmosphere, teacher-student relationships, peer relationships, and child behavior. Parents who are asked to complete additional rating scales for their child, if the child is selected as a target student, will be provided with a $10 gift card for the additional data collection procedures. If you are selected to complete additional rating scales, this gift card will be mailed at the end of the study (May 2014) to the address that you will provide on the additional data collection form. What if I have questions or concerns about this study? If you have any concerns or questions regarding this study, you may contact the researcher Erin Rappuhn through email (moraner1@msu.edu), regular mail (Erickson Hall Building, 620 Farm Lane Room 435, East Lansing, MI 48824), or by phone (413-250-8163). You may also contact Dr. John Carlson through email (carlsoj@msu.edu), regular mail (Erickson Hall Building, 620 Farm Lane Room 431, East Lansing, MI, 48824), or by phone (517-432-4856). If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University Human Research Protection Program by phone at 517-355-2180, fax at 517-432-4503, email at irb@msu.edu, or by regular mail at 408 West Circle Drive Room 207 Olds Hall, MSU, East Lansing, MI 48824. *If you do not wish to allow your child to be included in data collection for this study, please complete the attached form and return it to your child’s teacher within two weeks by the following date:_____________. 216 Waiver of Consent If you do not wish to allow your child to be included in data collection for this study, please complete the attached form and return it to your child’s teacher as soon within two weeks by the following date:_____________. I do not want my child to be included in data collection for this study. Child’s Name (please print):________________________________________ Child’s Teacher:________________________ Child’s School: _________________________ Parent’s Name (please print): ________________________________________ ___________________________________________ Signature of Parent 217 _______________ Date REFERENCES 218 REFERENCES Ackerson, J., Scogin, F., McKendree-Smith, N., & Lyman, R.D. (1998). Cognitive bibliotherapy for mild and moderate adolescent depressive symptomatology. Journal of Consulting and Clinical Psychology, 66, 685-690. doi: http://dx.doi.org/10.1037//0022006X.66.4.685 Albano, A., Chorpita, B., & Barlow, D. (2003). Childhood anxiety disorders. In E. Mash & R. Barkley (Eds.), Child psychopathology, New York, NY: The Guilford Press. Albano, A., Marten, P., Holt, C., Heimberg, R., & Barlow, D. (1995). Cognitive-behavioral group treatment for social phobia in adolescence: A preliminary study. Journal of Nervous and Mental Disease, 183, 649-656. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders, fourth edition, text revision (DSM-IV-TR). Washington DC: American Psychiatric Association. Anthony, B.J., Anthony, L.G., Morrel, T.M., & Acosta, M. (2005). Evidence for social and behavior problems in low-income urban preschoolers: Effects of site, classroom, and teacher. Journal of Youth and Adolescence, 34, 31-39. doi: 10.1007/s10964-005-1334-y Baker, J., Grant, S., & Morlock, L. (2008). The teacher-student relationship as a developmental context for children with internalizing or externalizing behavior problems. School Psychology Quarterly, 23, 3-15. doi:10.1037/1045-3830.23.1.3 Baker-Henningham, H., Walker, S., Powell, C., Meeks Gardner, J. (2009). A pilot study of the Incredible Years Teacher Training programme and a curriculum unit on social and emotional skills in community pre-schools in Jamaica. Child: Care, Health, and Development, 35, 624-631. doi:10.1111/j.1365-2214.2009.00964.x Barrera, M., Biglan, A., Taylor, T.K., Gunn, B.K., Smolkowski, K., Black, C., Ary, D.V., & Fowler, R.C. (2002). Early elementary school intervention to reduce conduct problems: A randomized trial with Hispanic and Non-Hispanic children. Prevention Science, 3, 8394. Retrieved from http://www.incredibleyears.com Barrett, P.M., & Turner, C.M. (2004). Prevention strategies. In T.L. Morris & J.S. March (Eds.), Anxiety disorders in children and adolescents, second edition, New York: The Guilford Press. Beauchaine, T., Webster-Stratton, C., & Reid, M. (2005). Mediators, moderators, and predictors of 1-year outcomes among children treated for early-onset conduct problems: A latent growth curve analysis. Journal of Consulting and Clinical Psychology, 73, 371-388. doi:10.1037/0022-006X.73.3.371 219 Beidel, D.C., Morris, T.L., & Turner, M.W. (2004). Social phobia. In T.L. Morris & J.S. March (Eds.), Anxiety disorders in children and adolescents, second edition, New York: The Guilford Press. Beidel, D., Turner, S., Morris, T. (2000). Behavioral treatment of childhood social phobia. Journal of Consulting and Clinical Psychology, 68, 1072-1080. doi:10.1037/0022 006X.68.6.1072 Bernstein, G., Layne, A., Egan, E., & Tennison, D. (2005). School-based interventions for anxious children. Journal of American Academy of Child and Adolescent Psychiatry, 44, 1118-1127. doi:10.1097/01.chi.0000177323.40005.a1 Briggs-Gowan, M., Carter, A., Moye Skuban, E., & McCue Horwitz, S. (2001). Prevalence of social-emotional and behavioral problems in a community sample of 1- and 2-year-old children. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 811819. doi:10.1097/00004583-200107000-00016 Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press. Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives. Developmental Psychology, 22, 723-742. doi: 10.1037//0012 1649.22.6.723 Cadima, J., Leal, T., & Burchinal, M. (2010). The quality of teacher-student interactions: associations with first graders' academic and behavioral outcomes. Journal of School Psychology, 48, 457-482. doi: 10.1111/1467-8624.00611 Caldarella, P., & Merrell, K. (1997). Common dimensions of social skills of children and adolescents: A taxonomy of positive behaviors. School Psychology Review, 26, 264-278. http://www.nasponline.org/publications/spr/about.aspx Campbell, S. (1995). Behavior problems in preschool children: A review of recent research. Journal of Child Psychology and Psychiatry, 36, 113-149. doi:10.1111/j.1469 7610.1995.tb01657.x Carlson, J., Tiret, H., Bender, S., & Benson, L. (2011). The influence of group training in the Incredible Years Teacher Classroom Management Program on preschool teachers’ classroom management strategies. Journal of Applied School Psychology, 27, 134-154. doi:10.1080/15377903.2011.565277 Caspi, A. (2000). The child is father of the man: Personality continuities from childhood to adulthood. Journal of Personality and Social Psychology, 78, 158-172. doi: 10.1037//0022-3514.78.1.158 220 Chafouleas, S.M., Briesch, A.M., Riley-Tillman, T.C., Christ, T.J., Black, A.C., & Kilgus, S.P. (2010). An investigation of the generalizability and dependability of Direct Behavior Rating Single Item Scales (DBR-SIS) to measure academic engagement and disruptive behavior of middle school students. Journal of School Psychology, 48, 219-246. doi:10.1016/j.jsp.2010.02.001 Chafouleas, S.M., Christ, T.J., Riley-Tillman, T.C., Briesch, A.M., & Chanese, J.A. (2007). Generalizability and dependability of direct behavior ratings to assess social behavior of preschoolers. School Psychology Review, 36, 63-79. http://www.nasponline.org/ publications/spr/about.aspx Chafouleas, S.M., Kilgus, S.P., & Hernandez, P. (2009). Using direct behavior rating (DBR) to screen for school social risk: A preliminary comparison of methods in a kindergarten sample. Assessment for Effective Intervention, 34, 214-223. doi: 10.1177/1534508409333547 Chafouleas, S.M., McDougal, J.L., Riley-Tillman, T.C., Panahon, C.J., & Hilt, A.M. (2005). What do daily behavior report cards (DBRCs) measure? An initial comparison for DBRCs with direct observation for off-task behavior. Psychology in the Schools, 42, 669-676. doi: 10.1002/pits.20102 Chafouleas, S.M., Riley-Tillman, T.C., & Christ, T.J. (2009). Direct Behavior Rating (DBR): An emerging method for assessing social behavior within a tiered intervention system. Assessment for Effective Intervention, 34, 195-200. doi: 10.1177/1534508409340391 Chafouleas, S.M., Sanetti, L.M.H., Jaffery, R., & Fallon, L.M. (2012). An evaluation of a classwide intervention package involving self-management and a group contingency on classroom behavior of middle school students. Journal of Behavioral Education, 21, 3457. doi: 10.1007/s10864-011-9135-8 Chafouleas, S.M., Sanetti, L.M.H., Kilgus, S.P., & Maggin, D.M. (2012). Evaluating sensitivity to behavioral change using direct behavior rating single-item scales. Exceptional Children, 78, 491-505. http://journals.cec.sped.org/ec/ Chorpita, B.F., & Southam-Gerow, M.A. (2006). Fears and anxieties. In E.J. Mash & R.A. Barkley (Eds.), Treatment of childhood disorders, third edition, New York: The Guilford Press. Christ, T.J., Riley-Tillman, T.C., Chafouleas, S., & Jaffery, R. (2011). Direct behavior rating: An evaluation of alternate definitions to assess classroom behaviors. School Psychology Review, 40, 181-199. http://www.nasponline.org/publications/spr/about.aspx Collaborative for Academic, Social, and Emotional Learning. (2011). What is SEL? Retrieved from http://casel.org/why-it-matters/what-is-sel/ Coplan, R., & Arbeau, K. (2009). Peer interactions and play in early childhood. In K. Rubin, 221 W. Bukowski, & B. Laursen (Eds.), Handbook of peer interactions, relationships, and groups. New York: The Guilford Press. Delate, T., Gelenberg, A.J., Simmons, V.A., & Motheral, B.R. (2004). Trends in the use of antidepressants in a national sample of commercially insured pediatric patients, 1998 to 2002. Psychiatric Services, 55(4), 387-391. http://ps.psychiatryonline.org/ journal.aspx?journalid=18 Dobbs, J., & Arnold, D. (2009). Relationship between preschool teachers’ reports of children’s behavior and their behavior toward those children. School Psychology Quarterly, 24, 95 105. doi: 10.1037/a0016157 Doll, B., & Yoon, J. (2010). The current status of youth prevention science. In B. Doll, W. Pfohl, & J. Yoon (Eds.), Handbook of youth prevention science. New York: Routledge Taylor & Francis Group. Domitrovich, C., Cortes, R., & Greenberg, M. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. The Journal of Primary Prevention, 28, 67-91. doi:10.1007/s10935-007-0081-0 Doumen, S., Verschueren, K., Buyse, E., De Munter, S., Max, K., & Moens, L. (2009). Further examination of the convergent and discriminant validity of the Student-Teacher Relationship Scale. Infant and Child Development, 18, 502-520. doi: 10.1002/icd.635 Dunson III, R., Hughes, J., & Jackson, T. (1994). Effect of behavioral consultation on student and teacher behavior. Journal of School Psychology, 32, 247-266. doi: 10.1111/1467 8624.00611 Durlak, J., Weissberg, R., Dymnicki, A., Taylor, R., & Schellinger, K. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. doi:10.1111/j.1467 8624.2010.01564.x Dwyer, K., & Van Buren, E. (2010). School mental health: Prevention at all levels. In B. Doll, W. Pfohl, & J. Yoon (Eds.), Handbook of youth prevention science. New York: Routledge Taylor & Francis Group. Eckert, T.L., & Hintze, J.M. (2000). Behavioral conceptions and applications of acceptability: Issues related to service delivery and research methodology. School Psychology Quarterly, 15, 123-148. doi: http://dx.doi.org/10.1037/h0088782 Egger, H., & Angold, A. (2006). Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry, 47, 313-337. doi:10.1111/j.1469-7610.2006.01618.x Emslie, G.J., & Mayes, T.L. (2001). Mood disorders in children and adolescents: 222 psychopharmacological treatment. Biological Psychiatry, 49, 1082-1090. doi:10.1016/S0006-3223(01)01149-0 Epstein, A.S. (2003). Holding your program accountable: Introducing High/Scope’s new preschool program quality assessment (PQA). Reprinted from High Scope Resource, A Magazine for Educators, 11-14. Retrieved from http://www.highscope.org/. Fabes, R., Martin, C., & Hanish, L. (2009). Children’s behaviors and interactions with peers. In K. Rubin, W. Bukowski, & B. Laursen (Eds.), Handbook of peer interactions, relationships, and groups. New York: The Guilford Press. Fox, N.A., Henderson, H.A., Marshall, P.J., Nicholas, K.E., & Ghera, M.M. (2005). Behavioral inhibition: Linking biology and behavior within a developmental framework. Annual Review of Psychology, 56, 235-262. doi:10.1146/annurev.psych.55.090902.141532 Gavigan, K., Kurtts, S., & Mimms, M. (2010). Bibliotherapy as an intervention approach for children and adolescents with emotional disabilities. In C. Norton (Eds.), Innovative Interventions in Child and Adolescent Mental Health, New York: Taylor & Francis e Library. Graham, J.W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549-576. doi: 10.1146/annurev.psych.58.110405.085530 Hamre, B., & Pianta, R. (2001). Early teacher-child relationships and the trajectory of children’s school outcomes through eighth grade. Child Development, 72, 625-638. doi:10.1111/1467-8624.00301 Hay, D., Caplan, M., & Nash, A. (2009). The beginnings of peer relations. In K. Rubin, W. Bukowski, & B. Laursen (Eds.), Handbook of peer interactions, relationships, and groups. New York: The Guilford Press. Henricsson, L., & Rydell, A. (2004). Elementary school children with behavior problems: Teacher-child relations and self-perception. A prospective study. Merrill-Palmer Quarterly, 50, 111- 138. doi:10.1353/mpq.2004.0012 Herman, K., Borden, L., Reinke, W., & Webster-Stratton, C., (2011). The impact of the Incredible Years parent, child, and teacher training programs on children’s co-occurring internalizing symptoms. School Psychology Quarterly, 26, 189-201. doi:10.1037/a0025228 Herman, K., & Ostrander, R. (2007). The effects of attention problems on depression: Developmental, academic, and cognitive pathways. School Psychology Quarterly, 22, 483-510. doi: 10.1037/1045-3830.22.4.483 Hirshfeld-Becker, D.R., Biederman, J., & Rosenbaum, J.F. (2004). Behavioral inhibition. In T.L. Morris & J.S. March (Eds.), Anxiety disorders in children and adolescents, second 223 edition, New York: The Guilford Press. Hirshfeld-Becker, D., Micco, J., Mazursky, H., Bruett, L., & Henin, A. (2011). Applying cognitive-behavioral therapy for anxiety to the younger child. Child and Adolescent Psychiatric Clinics of North America, 20, 349-368. doi:10.1016/j.chc.2011.01.008 Hoagwood, K., & Johnson, J. (2003). School psychology: A public health framework. Journal of School Psychology, 41, 3-21. doi:10.1016/S0022-4405(02)00141-3 Hokanson, J., & Rubert, M. (1991). Interpersonal factors in depression. In D.G. Gilbert & J.J. Connolly (Eds.), Personality, social skills, and psychopathology: An individual difference approach. New York: Plenum. Howes, C. (2000). Social-emotional classroom climate in child care, child-teacher relationships and children’s second grade peer relations. Social Development, 9, 191-204. doi:10.1111/1467-9507.00119 Huberty, T.J. (2008). Best practices in school-based interventions for anxiety and depression. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (Vol. 4, pp. 14731486). Bethesda, MD: National Association of School Psychologists. Hunter, L. (2003). School psychology: A public health framework. Journal of School Psychology, 41, 39-59. doi: 10.1016/S0022-4405(02)00143-7 Hutcherson, S., & Epkins, C. (2009). Differentiating parent- and peer-related interpersonal correlates of depressive symptoms and social anxiety in preadolescent girls. Journal of Social and Personal Relationships, 26, 875-897. doi:10.1177/0265407509345654 Hutchings, J., Daley, D., Jones, K., Martin, P., Bywater, T., & Gwyn, R. (2007). Early results from developing and researching the Webster-Stratton Incredible Years Teacher Classroom Management Training Programme in North West Wales. Journal of Children’s Services, 2, 15-26. Retrieved from http://www.incredibleyears. com/Library/default.asp Hymel, S., Rubin, K.H., Rowden, L., & LeMare, L. (1990). Children’s peer relationships: Longitudinal prediction of internalizing and externalizing problems from middle to late childhood. Child Development, 61, 2004-2021. doi: http://dx.doi.org/10.2307/1130854 Iaquinta, A., & Hipsky, S. (2006). Practical bibliotherapy strategies for the inclusive elementary classroom. Early Childhood Education Journal, 34, 209-213. doi: 10.1007/s10643-0060128-5 Ikeeda, M.J., Neesen, E., & Witt, J.C. (2008). Best practices in universal screening. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (Vol. 2, pp. 103-114). Bethesda, MD: National Association of School Psychologists. 224 Institute of Education Sciences (n.d.). What Works Clearinghouse. Retrieved from http://ies.ed.gov/ncee/wwc/findwhatworks.aspx January, A., Case, R., & Paulson, D. (2011). A meta-analysis of classroom-wide interventions to build social skills: Do they work? School Psychology Review, 40, 242-256. http://www. nasponline.org/publications/spr/about.aspx Jennings, J., & DiPrete, T. (2010). Teacher effects on social and behavioral skills in early elementary school. Sociology of Education, 83, 135-159. doi: 10.1111/1467-8624.00611 Jurecska, D.E., Hamilton, E.B., & Peterson, M.A. (2011). Effectiveness of the Coping Power Program in middle-school children with disruptive behaviours and hyperactivity difficulties. British Journal of Learning and Support, 26, 168-172. doi: http://dx.doi.org/10.1111/j.1467-9604.2011.01499.x Kauffman, J.M. (1997). Characteristics of emotional and behavioral disorders of children and youth: Sixth edition. New Jersey: Merrill/Prentice Hall. Kazdin, A.E. (2000). Perceived barriers to treatment participation and treatment acceptability among antisocial children and their families. Journal of Child and Family Studies, 9, 157-174. http://link.springer.com/journal/10826 Kendall, P.C., Furr, J.M., & Podell, J.L. (2010). Child-focused treatment of anxiety. In J.R. Weisz & A.E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents. New York: The Guilford Press. Kilfedder, C., Power, K., Karatzias, T., McCafferty, A., Niven, K., Chouliara, Z., Galloway, L., & Sharp, S. (2010). A randomized trial of face-to-face counseling versus telephone counseling versus bibliotherapy for occupational stress. Psychology and Psychotherapy, 83, 223-242. doi: 10.1348/147608309X476348 Ladd, G., & Troop-Gordon, W. (2003). The role of chronic peer difficulties in the development of children's psychological adjustment problems. Child Development, 74, 1344-1367. doi: 10.1111/1467-8624.00611 LaFreniere, P., & Capuano, F. (1997). Preventive intervention as means of clarifying direction of effects in socialization: Anxious-withdrawn preschoolers case. Development and Psychopathology, 9, 551-564. doi:10.1017/S0954579497001302 Lavigne, J., Arend, R., Rosenbaum, D., Binns, H., Christoffel, K., & Gibbons, R. (1998). Psychiatric disorders with onset in the preschool years: II. Correlates and predictors of stable case status. Journal of the American Academy of Child and Adolescent Psychiatry, 37, 1255-1261. doi:10.1097/00004583-199812000-00008 Lavigne, J., Gibbons, R., Christoffel, K., Arend, R., Rosenbaum, D., Binns, H., Dawson, N., Sobel, H., & Isaacs, C. (1996). Prevalence rates and correlates of psychiatric disorders 225 among preschool children. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 204-214. doi:10.1097/00004583-199602000-00014 LeBel, T.J., Kilgus, S.P., Briesch, A.M., & Chafouleas, S. (2010). The impact of training on the accuracy of teacher-completed direct behavior ratings (DBRs). Journal of Positive Behavior Interventions, 12, 55-63. doi: 10.1177/1098300708325265 Lee, V.E., & Bryk, A.S. (1989). A multilevel model of the social distribution of high school achievement. Sociology of Education, 62, 172-192. doi: http://dx.doi.org/10.2307/2112866 Lillas, C., & Turnbull, J. (2009). Infant/Child mental health, early intervention, and relationship-based therapies: A neurorelational framework for interdisciplinary practice. New York: W.W. Norton & Co., Inc. Luby, J. (2009). Mood disorders in children and adolescents: Psychopharmacological treatment. The American Journal of Psychiatry, 166, 974-979. doi:10.1176/appi.ajp.2009.08111709 Luby, J., Heffelfinger, A., Koenig-McNaught, A., Brown, K., & Spitznagel, E. (2004). The Preschool Feelings Checklist: A brief and sensitive screening measure for depression in young children. Journal of the American Academy of Child and Adolescent Psychiatry, 43, 708-717. doi:10.1097/01.chi.0000121066.29744.08 Luby, J., Heffelfinger, A., Mrakotsky, C., Brown, K., Hessler, M., Wallis, J., & Spitznagel, E. (2003). The clinical picture of depression in preschool children. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 340-348. doi:10.1097/00004583 200303000-00015 Luby, J., Lenze, S., & Tillman, R. (2012). A novel early intervention for preschool depression: Findings from a pilot randomized controlled trial. Journal of Child Psychology and Psychiatry, 53, 313-322. doi:10.1111/j.1469-7610.2011.02483.x Luby, J., Si, X., Belden, A., Tandon, M., & Spitznagel, E. (2009). Preschool depression: Homotypic continuity and course over 24 months. Archives of General Psychiatry, 66, 897-905. doi:10.1001/archgenpsychiatry.2009.97 Maag, J.W. (2001). Rewarded by punishment: Reflections on the disuse of positive reinforcement in schools. Exceptional Children, 67, 173-186. http://journals.cec.sped.org/ec/ Mash, E., & Barkley, R. (2006). Treatment of childhood disorders (3rd Edition). New York: Guilford Press. Masten, A. & Coatsworth, J. (1998). The development of competence in favorable and unfavorable environments. American Psychologist, 53, 205-220. doi: http://dx.doi.org.proxy1.cl.msu.edu/10.1037/0003-066X.53.2.205 226 McCoach, D.B. (2010). Hierarchical linear modeling. In J.R. Hancock & R.O. Mueller (Eds.), In the reviewer’s guide to quantitative methods in the social sciences. New York: Taylor & Francis. McIntosh, K., Campbell, A.L., Carter, D.R., & Dickey, C.R. (2009). Differential effects of a tier two behavior intervention based on function of problem behavior. Journal of Positive Behavior Intervention, 11, 82-93. doi: 10.1177/1098300708319127 Merrell, K. (2008). Behavioral, social, and emotional assessment of children and adolescents, 3rd Edition. New York, NY: Taylor & Francis Group, LLC. Merrell, K., & Buchanan, R. (2006). Intervention selection in school-based practice: Using public health models to enhance systems capacity of schools. School Psychology Review, 35, 167-180. doi: 10.1016/S0022-4405(02)00143-7 Mesman, J., & Koot, H. (2001). Early preschool predictors of preadolescent internalizing and externalizing DSM-IV diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1029-1036. doi:10.1097/00004583-200109000-00011 Michigan Department of Education. (n.d.). Great Start Readiness Program. Retrieved from http://www.michigan.gov/documents/mde/GSRP-Overview_410757_7.pdf. Morawski, C.M. (1997). A role for bibliotherapy in teacher education. Reading Horizons, 37, 243-259. Retrieved from http://scholarworks.wmich.edu/reading_horizons/ Morris, T.L. (2004). Social development. In T.L. Morris & J.S. March (Eds.), Anxiety disorders in children and adolescents, second edition, New York: The Guilford Press. Morris, T.L., & March, J.S. (2004). Anxiety Disorders in Children and Adolescents: Second Edition. New York: The Guilford Press. New Freedom Commission on Mental Health, Achieving the Promise: Transforming mental health care in America. Executive Summary. DHHS Pub. No. SMA-03-3831. Rockville, MD: 2003. Normandeau, S. & Guay, F. (1998). Preschool behavior and first-grade school achievement: The meditational role of cognitive self-control. Journal of Educational Psychology, 90, 111-121. doi:10.1037//0022-0663.90.1.111 O’Connor, E., & McCartney, K. (2006). Testing associations between young children’s relationships with mothers and teachers. Journal of Educational Psychology, 98, 97-98. DOI:10.1037/0022-0663.98.1.87 Ogg, J.A., & Carlson, J.S. (2009). The self-administered Incredible Years Parent Training Program: Perceived effectiveness, acceptability, and integrity with children exhibiting 227 symptoms of attention-deficit/hyperactivity disorder. Journal of Evidence-Based Practices for Schools, 10, 143-166. Retrieved from http://www.incredibleyears.com Perry, B. (1998). Neurophysiological aspects of anxiety disorders in children. The Child Trauma Academy. Retrieved from www.ChildTrauma.org. Peugh, J.L. (2010). A practical guide to multilevel modeling. Journal of School Psychology, 48, 85-112. doi: 10.1016/j.jsp.2009.09.002 Pianta, R.C. (2001). Student Teacher Relationship Scale Professional Manual. Psychological Assessment Resources, Inc. Retrieved from http://curry.virginia.edu/academics/ directory/robert-c.-pianta/measures Pine, D., Cohen, P., Gurley, D., Brook, J., & Ma, Y. (1998). The risk for early-adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Archives of General Psychiatry, 55, 56-64. Retrieved from http://archpsyc.jamanetwork.com/ Qi, C. & Kaiser, A. (2003). Behavior problems of preschool children from low income families: Review of the literature. Topics in Early Childhood Special Education, 23, 188-216. doi:10.1177/02711214030230040201 Rapee, R., Kennedy, S., Ingram, M., Edwards, S., & Sweeney, L. (2005). Prevention and early intervention of anxiety disorders in inhibited preschool children. Journal of Consulting and Clinical Psychology, 73, 488-497. doi:10.1037/0022-006X.73.3.488 Raudenbush, S., & Bryk, A.S. (1986). A hierarchical model for studying school effects. Sociology of Education, 59, 1-17. doi: http://dx.doi.org/10.2307/2112482 Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods second edition. Thousand Oaks, CA: Sage Publications, Inc. Raver, C., Jones, S., Li-Grining, C., Metzger, M., Champion, K., & Sardin, L. (2008). Improving preschool classroom processes: Preliminary findings from a randomized trial implemented in Head Start settings. Early Childhood Research Quarterly, 63, 253-255. doi:10.1016/j.ecresq.2007.09.001 Reid, M.J., Webster-Stratton, C., & Beauchaine, T. (2001). Parent training in head start: A comparison of program response among African American, Asian American, Caucasian, and Hispanic mothers. Prevention Science, 2, 209-227. http://www.springer.com/ public+health/journal/11121 Reinke, W., Stormont, M., Webster-Stratton, C., Newcomer, L., & Herman, K. (2012). The incredible years teacher classroom management program: using coaching to support generalization to real-world classroom settings. Psychology in the Schools, 49, 416-428. doi: 10.1002/pits 228 Reschley, D. (2008). School psychology paradigm shift and beyond. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (Vol. 1, pp. 3-16). Bethesda, MD: National Association of School Psychologists. Reynolds, C., & Kamphaus, R. (2004). Behavior Assessment System for Children Second Edition: Manual. Circle Pines, Minnesota: AGS Publishing. Riley-Tillman, T.C., Chafouleas, S.M., Christ, T., Briesch, A.M., & LeBel, T.J. (2009). The impact of item wording and behavioral specificity on the accuracy of direct behavior ratings (DBRs). School Psychology Quarterly, 24, 1-12. doi: 10.1037/a0015248 Riley-Tillman, T.C., Chafouleas, S.M., Sassu, K.A., Chanese, J.A., & Glazer, A.D. (2008). Examining the agreement of direct behavior ratings and systematic direct observation data for on-task and disruptive behavior. Journal of Positive Behavior Interventions, 10, 136-143. doi: http://dx.doi.org/10.1177/1098300707312542 Roblek, T., & Piacentini, J. (2005). Cognitive-behavior therapy for childhood anxiety disorders. Child and Adolescent Psychiatric Clinics of North America, 14, 863-876. doi: 10.1016/j.chc.2005.06.001 Rose, A., Carlson, W., Luebbe, A., Schwartz-Mette, R., Smith, R., & Swenson, L. (2011). Predicting difficulties in youth’s friendships: Are anxiety symptoms as damaging as depressive symptoms? Merrill-Palmer Quarterly, 57, 244-262. doi:10.1353/mpq.2011.0013 Rose-Krasnor, L., & Denham, S. (2009). Social-emotional competence in early childhood. In K. Rubin, W. Bukowski, & B. Laursen (Eds.), Handbook of peer interactions, relationships, and groups. New York: The Guilford Press. Rubin, K., Bowker, J., & Kennedy, A. (2009). Avoiding and withdrawing from the peer group. In K. Rubin, W. Bukowski, & B. Laursen (Eds.), Handbook of peer interactions, relationships, and groups. New York: The Guilford Press. Rubin, K., Bukowski, W., & Laursen, B. (2009). Handbook of Peer Interactions, Relationships, and Groups. New York: The Guilford Press. Rubin, K., Coplan, R., & Bowker, J. (2009). Social withdrawal in childhood. Annual review of psychology, 60, 141-71. doi:10.1146/annurev.psych.60.110707.163642 Sameroff, A.J., & Fiese, B.H. (2000). Transactional regulation: The developmental ecology of early intervention. In J. Shonkoff, & S. Meisels (Eds.), Handbook of early childhood intervention: Second Edition. New York: Cambridge University Press. Schoff D’Éramo, K., & Francis, G. (2004). Cognitive-behavioral psychotherapy. In T.L. Morris & J.S. March (Eds.), Anxiety disorders in children and adolescents, second 229 edition, New York: The Guilford Press. Schultz, B.L., Richardson, R.C., Barber, C.R., & Wilcox, D. (2011). A preschool pilot study of connecting with others: Lessons for teaching social and emotional competence. Early Childhood Education Journal, 39, 143-148. doi: 10.1007/s10643-011-0450-4 Segool, N., & Carlson, J. (2008). Efficacy of cognitive-behavioral and pharmacological treatments for children with social anxiety. Depression and Anxiety, 25, 620-631. doi:10.1002/da.20410 Shechtman, Z. (2000). An innovative intervention for treatment of child and adolescent aggression: An outcome study. Psychology in the Schools, 37, 157-167. doi: http://dx.doi.org/10.1002/(SICI)1520-6807(200003)37:2<157::AID-PITS7>3.0.CO;2-G Shernoff, E., & Kratochwill, T. (2007). Transporting an evidence-based classroom management program for preschoolers with disruptive behavior problems to a school: An analysis of implementation, outcomes, and contextual variables. School Psychology Quarterly, 22, 449-472. doi:10.1037/1045-3830.22.3.449 Silverman, W., Kurtines, W., Jaccard, J., Pina, A. (2009). Directionality of change in youth anxiety treatment involving parents: An initial examination. Journal of Consulting and Clinical Psychology, 77, 474-485. doi:10.1037/a0015761 Snyder, J., Low, S., Schultz, T., Barner, S., Moreno, D., Garst, M., Leiker, R., Swink, N., Schrepferman, L. (2011). The impact of brief teacher training on classroom management and child behavior in at-risk preschool settings: Mediators and treatment utility. Journal of Applied Developmental Psychology, 32, 336-345. doi:10.1016/j.appdev.2011.06.001 Spence, S., Donovan, C., & Brechman-Toussaint, M. (2000). The treatment of childhood social phobia: The effectiveness of a social skills training-based, cognitive-behavioural intervention, with and without parental involvement. Journal of Child Psychology and Psychiatry, 41, 713-726. doi: http://dx.doi.org.proxy2.cl.msu.edu/10.1111/1469 7610.00659 Stark, K.D., Sander, J., Hauser, M., Simpson, J., Schnoebelen, S., Glenn, R., & Molnar, J. (2006). Depressive disorders during childhood and adolescence. In E.J. Mash & R.A. Barkley (Eds.), Treatment of childhood disorders, third edition, New York: The Guilford Press. Stice, E., Shaw, H., Bohon, C., Marti, C.N., & Rohde, P. (2009). A meta-analytic review of depression prevention programs for children and adolescents: Factors that predict magnitude of intervention effects. Journal of Consulting and Clinical Psychology, 77, 486-503. doi:10.1037/a0015168 Strobel, D. (2011). Bibliotherapy: An intervention designed for siblings of children with autism. 230 Retrieved from ProQuest Information and Learning: Dissertation Abstracts International Section A: Humanities and Social Sciences. Sutherland, K., Lewis-Palmer, T., Stichter, J., & Morgan, P. (2008). Examining the influence of teacher behavior and classroom context on the behavioral and academic outcomes for students with emotional or behavioral disorders. The Journal of Special Education, 41, 223-233. doi: 10.1111/1467-8624.00611 Tan, C.S. (2007). Test Review: Reynolds, C. R., & Kamphaus, R. W. (2004). Behavior assessment system for children (2nd ed.). Circle Pines, MN: American Guidance Service. Assessment for Effective Intervention, 32, 121-124. doi: 10.1177/15345084070320020301 The Incredible Years, Inc. (2012). The Incredible Years Program. Retrieved from http://www.incredibleyears.com/ Thompson, R. (2005). The course and correlates of mental health care received by young children: Descriptive data from a longitudinal urban high-risk sample. Children and Youth Services Review, 27, 39-50. doi:10.1016/j.childyouth.2004.07.003 University of Connecticut. (2010). Direct Behavior Ratings. Retrieved from http://www.directbehaviorratings.com/cms/ U.S. Department of Health and Human Services. (1999). Mental health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services U.S. Department of Justice Office of Justice Programs. (n.d.). Program Profile: The Incredible Years. Retrieved from http://www.crimesolutions.gov/ProgramDetails.aspx?ID=194 Vasa, R.A., & Pine, D.S. (2004). Neurobiology. In T.L. Morris & J.S. March (Eds.), Anxiety disorders in children and adolescents, second edition, New York: The Guilford Press. Webb, M.L., & Neuharth-Pritchett, S. (2011). Examining factorial validity and measurement invariance of the Student-Teacher Relationship Scale. Early Childhood Research Quarterly, 26, 205-215. doi: http://dx.doi.org/10.1016/j.ecresq.2010.09.004 Webster-Stratton, C. (2008). The Incredible Years: Teachers and Children Series, Leader’s Guide. Seattle, WA: Incredible Years, Inc. Webster-Stratton, C. (2011). The Incredible Years: Parents, teachers, and children’s training series. Seattle, WA: Incredible Years, Inc. Webster-Stratton, C. (2012). Incredible teachers: Nurturing children’s social, emotional, and academic competence. Seattle, WA: Incredible Years, Inc. Webster-Stratton, C., & Herman, K. (2008). The impact of parent behavior-management 231 training on child depressive symptoms. Journal of Counseling Psychology, 55, 473-484. doi:10.1037/a0013664 Webster-Stratton, C., & Herman, K. (2010). Disseminating Incredible Years Series early intervention programs: Integrating and sustaining services between school and home. Psychology in the Schools, 47, 36-54. doi: 10.1002/pits Webster-Stratton, C., & Reid, M.J. (2003). The incredible years parents, teachers and children training series: A multifaceted treatment approach for young children with conduct problems. Guilford Press, New York, NY. Retrieved from http://ezproxy.msu.edu/login ?url=http://search.proquest.com/docview/620236612?accountid=12598 Webster-Stratton, C., & Reid, M.J. (2010). The Incredible Years Program for children from infancy to pre-adolescence: Prevention and treatment of behavior problems. In R.C. Murrihy et al (Eds.), Clinical Handbook of Assessing and Treating Conduct Problems in Youth. Springer Science+Business Media, LLC. doi: 10.1007/978-1-4419-6297-3 Webster-Stratton, C., Reid, M.J., & Hammond, M. (2001a). Preventing conduct problems, promoting social competence: A parent and teacher training partnership in Head Start. Journal of Clinical Child Psychology, 30, 283-302. doi:10.1207/S15374424JCCP3303_2 Webster-Stratton, C., Reid, J., & Hammond, M. (2001b). Social skills and problem-solving training for children with early-onset conduct problems: Who benefits? Journal of Child Psychology and Psychiatry, 42, 943-952. doi:10.1111/1469-7610.00790 Webster-Stratton, C., Reid, M.J., & Hammond, M. (2004). Treating children with early-onset conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child and Adolescent Psychology, 33, 105-124. doi:10.1207/S15374424JCCP3301_11 Webster-Stratton, C., Reid, M.J., & Stoolmiller, M. (2008). Preventing conduct problems and improving school readiness: Evaluation of the Incredible Years Teacher and Child Training Programs in high-risk schools. Journal of Child Psychology and Psychiatry, 49, 471-488. doi: 10.1111/j.1469-7610.2007.01861.x Webster-Stratton, C., Reinke, W.M., Herman, K.C., & Newcomer, L.L. (2011). The Incredible Years Teacher Classroom Management Training: The methods and principles that support fidelity of training delivery. School Psychology Review, 40, 509-529. http://www.nasponline.org/publications/spr/about.aspx Weist, M., & Paternite, C. (2006). Building an interconnected policy-training-practice-research agenda to advance school mental health. Education & Treatment of Children, 29, 173196. http://www.educationandtreatmentofchildren.net/ Weisz, J., McCarty, C., & Valeri, S. (2006). Effects of psychotherapy for depression in children and adolescents: A meta-analysis. Psychological Bulletin, 132, 132-149. 232 doi:10.1037/0033-2909.132.1.132 Wettig, H., Coleman, A., & Geider, F. (2011). Evaluating the effectiveness of Theraplay in treating shy, socially withdrawn children. International Journal of Play Therapy, 20, 2637. doi:10.1037/a0022666 Williford, A., & Shelton, T. (2008). Using mental health consultation to decrease disruptive behaviors in preschoolers: Adapting an empirically-supported intervention. Journal of Child Psychology and Psychiatry, 49, 191-200. doi:10.1111/j.1469-7610.2007.01839.x Wilson, S., & Thornton, S. (2007). “The factor that makes us more effective teachers”: Two pre-service primary teachers’ experience of bibliotherapy. Mathematics Teacher Education and Development, 9, 21-35. Retrieved from http://www.merga.net.au/node/43?volume=9 Wittchen, H., & Fehm, L. (2003). Epidemiology and natural course of social fears and social phobia. Acta Psychiatrica Scandinavica, 108, 4-18. doi: http://dx.doi.org/10.1034/j.1600-0447.108.s417.1.x Wittchen, H., Nelson, C., & Lachner, G. (1998). Prevalence of mental disorders and psychosocial impairments in adolescents and young adults Prevalence of mental disorders and psychosocial impairments in adolescents and young adults. Psychological Medicine, 28, 109-126. doi: 10.1017/S0033291797005928 Zhang, X., & Sun, J. (2011). The reciprocal relations between teachers’ perceptions of children’s behavior problems and teacher-child relationships in the first preschool year. The Journal of Genetic Psychology: Research and Theory on Human Development, 172, 176-198. doi:10.1080/00221325.2010.528077 233