A SELF-ADMINISTERED PARENT TRAINING INTERVENTION FOR BUILDING SOCIAL-EMOTIONAL COMPETENCE AMONG LOW-INCOME PRESCHOOLERS: A NON-CONCURRENT MULTIPLE-BASELINE DESIGN ACROSS THREE CASES By Rebecca N. Thomson 2018 A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of School Psychology – Doctor of Philosophy A SELF-ADMINISTERED PARENT TRAINING INTERVENTION FOR BUILDING SOCIAL-EMOTIONAL COMPETENCE AMONG LOW-INCOME PRESCHOOLERS: A NON-CONCURRENT MULTIPLE-BASELINE DESIGN ACROSS THREE CASES ABSTRACT By Rebecca N. Thomson The development of social-emotional skills during early childhood is essential for future success in school, at home, and in the community. Unfortunately, research indicates that between 10% and 32% of all preschoolers exhibit social-emotional challenges, with the highest rates found among students from low-income backgrounds (e.g., Head Start; McCabe & Altamura, 2011; Webster-Stratton & Hammond, 1998). Parent training interventions have demonstrated effectiveness in building positive parenting practices, and, in turn, improving social-emotional competencies in early childhood (Blueprints for Healthy Youth Development, 2016a). However, due to numerous practical barriers (e.g., scheduling problems, cost, transportation issues), these interventions are unavailable to and underutilized by families in low-income communities. Self- administered parent training programs have the potential to overcome these practical barriers, while still facilitating effective social-emotional outcomes for children (Elgar & McGrath, 2003). Studies have begun to explore parent training programs delivered in self-administered formats (e.g., Incredible Years parent training program, Triple P program), but some challenges to their widespread implementation remain (e.g., high cost of program materials, time-intensive training, significant time commitment for families). The parent training guide included as a part of the Devereux Early Childhood Assessment, Second Edition (DECA-P2) program (LeBuffe & Naglieri, 2012; Mackrain & Cairone, 2013) has the potential to be effective as an early intervention tool by removing some of the barriers inherent within other self-administered programs. Using a non-concurrent, multiple-baseline design, the present study explored the effectiveness, integrity, and acceptability of the DECA-P2 self-administered parent training guide with parents of Head Start preschoolers (N = 3). A replicated intervention effect was not documented for any of the child-level variables (i.e., social-emotional competence, behavior concerns) or parent-level variables (i.e., positive parenting practices, negative parenting practices, parenting stress), indicating that the DECA-P2 parent training intervention was not effective for these families. Parents reported being able to implement the intervention with high levels of integrity (i.e., above 80%) and rated the program as being moderately-to-highly acceptable. Copyright by REBECCA N. THOMSON 2018 ACKNOWLEDGEMENTS I am very thankful for the support (both professionally and personally) I received from many individuals throughout the dissertation process. First, I am grateful for the guidance, feedback, and encouragement provided by my dissertation chair, Dr. John Carlson, which allowed me to develop and carry out a rigorous project, as well as to continue growing as a scholar and researcher. Dr. Carlson was also instrumental in helping me to identify and secure necessary funding sources, including the Society for the Study of School Psychology Dissertation Grant and the Michigan State University Dissertation Completion Fellowship. Second, this project would not have been possible without the recruitment efforts of Head Start professionals across the mid-Michigan area. In particular, I would like to acknowledge Marilyn Gatewood (Family and Community Partnership Specialist, Washtenaw Intermediate School District), Katherine Brown (Family Support Specialist, Beatty Early Learning Center), Heather Berg (Family Support Specialist, Ford Early Learning Center), Dawn Bailey (Family Support Specialist, Perry Early Learning Center), Charlie Basham (Family Service Worker, Whitmore Lake Head Start & GSRP), Erika Lovaas (Mental Health Specialist, C.A.C.S Head Start), and Jennifer Lange (Head Start/GSRP Mental Health and Disabilities Coordinator, Livingston Intermediate School District) for their willingness to share study information with colleagues and families. Third, I am very appreciative of the feedback and support provided by my dissertation committee members, Dr. Kristin Rispoli, Dr. Summer Ferreri, and Dr. Troy Mariage. The ideas and perspectives shared following my dissertation proposal were incredibly valuable for improving the quality of my final dissertation project. I am also thankful for the efforts of my v two project assistants, Lauren Drew (Michigan State University Education Specialist student in School Psychology) and Julia Porter (Michigan State University undergraduate student). Last, but certainly not least, my family and friends have been an important part of the dissertation process. I am forever grateful for the support of my husband, Dave Kelly, whose consistent encouragement, positive attitude, and willingness to listen helped me to stay persistent and overcome challenges along the way. I am also lucky to have a wonderful cohort of fellow doctoral students who have supported me in all of my endeavors throughout graduate school. Finally, I want to thank my parents (Dan and Cathie Thomson) and sisters (Jennifer Lamberts, Julie Thomson) for being a fantastic support system and always believing in me. vi TABLE OF CONTENTS LIST OF TABLES………………………………………………………………………………. xi LIST OF FIGURES……………………………………………………………………………... xii CHAPTER 1……………………………………………………………………………………… 1 INTRODUCTION………………………………………………………………………………... 1 CHAPTER 2……………………………………………………………………………………...12 LITERATURE REVIEW………………………………………………………………………...12 Role of Parents in Early Childhood Social-Emotional Development……………………….. 12 Impact of poverty………………………………………………………………………... 14 Family stress theory…………………………………………………………………. 14 Resilience theories…………………………………………………………………... 15 Parenting Practices that Impact Early Childhood Social-Emotional Development…………. 19 Negative parenting practices…………………………………………………………….. 20 Social interaction learning theory…………………………………………………… 21 Positive parenting practices……………………………………………………………... 23 Head Start: An At-Risk Population…………………………………………………………..24 Child outcomes…………………………………………………………………………...25 Evidence-Based Parent Training Programs for Preschoolers……………………………….. 27 Parent Management Training – Oregon model (PMTO) ……………………………….. 30 Child FIRST……………………………………………………………………………... 32 HighScope preschool……………………………………………………………………. 34 Incredible Years parent training program……………………………………………….. 35 Triple P program………………………………………………………………………… 37 Summary………………………………………………………………………………… 38 Self-Administered Parent Training Interventions…………………………………………… 40 Devereux Early Childhood Assessment, Second Edition (DECA-P2; LeBuffe & Naglieri, 2012) parent training guide (Mackrain & Cairone, 2013)………………………………..41 Single-Case Design Research………………………………………………………………...45 Rationale for single-case design……………………………………………………….…45 Single-case design standards……………………………………………………………..46 Advantages of multiple-baseline designs within intervention research……………….… 47 Analysis of single-case design data……………………………………………………... 50 Visual analysis of single-case design data…………………………………………... 51 Statistical analysis of single-case design data……………………………………….. 54 Randomization tests……………………………………………………………... 54 Effect size calculations……………………………………………………….......55 Clinical significance of single-case design data…………………………………….. 56 Research Questions and Hypotheses………………………………………………………... 57 CHAPTER 3…………………………………………………………………………………….. 64 METHODS……………………………………………………………………………………… 64 vii Participants…………………………………………………………………………………... 64 Measures…………………………………………………………………………………….. 66 Multiple-baseline measures………………………………………………………………66 Primary measures……………………………………………………………………. 66 Direct behavior rating (DBR; Chafouleas, Riley-Tillman, et al., 2009)………… 66 Goal attainment scaling (GAS; Kiresuk & Sherman, 1968)…………………….. 68 Supplemental measures……………………………………………………………… 71 Devereux Early Childhood Assessment – Second Edition (DECA-P2) rating scale (LeBuffe & Naglieri, 2012)…………………………………………………… 71 Parent Behavior Inventory (PBI; Lovejoy et al., 1999)…………………………. 72 Coder Impressions Inventory (CII; Webster-Stratton, 1998)…………………….73 Relationship between measures……………………………………………………... 75 Pre-test/post-test measures………………………………………………………………. 77 Parenting Stress Index – Short Form (PSI-SF; Abidin, 2012)………………………. 77 Descriptive measures……………………………………………………………………. 78 Demographics……………………………………………………………………….. 78 Integrity……………………………………………………………………………… 78 Acceptability………………………………………………………………………… 79 Procedures…………………………………………………………………………………… 80 Research design…………………………………………………………………………..80 Recruitment……………………………………………………………………………… 80 Inclusion and exclusion criteria…………………………………………………………. 81 Project personnel and training……………………………………………………………82 Treatment phases…………………………………………………………………………83 Pre-test………………………………………………………………………………. 83 Baseline (A)…………………………………………………………………………. 84 Intervention (B)……………………………………………………………………… 84 Post-test……………………………………………………………………………… 85 Follow-up……………………………………………………………………………. 85 Data Analysis………………………………………………………………………………... 85 Research questions 1 and 2……………………………………………………………… 85 Research question 3………………………………………………………………………87 Research question 4……………………………………………………………………... 87 Research question 5……………………………………………………………………... 87 CHAPTER 4……………………………………………………………………………………...88 RESULTS……………………………………………………………………………………….. 88 Research Question 1………………………………………………………………………….88 Social-emotional competence…………………………………………………………… 88 Primary measure……………………………………………………………………...88 Supplemental measures……………………………………………………………… 90 Follow-up……………………………………………………………………………. 93 Behavior concerns……………………………………………………………………….. 94 Primary measure……………………………………………………………………...94 Supplemental measures……………………………………………………………… 94 Follow-up……………………………………………………………………………. 96 viii Research Question 2………………………………………………………………………….97 Positive parenting practices………………………………………………………………97 Primary measure……………………………………………………………………...97 Supplemental measures……………………………………………………………… 97 Follow-up…………………………………………………………………………... 101 Negative parenting practices…………………………………………………………… 102 Primary measure…………………………………………………………………… 102 Supplemental measures…………………………………………………………….. 102 Follow-up…………………………………………………………………………... 104 Research Question 3……………………………………………………………………….. 104 Research Question 4……………………………………………………………………….. 105 Research Question 5………………………………………………………………………...106 Individual Improvements…………………………………………………………………... 109 CHAPTER 5…………………………………………………………………………………… 110 DISCUSSION………………………………………………………………………………….. 110 Child Social-Emotional Competence and Behavior Concerns…………………………….. 112 Minimal room for improvement in social-emotional competence……………………...112 Lack of change in parenting practices…………………………………………………..115 Parenting Practices…………………………………………………………………………. 116 Minimal room for improvement in positive parenting practices………………………. 116 Factors associated with living in poverty………………………………………………. 120 Characteristics of the intervention……………………………………………………... 121 Parenting Stress…………………………………………………………………………...... 123 Integrity…………………………………………………………………………………….. 125 Acceptability……………………………………………………………………………….. 127 Limitations…………………………………………………………………………………. 129 Reliability, validity, and sensitivity of measures………………………………………. 129 Parent-reported measures………………………………………………………………. 132 Inclusion criteria………………………………………………………………………...133 Changes in measurement interval length between baseline and intervention phases….. 134 Implications for Research…………………………………………………………………...134 Implications for Practice…………………………………………………………………… 136 APPENDICES………………………………………………………………………………......139 Appendix A: Inclusion and Exclusion Criteria…………………………………………….. 140 Appendix B: Direct Behavior Rating Form………………………………………………... 141 Appendix C: Goal Attainment Scale (Example)…………………………………………… 142 Appendix D: Parent Behavior Inventory……………………………………………………143 Appendix E: Coder Impressions Inventory…………………………………………………145 Appendix F: Demographic Form…………………………………………………………... 147 Appendix G: Weekly Integrity Checklist…………………………………………………...148 Appendix H: Treatment Evaluation Questionnaire – Parent Form (TEQ-P) Acceptability Scale……………………………………………………………………………………… 149 Appendix I: Recruitment Flyer…………………………………………………………….. 150 Appendix J: Example Workbook Week…………………………………………………….151 ix Appendix K: Visual Analysis Guide………………………………………………………..156 REFERENCES………………………………………………………………………………….161 x LIST OF TABLES Table 1. Evidence-Based Parent Training Programs for Building Social-Emotional Competence in Early Childhood…………………………………………………………………………… 2 Table 2. Ten Steps Along the Intervention Research Trajectory (Sheridan, 2014)………………. 9 Table 3. Social-Emotional Competence in the Early Childhood Population…………………….18 Table 4. Blueprints for Healthy Youth Development Program Criteria………………………… 28 Table 5. Parenting Strategies from the DECA-P2 Parent Training Guide (Mackrain & Cairone, 2013)………………………………………………………………………………………… 43 Table 6. Six Features of Visual Analysis for Single-Case Design……………………………… 53 Table 7. Research Questions, Variables, Hypotheses, Measures, and Data Analyses………….. 58 Table 8. Demographic Characteristics and Baseline Ratings for the Four Enrolled Participants………………………………………………………………………………….. 65 Table 9. Assessments Planned for Each Phase………………………………………………….. 69 Table 10. Correlations between Outcome Measures……………………………………………. 76 Table 11. Elements of the Present Study’s Research Design in Alignment with What Works Clearinghouse Single-Case Design Standards……………………………………………… 81 Table 12. Means and Effect Sizes for Measures of Child Social-Emotional Competence and Behavior Concerns………………………………………………………………………….. 91 Table 13. Means and Effect Sizes for Measures of Positive and Negative Parenting Practices... 99 Table 14. Changes in PSI-SF Scores from Pre-Test to Post-Test………………………………105 Table 15. Average Percent Completion of Intervention Activities…………………………….. 106 Table 16. Answers on the TEQ-P Across Participants………………………………………… 107 Table 17. Parent Responses to Exit Interview Questions……………………………………… 108 xi Figure 1. Witt & Elliott’s (1985) Model of Treatment Acceptability…………………………....... 7 LIST OF FIGURES Figure 2. The Resilience Model (LeBuffe & Naglieri, 2012)……………………………………. 17 Figure 3. Relationship of Social and Emotional Competencies to Protective Factors (LeBuffe & Naglieri, 2012)………………………………………………………………………………. 18 Figure 4. Three-Step Coercive Sequence (Patterson, 2016)…………………………….………. 22 Figure 5. Parent DBR Ratings – Average Social-Emotional Competence……………………… 89 Figure 6. Parent DBR Ratings – Individual Social-Emotional Competence Items (Initiative, Self- Regulation, Attachment/Relationships)……………………………………………………... 90 Figure 7. Parent DBR Ratings – Behavior Concerns…………………………………………….95 Figure 8. GAS Ratings – Positive Parenting Practices…………………………………………..98 Figure 9. GAS Ratings – Negative Parenting Practices………………………………………...103 xii CHAPTER 1 Introduction The development of social-emotional competence in early childhood is an essential prerequisite to many future successes (e.g., school readiness, academic success, reduced rates of delinquency; McCabe & Altamura, 2011). From an early age, parents play a key role in children’s social-emotional development (Rimm-Kaufman & Wanless, 2012). In the interest of evidence-based practice (EBP), researchers have worked to “identify, disseminate, and promote the adoption of practices with demonstrated research support” (Kratochwill, 2007, p. 829), specifically focusing on parent training as an intervention for building social-emotional competence in early childhood (Reid, Webster-Stratton, & Baydar, 2004). To date, several parent training programs have been identified as evidence-based interventions (EBIs), or “research- based prevention and intervention programs that have a strong empirical basis and have demonstrated positive outcomes in multiple well-designed studies” (Forman et al., 2013), for building social-emotional competence among preschoolers (Blueprints for Healthy Youth Development, 2016a; see Table 1). While EBIs exist, research shows that these programs are less effective and/or underutilized among children and families from low-income backgrounds. In general, the dissemination and implementation of EBIs with children and families living in poverty has proved to be challenging due to numerous practical barriers. For instance, low-income families may lack knowledge about the availability of EBIs, have difficulties with transportation, be unable to access the locations where services are delivered, not be covered by health insurance, have insufficient time to schedule services, and/or experience challenges navigating the complex mental health service system. These barriers lead to low rates of service access (20%) and high 1 Table 1. Evidence-Based Parent Training Programs for Building Social-Emotional Competence in Early Childhood Program Name Author/Year Parent Management Training – Oregon model (PMTO) Sessions include: • Psychoeducation/ Five core practices: 1. Skill Forgatch & Patterson (2010) Treatment Techniques Therapeutic Content Format 60- to 90- minute individual or group sessions (therapist- led) Research Evidence Randomized controlled trial (N = 112) documented increases in child social-emotional competence (Amlund Hagen, Ogden, & Bjørnebekk, 2011; Ogden & Amlund Hagen, 2008). Randomized controlled trial (N = 157) documented decreases in child behavior concerns (Lowell et al., 2011). encouragement 2. Limit setting 3. Monitoring 4. Problem-solving 5. Positive involvement didactics • Role play • Home practice assignments Two core components: • “System of care” approach (i.e., connection to comprehensive, integrated services and supports) • Parent-child dyadic psychotherapy Child FIRST Lowell et al. (2011) Consistent home visits by a clinical team (mental health professionals and care coordinators) Content is guided by parental need rather than a fixed curriculum. Therapist works conjointly with the parent and child to enhance: • Maternal reflectivity and empathy with the child’s experience • Maternal sensitivity and responsiveness to the child 2 Table 1 (cont’d) HighScope preschool curriculum Weikart et al. (1971); Hohmann et al. (1979); Hohmann & Weikart (1995) Weekly 90- minute home visits conducted by teachers; monthly parent group meetings Incredible Years parent training program Webster- Stratton (2006) Two-hour weekly group sessions (therapist-led using video vignettes) Curriculum model is shared with parents through: • Psychoeducation/ didactics • Modeling The program aims to promote active learning by providing opportunities for children to: • Initiate their own activities • Take responsibility for the completion of chosen activities Sessions utilize: • Group discussion • Self-reflection • Modeling and practice rehearsals • Problem-solving • Sharing of ideas • Support networks Program topics include: • Academic, persistence, and self-regulation coaching • Effective use of praise and encouragement • Proactive discipline • Teaching children beginning problem- solving skills Longitudinal, randomized controlled trial (N = 123) documented increases in child social-emotional competence and decreases in child behavior concerns (Heckman, Pinto, & Savelyev, 2013). Meta-analyses of controlled experimental studies documented improvements in child behavior concerns (d = .27), social-emotional competence (d = .23), and positive parenting (Menting et al., 2013; Pidano & Allen, 2015) 3 Table 1 (cont’d) Triple P Program Sanders (1999) Multilevel prevention program Five core principles: 1. Ensure a safe, engaging environment. 2. Promote a positive learning environment. 3. Use assertive discipline. 4. Maintain reasonable expectations. 5. Take care of oneself as a parent. Meta-analysis of randomized controlled trials documented improvements in child behavior concerns (d = .48), parenting (d = .42), and parental well-being (d = .20; Nowak & Heinrichs, 2008). • Level 1: media and informational strategies • Level 2: 1 to 2 brief consultations; parenting seminars • Level 3: 4 brief consultations; active skills training; parenting tip sheets • Level 4: group-based; psychoeducation/didact ics; active skills training • Level 5: modules on partner communication, mood management and stress coping skills for parents, and additional practice sessions 4 rates of attrition (50%) for low-income children experiencing mental health concerns (Atkins et al., 2006; Gross, Julion, & Fogg, 2001). Given the numerous risk factors that children living in poverty are exposed to daily (e.g., violence, family instability, lack of social support; Qi & Kaiser, 2003), early intervention in the home environment is necessary to promote social-emotional development (Sheridan et al., 2014). Without strong social-emotional skills, which serve as protective factors (LeBuffe & Naglieri, 2012), exposure to these risk factors increases the likelihood of negative developmental outcomes (e.g., truancy, mental illness, delinquency; Andershed & Andershed, 2015). Not surprisingly, approximately one-third of preschoolers from low-income backgrounds (e.g., students enrolled in Head Start) experience social-emotional challenges (Webster-Stratton & Hammond, 1998), which is a considerably higher rate than the 10% to 15% of preschoolers within the general population who present with low social-emotional competence (McCabe & Altamura, 2011). Despite the need within this population, most current EBIs are structured as individual or small-group face-to-face treatments (Rotheram-Borus, Swendeman, & Chorpita, 2012), which minimizes access to families who face considerable practical barriers. To ensure that EBIs reach a broader population of consumers (including hard-to-reach populations, such as children living in poverty), Rotheram-Borus and colleagues (2012) suggest the need for disruptive innovations. Disruptive innovations “[provide] a simpler and less expensive alternative that meets most of the same needs for the majority of consumers. The new service is more accessible, scalable, replicable, and sustainable” (Rotheram-Borus et al., 2012, p. 467). As a disruptive innovation, Rotheram-Borus and colleagues recommend using novel delivery formats, such as self-administered interventions. Self-administered interventions, delivered via written materials (e.g., books) and/or multimedia (e.g., video, audio), are readily 5 available, inexpensive, and can be implemented at any time that is convenient for the client. Self- administered interventions may be disseminated and/or delivered by educators, psychologists, physicians, paraprofessionals, or other healthcare professionals. With such a level of flexibility, self-administered treatment approaches may expand the reach of current EBIs, moving service provision closer to children’s natural environments (Elgar & McGrath, 2003). Self-administered interventions have the potential to maintain the same level of effectiveness as current EBIs for building social-emotional competence in early childhood (Elgar & McGrath, 2003), but are likely to improve levels of integrity and acceptability that inhibit the dissemination and implementation of EBIs among low-income populations. Integrity refers to the degree to which an intervention is being implemented as planned, and acceptability refers to whether an intervention is perceived to be fair, reasonable, feasible, and appropriate by consumers and stakeholders (Forman et al., 2013; Villarreal, Ponce, & Gutierrez, 2015). According to Witt and Elliott (1985), treatment acceptability, treatment integrity, treatment use, and treatment effectiveness are reciprocally related (see Figure 1). If treatment acceptability is high, individuals are more likely to implement the treatment and follow treatment procedures with integrity. If treatment integrity is high, positive behavioral outcomes are more likely, which, in turn, should enhance perceptions of treatment acceptability (Eckert & Hintze, 2000). Given the interdependence of these variables, it is essential to consider treatment integrity and acceptability in order to maximize the potential of effective mental health interventions. Unfortunately, many of the practical barriers that low-income families face (e.g., insufficient time, scheduling problems, cost, difficulties with transportation) negatively impact their ability to complete mental health interventions with integrity (Gross et al., 2001). Self- administered interventions, however, may increase treatment integrity by allowing low-income 6 Figure 1. Witt & Elliott’s (1985) Model of Treatment Acceptability Acceptability of Treatment Effectiveness of Treatment Use of Treatment Integrity of Treatment families to complete program requirements at a time that is most convenient, and to be able to readily access program materials at no (or minimal) cost. Furthermore, there is evidence to suggest that self-administered interventions may be an acceptable treatment delivery format for families, given the high rate of self-help book purchases across the United States, relative to participation in therapy (Rotheram-Borus et al., 2012). Harwood and L’Abate (2010) assert that self-administered interventions may be the most acceptable treatment delivery format for as high as 15% of the population. Previous studies have explored the use of self-administered interventions for various mental health concerns [e.g., attention deficit/hyperactivity disorder (ADHD), sleep problems, depression, disruptive behavior; Elgar & McGrath, 2003]. Meta-analyses of self-administered treatments have found medium to large effect sizes (.57 - .96) when compared to control groups 7 (Marrs, 1995; Scogin, Bynum, Stephens, & Calhoon, 1990). However, despite being widely available to the public, self-administered parent training programs aimed at improving child behavior are not all empirically supported (Hahlweg, Heinrichs, Kuschel, & Feldmann, 2008). There is evidence to show that some self-administered parent training programs are effective, with studies documenting improvements in child social-emotional skills after participation in the self-administered Triple P program (de Graaf, Speetjens, Smit, de Wolff, & Tavecchio, 2008) and reductions in child behavior concerns after the Incredible Years self-administered parent training program (e.g., Webster-Stratton, Kolpacoff, & Hollinsworth, 1988; Webster-Stratton, 1990, 1992). Nonetheless, this body of research is relatively small, and some practical barriers remain among the self-administered interventions investigated to date. For instance, both the Triple P program and the Incredible Years self-administered parent training program require time-intensive facilitator training, costly materials, and ten weeks or more of treatment (Armstrong, Ogg, Sundman-Wheat, and Walsh, 2014; de Graaf et al., 2008; Webster-Stratton et al., 1988). Given these limitations, it is necessary to empirically investigate other self- administered parent training interventions for enhancing children’s social-emotional competence and reducing behavior concerns. One currently available self-administered program is the parent training guide included as a part of the Devereux Early Childhood Assessment, Second Edition (DECA-P2) program (LeBuffe & Naglieri, 2012). The purpose of the DECA-P2 parent training guide is to teach parents about healthy social and emotional development in young children, as well as to provide everyday strategies for enhancing children’s social and emotional skills in the home environment (Mackrain & Cairone, 2013). The characteristics of the guide (e.g., low cost, implementation flexibility, ease of dissemination, alignment with research-based assessment measures and 8 classroom resources) have the potential to overcome the barriers to treatment integrity and acceptability that frequently hinder low-income parents’ participation in parent training programs. In addition, the guide is less expensive and considerably shorter than existing self- administered interventions (e.g., Triple P, Incredible Years; Armstrong et al., 2014; de Graaf et al., 2008; Webster-Stratton et al., 1988), which may even further enhance treatment integrity and acceptability. To effectively develop and test interventions in applied settings, Sheridan (2014) provides a ten-step intervention research trajectory (see Table 2). After identifying an issue or problem and creating intervention strategies (i.e., DECA-P2 parent training guide), it is necessary to conduct a pilot study to assess the feasibility of the intervention within the selected setting (Sheridan, 2014). To date, the DECA-P2 parent training guide has been investigated in one small pilot study (N = 12). In the pilot study, children demonstrated increases in social- emotional competence and reductions in behavior concerns following parents’ participation in the DECA-P2 self-administered parent training program. Furthermore, parents completed the Table 2. Identify an issue or problem Pilot/assess feasibility Evaluate with intensity/precision; small sample Ten Steps Along the Intervention Research Trajectory (Sheridan, 2014) Step Description 1 2 Create strategies 3 4 5 Replicate and extend with new sample, problem, context 6 Develop theory 7 8 Assess mechanisms of change (theory) 9 10 Test effectiveness on a large scale Test on a larger scale Investigate influential contextual/situational variables 9 program with high levels of integrity, and rated the program as being highly acceptable for addressing their child’s needs (Thomson & Carlson, 2016). Given these promising exploratory findings, future research is necessary to explore the intervention in greater depth. The next step along Sheridan’s (2014) intervention research trajectory is to evaluate the intervention with intensity and precision within a small sample (i.e., single-case design). Single- case design is a valuable research method for exploring functional relationships between independent and dependent variables, and, therefore, has “proved essential in the early stages of intervention development, feasibility, and efficacy testing” (Sheridan, 2014, p. 299). Single-case designs allow researchers to explore those who respond to a treatment, as well as those who do not respond (Kratochwill et al., 2010), which can, in turn, inform study replications, theory development, and future larger investigations of intervention effects (i.e., randomized controlled trials; Sheridan, 2014). With current statistical advancements, single-case design studies can also contribute to the evidence-based practice literature through meta-analysis (Burns, 2012; Shadish, 2014). Using a non-concurrent, multiple-baseline design across three participants, the present study examined the effectiveness, integrity, and acceptability of the DECA-P2 parent training guide implemented with low-income children and families (i.e., Head Start students and their parents). More specifically, this study explored changes in child-level (e.g., social-emotional competence, behavior concerns) and parent-level risk and protective factors (e.g., positive parenting practices, negative parenting practices, parenting stress) after participation in the self- administered parent training intervention. This study also explored the integrity and acceptability of the intervention within this population, given that there are many barriers to low-income parents’ participation in mental health interventions (Eamon & Venkataraman, 2003; Gross et 10 al., 2001). Study findings may assist education and mental health professionals working within low-income communities who are considering ways to overcome these barriers and provide needed services to at-risk families. 11 CHAPTER 2 Literature Review In order to address the proposed need for the study, the following sections will describe a) the role of parents in early childhood social-emotional development; b) how negative and positive parenting practices are associated with children’s social-emotional outcomes; c) the importance of social-emotional development for a specific at-risk early childhood population (e.g., Head Start); d) the strengths and weaknesses of current parent training programs for building social-emotional competence in early childhood; e) the utility of self-administered parent training programs for at-risk populations; and f) the appropriateness of single-case design within intervention research. Role of Parents in Early Childhood Social-Emotional Development The first five years of life are a critical period for social and emotional development (Shonkoff & Phillips, 2000). During this time, children’s development is heavily influenced by their interactions with parents in the home setting (Sheridan et al., 2014). The family has been described as “the principal context in which human development takes place” (Bronfenbrenner, 1986, p. 723), especially during early childhood (Vick Whittaker, Harden, See, Meisch, & Westbrook, 2011), and parents are often described as “children’s first teachers” (Hindman, Skibbe, Miller, & Zimmerman, 2010, p. 237). Clearly, parents have a profound influence on children’s social-emotional development (Shepard & Dickstein, 2009), and this has been demonstrated in both direct and indirect ways. For example, parents influence children’s social- emotional development directly via face-to-face interactions that consist of instructions, modeling, commands, and restrictions. Indirect methods of influence include parental decisions about neighborhoods, schools, peers, resources, and other social contacts (Schaffer, 2006). 12 Along with other developmental forces (e.g., genetics), parental influences can either foster or inhibit the development of children’s social-emotional competencies (Schaffer, 2006). Social-emotional competence can be defined as “the ability of children to successfully interact with other children and adults in a way that demonstrates an awareness of, and ability to manage, emotions in an age- and context-appropriate manner” (LeBuffe, Shapiro, & Naglieri, 2009, p. 5). Young children with strong social-emotional competence exhibit qualities such as curiosity, persistence, confidence, and delayed gratification, while qualities such as inattention, aggression, insecurity, and impulsivity are signs of early behavior concerns (Dickstein, 2015). Developmental researchers identify the transition from toddlerhood to the preschool years as an especially vital time for building social-emotional competencies, as it is necessary for children to learn how to regulate their emotions and behavior. In fact, there is evidence to suggest that early delays in social-emotional development (e.g., poor social-emotional competence, high levels of behavior concerns) remain stable from toddlerhood through elementary school and adolescence, and can lead to later conduct disorders, antisocial behavior disorders, and academic problems (Vick Whittaker et al., 2011). The early development of social-emotional competence is essential, as it has profound implications for future wellness. Research shows that social-emotional skills are equally as influential for school success as cognitive and academic skills (Webster-Stratton & Reid, 2004), and the combination of social-emotional and cognitive skills is necessary for success in school and in the workplace. Children’s social-emotional competence in the early years has been found to be predictive of many positive developmental outcomes, including high school and college graduation, stable employment, and reduced rates of substance use and criminal activity (Jones, Greenberg, & Crowley, 2015). Social-emotional competence has also been associated with 13 school readiness, positive attitudes towards school, academic success, and fewer behavior problems in academic and social situations (Bulotsky-Shearer, Domínguez, Bell, Rouse, & Fantuzzo, 2010; McCabe & Altamura, 2011). Given this link with future outcomes, social- emotional functioning has been cited as a major contributing factor to many public health problems, such as substance abuse, obesity, and violence (Jones et al., 2015). Impact of poverty. Unfortunately, children living in poverty are at a disadvantage when it comes to the development of social-emotional competence, as “poverty is considered the single biggest threat to child well-being” (Odgers et al., 2012, p. 705). The harmful effects of poverty on child development are typically viewed as being transmitted through parents (Odgers, 2012), especially during the early childhood years (Duncan, Yeung, Brooks-Gunn, & Smith, 1998). In particular, the stressors associated with living in poverty (e.g., lack of material resources) can make it difficult for parents to maintain consistent, positive interactions with their children (Mortensen & Mastergeorge, 2014) – a necessary criterion for positive social-emotional development (Sheridan et al., 2014). Family stress theory. One major theoretical model used to explain the link between poverty and child outcomes is the family stress theory (Conger, Conger, & Martin, 2010; Conger et al., 1990). According to family stress theory, economic disadvantage is associated with an increase in parenting stress (i.e., an aversive psychological reaction to the demands of being a parent; Deater-Deckard, 1998). This results in a higher likelihood of negative parent-child interactions, which, ultimately, has harmful effects on children’s developmental outcomes. Positive parenting practices, however, have demonstrated “buffering” effects against these negative consequences of poverty. The family stress model, then, highlights the mediating effects of parent mental health (e.g., parenting stress) and parenting practices between poverty 14 and child developmental outcomes (Linver, Brooks-Gunn, & Kohen, 2002; Yeung, Linver, & Brooks-Gunn, 2002). Family stress theory has been previously applied to explain how a family’s level of income can specifically impact children’s social-emotional development. In a recent study with a low-income, early childhood population (e.g., Early Head Start), Vick Whittaker and colleagues (2011) applied the family stress model to investigate children’s social-emotional functioning. Results supported an indirect relationship between parent mental health (e.g., parenting stress) and children’s social-emotional functioning by way of parenting practices (e.g., maternal sensitivity) as a mediator (Vick Whittaker et al., 2011). This highlights the especially important role of parents in low-income communities for influencing children’s social-emotional competencies, and the need for early childhood intervention to focus on improving parent-child interaction. Resilience theories. While it is evident that investing in the development of social- emotional competence at an early age is fundamental for all children, social-emotional competence serves an especially important function as a protective factor for children living in poverty (LeBuffe, Ross, Fleming, & Naglieri, 2013). Even though all children may be exposed to mild (e.g., family discord) or severe risk factors (e.g., divorce, family violence) throughout the course of development (Lamb-Parker, LeBuffe, Powell, & Halpern, 2008), economic disadvantage is viewed as one of the most influential risk factors for children. Poverty is associated with a number of challenges, such as poor support networks, exposure to violence, family instability, and low parental education levels, which interact to place children at a heightened risk for behavior problems (Qi & Kaiser, 2003). A recent meta-analysis found that socioeconomic factors (i.e., low levels of income) were significantly associated with 15 externalizing behavior problems (i.e., delinquency, antisocial behavior, arrests, convictions, police contacts, alcohol/drug use, aggression) throughout childhood, adolescence, and adulthood (Andershed & Andershed, 2015). Risk factors and protective factors act as two opposing variables in the development of resilience, which refers to the ability to adapt positively in the face of adversity (Smith, LeBuffe, Alleyne, Mackrain, & Likins, 2014). Risk factors (e.g., economic disadvantage, behavior problems, family stress) increase the likelihood of negative developmental outcomes (e.g., truancy, mental illness, delinquency), while protective factors (e.g., social-emotional competence, positive parent-child relationships) increase the likelihood of positive developmental outcomes (Andershed & Andershed, 2015; Wright, Masten, & Narayan, 2013). Essentially, protective factors offset or buffer against the negative influences of risk and adversity (LeBuffe & Naglieri, 2012; see Figure 2). For this reason, promoting protective factors is especially important for young children who experience a high number of risk factors, such as those who live in poverty (Webster-Stratton & Taylor, 2001). Risk and protective factors exist within the individual (i.e., child), the family, and the environment (e.g., school, neighborhood), and all contribute uniquely to mental health outcomes (Garmezy, 1991; see Figure 3). Individual protective factors include child characteristics like temperament, intelligence, personality, and social-emotional competence. A specific focus on the promotion of social-emotional competence is ideal for three main reasons: 1) these skills are linked to successful outcomes in many domains (e.g., school, social relationships), 2) these skills are malleable, and 3) these skills are directly influenced by caregivers (e.g., parents, teachers). Luckily, building social-emotional competence does not require intensive efforts; rather, these 16 Figure 2. The Resilience Model (LeBuffe & Naglieri, 2012) skills can emerge through everyday experiences and interactions (LeBuffe & Naglieri, 2012). DiCorcia and Tronick (2011) refer to this as the “Everyday Stress Resilience Hypothesis,” in which resilience is viewed as “a process of regulating everyday life stressors” (p. 1593). According to this hypothesis, when children are able to cope with smaller, everyday stressors, this will prepare them to cope with later, more significant stressors (DiCorcia & Tronick, 2011). In early childhood, three specific behavioral indicators have been found to help children cope with daily stressors, and, therefore, are viewed as protective: 1) initiative, 2) self-regulation, and 3) attachment/relationships (see Table 3). Initiative refers to a “child’s ability to use 17 Figure 3. Relationship of Social and Emotional Competencies to Protective Factors (LeBuffe & Naglieri, 2012). Table 3. Description A child’s ability to think and act independently Social-Emotional Competence in the Early Childhood Population Examples Category Trying multiple ways to solve a Initiative problem; choose to do a difficult task Controlling anger; cooperating with others Showing affection for familiar adults; seeking help from children/adults when necessary A child’s ability to understand and express feelings Self- Regulation Attachment/ Relationships A child’s ability to develop relationships with adults and peers Note. Information adapted from LeBuffe and Naglieri (2012). independent thought and action to meet his or her needs” (LeBuffe & Naglieri, 2012, p. 92) and is evidenced by behaviors such as trying different ways to solve a problem, choosing to complete 18 challenging tasks, and starting or organizing play with other children. Self-regulation refers to a “child’s ability to express emotions and manage behavior in healthy ways” (LeBuffe & Naglieri, 2012, p. 92) and is evidenced by behaviors such as listening to and respecting others, sharing with other children, and handling frustration well. Attachment/relationships refers to a “child’s ability to promote and maintain mutual, positive connections with other children and significant adults” (LeBuffe & Naglieri, 2012, p. 92) and is evidenced by behaviors such as showing affection for familiar adults, appearing happy when playing with others, and seeking help from other children or adults when necessary (LeBuffe & Naglieri, 2012). Collectively, these factors provide an indication of a child’s level of resilience (Lamb-Parker et al., 2008). All in all, based on family stress theory (Conger, Conger, & Martin, 2010; Conger et al., 1990) and models of resilience development (e.g., Garmezy, 1991), a focus on enhancing social- emotional competence through positive parent-child interactions is especially warranted in low- income early childhood populations. Children exposed to accumulated risk (i.e., due to poverty) require strong social-emotional competencies in order to buffer against these risk factors. In the early years, parent-child interactions are one of the primary avenues for facilitating the development of these social-emotional competencies (Sheridan et al., 2014), establishing the trajectories for children’s social-emotional development from an early age (Vick Whittaker et al., 2011). Parenting Practices that Impact Early Childhood Social-Emotional Development Children’s social-emotional development is impacted differentially by the variety of practices used by parents. Positive parent-child interactions are associated with healthy social- emotional development, while negative parent-child interactions are linked with poor social- emotional outcomes (Vick Whittaker et al., 2011). While negative parenting practices have 19 historically received the most research attention in regard to children’s social-emotional development, more recent studies have begun to focus on the protective effects of positive parenting practices (Reuben et al., 2016). Contributions of both negative and positive parenting practices to child social-emotional development are reviewed. Negative parenting practices. Negative parenting practices are conceptualized as comprising behaviors “variously described as harsh, overreactive, emotionally negative, coercive, controlling, and authoritarian” (Chang, Schwartz, Dodge, & McBride-Chang, 2003, p. 599). As such, negative parenting practices may include behaviors such as rejecting, discounting, yelling, frequent negative commands, name calling, physical threats, and aggression, and are typically characterized by overt expressions of anger, irritability, frustration, and/or distress (Chang et al., 2003; Danzig, Dyson, Olino, Laptook, & Klein, 2015). Since parenting processes operate within ecological contexts, the use of parenting practices is impacted by both proximal and distal environmental variables (Bronfenbrenner & Ceci, 1994). Not surprisingly, then, negative parenting practices are more common in stressful circumstances (e.g., living in poverty; Deater-Deckard, Chen, Wang, & Bell, 2012), which takes a toll on children’s social-emotional development. The use of negative parenting practices is associated with poor social-emotional outcomes for children in both the short- and long-term (Danzig et al., 2015). For instance, when parents exhibit emotion dysregulation via negative parenting practices, this, in turn, hinders their children’s ability to regulate their emotions. Furthermore, when parent-child interactions are characterized by aggression, this can transfer to children’s interactions with their peers, leading to challenges in building peer relationships (Chang et al., 2003). In general, negative parenting practices increase the likelihood of future conduct problems, as they provide a negative model of 20 behavior and may inadvertently reinforce children’s behavior problems (Webster-Stratton & Taylor, 2001). Social interaction learning theory. Patterson’s (1975) social interaction learning theory provides a conceptual framework for understanding how patterns of negative parent-child interactions over time can impact children’s social-emotional development. According to the theory, positive and neutral social interactions within the family context enhance the development of children’s social-emotional competencies, while aversive exchanges centered on coercion lead to the development of behavior problems (e.g., aggression). Coercion between parents and children occurs when aversive behaviors (e.g., whining, noncompliance, tantrums) from one are used to control the behavior of the other. These aversive behaviors are maintained when they consistently result in a family member’s acquiescence and/or access to desired materials and activities (Snyder, 2016). An example of coercive family dynamics is provided in Figure 4. As illustrated, aversive behaviors are demonstrated by both the mother (i.e., scolding) and the boy (i.e., arguing). The boy’s arguing serves as a punishment for the mother’s scolding, resulting in the mother removing her demands. The mother’s submission serves as negative reinforcement for the boy (who will, in turn, be more likely to argue in the future), and when the boy stops arguing, this serves as negative reinforcement for the mother (who, in turn, will be more likely to acquiesce in the future; Patterson, 2016). Coercive processes within families can begin when children are as young as 2 or 3 years of age (Biglan, 2016). During the preschool years, children begin to display an increase in coercive behaviors as parents start to set limits in response to children’s improving physical mobility and less mature cognitive skills (Sitnick et al., 2015). If parents are able to effectively manage children’s coercive behaviors, this fosters positive parent-child interactions and offers 21 Figure 4. Three-Step Coercive Sequence (Patterson, 2016) opportunities for children to build social-emotional competencies. If not managed effectively, coercive interactions become more frequent and intense (Snyder, 2016). On average, a child with social-emotional challenges exhibits an aversive event once every three minutes, and research shows that coercive family processes are even more common in families living in poverty (Biglan, 2016). According to social interaction learning theory, strategies to increase positive parenting practices and decrease coercion can help parents more effectively facilitate the development of children’s social-emotional skills (Forgatch, Patterson, & Gewirtz, 2013). 22 Research indicates that it is not simply the presence of negative parent-child interactions, but also the absence of positive parent-child interactions, that adversely impacts children’s well- being (Odgers et al., 2012). Positive parenting practices. Positive parenting practices are conceptualized as being warm, nurturing, and/or supportive. Warm/nurturing parenting practices involve behaviors such as letting a child know that you care, acting loving and affectionate toward a child, and telling a child that you love them (Reuben et al., 2016). Supportive parent-child interactions are often “characterized by exchanges that are warm, sensitive, responsive, and adaptive to the needs of the child” (Mortensen & Mastergeorge, 2014, p. 337). Supportive parenting can refer to a variety of practices, including responsivity, language/cognitive stimulation, positive regard, positive emotional affect, and joint attention (Mortensen & Mastergeorge, 2014). Parents’ use of these positive parenting practices provides opportunities for children to learn and practice essential social-emotional skills via attention support (e.g., a parent directing a child’s attention), modeling (e.g., a parent problem-solving through a complex task while a child watches), and implicit activities (e.g., a parent and a child waiting for each other; Rimm-Kaufman & Wanless, 2012). Previous research has documented associations between high levels of positive parenting and vital social-emotional competencies, including emotion regulation and conflict-resolution skills (Reuben et al., 2016). Unfortunately, due to the physical and psychosocial stressors present in low-income contexts, positive parent-child interactions are often compromised (Odgers et al., 2012). For example, studies have shown that living in poverty reduces a mother’s ability to be responsive to her child’s needs (e.g., Evans, Boxhill, & Pinkaya, 2008). Additional factors common in low- income communities that have been found to negatively impact the use of positive parenting 23 practices include: a) lower levels of social support, b) a lack of positive parenting role models in the community, and c) a higher likelihood of parents experiencing mental health problems (Odgers et al., 2012). While many parents living in poverty engage in positive parent-child interactions, parenting practices are often described as inconsistent within this population. For instance, the same child behavior may lead to parental anger, satisfaction, and/or disregard, depending on the day (Puckering, 2004). Early childhood is an opportune time to initiate interventions that target positive parenting practices among at-risk populations, as parent-child interaction patterns become less responsive to change as time goes on (Sitnick et al., 2015). In a review of experimental studies investigating preventative parent training programs, Sandler, Schoenfelder, Wolchik, and MacKinnon (2011) found that positive parenting practices, such as responsiveness, warmth, and effective discipline, were successfully increased through intervention with at-risk parent populations (e.g., families living in poverty). In several studies, long-term positive effects (e.g., between one and six years after program participation) were documented. Specifically, parent training programs are proposed to facilitate increases in positive parenting practices through any of three main pathways: 1) parents learn new skills, which are maintained by positive reactions from children; 2) parent self-efficacy increases; and/or 3) barriers to effective parenting (e.g., parent mental health problems) are reduced (Sandler et al., 2011). Given that research has documented success in shaping positive parenting, it is important for researchers to consider how these methods can be applied within specific at-risk early childhood populations. Head Start: An At-Risk Population One population considered to be at-risk is children who participate in Head Start programs, given that the vast majority (90%) must meet at least one of four eligibility criteria: 1) 24 family income is equal to or below the poverty line, 2) family is eligible for public assistance, 3) child is homeless, and/or 4) child is in foster care (U.S. Department of Health and Human Services, 2015). Across the United States, Head Start programs serve a diverse population of children and families from low-income backgrounds. Data from the Head Start Family and Child Experiences Survey (FACES) provide an overview of the demographic characteristics of Head Start students and families. Among the cohort of students entering Head Start in 2009, half were three years old and half were four or older. The majority of children represented racial/ethnic minority groups (39% Hispanic/Latino; 32% African American), and many (28%) lived in households where the primary language spoken was not English. Spanish was the most common non-English language spoken in the homes of Head Start children. Fewer than half (43%) of the Head Start child population lived with both biological/adoptive parents (Aikens, Klein, Tarullo, & West, 2013). Child outcomes. The negative impact of poverty can be observed in a number of Head Start children’s developmental outcomes, as they score below norms, on average, in a variety of assessed areas at Head Start entry and exit (e.g., language, literacy, math, social-emotional skills). However, research shows that children make progress towards norms over time. For example, 2009 FACES data show that children gain about 6 standard score points in receptive vocabulary and about 5 standard score points in expressive vocabulary throughout Head Start [as measured using the Peabody Picture Vocabulary Test, Fourth Edition (PPVT-4; Dunn & Dunn, 2006) and the Expressive One-Word Picture Vocabulary Test (EOW PVT; Brownell, 2000)]. Furthermore, teachers report improved social-emotional skills and fewer behavior problems (e.g., aggressive behavior, hyperactive behavior) by the end of Head Start (Aikens et al., 2013). 25 Still, social-emotional development remains an area of concern for this population. While many preschoolers lack appropriate social-emotional skills (approximately 10% to 15%; McCabe & Altamura, 2011), children from low-income backgrounds (e.g., Head Start) experience these difficulties at nearly double this rate (Brinkman, Wigent, Tomac, Pham, & Carlson, 2007; McWayne & Cheung, 2009; Webster-Stratton & Hammond, 1998). For children living in poverty, early intervention is imperative to ensure that early social-emotional challenges do not escalate to a clinically significant level (Webster-Stratton & Reid, 2004). Resilience research supports the use of interventions for enhancing protective factors (e.g., social-emotional competence) within at-risk populations (Wright & Masten, 2005), especially those that involve significant adults in children’s lives (e.g., parent; Webster-Stratton & Reid, 2004). Head Start programs recognize the importance of parent involvement and aim to involve “parents as partners in their children’s education and as agents of change in their communities” (Henrich & Gadaire, 2008, p. 57). To achieve this, Head Start Performance Standards outline a number of ways to collaborate with parents, including helping to identify and work towards family goals, assisting parents in accessing health care and other social services, allowing parents to provide input about the Head Start curriculum, and engaging with parents via home visits. Though parent involvement is essential throughout all aspects of Head Start programs, Head Start Performance Standards specifically outline the role of parents in promoting children’s mental health. Namely, programs are required to gather information from parents; share observations; discuss strategies for appropriately responding to children’s behaviors and creating supportive, nurturing environments; and encourage parent participation in mental health interventions (U.S. Department of Health and Human Services, 2015). 26 Studies show that Head Start is successful in involving parents, with the majority of Head Start parents participating in home visits, parent-teacher conferences, classroom volunteering, parent workshops, and/or field trips (O’Brien et al., 2002). Parents are reported to be more involved in Head Start than in other early childhood education programs (Marcon, 1999), and this involvement has been associated with increased parent-reported social-emotional skills in children (O’Brien et al., 2002). However, universal parent involvement strategies, while effective for many children and families, do not always provide enough support. In cases where children need more targeted social-emotional supports, parent training is suggested as a helpful, and necessary, intervention (Reid et al., 2004), especially during early childhood when both children’s behavior and parenting practices are most malleable (Sitnick et al., 2015). Evidence-Based Parent Training Programs for Preschoolers A number of parent training programs have been developed to date with the goal to improve parenting practices and, in turn, child behavior. While parent training programs focus directly on changing parent behavior, research demonstrates that these programs also have an influence on child behavior. On average, parent training programs have been found to have a small to moderate effect (d = .31) on both child behaviors (i.e., social-emotional competence, behavior concerns) and parent behaviors (d = .47) among nonclinical (i.e., at-risk) samples (Lundahl, Risser, & Lovejoy, 2006). According to the Blueprints for Healthy Youth Development, there are five evidence-based parent training programs for building social- emotional protective factors among preschoolers: 1) Parent Management Training, 2) Child FIRST, 3) HighScope preschool, 4) Incredible Years, and 5) Triple P program (Blueprints for Healthy Youth Development, 2016a; see Table 1). Within the Blueprints registry, programs are rated as Promising, Model, or Model Plus according to a set of evaluation criteria (see Table 4). 27 Intervention Specificity Evaluation Quality Intervention Impact Dissemination Readiness Independent Replication Program description clearly defines: a) outcomes, b) risk/protective factors targeted, c) population, AND d) mechanisms of change. A minimum of: a) one rigorous randomized controlled trial, OR b) two rigorous quasi-experimental studies A minimum of: a) two rigorous randomized controlled trials, OR b) one rigorous randomized controlled trial plus one rigorous quasi-experimental study Research documents: a) significant positive change in intended outcomes, AND b) no evidence for harmful effects Positive impact lasts for at least 12 months after completion of the intervention The program: a) is currently ready for dissemination, AND b) has the necessary training, materials, and supports available for implementation At least one rigorous study is completed by researchers not affiliated with the program developer ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü Table 4. Blueprints for Healthy Youth Development Program Criteria Category Description Promising Model Model Plus Note. Information available online at http://www.blueprintsprograms.com/criteria. Promising programs meet the minimum standard of effectiveness, while Model and Model Plus programs meet a higher standard, providing greater confidence in the positive potential of the interventions (Blueprints for Healthy Youth Development, 2016b; Mihalic & Elliott, 2015). 28 Although these five interventions have been labeled as effective for building social- emotional protective factors in young children, it is also imperative to consider treatment integrity and acceptability (Brown-Chidsey, Steege, & Mace, 2008). Unfortunately, these variables are not evaluated by the Blueprints registry when classifying an intervention as “evidence-based” (Mihalic & Elliott, 2015). Witt and Elliott’s (1985) model of treatment acceptability highlights the reciprocal relationship between treatment acceptability, treatment integrity, treatment effectiveness, and treatment use (see Figure 1), which suggests that all variables must be considered to improve overall dissemination of an intervention. While measures of intervention effectiveness are necessary to determine how well an approach yields a desired outcome, measures of integrity and acceptability are key indicators of the transportability of an intervention (Chorpita, 2003). Treatment integrity represents the extent to which an intervention was carried out as intended and treatment acceptability measures how appropriate, relevant, and useful an intervention is perceived to be (i.e., social validity; Brown-Chidsey et al., 2008). In order to draw valid conclusions about an intervention, we must have a clear understanding of adherence to a treatment protocol (e.g., treatment integrity; Sanetti, Gritter, & Dobey, 2011) and the appropriateness of treatment strategies for a given setting and population (e.g., parent acceptability; Sterling-Turner & Watson, 2002). Among low-income populations, participation in parent training programs tends to be highly influenced by integrity and acceptability. Low-income youth are not likely to receive needed mental health intervention (i.e., via parent training programs), with rates as high as 80% to 90% of those in need not receiving mental health services (Kataoka, Zhang, & Wells, 2002). Of the low-income youth who are able to access mental health services, attrition rates tend to be high (greater than 50%; Atkins et al., 2006). One reason for this is the numerous factors that 29 interfere with parents’ ability to participate in parent training programs with integrity (e.g., inadequate time, work/school schedule conflicts, lack of child care; Gross et al., 2001). Therefore, if parent training programs are not easily implemented with integrity among low- income populations, parents are not likely to participate. Another reason for this discrepancy is a lower level of perceived acceptability. If a parent does not perceive a program to be acceptable, they are more likely to drop out and less likely to comply with all program components as prescribed. Some research suggests that low-income parents rate treatment acceptability levels differently than do middle- to upper-income parents (Eamon & Venkataraman, 2003); thus, it is important to consider the acceptability of proposed interventions for parents living in poverty. Parent Management Training – Oregon model (PMTO). Parent Management Training – Oregon Model (PMTO; Forgatch & Patterson, 2010) is a parent training program focused on providing parents with the skills to prevent and remediate their child’s conduct problems (e.g., aggression, externalizing behavior) and promote prosocial behavior (e.g., social-emotional competence; Forgatch & Kjøbli, 2016). Grounded in the social interaction learning model (Patterson, 1975), PMTO teaches parents five core practices: 1) skill encouragement, 2) limit setting, 3) monitoring, 4) problem solving, and 5) positive involvement. Through scaffolding and positive reinforcement, skill encouragement is used to teach children new prosocial behaviors. Limit setting, on the other hand, is used to discourage deviant behaviors through small contingencies. Monitoring refers to parental supervision of children’s activities and behaviors, problem solving refers to setting and working towards goals, and positive involvement refers to showing children love and interest (Pearl, 2009). Parents learn these skills in a step-by-step manner through 25 to 30 one-hour individual family sessions or 14 90-minute group sessions (Forgatch & Patterson, 2010). 30 PMTO has demonstrated effectiveness in building protective factors for children in a number of well-conducted studies, resulting in its rating as a Model program by the Blueprints registry (Blueprints for Healthy Youth Development, 2016a). In one randomized controlled trial, teacher-rated social-emotional competence increased among youth with conduct problems (N = 112) whose parents participated in PMTO, relative to youth receiving regular services within children’s service agencies (Amlund Hagen, Ogden, & Bjørnebekk, 2011; Ogden & Amlund Hagen, 2008). Other randomized controlled trials have found PMTO to be effective over the long-term (i.e., reduced delinquency and criminal behavior; Forgatch, Patterson, DeGarmo, & Beldavs, 2009), in improving positive parenting, in reducing coercive and harsh parenting practices (Forgatch, DeGarmo, & Beldavs, 2005), with families from diverse backgrounds (i.e., immigrant mothers; Bjørknes, Kjøbli, Manger, & Jakobsen, 2012), and when delivered in a brief format (i.e., 3 to 5 sessions; Kjøbli & Ogden, 2012). Although positive outcomes have been documented for PMTO, questions remain related to treatment integrity and acceptability. While researchers described PMTO therapists as “[delivering] treatment with satisfactorily levels of fidelity” (Amlund Hagen et al., 2011, p. 170), integrity ratings, on average, were at approximately 77% (Ogden & Amlund Hagen, 2008), which is lower than the 80% threshold proposed by Perepletchikova and Kazdin (2005) for high treatment integrity. This information suggests that there is likely room for growth in this area, and future research should prioritize the exploration of PMTO treatment integrity. In terms of acceptability, Ogden and Amlund Hagen (2008) found that parents were significantly more satisfied with PMTO than regular services. It is important to note, though, that the participants in this study represented a middle socioeconomic background, based on income and parent education levels (Ogden & Amlund Hagen, 2008). While PMTO has been successfully adapted 31 for parents within diverse cultural contexts (e.g., single mothers, stepfamilies, families living in high crime neighborhoods), it has not been specifically studied within low-income communities (Forgatch & Kjøbli, 2016). Therefore, it is unclear whether parents living in poverty would also rate this program as being acceptable for building children’s social-emotional competencies. Child FIRST. Child FIRST (Child and Family Interagency, Resource, Support, and Training) is a comprehensive, home-based intervention that targets at-risk children and families. The Child FIRST model includes two main components: 1) a “system of care” approach to increase access to needed services and supports, and 2) a relationship-based approach to promote positive parenting practices. Given that the family is the focus of the intervention, the program is implemented by a clinical team (including both mental health professionals and care coordinators) via consistent home visits. The intervention is driven by each family’s individual needs, rather than a structured curriculum (Lowell, Carter, Godoy, Paulicin, & Briggs-Gowan, 2011). The Child FIRST program has demonstrated success in one randomized controlled trial, and, thus, has been given a Promising rating by the Blueprints registry (Blueprints for Healthy Youth Development, 2016a). Children between the ages of 6 months and 36 months who received the Child FIRST intervention (N = 58) showed reductions in externalizing behavior six months after completion of the program, as compared to children receiving usual care (N = 59). Other positive outcomes included improvements in language skills and increased access to community-based services (e.g., health, early care, education). In addition, parents who participated in the Child FIRST program experienced reductions in stress and depressive symptoms, as well as improvements in overall well-being immediately following and/or six months after completion of the program. At a 30-month follow-up, families who received the 32 Child FIRST intervention had less involvement with Child Protective Services (CPS), as compared with families who received usual care (Lowell et al., 2011). Despite these initial positive results, the Child FIRST model has some inherent threats to acceptability and integrity. First and foremost, the intervention is both time-intensive and costly. While the length of the intervention varies, the average duration is several months [Lowell and colleagues (2011) found a mean of 22 weeks], which is a considerable commitment for families and professionals. Each individual home visit session lasts between 45 and 90 minutes, which can be challenging to schedule each week. Lowell and colleagues reported that there were “many missed and canceled appointments” throughout the study (p. 198). Funding is also a consideration, given that the intervention is estimated to cost approximately $4000 per family (without factoring in the costs of additional community-based services; Lowell et al., 2011). Given this investment, it may be more appropriate to begin with a less expensive approach prior to implementing the Child FIRST program. Second, treatment integrity is unclear for the Child FIRST program. Lowell and colleagues (2011) incorporated the use of a fidelity checklist after each home visit session; however, this data was only reviewed during clinical supervision and not formally analyzed. As a result, our knowledge is limited pertaining to whether the intervention can be easily and consistently carried out as intended. Although a manual is provided to “teach and guide the intervention” (Lowell et al., 2011, p. 199), substantial clinical judgment is required to align the intensity of the intervention and selection of services with family needs. Thus, clinical experience and training has the potential to impact treatment integrity. In sum, while the Child FIRST program has empirical evidence to support its effectiveness, future efforts are needed to explore both acceptability and integrity. 33 HighScope preschool. The HighScope preschool curriculum, developed by Weikart and his colleagues (Weikart, Rogers, Adcock, & McClelland, 1971; Hohmann, Banet, & Weikart, 1979; Hohmann & Weikart, 1995), is an educational model that emphasizes active learning. Based on Piaget’s constructivist theory of child development, the HighScope curriculum teaches children to engage in self-directed activities, participate in small- and large-group tasks, and spend time outdoors. Through these activities, children are thought to develop initiative and social reasoning, leading to future success in planning, decision making, critical thinking, and interpersonal interactions (Schweinhart & Weikart, 1997). While the majority of the curriculum focuses on the classroom (five half-days per week), there is also a parent training component, which includes: a) weekly 90-minute home visits conducted by teachers, and b) monthly parent group meetings (Nores, Belfield, Barnett, & Schweinhart, 2005). The HighScope curriculum, rated as a Promising program by the Blueprints registry (Blueprints for Healthy Youth Development, 2016a), was studied in a longitudinal, randomized controlled trial of African American preschool students (N = 123) living in poverty. Data collected on student outcomes through age 40 indicates that HighScope participants exhibited greater social-emotional competence (e.g., initiative, interest, persistence) and fewer externalizing behaviors (e.g., aggression, breaking rules, disruptive behavior), which, in turn, translated to future success in variety of domains (e.g., reduced criminal activity, increased income, stable employment; Heckman, Pinto, & Savelyev, 2013). In an independent re-analysis of the study’s results, researchers concluded that there are “strong positive impacts from participation in the program and strong positive gains for the general public in providing this program” (Nores et al., 2005, p. 256). 34 Though the HighScope preschool curriculum has demonstrated long-term effectiveness in promoting healthy social-emotional outcomes, research has not investigated the outcomes of the parent training component alone. To achieve positive outcomes, the HighScope curriculum likely must be implemented as a comprehensive unit, which may result in low integrity and/or acceptability in certain settings. For instance, it has been historically challenging to implement the curriculum within programs that do not mirror the structure of the HighScope Perry Preschool program (Schweinhart, 2010). In particular, there is evidence to suggest that the curriculum is not as effective within larger federal preschool programs (e.g., Head Start) due to several differences, including the availability of highly qualified teachers, inconsistencies in parent engagement, and challenges with regularly assessing program implementation (Schweinhart, 2013). As a result, programs may have difficulties implementing the HighScope preschool curriculum with integrity, which, in turn, would limit the program’s success. Since the program “ingredients” needed for the HighScope curriculum to be highly effective are not typically present within Head Start programs, this may also result in teachers and parents perceiving the intervention as having poor acceptability. As a result, Head Start programs may be unlikely to adopt the HighScope preschool curriculum altogether, given the direct link between treatment acceptability and treatment use (Witt & Elliott, 1985). Incredible Years parent training program. Another parent training intervention for promoting social-emotional competence in young children is the Incredible Years parent training program (Webster-Stratton, 2006). Through weekly sessions, parents learn strategies for promoting children’s self-control, communication skills, problem-solving skills, and self-care (Webster-Stratton & Reid, 2010). One feature unique to the Incredible Years program is the use of videotape vignettes to facilitate session content. The Incredible Years program is founded in 35 social learning theory, particularly Bandura’s (1986) modeling and self-efficacy theories and Patterson’s (1975) coercion theory. These theories form the basis for the parent training program, which emphasizes positive reinforcement and other proactive discipline strategies via live and video modeling, rehearsal, and self-reflection (The Incredible Years, 2013). The Incredible Years parent training program is supported by many rigorous research studies (developer- and independently-conducted) with diverse populations (Menting, de Castro, & Matthys, 2013; Pidano & Allen, 2015), leading to its rating as a Promising intervention (Blueprints for Healthy Youth Development, 2016a). In the meta-analysis conducted by Menting and colleagues, small effect sizes were found for decreases in child behavior concerns (d = .27) and increases in child social-emotional competence (d = .23) after parent participation in the Incredible Years training program. A few large-scale randomized trials have investigated the program specifically within the Head Start population (Borden, Schultz, Herman, & Brooks, 2010). Each of the studies demonstrated significant increases in children’s social-emotional competence after implementation of the Incredible Years program with parents, teachers, and family services workers (Webster-Stratton, 1998; Webster-Stratton, Reid, & Hammond, 2001). Furthermore, results suggest that these gains can be maintained over time (Borden et al., 2010). Even though there is solid empirical evidence to support the effectiveness of the Incredible Years parent training program (Pidano & Allen, 2015), the program’s success may, in some circumstances, be threatened by acceptability. For example, the program typically consists of 18 to 20 two-hour weekly sessions led by therapists (Webster-Stratton & Reid, 2010), which can be challenging for any busy family to consistently attend, let alone families living in poverty who often experience more chaotic and inflexible daily schedules (Gross et al., 2001). Additionally, all staff are required to attend a three-day training and purchase all program 36 materials prior to delivering the intervention, which can become quite costly ($1,395 or more; Armstrong et al., 2014). While these participation, training, and material requirements ensure that the program is carried out with integrity, it may not be feasible to meet these requirements in some settings, especially in low-income areas where cost is a major barrier to mental health treatment (Elgar & McGrath, 2003; Webster-Stratton & Reid, 2003). Triple P program. Another evidence-based program for young children is the Triple P Positive Parenting Program (Sanders, 1999), a multilevel prevention program focused on developing a “family friendly” environment in the home. Triple P has five intervention levels of increasing intensity, which target the social contexts that families are exposed to every day (e.g., media, health care, child care, school, work, religious organizations). Level 1 (Universal Triple P) is a media-based campaign to raise awareness of the importance of parenting practices and to share information about parenting programs. Level 2 (Selected Triple P) focuses on information and tips for minor developmental or behavioral issues, which may be delivered in a brief one-on- one, group, or telephone meeting. Level 3 (Primary Care Triple P) extends Level 2 by including multiple sessions (approximately four) focused on a specific problem behavior. Level 4 (Group Triple P, Self-Directed Triple-P) is a more thorough program (e.g., 8 to 10 sessions) aimed at developing a broad range of parenting skills. Level 5 (Enhanced Triple P) is an individualized program for families requiring more intense supports (Sanders, 2008). The Triple P program is rated as a Promising program by the Blueprints registry (Blueprints for Healthy Youth Development, 2016a). In a meta-analysis of randomized controlled trials by Nowak and Heinrichs (2008; N = 29), the Triple P Program had a small to moderate average effect size for child problem behavior (ES = .48), parenting (ES = .42), and parental well-being (ES = .20). However, many of the studies included in the meta-analysis were 37 completed by the program’s developer and/or his colleagues. The body of research exploring the Triple P program continues to expand, but the program remains relatively unstudied by independent researchers to date (Pidano & Allen, 2015). Therefore, in order to increase confidence in the effectiveness of the program, independent replication is necessary (Blueprints for Healthy Youth Development, 2016b). While the Triple P program has been implemented with high integrity and rated as being highly acceptable among culturally diverse populations (Hoath & Sanders, 2002; Morawska et al., 2011), there are still some barriers to consider. As with the other evidence-based parent training programs, funding is likely an issue for the dissemination of this program, given that training and materials cost between $1500 and $2000 for each practitioner. Additionally, Triple P program sessions typically last between one and two hours over 12 weeks (Armstrong et al., 2014), and since timing and location of services is another frequently cited barrier to accessing mental health treatments, families could be deterred from the program for this reason. Despite the many advantages to the Triple P program, additional efforts are needed to improve parent access and participation, particularly for culturally diverse parents (Morawska et al., 2011). Summary. Several parent training programs have demonstrated effectiveness for building social-emotional protective factors in early childhood. The therapeutic content included within each program reflects slightly different combinations of common evidence-based components of parent training programs, including principles of positive reinforcement, principles of effective limit-setting/punishment, parent-child relationship building, problem- solving skills, anger management, and affect education (Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008). Delivery formats range from individual- and group-based sessions to home visits to multilevel prevention. All programs (except for Child FIRST) deliver content via 38 psychoeducation/didactics and modeling/role play, highlighting a focus on both knowledge and practice. Programs differ in terms of child involvement, with Child FIRST emphasizing parent- child interaction within sessions (Lowell et al., 2011), and others (e.g., Incredible Years, Triple P) focusing primarily on parent skill development within sessions (Sanders, 2008; Webster- Stratton & Reid, 2010). While these programs are founded in evidence-based parent training techniques (Garland et al., 2008), they are also costly, lengthy, and time-intensive, which may limit the integrity and acceptability of these programs within low-income populations. Some of the evidence-based programs have been implemented with high levels of integrity (i.e., Incredible Years, Triple P; Hoath & Sanders, 2002; Webster-Stratton, 1998), though it is difficult to tell whether the programs could be implemented with the same level of integrity outside of a funded randomized controlled trial. For instance, programs require time-intensive training, expensive materials, and lengthy weekly sessions (i.e., 45 minutes or more), which may not be feasible in settings that do not have adequate funding or enough highly-qualified professionals (i.e., Head Start; Schweinhart, 2013). Even if settings have the required resources to implement these programs, low-income families may struggle to participate in all program components, given the practical barriers that exist (e.g., time constraints, scheduling conflicts, cost, problems with child care). Due to these barriers, previous studies of parent training programs focused on children’s behavior problems have found lower rates of participation among low-income participants versus higher-income participants (Heinrichs, Bertram, Kuschel, & Hahlweg, 2005). These evidence-based programs may also be less likely to be adopted within low-income communities due to low levels of treatment acceptability (Eamon & Venkataraman, 2003). Treatment acceptability is influenced by many different factors, including the time and effort 39 needed for an intervention, the severity of the presenting behaviors, and the difficulty of the intervention (Villarreal et al., 2015). Given the time, effort, and expenses associated with many of the evidence-based programs, schools and families may perceive lower-cost, higher- flexibility, and less time-intensive options as being more acceptable, especially for students who are not displaying clinical levels of behavior. Treatment effectiveness also impacts levels of treatment acceptability (Witt & Elliott, 1985), and since the experience of poverty has been associated with a smaller intervention impact (Dawson-McClure et al., 2015), this likely impacts the types of treatments that parents seek out within this community. Overall, these limitations suggest that adaptations must be made within low-income contexts to ensure adequate access to services and effective service delivery. Self-Administered Parent Training Interventions Although effective parent-focused social-emotional interventions exist (e.g., Barton et al., 2014), these programs remain largely unavailable to many families who need them (Forgatch et al., 2013), especially those at-risk due to poverty. At-risk parents often find it difficult to access and adhere to these training programs due to life stresses, such as work conflict, family discord, expenses, or transportation issues (Webster-Stratton & Reid, 2003). To improve access and adherence within this population, Rotheram-Borus and colleagues (2012) suggest the use of self- administered interventions as a disruptive innovation. By delivering services through practices such as video programming and written materials, self-administered interventions may be viable treatment options (e.g., high levels of treatment integrity and acceptability) for families experiencing considerable barriers to mental health treatment (Elgar & McGrath, 2003). Self- administered programs can be completed and evaluated prior to the need to conduct costly, time- consuming, and intrusive comprehensive assessments commonly reported within the early 40 childhood literature (e.g., Shernoff, Hill, Danis, Leventhal, & Wakschlag, 2014). The many advantages to self-administered methods of delivery, including low costs, convenience, and stigma reduction, have the potential to overcome barriers to treatment access, while still facilitating effective outcomes (Elgar & McGrath, 2003). Some parent training interventions targeting the early childhood population have already demonstrated positive results in self-administered formats (e.g., Incredible Years parent training program, Triple P program). For instance, according to a meta-analysis conducted by de Graaf and colleagues (2008), the therapist-led and self-directed formats of the Triple P program were equally effective. Similarly, a study of the self-administered Incredible Years parent training program found treatment effects (i.e., increases in children’s social-emotional competence, decreases in behavior concerns) comparable to those documented with the face-to-face group format (Webster-Stratton et al., 1988). A meta-analysis of 63 studies investigating parent training programs found that child and parent outcomes did not significantly differ based on the intervention format (i.e., face-to-face versus self-administered; Lundahl et al., 2006), suggesting that self-administered parent training programs have the potential to lead to significant positive outcomes for children and families. Furthermore, research indicates that self-administered parent training programs can be implemented with moderate-to-high integrity and acceptability (Ogg & Carlson, 2009; Stewart & Carlson, 2010). However, other studies have found inconclusive evidence for the effectiveness of self-administered programs (e.g., Kratochwill, Elliott, Loitz, Sladeczek, & Carlson, 2003), warranting additional research in this area. Devereux Early Childhood Assessment, Second Edition (DECA-P2; LeBuffe & Naglieri, 2012) Parent Training Guide (Mackrain & Cairone, 2013). The Devereux Early Childhood Assessment, Second Edition (DECA-P2) is a comprehensive, strengths-based 41 prevention program for building social-emotional competence in preschool children. The program utilizes a five-step problem solving process to identify and enhance social-emotional protective factors: 1) Collect student information via records and observations; 2) Parents and/or teachers rate children’s social-emotional competence (using the DECA-P2 rating scale); 3) Summarize results; 4) Implement strategies aligned with assessment results at school and at home; and 5) Evaluate student progress over time (Lamb-Parker et al., 2008). According to Armstrong and colleagues (2014), the DECA-P2 program is founded in resilience theories, given its focus on “developing child strengths known to lead to positive outcomes,” so that “children are better equipped to handle stressors that occur within their life” (p. 94). In the DECA-P2 program, home-based strategies are outlined in a short parent training guide, entitled Promoting Resilience For Now and Forever. The guide was created to a) provide parents and families with an understanding of healthy social and emotional development, and b) offer useful strategies to support the development of social and emotional skills in young children (see Table 5; Mackrain & Cairone, 2013). The guide boasts several strengths, which all support its potential to be effective as an early intervention tool. First, given its direct link with evidence-based assessment measures and classroom resources (Lamb-Parker et al., 2008), the DECA-P2 parent training guide is a convenient option for many parents to extend their children’s social-emotional learning into the home environment. Dawson-McClure and colleagues (2015) suggest that “framing parenting interventions as support for school success is likely to increase acceptability and capitalize on the motivation shared by parents from diverse cultures to help children achieve” (p. 280). Therefore, since the DECA-P2 guide connects directly with resources that can be used in the classroom, parents may be more likely to view it as a joint effort. Second, the DECA-P2 parent training guide is shorter and considerably less 42 Table 5. Parenting Strategies from the DECA-P2 Parent Training Guide (Mackrain & Cairone, 2013) Category Initiative Description 1. Get involved in your child’s play. 2. Let your child teach you how to do something. 3. Encourage your child’s interests. 4. Involve your child in doing simple daily tasks. 5. Do things as a family regularly. 6. Ask for your child’s help throughout the day. 7. Have fun together everyday. 8. Find what is special about your child. 9. Provide help if needed as your child learns something new. 10. Talk out loud as you help your child solve a problem. Self- Regulation Attachment/ Relationships 1. Help your child learn to calm down when frustrated. 2. Talk about it later. 3. Name and discuss your feelings. 4. Name and discuss your child’s feelings. 5. Practice taking turns. 6. Offer different choices. 7. Have simple rules and be consistent. 8. Look for clues that a problem is coming and have a plan. 9. Let your child know that all feelings are OK. 10. Use warnings to help your child get ready for change. 11. Explain what to do. 1. Hug and cuddle together. 2. Respond when your child wants to talk and play with you. 3. Use your child’s name often. 4. Create special hello and good-bye routines. 5. Show your child how to make friends. 6. Talk with your child about things she does well. 7. Help your child make and keep friends. expensive than the self-administered manual for other parent training programs (e.g., Incredible Years; see http://www.incredibleyears.com), which may further enhance its effectiveness in overcoming barriers to mental health treatment. Third, professionals can disseminate the parent training guide to parents without the need for extensive training or ongoing supervision, which 43 may improve the availability of the intervention among diverse populations and within large federal preschool programs (e.g., Head Start). The DECA-P2 parent training guide includes much of the same therapeutic content as is present in other evidence-based parent training programs (Garland et al., 2008). Strategies focus on principles of positive reinforcement (e.g., “Find what is special about your child”), principles of effective limit-setting/punishment (e.g., “Have simple rules and be consistent”), parent-child relationship building (e.g., “Create special hello and good-bye routines”), problem-solving skills (e.g., “Talk out loud as you help your child solve a problem”), anger management (e.g., “Help your child learn to calm down when frustrated”), and affect education (e.g., “Name and discuss your feelings”). Given that the DECA-P2 parent training guide is self-administered, content is delivered through psychoeducation/didactics, and the program does not offer the opportunity for modeling/role play (as do most other evidence-based parent training programs). Nonetheless, the DECA-P2 parent training guide offers opportunities for parents to practice skills through assigned weekly “homework” strategies. To date, the DECA-P2 parent training guide (Mackrain & Cairone, 2013) has only been investigated in one small pilot study with preschoolers (N = 12), the majority of whom participated in Head Start programs. In the pilot study, children’s social-emotional competence increased and behavior problems decreased (as rated by parents) following parents’ use of the DECA-P2 parent training guide as a self-administered intervention. While the guide can be implemented flexibly, a researcher-developed, structured workbook was used in the pilot study to facilitate the guide’s content over an eight-week implementation period. Workbook pages for each week followed the same structure. At the beginning of the week, parents read and/or reviewed short sections from the parent training guide and answered written questions to reflect 44 on the content (e.g., “Describe a time when anger and/or frustration affected how you responded to a situation. Looking back, what might you have done differently?”). Parents were then presented with three focus strategies based on content from the parent training guide and prompted to brainstorm ideas for using the strategies in their daily life (e.g., “Which of your own feelings might you discuss with your child?”). During the course of the week, parents tracked their use of each strategy using a log in the workbook. At the end of the week, parents summarized their use of the strategies, evaluated the week’s content, and rated their child’s social-emotional competencies (Thomson & Carlson, 2016). Overall, based on the strengths of the DECA-P2 parent training guide (e.g., alignment with classroom strategies, cost, ease of dissemination; Mackrain & Cairone, 2013) and its initial success within a pilot study (Thomson & Carlson, 2016), further research efforts are warranted. Intervention research requires a combination of methods, including single-case and randomized controlled designs; however, single-case design serves as a foundation for intervention development in the early stages of the research process, as researchers are able to answer questions with “intricate detail” (Sheridan, 2014, p. 300) and investigate the presence of a functional relationship. Additionally, single-case designs are especially useful when exploring treatment effects within particular contexts (e.g., families living below the poverty line; Sheridan, 2014). Single-Case Design Research Rationale for single-case design. Given the success of the DECA-P2 self-administered parent training intervention in a pilot study (Thomson & Carlson, 2016), the next step in the research trajectory, according to the process outlined by Sheridan (2014), is to evaluate the intervention with intensity and precision using a small sample size (i.e., single-case design; see 45 Table 2). According to Sheridan (2014), single-case designs are particularly useful in the early stages of intervention research when the major focus tends to be on efficacy. Furthermore, single-case designs have the capability to address the issues associated with conducting high- quality research in low-income, community contexts. Most importantly, single-case designs address the low participation rates of low-income families in parent training programs by only requiring a small number of participants to draw reliable and valid conclusions (Kratochwill et al., 2013). Additionally, single-case design allows for experimental control and high internal validity via systematic and repeated demonstration of the intervention effect, as well as both within- and between-subject comparisons (Horner et al., 2005). Findings from single-case designs can also be aggregated within meta-analyses, which has the potential to inform the evidence-based practice literature (Burns, 2012; Shadish, 2014). Single-case design standards. Single-case design research is experimental in nature and “its purpose is to document causal, or functional, relationships between independent and dependent variables” (Horner et al., 2005, p. 166). Kratochwill and colleagues (2013), in conjunction with the What Works Clearinghouse, specified four criteria of single-case designs that are necessary to “meet design standards” (p. 27). According to the first criterion, in an effort to minimize threats to internal validity, the researcher must systematically (or randomly) manipulate the independent variable (i.e., the intervention). Researcher-predicted data changes (i.e., upon introduction of the intervention) control for the potential influence of confounding variables, improving confidence that behavior changes occur as a result of the intervention. The second criterion indicates that each outcome variable must be assessed over time using multiple informants. In addition, inter-rater agreement must be documented for each outcome variable. The third criterion requires that studies must include at least three attempts to demonstrate an 46 intervention effect (e.g., include at least three participants in a multiple-baseline design). This criterion facilitates experimental control by providing the opportunity for researchers to observe “replication of an unlikely change in the pattern of the data correlated with the researcher either systematically or randomly manipulating the independent variable” (p. 28). The fourth criterion designates that each phase must include a minimum of three to five data points, which allows researchers to document a pattern of responding (Kratochwill et al., 2013). Advantages of multiple-baseline designs within intervention research. While there are many variations of single-case designs (e.g., ABAB, alternating treatments), the multiple- baseline design is uniquely suited for situations when it is not desirable or ethical to reverse a treatment effect (e.g., mental health outcomes). Multiple-baseline designs include two phases: (A) Baseline phase [no independent variable (i.e., no intervention)], and (B) Treatment phase [introduction of independent variable (i.e., intervention)]. During both the baseline and treatment phases, dependent variables are measured repeatedly. In typical AB designs, findings do not parse out competing explanations for changes in the dependent variable; however, multiple- baseline designs address this drawback in several ways. First, “manipulation of the independent variable is ‘active’ rather than ‘passive’” (p. 40), meaning that researchers decide when and how to introduce the independent variable (i.e., intervention). Second, multiple-baseline designs include at least three data series (i.e., repeated measurements), meaning that the dependent variable is measured across three or more conditions (e.g., participants, settings, behaviors). Third, in multiple-baseline designs, the introduction of the independent variable is staggered across at least three different points in time. In other words, baseline phases are different in length to determine if changes in the data correspond with the introduction of the independent 47 variable, as opposed to familiarity with the measurement process or other (uncontrolled) events (Horner & Odom, 2014). Multiple-baseline research can be designed to happen concurrently or non-concurrently. The concurrent approach involves simultaneous data collection across participants, behaviors, and/or settings, while the non-concurrent approach allows researchers to collect data at different times. The non-concurrent approach, proposed by Watson and Workman (1981), is a good fit for research in applied settings, given that it may not always be feasible to collect data on participants at the same time (e.g., due to recruitment issues; Christ, 2007). Considering the difficulties with participant recruitment and retention in low-income communities (Gross et al., 2001), the non-concurrent approach is ideal, such that it allows for greater flexibility in recruiting participants. In the non-concurrent approach, baseline phase durations are determined a priori, and once participants become available, they are randomly assigned to one of the baseline conditions. These two major characteristics of the non-concurrent approach (i.e., a priori determination of baseline phases, random assignment to a baseline/intervention schedule), combined with the other qualities of a multiple-baseline design, function to promote experimental control and internal validity (Christ, 2007). In the non-concurrent multiple-baseline approach, confounding variables (i.e., threats to internal validity), which may include ambiguous temporal precedence, history, maturation, regression to the mean, and testing effects, are controlled via multiple elements of the research design. First, the active manipulation of the independent variable (i.e., intervention) by the researcher via a priori determination of the baseline/intervention schedule ensures that the assumed cause precedes the assumed effect. Second, effect replication reduces the likelihood that changes in the dependent variable are due to extraneous variables (i.e., history, maturation, 48 testing). Third, repeated measurement of the dependent variable during the baseline and treatment phases allows researchers to more easily examine the data for regression effects, as compared with studies that only rely on pre-test and post-test data. Internal validity is further strengthened by randomly assigning participants to baseline/intervention schedules (Christ, 2007; Kratochwill & Levin, 2010). In general, the multiple-baseline across participants design is thought to be the strongest of all single-case designs in terms of internal validity (Kratochwill & Levin, 2010). While a priori randomization of baseline phase lengths improves both internal validity and statistical conclusion validity (Kratochwill & Levin, 2010), one weakness of the approach is that it does not allow for the baseline data to stabilize prior to initiating the intervention (Watson & Workman, 1981). Although high levels of variability in the baseline phase may pose a challenge for visual analysis, the a priori randomization method is advantageous for several reasons. First, determining baseline phase lengths a priori allows for data to be collected non- concurrently. This is an advantage within applied settings (e.g., home, school) where it may not be feasible to have participants referred at the same time. The non-concurrent multiple-baseline design is superior to the reversal design, which has been used previously to address pragmatic concerns in applied settings, for two main reasons: 1) The non-concurrent multiple-baseline design does not require the removal of the intervention (i.e., an ethical disadvantage of the reversal design), and 2) The reversal design assumes that treatment effects can be reversed, so maintenance effects can lead to false conclusions (i.e., a theoretical disadvantage; Watson & Workman, 1981). Second, determining baseline phase lengths a priori is said to be an important prerequisite for conducting formal statistical analysis within single-case design studies (i.e., multiple-baseline; Koehler & Levin, 1998). Calculation of effect sizes is useful, as that is what 49 can be used to aggregate findings within future meta-analyses (Burns, 2012; Shadish, 2014). Additionally, it is important to be able to conduct statistical analysis, which can help correct for any trend or variability in the baseline data (Kazdin, 2011). Overall, according to Christ (2007), “establishment of a priori baseline durations combined with the random assignment of subjects bolsters the design’s potential to demonstrate experimental control” (p. 454), which, in turn, leads to increased confidence in the validity of the inferences that can be drawn from the data. Generally speaking, one major concern with single-case designs relates to external validity (Lanovaz & Rapp, 2016). External validity “refers to the extent to which causal inference from a particular study holds over different contexts, settings, measures, populations, and so on” (Hitchcock, Kratochwill, & Chezan, 2015, p. 462), and can also be thought of as a component of generalization. In single-case design, replication is necessary to demonstrate external validity (Onghena & Edgington, 2005), and multiple-baseline designs in particular allow for replication across participants, settings, or behaviors (Kratochwill & Levin, 2010). Hawkins, Sanson-Fisher, Shakeshaft, D’Este, and Green (2007) argue that external validity can be further strengthened by randomly assigning intervention start points, as this decreases the likelihood that the baseline/intervention schedule is associated with participant characteristics (e.g., willingness to participate in the intervention, access to researchers). Therefore, because the non-concurrent multiple-baseline approach randomizes participants to baseline phase lengths (Christ, 2007), this method has the potential to improve overall external validity. Analysis of single-case design data. A comprehensive and reliable analysis of single- case design data should include a combination of visual and statistical analysis methods (e.g., randomization tests, effect size calculations; Heyvaert & Onghena, 2014). Visual analysis offers a broad, integrated, holistic view of the data by simultaneously examining several aspects, such 50 as changes in mean, trend, and variability; how rapidly behavior changes following the introduction of an intervention; and recurring patterns or cycles within the data (Parker & Hagan- Burke, 2007). Statistical analysis, on the other hand, allows researchers to assess the reliability and magnitude of an effect. Randomization tests should be used to rule out the null hypothesis that there is no differential effect of the intervention (Onghena & Edgington, 2005), and effect size calculations should be used to determine the magnitude of an effect. Statistical analysis is advantageous because it can a) be conducted with unstable or variable baseline data, b) detect small but important treatment effects, and c) produce consistent results regardless of the analyst (Heyvaert & Onghena, 2014). Heyvaert and Onghena (2014) also recommend measuring the clinical significance of the effect of an intervention, or “whether it makes a ‘real’ difference to the patient and/or to others with whom the patient interacts in everyday life” (p. 62). Methods for conducting visual and statistical analysis, as well as measuring clinical significance, within multiple-baseline designs are outlined below. Visual analysis of single-case design data. In single-case research, visual analysis is the first step in determining the presence of treatment effects and is conducted by examining graphed data of baseline and intervention phases (Kratochwill & Stoiber, 2002). Visual analysis allows researchers to determine whether a relationship exists between an independent and dependent variable, and if so, what the strength and magnitude of that relationship is. An effect is “documented when the data pattern in one phase (e.g., an intervention phase) differs more than would be expected from the data pattern observed or extrapolated from the previous phase (e.g., a baseline phase)” (Kratochwill et al., 2010, pp. 17-18). Kratochwill and colleagues (2010) outline four steps for conducting visual analysis: 1) Document a predictable baseline pattern of data, 2) Examine the data within each phase to assess within-phase patterns, 3) Compare data 51 from each phase with the data in the adjacent phase, and 4) Integrate information from all phases of the study to determine whether there are at least three demonstrations of an effect. To analyze within- and between-phase data patterns, visual analysis procedures involve the examination of six features (see Table 6). First, level is considered by calculating the mean score for the data within each phase. A change in the mean score in the hypothesized direction between baseline and intervention phases provides evidence for behavior change as a result of the introduced intervention. Second, trend, or the slope of the best-fitting straight line, is examined for each phase. Slopes that differ between phases provide evidence that behavior change occurred as a result of the intervention, rather than due to natural changes over time. Third, variability is assessed using range and/or standard deviation of scores within each phase. Minimal variability is ideal, as it provides confidence that behavior change is not simply due to natural variation in the behavior. Level, trend, and variability are examined separately for the baseline and intervention phases prior to examination of the next three features. Fourth, immediacy of the effect is considered by comparing the mean score of the last three data points in the baseline phase with the first three data points in the intervention phase. A more rapid effect provides stronger evidence that behavior changes occurred as a result of the introduced intervention. Fifth, overlap is assessed by calculating the proportion of data in the intervention phase that overlaps with the data in the baseline phase. Lower proportions of data overlap offer evidence in favor of an intervention effect. Sixth, data across similar phases (e.g., all “baseline” phases, all “intervention” phases) are examined for consistency, with high levels of consistency suggesting the presence of a causal relationship between the independent and dependent variables (Kratochwill et al., 2010). 52 Table 6. Six Features of Visual Analysis for Single-Case Design Feature 1) Level 2) Trend 3) Variability 4) Immediacy of the Effect 5) Overlap 6) Consistency of Data Patterns Across Similar Phases Description Mean score for the data within a phase Slope of the best-fitting straight line for the data within a phase Range or standard deviation of data about the best-fitting straight line Change in level between the last three data points in one phase and the first three data points of the next Proportion of data from one phase that overlaps with data from the previous phase Looking at data from all phases within the same condition (e.g., all baseline phases; all intervention phases) and examining the extent to which there is consistency in the data patterns from phases with the same conditions Outcome Demonstrating an Effect Mean scores that differ between phases Slopes that differ between phases Relatively stable variability within each phase The more rapid (or immediate) the effect, the more convincing the effect The smaller the proportion of overlapping data points (or conversely, the larger the separation), the more convincing the effect The greater the consistency, the more likely the data represent a causal relation Note. Descriptions and outcomes taken from Kratochwill et al. (2010). According to Kratochwill and colleagues (2013), Strong Evidence for an intervention effect is demonstrated when at least two trained visual analysts confirm the presence of a functional relationship, as evidenced by a) three replications of an intervention effect (at different points in time), and b) no observations of a non-effect. Moderate Evidence is demonstrated by three demonstrations of an effect and one (or more) demonstrations of a non-effect. No Evidence is demonstrated by a lack of three (or more) demonstrations of an effect (Kratochwill et al., 2013). 53 Statistical analysis of single-case design data. In addition to visual analysis procedures, statistical methods can also be used to analyze single-case design data, especially when there is considerable trend and/or variability in the data (Heyvaert & Onghena, 2014; Kazdin, 2011). Although documenting a stable baseline trend is the first step in the visual analysis process (Kratochwill et al., 2010), some researchers argue that “stable baselines often simply cannot be obtained” (Parker, Cryer, & Byrns, 2006, p. 421). For instance, Parker and colleagues (2005) found that across a sample of published single-case design studies, approximately 66% had clear positive or negative baseline trend, and more than 50% had high variability within the baseline phase. Therefore, while statistical analysis is recommended as a useful tool for analysis of all single-case design data (Heyvaert & Onghena, 2014), it is necessary for determining the presence of an intervention effect in many cases, as it can help to control for baseline variability (Kazdin, 2011; Parker et al., 2005; Parker et al., 2006). Two commonly recommended statistical analysis procedures for single-case designs include randomization tests (Ferron & Levin, 2014) and effect size calculations (Kratochwill et al., 2013). Randomization tests. Randomization tests are “statistical significance tests based on the random assignment of experimental units to treatments” (Heyvaert & Onghena, 2014, p. 52), and are used to determine if the outcome following an intervention is greater than what would be expected due to chance (Ridgway, 2013). Randomization tests improve statistical conclusion validity and allow researchers to draw causal inferences between independent and dependent variables (Ferron & Levin, 2014). In other words, randomization tests provide information about whether positive outcomes were likely due to an active treatment. Calculation of randomization tests does not require data to meet parametric assumptions (e.g., normality, homogeneity of 54 variance, independence), which makes these statistical tests especially applicable to single-case research (Ferron & Levin, 2014). While many different randomization tests exist, the Wampold and Worsham (1986) randomization test is the recommended method for analyzing multiple-baseline data (Bulté & Onghena, 2009). The Wampold and Worsham test is used when participants have been randomly assigned to baseline phase lengths. If there are N participants, then there are N! possible assignments. To calculate the test statistic (W), mean differences are calculated between the treatment (B) and baseline (A) phases and aggregated across participants. W is calculated for all possible permutations (N!) and significance is determined by finding the number of Ws as large as or larger than the W for the current data and dividing this number by N!. The test statistic can be compared to the conventional Type I error probability (α) of .05 (Levin, Ferron, & Gafurov, 2016). Effect size calculations. Effect size calculations can be used to quantify the magnitude of intervention effects in single-case design research (Heyvaert & Onghena, 2014). Several different effect size calculations have been proposed for use within single-case research, including parametric (e.g., R2), non-parametric [e.g., percentage of non-overlapping data (PND)], and standardized mean difference methods (Ross & Begeny, 2014). However, there are several limitations to using these methods, including an inability to compare with effect sizes from between-group studies (Kratochwill et al., 2010), lack of control for (or over control of) baseline trend (Parker, Vannest, Davis, & Sauber, 2011; Ross & Begeny, 2014), and vulnerability to outliers (Lenz, 2013). The Tau-U effect size, developed by Parker and colleagues (2011), addresses these limitations by controlling for baseline trend and combining non-overlap between phases with trend from the intervention phase. As a non-parametric method, Tau-U is 55 theoretically less influenced by sample size (Ross & Begeny, 2014). It is distribution free, does not require data to meet statistical assumptions, provides accurate calculations when there are few data points, has good statistical power, and manages autocorrelation well (Brossart, Vannest, Davis, & Patience, 2014; Parker et al., 2011). Due to these strengths, the Tau-U effect size has been recommended as an ideal approach for single-case design (Parker et al., 2011). According to Parker, Vannest, and Davis (2014), calculation of Tau-U effect size (in its simplest form) involves four major steps. The first step is to calculate the number of contrasted pairs between all of the data points for a participant (#Pairs) by multiplying the number of ratings in the baseline phase by the number of ratings in the intervention phase. Second, each contrasted pair is coded as either a) positive or improving over time (+), b) negative or decreasing over time (-), or c) tied (T). Third, Snovlap is calculated by finding the difference between the number of positive and negative codes (#pos - #neg). Fourth, the Tau-U statistic is calculated using the formula S / #Pairs. Tau-U can also be calculated to control for baseline trend by computing Strend (#pos - #neg in Phase A) and subtracting this value from Snovlap in the Tau-U formula (Snovlap – Strend / #Pairs). The Tau-U effect size statistic can be interpreted as the “percent of non-overlap minus overlap” (Parker et al., 2014, p. 130). For example, a Tau-U of .45 indicates that 45% of data did not overlap (i.e., showed improvement) between phase A and B (after controlling for phase A trend; Bruni et al., 2017). Clinical significance of single-case design data. According to Heyvaert and Onghena (2014), “a statistically significant difference in outcome measures between baseline and treatment phases does not necessarily mean that the treatment has had a clinically significant impact” (p. 62). Thus, it is important to measure clinical significance of single-case data, along with the reliability of the behavior change (i.e., randomization tests) and the magnitude of the 56 effect (i.e., effect size calculation; Heyvaert & Onghena, 2014). The Reliable Change Index (RCI; Jacobson & Truax, 1991) is one of the most frequently reported methods for assessing clinically significant change in intervention research. The RCI value indicates whether an observed difference in scores should be attributed to real change (i.e., clinical significance) or to measurement error (Lambert & Ogles, 2009). A RCI is calculated using the following formula: !"#= &'−&) *+,-- *+,--= .2 (*1)' where x1 equals the mean score in the baseline phase and x2 equals the mean score in the treatment phase. Sdiff represents the standard error of difference between the two test scores, which can be calculated using the following formula: where SE represents the standard error of measurement. RCIs greater than 1.96 reflect statistically reliable changes in behavior and, therefore, meaningful improvement (Jacobson & Truax, 1991). Research Questions and Hypotheses Using a non-concurrent, multiple-baseline, single-case design, the purpose of the present study was to explore the effectiveness, integrity, and acceptability of the DECA-P2 family guide (Mackrain & Cairone, 2013), carried out in a self-administered format within an at-risk population (e.g., Head Start). Specific research questions and hypotheses are as follows (see Table 7 for an overview of the research questions, variables, hypotheses, measures, and data analysis procedures): Question 1: Is the DECA-P2 self-administered parent training guide an effective intervention for building social-emotional competence and reducing behavior concerns in three Head Start preschoolers from low-income backgrounds? 57 Table 7. Research Questions, Variables, Hypotheses, Measures, and Data Analyses Questions Multiple-Baseline Question 1: Is the DECA- P2 self-administered parent training guide an effective intervention for building social-emotional competence and reducing behavior concerns in three Head Start preschoolers from low-income backgrounds? Question 2: Is the DECA- P2 self-administered parent training guide an effective intervention for building positive parenting practices and reducing negative parenting practices for three parents of Head Start preschoolers from low-income backgrounds? Dependent Variables Social-emotional competence: the ability of children to successfully interact with other children and adults in a way that demonstrates an awareness of, and ability to manage, emotions in an age- and context-appropriate manner Behavior concerns: problem or challenging behaviors, such as aggression, withdrawal, and inattention Positive parenting practices: behaviors conceptualized as being warm, nurturing, and/or supportive Negative parenting practices: behaviors described as harsh, overreactive, emotionally negative, coercive, controlling, and authoritarian Hypotheses The DECA-P2 parent training guide will result in a significant increase in children’s social- emotional competence and a significant decrease in children’s behavior concerns. The DECA-P2 parent training guide will result in a significant increase in positive parenting practices and a significant decrease in negative parenting practices. 58 Measures Primary 1) Direct Behavior Rating (DBR) – Average Social- Emotional Competence 2) Direct Behavior Rating (DBR) – Behavior Concerns Supplemental 1) Devereux Early Childhood Assessment – Second Edition (DECA- P2) rating scale 2) Coder Impressions Inventory (CII) Primary 1) Goal Attainment Scaling 2) Goal Attainment Scaling (GAS) – Positive Parenting Practice (GAS) – Negative Parenting Practice Supplemental 1) Parent Behavior Inventory 2) Coder Impressions Inventory (CII) Data Analyses Primary DBRs: Visual analysis; Wampold and Worsham (1986) randomization test; Tau-U effect-size Supplemental DECA-P2: Tau-U effect size; Reliable Change Index (RCI) CII: Tau-U effect size Primary GAS: Visual analysis; Wampold and Worsham (1986) randomization test; Tau-U effect size Supplemental PBI & CII: Tau-U effect size Table 7 (cont’d) Pre-Test/Post-Test Question 3: Will the DECA-P2 self- administered parent training guide lead to a significant reduction in parenting stress from pre- test to post-test for three parents of Head Start preschoolers from low- income backgrounds? Descriptive Question 4: Will the DECA-P2 self- administered parent training guide be implemented as intended (i.e., treatment integrity) by three parents of Head Start preschoolers from low-income backgrounds? Question 5: Is the DECA- P2 self-administered parent training guide an acceptable intervention option for three parents of Head Start preschoolers from low-income backgrounds? Parenting stress: aversive psychological reaction to the demands of being a parent Parenting stress will decrease from the pre-test to the post-test. 1) Parenting Stress Index – Short Form (PSI-SF) Reliable Change Index (RCI) Integrity: the extent to which an intervention was carried out as intended The DECA-P2 parent training guide will be implemented with high integrity (i.e., 80% or above). Acceptability: how appropriate, relevant, and useful an intervention is perceived to be The DECA-P2 parent training guide will be rated as being highly acceptable. 1) Weekly Integrity Checklist Descriptive analysis Primary 1) Treatment Evaluation Questionnaire – Parent (TEQ-P) acceptability scale Supplemental 1) Open-Ended Interview Primary Descriptive analysis Supplemental Informal review of responses 59 Although parents are the direct participants in self-administered training programs, previous studies have documented positive outcomes for children (i.e., increases in social- emotional competencies, reductions in behavior concerns), especially for at-risk preschoolers (e.g., Webster-Stratton et al., 1988). Therefore, it was hypothesized that child social-emotional competence would increase and child behavior concerns would decrease after exposure to the DECA-P2 parent training intervention, as measured by parent ratings and in-home direct observations. Child behavior is described as being especially adaptable in early childhood (Sitnick et al., 2015) and strongly influenced by interactions with parents (Sheridan et al., 2014), suggesting that behavior changes could be observed during the course of an eight-week parent- focused intervention. The primary outcome measure of social-emotional competence in this study was the mean of three Direct Behavior Ratings (DBRs; Chafouleas, Riley-Tillman, & Christ, 2009) completed by parents (i.e., initiative, self-regulation, attachment/relationships). Supplemental measures included the Total Protective Factors (TPF) scale of the Devereux Early Childhood Assessment – Second Edition (DECA-P2) rating scale (LeBuffe & Naglieri, 2012) and the Child Bonding with Parent scale of the Coder Impressions Inventory – Child (CII – Child; Webster-Stratton, 1998), which was completed after a brief direct observation in the child’s home environment. The primary outcome measure of behavior concerns was a DBR completed by parents. Supplemental measures included the Behavior Concerns (BC) scale of the DECA-P2 rating scale, and the Child Noncompliant/Aggressive scale of the CII – Child. Effectiveness was represented by significant changes in scores on primary outcome measures (based on both visual and statistical analysis) for all three participants (i.e., a replicated effect). Statistical analysis was used to determine whether supplemental measures produced findings similar to or different from DBRs (i.e., the primary measure). 60 Question 2: Is the DECA-P2 self-administered parent training guide an effective intervention for building positive parenting practices and reducing negative parenting practices for three parents of Head Start preschoolers from low-income backgrounds? Experimental research indicates that preventative parent training interventions (e.g., Incredible Years) are successful in enhancing parents’ use of positive practices (e.g., responsiveness, warmth; Sandler et al., 2011) and reducing parents’ use of negative practices (e.g., Marcynyszyn, Maher, & Corwin, 2011). Accordingly, it was hypothesized that parents in the present study would use more positive parenting practices and fewer negative parenting practices after exposure to the DECA-P2 self-administered parent training intervention, as evidenced by self-ratings and in-home direct observations. Considering the malleability of parenting practices while children are young (LeBuffe & Naglieri, 2012; Sitnick et al., 2015), it was anticipated that positive changes in parenting practices could be observed throughout the eight-week intervention period. The primary outcome measure of parenting practices was Goal Attainment Scaling (GAS; Kiresuk & Sherman, 1968), and effectiveness was represented by significant changes in scores (based on both visual and statistical analysis) for all three participants (i.e., a replicated effect). Supplemental measures included the Parent Behavior Inventory (PBI; Lovejoy, Weis, O’Hare, & Rubin, 1999) and the Nurturing/Supportive and Harsh/Critical scales of the Coder Impressions Inventory – Parent (CII – Parent; Webster- Stratton, 1998). Statistical analysis was used to determine whether supplemental measures produced findings similar to or different from GAS ratings (i.e., the primary measure). Question 3: Will the DECA-P2 self-administered parent training guide lead to a significant reduction in parenting stress from pre-test to post-test for three parents of Head Start preschoolers from low-income backgrounds? 61 According to the family stress model, parent mental health (e.g., parenting stress) and parenting practices mediate the relationship between poverty and child developmental outcomes (Linver et al., 2002). While the hardships associated with poverty (e.g., seeking public assistance, reducing the consumption of goods and services) are thought to increase parenting stress (Yeung et al., 2002), participation in parent training programs has been linked with decreases in parenting stress (e.g., Maaskant et al., 2017; Stattin, Enebrink, Özdemir, & Giannotta, 2015), including those that are self-administered (Tarver, Daley, Lockwood, & Sayal, 2014). As a result, it was hypothesized that parents would report lower levels of parenting stress after participation in the DECA-P2 self-administered parent training intervention. The primary outcome measure of parenting stress was the Parenting Stress Index – Short Form (PSI-SF; Abidin, 2012), and a significant reduction in scores was represented by a reliable change in the T-score from pre-test to post-test (Reliable Change Index > 1.96; Jacobson & Truax, 1991) for all three participants (i.e., a replicated effect). Question 4: Will the DECA-P2 self-administered parent training guide be implemented as intended (i.e., treatment integrity) by three parents of Head Start preschoolers from low- income backgrounds? To increase the likelihood of effective outcomes, interventions must be carried out as intended (i.e., high levels of integrity; Forman et al., 2013). Research suggests that self- administered parent training programs (e.g., Incredible Years) can be implemented with moderate-to-high treatment integrity (e.g., Kratochwill et al., 2003; Ogg & Carlson, 2009). This supports the hypothesis that the DECA-P2 program’s self-administered parent training guide would be implemented with high (80% and above) levels of treatment integrity (Perepletchikova & Kazdin, 2005), according to a researcher-developed integrity checklist. 62 Question 5: Is the DECA-P2 self-administered parent training guide an acceptable intervention option for three parents of Head Start preschoolers from low-income backgrounds? Treatment acceptability is influenced by various factors, including the time and effort needed to participate in an intervention, the straightforwardness of the intervention description and procedures, and intervention effectiveness (Villarreal et al., 2015). Given that the DECA-P2 self-administered parent training intervention can be implemented flexibly, is written in parent- friendly language, and has demonstrated effectiveness in a small pilot study (Thomson & Carlson, 2016), it was hypothesized that parents who participate in the program would rate the intervention to be highly acceptable. Investigations of other self-administered parent training programs (e.g., Incredible Years) have also documented high levels of treatment acceptability (Ogg & Carlson, 2009; Stewart & Carlson, 2010), providing support for the hypothesis that parents would also find the DECA-P2 parent training guide to be highly acceptable. The primary outcome measure of acceptability was the acceptability scale of the Treatment Evaluation Questionnaire – Parent Form (TEQ-P; Kratochwill et al., 2003). High levels of acceptability were determined by scores of 55 or above on this scale (Kratochwill et al., 2003), as rated by parents upon completion of the DECA-P2 self-administered parent training intervention. A short open-ended interview was used as a supplemental measure of acceptability. 63 Participants CHAPTER 3 Methods Participants included parents (N = 3) of 4- to 5-year-old children enrolled in Head Start programs across Michigan. Six potential participants were identified via the study recruitment efforts described in the Procedures section, four participants were enrolled in the study after qualifying based on inclusion and exclusion criteria (see Appendix A), and three participants (i.e., the final sample) completed the intervention. Table 8 provides an overview of the demographic and baseline data for each enrolled participant. The final sample included three biological parents (2 females, 1 male) with 4- to 5-year-old children enrolled in Head Start programs (2 males, 1 female). Families represented diverse racial/ethnic backgrounds, with Parent 1 identifying as Latino/Hispanic (Child 1 was of mixed race/ethnicity), Parent/Child 2 as Black/African American, and Parent/Child 3 as White/Caucasian. Parent ages ranged from 24 to 48 years old. All participating families were living in poverty, as indicated by household incomes below the federal poverty line (U.S. Department of Health and Human Services, 2016). Parent 4, a 33-year-old White/Caucasian biological mother, and Child 4, a 4-year-old White/Caucasian boy, enrolled in the study, but dropped out during the baseline phase due to scheduling conflicts. Children and parents demonstrated a need for the present intervention based on less-than- optimal social-emotional competence and use of positive parenting practices. All children presented with at-risk social-emotional development, as indicated by scores in the Area of Need range (i.e., T-scores between 28 and 40) on the DECA-P2 Total Protective Factors (TPF) scale or one or more of the individual DECA-P2 TPF subscales (LeBuffe & Naglieri, 2012). Based on initial DECA-P2 ratings, self-regulation was considered an Area of Need for all three 64 Table 8. Demographic Characteristics and Baseline Ratings for the Four Enrolled Participants Characteristic Parent Age/Sex Race/Ethnicity Relationship to Child Marital Status Parent/Child 1 48/M Latino/ Hispanic Biological Father Married Parent/Child 3 24/F Parent/Child 2 25/F Black/African American White/ Caucasian Biological Mother Biological Mother Living together as if Living together as if married married High School or GED Some college College Graduate $15,000-$19,000 Less than $4,999 Working part-time Working full-time Parent/Child 4* 33/F White/ Caucasian Biological Mother Separated Highest Level of Education High School or GED Annual Household Income Employment Status PBI – Supportive/Engaged Scale Items Rated 2 or Below $10,000-$14,999 Not working by choice (disability) “My child and I spend time playing games, doing crafts, or doing other activities” Child Age/Sex Race/Ethnicity Initial DECA-P2 TPF Score Initiative Self-Regulation Attachment/Relationships 5/M Mixed 46 50 40 51 *Parent/Child 4 dropped out of the study after the first baseline home visit. 65 “My child and I spend time playing games, doing crafts, or doing other activities” “My child and I spend time playing games, doing crafts, or doing other activities;” “I listen to my child’s feelings and try to understand them” 4/F Black/African American White/ Caucasian 52 53 36 66 30 34 28 33 4/M Less than $4,999 Not working by choice (disability) “My child and I spend time playing games, doing crafts, or doing other activities” 4/M White/ Caucasian 28 38 46 36 participants, while overall social-emotional competence (i.e., TPF scale) was only considered an Area of Need for one participant (Child 3). All parents reported infrequent use of one or more positive parenting practices, which was determined by a self-rating of 2 (Somewhat True) or below on at least one item on the Supportive/Engaged scale of the Parent Behavior Inventory (PBI; Lovejoy et al., 1999). Measures Multiple-baseline measures. Primary measures. Direct behavior rating (DBR; Chafouleas, Riley-Tillman, et al., 2009). Direct behavior ratings (DBRs) were used to monitor children’s progress related to the three categories of social- emotional competence (i.e., initiative, self-regulation, and attachment/relationships), as well as behavior concerns. DBRs are completed by having an observer (e.g., parents, teachers, school psychologists) use a pre-determined scale (e.g., 0 to 10; 0% to 100%) to provide a rating of a specified behavior over a set amount of time (e.g., day, class period). DBRs are meant to capture the strengths of both behavior rating scales (e.g., efficiency) and systematic direct observation (e.g., repeatability, flexibility; Chafouleas, Riley-Tillman, et al., 2009). DBRs have been found to significantly correlate with systematic direct observation (r ≥ .81, p < .01; Riley-Tillman, Chafouleas, Sassu, Chanese, & Glazer, 2008) and behavior rating scales (i.e., Social Skills Rating System; r ≥ .47, p < .05; Chafouleas, Kilgus, & Hernandez, 2009), indicating strong convergent validity. Research indicates that DBRs have strong temporal reliability (i.e., coefficients ≥ .7) with only a small number of DBR ratings [e.g., 2 to 10 (Kilgus, Riley-Tillman, Stichter, Schoemann, & Bellesheim, 2016), 5 to 10 (Chafouleas et al., 2013), 7 to 10 (Chafouleas, Christ, Riley-Tillman, Briesch, & Chanese, 2007)]. DBRs have also demonstrated 66 sensitivity to behavior change using a variety of change metrics (e.g., absolute change, percentage of change, effect size, RCI; Chafouleas, Sanetti, Kilgus, & Maggin, 2012). Overall, psychometric data suggest that DBRs can serve as reliable, valid, time-efficient options for behavioral data collection. In the present study, parents provided DBR ratings using an 11-point scale [ranging from 0 (Never) to 10 (Always)] for four behaviors: 1) “My child showed initiative today/this week (e.g., tried new things, asked questions, stuck with a task even when it was hard to do)” (Initiative), 2) “My child showed self-control today/this week (e.g., calmed down when upset, listened to others, showed patience)” (Self-Regulation), 3) “My child interacted well with peers and adults today/this week (e.g., showed affection for familiar adults, appeared happy when playing with others)” (Attachment/Relationships), and 4) “My child showed problem behaviors today/this week (e.g., difficulty concentrating, hurt others with actions or words, temper tantrums)” (Behavior Concerns; see Appendix B). Behaviors and examples were adapted from the information provided in the DECA-P2 self-administered parent training guide (Mackrain & Cairone, 2013). While state behaviors such as these are highly influenced by contextual variables (e.g., day, time, setting; Chafouleas et al., 2010), a similar multi-item DBR measuring social competence (i.e., works to resolve conflicts, interacts cooperatively) found that high levels of reliability could be obtained over just a few ratings (i.e., 7 to 10) within a sample of preschoolers (Chafouleas, Christ, et al., 2007). Additionally, Riley-Tillman, Chafouleas, Christ, Briesch, and LeBel (2009) found that ratings were more accurate when DBR forms included more global definitions of behavior (e.g., “academic engagement,” “disruptive behavior”), rather than more specific, discrete behaviors (e.g., “raising hand,” “following teacher directions”). An average score was computed for the first three behaviors on the DBR form (i.e., Initiative, Self- 67 Regulation, Attachment/Relationships) to get an overall rating of social-emotional competence (i.e., Average Social-Emotional Competence). Behavior concerns were measured using a single item (i.e., the fourth item on the DBR form). Higher scores for the average of the first three behaviors indicate higher social-emotional competence, while lower scores on the fourth behavior indicate fewer behavior concerns. Parents completed DBRs daily throughout the baseline phase and weekly throughout the treatment phase (see Table 9). Reliability of DBR scores in the present study was computed using intraclass correlation coefficients (ICCs; Shrout & Fleiss, 1979) and the formula for reliability estimates outlined by Kilgus and colleagues (2016): !""= $∗&'' (1+($−1)∗&'') where k is the lowest number of DBR ratings completed for a participant (k = 6 in the baseline phase and k = 8 in the intervention phase). Following the procedures outlined by Kilgus et al., reliability estimates were calculated separately for the baseline and intervention phases. During the baseline phase, reliability coefficients were unacceptable for Initiative scores (rxx = .19), questionable for Behavior Concerns scores (rxx = .62), good for Self-Regulation (rxx = .80) and Average Social-Emotional Competence scores (rxx = .84), and excellent for Attachment/Relationships scores (rxx = .90). During the intervention phase, all reliability coefficients were in the acceptable-to-excellent range (Initiative: rxx = .94; Self-Regulation: rxx = .70; Attachment/Relationships: rxx = .97; Average Social-Emotional Competence: rxx = 1.00), except for Behavior Concerns scores, which demonstrated unacceptable reliability (rxx = .29). Goal attainment scaling (GAS; Kiresuk & Sherman, 1968). Goal attainment scaling (GAS) is a criterion-referenced measure of progress towards individual goals. GAS involves selecting and defining a unique set of goals, specifying the range of outcomes from positive to 68 Table 9. Baseline Assessments Planned for Each Phase Phase Pre-Test Assessment Plan 1) Demographic Form 2) Parenting Stress Index – Short Form (PSI-SF) 1) Direct Behavior Rating (DBR) 2) Goal Attainment Scaling (GAS) 3) Devereux Early Childhood Assessment – Second Edition (DECA-P2) rating scale 4) Parent Behavior Inventory (PBI) 5) Coder Impressions Inventory (CII) Treatment 1) Direct Behavior Rating (DBR) 2) Goal Attainment Scaling (GAS) 3) Devereux Early Childhood Assessment – Second Edition (DECA-P2) rating scale 4) Parent Behavior Inventory (PBI) 5) Coder Impressions Inventory (CII) 6) Weekly Integrity Checklist 1) Treatment Evaluation Questionnaire – Parent (TEQ-P) acceptability scale 2) Parenting Stress Index – Short Form (PSI-SF) 3) Open-Ended Interview 1) Direct Behavior Rating (DBR) 2) Goal Attainment Scaling (GAS) Post-Test Follow-Up (4 weeks post- intervention) Completed By Parent (1x total) Parent (1x total) Parent (1x daily) Parent (1x daily) Parent (3x total) Parent (3x total) Researcher (3x total) Parent (1x weekly) Parent (1x weekly) Parent (3x total) Parent (3x total) Researcher (3x total) Parent (1x per week) Parent (1x total) Parent (1x total) Researcher (1x total) Parent (1x) Parent (1x) negative, and ranking the level of attainment at a given point in time (Brady, Busse, & Lopez, 2014). In the present study, GAS was used to measure parents’ perceptions of changes in their parenting practices. Parents rated their progress towards two parenting practice goals, selected based on their initial ratings of positive and negative parenting practices using the Parent Behavior Inventory (PBI; Lovejoy et al., 1999; see below). Specifically, the lowest-rated 69 parenting practice from the Supportive/Engaged scale and the highest-rated parenting practice from the Hostile/Coercive scale were selected for monitoring with GAS (see Appendix C for a sample GAS form). When there was a tie for the lowest-rated positive and/or the highest-rated negative parenting practice, parents were asked to subjectively select the practice that they used least or most often (of the tied items). Each goal was rated on a 5-point scale. If parents achieved at the expected level, a rating of 0 was selected. If they achieved at a higher-than-expected level, a rating of +1 (somewhat more) or +2 (much more) was selected. If they achieved at a lower- than-expected level, a rating of -1 (somewhat less) or -2 (much less) was selected (Turner- Stokes, 2009). Parents completed GAS ratings daily throughout the baseline phase and weekly throughout the treatment phase (see Table 9). Research has demonstrated evidence of strong reliability and validity for the GAS approach. For example, when using GAS in clinical settings, researchers have found high interrater reliability, with coefficients ranging from .87 to .93 (Cardillo & Smith, 1994; Shefler, Canetti, & Wiseman, 2001; Stolee, Rockwood, Fox, & Streiner, 1999). Cardillo and Smith (1994) also documented high test-retest reliability, with a coefficient of .84. Regarding validity, Schlosser (2004) indicated that low correlations can be expected between GAS and standardized measures, given the individualized nature of the GAS method. Nonetheless, strong correlations have been found between GAS and standardized measures assessing change, such as the Health- Sickness Rating Scale (r = .70, p < .001), the Target Complaints Scale (r = .50, p < .01), the Brief Symptom Inventory (r = .38, p < .05), and the Rosenberg Self-Esteem Scale (r = .34, p < .05; Shefler et al., 2001). Notably, the many strengths of the GAS approach, including time efficiency and user friendliness (Roach & Elliott, 2005), result in very high social validity 70 (Schlosser, 2004), which is an important characteristic for measures being used in applied settings. Reliability of GAS scores in the present study was computed using the formula outlined by Kilgus and colleagues (2016). During the baseline and intervention phases, reliability was good-to-excellent for both positive parenting practice ratings (Baseline: rxx = .97; Intervention: rxx = .93) and negative parenting practice ratings (Baseline: rxx = .84; Intervention: rxx = .97). Supplemental measures. Devereux Early Childhood Assessment – Second Edition (DECA-P2) rating scale (LeBuffe & Naglieri, 2012). Social-emotional competence and behavior concerns were measured with the Devereux Early Childhood Assessment – Second Edition (DECA-P2), a 38-item strengths-based assessment for preschool children between the ages of 2 and 5. Positive behaviors are examined through the 27-item Total Protective Factors (TPF) scale, which is further divided into three subscales: Initiative, Self-Regulation, and Attachment/Relationships. On the TPF scale (and each of the subscales), T-scores of 60 or above represent a Strength, scores between 41 and 59 represent Typical functioning, and scores of 40 or below represent an Area of Need. Behavior concerns (BC) are assessed through an 11-item screener. On the BC scale, scores of 59 and below are considered Typical and scores of 60 or above are considered an Area of Need (LeBuffe & Naglieri, 2012). Parents completed the DECA-P2 rating scale three times during the baseline phase and three times during the intervention phase (see Table 9). The DECA-P2 demonstrates strong psychometric properties, with all measures of reliability meeting or exceeding common standards. Internal consistency coefficients are .92 and .80 for the TPF and BC scales, respectively, which both exceed the minimum acceptable standard of .70 (Stemler & Tsai, 2011). Test-retest reliability is respectable, as well, with strong, 71 significant correlations between parents’ ratings of the same child at different time points for both the TPF and BC scales (r = .88, p < .01 and r = .78, p < .01, respectively). Further, pairs of parents rated the same children similarly on both the TPF and BC scales (i.e., inter-rater reliability), corrected r = .51, p < .01 and corrected r = .46, p < .01, respectively (LeBuffe & Naglieri, 2012). The DECA-P2 TPF and BC scales have been found to be stable over the long- term (i.e., 12 months; TPF scale: r = .65; BC scale: r = .53; Carlson & Voris, 2017). The DECA-P2 also exhibits solid validity. For instance, strong, significant differences in scores between a clinical sample (identified as having “emotional or behavioral disturbances”; N = 125) and a comparison sample (N = 126) have been documented (i.e., criterion-related validity). Additionally, the DECA-P2 TPF and BC scales demonstrate strong convergent validity with the Preschool Behavioral and Emotional Rating Scale (PreBERS; corrected r = .63 - .78; Epstein & Synhorst, 2009) and the Conners Early Childhood Scale (Conners EC; corrected r = .60 - .64; Conners, 2009), two other measures of social, emotional, and behavioral strengths and concerns (LeBuffe & Naglieri, 2012). Parent Behavior Inventory (PBI; Lovejoy et al., 1999). The Parent Behavior Inventory (PBI) is a 20-item rating scale of parenting practices that is relevant to preschool-age children (see Appendix D). The PBI includes two 10-item subscales: 1) Hostile/Coercive, and 2) Supportive/Engaged. Examples of items on the Hostile/Coercive scale include “I lose my temper when my child doesn’t do something I ask him/her to do” and “I grab or handle my child roughly.” Examples of items on the Supportive/Engaged scale include “My child and I hug and/or kiss each other” and “I thank or praise my child.” Parents rate each item on a 6-point Likert-type scale from 0 (Not at all true/I do not do this) to 5 (Very true/I often do this). Total scores for each of the two scales range from 0 to 50, with higher scores indicating more frequent 72 use of the parenting behaviors (Lovejoy et al., 1999). Parents completed the PBI three times during the baseline phase and three times during the intervention phase (see Table 9). The PBI has demonstrated solid psychometric properties. In terms of reliability, strong internal consistency has been documented with Cronbach’s alpha coefficients of .81 and .83 for the Hostile/Coercive and Supportive/Engaged scales, respectively (Lovejoy et al., 1999). In the present study, strong internal consistency was also documented for both scales (Supportive/Engaged: a = .82; Hostile/Coercive: a = .80). Lovejoy and colleagues (1999) found that PBI ratings one week apart were highly correlated (r = .69 for the Hostile/Coercive scale and r = .74 for the Supportive/Engaged scale), which suggests good test-retest reliability. Significant, though small, correlations were found between mothers’ ratings of their own parenting behavior and their husbands’ ratings of maternal parenting behavior (i.e., interrater reliability). In terms of validity, a confirmatory factor analysis supported the two-factor structure of the measure. Furthermore, PBI scores were significantly correlated with scores on other rating scales related to parenting, including the Parenting Stress Index – Short Form (PSI-SF; Abidin, 2012) and the Positive and Negative Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988; Lovejoy et al., 1999). Coder Impressions Inventory (CII; Webster-Stratton, 1998). To supplement parent ratings of children’s behavior and self-ratings of parenting practices, parent-child interactions were observed and recorded in the home environment. The Coder Impressions Inventory (CII; Webster-Stratton, 1998), adapted from the Oregon Social Learning Center’s Impression Inventory, is an 81-item behavioral observation measure that assesses parenting practices and child behaviors during parent-child social interactions (see Appendix E). Parents and children were observed over a 15-minute period, including 12 minutes of free play (using researcher- 73 provided toys and games) and 3 minutes of clean-up. Observations were video recorded for later analysis and review. While Webster-Stratton (1998) initially utilized the CII following a 30- minute observation of parent-child interactions, recent studies have demonstrated success in using the tool with shorter observation periods (e.g., 12 – 20 minutes; Deković et al., 2010; Rusby, Smolkowski, Marquez, & Taylor, 2008). Videos were recorded three times during the baseline phase and three times during the intervention phase (see Table 9). Video-recorded observations were coded separately by the project coordinator and one research assistant using the CII. When coding, researchers watched each video-recorded observation twice (once to rate child behavior and once to rate parenting practices). Using the CII, behaviors are typically rated using a three-point scale: 1) Did Not Occur, 2) 1-3 Examples, 3) 4 Or More Examples; however, for some items, raters can select 0 (No Basis) if there was no opportunity to observe the behavior in the 15-minute observation period. For child behavior, item ratings yield two subscale scores: 1) Child Bonding with Parent (6 items; e.g., physically/verbally affectionate, attached to parent), and 2) Child Noncompliant/Aggressive (6 items; e.g., noncompliance, physical aggression). Example items on the Child Bonding with Parent scale include “Child was verbally affectionate to parent” and “Child was physically affectionate with parent.” Example items on the Child Noncompliant/Aggressive scale include “Child was physically aggressive toward parent” and “Child shouted at parent.” In previous research, the Child Bonding with Parent and Child Noncompliant/Aggressive scales demonstrated questionable to acceptable internal consistency (Cronbach’s alphas = .61 and .73, respectively; Reid, Webster-Stratton, & Hammond, 2007). In the present study, the Child Bonding with Parent scale demonstrated unacceptable internal consistency (Cronbach’s alpha = .37) and the Child Noncompliant/Aggressive scale demonstrated questionable internal 74 consistency (Cronbach’s alpha = .60). To demonstrate interrater reliability, all videos were coded by the project coordinator and one project assistant. Interrater reliability was excellent for both the Child Bonding with Parent scale (ICC = .96) and the Child Noncompliant/Aggressive scale (ICC = .86). For parenting practices, the CII yields two subscales: 1) Nurturing/Supportive (13 items; e.g., positive encouragement, patience, physical/verbal affection), and 2) Harsh/Critical (12 items; e.g., criticism, anger/hostility, nagging, physical aggression). Example items on the Nurturing/Supportive scale include “Paid attention when child talked or asked questions” and “Parent was verbally affectionate to child (positive tone of voice, pet name, etc.).” Example items on the Harsh/Critical Scale include “The parent threatened punishment for misbehavior (‘If you do that again, I’ll hit you’)” and “Parent shouted at child.” Each of these scales has demonstrated high internal consistency, with Cronbach’s alpha scores of .88 and .89 for the Nurturing/Supportive and Harsh/Critical scales, respectively (Webster-Stratton, Reid, & Hammond, 2004). In the present study, the Nurturing/Supportive scale demonstrated questionable internal consistency (Cronbach’s alpha = .63), while the Harsh/Critical scale revealed acceptable internal consistency (Cronbach’s alpha = .75). Interrater reliability was excellent for both the Nurturing/Supportive scale (ICC =.90) and the Harsh/Critical scale (ICC = .90). Relationship between measures. Pearson correlation coefficients were calculated for child and parent outcome measures to assess the degree to which they were related for the participants in the present study (see Table 10). None of the three measures of child social- emotional competence (i.e., DBR – Average Social-Emotional Competence, DECA-P2 TPF scale, CII – Child Bonding with Parent) were significantly correlated. Regarding measures of 75 Table 10. Correlations between Outcome Measures Measures Child Outcome Measures DBR-SEC DBR-SEC DECA-P2 TPF CII-BOND DBR-BC DECA-P2 BC CII-N/A - .34 .19 -.52* -.39 -.10 DECA-P2 TPF .34 - -.13 -.12 -.88*** -.52* CII-BOND DBR-BC .19 -.13 - -.34 .16 -.30 -.52* -.12 -.34 - .11 -.01 DECA-P2 BC -.39 -.88*** .16 .11 - .54* CII-N/A -.10 -.52* -.30 -.01 .54* - GAS-POS - PBI-S/E .66** -.74*** CII-N/S GAS-NEG PBI-H/Co CII-H/Cr Parent Outcome Measures GAS-POS PBI-S/E CII-N/S GAS-NEG PBI-H/Co CII-H/Cr PSI Note. BC=Behavior Concerns. BOND=Bonding with Parent. CII=Coder Impressions Inventory. DBR=Direct Behavior Rating. GAS=Goal Attainment Scaling. H/Co=Hostile/Coercive. H/Cr=Harsh/Critical. N/A=Noncompliant/Aggressive. NEG=Negative Parenting Practice. PBI=Parent Behavior Inventory. POS=Positive Parenting Practice. PSI=Parenting Stress Index-Short Form. SEC=Social-Emotional Competence. TPF=Total Protective Factors. *p < .05, **p < .01, ***p < .001 PSI -.88* -.73 .10 -.15 .64 -.19 - .13 -.14 .28 - -.03 -.51* -.15 .16 .20 -.05 -.51* .12 - -.19 .66** .48* .13 -.66** .16 -.88* .48* .42 - .28 -.21 -.05 .10 - .42 -.14 .20 -.73 -.21 -.03 - .12 .64 -.66** -.74*** 76 child behavior concerns, there was a significant moderate correlation between DECA-P2 BC scale and CII – Child Noncompliant/Aggressive scale scores (r = .54, p < .05). As expected, there was a strong negative correlation between DECA-P2 TPF and BC scale scores (r = -.88, p < .001). There were also moderate negative correlations between DBR – Average-Social Emotional Competence and DBR – Behavior Concerns scores (r = -.52, p < .05) and DECA-P2 TPF and CII – Child Noncompliant/Aggressive scores (r = -.52, p < .05). Regarding measures of positive parenting practices, GAS ratings were strongly correlated with PBI – Supportive/Engaged scale scores (r = .66, p < .01) and moderately correlated with CII – Nurturing/Supportive scale scores (r = .48, p < .05). There was also a significant negative correlation between GAS – Positive Parenting Practice ratings and PSI-SF scores (r = -.88, p < .05). As anticipated, there was a strong negative correlation between PBI – Supportive/Engaged scale scores and PBI – Hostile/Coercive scale scores (r = -.74, p < .001). There was an unexpected negative correlation between GAS – Negative Parenting Practice ratings and CII – Harsh/Critical scale scores (r = -.51, p < .05). Pre-test/post-test measures. Parenting Stress Index – Short Form (PSI-SF; Abidin, 2012). The Parenting Stress Index – Short Form (PSI-SF; Abidin, 2012) is a 36-item questionnaire designed to measure stress in the parent-child relationship. The PSI-SF yields three subscales: 1) Parental Distress, 2) Parent-Child Dysfunctional Interaction, and (3) Difficult Child. The Parental Distress subscale measures parents’ perceptions about their child-rearing competencies and stresses associated with parenting. The Parent-Child Dysfunctional Interaction subscale measures parents’ perceptions about the child’s ability to meet expectations and the quality of their parent-child interactions. The Difficult Child subscale measures parents’ perceptions about their child’s 77 temperament and behavior (Reitman, Currier, & Stickle, 2002). Each subscale consists of 12 items rated using a Likert-type scale from 1 (Strongly Disagree) to 5 (Strongly Agree). Subscale scores range from 12 to 60, and the Total Stress score ranges from 36 to 180. Higher scores indicate greater levels of stress, with Total Stress scores in the 81st to 84th percentile considered to be borderline clinically significant and scores in the 85th percentile and above to be clinically significant. The PSI-SF demonstrates strong internal consistency (Cronbach’s alphas above .90 for all scales; Abidin, 2012), specifically with mothers of students enrolled in Head Start (Reitman et al., 2002). Parents completed the PSI-SF before and after the treatment phase (see Table 9). Descriptive measures. Demographics. Demographic data was collected at the initial home visit, using a researcher-developed demographic form (see Appendix F). Data collected about parents included date of birth, relationship to child, sex, marital status, level of education, household size, annual household income, employment status, and race/ethnicity. Data collected about children included date of birth, sex, and race/ethnicity. Integrity. Integrity data was collected weekly using a researcher-developed integrity checklist (see Appendix G for an example integrity checklist). This measure includes a checklist of intervention components that can be used to indirectly assess the degree to which intervention procedures were carried out as intended. Treatment integrity was reflected by the percentage of intervention components completed out of the total number of intervention components. According to Perepletchikova and Kazdin (2005), interventions carried out with 80% to 100% integrity are classified as having high levels of integrity, while treatments with 50% or less indicate low levels of integrity. 78 Acceptability. Acceptability data was collected using the acceptability scale of the Treatment Evaluation Questionnaire – Parent Form (TEQ-P; see Appendix H; Kratochwill et al., 2003). The TEQ-P, developed from the Treatment Evaluation Inventory (TEI; Kazdin, 1980), contains 21 items that assess treatment acceptability, perceived effectiveness, and amount of time for improvement. In the present study, only the 11 questions associated with the acceptability scale were provided to parents at the end of the intervention (see Table 9). Each item is rated using a 6-point Likert-type scale that ranges from 1 (Strongly Disagree) to 6 (Strongly Agree). Examples of questions on the modified TEQ-P acceptability scale include: “This was an acceptable intervention for the child’s problem behavior” and “I would suggest the use of this intervention to other parents.” Total scores on the TEQ-P acceptability scale range from 11 to 66, and scores of 55 and above are typically considered to reflect high acceptability (Kratochwill et al., 2003). Although psychometric properties for the TEQ-P have not been documented, research has revealed an internal consistency reliability of .97 for the TEI (Spirrison, Noland, & Savoie, 1992). The TEQ-P revealed strong internal consistency in the present study (a = .94). Use of the TEQ-P to measure acceptability is consistent with previous studies of self-administered parent training programs (e.g., Incredible Years; Stewart & Carlson, 2010). An open-ended interview was also used to gather qualitative information about the acceptability of the intervention. At the final home visit upon completion of the intervention, parents were asked three questions: 1) What were the best aspects of the program?, 2) What aspects did you not like or not find helpful?, and 3) How could the program have been improved to help you more? Interview responses were not formally analyzed; instead, responses were informally reviewed to inform future research and practice efforts. 79 Procedures Research design. A non-concurrent, multiple-baseline, single-case design was selected for use in the present study, and data were collected via parent ratings, self-ratings, and direct observations of parenting practices and child behavior in the home setting. To ensure methodological rigor, the design of the study was developed using the four What Works Clearinghouse standards for single-case design (see Table 11). First, the independent variable was systematically manipulated by the researcher, in that the intervention was introduced after a pre-specified baseline phase (determined a priori and randomly assigned upon enrollment). Second, each outcome was measured using a multi-informant approach (i.e., a combination of parent ratings, self-ratings, and direct observations). Third, the use of a multiple-baseline design with three baseline conditions (i.e., 3 participants) allowed for the potential to demonstrate at least three intervention effects. Fourth, each phase included a minimum of five data points (Kratochwill et al., 2013). Recruitment. The participant recruitment process consisted of four phases. In Phase 1 (August 2017), the study flyer (see Appendix I) was shared with Head Start professionals (N = 4) who worked with families in six Michigan counties (Washtenaw, Livingston, Clinton, Eaton, Ingham, and Shiawassee). Head Start professionals shared information about the study with families through phones blasts and parent newsletters, and by handing out flyers at student orientations. In Phase 2 (October 2017), information about the study was shared with parents via a) an in-person presentation at one Head Start center in Washtenaw county, and b) printed flyers sent home with all students at another Head Start center in Washtenaw county. In Phase 3 (December 2017), printed flyers were sent home with students at several Head Start centers in Livingston county and information about the study was again shared in a parent newsletter. In 80 Table 11. Elements of the Present Study’s Research Design in Alignment with What Works Clearinghouse Single-Case Design Standards Standard 1) An independent variable (i.e., intervention) Aligned Procedure in the Present Study Researcher introduces the intervention after a pre-specified baseline phase. must be systematically manipulated (i.e., by the researcher, rather than some naturally occurring event). 2) Each outcome variable must be measured over time by more than one informant. 3) The study must include at least three attempts to demonstrate an intervention effect at a different point in time. 4) Each phase must include a minimum of three data points. To meet standards, a multiple- baseline design must include a minimum of six phases with at least five data points per phase. Note. Standards taken from Kratochwill et al. (2013). All outcome variables (i.e., child behavior, parenting practices) are measured by multiple informants (e.g., parent ratings, self-ratings, and/or direct observation by the researcher). The present study will utilize a multiple- baseline design with five baseline conditions (i.e., 5 participants). In each of the present study’s five baseline phases, five to nine data points will be collected. In each of the five treatment phases, eight data points will be collected. Phase 4 (January 2018), study flyers were again posted in Head Start centers, added to phone blasts, and sent home with families in Washtenaw county. Six potential participants were identified via these methods. Of the six, one parent did not qualify for participation because she did not speak English fluently, and another parent did not qualify for participation because the family’s household income was above the poverty line. Four families were enrolled in the study after qualifying based on inclusion and exclusion criteria (see Appendix A); however, one parent dropped out during the baseline phase due to scheduling conflicts. Inclusion and exclusion criteria. In order to determine whether the DECA-P2 self- administered parent training intervention aligned with a family’s needs, parents completed the DECA-P2 rating scale and the PBI. Since the primary goal of the parent training guide is to teach 81 positive parenting strategies that will support the development of children’s social-emotional competencies (Mackrain & Cairone, 2013), it was important to assess whether positive parenting practices and children’s social-emotional competence were areas in need of intervention. Thus, families were included in the present study if 1) children were rated with T-scores between 28 and 40 on the DECA-P2 TPF scale or one or more of the TPF subscales (indicating that this was an Area of Need), and 2) parents rated at least one item on the PBI Supportive/Engaged scale as a 2 or below. Scores of 2, 1, or 0 on the PBI indicate that a parent only uses the practice somewhat, a little, or not at all, demonstrating that there is room for improvement. Additional inclusion criteria included: a) the child was between 3 and 5 years of age at their last birthday, b) the child was enrolled in Head Start, and c) annual household income was below the federal poverty line. Parents were excluded from the study if their child had been diagnosed with an intellectual or developmental disorder and/or the parent was not fluent in English (see Appendix A for all inclusion and exclusion criteria). Project personnel and training. Project personnel included a) the project coordinator, and b) two project assistants. The project coordinator was a Michigan State University graduate student in school psychology who completed this project in partial fulfillment of the requirements to obtain a Doctor of Philosophy in School Psychology. The project coordinator was responsible for: a) developing the methodology for the study, b) obtaining and organizing project materials, c) conducting recruitment efforts, d) obtaining consent and enrolling participants in the study, e) training and delegating responsibilities to the project assistants, f) managing the project timeline, g) collecting data from participants, h) coding video-recorded observations, and i) analyzing data. 82 The first project assistant was a first-year graduate student at Michigan State University working towards an Education Specialist degree in School Psychology. The primary responsibilities of this project assistant included: a) collecting data from participants, b) entering and organizing data, and c) coding video-recorded observations. This project assistant was reimbursed for hours spent working on the project ($8 per hour) and miles traveled (0.53 cents per mile). The project coordinator met with this project assistant prior to the start of the project to discuss a) the responsibilities of the project assistant, b) assessment measures and scoring, and c) home visits with participants. This project assistant observed two home visits conducted by the project coordinator prior to conducting home visits independently. This project assistant provided weekly project updates to the project coordinator via e-mail, phone, or in-person. The second project assistant was an undergraduate student at Michigan State University. The primary responsibility of this project assistant was to conduct visual analysis of the study graphs. Prior to conducting visual analysis, this project assistant participated in a brief training, which included reading the What Works Clearinghouse overview of single-case design (Kratochwill et al., 2010) and a practice activity with mock data. This project assistant was reimbursed for hours spent working on training activities and visual analysis ($10 per hour). Treatment phases. Parents and children who met inclusion criteria for the study were provided with a consent form that discussed the purpose and scope of the study, possible risks and benefits, procedures for ensuring privacy and confidentiality, and costs of participation. After obtaining informed consent, the baseline phase was initiated. Families were randomly assigned to between five and nine days for the baseline phase. The treatment phase involved an eight-week intervention based on the content in the DECA-P2 parent training guide. 83 Pre-test. Prior to the start of the baseline phase, parents completed the PSI-SF to obtain initial perceptions of parenting stress. Baseline (A). Parents completed the primary measures (i.e., DBRs and GAS) daily throughout the baseline phase to rate their child’s social-emotional competence and behavior concerns, as well as their use of positive and negative parenting practices. Supplemental measures (i.e., DECA-P2 rating scale, PBI, CII) were completed three times throughout the baseline phase (see Table 9). The length of the baseline phase ranged from five days to nine days, and the three participants were randomly assigned to a baseline phase once enrolled in the study. Parents received a monetary incentive of $25 at the end of the baseline phase. Intervention (B). The intervention followed the chapters of the DECA-P2 parent training guide (Mackrain & Cairone, 2013) over an eight-week implementation period. An accompanying workbook was provided to facilitate each week’s content, and the workbook pages for each week followed the same structure (see Appendix J for an example workbook week). At the beginning of the week, parents read or reviewed short sections from the parent training guide and answered written questions to reflect on the content (e.g., “Describe a time when anger and/or frustration affected how you responded to a situation. Looking back, what might you have done differently?”). Parents were then presented with three focus strategies based on content from the parent training guide and prompted to brainstorm ideas for using the strategies in their daily life (e.g., “Which of your own feelings might you discuss with your child?”). At the end of each week, parents completed the primary measures (i.e., DBRs and GAS), along with the Weekly Integrity Checklist. Parents completed the supplemental measures (i.e., DECA-P2 rating scale, PBI, CII) at the end of each of the three intervention modules: 1) Initiative (Weeks 1-3), 2) Self- Regulation (Weeks 4-6), and 3) Attachment/Relationships (Weeks 7-8; see Table 9). Parents 84 received a monetary incentive of $25 at each of the three home visits following each intervention module (for a total incentive of $100 across the baseline and intervention phases). Post-test. Upon completion of the eight-week intervention, parents completed the TEQ-P acceptability scale and the PSI-SF. They also participated in a short open-ended interview conducted by the project assistant. Some post-test data was also collected from Parent/Child 4 (who dropped out of the study during the baseline phase) to assess progress without participation in the intervention. Specifically, Parent 4 completed the DECA-P2 rating scale and the PSI-SF (i.e., standardized measures) approximately eight weeks after the end of the originally-scheduled baseline phase. Parent 4 received a $25 Meijer gift card for completing the post-test measures. Follow-up. Four weeks after completing the eight-week intervention, parents were contacted via phone or e-mail and asked to complete primary outcome measures (i.e., DBRs and GAS). Follow-up data was used to assess maintenance of treatment effects. Data Analysis Research questions 1 and 2. To answer the first and second research questions (i.e., effectiveness related to child behavior and parenting practices), a combination of visual and statistical analyses were used (Heyvaert & Onghena, 2014). First, the project coordinator and one project assistant reviewed graphs of DBR and GAS scores using the visual analysis procedures outlined by Kratochwill and colleagues (2010). A Visual Analysis Guide was followed to determine if there was evidence of intervention effects (see Appendix K). Inter-rater reliability between the project coordinator and the project assistant was calculated using Cohen’s kappa and indicated perfect agreement (k = 1.00). Second, statistical analysis included randomization tests and effect size calculations. The Wampold and Worsham (1986) randomization test was used to assess statistical significance and 85 was calculated using the ExPRT (Excel Package of Randomization Tests; Levin, Evmenova, & Gafurov, 2014) program. In ExPRT’s multiple-baseline randomization test program, all raw data were entered in sequential order (i.e., starting with the participant for which the intervention was introduced first). Then, information about the participants and research design was entered, including the first potential intervention start point, the number of potential intervention start points, the actual intervention start point for each case, the type of data (i.e., original), the desired significance level (i.e., .05), the number of tails (i.e., one-tailed), the hypothesized direction of change (i.e., phase A > B or phase B > A), the type of data to compare between phases (i.e., mean), and the type of comparison (i.e., within-series). After running the test, the p value provided by the ExPRT program was assessed to determine whether it was statistically significant (a < .05). Tau-U effect sizes were also calculated to assess the magnitude of change from the baseline phase to the intervention phase. An online software program (www.singlecaseresearch.org; Vannest, Parker, Gonen, & Adiguzel, 2016) was used to conduct Tau-U effect size calculations. After raw data were entered into the online program, effect size calculation involved three steps. First, phase A trends were reviewed to determine if any were significant. Like Dart, Collins, Klingbeil, and McKinley (2014) and Bruni and colleagues (2017), a liberal significance value (p < .15) was used to make this determination. Second, the Tau-U statistic was calculated for each AB contrast, controlling for baseline trend if necessary. Significant baseline trends opposite of the hypothesized direction (e.g., increasing behavior concerns during baseline) were not controlled (Dart et al., 2014). Third, the online software program was used to calculate a weighted mean of all AB contrasts to get a single Tau-U statistic for each measure. 86 Third, clinical significance was assessed using the RCI (Jacobson & Truax, 1991). The RCI was only calculated for DECA-P2 scores, because the standard error of measurement is not available for DBRs, GAS ratings, PBI scores, or CII scores. Research question 3. To answer the third research question (i.e., effectiveness related to parenting stress), a RCI was computed to measure clinical significance (Jacobson & Truax, 1991). Research question 4. Descriptive analysis was used to answer the fourth research question (i.e., integrity). Using parents’ scores on the Weekly Integrity Checklists, the percent of adherence to weekly intervention protocol was calculated, as well as an average percent adherence over the whole intervention phase. Weekly and average percentages were compared to the 80% standard suggested by Perepletchikova and Kazdin (2005) to determine whether parents were able to implement the intervention with high levels of integrity. Research question 5. Descriptive analysis was also used to answer the fifth research question (i.e., acceptability). Total scores for the TEQ-P acceptability scale were calculated and compared to the standard for high acceptability (i.e., scores of 55 and above) suggested by Kratochwill and colleagues (2003). Open-ended interview responses were informally reviewed to determine what parents liked and disliked about the program. 87 Research Question 1 CHAPTER 4 Results Is the DECA-P2 self-administered parent training guide an effective intervention for building social-emotional competence and reducing behavior concerns in three Head Start preschoolers from low-income backgrounds? Social-emotional competence. Primary measure. Visual analysis of the primary multiple-baseline measure, average parent DBR ratings of social-emotional competence (i.e., mean of Initiative, Self-Regulation, and Attachment/Relationships DBR items), did not find evidence of an intervention effect for any of the three participants (see Figure 5). While there was an average increase in DBR ratings of social-emotional competence from baseline (M = 6.34; SD = 1.08; Range = 5.37-7.50) to intervention (M = 7.21; SD = .08; Range = 7.13-7.29), this was not consistent across participants, with Child 1’s average DBR scores slightly decreasing over time (Baseline: M = 7.50; SD = 1.35; Range = 5.3-8.7; Intervention: M = 7.13; SD = 1.49; Range = 4.7-9.0). Figure 6 provides a breakdown of the three DBR items (i.e., Initiative, Self-Regulation, Attachment/Relationships) that contribute to the average rating of social-emotional competence. Visual analysis did not find evidence of an intervention effect for any of the three individual DBR items. Statistical analysis confirmed that there was not a significant change in social-emotional competence from baseline to intervention, yet it is important to recognize that parents initially rated their children as exhibiting social-emotional competencies “sometimes” or “often” (i.e., average baseline DBR scores ranging from 5.37 to 7.50; Vagias, 2006). The Wampold and Worsham (1986) randomization test was not significant for average social-emotional competence 88 Figure 5. Parent DBR Ratings – Average Social-Emotional Competence g n i t a R R B D e g a r e v A g n i t a R R B D e g a r e v A g n i t a R R B D e g a r e v A 10 8 6 4 2 0 10 9 8 7 6 5 4 3 2 1 0 10 8 6 4 2 0 Baseline Intervention Follow-Up Child 1 0 1 2 3 4 5 6 7 8 9 10 Rating Number 11 12 13 14 15 16 17 18 Child 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Child 3 0 1 2 3 4 5 6 7 9 8 10 Rating Number 11 12 13 14 15 16 17 18 Note. Average Social-Emotional Competence scores reflect the mean of the first three behaviors on the DBR form (i.e., Initiative, Self-Regulation, Attachment/Relationships). scores (p = .217), suggesting that any changes in parent DBR ratings of social-emotional competence were not likely due to the intervention. Tau-U effect size calculations for average social-emotional competence DBR scores were non-significant for all three participants (Child 1: Tau-U = -.21, p = .519; Child 2: Tau-U = .39, p = .203; Child 3: Tau-U = .47, p = .102; see Table 12). 89 Parent DBR Ratings – Individual Social-Emotional Competence Items (Initiative, Self- Regulation, Attachment/Relationships) Intervention Follow-Up Baseline Figure 6. g n i t a R R B D e g a r e v A g n i t a R R B D e g a r e v A g n i t a R R B D e g a r e v A 10 8 6 4 2 0 10 8 6 4 2 0 10 8 6 4 2 0 Child 1 0 1 2 3 4 5 6 7 8 9 10 Rating Number 11 12 13 14 15 16 17 18 Child 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Child 3 0 1 2 3 4 5 6 7 8 9 10 Rating Number 11 12 13 14 15 16 17 18 Initiative Self-Regulation Attachment/Relationships Supplemental measures. DECA-P2 TPF scale scores, on average, increased from baseline (M = 37.00; SD = 7.33; Range = 29.67-44.33) to intervention (M = 46.56; SD = 6.55; Range = 44.00-54.00), though it is important to note that Child 1’s mean TPF score decreased slightly from baseline (M = 44.33; SD = 1.53; Range = 43-46) to intervention (M = 44.00; SD = 90 Table 12. Means and Effect Sizes for Measures of Child Social-Emotional Competence and Behavior Concerns M Measure/Child Primary Measure: DBR Average Social-Emotional Competence Child 1 Child 2 Child 3 Average Behavior Concerns Child 1 Child 2 Child 3 Average Supplemental Measure: DECA-P2 Total Protective Factors (TPF) T- score Child 1 Child 2 Child 3 Average Baseline M (SD; Range) 7.50 (1.35; 5.3-8.7) 6.14 (1.12; 4.3-7.7) 5.37 (2.18; 3.0-9.0) 6.34 (1.08; 5.37-7.50) 5.00 (1.79; 3-8) 7.00 (1.63; 4-9) 5.44 (2.74; 2-9) 5.81 (1.05; 5.00-7.00) 44.33 (1.53; 43-46) 37.00 (13.08; 28-52) 29.67 (1.53; 28-31) 37.00 (7.33; 29.67-44.33) Intervention M (SD; Range) 7.13 (1.49; 4.7-9.0) 7.29 (2.03; 2.7-9.0) 7.21 (2.48; 3.0-9.3) 7.21 (.08; 7.13-7.29) 4.25 (2.38; 2-8) 4.38 (3.62; 0-10) 6.25 (2.92; 1-10) 4.96 (1.12; 4.25-6.25) 44.00 (6.08; 37-48) 54.00 (19.16; 32-67) 41.67 (16.50; 28-60) 46.56 (6.55; 44.00-54.00) Change -.37 +1.15 +1.84 +.87 -.75 -2.62 +.81 -.85 -.33 +17.00 +12.00 +9.56 Tau-U p RCI -.21 .39+ .47 .23 -.31 -.46 .35+ -.13 .33 .78 .44 .52 .519 .203 .102 .193 .333 .133 .229 .465 .513 .127 .383 .078 - - .08 4.25* 3.00* 2.39* 91 Table 12 (cont’d) Behavior Concerns (BC) T-score Child 1 Child 2 Child 3 Average Supplemental Measure: CII Child Bonding with Parent Child 1 Child 2 Child 3 Average Child Noncompliant/Aggressive Child 1 Child 2 Child 3 Average *Reliable change ( > 1.96) +Corrected for baseline trend 61.00 (4.58; 56-65) 71.33 (1.15; 70-72) 72.00 (0; 72-72) 68.11 (6.17; 61.00-72.00) 2.00 (.17; 1.83-2.17) 2.00 (.33; 1.67-2.33) 2.00 (.33; 1.67-2.33) 2.00 (0; 2.00-2.00) 1.50 (.17; 1.33-1.67) 1.95 (.25; 1.67-2.17) 1.33 (.17; 1.17-1.50) 1.59 (.32; 1.33-1.95) 61.67 (3.21; 58-64) 53.33 (17.24; 38-72) 61.67 (12.34; 48-72) 58.89 (4.81; 53.33-61.67) 1.95 (.25; 1.67-2.17) 1.94 (.42; 1.50-2.33) 2.33 (.17; 2.17-2.50) 2.07 (.22; 1.94-2.33) 1.28 (.25; 1.00-1.50) 1.56 (.54; 1.17-2.17) 1.33 (.17; 1.17-1.50) 1.39 (.15; 1.28-1.56) +.67 -18.00 -10.33 -9.22 -.05 -.06 +.33 +.07 -.22 -.39 0 -.20 .11 2.85* 1.63 1.46 - - .11 -.56 -.67 -.37 -.44+ -.11 .33+ -.07 -.56 -.44 0 -.33 .827 .275 .190 .208 .383 .827 .513 .801 .275 .383 1.00 .257 92 6.08; Range = 37-48). Tau-U effect size calculations for DECA-P2 TPF scale scores were non- significant for all three participants (Child 1: Tau-U = .33, p = .513; Child 2: Tau-U = .78, p = .127; Child 3: Tau-U = .44, p = .383; see Table 12). RCI calculation only indicated a reliable increase (i.e., a larger change than what would be expected due to unreliability of a measure) in average DECA-P2 TPF scale scores for Child 2 (RCI = 4.25) and Child 3 (RCI = 3.00). During the baseline phase, average DECA-P2 TPF scale scores were in the Typical range for one participant (Child 1) and Area of Need range for two participants (Child 2, Child 3). During the intervention phase, average DECA-P2 TPF scale scores were in the Typical range for all three participants. On average, mean scores for the CII – Child Bonding with Parent scale (i.e., direct behavior observation) remained relatively stable from baseline (M = 2.00; SD = 0; Range = 2.00- 2.00) to intervention (M = 2.07; SD = .22; Range = 1.94-2.33) and reflected fairly frequent demonstration of the social-emotional competencies measured (i.e., 1-3 Examples). Tau-U effect size calculations for the CII – Child Bonding with Parent scale scores confirmed that changes were non-significant for all three participants (Child 1: Tau-U = -.44, p = .383; Child 2: Tau-U = -.11, p = .827; Child 3: Tau-U = .33, p = .513; see Table 12). Follow-up. Follow-up parent DBR ratings of social-emotional competence (four weeks post-intervention) revealed an increase in score (i.e., improvement) for Child 2 and Child 3, and a decline in score for Child 1. Child 2’s follow-up DBR rating of social-emotional competence (mean DBR score = 9.7) represented a score increase of 2.41 over the level in the intervention phase (M = 7.29), and Child 3’s follow-up rating (mean DBR score = 8.3) represented a score increase of 1.09 over the level in the intervention phase (M = 7.21). Child 2 and Child 3’s mean DBR scores at follow-up indicate use of social-emotional competencies “often” to “always” 93 (Vagias, 2006). Compared to the level in the intervention phase (M = 7.13), Child 1’s follow-up DBR rating of social-emotional competence (mean DBR score = 7.00) represented a score decrease of .13. However, it is important to note that Child 1’s follow-up mean DBR score indicated that he exhibited social-emotional competencies “often” (Vagias, 2006). Behavior concerns. Primary measure. Visual analysis of parent DBR ratings of behavior concerns, the primary multiple-baseline measure, did not find evidence of an intervention effect for any of the three participants (see Figure 7). On average, there was a slight decrease in DBR ratings of behavior concerns from baseline (M = 5.81; SD = 1.05; Range = 5.00-7.00) to intervention (M = 4.96; SD = 1.12; Range = 4.25-6.25), but results varied across participants, with Child 3 demonstrating more frequent behavior concerns over time (Baseline: M = 5.44; SD = 2.74; Range = 2-9; Intervention: M = 6.25; SD = 2.92; Range = 1-10). On average, children were rated as exhibiting problem behavior “sometimes” during the baseline and intervention phases (Vagias, 2006). Statistical analysis confirmed that there was not a significant change in behavior concerns from baseline to intervention. The Wampold and Worsham (1986) randomization test was not significant for average behavior concern scores (p = .533), indicating that the DECA-P2 self-administered parent training intervention did not have a significant influence on children’s problem behaviors. Tau-U effect size calculations for behavior concern DBR scores were non- significant for all three participants (Child 1: Tau-U = -.31, p = .333; Child 2: Tau-U = -.46, p = .133; Child 3: Tau-U = .35, p = .229; see Table 12). Supplemental measures. DECA-P2 BC scale scores, on average, decreased from baseline (M = 68.11; SD = 6.17; Range = 61.00-72.00) to intervention (M = 58.89; SD = 4.81; Range = 53.33-61.67); however, individual mean changes were inconsistent across participants. 94 Figure 7. Parent DBR Ratings – Behavior Concerns g n i t a R R B D g n i t a R R B D g n i t a R R B D 10 8 6 4 2 0 10 9 8 7 6 5 4 3 2 1 0 10 8 6 4 2 0 Baseline Intervention Follow-Up Child 1 0 1 2 3 4 5 6 7 9 8 10 Rating Number 11 12 13 14 15 16 17 18 Child 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Child 3 0 1 2 3 4 5 6 7 8 9 10 Rating Number 11 12 13 14 15 16 17 18 Note. Behavior Concerns scores reflect the rating for the fourth behavior on the DBR form. In particular, Child 1’s mean BC scale score increased slightly from baseline (M = 61.00; SD = 4.58; Range = 56-65) to intervention (M = 61.67; SD = 3.21; Range = 58-64). Tau-U effect size calculations for BC scale scores were non-significant for all three participants (Child 1: Tau-U = .11, p = .827; Child 2: Tau-U = -.56; p = .275; Child 3: Tau-U = -.67, p = .190; see Table 12). RCI calculation only indicated a reliable decrease in average BC scale scores for Child 2 (RCI = 95 2.85). DECA-P2 BC scale scores were in the Area of Need range for all three participants at baseline, and only one participant’s average score (Child 2) decreased into the Typical range during the intervention phase. On average, mean scores for the CII – Child Noncompliant/Aggressive scale (i.e., direct behavior observation) decreased from baseline (M = 1.59; SD = .32; Range = 1.33-1.95) to intervention (M = 1.39; SD = .15; Range = 1.28-1.56), though Child 3’s mean score did not change over time (Baseline/Intervention: M = 1.33; SD = .17; Range = 1.17-1.50). Average CII – Child Noncompliant/Aggressive scale scores in both phases fell between Did Not Occur and 1-3 Examples, which indicates that children demonstrated some problem behaviors a couple times and did not demonstrate others at all. Tau-U effect size calculations for the CII – Child Noncompliant/Aggressive scale scores were non-significant for all three participants (Child 1: Tau-U = -.56, p = .275; Child 2: Tau-U = -.44, p = .383; Child 3: Tau-U = 0, p = 1.00; see Table 12). Follow-up. Follow-up parent DBR ratings of behavior concerns (four weeks post- intervention) showed a decrease in score (i.e., improvement) for Child 2 and Child 3, and an increase in score for Child 1. Child 2’s follow-up DBR rating of behavior concerns (DBR score = 3) represented a score decrease of 1.38 compared to the level in the intervention phase (M = 4.38), and Child 3’s follow-up rating (DBR score = 2) represented a score decrease of 4.25 compared to the level in the intervention phase (M = 6.25). Child 2 and Child 3’s DBR scores at follow-up indicate that behavior concerns “rarely” occurred (Vagias, 2006) in the four weeks following completion of the intervention. Child 1’s follow-up DBR rating of behavior concerns (DBR score = 8) represented a score increase of 3.75 over the level in the intervention phase (M 96 = 4.25). Child 1’s follow-up DBR rating indicates that he displayed behavior concerns “often” (Vagias, 2006) during the four weeks following completion of the intervention. Research Question 2 Is the DECA-P2 self-administered parent training guide an effective intervention for building positive parenting practices and reducing negative parenting practices for three parents of Head Start preschoolers from low-income backgrounds? Positive parenting practices. Primary measure. Visual analysis of GAS ratings of positive parenting practices, the primary multiple-baseline measure, did not find evidence of a replicated intervention effect, despite all three parents reporting more frequent use of positive parenting practices over time (see Figure 8). On average, parents provided higher GAS ratings of positive parenting practices during the intervention phase (M = .75; SD = .78; Range = .13-1.63) as compared to the baseline phase (M = -.18; SD = 1.22; Range = -1.43-1.00). Average GAS ratings of positive parenting practices revealed that parents used these practices “as much as expected” during the baseline phase, and “somewhat more than expected” during the intervention phase. The Wampold and Worsham (1986) randomization test indicated that this change was likely due to the intervention (p < .05); however, Tau-U effect size calculation only revealed a significant effect magnitude for one of the three participants (Parent 2; Tau-U = .70, p < .05). Statistical analysis suggested that while the effect was non-random, ratings did not change enough to be significant for all three participants (see Table 13). Supplemental measures. Average PBI – Supportive/Engaged scale scores increased from baseline (M = 34.67; SD = 2.31; Range = 32-36) to intervention (M = 44.33; SD = 6.43; Range = 37-49) for Parent 2, decreased from baseline (M = 48.00; SD = 2.00; Range = 46-50) to 97 Figure 8. GAS Ratings – Positive Parenting Practices g n i t a R S A G g n i t a R S A G g n i t a R S A G 2 1 0 -1 -2 2 1 0 -1 -2 2 1 0 -1 -2 Baseline Intervention Follow-Up Parent 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Parent 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Parent 3 0 1 2 3 4 5 6 7 9 8 10 Rating Number 11 12 13 14 15 16 17 18 98 Means and Effect Sizes for Measures of Positive and Negative Parenting Practices Measure/Child Table 13. Primary Measure: GAS Positive Parenting Practice Parent 1 Parent 2 Parent 3 Average Negative Parenting Practice Parent 1 Parent 2 Parent 3 Average Supplemental Measure: PBI Supportive/Engaged Parent 1 Parent 2 Parent 3 Average Hostile/Coercive Parent 1 Parent 2 Parent 3 Average Baseline M (SD; Range) 1.00 (0; 1.00-1.00) -1.43 (.79; -2-0) -.11 (1.27; -2-2) -.18 (1.22; -1.43-1.00) .17 (.98; -1-1) .43 (1.13; -1-2) .78 (1.72; -2-2) .46 (.31; .17-.78) Intervention M (SD; Range) 1.63 (.52; 1-2) .13 (1.25; -2-2) .50 (1.20; -2-2) .75 (.78; .13-1.63) .88 (.35; 0-1) -.38 (1.51; -2-2) 1.63 (.52; 1-2) .71 (1.01; -.38-1.63) 48.00 (2.00; 46-50) 34.67 (2.31; 32-36) 39.33 (3.21; 37-43) 40.67 (6.77; 34.67-48.00) 15.33 (1.53; 14-17) 29.00 (10.39; 17-35) 35.33 (4.51; 31-40) 26.56 (10.22; 15.33-35.33) 47.67 (.58; 47-48) 44.33 (6.43; 37-49) 39.33 (4.04; 37-44) 43.78 (4.19; 39.33-47.67) 17.00 (1.00; 16-18) 21.33 (11.02; 14-34) 23.33 (10.02; 13-33) 20.56 (3.24; 17.00-23.33) 99 M Change Tau-U +.63 +1.56 +.61 +.93 +.71 -.81 +.85 +.25 -.33 +9.66 0 +3.11 +1.67 -7.67 -12.00 -6.00 .63 .70 .31 .54 .42 -.34 .19 .09 -.44+ 1.00 -.11 .15 .67 -.78 -.78 -.30 p .053 .024* .290 .003** .197 .272 .501 .619 .383 .049* .827 .614 .190 .127 .127 .314 Table 13 (cont’d) Supplemental Measure: CII Nurturing/Supportive Parent 1 Parent 2 Parent 3 Average Harsh/Critical Parent 1 Parent 2 Parent 3 Average *p < .05; **p < .01 +Corrected for baseline trend 1.87 (.04; 1.85-1.92) 2.37 (.15; 2.25-2.54) 2.37 (.39; 1.92-2.62) 2.20 (.29; 1.87-2.37) 1.20 (.12; 1.09-1.33) 1.22 (.24; 1.08-1.50) 1.11 (.05; 1.08-1.17) 1.18 (.06; 1.11-1.22) 2.03 (.16; 1.85-2.15) 2.30 (.26; 2.08-2.58) 2.46 (.20; 2.23-2.58) 2.26 (.22; 2.03-2.46) 1.20 (.26; 1.00-1.50) 1.22 (.31; 1.00-1.58) 1.34 (.14; 1.17-1.42) 1.25 (.07; 1.20-1.34) +.16 -.07 +.09 +.06 0 0 +.23 +.08 .56 -.33 -.11 .04 -.11 .11 .89 .30 .275 .513 .827 .900 .827 .827 .081 .314 100 intervention (M = 47.67; SD = .58; Range = 47-48) for Parent 1, and did not change from baseline (M = 39.33; SD = 3.21; Range = 37-43) to intervention (M = 39.33; SD = 4.04; Range = 37-44) for Parent 3. Tau-U effect size calculation for PBI – Supportive/Engaged scale scores only revealed a significant effect for one of the three participants (Parent 2; Tau-U = 1.00, p < .05). On average, CII – Nurturing/Supportive scale scores (i.e., direct behavior observation) increased slightly from baseline (M = 2.20; SD = .29; Range = 1.87-2.37) to intervention (M = 2.26; SD = .22; Range = 2.03-2.46), though Parent 2’s score decreased slightly over time (Baseline: M = 2.37; SD = .15; Range = 2.25-2.54; Intervention: M = 2.30; SD = .26; Range = 2.08-2.58). Average CII – Nurturing/Supportive scale scores fell between 1-3 Examples and 4+ Examples, which indicates that parents demonstrated nurturing/supportive behaviors on a frequent basis across both phases. Tau-U effect size calculations for the CII – Nurturing/Supportive scale were not significant for any of the three participants (Parent 1: Tau-U = .56, p = .275; Parent 2: Tau-U = -.33, p = .513; Parent 3: Tau-U = -.11, p = .827; see Table 13). Follow-up. Follow-up GAS ratings of positive parenting practices (four weeks post- intervention) showed an increase in score (i.e., improvement) for Parent 2 and Parent 3, and a decline in score for Parent 1. Parent 2’s follow-up GAS rating of the targeted positive parenting practice (GAS rating = 2) represented a score increase of 1.33 compared to the level in the intervention phase (M = .67), and Parent 3’s follow-up rating (GAS rating = 2) represented a score increase of 1.14 compared to the level in the intervention phase (M = .86). Parent 1’s follow-up GAS rating of the targeted positive parenting practice (GAS rating = 1) represented a score decrease of .63 over the level in the intervention phase (M = 1.63). Parent 2 and 3’s follow- up GAS ratings indicated that they were using the positive parenting practice “much more than 101 expected,” and Parent 1’s follow-up rating indicated that he was using the positive parenting practice “somewhat more than expected.” Negative parenting practices. Primary measure. Visual analysis of GAS ratings of negative parenting practices, the primary multiple-baseline measure, did not find evidence of an intervention effect for any of the three participants (see Figure 9). In fact, two of the three parents reported more frequent use of negative parenting practices over time, with average GAS ratings of negative parenting practices increasing from .46 (SD = .31; Range = .17-.78) during the baseline phase to .71 (SD = 1.01; Range = -.38-1.63) during the intervention phase. Average GAS ratings of negative parenting practices revealed that parents used these practices “as much as expected” during the baseline phase and “somewhat more than expected” during the intervention phase. Statistical analysis revealed that there was not a significant change from baseline to intervention [Wampold and Worsham (1986) randomization test: p = .283; Tau-U weighted average = .09, p = .619; see Table 13]. Supplemental measures. On average, PBI – Hostile/Coercive scale scores decreased from baseline (M = 26.56; SD = 10.22; Range = 15.33-35.33) to intervention (M = 20.56; SD = 3.24; Range = 17.00-23.33), thought Parent 1’s score increased slightly over time (Baseline: M = 15.33; SD = 1.53; Range = 14-17; Intervention: M = 17.00; SD = 1.00; Range = 16-18). Tau-U effect size calculations for the PBI – Hostile/Coercive scale were non-significant for all three participants (Parent 1: Tau-U = .67, p = .190; Parent 2: Tau-U = -.78, p = .127; Parent 3: Tau-U = -.78, p = .127; see Table 13). Average CII – Harsh/Critical scale scores (i.e., direct behavior observation) did not change over time for Parent 1 or Parent 2 and increased slightly for Parent 3 (Baseline: M = 1.11; 102 Figure 9. GAS Ratings – Negative Parenting Practices Baseline Intervention Follow-Up Parent 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Parent 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Parent 3 0 1 2 3 4 5 6 7 8 9 10 Rating Number 11 12 13 14 15 16 17 18 g n i t a R S A G g n i t a R S A G g n i t a R S A G 2 1 0 -1 -2 2 1 0 -1 -2 2 1 0 -1 -2 SD = .05; Range = 1.08-1.17; Intervention: M = 1.34; SD = .14; Range = 1.17-1.42). Average CII – Harsh/Critical scale scores fell between Did Not Occur and 1-3 Examples, which indicates that parents demonstrated some harsh/critical behaviors a couple times and did not demonstrate others at all. Tau-U effect size calculations for CII – Harsh/Critical scale scores were non- 103 significant for all three participants (Parent 1: Tau-U = -.11, p = .827; Parent 2: .11, p = .827; Parent 3: Tau-U = .89, p = .081; see Table 13). Follow-up. Follow-up GAS ratings of negative parenting practices (four weeks post- intervention) showed a decrease in score (i.e., improvement) for Parent 2 and Parent 3, and an increase in score for Parent 1. Parent 2’s follow-up GAS rating of the targeted negative parenting practice (GAS rating = -2) represented a score decrease of 2.60 compared to the level in the intervention phase (M = .60), and Parent 3’s follow-up rating (GAS rating = 0) represented a score decrease of 1.63 compared to the level in the intervention phase (M = 1.63). Parent 1’s follow-up GAS rating of the targeted negative parenting practice (GAS rating = 2) represented a score increase of 1.12 over the level in the intervention phase (M = .88). Follow-up GAS ratings indicated that Parent 2 was using the negative parenting practice “much less than expected,” Parent 3 was using the practice “as much as expected,” and Parent 1 was using the practice “much more than expected.” Research Question 3 Will the DECA-P2 self-administered parent training guide lead to a significant reduction in parenting stress from pre-test to post-test for three parents of Head Start preschoolers from low-income backgrounds? There was not a significant reduction in PSI-SF scores from pre-test to post-test for all three of the parents of Head Start preschoolers who participated in the present study. However, two of the parents did report a significant decrease in parenting stress over time (Parent 2: RCI = 4.11; Parent 3: RCI = 5.69; see Table 14). At pre-test, Parent 2’s ratings were in the clinically significant range (87th percentile), Parent 3’s ratings were in the borderline clinically significant range (83rd percentile), and Parent 1’s ratings were in the average range (69th percentile). At post- 104 test, all three parents’ ratings were in the average range (Parent 1: 67th percentile; Parent 2: 61st percentile; Parent 3: 35th percentile). Table 14. Changes in PSI-SF Scores from Pre-Test to Post-Test Post-Test T-Score Participant Pre-Test T-Score (Percentile) 54 (69th) 64 (87th) 62 (83rd) Parent 1 Parent 2 Parent 3 Average *Reliable change ( > 1.96) (Percentile) 53 (67th) 51 (61st) 44 (35th) RCI .32 4.11* 5.69* 3.37* 60.00 (SD = 5.29) 49.33 (SD = 4.73) Research Question 4 Will the DECA-P2 self-administered parent training guide be implemented as intended (i.e., treatment integrity) by three parents of Head Start preschoolers from low-income backgrounds? The three parents of Head Start preschoolers who participated in the study demonstrated high levels of treatment integrity (see Table 15). Scores of 80% or above reflect high treatment integrity (Perepletchikova & Kazdin, 2005), and average integrity scores across the eight weeks of the intervention were 80% or higher for all three participants (Parent 1: 86%; Parent 2: 80%; Parent 3: 93%). Specifically, Parent 1 reported high integrity (i.e., above 80%) for four of the eight weeks of the intervention, Parent 2 reported high integrity for six of the eight weeks, and Parent 3 reported high integrity for seven of the eight weeks. Overall treatment integrity (across participants and weeks) averaged 86%. 105 Table 15. Parent 1 Parent 2 Parent 3 Average Average Percent Completion of Intervention Activities Week 1 2 3 4 5 6 7 8 Average Note: Scores of 80% or higher are considered to reflect high integrity (Perepletchikova and Kazdin, 2005). 75% 86% 100% 100% 100% 75% 75% 75% 86% 88% 100% 88% 100% 71% 100% 100% 100% 93% 88% 100% 100% 88% 100% 88% 38% 38% 80% 83% 95% 96% 96% 91% 88% 71% 71% 86% Research Question 5 Is the DECA-P2 self-administered parent training guide an acceptable intervention option for three parents of Head Start preschoolers from low-income backgrounds? Two of the three parents found the DECA-P2 self-administered parent training guide to be highly acceptable. A score of 55 or higher on the TEQ-P acceptability scale indicates an intervention is highly acceptable (Kratochwill et al., 2003), and Parent 1 and Parent 3’s ratings exceeded this standard (see Table 16). Parent 2’s rating was slightly below 55 (Total Score = 52). Two of three parents “strongly agreed” that the DECA-P2 self-administered parent training guide was an acceptable intervention for building children’s social-emotional skills, with Parent 2 indicating she “agreed” with this statement. All parents “agreed” or “strongly agreed” that the parent training guide was a good way to build children’s social-emotional skills and that they liked the procedures used in the intervention. Parents provided slightly lower ratings (“slightly agree” to “agree”) in response to the statement, “My child’s social and emotional skills were lacking enough to warrant use of this intervention.” 106 Table 16. Answers on the TEQ-P Across Participants Statement 1. This was an acceptable intervention for building my child’s social and emotional skills. 2. Most parents would find this intervention appropriate for building children’s social and emotional skills. 3. This intervention was effective in building my child’s social and emotional skills. 4. I would suggest the use of this intervention to other parents. 5. My child’s social and emotional skills were lacking enough to warrant use of this intervention. 6. Most parents would find this intervention suitable for building social and emotional skills. 7. The intervention did not result in negative side effects for my child. 8. The intervention would be appropriate for a variety of children. 9. The intervention was a fair way to build my child’s social and emotional skills. 10. I liked the procedures used in the intervention. 11. The intervention was a good way to build my child’s social and emotional skills. Parent 1 Parent 2 Parent 3 Average 6 5 5 5 4 5 6 6 5 5 6 5 6 4 5 4 5 5 4 4 5 5 6 6 6 6 5 6 6 6 6 6 6 5.67 5.67 5.00 5.33 4.33 5.33 5.67 5.33 5.00 5.33 5.67 Total Score Note: Ratings range from 1 (Strongly Disagree) to 6 (Strongly Agree). Total scores of 55 or higher are considered to reflect an acceptable intervention. 58.33 65 52 58 Open-ended interview responses provided additional information about the acceptability of the program. Table 17 provides a summary of parent responses to each of the three interview questions. Interestingly, two of the three parents reported that home visits were one of the best 107 Table 17. Parent Responses to Exit Interview Questions Question 1. What were the best aspects of the program? Responses “The whole thing! Particularly liked the home visit aspect of the program” (Parent 1) “Helpful reminder of parenting strategies” (Parent 2) “The theory behind each strategy was interesting to read about and was helpful for conceptualization/brainstorming” (Parent 2) “Home visits were a source of excitement for the child” (Parent 2) “Doing the program made ideas about parenting more fresh in my mind, which led to better parenting practices” (Parent 3) “Allowed me to slow down and take time to do things with the kids” (Parent 3) “Questions were asked about the adult’s life as well as the child’s life” (Parent 3) “Nothing! Everything was helpful and enjoyable to work on with the kids” (Parent 1) “It was sometimes difficult to find time to sit down and read” (Parent 2) “Questionnaires were organized in a confusing way or had repetitive questions” (Parent 2) “Sometimes repetitive, particularly in the workbook activities” (Parent 3) “No suggestions at the moment” (Parent 1) “An electronic format for the book would have been helpful” (Parent 2) “More clear exchange of contact information at the beginning would have been helpful” (Parent 3) 2. What aspects did you not like or not find helpful? 3. How could the program have been improved to help you more? aspects of the program, even though the primary purpose of the home visits was to collect data. Additionally, two of the three parents liked that the program offered reminders of helpful parenting practices and allowed them to think more deeply about the strategies [e.g., “the theory behind each strategy” (Parent 2), “questions were asked about the adult’s life” (Parent 3)]. Two 108 of the three parents found that certain aspects of the program were repetitive (e.g., workbook, questionnaires). Regarding suggestions for improvement, Parent 2 noted that she would have preferred an electronic format and Parent 3 requested a clearer exchange of contact information upon enrollment in the study. Individual Improvements Despite the fact that there were not any replicated treatment effects documented in the present study, parents found the intervention to be moderately-to-highly acceptable. Therefore, it is important to review individual improvements for each parent/child duo to further understand acceptability ratings. Two of the three parent/child duos (Parent/Child 2, Parent/Child 3) showed significant improvements following participation in the DECA-P2 self-administered parent training intervention. Parent 2 experienced a reliable decrease in parenting stress (PSI-SF RCI = 4.11; see Table 14) and a significant increase in her use of positive parenting practices (GAS – Positive Tau-U = .70, p < .05; PBI Supportive/Engaged scale Tau-U = 1.00, p < .05; see Table 13). Child 2 experienced a reliable increase in social-emotional competence and a reliable decrease in behavior concerns, as measured by the DECA-P2 rating scale (TPF scale RCI = 4.25; BC scale RCI = 2.85; see Table 12). Parent 3 also experienced a reliable decrease in parenting stress (PSI-SF RCI = 5.69; see Table 14), and Child 3 also demonstrated a reliable increase in social-emotional competence (DECA-P2 TPF scale RCI = 3.00; see Table 12). There were not any significant improvements in child behavior or parenting practices for Parent/Child 1. 109 CHAPTER 5 Discussion Children living in poverty experience social-emotional challenges at a higher rate (33%; Webster-Stratton & Hammond, 1998) than children within the general population (10-15%; McCabe & Altamura, 2011), yet there are substantial barriers that prevent them from receiving needed treatment (Elgar & McGrath, 2003). Given that young children’s social-emotional skills are associated with positive outcomes in the short-term (e.g., school success; Webster-Stratton & Reid, 2004) and long-term (e.g., high school and college graduation, stable employment; Jones et al., 2015), it is essential to identify EBIs that are easily disseminated and implemented within low-income populations. Self-administered interventions are suggested as one disruptive innovation for improving access, scalability, replicability, and sustainability of mental health treatments (Rotheram-Borus et al., 2012), especially for families living in poverty. Self- administered interventions offer a number of advantages, including accessibility, low costs, flexible implementation, reduced stigma, and the potential to be disseminated by a variety of professionals (e.g., educators, psychologists, physicians; Elgar & McGrath, 2003). Parent training programs, an EBI for building social-emotional skills (e.g., Lundahl et al., 2006), have been developed and studied in self-administered formats, yet some barriers still remain with the current self-administered program options. For example, while the Triple P program allows for flexible implementation within a multi-tier model, materials are expensive (between $1,450 and $1,905 depending on the level) and considerable training is required (Armstrong et al., 2014). The Incredible Years parent training program also requires costly materials and training, and can be relatively lengthy (i.e., 10-12 weeks; Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992). Although these programs have demonstrated effectiveness in improving parent and child 110 behavior, the drawbacks may limit the availability of professionals to disseminate the programs and the opportunity for families to participate. The DECA-P2 family guide (Mackrain & Cairone, 2013) has the potential to overcome some of these weaknesses due to its extremely low cost, ease of dissemination, and shorter time commitment. A pilot study indicated that the family guide, when implemented in a self- administered workbook format, was effective in improving Head Start children’s (N = 12) social- emotional competence and reducing behavior concerns (Thomson & Carlson, 2016). Therefore, following Sheridan’s (2014) intervention research trajectory, the present study aimed to investigate the DECA-P2 self-administered parent training program with increased methodological rigor within a small sample (i.e., single-case design). This increased rigor, which ensured internal validity (i.e., by controlling for confounding variables; Christ, 2007) and external validity (i.e., via replication; Onghena & Edgington, 2005), did not provide evidence for the effectiveness of the DECA-P2 parent training intervention for three parents of Head Start preschoolers. Notably, though, positive outcomes were documented related to implementation (i.e., integrity, acceptability). Despite the non-significant findings, single-case design methodology offers several strengths, and close examination of the results has the potential to contribute to future science and practice. While the pilot study documented significant increases in children’s social- emotional competence and decreases in behavior concerns following parents’ participation in the intervention, the pre-test/post-test design of the study did not allow researchers to draw conclusions about a causal relationship (Thomson & Carlson, 2016). Therefore, the present study aimed to explore a causal relationship using more rigorous methodology (i.e., multiple-baseline design); however, the inconsistent results across participants did not meet the standards required 111 to show evidence of a causal relation (i.e., replication across participants; Kratochwill et al., 2013). One of the advantages of single-case design is that it allows for a thorough review of both responders and non-responders, as opposed to simply viewing non-responders as “outliers” as is done in traditional group designs. Examination of non-responders is important because it allows for the development of iterative hypotheses about outcome variation, which, in turn, can inform study replications and theory/intervention development (Sheridan, 2014). In the present study, examination of both significant and non-significant results has led to hypotheses about the context (i.e., target population) and intervention structure that can inform future work. Specific findings and hypotheses are summarized below and discussed in the context of previous research. Child Social-Emotional Competence and Behavior Concerns Visual and statistical analysis revealed that there was not a significant improvement in children’s social-emotional competence or a reduction in behavior concerns following parents’ participation in the DECA-P2 self-administered parent training intervention. This finding is surprising, given that other self-administered parent training programs (e.g., Incredible Years, Triple P) have been successful in improving children’s behavior (i.e., increasing social-emotional competence, decreasing behavior concerns; de Graaf et al., 2008; Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992). Potential reasons for this discrepancy in findings may include: 1) minimal room for improvement in social-emotional competence, and/or 2) lack of change in parenting practices. Minimal room for improvement in social-emotional competence. One potential reason that a significant improvement in social-emotional competence was not documented in the present study is that children had moderate levels of functioning at baseline. When participants 112 are already performing well on a measure at baseline, the measure will not be able to capture much change (i.e., ceiling effects; Coster, 2013). In the present study, children’s mean DBR ratings of social-emotional competence ranged from 5.37 to 7.50 (out of 10) during the baseline phase and two children’s DECA-P2 TPF scale T-scores were in the Typical range at the initial rating, leaving little room for growth. Parents seemed to agree that children’s social-emotional skills were not necessarily lacking enough to warrant the use of the present intervention, with the two parents whose children scored in the Average range on the DECA-P2 TPF scale at baseline (Child 1, Child 2) only “slightly agreeing” with this statement on the TEQ-P. While all children met LeBuffe and Naglieri’s (2012) criteria for at-risk social-emotional development (based on DECA-P2 TPF scale T-scores), two of the three children (Child 1, Child 2) only demonstrated need in one area of social-emotional competence (i.e., self-regulation), rather than more global deficits in social-emotional competence. However, supplemental data analysis found that even when effect size calculations were isolated to just self-regulation scores (i.e., DBR-Self- Regulation ratings, DECA-P2 Self-Regulation subscale scores), a replicated intervention effect was not documented. Therefore, it is possible that the comprehensive nature of the program on social-emotional competence did not align with the more targeted needs (i.e., self-regulation deficits) of the participants in the study. In the pilot study of the DECA-P2 self-administered parent training intervention (Thomson & Carlson, 2016), the same inclusion criteria were used; however, the pilot study sample reflected a group of children with an overall greater level of need. For example, in the pilot study, the mean pre-test DECA-P2 TPF scale T-score was 36.75 (SD = 6.73; Range = 28- 52; Thomson & Carlson, 2016), and in the present study, the mean initial DECA-P2 TPF scale T-score was 42.67 (SD = 11.37; Range = 30-52). Therefore, children in the present study started, 113 on average, with more typical social-emotional functioning and had less room to show improvement. Previous studies of self-administered parent training programs (e.g., Incredible Years, Triple P) also included children with greater overall levels of need, which allowed more room to see improvement. All studies used the Eyberg Child Behavior Inventory (ECBI; Eyberg & Robinson, 1983) to identify children with behavior problems in the clinical range. Studies of the Incredible Years self-administered parent training program (Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992) included samples of children with mean ECBI Total Problems scores above 20. This far exceeds the cutoff score of 11 (Robinson, Eyberg, & Ross, 1980), which indicates that these children scored well into the clinical range. Studies of the Triple P self-administered parent training program (e.g., Connell, Sanders, & Markie-Dadds, 1997; Markie-Dadds & Sanders, 2006) used the Intensity scale of the ECBI to determine whether children had clinical-level problem behaviors (i.e., scores ³ 127; Robinson et al., 1980). In these studies, children’s mean ECBI Intensity scores ranged from 145.75 to 160.20, which again far exceeds the cutoff score (Connell et al., 1997; Markie-Dadds & Sanders, 2006). The inclusion criteria in the present study focused on ratings of social-emotional competence rather than behavior concerns, given the primary focus of the DECA-P2 family guide (Mackrain & Cairone, 2013) and the criteria for being “at-risk” based on the DECA-P2 rating scale (LeBuffe & Naglieri, 2012). However, children’s pre-test scores in these studies of other self-administered parent training interventions clearly indicate more significant, comprehensive behavioral needs. While it may have been difficult to see statistically significant changes in children’s social-emotional competence due to their relatively high level of functioning at baseline, there was indication of clinical significance for two of the three children in the present study. In 114 particular, RCIs for DECA-P2 TPF scale T-scores were significant for Child 2 (RCI = 4.25) and Child 3 (RCI = 3.00). Additionally, other studies have used scores within the normal or non- clinical range of functioning to document clinical significance (Webster-Stratton, 1992), and in the present study, all mean DECA-P2 TPF scale T-scores were in the Typical range of functioning during the intervention phase (Range = 41.67-54.00). Additionally, Child 2 had a clinically significant decrease in mean DECA-P2 BC scale T-scores (RCI = 2.85; Typical range). In contrast, supplemental analysis revealed that Child 4 experienced no change in social- emotional competence (i.e., DECA-P2 TPF scale; Pre-test: T-score = 28; Post-test: T-score = 29; RCI = .25) and a reliable (i.e., clinically significant) increase in behavior concerns (i.e., DECA- P2 BC scale) over time (Pre-test: T-score = 50; Post-test: T-score = 68; RCI = 2.85). This indicates that Child 4’s behaviors became more problematic over an eight-week period of time without participation in the intervention. However, without demonstration of statistical significance, it is unclear whether these results occurred by chance (Ranganathan, Pramesh, & Buyse, 2015). Lack of change in parenting practices. Another potential reason that changes in children’s behaviors were not observed in the present study (contrary to hypotheses) is that there were not significant changes in parenting practices (i.e., the proposed mechanism of change; Vick Whittaker et al., 2011). The purpose of parent training programs is “to proximally modify parents’ childrearing practices and attitudes and, if successful, distally modify children’s behavior” (Lundahl et al., 2006, p. 88). As a result, changes in parenting practices are an essential prerequisite to changes in child behavior. In the present study, significant changes in parenting practices were not observed, which is likely a primary reason that changes in child 115 behavior were not documented either. Parent-level variables will be discussed further in the following section. Parenting Practices Visual and statistical analysis revealed that there was not an improvement in positive parenting practices or a reduction in negative parenting practices following parents’ participation in the DECA-P2 self-administered parent training intervention. These findings contrast with previous research, which has consistently documented improvements in parenting practices with the use of self-administered parent training programs (Lundahl et al., 2006; Reyno & McGrath, 2006). For example, parents who participated in the Incredible Years self-administered parent training program were observed to have increased positive affect, use more frequent praise, and reduce their use of criticisms and no-opportunity commands, as compared with a waitlist control group (Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992). Additionally, parents who participated in the Triple P self-administered parent training intervention reported significantly less laxness, verbosity, and overreactivity (as measured by the Parenting Scale; Arnold, O’Leary, Wolff, & Acker, 1993) compared to a control group (Connell et al., 1997; Markie-Dadds & Sanders, 2006). Potential reasons for this discrepancy in findings may include: 1) minimal room for improvement in positive parenting practices, 2) factors associated with living in poverty, and/or 3) characteristics of the intervention. Minimal room for improvement in positive parenting practices. One potential reason that a significant improvement in positive parenting practices was not documented in the present study is that parents reported relatively frequent use of positive parenting practices at baseline (i.e., ceiling effects; Coster, 2013). For example, baseline average scores on the PBI – Supportive/Engaged scale (M = 41.00; maximum score = 50) and CII – Nurturing/Supportive 116 scale (M = 2.20; maximum score = 3) were quite high and, thus, there was little room for growth. Inclusion criteria required parents to report infrequent use of one or more positive parenting practices in an attempt to identify an “at-risk” sample (rather than difficulties at a clinical level), which aligns well with the intended purpose of the DECA-P2 program (LeBuffe & Naglieri, 2012). However, other researchers investigating programs that target a similar population (e.g., primary care Triple P) have taken a different approach by including families with problems at a clinical level to ensure stable baseline data and to increase the chances of seeing a significant effect following treatment (Boyle et al., 2010). It is important to note that statistical analyses provided evidence of some improvement in positive parenting practices following participation in the DECA-P2 self-administered intervention, but the magnitude of the change was not significant. For instance, the Wampold and Worsham (1986) randomization test suggested that there was a significant increase in positive parenting practices (rated using the GAS method) from baseline to intervention (i.e., greater than what would be expected due to chance). However, individual effect size calculations only revealed a strong, significant effect magnitude for one of the three participants (Parent 2; Tau-U = .70, p < 05). This means that the DECA-P2 self-administered parent training intervention likely resulted in some increases in positive parenting practices, though the increases were not large enough to be significant or meaningful for all three participants. This distinction between the non-randomness of the effect and the magnitude of the effect is important, given that p values obtained from significance tests do not necessarily reflect practical or clinical significance (Onghena & Edgington, 2005). Here, it is evident that changes in positive parenting practices were minimal, suggesting that they likely did not have a meaningful impact on parents’ functioning throughout their day-to-day lives. 117 In comparison to previous studies of other self-administered interventions (e.g., Incredible Years), parents’ baseline scores in the present study seem to reflect less room for improvement in positive parenting practices. For example, mothers in the Webster-Stratton and colleagues (1988) and Webster-Stratton (1990, 1992) studies were only observed to give, on average, between 3 and 7 praise statements during a 30-minute pre-test observation. This was significantly lower than the minimally-recommended 2:1 positive-to-negative statement ratio (Zemp, Merrilees, & Bodenmann, 2014), as the mothers were also observed to give between 15 and 17 criticisms during the same 30-minute observation (Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992). In the present study, parents were observed to use more praise statements than criticisms during baseline observations, as the average baseline CII rating for the item, “Parent is a positive and reinforcing parent” was a 2.11 (out of 3; between 1-3 examples and 4+ examples) and a 1.78 for the item, “The parent showed disapproval or criticized child” (out of 3; between Did not occur and 1-3 examples). Additionally, parents’ average baseline rating on the PBI item, “I thank or praise my child” was a 4.33 (out of 5; between Quite a bit true and Very true). Parents in the Webster-Stratton studies also exhibited mostly neutral affect at baseline (Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992), while parents in the present study were observed to have overall positive affect (i.e., mean CII rating of 2.33 out of 3 for the item, “Parent seemed to enjoy parenting,” which was primarily based on facial expressions, tone of voice, and nonverbal behaviors). Although use of positive parenting practices was not a part of the inclusion criteria for the Webster-Stratton studies (Webster- Stratton et al., 1988; Webster-Stratton, 1990, 1992), it is apparent that these samples of parents demonstrated more maladaptive parenting, which likely allowed for a greater chance to see significant improvements following the intervention. 118 Unfortunately, some research suggests that self-selection to participate in parent training programs likely reflects participants with stronger parenting skills, which may make it difficult to recruit participants with more maladaptive parenting in future research. For example, Costigan and Cox (2001) found that fathers who elected to participate in a longitudinal family research project were more highly educated and used more positive parenting practices (as assessed via in-home observations) than those who chose not to participate. While data is not available about non-participants in the present study, the three parents who participated in the intervention appeared to be already using many positive parenting practices upon enrollment into the study (as evidenced by high scores on the PBI – Supportive/Engaged scale and the CII – Nurturing/Supportive scale). Given the difficulties with recruiting and retaining low-income families within current and previous research (Gross et al., 2001), it will be imperative to explore ways to motivate more parents to participate in parent training programs, especially those with lower parenting skills. Studies have investigated the use of incentives to improve motivation [e.g., childcare discounts (Gross et al., 2011), monetary rewards (Irvine, Biglan, Smolkowski, Metzler, & Ary, 1999; Orrell-Valente, Pinderhughes, Valente, Laird, & Conduct Problems Research Groups, 1999)], though they have not had a significant effect on parent attendance, motivation, or engagement. This may be due to the fact that non-participation in parent training programs typically occurs as a result of logistical issues (e.g., scheduling conflicts, time demands; Heinrichs et al., 2005), which cannot be solved with incentives. While information is not known about why most parents who received the present study’s flyer did not respond with interest in participating in the intervention, it was evident that logistical issues (i.e., scheduling conflicts) were the reason that Parent 4 had to drop out of the study during the baseline phase. It is also 119 important to note that privacy issues (i.e., not wanting to be video-recorded) have been reported as barriers particularly for families living in poverty (Spoth, Redmond, Hockaday, & Shin, 1996), so this could have minimized parents’ interest in participating in the present intervention upon receiving the flyer. Future efforts should focus on ways to differentiate recruitment efforts within low-income communities to ensure higher levels of participation in self-administered parent training programs. Factors associated with living in poverty. Another potential reason for the discrepancy in findings between the present study and previous studies of self-administered parent training programs relates to the socioeconomic status of the participants. Previous studies that documented improvements in parenting practices (and child behavior) following parents’ participation in self-administered parent training programs did not include only participants living in poverty (e.g., Connell et al., 1997; Webster-Stratton et al., 1988). Similarly, while the pilot study of the DECA-P2 self-administered parent training intervention did not assess parenting practices, it is important to note that participants had higher levels of educational attainment than the national Head Start population, indicating higher socioeconomic status (household income data was not collected as a part of the study; Thomson & Carlson, 2016). Given that low family income has been found to strongly predict treatment outcomes for parent training programs (Lundahl et al., 2006; Reyno & McGrath, 2006), it is important to consider how this demographic factor might have influenced outcomes in the present study. Generally speaking, low socioeconomic status “is believed to undermine efficacy of parent training interventions by disrupting parent training processes and implementation of recommendations” (Lundahl et al., 2006, p. 87). In other words, parents living in poverty might have difficulty engaging in the learning process and consistently using strategies due to factors 120 associated with low-income status (e.g., family instability, insufficient time; Atkins et al., 2006; Qi & Kaiser, 2003). While self-administered parent training programs are able to thwart some of the challenges that low-income families face (such as difficulties with transportation and lack of health insurance), other challenges may still be present. Lundahl and colleagues (2006) found that family adversity (i.e., low socioeconomic status) “significantly undermined positive changes in parental behavior” (p. 96). If parents are unable to implement strategies consistently in the home environment due to cumulative risk factors associated with living in poverty, it is possible that they would not see significant outcomes or would need a longer period of implementation to see intended benefits. Although parents in the present study reported high levels of integrity to program components (86%), it is important to remember that data was self-reported and, therefore, may reflect a higher estimate than what is accurate. Characteristics of the intervention. A third potential reason for the discrepancy between the findings in the present study and previous studies of self-administered parent training programs relates to differences in intervention characteristics (e.g., components, length). While the benefits of parent training in general have been well-documented (Reid et al., 2004), outcomes can vary based on the specific components included within programs. A meta-analysis conducted by Wyatt Kaminski, Valle, Filene, and Boyle (2008) highlighted three components of parent training programs that result in greater improvements in parenting behavior: 1) positive interactions with children, 2) emotional communication, and 3) practicing with their own child. While the DECA-P2 family guide includes a focus on both positive interactions with children and emotional communication (Mackrain & Cairone, 2013), it does not offer opportunities for rehearsal or role-play (with feedback from professionals), given the self-administered nature of the program. The DECA-P2 self-administered parent training program instructs parents to 121 practice the three focus strategies each week as “homework,” but parents do not have the opportunity to troubleshoot any challenges or receive feedback from professionals about their performance, which may impact their accuracy and/or consistency of strategy use. Unfortunately, adding a feedback/troubleshooting component to the current intervention could potentially undermine efforts to reduce barriers to treatment by requiring scheduled face- to-face sessions and availability of qualified personnel. Nonetheless, other studies of self- administered parent training programs have attempted to enhance the interventions by including scheduled phone consultation with a therapist (Markie-Dadds & Sanders, 2006) and the option to call the therapist anytime throughout the intervention period (Webster-Stratton, 1990). These enhancements demonstrated some added benefits, such as lower levels of disruptive/deviant child behavior, as compared with children whose parents participated in the regular self- administered program (Markie-Dadds & Sanders, 2006; Webster-Stratton, 1990). Therefore, these enhancements could potentially be options for boosting the effectiveness of the DECA-P2 self-administered parent training intervention. Additionally, telehealth is another promising strategy that could allow for the addition of a feedback/troubleshooting component without compromising the benefits of a self-administered program; however, there may be logistical and financial barriers to this approach (e.g., limited Internet access, increased costs). Early research has documented success with a telehealth parent training program (with coaching) for parents of children with autism spectrum disorder (ASD; Wainer & Ingersoll, 2015). Future research should continue to explore this area and its feasibility within applied settings and with hard-to-reach populations (e.g., families living in poverty). Another intervention characteristic that may have influenced the outcomes in the present study is the intervention length. Parenting practices are said to be especially malleable while 122 children are young (LeBuffe & Naglieri, 2012; Sitnick et al., 2015) and for this reason, it was anticipated that changes in parenting practices would be observed over the eight-week intervention period. However, other self-administered parent training interventions (e.g., Incredible Years, Triple P) were between 10 and 12 weeks in length (Connell et al., 1997; Markie-Dadds & Sanders, 2006; Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992), and these extra few weeks may have allowed the time necessary to see significant improvements. For instance, follow-up GAS ratings (i.e., four weeks post-intervention, which equates to the 12- week mark) showed continued improvement in use of positive parenting practices for two of the three parents in the present study (Parent 2, Parent 3). These parents also rated their children as showing continued improvement in social-emotional competence and declines in behavior concerns (based on DBR ratings) at follow-up. Given that repeated and consistent use of positive parenting practices is necessary to promote child well-being (Mortensen & Mastergeorge, 2014), it could be that a few extra intervention weeks would allow time for changes to reach a significant level. Parenting Stress There was not a significant reduction in parenting stress for all three of the parents of Head Start preschoolers who participated in the present study. However, it is important to note that two parents (Parent 2, Parent 3) reported a reliable (i.e., clinically significant) decrease in PSI-SF scores from pre-test to post-test, and the third parent (Parent 1) did not begin the program with a high level of parenting stress. Notably, all three parents’ ratings on the PSI-SF were in the Average range at post-test. In contrast, supplemental analysis of Parent 4’s pre-test and post-test PSI-SF scores indicated the opposite, such that there was a reliable (i.e., clinically significant) increase in parenting stress over time (Pre-test: T-score = 48; 52nd percentile; Average range; 123 Post-test: T-score = 63; 85th percentile; Clinically Significant range). This provides some additional support for the benefits of program participation in reducing parenting stress, as Parent 4 did not follow-through with participation in the intervention, and her level of stress increased significantly over time. Previous research has documented decreases in parenting stress following participation in self-administered training programs (Tarver et al., 2014), though many of the programs previously associated with reductions in parenting stress (e.g., Incredible Years, Triple P) were more intensive and costly than the DECA-P2 self-administered parent training intervention. For example, these programs have longer intervention lengths, added components (i.e., videotapes, telephone consultation), expensive materials, and/or required practitioner training (Armstrong et al., 2014). Therefore, it is encouraging that two of the three parents who participated in this brief, simple, flexibly-implemented self-administered program also experienced such significant reductions in parenting stress, as this type of program has the potential to reach a wider range of families. Despite these promising findings, results should be interpreted with caution, given that parenting stress was explored using a pre-test/post-test design in the present study and significant changes were only observed for two of the three participants. There are limitations to a pre- test/post-test research design that pose threats to internal validity (e.g., maturation, regression to the mean, testing effects; Knapp, 2016), which may influence the reliability of the results. Additionally, the inclusion of only three participants limits the generalizability of the findings (i.e., external validity). Still, the positive changes in parenting stress in the present study highlight the need to explore this area further with self-administered parent training programs that do not require significant financial resources, costly and time-intensive training, or lengthy 124 time commitments, such as the DECA-P2 family guide. Cost-effective and time-efficient solutions for reducing parenting stress can be especially useful within low-income communities, where the need for mental health services is high (Deater-Deckard et al., 2012), but access and participation is low (Atkins et al., 2006; Gross et al., 2001). Integrity On average, parents who participated in the DECA-P2 self-administered parent training intervention reported 86% integrity to the intervention components (Parent 1: 86%; Parent 2: 80%; Parent 3: 93%), which is considered a high level of treatment integrity according to the standard suggested by Perepletchikova and Kazdin (2005; i.e., 80% or higher). This finding is similar to previous work, which has documented moderate (i.e., 71%) to high levels of treatment integrity with the use of the Incredible Years self-administered parent training program (Kratochwill et al., 2003; Ogg & Carlson, 2009) and high integrity (i.e., 95%) with the use of the DECA-P2 self-administered parent training intervention (Thomson & Carlson, 2016). Some research has indicated higher integrity to some components of self-administered parent training interventions (i.e., 98% of videotapes watched), but lower integrity to other components (i.e., 44% of the workbook completed, 70% of strategies implemented; Ogg & Carlson, 2009), while other research has documented high integrity to all intervention components (Thomson & Carlson, 2016). Parents in the present study had lower integrity to completing written questions in the workbook (71%), but higher integrity to completing the reading (92%) and implementing the focus strategies (90%). The overall high level of parent-reported integrity in the present study suggests that the intervention components (e.g., reading, written questions, use of strategies) are feasible for families to implement within the context of the home setting. 125 Other studies of self-administered parent training programs (e.g., Incredible Years, Triple P) did not formally assess parents’ integrity to the intervention process. For instance, Webster- Stratton and colleagues (1988) only reported therapist integrity for the group discussion treatment condition but did not report integrity for the self-administered treatment condition. Webster-Stratton (1990, 1992), Connell and colleagues (1997), and Markie-Dadds and Sanders (2006) did not report integrity at all. It is essential to measure integrity within intervention research to determine whether EBIs can be feasibly disseminated in applied contexts, especially for those populations that face many barriers to mental health treatment (e.g., families living in poverty; Gross et al., 2001). Therefore, while it is evident that the Incredible Years and Triple P self-administered parent training interventions are effective in improving parenting practices and child behavior (Connell et al., 1997; Markie-Dadds & Sanders, 2006; Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992), it is unclear whether diverse populations of parents could implement them with high levels of integrity. Although overall integrity was high in the present study, it is important to note a few concerns relating to integrity scores. First, there were two weeks for which average integrity was below 80% (Week 7: 71%; Week 8: 71%). These two weeks corresponded to the Attachment/Relationships strategies, which was the strongest area of social-emotional competence for two of the three children at baseline based on DECA-P2 Attachment/Relationships subscale T-scores [Child 1: T-score = 51 (Typical); Child 2: T-score = 66 (Strength)]. Therefore, it is possible that the parents of these children did not view this information as being relevant or helpful, given that their individual integrity scores were both below 80% for these two weeks of the intervention (Parent 1: 75%; Parent 2: 38%). Another concern was that there were some inaccuracies in Parent 1’s integrity ratings. Items related to 126 answering written questions in the workbook were cross-checked, and for two intervention weeks, Parent 1 inaccurately reported that he completed the written workbook questions, when, in fact, he did not. These errors raise some concerns for other potential oversights in his integrity ratings. Acceptability All three parents rated the DECA-P2 self-administered parent training intervention as acceptable. Two parents (Parent 1, Parent 3) rated it as being highly acceptable, and one parent (Parent 2) rated it as being moderately acceptable. This indicates that parents generally found the DECA-P2 family guide to be a fair, appropriate, and effective intervention for building their children’s social-emotional competence. These results mirror the high acceptability (mean TEQ- P acceptability scale score = 59.67) found in the pilot study of the DECA-P2 self-administered parent training intervention (Thomson & Carlson, 2016) and in other studies of self-administered parent training interventions (e.g., Incredible Years; mean TEQ-P acceptability scale scores = 55-58; Kratochwill et al., 2003; Stewart & Carlson, 2010). The moderate-to-high acceptability documented in the present study is an important indicator of the transportability of the DECA-P2 self-administered parent training intervention into real-world contexts (Stewart & Carlson, 2010), given that acceptability is linked to the adoption of interventions and compliance with their procedures (Witt & Elliott, 1985). It is interesting that parents found the program to be moderately-to-highly acceptable, given that treatment effectiveness has a strong influence on acceptability (Witt & Elliott, 1985), and replicated intervention effects were not documented for child or parent behavior in the present study. There were, however, some individual changes that may explain why parents perceived the intervention as being acceptable. For example, Parent 2 experienced a reliable 127 decrease in parenting stress, a significant improvement in her use of positive parenting practices (i.e., GAS ratings, PBI – Supportive/Engaged scale), and reported reliable changes in her child’s social-emotional competence (i.e., DECA-P2 TPF scale) and behavior concerns (i.e., DECA-P2 BC scale). Parent 3 also experienced a reliable decrease in parenting stress and reported a reliable increase in her child’s social-emotional competence (i.e., DECA-P2 TPF scale). However, Parent/Child 1 did not experience any significant changes based on participation in the program. While acceptability is highly influenced by intervention effectiveness (Witt & Elliott, 1985), high acceptability has occasionally been documented in the absence of evidence for effectiveness (e.g., suspensions for truancy; Von Brock & Elliott, 1987). Despite the fact that Parent/Child 1 did not experience any significant improvements, average DECA-P2 TPF scale scores were in the Typical range for all three children during the intervention phase, which suggests that parents perceived their children’s social-emotional functioning as being within normal limits (and likely contributed to their moderate-to-high acceptability ratings). Other potential reasons why parents rated the intervention as being moderately-to-highly acceptable may relate to a) perceptions of support, b) the delivery format, and/or c) the time efficiency. While replicated intervention effects for child and parent behaviors were not observed in the present study, parents may have felt supported in managing their child’s behaviors through participation in the program. Parents’ responses to open-ended interview questions suggested that they believed they gained something as a result of participation in the program, such as a “helpful reminder of parenting strategies” (Parent 2), an understanding of the “theory behind each strategy” (Parent 2), and time to “slow down” and “do things with the kids” (Parent 3). Previous research has also drawn connections between acceptability and intervention delivery format, with parents reporting that interventions delivered in the home setting are more helpful 128 (Lewis, Feely, Seay, Fedoravicis, & Kohl, 2016). Two of the three parents in the present study reported that they enjoyed the home visits (Parent 1, Parent 2) and all three “agreed” or “strongly agreed” that they liked the study procedures, which suggests that the intervention delivery format may have had an influence on their ratings of acceptability. Finally, acceptability is influenced by the amount of time it takes to carry out an intervention (Witt, Elliott, & Martens, 1984). Therefore, the time-efficient nature of the DECA-P2 self-administered parent training program likely contributed to the moderate-to-high acceptability ratings in the present study. Future research may seek to incorporate study data within the interview process to better understand acceptability when positive outcomes are not observed. Limitations The present study is limited by: a) issues with the reliability, validity, and sensitivity of outcome measures, b) the use of parent-reported data to measure integrity, c) inclusion criteria that did not identify a high-needs sample, and d) changes in measurement interval length between baseline and intervention phases. Reliability, validity, and sensitivity of measures. There are several concerns relating to the reliability and validity of the primary outcome measure of child behavior (i.e., DBRs), as well as the direct behavior observation measure (i.e., CII). While behavior rating scales [e.g., Child Behavior Checklist (Achenbach & Edelbrock, 1983), ECBI (Eyberg & Robinson, 1983)] and observational methods [e.g., Dyadic Parent-Child Interactive Coding System (Robinson & Eyberg, 1981)] are typically used as primary outcome measures when exploring the effectiveness of parent training programs (Lundahl et al., 2006; Reyno & McGrath, 2006), DBRs were selected as the primary outcome measure in the present study, as it is a more appropriate and feasible method for collecting data on a daily/weekly basis than lengthy behavior rating scales or 129 direct behavior observations. Reliability coefficients were good-to-excellent for Average Social- Emotional Competence scores; however, reliability was questionable for Behavior Concerns scores in the baseline phase and unacceptable in the intervention phase. It is unclear why reliability was lower for this variable, given that the current DBR measure was structured in the same way as DBR measures within studies documenting high reliability, including a) asking raters to place a mark on a line indicating the proportion of time that a target behavior was observed, and b) using a scale from 0% to 100% with qualitative anchors at 0% (Never), 50% (Sometimes), and 100% (Always; e.g., Chafouleas et al., 2010; Kilgus et al., 2016). Additionally, research indicates that the number of ratings collected during the baseline phase (6-9) should be enough to achieve high reliability (e.g., Chafouleas et al., 2013; Kilgus et al., 2016). Despite similarities between measures, one key difference between the current and previous studies relates to the rater. Previous studies used teachers (e.g., Chafouleas et al., 2010; Chafouleas et al., 2013; Kilgus et al., 2016) and/or outside evaluators (i.e., undergraduate students; Riley-Tillman, Christ, Chafouleas, Boice-Mallach, & Briesch, 2011) as raters, so it is difficult to conclude whether results would differ for parent raters. While DBR developers indicate that parents can be raters (Christ, Riley-Tillman, Chafouleas, & Boice, 2010), research has not been conducted on the reliability of parent-rated DBRs. It is likely that teachers and other professionals have more prior experience with DBR-like tools (e.g., Daily Behavior Report Card; Chafouleas, Riley-Tillman, & McDougal, 2002) than parents, so parents may require brief training to ensure reliable, accurate DBR ratings. Validity of the DBR measure in the present study was also questionable, especially given that reliability is a necessary condition of validity (Kilgus et al., 2016). Previous research has documented convergent validity for DBR ratings, as evidenced by correlations with systematic 130 direct observation (Riley-Tillman et al., 2008) and behavior rating scales (Chafouleas et al., 2009). Yet, current findings were not reflective of this. In contrast to previous research, parent DBR ratings of average social-emotional competence and behavior concerns were not correlated with rating scale scores (i.e., DECA-P2 TPF and BC scales) or direct behavior observation measures (i.e. CII – Child Bonding with Parent and Child Noncompliant/Aggressive scales). There were some reliability concerns with direct behavior observation measures (i.e., low internal consistency), which may explain the low correlations with DBR ratings. However, DBR ratings should have correlated strongly with DECA-P2 scores, given that these measures were assessing the same behaviors and the DECA-P2 is a well-established, psychometrically-sound measurement tool (LeBuffe & Naglieri, 2012). Therefore, the low correlations between DBR ratings and DECA-P2 scores raise some concerns about the concurrent validity of the DBR measure developed for use in this study. There were also reliability concerns with the CII. First and foremost, three of the four CII scales did not demonstrate acceptable levels of internal consistency (Child Bonding with Parent, Child Noncompliant/Aggressive, Nurturing/Supportive). Therefore, this calls into question whether these CII scales were accurately measuring what they are intended to measure. Low Cronbach’s alpha values can result from several factors, including a low number of items on a measure, poor interrelatedness between items, or inclusion of multiple constructs (Tavakol & Dennick, 2011). It is likely that the low alpha values for the CII – Child scales (Child Bonding with Parent, Child Noncompliant/Aggressive) resulted from the low number of items on these scales, given that each scale only has 6 items. Since direct behavior observation was a supplemental measurement approach in the present study, a lengthy coding system was not selected for use; however, more items may be necessary to demonstrate stronger reliability. The 131 Nurturing/Supportive scale, on the other hand, included three items with weak item-total correlations (i.e., less than .3), indicating that these three items should potentially be removed from the scale to improve reliability (Field, 2013). Another potential concern with direct behavior observation relates to reactivity effects, or the tendency for individuals to change their behavior when an observer is present (Gittelsohn, Shankar, West, Ram, & Gnywali, 1997). In the present study, observations were video-recorded by researchers in the home setting. While researchers attempted to make their presence as minimally noticeable as possible, this was difficult to do in many cases due to the small size of the homes. Additionally, observations were conducted during free play, which appeared to be highly reinforcing to the children and, therefore, may not accurately reflect their behavior across the day. Finally, sensitivity to behavior change may have been affected by the restricted range and ceiling effects of some of the outcome measures (Coster, 2013). For example, the small range of potential scores for the GAS ratings (Range = 4) and CII ratings (Range = 3) may have limited the capability for these measures to detect changes over time. Additionally, while DBR measures have shown high treatment sensitivity in previous research (Chafouleas et al., 2012), children had fairly high DBR scores of social-emotional competence at baseline and, therefore, ceiling effects may have prevented the measure from being able to detect meaningful change (Coster, 2013). Ceiling effects may have also impacted sensitivity to change for PBI – Supportive/Engaged scale scores, CII – Child Bonding with Parent scale scores, and CII – Nurturing/Supportive scale scores. Parent-reported measures. Another limitation was that the present study relied solely on a parent-reported measure to assess integrity. While observational measures would have 132 provided a more reliable assessment of integrity (Roach & Elliott, 2008), a parent-reported measure was selected due to its time- and cost-effectiveness. Limitations of parent-reported measures include the vulnerability to biases and inaccurate reporting (Wainer & Ingersoll, 2013). For example, parents’ responses may be susceptible to social desirability biases. Social desirability biases refer to participants responding in ways that they believe are socially acceptable within a specific context (Nederhof, 1985). In the present study, parents may have reported high levels of integrity to align with the perceived expectations of the study. Furthermore, integrity tends to be overestimated by the individuals who are implementing a given intervention (Roach & Elliott, 2008). In the present study, researchers cross-checked two items on the integrity checklist (i.e., whether the parent completed the reflection and brainstorming questions in the workbook); however, there was no way to determine whether parents overestimated integrity to the rest of the items on the checklist. Therefore, overall integrity scores should be interpreted with caution. Inclusion criteria. To be included within the study, children needed to demonstrate at- risk social-emotional development, as evidenced by scores of 40 or below (i.e., in the Area of Need range) on the DECA-P2 TPF scale or one or more of the TPF subscales (LeBuffe & Naglieri, 2012). Additionally, parents needed to demonstrate infrequent use of one or more positive parenting practices, as evidenced by a rating of 2 (Somewhat True) or below on one or more PBI – Supportive/Engaged scale items. These inclusion criteria led to a sample of children with relatively high levels of social-emotional competence at baseline, and a sample of parents with relatively frequent use of positive parenting practices at baseline. As a result, there was little room for improvement in these outcome variables. While the participants did show indication of being at-risk, other researchers have suggested targeting a sample with a greater level of need to 133 ensure stable baseline data and to increase the chances of seeing a significant effect following treatment (Boyle et al., 2010). Changes in measurement interval length between baseline and intervention phases. The change from daily data collection during the baseline phase to weekly data collection during the intervention phase may have influenced outcomes by introducing a confounding variable. The length between measurements was increased during the intervention phase to improve feasibility for families and to minimize intrusiveness. This technically is not in violation of single-case design standards, given that the literature does not provide specific guidelines regarding consistent measurement intervals across phases (e.g., Kratochwill et al., 2010; Tate et al., 2016). Instead, Tate and colleagues (2016) simply recommend that investigators must be clear about the length of time between data points. Despite this, it is important to recognize that shifting measurement interval lengths between phases could potentially pose a threat to internal validity. Implications for Research Although the DECA-P2 self-administered parent training intervention did not lead to positive outcomes for the three parents and children in the present study, findings should be used to inform future studies of the intervention (Sheridan, 2014). Following Sheridan’s (2014) intervention research trajectory, Step 1 focused on identifying a problem (i.e., the need for parent training programs that overcome barriers to mental health care for families living in poverty) and Step 2 focused on selecting strategies (i.e., the already-published DECA-P2 family guide; Mackrain & Cairone, 2013). In Step 3, a pilot study was conducted to assess the feasibility of the intervention, and findings supported the effectiveness, integrity, and acceptability of the program (Thomson & Carlson, 2016). The present study fulfilled Step 4 by evaluating the DECA-P2 self- 134 administered parent training intervention with increased methodological rigor within a small sample (i.e., single-case design), which provided a better understanding of some of the nuances of the intervention that can be explored further in future research. These nuances can be incorporated within Step 5 (i.e., replicating and extending research within a new sample, problem, or context) to learn more about how the intervention works, for whom it works, and in what contexts it works (Sheridan, 2014). Study replication should explore a slightly different target population (using a multiple- baseline across participants design, given its strong internal and external validity; Kraotchwill & Levin, 2010). In particular, inclusion criteria should be adjusted to include children with more problematic behavior and parents with more maladaptive parenting practices. While the DECA- P2 resources are intended for at-risk children and families (LeBuffe & Naglieri, 2012), it would potentially be easier to establish a stable baseline pattern and/or show significant change if participants with more problematic behaviors were included (Boyle et al., 2010). The DECA-P2 rating scale does not offer a T-score cut-off for clinical-level behaviors, but future research could require a lower T-score at baseline for inclusion, such as two standard deviations below the mean (i.e., 30 or below). Researchers could also use another broad-band rating scale with clinical cut- off scores (e.g., Child Behavior Checklist; Achenbach & Edelbrock, 1983) to determine whether children would qualify for participation in the study, though this would not be as well-aligned with the three components of social-emotional competence targeted by the DECA-P2 family guide (Mackrain & Cairone, 2013). Study extension(s) should test the potential benefit of some modifications to the intervention. For example, the intervention could be extended over 10 to 12 weeks, similar to the timeframe used in other studies of self-administered parent training programs (e.g., Incredible 135 Years, Triple P; Connell et al., 1997; Markie-Dadds & Sanders, 2006; Webster-Stratton et al., 1988; Webster-Stratton, 1990, 1992). Furthermore, research should assess the potential added benefit of phone and/or telehealth consultation (including coaching) with a therapist, as has been incorporated in previous literature (Markie-Dadds & Sanders, 2006; Webster-Stratton, 1990; Wainer & Ingersoll, 2015). Finally, future studies should seek to improve recruitment efforts to encourage more families to participate in the intervention. All future work should address one of the major limitations of the present study by incorporating primary outcome measures with greater reliability, validity, and sensitivity. While DBRs have been shown to be reliable, valid, and sensitive to behavior change (e.g., Chafouleas et al., 2012; Chafouleas, Kilgus, et al., 2009; Kilgus et al., 2016; Riley-Tillman et al., 2008), there is some variation in the literature and psychometric properties have not yet been documented with parents as raters. The DECA-P2 rating scale, on the other hand, has demonstrated strong reliability and validity when rated by parents (LeBuffe & Naglieri, 2012), but given that the DECA-P2 is a lengthy measure (i.e., 38 items), it is not appropriate for repeated measurement (i.e., a requirement of single-case design). Therefore, future studies should seek to identify more reliable and valid assessment tools that are appropriate for daily and/or weekly data collection. Implications for Practice There is a small, but growing, body of literature suggesting that self-administered interventions may hold considerable promise for building social-emotional competence and reducing behavior concerns among young children (de Graaf et al., 2008; Webster-Stratton et al., 1988). This disruptive innovation has the potential to increase access to mental health services among hard-to-reach populations (e.g., families living in poverty; Rotheram-Borus et al., 2012) 136 by increasing integrity and acceptability, which are important indicators of the transportability of an intervention (Chorpita, 2003). Similar to previous studies of self-administered parent training interventions (e.g., Kratochwill et al., 2003; Ogg & Carlson, 2009; Stewart & Carlson, 2010), parents in the present study rated the intervention as being acceptable and were able to carry out the intervention components with a high level of integrity. This is a positive finding, given the many barriers low-income families face to accessing and participating in mental health services (Gross et al., 2001). Although a replicated intervention effect was not documented for the three parents and children who participated in the present study, Sheridan (2014) indicates that the “question of impact is not a static or unidimensional one” (p. 303) and instead, interventions can be refined for different contexts, participants, and conditions. Therefore, one consideration for practice may be a tiered model of service delivery, given the response variation among the participants in the present study. In particular, Parent/Child 2 and Parent/Child 3 showed some indication of individual improvements (i.e., DECA-P2 TPF and BC scales, GAS ratings of positive parenting practices, PBI – Supportive/Engaged scale scores, PSI-SF scores), while Parent/Child 1 did not experience any individual changes after participating in the intervention. This suggests that some families might benefit from a self-administered approach alone, while others may need additional supports (i.e., phone consultation with a therapist, group discussion). Previous studies have found some added benefits when self-administered parent training programs are supplemented with therapist consultation via phone (Markie-Dadds & Sanders, 2006; Webster- Stratton, 1990) or group discussion (Webster-Stratton et al., 1988). Continued exploration of the DECA-P2 self-administered parent training intervention in applied settings will help to uncover 137 whether any of these enhancements could be integrated into a tiered model of service delivery to address each family’s level of need. 138 APPENDICES 139 APPENDIX A Inclusion and Exclusion Criteria Please check all those that apply. If all criteria are not met, then the family is not eligible for participation in this research project. 1. Inclusions: CHILD ______ ______ ______ PARENT/FAMILY ______ ______ Male or female between 3 and 5 years of age at their last birthday T-score between 28 and 40 on the DECA-P2 Total Protective Factors (TPF) scale or one or more of the TPF subscales Child is currently enrolled in Head Start Household income below the federal poverty line At least one item rated a 2 or below on the Parent Behavior Inventory (PBI) Supportive/Engaged scale 2. Exclusions: CHILD ______ Please write “NO” in the blank provided for all criteria that do not apply. If any of the criteria are not negated, the child is not eligible for participation in this research project Diagnosed with an intellectual or developmental disorder Not fluent in English PARENT ______ 140 APPENDIX B Direct Behavior Rating Form Directions: Place a mark along the line that best reflects the percentage of total time your child exhibited each target behavior throughout the day. Note that percentages do not need to total 100% across behaviors since behaviors may co-occur. 1. My child showed initiative today (e.g., tried new things, asked questions, stuck with a task even when it was hard to do). 2. My child showed self-control today (e.g., calmed down when upset, listened to others, showed patience). 3. My child interacted well with peers and adults today (e.g., showed affection for familiar adults, appeared happy when playing with others). 4. My child showed problem behaviors today (e.g., difficulty concentrating, hurt others with actions or words, temper tantrums). 141 APPENDIX C Goal Attainment Scale (Example) Directions: Using the scale below, rate your level of attainment for each parenting practice. At the bottom of this page, place an ‘X’ in the box that matches your rating for a given day. Ratings should be selected daily. Level of Attainment +2 Much more than expected +1 Somewhat more than expected At expected level Somewhat less than expected Much less than expected 0 -1 -2 Parenting Practice 1: Teach my child new things Parenting Practice 2: Say mean things to my child Today, I taught my child new things much more than I did previously. Today, I said mean things to my child much more than I did previously. Today, I taught my child new things somewhat more than I did previously. Today, I taught my child new things as much as I did previously. Today, I said mean things to my child somewhat more than I did previously. Today, I said mean things to my child as much as I did previously. Today, I taught my child new things somewhat less than I did previously. Today, I said mean things to my child somewhat less than I did previously. Today, I taught my child new things much less than I did previously. Today, I said mean things to my child much less than I did previously. Parenting Practice 1: Teach my child new things +2 +1 0 -1 -2 Rating Date 1 2 3 4 5 6 +2 +1 0 -1 -2 Parenting Practice 2: Say mean things to my child 6 Rating Date 2 3 4 1 5 7 7 8 8 9 9 142 APPENDIX D Parent Behavior Inventory Parent Behavior Inventory Read each statement carefully. Think about how you and your child generally get along. Tell us how well the statement describes the way you usually act with your child. Example: Rating _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ _____ 0 1 2 3 4 5 not at all true (I do not do this) a little true somewhat true moderately true quite a bit true very true (I often do this) I argue with my child. If you spend a great deal of time arguing with your child, you would mark a 5. If you never argue with your child, you would mark a 0. If you argue sometimes, but not much, you would mark a 1 or 2. If you argue often, but not all of the time, you would mark a 3 or 4. Statements I lose my temper when my child doesn’t do something I ask him/her to do. I have pleasant conversations with my child. I grab or handle my child roughly. I try to teach my child new things. I demand that my child does something (or stop doing something) right away. My child and I hug and/or kiss each other. I complain about my child’s behavior or tell my child I don’t like what s/he is doing. I laugh with my child about things we find funny. When my child misbehaves, I let him/her know what will happen if s/he doesn’t behave. My child and I spend time playing games, doing crafts, or doing other activities. I listen to my child’s feelings and try to understand them. I thank or praise my child. I spank or use physical punishment with my child. I offer to help, or help my child with things s/he is doing. I threaten my child. I comfort my child when s/he seems scared, upset, or unsure. I say mean things to my child that can make him/her feel bad. 143 _____ _____ _____ I hold or touch my child in an affectionate way. When I am disappointed in my child’s behavior, I remind him/her about what I’ve done for him/her. When my child asks for help or attention, I ignore him/her to make him/her wait until later. 144 APPENDIX E Coder Impressions Inventory Nurturing/Supportive Parenting Circle the number that best describes what was observed: Paid attention when child talked or asked questions Did not pay attention when child talked Parent was verbally affectionate to child (positive tone of voice, pet name, etc.) Parent seemed to enjoy parenting Parent was physically affectionate with child Parent was patient with child Parent is a positive and reinforcing parent Friendly relations between parent and child Parent seemed distant, detached from child Parent treated child with respect (acceptance) Parent seems supportive and empathetic Parent relates positive comments about child to coder The child did something clearly prosocial (extra nice, volunteered, did something for family, was cooperative), and the parent ignored it (or paid no attention) Harsh/Critical Parenting Circle the number that best describes what was observed: The parent threatened punishment for misbehavior (“If you do that again, I’ll hit you”) The parent made unreasonable request(s) (e.g., age- inappropriate, too high standards, impossible to comply with) The parent showed disapproval or criticized child The parent used guilt induction to get compliance The parent seemed to provoke the child into arguments The parent used sarcasm in a denigrating or hurtful way Parent did or said things to clearly indicate anger, irritability, or frustration 145 No basis 0 Did not occur 1 3 1 1 1 1 1 1 3 1 1 1 3 1-3 examples 4+ examples 2 2 2 2 2 2 2 2 2 2 2 2 2 3 1 3 3 3 3 3 3 1 3 3 3 1 No basis 0 0 Did not occur 1 1 1 1 1 1 1 1-3 examples 4+ examples 2 2 2 2 2 2 2 3 3 3 3 3 3 3 Parent was physically aggressive toward child Parent shouted at child Parent was overly strict, authoritarian, and/or oppressive Parent used nagging to get compliance Parent showed anger/hostility while disciplining Child Bonding with Parent Circle the number that best describes what was observed: Child was verbally affectionate to parent Child appeared depressed, sad, bummed out, tired, with flat affect Child was physically affectionate with parent The child seemed to enjoy parent’s verbal praise or encouragements Friendly relations between parent and child Child seemed aloof, distant, or unattached to parent Child Noncompliant/Aggressive Circle the number that best describes what was observed: The child did not comply with at least one parental request/command Child said or did things to clearly indicate anger, irritability, or frustration Child was physically aggressive toward parent Child shouted at parent Child seemed to have hostile, arrogant, or noncompliant set to parent Child treated parent with respect Scores Scales Harsh/Critical Nurturing/ Supportive 0 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 No basis 0 No basis 0 Did not occur 1 3 1 1 1 3 Did not occur 1 1 1 1 1 3 1-3 examples 4+ examples 2 2 2 2 2 2 3 1 3 3 3 1 1-3 examples 4+ examples 2 2 2 2 2 2 3 3 3 3 3 1 Child Bonding with Parent Child Noncompliant/ Aggressive Total Score Mean Score 146 APPENDIX F Demographic Form 147 APPENDIX G Weekly Integrity Checklist Week 1 Integrity Checklist Initiative: Part 1 Directions: Circle Yes or No to indicate whether each intervention task was completed within the last week. Intervention Task Completed? Comments 1. Read the Week 1 Overview (p. 4) Yes No 2. Read the assigned chapter(s) from Promoting Resilience for Now and Forever [Chapter 1 & Chapter 2] 3. Answered the reflection questions in the workbook (p. 5) 4. Reviewed the three focus strategies for the week (p. 6) 5. Answered the brainstorming questions in the workbook (p. 7) 6. Used Strategy 1 [Do things as a family regularly] at least once throughout the week Yes No Yes No Yes No Yes No Yes No 7. Used Strategy 2 [Have fun together every day] at least once throughout the week Yes No 8. Used Strategy 3 [Find what is special about your child] at least once throughout the week 9. Completed the daily self-monitoring log (p. 8) Yes No Yes No Number of Tasks Completed (Yes) = _______ / 9 148 Treatment Evaluation Questionnaire – Parent Form (TEQ-P) Acceptability Scale APPENDIX H Treatment Evaluation Questionnaire – Parent You and your child recently completed participation in a research study on an intervention approach for building social-emotional competence. Please evaluate the intervention by circling the number which best describes your agreement or disagreement with each statement. Please answer each question. Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree 1. This was an acceptable intervention for building my child’s social and emotional skills. 2. Most parents would find this intervention appropriate for building children’s social and emotional skills. 3. This intervention was effective in building my child’s social and emotional skills. 4. I would suggest the use of this intervention to other parents. 5. My child’s social and emotional skills were lacking enough to warrant use of this intervention. 6. Most parents would find this intervention suitable for building social and emotional skills. 7. The intervention did not result in negative side effects for my child. 8. The intervention would be appropriate for a variety of children. 9. The intervention was a fair way to build my child’s social and emotional skills. 10. I liked the procedures used in the intervention. 11. The intervention was a good way to build my child’s social and emotional skills. 1 1 1 1 1 1 1 1 1 1 1 149 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 APPENDIX I Recruitment Flyer 150 APPENDIX J Example Workbook Week Week 1 Checklist Initiative: Part 1 At the start of the week: (cid:0) Read the Week 1 Overview (p. 4) (cid:0) Read from Promoting Resilience for Now and Forever: Chapter 1 (pp. 5-8) Chapter 2 (pp. 9-13) *Do NOT complete “Thinking about Your Initiative” questions (pp. 13-14) (cid:0) Answer reflection questions in the workbook (p. 5): (cid:0) Review the three focus strategies for building initiative (p. 6): Do things as a family regularly. (p. 12) Have fun together every day. (p. 12) Find what is special about your child. (p. 13) (cid:0) Brainstorm ideas for using the three focus strategies (p. 7) During the week: (cid:0) Use the three focus strategies as many times as possible! At the end of the week: (cid:0) Complete the Week 1 Integrity Checklist (p. 9) (cid:0) Complete the DECA-P2 rating scale (cid:0) Complete the Parenting Practices Survey Note: Bolded page numbers are from the parent workbook. Italicized page numbers are from the family guide (Promoting Resilience for Now and Forever). 151 Week 1 Overview Initiative: Part 1 Introduction: Social and emotional health is essential for a child’s success in school and in life. Initiative is one factor that leads to positive social and emotional development. Children with initiative are eager to engage with others, interested in trying new things, and able to problem-solve effectively. This week, you will focus on building initiative by helping children learn the importance of engaging with others. Goals: 1. Understand the importance of social and emotional health for preschoolers 2. Understand the importance of initiative for social and emotional health 3. Learn and use strategies to build your child’s initiative 152 Week 1 Reflection Questions Initiative: Part 1 1. Describe one of your favorite family memories from the past year. Why was it so special? _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ 2. Do you think this experience had a lasting effect on your family’s culture? Why or why not? _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ 153 Week 1 Strategies Initiative: Part 1 1. Do things as a family regularly. Taking part in regular events helps children to be ready for them. Children feel secure when life has some order and they know what happens next. Examples: -Eat breakfast together each day -Read stories at bedtime -Go to a local event -Visit friends -Play games 2. Have fun together every day. Turn ordinary events into opportunities for fun. Examples: -Take turns pushing each other on the swings at the park -Talk about favorite foods at the grocery store -Play games or sing songs in the car 3. Find what is special about your child. Each child is special in his own way. Comment on his special strengths. Examples: -“Jackson, you tell silly jokes that make me laugh!” 154 Week 1 Ideas Initiative: Part 1 1. What activities might you engage in as a family this week? • _______________________________________________________________ • _______________________________________________________________ • _______________________________________________________________ 2. What ordinary events could you turn into opportunities for fun? • _______________________________________________________________ • _______________________________________________________________ • _______________________________________________________________ 3. What special strengths does your child have? (Be sure to comment about these strengths during the week!) • _______________________________________________________________ • _______________________________________________________________ • _______________________________________________________________ 155 APPENDIX K Visual Analysis Guide DBR-Average Social-Emotional Competence, Child # ______ Visual Analysis Guide: Directions: Complete the following form for each child, using the graph of his/her DBR- Average Social-Emotional Competence scores. Preliminary Analysis. Is there evidence of problem behavior (i.e., low DBR scores)? Are baseline data relatively stable? ___ Yes to both (Continue to Step 1) ___ No to either (Discontinue analysis) Step 1: Level (Mean). Is there a noticeable change in the level (mean) between the baseline phase and the intervention phase? In other words, is the intervention phase level visually higher than the baseline phase? ___ Yes, the intervention phase level is visually higher than baseline (Continue to Step 2) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 2: Trend (Slope). Is the trend (slope) distinctly more prominent during the intervention phase compared to the baseline phase? Is it trending in the anticipated direction (i.e., upward trend over time)? ___ Yes, the trend is more prominent in the anticipated direction (Continue to Step 3) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 3: Variability. Are data from the intervention phase relatively stable (i.e., with minimal vertical spread)? ___ Yes, data from the intervention phase are stable (Continue to Step 4) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 4: Immediacy of Effect. Are these changes noticeable within the first 3 weeks after the intervention start point (i.e., first three data points in the intervention phase)? ___ Yes, observable changes occurred within 3 weeks after intervention onset (Mark “Noticeable treatment effect” at bottom of page) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) ________ Noticeable treatment effect 156 _______ No noticeable treatment effect Visual Analysis Guide: DBR-Behavior Concerns, Child # ______ Directions: Complete the following form for each child, using the graph of his/her DBR- Behavior Concerns scores. Preliminary Analysis. Is there evidence of problem behavior (i.e., high DBR scores)? Are baseline data relatively stable? ___ Yes to both (Continue to Step 1) ___ No to either (Discontinue analysis) Step 1: Level (Mean). Is there a noticeable change in the level (mean) between the baseline phase and the intervention phase? In other words, is the intervention phase level visually lower than the baseline phase? ___ Yes, the intervention phase level is visually lower than baseline (Continue to Step 2) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 2: Trend (Slope). Is the trend (slope) distinctly more prominent during the intervention phase compared to the baseline phase? Is it trending in the anticipated direction (i.e., downward trend over time)? ___ Yes, the trend is more prominent in the anticipated direction (Continue to Step 3) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 3: Variability. Are data from the intervention phase relatively stable (i.e., with minimal vertical spread)? ___ Yes, data from the intervention phase are stable (Continue to Step 4) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 4: Immediacy of Effect. Are these changes noticeable within the first 3 weeks after the intervention start point (i.e., first three data points in the intervention phase)? ___ Yes, observable changes occurred within 3 weeks after intervention onset (Mark “Noticeable treatment effect” at bottom of page) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) ________ Noticeable treatment effect 157 _______ No noticeable treatment effect Visual Analysis Guide: GAS-Positive Parenting Practice, Child # ______ Directions: Complete the following form for each parent, using the graph of his/her GAS- Positive Parenting Practice scores. Preliminary Analysis. Is there evidence of problem behavior (i.e., low GAS scores)? Are baseline data relatively stable? ___ Yes to both (Continue to Step 1) ___ No to either (Discontinue analysis) Step 1: Level (Mean). Is there a noticeable change in the level (mean) between the baseline phase and the intervention phase? In other words, is the intervention phase level visually higher than the baseline phase? ___ Yes, the intervention phase level is visually higher than baseline (Continue to Step 2) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 2: Trend (Slope). Is the trend (slope) distinctly more prominent during the intervention phase compared to the baseline phase? Is it trending in the anticipated direction (i.e., upward trend over time)? ___ Yes, the trend is more prominent in the anticipated direction (Continue to Step 3) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 3: Variability. Are data from the intervention phase relatively stable (i.e., with minimal vertical spread)? ___ Yes, data from the intervention phase are stable (Continue to Step 4) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 4: Immediacy of Effect. Are these changes noticeable within the first 3 weeks after the intervention start point (i.e., first three data points in the intervention phase)? ___ Yes, observable changes occurred within 3 weeks after intervention onset (Mark “Noticeable treatment effect” at bottom of page) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) ________ Noticeable treatment effect 158 _______ No noticeable treatment effect Visual Analysis Guide: GAS-Negative Parenting Practice, Child # ______ Directions: Complete the following form for each parent, using the graph of his/her GAS- Negative Parenting Practice scores. Preliminary Analysis. Is there evidence of problem behavior (i.e., high GAS scores)? Are baseline data relatively stable? ___ Yes to both (Continue to Step 1) ___ No to either (Discontinue analysis) Step 1: Level (Mean). Is there a noticeable change in the level (mean) between the baseline phase and the intervention phase? In other words, is the intervention phase level visually lower than the baseline phase? ___ Yes, the intervention phase level is visually lower than baseline (Continue to Step 2) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 2: Trend (Slope). Is the trend (slope) distinctly more prominent during the intervention phase compared to the baseline phase? Is it trending in the anticipated direction (i.e., downward trend over time)? ___ Yes, the trend is more prominent in the anticipated direction (Continue to Step 3) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 3: Variability. Are data from the intervention phase relatively stable (i.e., with minimal vertical spread)? ___ Yes, data from the intervention phase are stable (Continue to Step 4) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) Step 4: Immediacy of Effect. Are these changes noticeable within the first 3 weeks after the intervention start point (i.e., first three data points in the intervention phase)? ___ Yes, observable changes occurred within 3 weeks after intervention onset (Mark “Noticeable treatment effect” at bottom of page) ___ No (Discontinue analysis, mark “No noticeable treatment effect” at bottom of page) ________ Noticeable treatment effect 159 _______ No noticeable treatment effect Visual Analysis Guide: Consistency of Change Check (CCC) Directions: Complete one CCC for each measure (4 total). After reviewing data for all three cases, carefully examine all of the within-phase data together for each measure (i.e., three together for DBR-Average Social-Emotional Competence, three together for DBR-Behavior Concerns, three together for GAS-Positive Parenting Practice, three together for GAS-Negative Parenting Practice). Look first at the baseline data across all cases. Next, examine the intervention phases across all cases. Answer the following question: Step 5: Consistency of Data. When analyzing all of the baseline phases and then all of the treatment phases at one time, do the observable changes appear to be consistently occurring during the intervention phase for all cases (i.e., a treatment effect is clearly replicated)? Mark one. ______ Yes, there is a clear replicated effect ______ Mixed results because (please explain): ______________________________________ _____________________________________________________________________________ _____________________________________________________________________________ _____________________________________________________________________________ _____ No, there was no replicated effect 160 REFERENCES 161 REFERENCES Abidin, R. R. (2012). Parenting Stress Index: Professional manual (4th ed.). Odessa, FL: Psychological Assessment Resources. Achenbach, T. M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington, VT: Author. Aikens, N., Klein, A. K., Tarullo, L., & West, J. (2013). Getting ready for kindergarten: Children’s progress during Head Start. FACES 2009 report. In J. M. O’Sullivan (Ed.), Head Start and child outcomes: Select analyses of FACES 2009 (pp. 1-20). New York, NY: Nova Science Publishers. Amlund Hagen, K., Ogden, T., & Bjørnebekk, G. (2011). Treatment outcomes and mediators of parent management training: A one-year follow-up of children with conduct problems. Journal of Clinical Child & Adolescent Psychology, 40, 165-178. doi: 10.1080/15374416.2011.546050 Andershed, A., & Andershed, H. (2015). Risk and protective factors among preschool children: Integrating research and practice. Journal of Evidence-Informed Social Work, 12, 412- 424. doi: 10.1080/15433714.2013.866062 Armstrong, K. H., Ogg, J. A., Sundman-Wheat, A. N., & Walsh, A. S. (2014). Evidence-based interventions for children with challenging behavior. New York, NY: Springer. Arnold, D. S., O’Leary, S. G., Wolff, L. S., & Acker, M. M. (1993). The Parenting Scale: A measure of dysfunctional parenting in discipline situations. Psychological Assessment, 5, 137-144. doi: 10.1037/1040-3590.5.2.137 Atkins, M. S., Frazier, S. L., Birman, D., Abdul Adil, J., Jackson, M., Graczyk, P. A., … McKay, M. M. (2006). School-based mental health services for children living in high poverty urban communities. Administration and Policy in Mental Health and Mental Health Services Research, 33, 146-159. doi: 10.1007/s10488-006-0031-9 Bandura, A. (1986). Social foundations of thought and action: A social-cognitive theory. Englewood Cliffs, NJ: Prentice-Hall. 162 Barton, E. E., Steed, E. A., Strain, P., Dunlap, G., Powell, D., & Payne, C. J. (2014). An analysis of classroom-based and parent-focused social-emotional programs for young children. Infants & Young Children, 27, 3-29. doi: 10.1097/iyc.0000000000000001 Biglan, A. (2016). Coercion and public health. In T. J. Dishion & J. J. Snyder (Eds.), The oxford handbook of coercive relationship dynamics (pp. 356-362). New York, NY: Oxford University Press. Bjørknes, R., Kjøbli, J., Manger, T., & Jakobsen, R. (2012). Parent training among ethnic minorities: Parenting practices as mediators of change in child conduct problems. Family Relations, 61, 101-114. doi: 10.1111/j.1741-3729.2011.00683.x Blueprints for Healthy Youth Development. (2016a). Blueprints for healthy youth development. Retrieved from http://www.blueprintsprograms.com/ Blueprints for Healthy Youth Development. (2016b). Program criteria. Retrieved from http://www.blueprintsprograms.com/criteria Borden, L. A., Schultz, T. R., Herman, K. C., & Brooks, C. M. (2010). The Incredible Years parent training program: Promoting resilience through evidence-based prevention groups. Group Dynamics: Theory, Research, and Practice, 14, 230-241. doi: 10.1037/a0020322 Boyle, C. L., Sanders, M. R., Lutzker, J. R., Prinz, R. J., Shapiro, C., & Whitaker, D. J. (2010). An analysis of training, generalization, and maintenance effects of primary care Triple P for parents of preschool-aged children with disruptive behavior. Child Psychiatry & Human Development, 41, 114-131. doi: 10.1007/s10578-009-0156-7 Brady, J., Busse, R. T., & Lopez, C. J. (2014). Monitoring school consultation intervention outcomes for data-based decision making: An application of the goal attainment scaling method. Counseling Outcome Research and Evaluation, 5, 64-70. doi: 10.1177/2150137814527605 Brinkman, T.M., Wigent, C.A., Tomac, R.A., Pham, A.V., & Carlson, J.S. (2007). Using the Devereux Early Childhood Assessment to identify behavioral risk and protective factors within a Head Start population. Canadian Journal of School Psychology, 22(2), 136-151. doi: 10.1177/0829573507307612 Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives. Developmental Psychology, 22, 723-742. doi: 10.1037//0012-1649.22.6.723 163 Bronfenbrenner, U., & Ceci, S. J. (1994). Nature-nurture reconceptualized in developmental perspective: A biological model. Psychological Review, 101, 568-586. doi: 10.1037/0033-295X.101.4.568 Brossart, D. F., Vannest, K. J., Davis, J. L., Patience, M. A. (2014). Incorporating nonoverlap indices with visual analysis for quantifying intervention effectiveness in single-case experimental designs. Neuropsychological Rehabilitation, 24, 464-491. doi: 10.1080/09602011.2013.868361 Brown-Chidsey, R., Steege, M. W., & Mace, F. C. (2008). Best practices in evaluating the effectiveness of interventions using case study data. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 2177-2191). Bethesda, MD: National Association of School Psychologists. Brownell, R. (2000). Expressive one-word picture vocabulary test. San Antonio, TX: Harcourt Assessment, Inc. Bruni, T. P., Drevon, D., Hixson, M., Wyse, R., Corcoran, S., & Fursa, S. (2017). The effect of functional behavior assessment on school-based interventions: A meta-analysis of single- case research. Psychology in the Schools, 54, 351-369. doi: 10.1002/pits.22007 Bulotsky-Shearer, R. J., Domínguez, X., Bell, E. R., Rouse, H. L., & Fantuzzo, J. W. (2010). Relations between behavior problems in classroom social and learning situations and peer social competence in Head Start and kindergarten. Journal of Emotional and Behavioral Disorders, 18, 195-210. doi: 10.1177/1063426609351172 Bulté, I., & Onghena, P. (2009). Randomization tests for multiple-baseline designs: An extension of the SCRT-R package. Behavior Research Methods, 41, 477-485. doi: 10.3758/BRM.41.2.477 Burns, M. K. (2012). Meta-analysis of single-case design research: Introduction to the special issue. Journal of Behavioral Education, 21, 175-184. doi: 10.1007/s10864-012-9158-9 Cardillo, J. E., & Smith, A. (1994). Reliability of goal attainment scores. In T. J. Kiresuk, A. Smith, & J. E. Cardillo (Eds.), Goal attainment scaling: Application, theory, and measurement (pp. 213–242). Hillsdale, NJ: Lawrence Erlbaum. Carlson, J. S., & Voris, D. S. T. (2017). One-year stability of the Devereux Early Childhood Assessment for preschoolers, second edition. Journal of Psychoeducational Assessment. Advance online publication. doi: 10.1177/0734282917710890 164 Chafouleas, S. M., Christ, T. J., Riley-Tillman, T. C., Briesch, A. M., & Chanese, J. A. M. (2007). Generalizability and dependability of direct behavior ratings to assess social behavior of preschoolers. School Psychology Review, 36, 63-79. Chafouleas, S. M., Kilgus, S. P., & Hernandez, P. (2009). Using direct behavior rating (DBR) to screen for school social risk: A preliminary comparison of methods in a kindergarten sample. Assessment for Effective Intervention, 34, 214-223. doi: 10.1177/1534508409333547 Chafouleas, S. M., Kilgus, S. P., Jaffery, R., Riley-Tillman, T. C., Welsh, M., & Christ, T. J. (2013). Direct behavior rating as a school-based behavior screener for elementary and middle grades. Journal of School Psychology, 51, 367-385. doi: 10.1016/j.jsp.2013.04.002 Chafouleas, S. M., Riley-Tillman, T. C., & Christ, T. J. (2009). Direct behavior rating (DBR): An emerging method for assessing social behavior within a tiered intervention system. Assessment for Effective Intervention, 34, 195-200. doi: 10.1177/1534508409340391 Chafouleas, S. M., Riley-Tillman, T. C., & McDougal, J. L. (2002). Good, bad, or in-between: How does the daily behavior report card rate? Psychology in the Schools, 39, 157-169. doi: 10.1002/pits.10027 Chafouleas, S. M., Sanetti, L. M., Kilgus, S. P., & Maggin, D. M. (2012). Evaluating sensitivity to behavioral change using direct behavior rating single-item scales. Exceptional Children, 78, 491-505. doi: 10.1177/001440291207800406 Chang, L., Schwartz, D., Dodge, K. A., & McBride-Chang, C. (2003). Harsh parenting in relation to child emotion regulation and aggression. Journal of Family Psychology, 17, 598-606. doi: 10.1037/0893-3200.17.4.598 Chorpita, B. (2003). The frontier of evidence-based practice. In A.E. Kazdin & J.R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 42-59). New York: Guilford Press. Christ, T. J. (2007). Experimental control and threats to internal validity of concurrent and nonconcurrent multiple baseline designs. Psychology in the Schools, 44, 451-459. doi: 10.1002/pits.20237 Christ, T. J., Riley-Tillman, T. C., Chafouleas, S. M., & Boice, C. H. (2010). Direct behavior rating (DBR): Generalizability and dependability across raters and observations. 165 Educational and Psychological Measurement, 70, 825-843. doi: 10.1177/0013164410366695 Conger, R. D., Conger, K. J., & Martin, M. J. (2010). Socioeconomic status, family processes, and individual development. Journal of Marriage and Family, 72, 685-704. doi: 10.1111/j.1741-3737.2010.00725.x Conger, R. D., Elder, G. H., Lorenz, F. O., Conger, K. J., Simons, R. L., Whitbeck, L. B., … Melby, J. N. (1990). Linking economic hardship to marital quality and instability. Journal of Marriage and Family, 52, 643-656. doi: 10.2307/352931 Connell, S., Sanders, M. R., & Markie-Dadds, C. (1997). Self-directed behavioral family intervention for parents of oppositional children in rural and remote areas. Behavior Modification, 21, 379-408. doi: 10.1177/01454455970214001 Conners, C. K. (2009). Conners early childhood manual. Toronto, Canada: Multi-Health Systems. Coster, W. J. (2013). Making the best match: Selecting outcome measures for clinical trials and outcome studies. The American Journal of Occupational Therapy, 67, 162-170. doi: 10.5014/ajot.2013.006015 Costigan, C. L., & Cox, M. J. (2001). Fathers’ participation in family research: Is there a self- selection bias? Journal of Family Psychology, 15, 706-720. doi: 10.1037/0893- 3200.15.4.706 Danzig, A. P., Dyson, M. W., Olino, T. M., Laptook, R. S., & Klein, D. N. (2015). Positive parenting interacts with child temperament and negative parenting to predict children’s socially appropriate behavior. Journal of Social and Clinical Psychology, 34, 411-435. doi: 10.1521/jscp.2015.34.5.411 Dart, E. H., Collins, T. A., Klingbeil, D. A., & McKinley, L. E. (2014). Peer management interventions: A meta-analytic review of single-case research. School Psychology Review, 43, 367-384. doi: 10.17105/spr-14-0009.1 Dawson-McClure, S., Calzada, E., Huang, K., Kamboukos, D., Rhule, D., Kolawole, B., …& Brotman, L. M. (2015). A population-level approach to promoting healthy child development and school success in low-income, urban neighborhoods: Impact on parenting and child conduct problems. Prevention Science, 16, 279-290. doi: 10.1007/s11121-014-0473-3 166 Deater-Deckard, K. (1998). Parenting stress and child adjustment: Some old hypotheses and new questions. Clinical Psychology: Science and Practice, 5, 314-332. doi: 10.1111/j.1468- 2850.1998.tb00152.x Deater-Deckard, K., Chen, N., Wang, Z., & Bell, M. A. (2012). Socioeconomic risk moderates the link between household chaos and maternal executive function. Journal of Family Psychology, 26, 391-399. doi: 10.1037/a0028331 de Graaf, I., Speetjens, P., Smit, F., de Wolff, M., & Tavecchio, L. (2008). Effectiveness of the Triple P Positive parenting program on parenting: A meta-analysis. Family Relations, 57, 553-566. doi: 10.1111/j.1741-3729.2008.00522.x Deković, M., Asscher, J. J., Hermanns, J., Reitz, E., Prinzie, P., & van den Akker, A. L. (2010). Tracing changes in families who participated in the Home-Start parenting program: Parental sense of competence as mechanism of change. Prevention Science, 11, 263-274. doi: 10.1007/s11121-009-0166-5 Dickstein, S. (2015). The family couch: Considerations for infant/early childhood mental health. Child and Adolescent Psychiatric Clinics of North America, 24, 487-500. doi: 10.1016/j.chc.2015.02.004 DiCorcia, J. A., & Tronick, E. (2011). Quotidian resilience: Exploring mechanisms that drive resilience from a perspective of everyday stress and coping. Neuroscience and Behavioral Reviews, 35, 1593-1602. doi: 10.1016/j.neubiorev.2011.04.008 Duncan, G. J., Yeung, W. J., Brooks-Gunn, J., & Smith, J. R. (1998). How much does childhood poverty affect the life chances of children? American Sociological Review, 63, 406-423. doi: 10.2307/2657556 Dunn, L. M., & Dunn, D. M. (2006). Peabody picture vocabulary test (4th ed.). Circle Pines, MS: American Guidance Service. Eamon, M. K., & Venkataraman, M. (2003). Implementing parent management training in the context of poverty. The American Journal of Family Therapy, 31, 281-293. doi: 10.1080/01926180390201972 Eckert, T. L., & Hintze, J. M. (2002). Behavioral conceptions and applications of acceptability: Issues related to service delivery and research methodology. School Psychology Quarterly, 15, 123-148. doi: 10.1037/h0088782 167 Elgar, F. J., & McGrath, P. J. (2003). Self-administered psychosocial treatments for children and families. Journal of Clinical Psychology, 59, 321-339. doi: 10.1002/jclp.10132 Epstein, M. H., & Synhorst, L. (2009). Preschool Behavioral and Emotional Rating Scale, Examiner’s manual. Austin, TX: Pro-Ed. Evans, G. W., Boxhill, L., & Pinkaya, M. (2008). Poverty and maternal responsiveness: The role of maternal stress and social resources. International Journal of Behavioral Development, 32, 232–237. doi: 10.1177/0165025408089272 Eyberg, S. M., & Robinson, E. A. (1983). Conduct problem behavior: Standardization of a behavioral rating scale with adolescents. Journal of Clinical Child Psychology, 12, 347– 354. doi: 10.1207/s15374424jccp1203_19 Ferron, J. M., & Levin, J. R. (2014). Single-case permutation and randomization statistical tests: Present status, promising new developments. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case intervention research: Methodological and statistical advances (pp. 153- 184). Washington, DC: American Psychological Association. Field, A. (2013). Discovering statistics using IBM SPSS statistics. Thousand Oaks, CA: SAGE Publications Inc. Forgatch, M. S., DeGarmo, D. S., & Beldavs, Z. G. (2005). An efficacious theory-based intervention for stepfamilies. Behavior Therapy, 36, 357-365. doi: 10.1016/S0005- 7894(05)80117-0 Forgatch, M. S., & Kjøbli, J. (2016). Parent management training – Oregon model: Adapting intervention with rigorous research. Family Process, 55, 500-513. doi: 10.1111/famp.12224 Forgatch, M. S., & Patterson, G. R. (2010). Parent management training – Oregon model: An intervention for antisocial behavior in children and adolescents. In J. R. Weisz & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 159–178). New York: Guilford. Forgatch, M. S., Patterson, G. R., DeGarmo, D. S., & Beldavs, Z. (2009). Testing the Oregon delinquency model with 9-year follow-up of the Oregon Divorce Study. Development and Psychopathology, 21, 637-660. doi: 10.1017/S0954579409000340 168 Forgatch, M. S., Patterson, G. R., & Gewirtz, A. (2013). Looking forward: The promise of widespread implementation of parent training programs. Perspectives on Psychological Science, 8, 682-694. doi: 10.1177/1745691613503478 Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy, L. A., Rosenfield, S. A., … Stoiber, K. C. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77-100. doi: 10.1037/spq0000019 Garland, A. F., Hawley, K. M., Brookman-Frazee, L., & Hurlburt, M. S. (2008). Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior problems. Journal of the American Academy of Child & Adolescent Psychiatry, 47, 505-514. doi: 10.1097/CHI.0b013e31816765c2 Garmezy, N. (1991). Resiliency and vulnerability to adverse developmental outcomes associated with poverty. The American Behavioral Scientist, 34, 416-430. doi: 10.1177/0002764291034004003 Gittelsohn, J., Shankar, A. V., West, K. P., Ram, R. M., & Gnywali, T. (1997). Estimating reactivity in direct observation studies of health behaviors. Human Organization, 56, 182-189. doi: 10.17730/humo.56.2.c7x0532q2u86m207 Gross, D., Johnson, T., Ridge, A., Garvey, C., Julian, W., Brusius Treysman, A., … Fogg, L. (2011). Cost-effectiveness of childcare discounts on parent participation in preventative parent training in low-income communities. Journal of Primary Prevention, 32, 283-298. doi: 10.1007/s10935-011-0255-7 Gross, D., Julion, W., & Fogg, L. (2001). What motivates participation and dropout among low- income urban families of color in a prevention intervention? Family Relations, 50, 246- 254. doi: 10.1111/j.1741-3729.2001.00246.x Hahlweg, K., Heinrichs, N., Kuschel, A., & Feldmann, M. (2008). Therapist-assisted, self- administered bibliotherapy to enhance parental competence: Short- and long-term effects. Behavior Modification, 32, 659-681. doi: 10.1177/0145445508317131 Harwood, T. M., & L’Abate, L. (2010). Self-help in mental health: A critical review. New York, NY: Springer. doi:10.1007/978-1-4419-1099-8 Hawkins, N. G., Sanson-Fisher, R. W., Shakeshaft, A., D’Este, C., & Green, L. W. (2007). The multiple baseline design for evaluating population-based research. American Journal of Preventative Medicine, 33, 162-168. doi: 10.1016/j.amepre.2007.03.020 169 Heckman, J., Pinto, R., & Savelyev, P. (2013). Understanding the mechanisms through which an influential early childhood program boosted adult outcomes. American Economic Review, 103, 2052-2086. doi: 10.1257/aer.103.6.2052 Heinrichs, N., Bertram, H., Kuschel, A., & Hahlweg, K. (2005). Parent recruitment and retention in a universal prevention program for child behavior and emotional problems: Barriers to research and program participation. Prevention Science, 6, 275–286. doi: 10.1007/s11121-005-0006-1 Henrich, C. C., & Gadaire, D. M. (2008). Head start and parent involvement. Infants & Young Children, 21, 56-69. doi: 10.1097/01.iyc.0000306373.48038.e6 Heyvaert, M., & Onghena, P. (2014). Randomization tests for single-case experiments: State of the art, state of the science, and state of the application. Journal of Contextual Behavioral Science, 3, 51-64. doi: 10.1016/j.jcbs.2013.10.002 Hindman, A. H., Skibbe, L. E., Miller, A., & Zimmerman, M. (2010). Ecological contexts and early learning: Contributions of child, family, and classroom factors during Head Start, to literacy and mathematics growth through first grade. Early Childhood Research Quarterly, 25, 235-250. doi: 10.1016/j.ecresq.2009.11.003 Hitchcock, J. H., Kratochwill, T. R., & Chezan, L. C. (2015). What Works Clearinghouse standards and generalization of single-case design evidence. Journal of Behavioral Education, 24, 459-469. doi: 10.1007/s10864-015-9224-1 Hoath, F. E., & Sanders, M. R. (2002). A feasibility study of enhanced group Triple P – Positive Parenting Program for parents of children with attention-deficit/hyperactivity disorder. Behaviour Change, 19, 191-206. doi: 10.1375/bech.19.4.191 Hohmann, M., Banet, B., & Weikart, D. P. (1979). Young children in action: A manual for preschool educators. Ypsilanti, MI: High/Scope Press. Hohmann, M., & Weikart, D. P. (1995). Educating young children: Active learning practices for preschool and child care programs. Ypsilanti, MI: High/Scope Press. Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179. doi: 10.1177/001440290507100203 170 Horner, R. H., & Odom, S. L. (2014). Constructing single-case research designs: Logic and options. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case intervention research: Methodological and statistical advances (pp. 27-52). Washington, DC: American Psychological Association. Irvine, A. B., Biglan, A., Smolkowski, K., Metzler, C., & Ary, D. (1999). The effectiveness of a parenting skills program for parents of middle school students in small communities. Journal of Consulting and Clinical Psychology, 67, 811-825. doi: 10.1037//0022- 006x.67.6.811 Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19. doi: 10.1037//0022-006X.59.1.12 Jones, D. E., Greenberg, M., & Crowley, M. (2015). Early social-emotional functioning and public health: The relationship between kindergarten social competence and future wellness. American Journal of Public Health, 105, 2283-2290. doi: 10.2105/ajph.2015.302630 Kataoka, S. H., Zhang, L., & Wells, K. B. (2002). Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. American Journal of Psychiatry, 159, 1548–1555. doi: 10.1176/appi.ajp.159.9.1548 Kazdin, A. E. (1980). Acceptability of alternative treatments for deviant child behavior. Journal of Applied Behavior Analysis, 13, 259-273. doi: 10.1901/jaba.1980.13-259 Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). New York, NY: Oxford University Press. Kilgus, S. P., Riley-Tillman, T. C., Stichter, J. P., Schoemann, A. M., & Bellesheim, K. (2016). Reliability of direct behavior ratings – social competence (DBR-SC) data: How many ratings are necessary? School Psychology Quarterly, 31, 431-442. doi: 10.1037/spq0000128 Kiresuk, T. J., & Sherman, R. E. (1968). Goal attainment scaling: A general method for evaluating comprehensive community mental health programs. Community Mental Health Journal, 4, 443-453. doi: 10.1007/BF01530764 Kjøbli, J. & Ogden, T. (2012). A randomized effectiveness trial of brief parent training in primary care settings. Prevention Science, 13, 616-626. doi: 10.1007/s11121-012-0289-y 171 Knapp, T. R. (2016). Why is the one-group pretest-posttest design still used? Clinical Nursing Research, 25, 467-472. doi: 10.1177/1054773816666280 Koehler, M. J., & Levin, J. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. Psychological Methods, 3, 206-217. doi: 10.1037/1082-989X.3.2.206 Kratochwill, T. R. (2007). Preparing psychologists for evidence-based school practice: Lessons learned and challenges ahead. American Psychologist, 62, 829-844. doi: 10.1037/0003- 066X.62.8.829 Kratochwill, T. R., Elliott, S. N., Loitz, P. A., Sladeczek, I., & Carlson, J. S. (2003). Conjoint consultation using self-administered manual and videotape parent-teacher training: Effects on children’s behavioral difficulties. School Psychology Quarterly, 18, 269-302. doi: 10.1521/scpq.18.3.269.22574 Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case design technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26-38. doi: 10.1177/0741932512452794 Kratochwill, T. R., & Levin, J. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124-144. doi: 10.1037/a0017736 Kratochwill, T., & Stoiber, K. (2002). Evidence-based interventions in school psychology: Conceptual foundations of the procedural and coding manual of division 16 and society for the study of school psychology task force. School Psychology Quarterly, 17, 341-389. doi: 10.1521/scpq.17.4.341.20872 Lambert, M. J., & Ogles, B. M. (2009). Using clinical significance in psychotherapy outcome research: The need for a common procedure and validity data. Psychotherapy Research, 19, 493-501. doi: 10.1080/10503300902849483 Lamb-Parker, F., LeBuffe, P., Powell, G., & Halpern, E. (2008). A strengths-based, systemic mental health approach to support children’s social and emotional development. Infants and Young Children, 21, 45-55. doi: 10.1097/01.IYC.0000306372.40414.a7 172 Lanovaz, M. J., & Rapp, J. T. (2016). Using single-case experiments to support evidence-based decisions: How much is enough? Behavior Modification, 40, 377-395. doi: 10.1177/0145445515613584 LeBuffe, P. A., & Naglieri, J. A. (2012). Devereux Early Childhood Assessment for Preschoolers, second edition (DECA-P2): User’s guide and technical manual. Lewisville, NC: Kaplan Early Learning Company. LeBuffe, P. A., Ross, K. M., Fleming, J. L., & Naglieri, J. A. (2013). The Devereux Suite: Assessing and promoting resilience in children ages 1 month to 14 years. In S. Prince- Embury & D.H. Saklofske (Eds.), Resilience in children, adolescents, and adults: Translating research into practice (pp.45-59). New York: Springer. LeBuffe, P. A., Shapiro, V. B., & Naglieri, J. A. (2009). The Devereux Student Strengths Assessment (DESSA): Technical manual and user’s guide. Lewisville, NC: Kaplan. Lenz, A. S. (2013). Calculating effect size in single-case research: A comparison of nonoverlap methods. Measurement and Evaluation in Counseling and Development, 46, 64-73. doi: 10.1177/0748175612456401 Levin, J. R., Evmenova, A. S., & Gafurov, B. S. (2014). The single-case data-analysis ExPRT (Excel package of randomization tests). In T. R. Kratochwill & J. R. Levin (Eds.), Single- case intervention research: Methodological and statistical advances (pp. 185-220). Washington, DC: American Psychological Association. Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2016). Comparison of randomization-test procedures for single-case multiple-baseline designs. Developmental Neurorehabilitation. Advance online publication. doi: 10.1080/17518423.2016.1197708 Lewis, E. M., Feely, M., Seay, K. D., Fedoravicis, N., & Kohl, P. L. (2016). Child welfare involved parents and Pathways Triple P: Perceptions of program acceptability and appropriateness. Journal of Child and Family Studies, 25, 3760-3770. doi: 10.1007/s10826-016-0526-2 Linver, M. R., Brooks-Gunn, J., & Kohen, D. E. (2002). Family processes as pathways from income to young children’s development. Developmental Psychology, 38, 719-734. doi: 10.1037//0012-1649.38.5.719 173 Lovejoy, M. C., Weis, R., O’Hare, E., & Rubin, E. C. (1999). Development and initial validation of the Parent Behavior Inventory. Psychological Assessment, 11, 534-545. doi: 10.1037/1040-3590.11.4.534 Lowell, D. I., Carter, A. S., Godoy, L., Paulicin, B., & Briggs-Gowan, M. J. (2011). A randomized controlled trial of Child FIRST: A comprehensive home-based intervention translating research into early childhood practice. Child Development, 82, 193-208. doi: 10.1111/j.1467-8624.2010.01550.x Lundahl, B., Risser, H. J., & Lovejoy, M. C. (2006). A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review, 26, 86-104. doi: 10.1016/j.cpr.2005.07.004 Maaskant, A. M., van Rooij, F. B., Overbeek, G. J., Oort, F. J., Arntz, M., & Hermanns, J. M. A. (2017). Effects of PMTO in foster families with children with behavior problems: A randomized controlled trial. Journal of Child and Family Studies, 26, 523-539. doi: 10.1007/s10826-016-0579-2 Mackrain, M., & Cairone, K. B. (2013). Promoting resilience for now and forever: A family guide for supporting the social and emotional development of preschool children (2nd ed.). Lewisville, NC: Kaplan Early Learning Company. Marcon, R. A. (1999). Positive relationships between parent school involvement and public school inner-city preschoolers’ development and academic performance. School Psychology Review, 28, 395–412. Marcynyszyn, L. A., Maher, E. J., & Corwin, T. W. (2011). Getting with the (evidence-based) program: An evaluation of the Incredible Years parent training program in child welfare. Children and Youth Services Review, 33, 747-757. doi: 10.1016/j.childyouth.2010.11.021 Markie-Dadds, C., & Sanders, M. R. (2006). A controlled evaluation of an enhanced self- directed behavioural family intervention for parents of children with conduct problems in rural and remote areas. Behaviour Change, 23, 55-72. doi: 10.1375/bech.23.1.55 Marrs, R. W. (1995). A meta-analysis of bibliotherapy studies. American Journal of Community Psychology, 23, 843-870. doi: 10.1007/BF02507018 McCabe, P. C., & Altamura, M. (2011). Empirically valid strategies to improve social and emotional competence of preschool children. Psychology in the Schools, 48, 513-540. doi: 10.1002/pits.20570 174 McWayne, C., & Cheung, K. (2009). A picture of strength: Preschool competencies mediate the effects of early behavior problems on later academic and social adjustment for Head Start children. Journal of Applied Developmental Psychology, 30(3), 273-285. doi: 10.1016/j.appdev.2008.12.014 Menting, A. T. A., de Castro, B. O., & Matthys, W. (2013). Effectiveness of the Incredible Years parent training to modify disruptive and prosocial child behavior: A meta-analytic review. Clinical Psychology Review, 33, 901-913. doi: 10.1016/j.cpr.2013.07.006 Mihalic, S. F., & Elliott, D. S. (2015). Evidence-based programs registry: Blueprints for healthy youth development. Evaluation and Program Planning, 48, 124-131. doi: 10.1016/j.evalprogplan.2014.08.004 Morawska, A., Sanders, M., Goadby, E., Headley, C., Hodge, L., McAuliffe, C., …& Anderson, E. (2011). Is the Triple P-Positive parenting program acceptable to parents from culturally diverse backgrounds? Journal of Child and Family Studies, 20, 614-622. doi: 10.1007/s10826-010-9436-x Mortensen, J. A., & Mastergeorge, A. M. (2014). A meta-analytic review of relationship-based interventions for low-income families with infants and toddlers: Facilitating supportive parent-child interactions. Infant Mental Health Journal, 35, 336-353. doi: 10.1002/imhj.21451 Nederhof, A.J. (1985). Methods of coping with social desirability bias: A review. European Journal of Social Psychology, 15(3), 263-280. doi: 10.1002/ejsp.2420150303 Nores, M., Belfield, C. R., Barnett, W. S., & Schweinhart, L. (2005). Updating the economic impacts of the High/Scope Perry preschool program. Educational Evaluation and Policy Analysis, 27, 245-261. doi: 10.3102/01623737027003245 Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of Triple P-Positive parenting program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical Child and Family Psychology Review, 11, 114–144. doi: 10.1007/s10567-008-0033-0 O’Brien, R.W., D’Elio, M. A., Vaden-Kiernan, M., Magee, C., Younoszai, T., Keane, M. J.,…& Hailey, L. (2002). A descriptive study of Head Start families: FACES technical report I. Washington, DC: U.S. Department of Health and Human Services. 175 Odgers, C. L., Caspi, A., Russell, M. A., Sampson, R. J., Arseneault, L., & Moffitt, T. E. (2012). Supportive parenting mediates neighborhood socioeconomic disparities in children’s antisocial behavior from ages 5 to 12. Development and Psychopathology, 24, 705-721. doi: 10.1017/S0954579412000326 Ogden, T., & Amlund Hagen, K. (2008). Treatment effectiveness of parent management training in Norway: A randomized controlled trial of children with conduct problems. Journal of Consulting and Clinical Psychology, 76, 607-621. doi: 10.1037/0022-006X.76.4.607 Ogg, J., & Carlson, J. S. (2009). The self-administered incredible years parent training program: Perceived effectiveness, acceptability, and integrity with children exhibiting symptoms of attention-deficit/hyperactivity disorder. Journal of Evidence-Based Practices for Schools, 10, 143-166. Onghena, P., & Edgington, E. S. (2005). Customization of pain treatments: Single-case design and analysis. The Clinical Journal of Pain, 21, 56-68. doi: 10.1097/00002508- 200501000-00007 Orrell-Valente, J., Pinderhughes, E., Valente, E., Laird, R., & Conduct Problems Prevention Research Group. (1999). If it’s offered, will they come? Influences on parents’ participation in a community-based conduct problems prevention program. American Journal of Community Psychology, 27, 753-783. doi: 10.1023/a:1022258525075 Parker, R. I., Brossart, D. F., Vannest, K. J., Long, J. R., Garcia De-Alba, R., Baugh, F. G., & Sullivan, J. R. (2005). Effect sizes in single case research: How large is large? School Psychology Review, 34, 116-132. Parker, R. I., Cryer, J., & Byrns, G. (2006). Controlling baseline trend in single-case research. School Psychology Quarterly, 21, 418-443. doi: 10.1037/h0084131 Parker, R. I., & Hagan-Burke, S. (2007). Useful effect size interpretations for single case research. Behavior Therapy, 38, 95-105. doi: 10.1016/j.beth.2006.05.002 Parker, R. I., Vannest, K. J., & Davis, J. L. (2014). Non-overlap analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case intervention research: Methodological and statistical advances (pp. 127-152). Washington, DC: American Psychological Association. 176 Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42, 284-299. doi: 10.1016/j.beth.2010.08.006 Patterson, G. R. (1975). A social learning approach to family intervention. Eugene, OR: Castalia Publishing Company. Patterson, G. R. (2016). Coercion theory: The study of change. In T. J. Dishion & J. J. Snyder (Eds.), The oxford handbook of coercive relationship dynamics (pp. 7-22). New York, NY: Oxford University Press. Pearl, E. S. (2009). Parent management training for reducing oppositional and aggressive behavior in preschoolers. Aggression and Violent Behavior, 14, 295-205. doi: 10.1016/j.avb.2009.03.007 Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365-383. doi: 10.1093/clipsy.bpi045 Pidano, A. E., & Allen, A. R. (2015). The Incredible Years series: A review of the independent research base. Journal of Child and Family Studies, 24, 1898-1916. doi: 10.1007/s10826- 014-9991-7 Puckering, C. (2004). Parenting in social and economic adversity. In M. Hoghughi & N. Long (Eds.), Handbook of parenting: Theory and research for practice (pp. 38-54). Thousand Oaks, CA: SAGE Publications Inc. Qi, C. H., & Kaiser, A. P. (2003). Behavior problems of preschool children from low-income families: Review of the literature. Topics in Early Childhood Special Education, 23, 188- 216. doi: 10.1177/0271121403023004020 Ranganathan, P., Pramesh, C. S., & Buyse, M. (2015). Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspectives in Clinical Research, 6, 169-170. doi: 10.4103/2229-3485.159943 Reid, M. J., Webster-Stratton, C., & Baydar, N. (2004). Halting the development of conduct problems in Head Start children: The effects of parent training. Journal of Clinical Child & Adolescent Psychology, 33, 279-291. doi: 10.1207/s15374424jccp3302_10 177 Reid, M. J., Webster-Stratton, C., & Hammond, M. (2007). Enhancing a classroom social competence and problem-solving curriculum by offering parent training to families of moderate- to high-risk elementary school children. Journal of Clinical Child & Adolescent Psychology, 36, 605-620. doi: 10.1080/15374410701662741 Reitman, D., Currier, R. O., & Stickle, T. R. (2002). A critical evaluation of the Parenting Stress Index – Short Form (PSI-SF) in a Head Start population. Journal of Clinical Child & Adolescent Psychology, 31, 384-392. doi: 10.1207/S15374424JCCP3103_10 Reuben, J. D., Shaw, D. S., Neiderhiser, J. M., Natsuaki, M. N., Reiss, D., & Leve, L. D. (2016). Warm parenting and effortful control in toddlerhood: Independent and interactive predictors of school-age externalizing behavior. Journal of Abnormal Child Psychology, 44, 1083-1096. doi: 10.1007/s10802-015-0096-6 Reyno, S. M., & McGrath, P. J. (2006). Predictors of parent training efficacy for child externalizing behavior problems – a meta-analytic review. Journal of Child Psychology and Psychiatry, 47, 99-111. doi: 10.1111/j.1469-7610.2005.01544.x Ridgway, J. (2013). Single-case and small-n experimental designs: A practical guide to randomization tests. Research in Mathematics Education, 15, 208-212. doi: 10.1080/14794802.2013.797783 Riley-Tillman, T. C., Chafouleas, S. M., Christ, T., Briesch, A. M., & LeBel, T. J. (2009). The impact of item wording and behavioral specificity on the accuracy of direct behavior ratings (DBRs). School Psychology Quarterly, 24, 1-12. doi: 10.1037/a0015248 Riley-Tillman, T. C., Chafouleas, S. M., Sassu, K. A., Chanese, J. A., & Glazer, A. D. (2008). Examining the agreement of direct behavior ratings and systematic direct observation data for on-task and disruptive behavior. Journal of Positive Behavior Interventions, 10, 136-143. doi: 10.1177/1098300707312542 Riley-Tillman, T. C., Christ, T. J., Chafouleas, S. M., Boice-Mallach, C. H., & Briesch, A. (2011). The impact of observation duration on the accuracy of data obtained from direct behavior rating (DBR). Journal of Positive Behavior Interventions, 13, 119-128. doi: 10.1177/1098300710361954 Rimm-Kaufman, S. E., & Wanless, S. B. (2012). An ecological perspective for understanding the early development of self-regulatory skills, social skills, and achievement. In R. C. Pianta (Ed.), Handbook of early childhood education (pp. 299-323). New York, NY: The Guilford Press. 178 Roach, A. T., & Elliott, S. N. (2005). Goal attainment scaling: An efficient and effective approach to monitoring student progress. Teaching Exceptional Children, 37, 8-17. doi: 10.1177/004005990503700401 Roach, A. T., & Elliott, S. N. (2008). Best practices in facilitating and evaluating intervention integrity. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 195-208). Bethesda, MD: National Association of School Psychologists. Robinson, E. A., & Eyberg, S. M. (1981). The dyadic parent-child interaction coding system: Standardization and validation. Journal of Consulting & Clinical Psychology, 49, 245– 250. doi: 10.1037//0022-006x.49.2.245 Robinson, E. A., Eyberg, S. M., & Ross, A. W. (1980). The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child Psychology, 9, 22-29. doi: 10.1080/15374418009532938 Ross, S. G., & Begeny, J. C. (2014). Single-case effect size calculation: Comparing regression and non-parametric approaches across previously published reading intervention data sets. Journal of School Psychology, 52, 419-431. doi: 10.1016/j.jsp.2014.06.003 Rotheram-Borus, M. J., Swendeman, D., & Chorpita, B. F. (2012). Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist, 67, 463- 476. doi: 10.1037/a0028180 Rusby, J. C., Smolkowski, K., Marquez, B., & Taylor, T. K. (2008). A small-scale randomized efficacy trial of Carescapes: Enhancing children’s social development in child care homes. Early Childhood Research Quarterly, 23, 527-546. doi: 10.1016/j.ecresq.2008.08.003 Sanders, M. R. (1999). The Triple P-Positive parenting program: Towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clinical Child and Family Psychology Review, 2, 71–90. doi: 10.1023/A:1021843613840 Sanders, M. R. (2008). Triple P-Positive parenting program as a public health approach to strengthening parenting. Journal of Family Psychology, 22, 506-517. doi: 10.1037/0893- 3200.22.3.506 Sandler, I. N., Schoenfelder, E. N., Wolchik, S. A., MacKinnon, D. P. (2011). Long-term impact of prevention programs to promote effective parenting: Lasting effects but uncertain 179 processes. Annual Review of Psychology, 62, 299-329. doi: 10.1146/annurev.psych.121208.131619 Sanetti, L. M. H., Gritter, K. L., & Dobey, L. M. (2011). Treatment integrity of interventions with children in the school psychology literature from 1995 to 2008. School Psychology Review, 40(1), 72-84. Schaffer, H. R. (2006). Key concepts in developmental psychology. Thousand Oaks, CA: SAGE Publications Inc. Schlosser, R. W. (2004). Goal attainment scaling as a clinical measurement technique in communication disorders: A critical review. Journal of Communication Disorders, 37, 217-239. doi: 10.1016/j.jcomdis.2003.09.003 Schweinhart, L. J. (2010). The challenge of the HighScope Perry preschool study. In A. J. Reynolds, A. J. Rolnick, M. E. Englund, & J. A. Temple (Eds.), Childhood programs and practices in the first decade of life: A human capital integration (pp. 157-167). New York, NY: Cambridge University Press. Schweinhart, L. J. (2013). Long-term follow-up of a preschool experiment. Journal of Experimental Criminology, 9, 389-409. doi: 10.1007/s11292-013-9190-3 Schweinhart, L. J., & Weikart, D. P. (1997). The High/Scope preschool curriculum comparison study through age 23. Early Childhood Research Quarterly, 12, 117-143. doi: 10.1016/s0885-2006(97)90009-0 Scogin, F., Bynum, J., Stephens, G., & Calhoon, S. (1990). Efficacy of self-administered treatment programs: A meta-analytic review. Professional Psychology: Research and Practice, 21, 42–47. doi: 10.1037/0735-7028.21.1.42 Shadish, W. R. (2014). Analysis and meta-analysis of single-case designs: An introduction. Journal of School Psychology, 52, 109-122. doi: 10.1016/j.jsp.2013.11.009 Shefler, G., Canetti, L., & Wiseman, H. (2001). Psychometric properties of goal attainment scaling in the assessment of Mann’s time-limited psychotherapy. Journal of Clinical Psychology, 57, 971-979. doi: 10.1002/jclp.1063 Shepard, S. A., & Dickstein, S. (2009). Preventative intervention for early childhood behavioral problems: An ecological perspective. Child and Adolescent Psychiatric Clinics of North America, 18, 687-706. doi: 10.1016/j.chc.2009.03.002 180 Sheridan, S. M. (2014). Single-case designs and large-N studies: The best of both worlds. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case intervention research: Methodological and statistical advances (pp. 299-308). Washington, DC: American Psychological Association. Sheridan, S. M., Knoche, L. L., Edwards, C. P., Kupzyk, K. A., Clarke, B. L., & Kim, E. M. (2014). Efficacy of the Getting Ready intervention and the role of parental depression. Early Education and Development, 25, 746-769. doi: 10.1080/10409289.2014.862146 Shernoff, E. S., Hill, C., Danis, B., Leventhal, B. L., & Wakschlag, L. S. (2014). Integrative consensus: A systematic approach to integrating comprehensive assessment data for young children with behavior problems. Infants & Young Children, 27, 92-110. doi: 10.1097/IYC.0000000000000008 Shonkoff, J. P., & Phillips, D. A. (2000). From neurons to neighborhoods: The science of early childhood development. Washington, DC: National Academies Press. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428. doi: 10.1037//0033-2909.86.2.420 Sitnick, S. L., Shaw, D. S., Gill, A., Dishion, T., Winter, C., Waller, R., …& Wilson, M. (2015). Parenting and the family check-up: Changes in observed parent-child interaction following early childhood intervention. Journal of Clinical Child & Adolescent Psychology, 44, 970-984. doi: 10.1080/15374416.2014.940623 Smith, G. T., LeBuffe, P. A., Alleyne, D., Mackrain, M., & Likins, L. (2014). Bringing a resilience perspective to children in the child welfare system: A curriculum for caregivers. In S. Prince-Embury & D. H. Saklofske (Eds.), Resilience interventions for youth in diverse populations (pp. 159-179). New York, NY: Springer Science+Business Media. Snyder, J. J. (2016). Coercive family processes and the development of child social behavior and self-regulation. In T. J. Dishion & J. J. Snyder (Eds.), The oxford handbook of coercive relationship dynamics (pp. 101-113). New York, NY: Oxford University Press. Spirrison, C. L., Noland, K. A., & Savoie, L. B. (1992). Factor structure of the Treatment Evaluation Inventory: Implications for measurement of treatment acceptability. Journal of Psychopathology and Behavioral Assessment, 14, 65-79. doi: 10.1007/BF00960092 181 Spoth, R., Redmond, C., Hockaday, C., & Shin, C. (1996). Barriers to participation in family skills preventive interventions and their evaluations: A replication and extension. Family Relations, 45, 247–254. doi: 10.2307/585496 Stattin, H., Enebrink, P., Özdemir, M., & Giannotta, F. (2015). A national evaluation of parenting programs in Sweden: The short-term effects using an RCT effectiveness design. Journal of Consulting and Clinical Psychology, 83, 1069-1084. doi: 10.1037/a0039328 Stemler, S. E., & Tsai, J. (2011). Best practices in interrater reliability: Three common approaches. In J. Osborne (Ed.), Best practices in quantitative methods (pp. 29-49). Thousand Oaks, CA: SAGE Publications. Sterling-Turner, H. E., & Watson, T. S. (2002). An analog investigation of the relationship between treatment acceptability and treatment integrity. Journal of Behavioral Education, 11(1), 39-50. doi: 10.1023/A:1014333305011 Stewart, L. S., & Carlson, J. S. (2010). Investigating parental acceptability of the Incredible Years self-administered parent training program for children presenting externalizing behavior problems. Journal of Applied School Psychology, 26, 162-175. doi: 10.1080/15377901003709369 Stolee, P., Rockwood, K., Fox, R. A., & Streiner, D. L. (1992). The use of goal attainment scaling in a geriatric care setting. Journal of the American Geriatric Society, 40, 574-578. doi: 10.1111/j.1532-5415.1992.tb02105.x Tarver, J., Daley, D., Lockwood, J., & Sayal, K. (2014). Are self-directed parenting interventions sufficient for externalising behaviour problems in childhood? A systematic review and meta-analysis. European Child & Adolescent Psychiatry, 23, 1123-1137. doi: 10.1007/s00787-014-0556-5 Tate, R. L., Perdices, M., Rosenkoetter, U., McDonald, S., Togher, L., Shadish, W., … & Vohra, S. (2016). The single-case reporting guideline in behavioural interventions (SCRIBE) 2016: Explanation and elaboration. Archives of Scientific Psychology, 4, 10-31. doi: 10.1037/arc0000027 Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53-55. doi: 10.5116/ijme.4dfb.8dfd 182 The Incredible Years. (2013). Theoretical assumptions. Retrieved from http://incredibleyears.com/for-researchers/ Thomson, R. N., & Carlson, J. S. (2016). A pilot study of a self-administered parent training intervention for building preschoolers’ social-emotional competence. Early Childhood Education Journal, 45, 419-426. doi: 10.1007/s10643-016-0798-6 Turner-Stokes, L. (2009). Goal attainment scaling (GAS) in rehabilitation: A practical guide. Clinical Rehabilitation, 23, 362-370. doi: 10.1177/0269215508101742 U.S. Department of Health and Human Services. (2015). Head start program performance standards. Retrieved from http://eclkc.ohs.acf.hhs.gov/hslc/standards/hspps/45-cfr- chapter-xiii/45-cfr-chap-xiii-eng.pdf U.S. Department of Health and Human Services. (2016). Annual update of the HHS poverty guidelines. Federal Register, 81, 4036-4037. Vagias, W. M. (2006). Likert-type response anchors. Retrived from http://www.peru.edu/oira/wp-content/uploads/sites/65/2016/09/Likert-Scale- Examples.pdf Vannest, K. J., Parker, R. I., Gonen, O., & Adiguzel, T. (2016). Single case research: Web based calculators for SCR analysis (Version 2.0) [Web-based application]. Available from singlecaseresearch.org Vick Whittaker, J. E., Harden, B. J., See, H. M., Meisch, A. D., & Westbrook, T. R. (2011). Family risks and protective factors: Pathways to Early Head Start toddlers’ social- emotional functioning. Early Childhood Research Quarterly, 26, 74-86. doi: 10.1016/j.ecresq.2010.04.007 Villarreal, V., Ponce, C., & Gutierrez, H. (2015). Treatment acceptability of interventions published in six school psychology journals. School Psychology International, 36, 322- 332. doi: 10.1177/0143034315574153 Von Brock, M. B., & Elliott, S. N. (1987). Influence of treatment effectiveness on the acceptability of classroom interventions. Journal of School Psychology, 25, 131-144. doi: 10.1016/0022-4405(87)90022-7 183 Wainer, A., & Ingersoll, B. (2013). Intervention fidelity: An essential component for understanding ASD parent training research and practice. Clinical Psychology: Science and Practice, 20, 335-357. doi: 10.1111/cpsp.12045 Wainer, A. L., & Ingersoll, B. R. (2015). Increasing access to an ASD imitation intervention via a telehealth parent training program. Journal of Autism and Developmental Disorders, 45, 3877-3890. doi: 10.1007/s10803-014-2186-7 Wampold, B. & Worsham, N. (1986). Randomization tests for multiple-baseline designs. Behavioral Assessment, 8, 135-143. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063-1070. doi: 10.1037/0022-3514.54.6.1063 Watson, P., & Workman, E. (1981). The non-concurrent multiple baseline across-individuals design: An extension of the traditional multiple baseline design. Journal of Behavior Therapy and Experimental Psychiatry, 12, 257- 259. doi: 10.1016/0005-7916(81)90055-0 Webster-Stratton, C. (1990). Enhancing the effectiveness of self-administered videotape parent training for families with conduct-problem children. Journal of Abnormal Child Psychology, 18, 479-492. doi: 10.1007/BF00911103 Webster-Stratton, C. (1992). Individually administered videotape parent training: “Who benefits?” Cognitive Therapy and Research, 16, 31-35. doi: 10.1007/BF01172955 Webster-Stratton, C. (1998). Preventing conduct problems in Head Start children: Strengthening parent competencies. Journal of Consulting and Clinical Psychology, 66, 715-730. doi: 10.1037/0022-006X.66.5.715 Webster-Stratton, C. (2006). The Incredible Years preschool basic parenting program. Seattle, WA: Incredible Years Press. Webster-Stratton, C., & Hammond, M. (1998). Conduct problems and level of social competence in Head Start children: Prevalence, pervasiveness, and associated risk factors. Clinical Child Psychology and Family Psychology Review, 1, 101-124. doi: 10.1023/A:1021835728803 Webster-Stratton, C., Kolpacoff, M., & Hollinsworth, T. (1988). Self-administered videotape therapy for families with conduct-problem children: Comparison with two cost-effective 184 treatments and a control group. Journal of Consulting and Clinical Psychology, 56, 558- 566. doi: 10.1037/0022-006X.56.4.558 Webster-Stratton, C., & Reid, M. J. (2003). Treating conduct problems and strengthening social and emotional competence in young children: The Dina Dinosaur treatment program. Journal of Emotional and Behavioral Disorders, 11, 130-143. doi: 10.1177/10634266030110030101 Webster-Stratton, C., & Reid, M. J. (2004). Strengthening social and emotional competence in young children – the foundation for early school readiness and success: Incredible years classroom social skills and problem-solving curriculum. Infants and Young Children, 17, 96-113. doi: 10.1097/00001163-200404000-00002 Webster-Stratton, C., & Reid, M. J. (2010). The incredible years parents, teachers, and children training series: A multifaceted approach for young children with conduct problems. In A.E. Kazdin & J.R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 194-210). New York: Guilford Press. Webster-Stratton, C., Reid, M.J., & Hammond, M. (2001). Preventing conduct problems, promoting social competence: A parent and teacher training partnership in Head Start. Journal of Clinical Child Psychology, 30, 283-302. doi: 10.1207/s15374424jccp3003_2 Webster-Stratton, C., Reid, M. J., & Hammond, M. (2004). Treating children with early-onset conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child & Adolescent Psychology, 33, 105-124. doi: 10.1207/S15374424JCCP3301_11 Webster-Stratton, C., & Taylor, T. (2001). Nipping early risk factors in the bud: Preventing substance abuse, delinquency, and violence in adolescence through interventions targeted at young children (ages 0-8 years). Prevention Science, 2, 165-192. Weikart, D. P., Rogers, L., Adcock, C., & McClelland, D. (1971). The cognitively oriented curriculum: A framework for preschool teachers. Urbana, IL: University of Illinois. Witt, J. C., & Elliott, S. N. (1985). Acceptability of classroom intervention strategies. In T. R. Kratochwill (Ed.), Advances in school psychology (Vol. 4, pp. 251-288). Hillsdale, NJ: Erlbaum. Witt, J. C., Elliott, S. N., & Martens, B. K. (1984). Acceptability of behavioral interventions used in classrooms: The influence of amount of teacher time, severity of behavior problem, 185 and type of intervention. Behavioral Disorders, 9, 95-104. doi: 10.1177/019874298400900211 Wright, M. O., & Masten, A. S. (2005). Resilience processes in development: Fostering positive adaptation in the context of adversity. In S. Goldstein & R.B. Brooks (Eds.), Handbook of resilience in children (pp. 17-37). New York, NY: Springer. Wright, M. O., Masten, A. S., & Narayan, A. J. (2013). Resilience processes in development: Four waves of research on positive adaptation in the context of adversity. In S. Goldstein & R. B. Brooks (Eds.), Handbook of resilience in children (2nd ed., pp. 15-37). New York, NY: Springer. Wyatt Kaminski, J., Valle, L. A., Filene, J. H., & Boyle, C. L. (2008). A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology, 36, 567-589. doi: 10.1007/s10802-007-9201-9 Yeung, W. J., Linver, M. R., & Brooks-Gunn, J. (2002). How money matters for young children’s development: Parental investment and family processes. Child Development, 73, 1861-1879. doi: 10.1111/1467-8624.t01-1-00511 Zemp, M., Merrilees, C. E., & Bodenmann, G. (2014). How much positivity is needed to buffer the impact of parental negativity on children? Family Relations, 63, 602-615. doi: 10.1111/fare.12091 186