MODIFIED PARENT-CHILD INTERACTION THERAPY TO ADDRESS SUBCLINICAL LEVELS OF BEHAVIORAL CONCERNS: A NONCONCURRENT MULTIPLE PROBE ACROSS PARTICIPANTS DESIGN By Nicole E. Mathes A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of School Psychology –Doctor of Philosophy 2021 ABSTRACT MODIFIED PARENT-CHILD INTERACTION THERAPY TO ADDRESS SUBCLINICAL LEVELS OF BEHAVIORAL CONCERNS: A NONCONCURRENT MULTIPLE PROBE ACROSS PARTICIPANTS DESIGN By Nicole E. Mathes Externalizing behavior problems are the most common referral for young child mental health services (Egger & Angold, 2006) and are associated with poor long-term outcomes (e.g., later conduct problems, peer conflict, social maladjustment, and high school dropout). Thus, early intervention for preschool children who are exhibiting behavior concerns is imperative for the prevention of more severe behavior problems. Interventions for young child behavior problems are typically aimed at improving parenting practices; however, children who are at-risk for the development of clinically significant behavior problems are likely to exhibit emerging (i.e., subclinical) levels of behavior during the preschool years. Therefore, there is potential for a brief, preventive method to reduce the need for more time and resource-intensive supports in the future. One way to develop a preventive approach to service delivery and improve outcomes for at-risk children is to adapt an evidence-based parent training intervention designed for children with significant behavioral concerns. Parent-Child Interaction Therapy (PCIT) is one such intervention that has particularly strong empirical support in the treatment of child behavior problems. The present study utilized a nonconcurrent multiple-probe across participants design to examine the fidelity, efficacy, and acceptability of a modified version of PCIT in increasing positive parenting skills and decreasing parent-rated child externalizing behaviors with four mothers and their preschool-aged children who demonstrated emerging problem behaviors. Results indicated that the intervention was implemented with excellent adherence (i.e., average = 97%). Visual analysis revealed at least three demonstrations of a treatment effect for all three positive parenting skills (i.e., improvement in Labeled Praises, Behavioral Descriptions, Reflections), as well as for a reduction in child behavior problems. Tau-U analyses indicated moderate to large effects for all four dyads’ use of Labeled Praises, large effects for Behavioral Descriptions across three dyads, and a large effect for one dyad’s use of Reflections. Large effects for a reduction in observed child behavior problems were found for two children, and standardized ratings of child behavior problems fell below the sub-clinical range for three children following intervention implementation. Additionally, mothers reported high levels of satisfaction with the intervention. Keywords: early intervention, prevention, parent training, externalizing behaviors, single- case design ACKNOWLEDGEMENTS Words cannot fully express the amount of gratitude I have for the people who have supported me through this marathon of graduate school. This challenging and rewarding journey would not have been possible without the love, guidance, patience, and encouragement of so many wonderful people. First, I would like to thank my advisor, Dr. Kristin Rispoli, for her constant encouragement, patience, support, and mentorship. She welcomed me to her research team, was open to my research interests, and taught me what quality, meticulous research looks like. I am a better scholar because of my time on the FSC Lab. Through many meetings and dissertation drafts, Dr. Rispoli helped me to develop, and eventually refine, both my research and clinical interests essentially from scratch. She continually encouraged me to think critically and apply research to practice. She never gave up on me as a student and helped support me across the finish line. Dr. Rispoli: Thank you for cheering me on and believing in me. I would like to express my appreciation to my dissertation committee, Dr. Jodene Fine, Dr. Marisa Fisher, Dr. Evelyn Oka, and Dr. Martin Volker, for their feedback, flexibility, guidance, knowledge, and understanding. I so appreciate their willingness to review many modifications to the study and their understanding of the challenges of implementing a dissertation study during a global pandemic. Thank you all for guiding me during this challenging process. I would also like to thank Dr. Matthew Brodhead for his instruction in single-case design methodology, for reading every draft of my dissertation proposal, and for his detailed feedback to ensure I implemented a high-quality dissertation study. Thank you for your genuine interest in my learning, the positive feedback, and the fun stickers left on hardcopies of edited drafts. iv This study would not have been possible without Nick Warren, my research assistant. Thank you for conducting fidelity checks and inter-rater observations for parenting and child behaviors. Your diligence in training and learning these responsibilities were impressive and much appreciated. Additionally, I would not have been able to complete my graduate school journey without my friends. To my School Psychology cohort (Justina, Chim, and Rachel): thank you for the writing sessions, game nights, work-life balance adventures, laughter, and celebrations of big and small milestones. To Rick, Adrianna, and Amy: thank you for the frequent check-ins, mentorship, guidance, encouragement, words of wisdom, validation, continual humor and positivity, for pushing me outside of my comfort zone and challenging me in my thinking, and most of all, for your friendship. To my Internship cohort (Emily, Katie, Amy, and Anna): I could not imagine a better group to navigate this crazy year with. Thank you for the support, words of encouragement and reminders to keep going, weekend work sessions, potluck dinners, movie nights, and beach days. To all my friends (I trust you know who you are!): thank you for loving me and supporting me through my graduate school journey, whether that was through words of encouragement, a visit, a hug, a cup of coffee, a meal, lending an ear to listen, or physically sitting with me to ensure I got my dissertation drafts done. Thank you for the reminder that there is life outside of school. Lastly, to my brothers (Kyle and Mark) and to my parents: a HUGE thank you for your unconditional love, support, and faith in my abilities. Mom and Dad: thank you for walking alongside me, even when it was not easy, and for encouraging me to continue on this long journey. I am blessed to have you both as parents. v TABLE OF CONTENTS LIST OF TABLES ......................................................................................................................... ix LIST OF FIGURES ....................................................................................................................... xi CHAPTER I. INTRODUCTION .................................................................................................... 1 Significance of the Problem ..................................................................................................................... 1 Theoretical Framework ............................................................................................................................ 4 Attachment Theory ........................................................................................................................... 4 Social Learning Theory .................................................................................................................... 7 Purpose of this Study ................................................................................................................................ 8 CHAPTER II. LITERATURE REVIEW ..................................................................................... 10 Historical Background ............................................................................................................................ 10 Child Risk Factors .................................................................................................................................. 11 Public Health Model ............................................................................................................................... 13 Positive Parenting Practices.................................................................................................................... 17 Parent Training Interventions ................................................................................................................. 20 Parent Management Training –Oregon Model ............................................................................... 23 Incredible Years Parent Training Program ..................................................................................... 27 Triple-P Positive Parenting Program .............................................................................................. 31 Parent-Child Interaction Therapy ................................................................................................... 37 Summary of Evidence-Based Training Programs .................................................................................. 41 Parent-Child Interaction Therapy (PCIT) ............................................................................................... 42 PCIT Structure and Content ........................................................................................................... 42 Detailed CDI Components ............................................................................................................. 45 Adaptations to Standard PCIT ................................................................................................................ 48 The Present Study ................................................................................................................................... 51 Rationale for Modified PCIT.......................................................................................................... 52 Modifications Due to Covid-19 ...................................................................................................... 53 Proposed Single-Case Design......................................................................................................... 55 Research Questions and Hypotheses ...................................................................................................... 56 Research Question 1 ....................................................................................................................... 56 Research Question 2 ....................................................................................................................... 57 Research Question 3 ....................................................................................................................... 58 Research Question 4 ....................................................................................................................... 60 CHAPTER III. METHOD ............................................................................................................ 61 Participants ............................................................................................................................................. 61 Recruitment Process ....................................................................................................................... 61 Recruitment .................................................................................................................................... 62 Inclusion and Exclusion Criteria .................................................................................................... 63 Participant Descriptions.................................................................................................................. 65 Constructs and Measures ........................................................................................................................ 68 Behavioral Risk .............................................................................................................................. 68 Positive Parenting Skills ................................................................................................................. 69 Externalizing Behaviors ................................................................................................................. 72 vi Integrity of Intervention Implementation ....................................................................................... 75 Parental Satisfaction ....................................................................................................................... 77 Research Design ..................................................................................................................................... 78 Procedures .............................................................................................................................................. 79 Study Personnel and Training ........................................................................................................ 80 Study Phases ................................................................................................................................... 81 Data Analysis.......................................................................................................................................... 89 Visual Analysis............................................................................................................................... 89 Effect Size ...................................................................................................................................... 91 Analysis by Research Question .............................................................................................................. 94 Research Question 1 ....................................................................................................................... 94 Research Question 2 ....................................................................................................................... 95 Research Question 3 ....................................................................................................................... 96 Research Question 4 ....................................................................................................................... 98 CHAPTER IV. RESULTS ............................................................................................................ 99 Research Question 1 ............................................................................................................................... 99 Positive Parenting Skills ................................................................................................................. 99 Negative Parenting Skills ............................................................................................................. 105 Research Question 2 ............................................................................................................................. 111 ECBI Ratings ................................................................................................................................ 111 Direct Child Behavioral Observation ........................................................................................... 117 BASC-3 Ratings ........................................................................................................................... 118 Summary of Results ............................................................................................................................. 119 Research Question 3 ............................................................................................................................. 120 Adherence..................................................................................................................................... 120 Dosage .......................................................................................................................................... 121 Quality of Intervention Delivery .................................................................................................. 121 Participant Responsiveness .......................................................................................................... 122 Research Question 4 ............................................................................................................................. 123 CHAPTER V. DISCUSSION ..................................................................................................... 127 Purpose of Study................................................................................................................................... 127 Parenting Skills..................................................................................................................................... 128 Positive Parenting Skills ............................................................................................................... 128 Summary of Positive Parenting Skill Changes ............................................................................. 133 Negative Parenting Skills ............................................................................................................. 135 Child Behavior...................................................................................................................................... 138 Summary of Child Behavior Outcomes ....................................................................................... 142 Intervention Fidelity ............................................................................................................................. 142 Adherence..................................................................................................................................... 142 Dosage .......................................................................................................................................... 143 Quality of Intervention Delivery .................................................................................................. 143 Participant Responsiveness .......................................................................................................... 144 Satisfaction ........................................................................................................................................... 145 Limitations and Implications for Future Research ............................................................................... 146 Lessons Learned from Telehealth Service Delivery............................................................................. 151 Implications for Practice....................................................................................................................... 153 Conclusion ............................................................................................................................................ 158 vii APPENDICES ............................................................................................................................ 159 Appendix A CDI Mastery Criteria ....................................................................................................... 160 Appendix B PCIT Trainer Levels......................................................................................................... 161 Appendix C PRIDE Skills Handout ..................................................................................................... 164 Appendix D Toys for CDI Handout ..................................................................................................... 165 Appendix E PCIT, Standard and Modified Versions ........................................................................... 166 Appendix F Treatment Integrity Checklist for Pre-Treatment Assessments ........................................ 169 Appendix G Treatment Integrity Checklist for CDI Teach Session .................................................... 170 Appendix H Treatment Integrity Checklist for CDI Coach 1............................................................... 171 Appendix I Treatment Integrity Checklist for CDI Coach 2 ................................................................ 172 Appendix J Treatment Integrity Checklist for CDI Coach 3 ................................................................ 173 Appendix K Treatment Integrity Checklist for CDI Coach 4 and Beyond .......................................... 174 Appendix L Treatment Integrity Checklist for Post-Treatment and Graduation Session..................... 175 Appendix M Treatment Integrity Checklist for Follow-Up Session .................................................... 176 Appendix N CDI Homework Sheet ...................................................................................................... 177 Appendix O DPICS In-Session CDI Coding Sheet .............................................................................. 178 Appendix P Child Behavior Observation In-Session CDI Coding Sheet ............................................. 179 REFERENCES ........................................................................................................................... 180 viii LIST OF TABLES Table 1. General Outline of PCIT CDI Sessions ………………………………………………. 43 Table 2. Descriptive Statistics for Positive Parenting Skills …………………………………...104 Table 3. Percentage of All Non-Overlapping Data for Positive Parenting Skills ……………. 105 Table 4. Descriptive Statistics for Negative Parenting Skills ………………………………….110 Table 5. Percentage of All Non-Overlapping Data for Negative Parenting Skills …………….110 Table 6. Descriptive Statistics for ECBI ………………………………….................................116 Table 7. Percentage of All Non-Overlapping Data for ECBI ………………………………….117 Table 8. Partial Interval Coding Child Behavioral Observation ……………………………….117 Table 9. BASC-3 Externalizing Composite Pre-Post Scores …………………………………. 118 Table 10. Visual Analysis and Effect Size Calculations Results …..…………………………..119 Table 11. Adherence to Treatment Protocol for CDIT ………………………………….……..121 Table 12. Quality of Intervention Delivery ………………………………….…………………122 Table 13. Participant Responsiveness: Homework Completion ……………………………….123 Table 14. Parent-Rated Intervention Satisfaction, Acceptability, and Feasibility for CDIT...…123 Table 15. CDI Mastery Criteria...………………………………………………………………160 Table 16. PCIT Trainer Levels...……………………………………………………………….161 Table 17. Pride Skills Handout...……………………………………………………………….164 Table 18. PCIT, Standard and Modified Versions…………………………………………...…166 Table 19. Pre-Treatment Integrity Checklist………………………………………………...…169 Table 20. CDI Teach Integrity Checklist……..……………………………………………...…170 Table 21. CDI Coach 1 Integrity Checklist………………………………………………...…..171 ix Table 22. CDI Coach 2 Integrity Checklist………………………………………………...…..172 Table 23. CDI Coach 3 Integrity Checklist………………………………………………...…..173 Table 24. CDI Coach 4 and Beyond Integrity Checklist……………………………………….174 Table 25. Post-Treatment and Graduation Session Integrity Checklist…………………...……175 Table 26. Follow-Up Session Integrity Checklist…………………………………………..…..176 Table 27. CDI Homework Sheet…………………….……………………………………...…..177 Table 28. DPICS Treatment Session ………………………………...……...………………....178 Table 29. DPICS Categories………………………………………………...……………….....178 Table 30. Child Behavior Observation Treatment Session……………………………………..179 Table 31. Child Behavior Observation Coding ………………………………..…………...…..179 x LIST OF FIGURES Figure 1. Positive Parenting Skills ……………………………………………………………. 103 Figure 2. Dyad 4 Father Positive Parenting Skills ……………………………………………. 104 Figure 3. Negative Parenting Skills ……………………………………………………………109 Figure 4. Dyad 4 Father Negative Parenting Skills ……………………………………………110 Figure 5. Child Behavior Ratings …………………………………………………………….. 115 Figure 6. Dyad 4 Father Child Behavior Ratings ……………………………………………...116 xi CHAPTER I. INTRODUCTION Significance of the Problem Research suggests that there is a strong need for addressing behavior problems in preschool children (Chacko et al., 2018). Disruptive behavior is characterized by observable, non-preferred actions presenting as noncompliance, opposition, aggression, and hyperactivity (Campbell, 2000; Ogundele, 2018; Tremblay, 2010). Between 10% and 15% of preschool children exhibit mild to moderate behavioral difficulties (Campbell, 2000). According to the U.S. Department of Health and Human Services (2001), one in five children and adolescents have a diagnosable emotional, behavioral, or mental health problem during their school years that significantly impairs how they function at home, school, or in the community. More specifically, between 9 and 14% of children ages birth to five years old experience social emotional problems that negatively influence their functioning, development, and school readiness (Kessler et al., 2005). Despite research that supports prevention and early intervention, most prevention efforts target older children and adolescents (Tremblay, 2010). This is problematic, as preschool years are a critical development period for the prevention of long-term difficulties. Childhood disorders are becoming increasingly recognized as an area of major public health concern, with several studies reporting higher prevalence rates in the early childhood population (Bufferd et al., 2012). Disruptive behavior disorders, including Oppositional Defiant Disorder (ODD), Conduct Disorder (CD), and Attention-Deficit/Hyperactivity Disorder (ADHD) affect as many as 16% of children and are the most common referral reasons for treatment in young children (National Academies of Sciences, 2016). Disruptive behaviors in early childhood, including oppositional, aggressive, and hyperactive behaviors, are also predictive of negative mental health outcomes in later life, including peer conflict, social maladjustment, 1 school failure, substance abuse, and criminality (Bellanti & Bierman, 2000; Vitaro, et al., 2005). Furthermore, the cost of untreated disruptive behavior disorders to caregivers and to society is high. First, such disorders place significant strain on parent mental health (Evans, Sibley, & Serpell, 2009), which often results in a need for individual treatment for parents. Second, there is a financial burden created for schools, public health agencies, and society (Insel, 2008). For example, one study estimated that up to $2.3 million dollars per child could be saved if successful interventions were implemented for at-risk youth who demonstrate disruptive behaviors (Cohen, 1998) due to the estimated costs of a young offender delinquent to society, such as expenses associated with parental mental health, the juvenile justice or prison system (e.g., mean number of offenses, cost of the investigation and verdict, incarceration, foregone earnings), and the costs associated with the crime (e.g., cost to the individual victim and/or public). The cost per child is likely higher today, given that this study occurred almost 20 years ago. In another study examining the public costs resulting from youths’ involvement in the juvenile justice system, child welfare, and special education systems over a seven-year period, the average total public cost was over $41,000 for individuals with CD and over $23,000 for individuals with ODD (Foster & Jones, 2005). Preventative efforts and early intervention may reduce the frequency of future criminal offenses and the number of individuals in the juvenile justice and prison systems (Cohen & Piquero, 2009). Early and accurate identification of children who are likely to develop more severe behavioral problems is crucial so that prevention efforts can be maximized. For young children who are at greater risk for such problems, early intervention is critical (Hess et al., 2012) and is often directed at parenting practices. According to a recent review of treatments for disruptive behaviors in children, interventions that aim to improve parenting practices are most effective 2 (Kaminski & Claussen, 2017). There is much research on the effectiveness of behavioral interventions that aim to teach behavioral management strategies and responsive strategies to train parents of children with externalizing behavior difficulties. These interventions focus on relationship enhancement within the parent-child dyad and attempt to increase positive interactions between the parent and child, while also aiming to reduce coercive and inconsistent parenting practices by providing direct coaching of parenting skills (Dekovic et al., 2010). Although such parenting interventions are effective in reducing externalizing behavior in children, these interventions are typically intensive, aim to treat children with clinically significant behaviors, and often focus on treating the individual child and the parent (Berkovits et al., 2010). Despite research that supports prevention efforts, mental health services are typically targeted towards children who have been identified as having a clinically significant (i.e., severe) problem, rather than subclinical (i.e., emerging) problems. Although this traditional service delivery model directs services toward those who need them, it is a reactive approach. Additionally, if an increasing number of children require intensive services, the service delivery system will not be able to provide adequate and appropriate services to address such clinically significant problems. The provision of services can be made more efficient by increasing prevention efforts and, consequently, reducing the number of children who require more intensive intervention. Given that early identification and intervention of disruptive behavior problems are associated with better mental health outcomes (Hess et al., 2012), there is a need to consider a preventative approach to service delivery. Due to the significant role that parents play in the healthy development of children, research on parent training interventions for the treatment of disruptive behaviors, and strong empirical support for prevention efforts, it is critical to develop 3 and test feasible parent training prevention programs to reduce risk for long-term behavioral difficulties. Theoretical Framework A child’s healthy development is a function of the stability, security, and consistency of trusting relationships that are developed during the early years (Lezin et al., 2004). Additionally, parent-child interactions and the modeling of appropriate prosocial behavior reduce the likelihood of challenging behavior (Fox, et al., 2003). Theories of attachment and social learning, together, provide a conceptual framework for how various factors interact within the parent-child dyad to influence the development of children. Attachment Theory Research on the parent-child relationship has foundations in attachment theory (Ainsworth & Bowlby, 1991). Attachment theory emphasizes the nature of the relationship between children and their parents and posits that a child’s first attachment experience profoundly shapes their social, cognitive, and emotional development (Bowlby, 1969). Patterns of attachment develop within the context of several everyday interactions between an infant and her/his attachment figure. Attachment refers to staying safe, gaining comfort, and seeking predictability and is also one aspect of the parent-child relationship. A child’s social-emotional health is affected by the formation, maintenance, disruption, and renewal of attachment relationships (Benoit, 2004). The relationships and patterns of interactions formed during the early stages of a child’s life serve as a foundation for interactions later in life (Hong & Park, 2012). The way a parent responds to a child’s needs creates a sort of “template” or schema. This template guides how the child recognizes and responds to her/his own emotions, as well as how he/she interacts with her/his parent and other adults. 4 The type of attachment relationship that a parent and child form early on is related to later child outcomes. According to Ainsworth and Bowlby (1991), there are four types of infant- parent attachment: secure, insecure-avoidant, insecure-resistant, and disorganized. The quality of attachment that an infant develops with the parent is determined by the parent’s response to the infant when her/his feelings of safety and security are threatened (Benoit, 2004). Secure attachments have been associated with positive outcomes for children. A secure attachment is characterized by parents who consistently respond to the child’s distress in sensitive or affectionate ways (e.g., picking up the child, calming the child) so that the child feels secure in their knowledge that they can freely express a negative emotion and the parent will provide comfort. Children who form secure attachments with their parents seek proximity to and maintain contact with the parent until they feel safe (Ainsworth & Bowlby, 1991; Benoit, 2004). According to Ainsworth and Bowlby (1991), parental warmth and consistent responsiveness underlie the development of a secure parent-child relationship, which contributes to the child’s understanding of relationships and to greater social-emotional regulation. Infants who displayed secure patterns of attachment behavior also showed more positive social behaviors toward both parents and peers throughout the preschool years and were less disruptive, aggressive, and more mature compared to children with non-secure attachments (Lyons-Ruth et al., 2006). In contrast, poor attachment relationships, including insecure-avoidant, insecure-resistant, and disorganized, stem from parental inconsistency and neglect. Insecure-avoidant attachment is categorized by parents who consistently respond to the child’s distress in insensitive ways (e.g., ignoring or becoming annoyed with the child); consequently, the child learns to avoid her/his parent in times of distress (Ainsworth & Bowlby, 1991; Benoit, 2004). Insecure-resistant attachment includes parents who respond in inconsistent or unpredictable ways to the child’s 5 distress; thus the child learns to display extreme negative emotion in order to obtain the attention of her/his parent. Both insecure-avoidant and insecure-resistant attachments are associated with adverse outcomes that negatively impact a child’s social-emotional health and development, including maladjustment and the development of oppositional, defiant, and aggressive behaviors (Fearon et al., 2010; Lezin et al., 2004). Children who develop disorganized attachment relationships are at the greatest risk for the development of later social, emotional, and behavioral issues. A disorganized attachment is categorized by distorted and atypical parent behaviors that are displayed by the parent during her/his interactions with the child (Benoit, 2004). Parents who display atypical behaviors often have a history of unresolved emotional, physical, or sexual trauma (Waters & Cumming, 2000). Children with disorganized attachment relationships are more vulnerable to stress, have difficulty with emotion regulation, display oppositional and aggressive behaviors, and have higher levels of teacher- and parent-rated social and behavioral difficulties in class (Benoit, 2004; Fearon et al., 2010). Thus, the formation of a warm and secure attachment is crucial to a child’s healthy development. Parenting practices that parents learn through parent training interventions can foster warmth in the parent-child relationship and help develop positive parent-child interactions. Parent-child Interaction Therapy (PCIT) is an intervention approach that aims to support healthy parent-child attachments by establishing warmth in the parent-child relationship through the teaching of skills that help children feel secure in their relationship with their parents (McNeil & Hembree-Kigin, 2001). 6 Social Learning Theory Social learning theory (Bandura, 1977) provides an explanation for the processes underlying how parents model appropriate behavior and emotion regulation, and how children learn to behave from their parents. It describes how everyday interactions between parents and children shape long-term patterns of behavior for both the parent and the child (Forgatch & Patterson, 2010). Social learning theory is a theory of learning and social behavior, which assumes that people learn behaviors, attitudes, and values from observing and imitating one another (Bandura, 1977). Children’s real-life experiences and exposures in their environments directly or indirectly shape their behavior. For a child, each moment-to-moment exchange with their parents is crucial, as child behavior can be shaped through repeated, microlevel dyadic parent-child interactions (Granic & Patterson, 2006). From parent-child exchanges, a child can learn strategies about managing their own emotions, resolving conflict, and engaging with others by observing the way in which their own reactions were responded to by their parents. For example, parents who are warm and responsive in their interactions with the child can help facilitate a positive parent-child relationship, as well model appropriate behavior and a calm demeanor to the child. Parents serve as influential models for their children, and the family environment is the primary context in which children observe and learn how to behave appropriately, regulate their emotions, resolve conflict, and engage with others. Parental affect during parent-child interactions may also influence child outcomes (Rueger et al., 2011). For example, high levels of maternal negativity towards the child is correlated with externalizing behavior problems in preschool children (Cole et al., 2003). Effective parent training programs aimed at reducing externalizing problems have demonstrated 7 that altering parents’ affective responses can lead to positive improvement in the parent-child relationship (Webster-Stratton et al., 2011). Parents can model appropriate methods to calm down and regulate their emotions and teach children to identify their feelings. In this way, a social learning framework recognizes that the environment and interactions between parents and children are the source of difficulty in perpetuating behavioral concerns, rather than suggesting that the individual child or parent is solely responsible. Social learning theory provides a foundation for incorporating positive dimensions of parenting as a way of promoting positive child behaviors and affect, as well as to improve parent-child interactions. Purpose of this Study Given the importance of a positive parent-child relationship and parent-child interactions, research on parent training interventions in the treatment of disruptive child behaviors, and the need for greater prevention efforts aimed at reducing disruptive behaviors in young children, the purpose of this study was to critically examine the treatment adherence, effectiveness, and acceptability of a preventative parent training intervention. Specifically, this study proposed adapting PCIT (Eyberg, 1988), an evidence-based parent training intervention (McNeil & Hembree-Kigin, 2001), to be implemented as a preventative program for the treatment of subclinical levels of child externalizing behavior in preschool-aged children. PCIT is a well-established parent training intervention for parents with children ages two to seven years who exhibit clinical levels of disruptive behavior (Thomas & Zimmer-Gernbeck, 2007). The goal of PCIT is to reduce child disruptive behavior by enhancing the parent-child relationship and improving parenting skills (Eyberg & Funderburk, 2011; McNeil & Hembree- Kigin, 2011). In PCIT, parents receive didactic training in positive parenting practices, as well as in-vivo practice and coaching on their parenting skills during sessions (Eyberg & Funderburk, 8 2011). Research on PCIT has demonstrated that PCIT is associated with positive effects, including improvements in child externalizing behaviors with effect sizes ranging from 1.06 to 1.65 (e.g., Cooley et al., 2014; Eyberg, Nelson, & Boggs, 2008; Ward et al., 2016), gains in positive parenting skills, and increased parental confidence in their ability to manage challenging child behaviors (Abrahamse et al., 2012; Bjørseth & Wichstrøm, 2016). Despite the strong research support behind PCIT, it has seldom been studied as a preventative intervention. Given its emphasis on strengthening the parent-child relationship and improving parenting skills with the goal of reducing child disruptive behaviors, PCIT has the potential to be used as a parent training prevention program, thus expanding the literature on interventions that may reduce risk for long-term behavioral difficulties. The present study critically examined the treatment adherence, effectiveness, and acceptability of an adapted version of PCIT with four mother-child dyads implemented via telehealth. Participating children demonstrated subclinical, yet elevated, levels of disruptive behaviors to be included in the study. A single-case experimental design (nonconcurrent multiple-probe across participants) was used to examine parent and child outcomes. Finally, acceptability of the adapted intervention was assessed through parent ratings of satisfaction with treatment. 9 CHAPTER II. LITERATURE REVIEW This literature review draws on research from several relevant areas. First, a brief historical background is provided to give context to the significance of child disruptive behaviors and need for preventive interventions. Next, child risk factors for disruptive behavior disorders are described and evidence-based treatments to address such problems are reviewed. Subsequently, a description of the public health model is given and its application to mental health service delivery is presented. Then, the limitations of relevant evidence-based treatments are considered. Finally, current research about a well-established treatment (Parent-Child Interaction Therapy) is further discussed to provide support for its use as a preventive program. Historical Background Research suggests that there is a strong need for addressing behavior problems early. Disruptive behavior is characterized by observable, non-preferred actions presenting as noncompliance, opposition, aggression, and hyperactivity (Campbell, 2000; Tremblay, 2010). Between 10 to 15 percent of preschool children exhibit mild to moderate behavioral difficulties (Campbell, 2000). Therefore, for a substantial portion of preschool children, behavioral symptoms present a challenge for parents and educators; however, these symptoms do not meet clinical significance warranting intensive intervention. The literature demonstrates that early behavior problems such as aggression, opposition, and hyperactivity are associated with more severe problems later in life (Bellanti & Bierman, 2000; Vitaro, Brendgen, Larose, & Tremblay, 2005). Thus, there is a need to intervene and prevent more severe behaviors later on. Two factors that have been empirically supported in the prevention of later conduct problems include positive parenting practices and support and positivity within the parent-child relationship (Hong & Park, 2012; Lezin et al., 2004). To better inform the use of interventions 10 designed to address the parent-child relationship and parenting practices, it is necessary to understand the factors that put a child at risk for the development of problematic behaviors, as well as how parents can promote the development of healthy behaviors. Additionally, it is important to examine the multiple factors that influence the development of a positive parent- child relationship and positive parenting practices. Child Risk Factors Specific family and environmental factors can make a child more vulnerable to social, emotional, and behavioral problems. The intrapersonal risk factors that have been identified as placing a child at risk for clinically significant disruptive behavior issues include genetic dispositional risk factors such as (a) neurochemical imbalances (e.g., low serotonin), (b) temperamental vulnerabilities (e.g., poor emotion regulation, self-regulation, high emotional reactivity; Barkley 2010; Barkley, 2013), and (c) personality predispositions (e.g., impulsivity, irritability; Mash & Barkley, 2014). In addition to intrapersonal factors, there are contextual risk factors that are linked to later behavioral problems including (a) prenatal factors (e.g., exposure to toxins, alcohol use, malnutrition; Centers for Disease Control and Prevention [CDC], 2018; Latimer et al., 2012), (b) characteristics of the environment (e.g., low quality child care, exposure to violence), (c) family variables (e.g., ineffective discipline, poor attachment, low socioeconomic status, parental criminality, maternal depression; Harvey, et al., 2011; Lavigne et al., 2012), (d) poor peer relationships (e.g., friendships with deviant peers, peer rejection; McCabe et al., 2001), and (e) neighborhood characteristics (e.g., high levels of violence, impoverished neighborhoods; CDC, 2018; Mash & Barkley, 2014). 11 While there are many factors associated with the development of disruptive behavioral problems, the most central and consistent influence on the development and maintenance of such behaviors are dysfunctional parenting practices (Mash & Barkley, 2014; Stormshak et al., 2000). Specific parenting practices that have been associated with the development of disruptive behavior disorders include the use of harsh and inconsistent discipline (McDonald et al., 2011; Waller et al., 2012), low levels of parental warmth and positive involvement (Kroneman et al., 2011; Pasalich et al., 2011), physical aggression (hitting, spanking), and failure to use positive reinforcement (Mash & Barkley, 2010). Coercive, inconsistent, and harsh parenting practices are related to the emergence of child oppositional and conduct behaviors (Muratori et al., 2017). Additionally, parent follow-through with consequences is important to gain child compliance. Patterson and colleagues (1986) noted that parental failure to follow through with commands resulted in reinforcement of child non- compliance. Furthermore, children with unpredictable and inconsistent parents are more likely to engage in oppositional and defiant behaviors in order to elicit predictable, yet negative, responses from the parent (Hong & Park, 2012). Parents who utilize physically aggressive disciplinary practices to gain compliance of children are also more likely to have children who are already engaging in more severe forms of aggressive behavior (Waller et al., 2012). Additionally, low levels of parental warmth and positive involvement are associated with conduct problems (Kroneman et al., 2011). Parental warmth is the expression of positive affect, affection, and admiration (Davidov & Grusec, 2006). Parent-child relationships characterized by low levels of warmth are correlated with child emotion regulation difficulties, including frequent child temper tantrums, whining, and noncompliance (Pasalic et al., 2011). These behaviors are characteristic of Oppositional Defiant Disorder. Low parental warmth can interfere with a child’s 12 ability to regulate his/her emotions, thus making it difficult to restrain problematic behavior (Etkin et al., 2014). Given the influence of poor parenting practices on negative child outcomes, a focus on positive parenting practices is central in the prevention of child behavior problems. For young children who are at greater risk for disruptive behavior problems, early intervention is crucial. In order to prevent the development of more severe behaviors, it is important to consider a preventative approach to intervention to address these child risk factors early on. Additionally, a preventative approach can reduce the number of individuals requiring intensive treatment services and the costs to society that are associated with treating problems after they become clinically significant (Insel, 2008). Public Health Model In general, a preventative approach to intervention aims to address risk factors associated with later problems, which decreases the likelihood that an individual will have more severe problems later on. One service delivery model that focuses on the prevention of more severe (i.e., clinically significant) problems is the public health model. The public health model aims to provide the maximum benefit for the largest number of people by identifying risk factors and targeting those factors for early intervention (Hess et al., 2012). Thus, services are directed towards populations rather than individuals. The public health model emphasizes prevention by attempting to reduce or prevent negative outcomes. Furthermore, the public health approach to service delivery promotes the provision of a full continuum of services available to meet the broadest of needs. This continuum of services is based on a system of tiered levels that are designed to meet the specific needs of the population. Problems are targeted through the implementation of universal interventions at the primary level, targeted interventions at the secondary level, and intensive selective interventions at the tertiary levels. The public health 13 model has typically been used in the prevention and treatment of drug and alcohol abuse, violent behaviors, and childhood obesity (Chacko et al., 2018). However, the public health approach can also be applied to mental health concerns, such as the prevention and treatment of disruptive behaviors. The public health model approach is a four-step process. The first step towards the prevention of a problem is to define the magnitude of the problem. This includes identifying what the problem is, who the problem affects (e.g., the target population), when and how best to intervene, and where services can be delivered (Centers for Disease Control and Prevention [CDC], 2018; National Academies of Sciences, Engineering, and Medicine, 2017). Relevant to this study, the problem is the emergence of disruptive behavior disorders, which are characterized as any non-preferred, observable behavior (Campbell, 2000). Disruptive behavior disorders are most commonly exhibited in children ages 2-7 years, but early intervention via parent training interventions with parents and their children during the preschool years is most beneficial (Chacko et al., 2018). The second step in the public health model includes the identification of risk and protective factors (Chacko et al., 2018; National Academies of Sciences, Engineering, and Medicine, 2017). In order to prevent a more severe problem from developing later on, it is important to understand both the risk and protective factors that may lead to a problem in order to identify where prevention efforts should be focused. Risk factors are any characteristics – genetic and contextual—that increase the likelihood of a problem persisting (e.g., prenatal factors, low quality childcare, poor attachment, low parental warmth, poor parenting practices). Contrastingly, protective factors are any characteristics that decrease the likelihood of a problem (e.g., authoritative parenting practices, secure attachment, self-regulation skills). 14 Based on the identification of risk and protective factors, the third step is to develop and evaluate prevention strategies and interventions (Chacko et al., 2018; National Academies of Sciences, Engineering, and Medicine, 2017). This step is a broad one that includes relying on empirical literature to ensure the use of evidence-based interventions, using data from assessments and relevant stakeholders, and critical evaluation of programs to determine what works. At this step in the process, multi-tiered, comprehensive services are provided in a systematic way at three tiers: primary, secondary, and tertiary. At the universal or primary prevention level, no individual is identified as having special needs or problems. Instead, supports and programs are designed to broadly address causal risk factors that are associated with negative outcomes (Hess et al., 2012). For example, in working to prevent childhood obesity, efforts at the primary level may include improving food selection choices in cafeterias and educating children about nutrition. In working to prevent disruptive behavior problems in children, prevention efforts may include ensuring that parents receive general education in positive parenting practices or the application of school-wide positive behavioral interventions and supports (PBIS) in school systems (Horner et al., 2005). Targeted secondary services are provided to individuals who are at-risk for the development of more severe behaviors later on and/or who exhibit subclinical levels of behavior (Wakschlag et al., 2015). For example, children who display difficult, but subclinical behavioral concerns have a higher risk for developing more severe behavior problems. To prevent clinically significant problems later on, the caregiver(s) may receive parent training to learn how to address child noncompliance and externalizing behaviors. Intensive selective tertiary services are provided to individuals who demonstrate clinically significant behaviors. For example, individuals at the tertiary level could include obese 15 children, children with a clinically significant disruptive behavior disorder, and individuals who abuse alcohol. Tertiary services may include an individualized exercise and nutrition program for an obese child, psychological services from a clinic or community mental health agency for a child with a disruptive behavior disorder, and rehabilitation services for an abuser of alcohol. No matter the level of service provision, programs are continually evaluated to determine their effectiveness for various populations (Chacko et al., 2018; National Academies of Sciences, Engineering, and Medicine, 2017). The final step in the public health model is the assurance of widespread adoption of prevention efforts. This step is critical to ensuring the adoption of a prevention approach. The reduction of a problem through tertiary services (e.g., psychosocial treatment approaches) will not have maximal impact unless factors of health (e.g., externalizing behaviors within normal limits) are addressed through mental health promotion approaches (Chacko et al., 2018). A public health approach maintains that factors that can place children at risk for negative mental health outcomes or promote positive mental health are at the broadest level (i.e., the community). Thus, the implementation of effective and sustainable public health approaches requires integrated efforts from multiple sectors (e.g., government, schools, community mental health agencies, primary care providers, caregivers). The more integrated these sectors, the greater the dissemination efforts, and the more likely a prevention framework can be adopted. Widespread dissemination efforts can include education to relevant stakeholders about a specific problem, may occur via training (e.g., for educators, community mental health agencies, parents, primary care providers, etc.), through networking, and via screenings for potential problems, as well as program evaluation (Chacko et al., 2018; National Academies of Sciences, Engineering, and Medicine, 2017). 16 Since an increasing number of children are exhibiting more significant behavior problems (Kessler et al., 2005; U.S. Department of Health and Human Services, 2001), a preventative and systematic approach to service delivery is needed. Preschool behavior problems are a vital target for intervention, given findings that preschoolers are expelled at a rate 3.2 times higher than school-age children (Gilliam, 2005) and that behavioral problems among young children predict later concerns (Bellanti & Bierman, 2000; Vitaro et al., 2005). While traditional treatments address problems at clinical or non-normative levels, prevention programs focus on improving subclinical concerns and/or treating at-risk individuals. If larger populations of children are exhibiting clinically significant problems that require more intensive services, the system will not be able to provide appropriate intervention. Thus, preventative efforts need to increase, as a reduction in problematic behavioral symptoms decreases the need for services. The proposed study aims to address the third step in the public health model by evaluating a potential preventative parent training program at the secondary level for children exhibiting emerging behavior problems. Due to the influence that poor parenting practices can have on child outcomes, targeting positive parenting practices is an important component in the prevention of child behavior problems. Positive Parenting Practices The term “parenting practices” refers to parenting behaviors or approaches to childrearing that can shape how the child develops (Breiner et al., 2016). Although negative parenting practices are associated with later child behavior problems, there are positive parenting practices that serve as protective factors against poor child behavioral outcomes. Authoritative parenting combines high levels of parental warmth with moderate levels of control (i.e., firm but fair disciplinary practices). Authoritative parenting is associated with the 17 healthy development of children and the emergence of important skills such as self-regulation, cognitive, and social skills (Hong & Park, 2012). Developmental research demonstrates that high levels of parental warmth, involvement, and communication support the development of child negotiation and conflict-resolution skills, which are skills that children need to manage interpersonal relationships and reduce their reliance on noncompliant or oppositional behaviors and tactics (Lezin et al., 2004). While parent training interventions vary in the number and type of parenting practices that parents are taught, four core parenting practices (skills) remain consistent throughout the empirical research. These practices include the use of parental warmth, effective monitoring (or supervision), parental involvement (or parental initiation), and the encouragement of prosocial skill acquisition (e.g., Forgatch & Patterson, 2010; Walton & Flouri, 2010). Parental warmth (or positive affect and affection) is correlated with positive child outcomes. High parental warmth has been linked with the development of children’s emotion regulation skills, which can be adaptive in high conflict environments (Walton & Flouri, 2010). For example, warm parents may help children learn to self-regulate their emotions and behaviors, which may help children to repress their aggressive and defiant actions due to higher emotion regulation capacities (Walton & Flouri, 2010). Additionally, children are more likely to positively view the parent-child relationship if parents show higher levels of affection and admiration (Etkin et al., 2014). Parental warmth may also let the child know his/her needs are important (Cummings & Davies, 2010). Most notably, the expression of parental warmth has been correlated to fewer problem behaviors over the course of adolescence (Etkin et al., 2014). Another parenting skill that is associated with positive child outcomes is monitoring. Monitoring includes awareness of and attending to the activities of children (Forgatch & 18 Patterson, 2010). Parents can monitor a child’s activities by describing what the child is doing behaviorally (e.g., “You are drawing a picture” or “You are building a tall tower”) or by joining in on the child’s activity. Monitoring is associated with decreases in antisocial behavior (Loukas et al., 2007) and decreased adolescent externalizing behavior (Gonzales et al., 2012). Monitoring shows the child that the parent is interested, demonstrates that the parent approves of the child’s choice of play, and helps organize the child’s thoughts about play (Eyberg & Funderburk, 2011; Forgatch & Patterson, 2010). Positive parent involvement is another parenting skill that is related to healthy child development. Positive parent involvement includes showing love and directing attention towards the child (Forgatch & Patterson, 2010). Parent involvement may consist of spending quality time with the child (e.g., joining in play) and attending to what the child says and does (e.g., describing what the child is doing; commenting, reflecting the child’s statements, and/or responding to a child’s verbalizations). According to Davidson and Cardemil (2009), higher levels of communication between a parent and a child have been associated with fewer child externalizing problems. Active listening to the child is also a way to foster a healthy parent-child relationship and lets the child know the parent is invested in what (s)he says (Forgatch & Patterson, 2010). Parents can also play an important role in the child’s development of prosocial skills. Parents can teach children prosocial behaviors using verbal praise and other forms of positive reinforcement (e.g., parental attention for desired behaviors) to shape prosocial behaviors (Forgatch & Patterson, 2010). Verbal praise is also a way to show warmth, positive affect, and direct attention towards the child. In addition, modeling is another way to encourage prosocial behaviors. Parent-child interactions provide an opportunity for the parent to demonstrate desired 19 behaviors, as children continuously watch their parents. For example, if a parent wants his/her child to share, (s)he can demonstrate sharing with the child. Furthermore, a parent can model using a calm voice and non-aggressive behaviors when upset. Overall, the aforementioned parenting practices are associated with positive child behavior outcomes. High quality early childcare is characterized by sensitivity, warmth, and responsiveness from the parent and is correlated with lower levels of disruptive behavior problems in children, including decreased impulsivity and greater compliance (Belsky et al., 2007; van Dijk et al., 2017). Given that parenting practices have a large influence on children’s development, treatment of disruptive behavior problems occurs through parent training interventions that foster positive parenting practices. Parent training interventions have long been identified as an evidence-based treatment for the treatment of disruptive child behavior disorders (Chorpita et al, 2011; Comer et al., 2013). In order to identify which parent training intervention(s) can potentially be used to treat subclinical levels of problem child behavior, and thus prevent the development of later behavior problems, it is important to critically examine which parent training interventions are empirically supported. Parent Training Interventions The primary purpose of parent training programs is to teach parents how to interact with their children in order to reduce externalizing behaviors. Research has demonstrated that parent training interventions are effective means of targeting child behavioral and emotional problems (Maughan et al., 2005). In general, parent training interventions aim to enhance the parent-child relationship and teach behavioral management strategies to parents. During such interventions, parents receive education about, practice with, and/or direct coaching of skills to reduce coercive 20 and inconsistent parenting practices and increase positive interactions with the child (Dekovic et al., 2010). There are many components of parent training interventions that are empirically supported and correlated with positive outcomes for children and their parents. First, many evidence-based parenting interventions have foundations in the social learning theory (Bandura, 1977; Forgatch & Patterson, 2010). This theory posits that behavior is shaped through reinforcing contingencies provided during repeated interactions with “key” people (e.g., mothers, fathers) in the social environment (Forgatch & Patterson, 2010). For young children, parents are primary socializing agents. Thus, practitioners focus their efforts on parents as agents of change, teaching parents skills to prevent and remediate children’s behavior problems. In parent training programs, it is imperative that parents are recognized as meaningful agents for facilitating children’s healthy development. Collaborative partnerships among parents and professionals correlate with positive social-emotional and behavioral outcomes for children and families and the efficacy and efficiency of interventions (Sheridan et al., 2008). The use of dyadic and triadic strategies is important and emphasizes the significance of parent-child and parent-child-therapist interactions. Dyadic strategies are those used by the parent during interactions with the child to help support the child’s development. Dyadic strategies include being sensitive and responsive to the child, following the child’s lead during play, turn taking, elaborating on the child’s communicative attempts, and commenting on the child’s activities, and interests (McCollum et al., 2001; McCollum & Yates, 1994). Triadic strategies are those used by the professional (i.e., therapist) during the course of parent-child interactions to build and expand on interactions that are enjoyable for the parent and the child and are also supportive of the child’s development (McCollum & Yates, 1994). Triadic strategies 21 include affirming parenting competence, modeling dyadic strategies, and providing suggestions for the parent to try with the child (McCollum & Yates, 1994). In-vivo coaching is a specific triadic strategy that has been empirically examined on its own. Shanley and Niec (2010) compared parent outcomes of 60 mothers and their children (ages 2-7 years) who were randomly assigned to a coaching or no coaching intervention group. Mothers who received in-vivo coaching displayed significant improvements in their parenting skills (e.g., labeled praises, behavioral descriptions, and reflections) compared to mothers in the group who received no in vivo coaching. Barnett and colleagues (2017) further examined how the therapist’s in-vivo coaching influenced parent attrition and skill acquisition with 51 parent- child dyads. They found that parents who received more responsive coaching acquired positive parenting skills more quickly and were more likely to complete treatment, while parents who had therapists that utilized less responsive techniques were more likely to drop out of treatment (Barnett et al., 2017). Together, these findings suggest that providing parents with responsive feedback through in vivo coaching is an important mechanism of change and influences parents’ engagement with treatment. Additionally, the quality of triadic interactions can be enhanced through a strengths-based approach, meaning that the therapist empowers the parents to use the skills they already have and expand upon them. Professionals can empower parents and increase skill development through activities such as role-playing in sessions, homework to practice at home, and collaborative discussions about parental issues and concerns. The connections among parents and professionals are an important relationship that is predictive of healthy child outcomes (Henderson & Mapp, 2002). 22 Several parent training programs have been found to be efficacious in treating behavior problems in young children. These parent training interventions include the aforementioned active components of parent interventions. The four most widely researched parenting behavior training programs that have demonstrated effectiveness in improving child behavior include Parent Management Training—the Oregon Model, Incredible Years, Triple-P Positive Parenting Program, and Parent-Child Interaction Therapy. Parent Management Training –Oregon Model Parent Management Training –the Oregon Model (PMTO; now referred to as GenerationPMTO), is a set of evidence-based parent training programs for parents of children 2- 18 years old with disruptive behaviors, including conduct disorder, oppositional defiant disorder, and antisocial behaviors (California Evidence-Based Clearinghouse, 2016; Forgatch & Kjobli, 2016). Delivery format for PMTO includes sessions with individual families in agencies or families’ homes, parent groups, and web-based and telehealth communication. The goal of PMTO is to teach parents how to reduce coercive parenting practices and replace them with five effective parenting practices including skill encouragement (via verbal praise and positive reinforcement), effective limit setting, monitoring and supervision, interpersonal problem solving (i.e., responding to rule-breaking behaviors), and positive involvement with the child (Forgatch & Kjobli, 2016; Forgatch & Patterson, 2010; Thijssen et al., 2016). During PMTO, parents attend weekly, one-hour sessions with the therapist. Parents serve as the primary agents of change, thus the therapists focus their efforts on teaching the parents. Children are not typically present for sessions unless their presence is relevant to the session. In intervention sessions, parents role-play with the therapist to learn and practice parenting skills, as well as problem solve to address any concerns that arose at home during the week. 23 There are a variety of PMTO programs available for families that vary in duration, format, and cost depending on the severity of the problem behavior and needs of the family. PMTO programs range in focus from preventative interventions with at-risk samples (e.g., single mothers, families living in high crime neighborhoods) to clinical treatment for serious behavior problems in preschoolers through adolescents and cost a total of $1,089 - $1,634 (Dishion et al., 2016). Treatment can range from 10-25 weekly sessions for severe problems, 6 to 14 sessions for group and telehealth sessions, and 6-8 individual sessions for mild problems or prevention (Dishion et al., 2016; Thijssen et al., 2016). PMTO has been rigorously examined in efficacy and effectiveness trials in different contexts, cultures, and delivery formats (see Dishion et al., 2016 for a complete list). Adaptations have been added to the core and supporting PMTO components to address particular contexts or populations (e.g., adding sessions on parenting alone, child support, negotiating visitation arrangements; Forgatch & DeGarmo, 1999). Studies conducted within the United States have shown that PMTO is efficacious in reducing child behavior problems and noncompliance at home and school for children who demonstrate clinically significant behaviors and in improving parenting practices for families of divorced mothers, biological mothers and stepfathers, and foster families with sample sizes ranging from 67 families to over 1,000 (Forgatch & DeGarmo, 1999; Forgatch et al., 2005). More specifically, studies have reported small standardized mean- difference effect sizes (d = 0.20-0.30) for improving child behavior problems (e.g., Forgatch et al., 2009), for improving police arrests in adolescence (d = 0.28; Forgatch et al., 2009), and for improving parenting practices (d= 0.33; e.g., DeGarmo et al., 2004; DeGarmo & Forgatch, 2007). 24 The effectiveness of PMTO has been investigated in Norway, Iceland, Europe, the Netherlands, and a few states within the U.S. (e.g., Montana, Michigan, New York, and Kansas) with study samples ranging from over 100 participants to over 1,000 (e.g., Dishion et al., 2016). In these samples, child participants (ages 4-12 years old) had a diagnosis of at least one disruptive behavior disorder (e.g., CD, ODD). Results indicated that PMTO was effective in reducing parent-reported child externalizing behaviors, improving teacher-reported social competence, and enhancing parental discipline (Dishion et al., 2016; Ogden & Hagen, 2008). More specifically, parents who received PMTO improved in their limit setting and discipline techniques, and monitoring, compared to parents who received treatment as usual, regardless of the child’s age. Additionally, younger children of parents who received PMTO were rated lower on externalizing behaviors by both parents and teachers and on compliance (Ogden & Hagen, 2008). Although empirical studies demonstrate that parents who receive PMTO have shown improvements in the five core parenting practices (skill encouragement, limit setting, monitoring, problem-solving, and positive involvement) and their children demonstrate decreased externalizing behavior, there are several limitations to note. First, in regard to efficacy studies of PMTO in the United States, the majority of studies have only examined outcomes in school-aged children (e.g., in kindergarten and up) rather than in younger children, although PMTO studies abroad have examined child outcomes in preschool-aged children (e.g., Ogden & Hagen 2008). Furthermore, several of the PMTO studies cite small median effect sizes in improving child behaviors and positive parenting practices (Dishion et al., 2016). Secondly, there is a lack of studies that examine the effects of PMTO on positive parenting practices and child externalizing behavior within the United States. Limitations across 25 the majority of PMTO studies are small sample sizes, lack of diversity in the sample in regards to race, ethnicity, and gender, issues with generalizability of results, and lack of post-intervention follow-up (Akin et al., 2014; DeGarmo & Forgatch, 2007; Hukkelberg, S., & Ogden, 2013; Ogden, & Hagen, 2008). An additional major limitation of PMTO is the use children with clinically significant levels of behavior. PMTO has been examined primarily with populations of children who are expected to demonstrate more clinically significant behaviors. For example, children from divorced families, children from high crime neighborhoods, maltreated children, preschoolers in foster care, and children labeled as “delinquent” (Forgatch & Patterson, 2010; Dishion et al., 2016) are likely to exhibit higher levels of externalizing behaviors, given the environmental circumstances. Given that parent training can help prevent more severe behaviors later on, it would be advantageous to examine the effects of PMTO with parents of children with mild behavioral concerns. In addition to limitations in the research, there are also practical limitations to PMTO. Like other parent training interventions, PMTO therapists view parents as the primary agents of change; thus, children are not present for sessions. However, as noted in Forgatch and DeGarmo (1999), this approach to treatment assumes that parents will apply newly acquired parenting skills (e.g., limit setting, positive involvement, monitoring, skill encouragement) to child problems. In other words, it is up to the parents to decide which strategies to use based on the target problem(s) that are most concerning. Prior research has demonstrated the advantages to focusing on dyadic interactions (e.g., parent-child), as well as triadic interactions (e.g., parent- child-therapist; McCollum et al., 1994; McCollum et al., 2007). Furthermore, research on in-vivo coaching of parents indicates that parents who receive direct feedback on their parenting skills 26 acquire such skills faster and displayed more significant improvements in parenting skills compared to families who did not receive coaching (Shanely & Niec, 2010). Coaching techniques that are responsive (e.g., reinforce the parent’s use of the skill in the moment) have been associated with positive parental behavior changes, while parent training sessions that do not include coaching are predictive of early dropout (McNeil & Hembree-Kigin, 2011). Thus, parent training programs that include the use of dyadic and triadic strategies, as well as include in-vivo coaching are likely to be more effective in improving parenting skills and child behavior. Another practical limitation of PMTO includes the length of time and cost of treatment. According to Dishion and colleagues (2016), PMTO takes 25 sessions, on average. This is a long time to be in treatment, particularly for families of children with severe behavior problems experiencing adversity. Furthermore, the total cost of treatment ($1,089 - $1,634) may be too expensive. For clinicians, training in PMTO requires participation in three workshops across ten days, a minimum of 12 coaching sessions, and recertification. The cost of training and certifying clinicians is about $19,165 per person (Generation, PMTO, 2017). For both parents and clinicians, the cost of treatment is high. Incredible Years Parent Training Program The Incredible Years Parenting Training Program (Webster-Stratton, 1980) is series of evidence-based programs to prevent and treat behavior problems in young children. Parenting programs are grouped according to age and include infants (0-12 months), toddlers (1-3 years), preschoolers (3-6 years), and school-age (6-12 years). Each program aims to strengthen parent competencies and foster parent involvement in children’s school experiences, promote children’s academic, social, and emotional skills, and reduce conduct problems. 27 While there are extended parent programs, the BASIC series is considered a “core series” that emphasizes parenting skills known to promote children’s social competence and reduce behavior problems. These parenting skills include how to play with children; social, emotional, academic and persistence skills coaching; effective praise and use of incentives; establishing predictable routines and rules and promoting responsibility; effective limit setting; behavior management techniques; and teaching children to problem solve. In the BASIC program for preschool-aged children, parents meet in 18-20 weekly 2-hour group sessions that are led by two group facilitators. During the sessions, parents watch video vignettes depicting parent models interacting with their children in various situations. Parents discuss the video vignettes, engage in role-play practice exercises, and review home activities. Children are not present during group sessions. The Preschool BASIC program costs $1,670 total (The Incredible Years Inc., 2013). Although the Incredible Years BASIC parent training program was originally developed to be used as a clinic-based treatment program, it has also been used as a prevention program with high-risk populations (e.g., Head Start children; Webster-Stratton et al., 2001). The type of population who is receiving services determines whether the intervention is classified as treatment (an intervention for families who sought help), indicated prevention (an intervention aimed at children who are identified as having minimal, but detectable signs or symptoms of behavior problems), selective prevention (an intervention targeted at children who are considered at high-risk due to biological, psychological, or contextual factors), or as universal prevention (an intervention targeted at children who have not been identified on the basis of individual risk; Menting et al., 2013). The Incredible Years BASIC parent program is an evidence-based program that has demonstrated effectiveness in strengthening parent management skills, improving children’s 28 social and emotional competence and school readiness, and reducing behavior problems (Webster-Stratton & Reid, 2010). To date, there is only one meta-analysis that has examined the effects of the BASIC parent training program on child outcomes. This meta-analysis is comprised of 50 studies that were conducted over a 30-year period and included 4,745 participants. The majority of child participants were male and the mean age of the child sample ranged from three to nine years old. Results of the meta-analysis indicated that the Incredible Years BASIC parent training program resulted in small effect sizes for child behavior outcomes. Positive changes in child disruptive behavior (d = 0.27) and prosocial behavior were reported (d = 0.30; Menting et al., 2013). Mean effect sizes based on observations (d = 0.37) were larger than mean effect sizes based on parents’ (d = 0.30) or teachers’ judgments (d = 0.13). Furthermore, parent-rated effect sizes differed between treatment studies (d = 0.50), and prevention studies (d = 0.13 for selective prevention and d = 0.20 for indicated prevention). Pre-treatment intensity of children’s problem behavior was the strongest predictor of the intervention’s effects on parents’ reports, with larger effects for studies that included children with more severe behavior (Menting et al., 2013). While this meta-analysis did not examine whether parenting behaviors improved following participation in the BASIC parent training program, it is considered a “well established” intervention (Menting et al., 2013). Results from individual studies examining the Incredible Years BASIC program indicate improvements in parenting domains such as reductions in harsh parenting, criticism, negative commands, and inconsistent discipline, and increases in positive parenting, responsiveness, and overall parenting skills (Hutchings et al., 2007; McIntyre, 2008; Webster-Stratton & Reid, 2010). Over 10 randomized controlled trials have examined the use of the BASIC parent training 29 program with parents of children (ages 3-8) who exhibit disruptive behavior problems. Findings demonstrate that the program is effective in the treatment of problematic behavior for children (ages 3-8 years) with ODD and CD in mental health clinics (e.g., Axberg & Broberg, 2012; Larsson et al., 2009). While the Incredible Years Parent Training Program is a well-established, evidence- based program for parents of children with disruptive behaviors, there are a few practical limitations that should be noted. First, children are not part of the sessions. Although parents watch video vignettes of parent models, participate in role-play activities to practice parenting skills, and engage in discussions regarding how to improve parenting skills and address challenging behavior, they do not receive feedback on their own parenting skills. The purpose of the videotaped vignettes and discussion sessions are to learn and practice positive parenting skills; however, the focus of the sessions is not on individual issues and parent concerns. Similarly to PMTO, parents are expected to practice skills at home. Parents may benefit from in- vivo feedback on their parenting skills, as research on in-vivo coaching demonstrates that parents who receive direct and responsive feedback on their coaching show more significant improvements in parenting skills (Shanely & Niec, 2010). Thus, it may be advantageous for the therapist to observe parent-child interactions in real time. Furthermore, the group format of the BASIC program may also be challenging, as parents who have children with less severe behavior problems may have different questions and concerns compared to parents with children who have more severe behavior problems. Lastly, for clinicians and their agencies, the cost of materials, training, and time commitment needed to implement the Preschool BASIC parent training program ($1,670) can be expensive, depending on the funding available for the mental health agency. To implement the 30 program, clinicians must attend a three-day training in Seattle ($660 for the training plus travel costs) or arrange for a trainer to come to their site ($1,100-1,500). Additionally, clinicians must consider the cost of parent materials ($17.95 per book), group leader books ($26.95 per book), consultation fees with trainers during training and recertification ($200 an hour), and an Incredible Years certification fee. Triple-P Positive Parenting Program The Triple P—Positive Parenting Program is a series of multi-tiered behavioral family interventions of increasing intensity designed to promote positive parenting and foster caring parent-child relationships among parents and children ages 2-16 years (Sanders, 2003). Originally, the program was tailored for at-risk children and their parents; however, five different levels of implementation were designed in order to take a universal, public health approach (Sanders, 2008). Within each level, there is also a choice of delivery methods to ensure that the needs of each family are met, while also encouraging self-sufficiency. At each level, families are given information on positive parenting practices and behavior management strategies. Level 1 (Universal Triple P) is a communication method designed to reach the greatest number of people with positive parenting information. It is not a course or personal intervention directly delivered to parents, but rather a “communications campaign.” At Level 1, parents have access to a range of materials, including brochures, posters, newspaper columns, and billboards. The purpose of widespread dissemination is to increase awareness of the need to support parents in childrearing, destigmatize and normalize the process of seeking parenting help, encourage parents to participate in positive parenting interventions, and increase the reach of positive parenting interventions (Triple P International, 2018). 31 Level 2 (“light touch”) includes two intervention options: Brief Primary Care and Triple P Selected Seminar Series. Brief Primary Care is a 15-30-minute one-on-one consultation session created to assist parents who are generally coping well, but have one or two concerns regarding their child’s behavior. Individuals who are involved in education, social or healthy services, and voluntary organizations can be trained to deliver Level 2 interventions (Triple P International, 2018). Level 3 (targeted counseling) is for parents of children with mild to moderate behavioral difficulties. Level 3 interventions can be delivered as Primary Care Triple P or Triple P Discussion Groups by individuals who can provide occasional support for the client and therapeutic interventions (e.g., teachers, school counselors, nurses, home visitors, pediatricians, health professionals). Primary Care Triple P includes four 15-30 minute individual counseling sessions with parents to address a specific child problem behavior or issue. These sessions can be delivered face-to-face or via telephone. Triple P Discussion Groups consist of four, two-hour small group sessions that target a specific behavior or issue (Triple P International, 2018). Level 4 interventions are for parents of children with severe behavioral difficulties and teach 17 core positive parenting skills. Level four can be delivered as Group Triple P, Standard Triple P, Triple P Online, or Self-directed Triple P. Parents utilize DVDs and workbooks to learn and practice parenting strategies. In the Standard Triple P format, families who require intensive support receive 10, one-hour, in-person sessions with a facilitator. Individuals who are able to provide individualized regular interventions such as school counselors, nurses, psychologists, and social workers are able to be trained in Group and Standard Triple P (Triple P International, 2018). 32 Level 5 (intensive intervention) is for parents whose family situation is complicated by concerns such as partner conflict, stress, or other mental health issues. Interventions at Level 5 can be delivered via Enhanced Triple P or Pathways Triple P by individuals who can provide individualized interventions (e.g., school counselors, nurses, psychologists, and social workers). The Enhanced format is for parents whose family situation is complicated by other problems such as partner conflict, stress, or mental health issues. Enhanced Triple P includes three online modules that are tailored to parents’ personal needs. Pathways Triple P is for parents at risk of child maltreatment. This program covers anger management and other behavior strategies in order to improve a parent’s ability to cope with raising children (Triple P International, 2018). The overarching goal of the Triple P program is to enhance the knowledge, skills, and confidence of parents to prevent later behavior and emotional problems in children (Sanders, 2008). This goal is achieved through didactic presentations, individual or small group activities and homework, teaching of differential reinforcement, communication skills, and effective consequences for misbehavior (Triple P International, 2018). The Triple P Positive parenting program is a well-established evidence-based program for the treatment of disruptive behavior disorders. A meta-analysis that included 55 studies found that Triple P resulted in positive changes in parenting skills, child problem behavior, and parental well-being across all five levels, with small to moderate effect sizes (Nowak & Heinrichs, 2008). A more in-depth meta-analysis of 101 studies conducted over a 33-year period examined parent and child outcomes at each specific level within the multi-tiered program (Sanders et al., 2014). More specifically, child social emotional and behavioral outcomes, parenting practices, parenting satisfaction and efficacy, and parents’ relationship were examined. A total of 16,099 families were included, with sample sizes ranging from 8 to 2207. The number of studies that evaluated 33 each Triple P level varied considerably between Level 1 (n = 4), Level 2 (n = 9), Level 3 (n = 7), Level 4 (n = 86), and Level 5 (n = 12). The average child age across trials was 5.85 years, with the majority of participants being male. Effect sizes were reported for Triple P overall and per level for each child and parent outcome. Results of the study demonstrated that Triple P (all levels combined) demonstrated small-to-medium effects for children’s social emotional, and behavioral outcomes (d = 0.47), parenting practices (d = 0.58), parental satisfaction and efficacy (d = 0.52), and parental relationship (d = 0.23). In regard to children’s social emotional and behavioral outcomes, significant effect sizes were found at all five levels (d = 0.35 – d = 0.53), with the largest effect size reported at Level 5. Furthermore, small to large effect sizes were found for parenting practices at each level (d = 0.32 – d = 0.82, with the largest effect size reported at Level 3. At follow-up, significant effects were found for all parent and child outcomes, although smaller effect sizes were reported (Sanders et al., 2014). The results from this meta-analysis show that Triple P—at all levels and in all delivery formats—is an effective intervention for improving social, emotional, and behavioral outcomes in children, and that it has benefits for participating parents (Sanders et al., 2014). Despite strong empirical support and positive outcomes for parents and children, the Triple P Program has practical limitations that should be noted. First, Triple P promotes the use of 17 core parenting skills, particularly at Levels 4 and 5. These 17 skills fall under four main domains, including developing positive relationships (e.g., showing affection, spending quality time), encouraging desirable behavior (e.g., giving attention, descriptive praise), teaching new skills and behaviors (e.g., setting a good example), and managing challenging behaviors (e.g., planned ignoring, clear and calm instructions, time-out). While these are important skills to 34 learn, the acquisition of 17 skills by the end of 8-10 sessions is likely to be difficult in applied settings. Another limitation includes the delivery format of Triple P programs. Although a range of service delivery modalities can allow flexibility for families, especially those in rural/remote or low-income areas where access to parenting services may be more limited, some Triple P formats require families to have access to technological resources (e.g., computer, internet service, telephone). An additional limitation includes the research on the various Triple P programs. There have been relatively fewer studies of the lower levels (e.g., Levels 1-3) that have examined parent and child outcomes (Sanders et al., 2014). Thus, it is difficult to conclude how effective the prevention-oriented Triple P levels are for various populations. Additionally, throughout all five levels of Triple P, parents receive psychoeducation, consultation, counseling, or a combination of all three depending on the severity of the child’s behavior problems. While parents have the opportunity to learn parenting skills and receive consultation based on their concerns, parents do not have the opportunity to practice parenting skills in group or individual sessions (e.g., via role-play) and thus, do not receive feedback on their skill development. As noted previously, parents who received coaching (e.g., in-vivo feedback) displayed significant improvements in their skills and acquired positive parenting skills faster than parents who were not coached (Barnett et al., 2017; Shanely & Niec, 2010). Furthermore, consultation sessions are reportedly 15-30 minutes (Triple P International, 2018). This is likely a short length of time to collaborate with parents and problem-solve together, as well as adequately address parents’ concerns. 35 A major limitation of the Triple P programs is the target population. Consistent with the Triple P public health model, approximately half of the studies in a recent meta-analysis appeared to target parents and children within the normal range of functioning. For 22 of the study samples included, the mean child problem behavior score fell in the normal range of functioning, 10 study samples fell within the at-risk (subclinical) range, and 13 studies fell within the clinically significant range (Nowak & Heinrichs, 2008). This data indicates that research on the application of the Triple P programs with children outside of the “normal range of behavioral functioning” is needed. Lastly, the cost of each Triple P program is not publicly listed. Interested parents must email or call to request information. Only the Triple P online format is advertised starting at $80 (Triple P International, 2018). Based on the costs of other evidence-based parent training programs (e.g., Incredible Years BASIC program, PMTO, PCIT), Triple P programs are likely comparable in price and presumably increase in price the more intense the intervention. Given that families must email or call to request information about Triple P programs, it may also be difficult for parents to access this treatment, as Triple P is offered in the United States, Australia, and a few European countries. For clinicians, the time commitment and cost of training differs depending on the program. Training generally consists of 5-6 days and includes pre-accreditation, training, and accreditation workshop days and costs between $1,525 - $2,520. However, there are limited training options within the United States—the majority of training workshops occur in Australia. For example, there were only two training workshops listed for the 2019 calendar year in California and South Carolina. Since the global pandemic, Triple P International has offered an increasing number of trainings online to U.S. participants for the 2021 calendar year; no in- 36 person trainings are offered. However, most workshops are for training in Level 4 and Level 5 programs, which are programs for children with more severe behaviors. No trainings are being offered for Level 1 or Level 2 Brief Primary Care for the 2021 calendar year. Despite the reported benefits of the Triple P program, necessary training to implement such programs can be difficult to obtain within the U.S. Parent-Child Interaction Therapy Parent-Child Interaction Therapy (PCIT; Eyberg, 1988) is an evidence-based parent training intervention for parents with children between the ages of 2 to 7 years (McNeil & Hembree-Kigin, 2011). The goal of PCIT is to reduce child disruptive behavior by enhancing the parent-child relationship and improving parenting skills (McNeil & Hembree-Kigin, 2011) through two intervention phases: Child-Directed Interaction (CDI) and Parent-Directed Interaction (PDI). In CDI, the goal is to establish warmth in the parent-child relationship, while in PDI, the goal is to teach parents how to manage the most challenging child behaviors (Eyberg & Funderburk, 2011; McNeil & Hembree-Kigin, 2011). PCIT is unique from other parent training interventions in that parents receive didactic training in positive parenting practices, as well as in-vivo practice and coaching on their parenting skills during sessions. Parent coaching on skill acquisition has been cited as an important mechanism of change (Barnett et al., 2017; Shanley & Niec, 2010). On average, families receive between 12-20 one-hour weekly sessions of PCIT (McNeil & Hembree-Kigin, 2011). PCIT has a well-established research base that suggests the efficacy and effectiveness of the intervention in treating child behavior problems. Several empirical studies have demonstrated that PCIT is associated with positive effects, including improvements in child externalizing behaviors (e.g., Eyberg et al., 2008; Thomas & Zimmer-Gernbeck, 2007), gains in positive 37 parenting skills, decreases in parental stress, and increased parental confidence in their ability to manage challenging child behaviors (Abrahamse et al., 2012; Bjørseth & Wichstrøm, 2016; Hood & Eyberg; 2003; Nixon et al., 2003). PCIT has also been found superior to waitlist control conditions in reducing disruptive behavior in young children (Bjørseth & Wichstrøm, 2016; Nixon, et al., 2003; Schuhmann et al., 1998). A meta-analysis of 12 PCIT studies found that PCIT had a large effect (d = 1.65) on improving externalizing behavior problems in children with clinically significant disruptive behavior disorders from pre-and post-treatment behavioral outcomes (Ward et al., 2016). Another meta-analysis of 11 studies found that PCIT demonstrated a large mean effect size (d = 1.06) in reducing child externalizing behaviors, as well as moderate to large effect sizes (d = 0.73 – 0.94) in reducing parenting stress (Cooley et al., 2014). In a comprehensive meta-analysis of 24 studies comparing the effects of PCIT with the Triple P Positive Parenting Program, PCIT demonstrated significantly larger effect sizes for reducing negative parent behaviors (d = 1.11 compared to d = 0.70), and child behavior problems (d = 1.45 compared to d = 0.51 –0.69) than did most or all forms of Triple P (Thomas & Zimmer-Gembeck, 2007). While PCIT has been established as an efficacious intervention in reducing child externalizing behaviors and improving positive parenting skills, a few limitations should be noted. First, research demonstrates that PCIT is an efficacious treatment for children with disruptive behavior disorders (e.g., ODD and CD). Additionally, although several studies have demonstrated PCIT to be effective in community-based settings (e.g., Budd, et al., 2011; Lyon & Budd, 2010) the majority of PCIT studies have utilized randomized controlled trials to establish the efficacy of PCIT. For example, several studies that have examined PCIT in community mental health agencies are randomized controlled trials (Lieneman, et al., 2017). Both Ward and 38 colleagues (2016) and Cooley and colleagues (2014) indicated a need for community-based effectiveness studies to determine the effectiveness of PCIT among individual populations in community settings, including community mental health agencies, outpatient clinics, child- welfare agencies, and home-based community services. Moreover, to date, the majority of PCIT efficacy and effectiveness studies target children between the ages of 2-5 years with clinically significant behavior problems (Ward et al., 2016). Future research examining the use of PCIT with other populations (e.g., children with subclinical behavior problems) in community-based settings may increase the transportability of PCIT, as well as increase access to treatment for families. Costs Associated with PCIT. Although PCIT is effective in the treatment of disruptive behaviors, it is a costly intervention in terms of training for the therapist and treatment costs for the family. For families, treatment costs range from $1200-1500 for one child (Goldfine et al., 2008). First, certified therapists deliver PCIT. Certified PCIT therapists are individuals who have received appropriate and sufficient PCIT training to be qualified to provide PCIT services (PCIT International, 2013). In order to become a certified therapist, an individual must: 1) have a master’s degree or higher in a mental health field and be an independently licensed mental health service provider (e.g., licensed psychologist, licensed clinical social worker, etc.) OR be at least a fourth year psychology doctoral student who is conducting clinical work under the supervision of a licensed mental health service provider; 2) complete 40 hours of initial training with a PCIT trainer, which includes coding practice, case observations, and coaching with families; and 3) complete continuation training over the course of 1 year, following completion of initial training. In continuation training, the individual must serve as the therapist for a minimum of two PCIT cases and see them to graduation criteria (as defined by the 2011 PCIT Protocol; Eyberg & 39 Funderburk, 2011) and participate in twice a month consultation with a PCIT trainer for supervision to assess coaching and coding skills and address complex treatment issues. To date, there are three levels of certified trainers: Level 1 Trainer, Level II Trainer, and a Master Trainer (please refer to Appendix B for a complete list of qualifications and training requirements). Once an individual becomes a Certified PCIT Therapist, (s)he is required to obtain at least 3 hours of PCIT continuing education credit every two years in order to become re- certified. Initial training plus certification training costs $4000 to $4700, while PCIT continuing education credits cost $35 to $45 per credit (one-hour long videos; PCIT International, 2013). In addition to the cost of intensive PCIT training and recertification, there are start-up and ongoing expenses associated with implementing PCIT. Initial expenses for PCIT include purchases of audio-visual equipment to enable coaching behind a one-way mirror using a bug-in- the-ear device, while maintenance expenses include treatment assessment measures, office space, time-out chairs, and child toys. The cost of equipment needed to implement PCIT is about $14,000 (Goldfine et al., 2008). Given that the costs of implementing PCIT were examined in 2008, it is likely that these costs are considerably higher today. The cost of implementing PCIT may discourage clinicians from becoming certified, thus resulting in less certified PCIT therapists and decreasing the availability of an effective parent-child treatment for families. Availability of PCIT therapists is variable in the United States. According to PCIT International (updated May 2021), there are certified PCIT therapists in all 50 United States. However, the number of certified therapists in each state ranges from one to fifty-six. While there may be a high number of certified therapists in one state (e.g., 56), these therapists are typically located in major cities (e.g., Chicago, Miami, New York City) and/or at major research institutions (e.g., Stanford University, Colombia University Clinic, Duke University Medical 40 Clinic). Thus, it is likely that families distal to major cities who are seeking PCIT treatment often face additional expense and the burdens associated with travel. Treatment of PCIT can also be expensive for families. According to Goldfine and colleagues (2008), PCIT treatment costs range from $1200-1500 for one child. If accepted by the provider, insurance may partially cover the cost of PCIT; however, many therapists require families to pay out of pocket, depending on their health plan. While insurance may partially cover PCIT, there is a wide variability in coverage, depending on the health plan. For example, Blue Cross covers 80 percent of PCIT after the deductible (which ranges from $1200-$7000) is met; Blue Cross Blue Shield Association, 2018). Coverage also may depend on whether PCIT treatment is billed as individual psychotherapy or family therapy, which varies among PCIT therapists. There are many complexities in determining insurance coverage of PCIT. It is not possible to determine the exact amount of coverage that specific insurance companies will pay, as the coverage differs depending on the family’s insurance plan and the treatment code that the therapist uses for billing PCIT. Thus, only families who are able to afford mental health services, either through out-of-pocket payment and/or health insurance, are able to access treatment. Summary of Evidence-Based Training Programs There are several empirically supported parent training interventions that have proven to be efficacious and effective in decreasing child-externalizing behaviors through the improvement of parenting practices. While PMTO, Incredible Years BASIC program, Triple P, and PCIT are all evidence-based interventions for the treatment and prevention of disruptive behavior disorders and aim to address key positive parenting practices, only PCIT contains the most active parent training components that are correlated with positive parent and child outcomes. Most 41 notably, PCIT focuses on dyadic (i.e., parent-child) and triadic (i.e., parent-child-professional) strategies by including the child in treatment and providing in-vivo feedback during sessions. Given that PCIT has a well-established research base that suggests the efficacy of the intervention in treating child behavior problems, aims to improve the parent-child relationship and increase positive parenting practices, includes the parent(s) and the child during treatment, and allows parents to receive feedback on their skills in real time, PCIT is likely to have the strongest impact on the prevention of child problem behaviors compared to other parent training programs. Parent-Child Interaction Therapy (PCIT) The following section describes the components of PCIT in greater detail and critically reviews the current empirical research to provide support for the potential benefits of a brief, modified version of PCIT that is aimed at preventing the emergence of clinically significant behavioral issues in young children. PCIT Structure and Content PCIT is traditionally implemented across two treatment phases. Across the two treatment phases, core elements include didactic training in specific parenting skills, role-playing and modeling, observations of parent-child interactions, and in-vivo coaching (refer to Table 1 for a general outline of PCIT sessions). The first phase of treatment is called child-directed interaction (CDI). In CDI, the goal is to establish warmth in the parent-child relationship through learning and applying skills that have been proven to help children feel calm, secure in their relationship with their parents, and good about themselves (McNeil & Hembree-Kigin, 2011). The skills parents learn during CDI are called PRIDE skills, which stand for praising the child’s appropriate behavior, reflecting the child’s appropriate talk, imitating the child’s appropriate 42 play, describing the child’s appropriate behavior, and enjoying the interaction with the child (Eyberg & Funderburk, 2011; refer to Appendix C). PRIDE skills are also referred to as “CDI skills.” In the CDI phase of treatment, parents also learn what not to do during CDI sessions. Parents are instructed to avoid asking questions (e.g., “What should we do next?”), using criticism (e.g., “The train track does not go that way” or “You’re not being very nice”), and giving commands (e.g., “Get off the chair” or “Sit next to me;” Eyberg & Funderburk, 2011). Table 1. General Outline of PCIT CDI Sessions Treatment Phase Session Goal of Session Pre-treatment Code parent-child interactions for praise, Phase 1: CDI reflections, behavioral descriptions, questions, commands, and negative talk. CDI Teach Give an overview of PCIT, introduce CDI, demonstrate and practice PRIDE skills, give CDI homework. CDI Coach 1 Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on behavioral descriptions. CDI Coach 2 Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on reflections and avoiding questions. CDI Coach 3 Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on labeled praises. CDI Coach 4+ Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on whichever PRIDE skill is the weakest. Note: General outline was created using the PCIT Protocol (Eyberg & Funderburk, 2011). As part of treatment, families are assigned homework. For homework, the parent should spend five minutes at home with the child, individually, every day for “special time.” Special time provides an opportunity for parents to practice the parenting skills they learned during the 43 PCIT sessions, and strengthens the parent-child relationship (Blizzard et al., 2017). For special time, parents are instructed to provide the child with three toy options that (s)he can play with during special time that are constructive and allow for creativity, such as building blocks, Legos, crayons and paper, doll houses, train tracks, etc. Parents are advised to avoid toys during special time that encourage rough play (e.g., bats, balls), may lead to aggressive play (e.g., toy guns or swords, super-hero action figures), require limit setting (e.g., painting), have pre-set rules (e.g., board games, card games), and discourage conversation (e.g., books, video games). Such toys should be avoided so that the parent(s) can follow the child’s lead during play and so that special time remains positive. For example, a board game that has pre-set rules can lead to a negative interaction if someone cheats or is competitive and gets upset by the outcome of the game. Once parents meet CDI mastery criteria (discussed in the next section), they progress to the second phase of treatment: PDI. This second phase of treatment builds on parents’ skills by teaching them a sequential discipline method that emphasizes consistency, predictability, and following through (Eyberg & Funderburk, 2011). In PDI, the goal is to teach parents how to manage the most challenging child behaviors while remaining confident, calm, and consistent in their approach (Eyberg & Funderburk, 2011). During PDI, parents learn strategies to help the child accept limits, comply with directions, respect house rules, and demonstrate appropriate behavior in public. These strategies include giving effective commands and appropriately using a scripted time-out procedure for noncompliance. It should be noted that the intervention proposed for the present study will not include the PDI phase of treatment. As stated above, parents learn a structured and consistent approach to discipline during the PDI phase in order to manage more severely defiant behaviors (Eyberg et al., 2008). Given that participating children in this study will exhibit elevated, yet subclinical 44 levels of behavior, it is expected that parents will be able to effectively improve their child’s behavior with CDI skills alone. PCIT requires the participation of only one parent; however, both parents can participate if they desire. Attendance in PCIT is important to make progress in treatment. If families miss three or more sessions without rescheduling, the clinician may terminate treatment, as treatment will not be effective without regular attendance (Eyberg & Funderburk, 2011). With consistent attendance and homework completion, traditional PCIT (CDI + PDI) can be completed within 12-20 sessions (McNeil & Hembree-Kigin, 2011). Treatment is considered complete when the parent has mastered both CDI and PDI skills and rates the child within normal limits of behavior on the Eyberg Child Behavior Inventory (ECBI; Eyberg & Funderburk, 2011; Eyberg and Pincus, 1999). Detailed CDI Components Prior to beginning PCIT, the parent(s) and child complete a pre-treatment observation that is rated using the Dyadic Parent-Child Interaction Coding System (DPICS; Eyberg et al., 2013). During a DPICS observation, the parent and child play together in the treatment room with five toy options. The therapist observes from behind a one-way mirror and codes the parent for specific CDI skills, including the number of behavioral descriptions, reflections, labeled praises, commands, negative talk, and questions (s)he uses. Following the DPICS observation, the family begins didactic training with the therapist. Didactic training in CDI skills occurs in one, one-hour session before the CDI phase of treatment begins and is referred to as the “CDI teach” session. During the CDI teach session, the parent(s) meet with the therapist without the child to learn the CDI (i.e., PRIDE skills) and practice role- playing situations with the therapist using CDI skills. 45 After the CDI teach session, parents start CDI coaching sessions. During coaching sessions, the therapist meets with the parents to check in, review the ECBI scores and homework, observe the parent-child interactions, and provide coaching for and feedback on the parent’s skills. During CDI coaching sessions, the parent and child are in a playroom while the therapist observes the parent and child interact through a one-way mirror. Parents wear a “bug-in-the-ear” device (e.g., headphones) so the therapist can provide in-vivo coaching on the skills the parent is learning to manage the child’s behavior without interfering with the parent-child interactions. At the beginning of each CDI coaching session, the therapist observes the parent-child interaction for five minutes to assess the parent’s use of PRIDE skills. These coded observations are used to determine when CDI mastery criteria have been met and the PDI phase of treatment can begin. According to the PCIT Protocol (Eyberg & Funderburk, 2011), the parent has met mastery in CDI skills when (s)he is observed by the therapist to give 10 behavioral descriptions, 10 reflections, and 10 labeled praises in at least one of the 5 minute CDI DPICS observations (refer to Appendix A for examples). Typically, CDI mastery criteria is met by the fourth CDI coaching session; however, parents may require additional CDI coaching sessions in order to meet mastery (Eyberg & Funderburk, 2011). Progress Monitoring of Problematic Behavior. Throughout PCIT treatment, the therapist uses the ECBI to progress monitor the child’s problematic behavior. Before every PCIT session, the parent(s) fill out an ECBI to rate the child’s behavior from the previous week. The ECBI is an established 36-item parent rating scale that measures disruptive problem behaviors in children between 2 and 16 years old (Abrahamse, Junger, Leijten, Lindeboom, Boer, & Lindauer, 2015). The ECBI has two different scales to assess disruptive behavior: Intensity Scale and Problem Scale. For every item, parents are asked how often their child exhibits the 46 disruptive behavior (Intensity Scale) and whether or not they find this behavior problematic (Problem Scale; Eyberg and Pincus, 1999). The ECBI is widely used and evidences strong psychometric properties. It has been used in the United States, Europe (e.g., Sweden, Norway, Netherlands), and Asia (e.g., Japan, South Korea, and China) and translated into several languages (Abrahamse et al., 2015). The ECBI has been used in clinical and research settings for early screening of disruptive child behavior (Abrahamse et al., 2015) and has shown to be sensitive in measuring the effect of treatment on such behavior problems (Nixon et al., 2003). Additionally, the ECBI has good retest reliability (r = 0.75) over a 10-month period (Funderburk, Eyberg, Rich, & Behar, 2003) and the two scales show high internal consistency (α > 0.90; Colvin, Eyberg, & Adams, 1999). The ECBI Intensity Scale also has good construct validity with other questionnaires that assess child behavior problems, including the Child Behavior Checklist (CBCL; r = 0.75; Boggs, Eyberg, & Reynolds, 1990) and the Strengths and Difficulties Questionnaire (SDQ; r = 0.68; Axberg, Johansson, & Broberg, 2008; Butler, 2011). Furthermore, the ECBI has demonstrated sensitivity to treatment effects (Schumann et al., 1998). Graduation Criteria. In order to graduate from traditional PCIT, the parent(s) must have previously met CDI and PDI mastery criteria, respectively. Additionally, the child’s ECBI Intensity Score must also be below a raw score of 114 (i.e., a half standard deviation above the normative mean; Eyberg & Funderburk, 2011). If the ECBI score has not decreased over the course of treatment, the therapist will have previously worked to problem solve with the parent to address behavior problems. If the ECBI score is still high after the parent has met CDI and PDI mastery, the therapist may potentially suggest individualized therapy for the child to address “leftover” problem behavior after completing PCIT (Eyberg & Funderburk, 2011). 47 Prior to the graduation ceremony, families complete a post-treatment DPICS observation. During the DPICS observation, the parent(s) must again demonstrate mastery of CDI and PDI skills or be close to mastery (i.e., one or two marks away from 10 behavioral descriptions, reflections, or praises and close to giving 75 percent of effective commands; Eyberg & Funderburk, 2011). If both parents are participating in PCIT, each parent must meet graduation criteria unless the primary caregiver has reached the mastery criteria and the family either has no problems with public behavior and aggressive behavior requiring house rules or the family has successfully applied the house rules and public behavior procedures during the course of treatment (which could occur during the fifth or later PDI sessions; Eyberg & Funderburk, 2011). Adaptations to Standard PCIT In addition to demonstrating the efficacy of PCIT for young children with disruptive behavior problems, researchers have begun to examine the effectiveness of PCIT within various community treatment settings. Many studies have demonstrated improvements in child behavior and positive parenting skills, and decreases in negative parenting skills for families receiving PCIT in community treatment settings (e.g., community mental health centers, domestic violence shelters, child welfare agencies, and in-home services by community agencies) in the U.S. (e.g., Budd, Hella, Bae, Meyerson, & Watkin, 2011; Danko, Garbacz, & Budd, 2016; Lyon & Budd, 2010). Improvements in positive parenting skills and decreases in child disruptive behavior were also shown in child-welfare settings (Lanier, Kohl, Benz, Swinger, & Drake, 2014). In some standard PCIT studies, CDI training (CDIT) alone has been shown to reduce behavior problems to below clinical levels for almost half of children who presented with a clinically significant behavior disorder prior to treatment (Bagner & Eyberg, 2007; Eisenstadt et al.; 1993; Harwood, et al., 2009). CDI focuses on increasing parental warmth and strengthening 48 the parent-child relationship by creating an environment in which the child views play and parent-child interactions as positive, rewarding experiences (Masse, et al., 2007). Components of CDI, including parent implementation, utilization of a naturalistic environment, modeling, and differential reinforcement have been established as evidence-based practices to reduce disruptive behavior in children (Wong et al., 2013). A study by N’zi and colleagues (2016) examined how application of CDIT influenced parenting skills and child behavior outcomes. The researchers implemented a pilot study to examine the feasibility and outcomes of a twice weekly, 8-session CDIT program for children living with kinship caregivers (e.g., grandparents). Participants included 14 grandmothers and great-grandmothers with their 2-7-year-old children (n = 7 participants in the control group and n = 7 participants in the treatment group). Participating children could not have a diagnosis of Autism Spectrum Disorder and needed a caregiver rating one standard deviation above the normative mean on the Problem Scale of the Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999). Children in the study were 50% female with a mean age of 5.2 years. The CDIT sessions followed the standard PCIT protocol (Eyberg & Funderburk, 2011) for the child- directed interaction phase and were implemented in a room in a public library. In the first session, caregivers learned the “do skills” (i.e., positive parenting skills such as behavior descriptions, reflections, and labeled/unlabeled praise) and “don’t skills” (i.e., criticisms, questions, and commands). Caregivers were coached in their skill development in the following seven sessions and were required to complete weekly homework (e.g., to have five minutes of “special time” at home). In regard to the feasibility of CDIT, attrition was 0% during the intervention; however, one participant in the treatment group was lost to a 3-month follow-up. Five of the seven 49 caregivers (71%) who received CDIT achieved CDIT mastery criteria for positive parenting skills and four of seven caregivers (57%) met mastery criteria for avoiding negative parenting behaviors. On average, homework completion rate was 62%. CDIT trainers obtained 96% accuracy with regard to treatment fidelity. Results of the study also indicated that kinship caregivers in the treatment group significantly improved in positive parenting skills and utilized less negative parenting behaviors compared to caregivers in the control group. Additionally, caregivers in the treatment group rated lower levels of child externalizing behavior problems and lower levels of stress (N’zi, Stevens, & Eyberg, 2016). Ginn and colleagues (2017) examined outcomes for children with ASD and their mothers following 8 weeks of the CDI phase of treatment only, as several components of CDI (e.g., parent implementation, naturalistic environment, modeling, differential reinforcement) have been established as evidence-based practices for children with ASD (Wong et al., 2013). Sessions were 60-75 minutes and held once per week. Thirty mother-child dyads with children ages 3-7 years with an ASD diagnosis participated in this randomized controlled study (n = 15 in the treatment group and n = 15 in the control group). Each CDIT session followed the standard CDI sessions from the PCIT protocol (Eyberg & Funderburk, 2011). Mothers were taught positive parenting skills (e.g., behavior descriptions, reflections, labeled praises) and were instructed to avoid negative parenting behaviors (e.g., questions, commands, critical statements) in the first CDIT teach session and received coaching sessions in the subsequent weeks (Ginn et al., 2017). Posttreatment outcomes of the study provide positive support for the use of CDIT. Mothers in the treatment group reported practicing their skills at home 70% of the days and 50% of mothers obtained CDIT skill mastery. Findings demonstrated significant changes in parenting behaviors and reductions in disruptive behavior following the CDI phase of treatment (Ginn et 50 al., 2017). More specifically, mothers in the CDIT group demonstrated significantly more positive parenting skills and less negative parenting behaviors. They also reported lower levels of stress. Mothers in the CDIT group also reported fewer disruptive behaviors in their children. At the 6-week follow-up, maternal stress was reduced, and parent and child outcomes were maintained for mothers in the treatment group. Positive findings from these studies indicate that caregivers are able to implement CDI (i.e., positive parenting) skills with children to significantly change their child’s problematic behavior, thus providing initial support for the use of the CDI phase only. Based on these results, a relatively brief parent training intervention, such as CDIT, shows promise in decreasing child disruptive behaviors. Brief parent training interventions like CDIT also increase accessibility for parents. By decreasing problematic behaviors early on in children who demonstrate emerging (subclinical) behavior problems, an intervention such as CDIT can help prevent the development of more severe problems. Thus, the present study proposes an extension of the research that has been done with CDIT by proposing the use of only the CDI phase of PCIT with parents of children who exhibit emerging behavior problems in order to prevent later development of more severe problems. The Present Study PCIT is well established as an efficacious intervention for the treatment of disruptive behavior problems. However, traditional PCIT is not accessible to all parents and children requiring services. As demonstrated in several empirical studies, a child typically has to hold a diagnosis of a disruptive behavior disorder (e.g., ADHD, ODD, or CD) in order to receive the treatment. If a child already has a diagnosis, this indicates that the child is already exhibiting clinically significant levels of disruptive behavior by the time they seek treatment. Furthermore, 51 if an increasing number of individuals require intensive intervention, the service delivery system will not be able to provide adequate supports and intervention, which results in negative and costly outcomes for the individuals, their families, and the greater public. Thus, preventative efforts need to increase, as a reduction in problematic behavior symptoms decreases the need for services. The current study draws upon principles of prevention by proposing an adaptation to PCIT to treat children exhibiting subclinical behaviors, thus intervening before the behavior reaches clinically significant levels. Currently, large gaps in the research exist concerning preventive programs for preschool-age children with emerging behavior problems (Berkovits et al., 2010). Rationale for Modified PCIT The proposed modified version of PCIT has theoretical foundations in attachment and social learning theories, as well as empirical support. The focus of this modified PCIT version is on child-directed interactions (CDI) and teaching positive parenting skills. More recently, researchers have begun to examine caregiver and child outcomes following implementation of the CDI phase of PCIT only. Results of these studies indicated that caregivers significantly improved in positive parenting skills, child externalizing behavior problems were reduced, and caregivers reported lower levels of stress (Ginn et al., 2017; N’zi, Stevens, & Eyberg, 2016). Positive findings from these studies suggest that parents are able to implement CDI with children to reduce their child’s problematic behavior, thereby providing initial support for the use of the CDI phase only. The modification proposed in the current study adds to existing research by examining the use of a modified version of CDI with mothers and their children who exhibit subclinical behavior concerns. The CDI-only phase of PCIT has shown promising results with 52 other populations (e.g., Ginn et al., 2017; N’zi, Stevens, and Eyberg; 2016), but has yet to be examined with children exhibiting subclinical behavior concerns. Furthermore, in standard PCIT studies, the majority of child participants have clinically significant externalizing behavior problems (i.e., a disruptive behavior disorder such as ADHD, CD, and/or ODD; Ward et al., 2016). Given that participating children in the present study must exhibit elevated, yet subclinical levels of behavior, participating parents are not likely to need extensive training in the disciplinary procedures provided in the PDI phase of PCIT (Eyberg, Nelson, & Boggs, 2008). In some studies, CDI alone has been shown to reduce child behavior problems to below clinical levels for almost half of children who presented with clinically significant behavior prior to treatment (e.g., Bagner & Eyberg, 2007; Eisenstadt et al., 1993). Modifications Due to Covid-19 After the initial study was proposed, the Covid-19 pandemic occurred. Given the spread of COVID-19 and the need to reduce in-person contact, mental health services were delivered via Telehealth. According to the World Health Organization (WHO), Telehealth involves the use of telecommunications and virtual technology to deliver health care services outside of standard- care facilities, thus expanding the scope of healthcare services (2020). Virtual technologies include use of videoconferencing, the internet, streaming media, and wireless communication (e.g., Bluetooth earpieces). Thus, the initial study was modified from in-person service delivery in a community-based setting to delivery via Telehealth. Comer and colleagues (2015) have suggested that PCIT may be feasible via video- teleconferencing format. Internet-delivered PCIT (I-PCIT; Comer et al., 2015) follows standard, clinic-based PCIT, but uses videoconferencing technologies to allow therapists to remotely 53 deliver treatment to families in their homes. I-PCIT can afford a comparable quantity of therapist contact relative to standard, clinic-based PCIT. In standard PCIT, the therapist monitors the parent-child interactions from another room and provides real-time feedback to the parents via a bug-in-the-ear device. With I-PCIT, therapists can remotely provide real-time feedback during parent-child interactions in the home (e.g., if a child has a behavioral outburst during an intervention session), whereas they would not have such opportunity to receive coaching in a more naturalistic environment in clinic based PCIT. Evidence for I-PCIT. Although the provision of mental health services via Telehealth is more recent, the use of I-PCIT has been researched. Comer and colleagues (2017) compared I- PCIT to standard, clinic-based PCIT using a randomized controlled trial with 3-5-year-olds (N = 40) with a diagnosed disruptive behavior disorder and their caregiver(s). The researchers evaluated child diagnostic outcomes, global severity and impairment, disruptive behavior symptoms, caregiver burden, barriers to treatment, and treatment satisfaction across the two conditions at several timepoints. Results demonstrated that I-PCIT and clinic-based PCIT were largely comparable on most outcomes. For example, treatment satisfaction and engagement were high across both conditions as rated on the Therapy Attitude Inventory (TAI; Minternet = 45.9 vs, Mclinic = 45.1; highest possible score = 50), there was no reported difference of therapeutic alliance across conditions, and child externalizing behavior problems improved significantly over time across the two conditions as rated on the Eyberg Child Behavior Inventory (ECBI; pre-Minternet =153 vs. pre-Mclinic = 164; post-Minternet = 74 vs. post-Mclinic = 73) . However, I-PCIT showed a significantly higher rate of “excellent responders” (meaning they received a CGI Improvement score of 1, the highest score of improvement on the Clinical Global Impression-Severity and 54 Improvement Scales [CGI-S/I; Guy & Bonato, 1970]). Additionally, 65% of children who received I-PCIT lost their initial disruptive behavior diagnosis at posttreatment (d = .27) compared to 50% of children who received clinic-based PCIT (d = .36). Although children in both conditions demonstrated maintenance of gains, children in the I-PCIT condition had slightly higher rates. Finally, caregivers receiving I-PCIT reported significantly fewer barriers to treatment than caregivers receiving clinic-based PCIT. These findings show promise for the use of I-PCIT to remotely deliver parenting training in the home (Comer et al., 2017). Proposed Single-Case Design As an important step in examining the use of modified PCIT (referred to as CDIT) as a potential evidence-based preventative program for children with subclinical levels of behavior, the current study examined the feasibility, effectiveness, and acceptability of CDIT with four mothers and their preschool-age children exhibiting emerging behavior problems using a nonconcurrent multiple probe design. Although there are many variations of single-case designs that can be used to critically examine intervention effectiveness, a multiple-probe design avoids most ethical and practical issues, while still utilizing rigorous methods to investigate a treatment effect. For example, a multiple-probe design does not require that treatment be withdrawn in order to demonstrate a treatment effect, as the intervention is staggered across participants (Gast & Ledford, 2014). Additionally, a multiple-probe single-case design is rigorous in its evaluation of threats to internal validity and has been used to investigate the effectiveness of interventions in educational and clinical settings (Gast & Ledford, 2014). In this design, threats to internal validity are reduced due to the characteristics of the multiple-probe design. These characteristics include: (a) well-defined experimental manipulations of the independent variable, (b) hypotheses proposed a 55 priori, (c) frequent formative assessments of the dependent variables, (d) visual depictions of whether changes in the dependent variables coincide with manipulation of the independent variable, and (e) opportunities to demonstrate replicated effects across multiple data series or subjects (Gast & Ledford). At least three attempts to demonstrate the intervention effect (e.g., by including at least three participants) at three different points in time are necessary to draw valid conclusions about the overall effect of the intervention using a multiple-probe design (Kratochwill et al., 2013). The greater the number of data points in a phase, the more confidence there is in capturing the pattern of data. To ensure methodological rigor, the design of the study was developed using the What Works Clearinghouse standards for single-case design (Kratochwill et al., 2013). Research Questions and Hypotheses Data was collected through measures of treatment fidelity, systematic observations of parenting skills, parent reports of child externalizing behaviors, and parent ratings of treatment acceptability and satisfaction. The following four research questions and related hypotheses were examined. Research Question 1 Are there changes in parents’ parenting skills following implementation of the CDIT intervention? Research Question 1a: Are improvements in parents’ positive parenting skills (primary outcome variable) evident following implementation of the CDIT intervention? Research Question 1b: Are decreases in parents’ negative parenting skills evident following implementation of the CDIT intervention? The primary goal of the current study was to provide evidence for the potential efficacy of CDIT in increasing positive parenting skills among parents of young children with behavioral 56 risk. It was hypothesized that positive parenting skills (“Do Skills”), including giving labeled praise, using behavior descriptions, and reflecting children’s verbalizations, would increase following the introduction of the intervention, while negative parenting skills (“Don't Skills”), including using criticism, asking questions, and giving negative commands, would decrease. This hypothesis was consistent with results from N’zi and colleagues (2016) who reported that 71% of caregivers achieved CDIT mastery for positive parenting skills and 57% of caregivers met mastery criteria for avoiding negative parenting behaviors. Ginn and colleagues (2017) provide further support. They reported that mothers in the CDIT group demonstrated significantly more positive parenting skills and less negative parenting behaviors compared to mothers in the control group (Ginn et al., 2017). Research Question 2 Do children’s externalizing behaviors (defined as “any non-preferred behavior” including aggression, opposition, and noncompliance) decrease after implementation of the CDIT intervention? A second goal of the current study was to provide support for the potential effectiveness of CDIT in reducing child externalizing behaviors. It was hypothesized that children’s externalizing behaviors would decrease following the introduction of the intervention and remain lower than baseline levels. This hypothesis was also supported by findings from N’zi and colleagues (2016) and Ginn and colleagues (2017). Both studies reported lower levels of child externalizing behavior problems following CDIT. It should be noted that reductions in children’s problem behaviors were not expected to be as sizable as those of children with clinically significant behavior problems. However, decreases in parent-reported child externalizing behaviors were expected, as prior studies have demonstrated that children who have parent- 57 reported subclinical levels of behavior concerns pre-treatment decreased in their levels of problem behavior after receiving an abbreviated version of PCIT (Berkovits et al., 2010). Research Question 3 Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth? Research Question 3a: Can CDIT be implemented with integrity by a doctoral- level student (the study clinician) via Telehealth as indicated by high adherence to the treatment protocol? Research Question 3b: Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth as indicated by high quality of intervention delivery? Research Question 3c: Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth as indicated by high dosage (i.e., parent attendance) of the intervention? Research Question 3d: Can CDIT be implemented with integrity by a doctoral- level student (the study clinician) via Telehealth as indicated by high participant responsiveness to the intervention? Treatment integrity was analyzed across several dimensions, including adherence to the treatment protocol, dosage of the intervention, quality of intervention delivery, and participant responsiveness. First, adherence to the treatment protocol was considered. One of the benefits of CDIT is that it follows the manualized standard PCIT protocol for child-directed interaction (Eyberg & Funderburk, 2011). The use of manualized treatments increases the likelihood that treatments will be carried out as intended (Perepletchikova & Kazdin, 2005). Four mothers and their children with emerging behavior problems received at least six treatment sessions from a doctoral student in a home-based setting. The use of the standard PCIT protocol (Eyberg & Funderburk, 2011) for the CDI phase of treatment and integrity checklists for each session (derived and adapted from the standard PCIT protocol; refer to Appendices F-M for integrity 58 checklists) helped support the clinician in the implementation of CDIT for all four cases. It was hypothesized that the clinician would implement CDIT with high treatment adherence. More specifically, an overall adherence percentage of 80% or higher was expected per individual sessions across all cases and per mother-child dyad, which is considered to be the minimum rating required for adequate treatment adherence (Kratochwill et al., 2013). Excellent treatment adherence for CDIT by treatment adherence reports from Ginn and colleagues (2017; M = 98%) and N’zi and colleagues (2016; M = 96%) supported this hypothesis and using this standard. Dosage of the intervention and quality of intervention delivery were also evaluated as measures of treatment fidelity. Dosage of the intervention was evaluated by examining attendance at treatment. Quality of intervention delivery referred to the degree to which the clinician implemented triadic and collaborative strategies (e.g., clinician was warm and sensitive towards families, provided ample opportunities for collaboration, initiated meaningful conversations with the parents, did not interrupt the parent while talking, asked open-ended questions, frequently encouraged parental participation). Quality of intervention delivery was measured by a global rating of the clinician’s use of triadic and collaborative strategies using a 4- point Likert scale (1= low quality and 4 = high quality). This scale was consistent with measures of the quality of intervention delivery from prior parent-training intervention work (e.g., Sheridan et al., 2008). It was hypothesized that participants would consistently attend weekly treatment sessions (e.g., 90-100% of sessions), given the brevity of the intervention. This hypothesis was supported by Ginn and colleagues (2017) who reported low attrition rates, as well as by N’zi and colleagues (2016) who reported 0% attrition rates. Additionally, it was hypothesized that the study clinician would have high ratings (as indicated by an average overall score of 3 or higher 59 on the Likert scale). High quality intervention delivery was expected based on the lead clinician’s training in PCIT and given that the PCIT protocol emphasizes the use of triadic and collaborative strategies (Eyberg & Funderburk, 2011). Participants’ engagement in the intervention was also evaluated as a measure of treatment fidelity. This variable was evaluated by examining the completion of homework assignments. It was expected that, on average, homework completion rates would be above 50%. These hypotheses were supported by Ginn and colleagues (2017) who reported low attrition rates and homework completion rates above 50%, as well as by N’zi and colleagues (2016) who reported 0% attrition rates and homework completion rates above 62%. Research Question 4 What are parent ratings of intervention satisfaction, which includes intervention acceptability and feasibility, after receiving the CDIT intervention? High overall treatment acceptability and satisfaction across all five mothers was anticipated. Consistent with previous psychometric work and clinical work, a total score of acceptability and feasibility was used from a psychometrically validated measure typically used to evaluate parent satisfaction with PCIT. While prior studies examining CDIT did not measure caregiver satisfaction, this hypothesis was based on results from prior studies of standard PCIT in which high levels of caregiver satisfaction were reported (e.g., Bagner et al., 2013; Berkovits et al., 2010). Furthermore, high ratings of maternal satisfaction and intervention feasibility were also expected, given the present study’s attempt to reduce the length of treatment and prevent more severe externalizing behaviors. 60 CHAPTER III. METHOD Participants Four child participants between the ages of three and six years old and their mothers participated in the study. Secondary caregivers were also invited to participate in the study (rationale discussed in the “study phases” section). Only one father chose to participate in the intervention, along with the mother and child. Data from the father was considered a case study and not part of the multiple probe design, as it was optional for the second caregiver to participate in the study. The first four eligible families who met the study’s inclusion criteria were enrolled in the study. Recruitment Process Several changes were made to the originally proposed recruitment process due to the Covid-19 pandemic. An in-person pilot study (discussed later) was conducted in January 2020. Recruitment for the originally proposed study occurred in February through early March 2020 and was limited to dyads in Michigan. From this process, 14 mothers expressed interest in the study. Of the 14 mothers, only seven responded to invitations to complete the eligibility screening phone call. Of the seven mothers, four mother-child dyads qualified for the study. Two dyads did not qualify, as one of the children had a diagnosis of ASD and the other child had behavior ratings in the normative range; another mother did not complete the BASC-3 as part of eligibility screening. Due to Covid-19, all in-person research was suspended and changes to the original study were made in order to implement the study via Telehealth. Recruitment resumed in May 2020 and was extended to include dyads in all 50 United States. Between May and August 2020, two additional mothers expressed interest in the study; however, their child’s behaviors were in the 61 clinically significant range, so they did not qualify for the study at the time. Changes were made to the inclusion and exclusion criteria (discussed below), as well as to the study design to enhance recruitment efforts. Recruitment continued until January 2021. Between September and December 2020, four mother-child dyads were recruited and qualified for participation in the study. Recruitment After receiving approval for the study from the Michigan State University Human Research Protection Program (MSU HRPP), four child participants between the ages of 3-6 years old and their mothers were recruited for involvement in this study. Fliers outlining the goal, length, and potential risks and benefits of the study were distributed to potential families. Interested families contacted the study clinician via email to set up a time for a phone screening. During the phone screening, the study clinician asked the mother about her child’s behavioral concerns to determine if they were eligible for the study. Potential eligible families were then sent the BASC-3 PRS-P electronically to fill out in order to determine whether the child met the inclusion criteria. The first four eligible families were enrolled in the study. Parental consent was obtained prior to the first pre-treatment (baseline) session. Parents received information regarding the risks and benefits of participating in the study, as well as any anticipated costs to the family for participation. To meet single case research design standards, a minimum of at least three participants were needed (Kratochwill et al., 2013). Parents received $100 for participation in the entire study ($30 for the completion of baseline sessions, $50 for completion of intervention sessions, and $20 for completion of follow-up sessions). All four dyads completed the full study; no dyad withdrew. Efforts that likely contributed to participant retention included rapport building at the beginning of every session, check-ins with mothers to 62 learn how things were going outside of CDIT and their child’s behavior, and addressing barriers to participation in the study and homework completion. The lead clinician and dyads used computers to video conference using a HIPPA-compliant Zoom link. Due to the Covid-19 pandemic, dyads were able to broadcast home-based interactions with the study clinician and the study clinician was able to provide live coaching with this technology. Prior to treatment, the study clinician reviewed the risks of internet-based services and obtained participants’ consent to I-PCIT. The study clinician discussed the logistics of remote parent training with parents, including technology required for participation, suggestions for rooms to use in the home during sessions, protocols for technology issues (e.g., lost internet connection, dropped phone call, etc.), and how to set up the room for sessions (e.g., number of toys, types of toys to use, etc.). The study clinician and parents also tested out the technology prior to the start of the study. Parents wore headphones that connected to their phone so that the study clinician could coach the parent without the child directly hearing. The lead clinician called the caregiver’s phone number to provide coaching; both the therapist and caregiver muted their volume on the laptop so that the video and audio from the parent-child interactions could still be used for coding purposes after the session ended. The lead clinician recorded the audio of the coaching from her computer. Inclusion and Exclusion Criteria Eligible mothers were English-speaking and over the age of 21 years. Eligible child participants were English-speaking and between the ages of three and six years old (preschool age). All four children exhibited subclinical levels of behavior concerns, as defined by T- scores between 55-74 on the Behavior Assessment System for Children Third Edition Parent Rating Scales for preschoolers (BASC-3 PRS-P; Reynolds & Kamphaus, 2015) Externalizing 63 Problems composite or on one or more of the Externalizing Problems subscales (i.e., hyperactivity, aggression, conduct problems). In practice, the at-risk range on the BASC-3 is indicated by T-scores between 60-69 and the clinically significant range is indicated by T-scores above 70. Since the BASC-3 has a standard deviation of 10, the at-risk (i.e., subclinical) eligibility range was increased by half a standard deviation in both directions for the purposes of this study. Therefore, children who received T-scores of 55-59 and/or T-scores of 70-74 on the Externalizing Problems composite or on one or more of the Externalizing Problems subscales were included in the study. Children who received T-scores above 75 on the Externalizing Problems composite or on one or more of the Externalizing Problems subscales were excluded from the study. Additionally, children who had a diagnosis of a disruptive behavior disorder (e.g., oppositional defiant disorder, conduct disorder) or neurodevelopmental disorder (e.g., Attention Deficit Hyperactivity Disorder, Autism Spectrum Disorder, Global Developmental Delay) were not eligible to participate in this study, as children with such disorders typically display clinically significant behavior concerns and their response to treatment would likely be complicated by comorbid symptomology. There were two reasons for expanding the eligibility criteria from BASC-3 T-scores of 60-69 to 55-74. Due to the COVID-19 pandemic, it was expected that mothers would perceive their child’s behavior to be more problematic (Panda et al., 2021). Given that families had been spending significant time together in confined spaces, it may have been likely that mothers were noticing changes in their children’s behavior. It may also have been the case that children were exhibiting more externalizing behaviors, such as hyperactivity, in response to the pandemic (Panda et al., 2021). With disruptions to routines, daycare and schooling, and adjustments made at home, it would be natural for a child to exhibit externalizing behaviors more than usual. 64 Consequently, it was hypothesized that mothers would rate their child’s behavior as more problematic. Additionally, parent-child dyads would also benefit from the study if T-scores were lower (e.g., T-score below 60), as parent concerns were still being noted. The purpose of the intervention was to increase positive parenting skills and decrease child behavior problems; it would be possible to improve parenting skills and demonstrate a decrease in child behavior problems with T-scores below 60. Secondly, although T-scores above 70 on the BASC-3 are considered to be in the clinically significant range, children who receive scores just above 70 (e.g., 70-74) may not qualify for standard PCIT. Therefore, children who receive T-scores between 70-74 on the BASC-3 would likely not receive appropriate intervention and problem behaviors would continue to increase as the child grew older. For the purposes of this study, it was deemed appropriate to include children who receive slightly higher BASC-3 T-scores (between 70-74), as the goal was prevention of more severe behaviors in the future. Participant Descriptions The first mother-child dyad (Dyad 1) lived in Missouri. The child was a White, English- speaking 4-year old male who was diagnosed with a speech fluency disorder. The child attended an in-home daycare fulltime and was receiving speech therapy at the time of the intervention. His biological mother was a White, English-speaking female who worked fulltime outside the home and held a graduate degree in school psychology. She was married, had one other biological child (son) living in the home, and reported a household gross income between $100,000 - $200,000. The father (spouse) chose not to participate in the intervention. The mother reported she had previously participated in a behavior consultation through an early childhood coalition but had not received direct intervention to address any child behavioral concerns. 65 Parent concerns included difficulty following directions, aggression towards mom, escalation of behaviors when denied access to something the child wanted, and non-compliance. On the pre- intervention BASC-3, overall Externalizing Behaviors (T-score =70) and Aggression (T-score = 70) were reported to be in the clinically significant range, while Hyperactivity (T-score = 66) was reported to be in the at-risk range. The second mother-child dyad (Dyad 2) lived in Michigan. The child was a White, English-speaking 5-year-old male who had no previous diagnoses. The child attended a regular education preschool program full time. His biological mother identified as a White and Hispanic, English-speaking female who worked parttime outside the home and attended some level of college. She was a single mother, had one other biological child (daughter) living in the home, and reported a household gross income between $30,000 - $35,000. The mother did not have a spouse/partner. The mother reported her son had previously received counseling through Community Mental Health (CMH). Throughout the intervention, her son lived one full week with her and one full week with his biological father; thus, although the intervention occurred weekly, the mother had several days where she did not see her son. Additionally, the mother and child contracted COVID-19 and experienced a related family death towards the end of the intervention and could not participate for 3 weeks. This pause occurred between session 11 and 12. Parent concerns included aggression towards an older sister, hyperactivity, and non- compliance, especially during non-preferred activities. On the pre-intervention BASC-3, overall Externalizing Behaviors (T-score = 70) was reported to be in the clinically significant range, while Aggression (T-score = 67) and Hyperactivity (T-score = 69) were reported to be in the at- risk range. 66 The third mother-child dyad (Dyad 3) lived in Illinois. The child was a White, English- speaking 5-year-old female who had no previous diagnoses. The child attended full day kindergarten in a general education classroom. Her biological mother identified as a White, English-speaking female who worked fulltime outside the home and held a graduate degree in education. She was married and reported a household gross income between $100,000 - $200,000. The father chose not to participate in the intervention. The mother reported her daughter had received occupational therapy and developmental therapy between the ages of 1-3 years old. Parent concerns included hyperactivity and arguing with (or “questioning”) the parents when asked to do something. On the pre-intervention BASC-3, overall Externalizing Behaviors (T-score =60) and Hyperactivity (T-score = 65) were reported to be in the at-risk range, while Aggression (T-score = 54) was reported to be in the normative range. The fourth mother-child dyad (Dyad 4) lived in Michigan. The child was a White, English-speaking 3-year old female (who turned four during the intervention) who had no previous diagnosis. She attended full day daycare and transitioned to a more structured daycare setting during the intervention. The child was also receiving speech and occupational therapy. Her biological mother identified as a White, English-speaking female who worked fulltime outside the home and held a graduate degree in marital counseling. She was married with a reported household gross income between $100,000 - $200,000. The father also participated in the intervention as to increase consistent use of positive parenting skills in the home in order to lead to sustained reductions in child behavioral issues. No demographic information was provided for the participating father. Parent concerns included aggression (“some with malice and some with sensory-seeking”), difficulties with social interactions, strong-will, and difficulty redirecting the child. On the pre-intervention BASC-3, overall Externalizing Behaviors (T-score 67 =67), Aggression (T-score = 69), and Hyperactivity (T-score = 63) were reported to be in the at- risk ranges. Constructs and Measures Multiple measures were used to assess the following dependent variables: behavioral risk, observed positive parenting skills, parent-reported child externalizing behaviors, observed child externalizing behaviors, intervention integrity, and parental satisfaction with the intervention. Behavioral Risk During the recruitment phase, the BASC-3 parent rating scale for preschoolers (PRS-P) was given to interested parents to complete to determine whether children met the inclusion criteria by exhibiting subclinical levels of behavior (i.e., T-scores between 55-74 on the Externalizing Problems composite or on one or more of the Externalizing Problems subscales). Additionally, the BASC-3 was given post-intervention. Pre- and posttest BASC-3 T-scores were used as a secondary measure to corroborate individual-level changes in behavioral concerns with the ECBI scores. The BASC-3 is a norm-referenced, standardized measure used to evaluate behavior in children and young adults ages 2 to 25 years (Reynolds & Kamphaus, 2015). Versions of the BASC-3 include teacher rating scales, parent-rating scales, and self-report scales for three age groups: preschool (ages 2-5), child (ages 6-11), and adolescent (ages 12-21). A self-report scale is available for young adults attending postsecondary education (ages 22-25). For the purposes of this study, only the BASC-3 PRS-P was used. The BASC-3 PRS-P measures externalizing problems, internalizing problems, school problems, behavioral symptoms, and adaptive skills. Parents are asked to indicate the frequency of behavior on a four-point scale (i.e., never, sometimes, often, and almost always). The BASC-3 68 results are reported as T-scores, which have a mean of 50 and a standard deviation of 10 (Reynolds & Kamphaus, 2015). For the Clinical Scales (which include the externalizing problems, internalizing problems, and school problems composites), T-scores of 70 or higher are in the clinically significant range and suggest a high level of maladjustment. T-scores between 60 and 69 are in the at-risk range and may identify problems that may not be severe enough to require formal treatment or may indicate the potential for a developing a problem that needs careful monitoring (Reynolds & Kamphaus, 2015). Scores on the Adaptive scales between 31 and 40 are at-risk, while scores of 30 and lower are clinically significant (Reynolds & Kamphaus, 2015). Percentile ranks range from 0% to 100% with percentile ranks from 25% to 75% comprising the average range. The BASC-3 PRS-P has good psychometric properties. For example, it demonstrates high internal consistency across ages and genders (PRS Composite median α = 0.93, range = 0.91-0.96; Clinical and Adaptive scales median α = 0.86, range = .78-.90), adequate to high ranges of reliability (PRS test-retest reliability Composite median = 0.92, range = 0.90-0.93; Clinical and Adaptive scales median = 0.87, range = 0.80-0.93; inter-rater reliability across forms PRS Composite median = 0.78, range = .0.70-0.82; Clinical and Adaptive scales median = 0.72, range = 0.60-0.80; Reynolds & Kamphaus, 2015) and adequate construct validity (Cronbach’s alpha between 0.42-0.90). Positive Parenting Skills The Dyadic Parent-Child Interaction Coding System Fourth Edition (DPICS-IV; Eyberg et al., 2013) was used to measure parenting practices during the baseline and treatment phases. The DPICS is a direct observation of the quality of parent-child social interactions, and measures child compliance and parenting skills across child-led play situations, parent-led play situations, 69 and clean-up situations. In child-led situations, the child plays with whatever toys he/she chooses, while the parent follows her/his lead and plays along with the child. In parent-led situations, the parent chooses the toys to play with and engages the child to play according to the parent’s rules. In clean-up situations, the parent tells the child that it is time to leave and the toys must be put away. In this situation, the child must put the toys away. Categories coded during the DPICS observations include positive parenting “Do Skills” (i.e., labeled praises, behavioral descriptions, reflections) and “Don’t Skills” (i.e., questions, commands, criticism, sarcasm), as well as whether or not the child complies with the command or has no opportunity to comply. For the purposes of this study, DPCIS observations was limited to child-led situations given this is the focus of the CDIT intervention. Weekly DPICS observations of parent-child dyad interactions across child-led situations occurred for five minutes at the onset of all sessions to assess parenting skills. Frequency counts of each “Do Skill” and “Don’t Skill” were tallied live during each DPICS observation. To date, no psychometric work has been published on the DPICS fourth edition. However, there is a large amount of psychometric evidence for the reliability, validity, and treatment sensitivity of earlier versions of the DPICS (Eyberg et al., 2010). The DPICS-IV was published to add new detailed guidelines, refine coding categories, and clarify distinctions between coding categories to increase inter-coder reliability (Eyberg et al., 2013). No coding categories were added or deleted between the DPICS-IV and DPICS-III editions. Despite the lack of research on the DPICS-IV, the codes that capture the primary outcome variable in this study (i.e., positive parenting skills) were psychometrically validated using the DPICS-III and have not been revised in the DPICS-IV. The coding categories that were refined include negative parent codes (e.g., questions, commands, negative talk), which are not primary outcome 70 variables in this study, and child codes (e.g., responses to commands), which are not relevant to CDIT and are therefore not included as outcomes for the current study. Several studies have demonstrated adequate discriminant validity, convergent validity, and treatment sensitivity for earlier versions of the DPICS (see Eyberg et al., 2013 for a review). For example, the DPICS has been shown to discriminate between clinic-referred and non-clinic referred families (e.g., Bjorseth et al., 2015; Foote, 2000) and detects changes in the quality of parent-child interactions following the completion of PCIT (e.g., Chaffin et al., 2004; Nieter et al., 2013). Additionally, the DPICS parent categories were significantly correlated with the ECBI Intensity score (r = 0.35; Bessemer, 1998). Convergent validity was demonstrated by the DPICS by accounting for a significant proportion of variance in reports of child problem behavior (ECB Intensity score: 45% of variance), parental locus of control (Parenting Locus of Control: 27% of variance), and parenting stress (Parenting Stress Index 19% of variance; Bessmer, 1998; Foote, 2000). To date, only one dissertation study aimed to provide updated evidence for convergent validity for the DPICS-IV with parent-reported measures of child functioning, including the ECBI and BASC-2-PRS (Cotter, 2016). No significant associations were reported between parent-reported ratings of child behavior (ECBI) and child externalizing and internalizing symptoms (BASC-2) and DPICS parent codes (Cotter, 2016). However, significant correlations were found between parent-reported ratings of child behavior (ECBI Intensity and Problem scores, r = 0.21 – 0.32), BASC-2 externalizing scores (r = 0.23 -0.29), and DPICS child codes (Cotter, 2016). It should be noted that this finding is not consistent with previous findings (e.g., Bessmer, 1998; Foote, 2000). The author notes that these results may be due to the inclusion of a wide range of diagnoses in the analyses (e.g., children with disruptive behavior disorders and 71 other comorbid diagnoses) compared to prior studies, generational differences in how parents interact with their children given that prior psychometric studies are decades old, and/ or differences in study methodology (e.g., the use of one vs. two DPICS observations at pre- treatment; Cotter, 2016). Findings from Cotter (2016) suggest that child behavior during the DPICS may be more representative of the presence and severity of parent-rated child behavior problems compared to parent behaviors. The use of parent codes still has important clinical implications, as parent codes may be more relevant for treatment purposes in highlighting behaviors that need to be modified (e.g., amount of praise given; the number of reflections or behavior descriptions compared to questions; Cotter, 2016). In regard to reliability, high to adequate inter-observer agreement has been demonstrated for coding of both parent and behaviors (Shanely & Niec, 2011). In a more recent study, kappa reliabilities for DPICS parent categories ranged from 0.80 to 1.00 (Shanley & Niec, 2011). Kappa reliabilities of 0.40 and higher are considered adequate (Gast & Ledford, 2014). For parent codes, test-retest reliability estimates ranged from 0.34 to 0.57 (Brinkmeyer, 2005). Although these values are lower than values typically considered acceptable, they are typical for brief behavioral observations (Eyberg et al., 2013). The DPICS manual recommends establishing a stable baseline of pre-treatment behavior by using more than one set of observations, conducted on different days. Externalizing Behaviors The Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999) was used to progress monitor child externalizing behaviors weekly across baseline and treatment phases for all four children. The ECBI is an established, norm-referenced 36-item parent rating scale that measures disruptive problem behaviors in children between 2 and 16 years old (Abrahamse et al., 72 2015). The ECBI has two different scales to assess disruptive behavior: the Intensity Scale and Problem Scale. The Intensity Scale measures the severity of the child’s behavior, while the Problem Scale is a measure of parental tolerance of the behavior. For every item, parents are asked how often their child exhibits the disruptive behavior on a seven-point scale (ranging from never to always) and whether or not they find this behavior problematic (i.e., yes or no; Eyberg & Pincus, 1999). Scores range from 36 to 252 for the Intensity Scale and 0 to 36 for the Problem Scale. The clinical cutoff scores are 132 for the Intensity Scale and 15 for the Problem Score (Eyberg & Pincus, 1999). Scores between 115-131 on the Intensity Scale are considered to be in the at-risk range (Eyberg & Pincus, 1999). Scores above 15 on the Problem Scale are expected for children exhibiting a high level of behavior concerns. Before every session, parents completed an ECBI to rate their child’s behavior from the previous week. The ECBI takes approximately 10 minutes to complete and can be hand-scored (Eyberg & Pincus, 1999). Several studies have demonstrated that the ECBI is a reliable and valid measure in assessing problem behavior and is also sensitive to behavior change. The ECBI has been used in clinical and research settings for early screening of disruptive child behavior (Abrahamse et al., 2015) and has been shown to be sensitive in measuring the effect of treatment on such behavior problems (Nixon et al., 2003). The ECBI has also detected changes in subclinical behavior problems in prior studies (Berkovits et al., 2010; Brestan et al., 1997). For example, 30 mother- child dyads participated in an abbreviated, preventative version of PCIT in primary care settings. Participating children had ECBI Intensity scores below the clinical cutoff and ranged between 68 and 132. Prior to the intervention, the mean ECBI score across children in the abbreviated PCIT group was 107.71. Substantial improvements in child behavior were reported following the 73 intervention (M = 87.40 post-treatment and M = 87.10 at six-month follow-up; Berkovits et al., 2010). Additionally, the ECBI has good test-retest reliability (r = 0.80 for Intensity scale and r = 0.85 for Problem Scale) over 12 weeks and 0.75 for both scales across a 10-month period (Funderburk et al., 2003) and the two scales show high internal consistency (α = 0.90; Colvin et al., 1999). The ECBI Intensity Scale also has good convergent validity with other questionnaires that assess child behavior problems, including the Child Behavior Checklist (CBCL; r = 0.75; Boggs et al., 1990) and the Strengths and Difficulties Questionnaire (SDQ; r = 0.68; Axberg et al., 2008; Butler, 2011). In order to corroborate the parents’ ECBI ratings, a direct child behavioral observation via partial interval recording of specific operationalized behaviors was used to measure child externalizing behaviors during the DPICS observations in both the baseline and treatment phases. The child behavior observation period occurred during the 5-minute DPICS observation and behavior was measured in 30-second intervals. Since every session with the mother and child was videotaped, and given that the DPICS coding was live, coding of the child externalizing behaviors occurred directly after each session. Child observations were not shared with the mother in order to avoid potential bias in the ECBI ratings. Categories coded during the child observation included non-compliance/refusal (e.g., does not comply with a request or command within five seconds), argumentation, verbal aggression (e.g., yelling, screaming, whining), physical aggression (e.g., hits, kicks, bites, throws or destroys toys), passive dysregulation (e.g., dawdles, careless with toys, slow in completing a task), and active dysregulation (e.g., overactive or restless, short attention span, easily distracted; refer to Appendix P for the coding sheet and coding definitions). Toys that are typically used during CDI 74 (e.g., blocks, Legos, puzzles, bristle blocks) may naturally elicit child externalizing behaviors (e.g., the child may have difficulty stacking the blocks; the child may be careless with the toys). Integrity of Intervention Implementation Integrity of intervention implementation is a multidimensional construct that is characterized across four dimensions: adherence, dosage, quality of intervention delivery, and participant responsiveness (Knoche et al., 2010). In order to assess adherence to the intended delivery of intervention procedures for each session, checklists were adapted for each intervention session based on the treatment integrity checklists from the standard PCIT protocol (Eyberg & Funderburk, 2011; refer to Appendices F-M for session integrity checklists). During each session, the study clinician indicated whether each session component was implemented by marking “Yes,” “No,” or “NA.” The research assistant completed fidelity checks for 30% of all sessions. The percentage of adherence for each session was derived by summing the number of components marked as “Yes” and dividing that number by the total number of required components and multiplied by 100 to yield a percentage (Gast & Ledford, 2014). The total adherence percentage for each individual participant was reported, as well as the range of adherence percentages for each individual session. At the end of the intervention, the total adherence percentages were averaged across all four participants to determine overall treatment adherence for this study. Additionally, inter-observer agreement was calculated across 30% of sessions for each dyad to assess for overall adherence to intervention procedures, aligned with What Works Clearinghouse (WWC) standards for single case design (Kratochwill et al., 2013). Researchers (e.g., the lead clinician and a research assistant) received checklists for the observed session (i.e., Session 2 checklist if observing a family’s second treatment session) to score the number of 75 items completed. Inter-observer agreement was calculated using percentages of agreement between study clinicians. According to Kratochwill (2013), a minimum inter-observer agreement of 80% or higher is adequate when using this approach. Quality of intervention delivery refers to the degree to which the clinician implements triadic and collaborative strategies (e.g., clinician is warm and sensitive towards families, provides ample opportunities for collaboration, initiates meaningful conversations with the parents, does not interrupt the parent while talking, asks open-ended questions, frequently encourages parental participation). Quality of intervention delivery was measured by a global rating of the clinician’s use of triadic and collaborative strategies during each intervention session using a 4-point Likert scale (1= low quality and 4 = high quality). This scale is consistent with measures of the quality of intervention delivery from prior parent training intervention work (e.g., Sheridan et al., 2008). Each week, the lead clinician rated the quality of intervention delivery using the 4-point Likert scale (refer to Appendices F-M to see the question on the integrity checklists). The research assistant also rated the quality of the intervention delivery across 30% of all sessions for each dyad. To evaluate the average quality of overall intervention delivery, an average rating for the clinician was calculated by summing the quality of intervention delivery scores from each week and dividing by the total possible score. Additionally, an average quality of intervention delivery was also calculated per mother-child dyad. The range of quality of intervention scores for each mother-child dyad was reported. Dosage of the intervention was evaluated by examining attendance at treatment. To assess dosage of the intervention, the number of full sessions that a participant attended was summed, then divided by the total number of possible sessions (e.g., 22) and multiplied by 100 to 76 yield a percentage. This resulted in a percentage of dosage of the intervention. The higher the percentage, the greater the dosage of the intervention. Participant responsiveness is the participants’ level of engagement in and receptiveness to intervention programming. Each mother completed weekly homework sheets in order to assess participant engagement in the intervention. Homework was assigned after every treatment session and was collected the following week. Each week, percentage of homework completion was calculated by summing up the number of days the mother completed her homework, divided by the total number of days the mother could have completed her homework, and multiplied by 100 to yield a percentage. An average homework completion score was taken by summing the weekly percentages of homework completion, divided by the total number of weeks, and multiplied by 100 to yield a percentage. A range of homework scores for each individual session was reported. Parental Satisfaction Parental satisfaction with treatment itself and the outcome may contribute towards the maintenance of parenting skills (Brestan et al., 1999). Thus, parental satisfaction ratings of the CDIT intervention was collected for all four cases following completion of the intervention (i.e., during the graduation session). The Therapy Attitude Inventory (TAI; Eyberg, 1974) was used to measure parent satisfaction, including intervention acceptability and feasibility, with treatment. The TAI is a 10-item parent report that has been used in prior PCIT studies to assess parent satisfaction with treatment in areas such as parenting skills learned, the child’s behavior changes, and the type of treatment program used (Brestan et al., 1999). The TAI was designed specifically for use with parent training, parent-child treatments, and behavioral family therapy (Brestan et al., 1999). Items on the TAI are designed to reflect the goals of parent-child treatments (e.g., to 77 promote parent-child relationship skills, to decrease negative parenting skills and child behaviors). For each item, parents rate their level of satisfaction on a five-point scale (score of 1 = dissatisfaction, score of 5 = high satisfaction) by answering questions about the techniques they have learned and how they feel about the treatment. A total TAI score is given by summing the scores on the 10 items (total score can range between 10—50). For the purposes of this study, only 7 items on the TAI were given, as three items ask specifically about disciplinary techniques and child compliance to commands, which are areas that CDIT does not address. Thus, a total score can range between 7—35). Studies that have examined the psychometric properties of the TAI found that it demonstrates high internal consistency (α = 0.91; Brestan et al., 1999; α = 0.88; Eisenstadt et al., 1993), indicating high reliability. The stability coefficient across a 4-month period was also high (r = 0.85; Brestan et al., 1999). External validity of the TAI has been demonstrated by moderate correlations (r = 0.36 to 0.49) between TAI scores and changes during treatment measured by pre- to posttreatment difference scores on the ECBI (Brestan et al., 1999; Reeve, 2015). Additionally, the TAI has been found to be sensitive to the variability in satisfaction among parent training models (e.g., higher scores on the TAI were reported after a 5-week program of individual parent-child treatment compared to scores after a 5-week group parent training; Eyberg & Matarazzo, 1980). Research Design This study used a nonconcurrent multiple-probe across participants single-case design (Gast & Ledford, 2014). It aimed to provide support for the effectiveness, feasibility, and acceptability of an adapted version of PCIT as a suitable treatment for the prevention of severe child externalizing behaviors. Intervention research requires a rigorous methodology to 78 demonstrate a causal relationship between an intervention and expected outcomes, such as randomized controlled trials (RCT). However, researchers in educational and behavioral sciences have begun to use single-case research designs to draw valid conclusions about intervention effectiveness (Kazdin, 2011). In addition, single-case design serves as a foundation for intervention development in the early stages of the research process and examines the presence of a functional relationship between the intervention and targeted outcomes (Kratochwill et al., 2013). Furthermore, single-case research eliminates the practical issue of recruiting a large sample by only requiring a small number of participants to draw reliable and valid conclusions (Kazdin, 2011). In single-case research studies, each participant serves as her or his own control (Gast & Ledford, 2014). Given that the current study adapted traditional PCIT to be a preventative intervention, which is a newer area of research, and that a well-developed RCT would require significant time, resources, and funding, single-case design was a more suitable method that allowed for investigation of intervention effectiveness. The use of a multiple probe design across participants allowed for the examination of the effectiveness, feasibility, and acceptability of a modified version of PCIT, while also aiming to control for threats to internal validity by reducing the number of baseline sessions required and systematically introducing the intervention across participants. Procedures The purpose of the present study was to examine the effectiveness, integrity, and acceptability of a brief, modified version of traditional PCIT for four preschool-aged children exhibiting subclinical levels of behavioral concerns with no prior diagnoses. 79 Study Personnel and Training Study personnel included a lead clinician and a research assistant. The lead clinician was an advanced graduate student who received training in PCIT, observed at least two PCIT cases to completion, and co-led one PCIT case to completion. The lead clinician was responsible for implementing the intervention, including collecting data forms from parents, scoring assessments, conducting observations, and providing in-vivo coaching during treatment sessions. The lead clinician received supervision from a licensed psychologist who was certified in PCIT. The research assistant was also a graduate student. The research assistant was responsible for double-coding 30% of the five-minute DPICS observations, as well as 30% the child behavioral observations in each phase of the study for each participant to ensure coding integrity. The lead clinician trained the research assistant in using the DPICS coding system. Training included a two-hour didactic on the positive parenting skills and in the use of the DPICS coding system to evaluate parenting skills in the CDI phase of treatment. The research assistant was provided with the DPICS coding manual in order to learn about the operational definitions of PRIDE skills and examples of parent verbalizations and behaviors that could be coded. To ensure that the research assistant’s DPICS codes were reliable, the lead clinician and research assistant watched five-minute video clips of parent-child interactions and practiced coding using the DPICS coding system. The research assistant was required to have three demonstrations of at least 80 percent reliability with the lead clinician before he was able to code independently. Inter-rater reliability was monitored throughout the study through double-coding (by the lead clinician and research assistant). Inter-rate agreement was established for 30% of DPICS and behavioral observations for each mother-child dyad. The lead clinician and research assistant compared their codes and discussed any disagreements. 80 All but five sessions total were able to be recorded for the purpose of establishing inter- rater agreement. Five were not recorded due to technology issues. Agreement was established for the DPICS CDI skills by dividing the frequency count of each of the three positive parenting skills (e.g., Labeled Praises, Behavioral Descriptions, Reflections) obtained by the study clinician by the frequency count obtained from the research assistant. This same procedure was repeated for negative parenting behaviors (e.g., Questions, Commands, Criticism). For the direct child behavioral observation, the number of intervals in which raters recorded the same behavior (e.g., argumentation) was divided by the total number of intervals. In regard to positive parenting skills, inter-rater agreement ranged from 85-100% for Labeled Praises, 80-100% for Reflections, and 83-100% for Behavioral Descriptions. In regard to negative parenting behaviors, inter-rater agreement ranged from 90-100%. In regard to direct child behavioral observations, inter-rater agreement ranged from 94-100% for Argumentative, 100% for Verbal Aggression, 100% Physical Aggression, and 90-100% for Active Dysregulation. Study Phases Pilot. After the research assistant was trained in the DPICS observations, a pilot of the adapted PCIT intervention was conducted with one mother-child dyad who met the aforementioned inclusion criteria to evaluate the feasibility, time, and cost of the intervention so that improvements to the intervention structure and content could be made before the full study began. The pilot study followed the structure of the adapted PCIT intervention (as outlined in Appendix E) and the intervention was implemented in-person, prior to the global pandemic. Based on results from the pilot study, changes to the initial study included allowing the second caregiver to receive live coaching. Results of the pilot study showed that the pilot mother’s use of positive parenting skills increased and, initially, child problem behavior did 81 decrease. However, observed child behavioral concerns increased incrementally across the treatment, and the mother reported an increase in child-problematic behavior posttreatment. It is possible that this was due to a wider gap between the mother's and father's parenting skills/techniques, as the mother reported that she and her spouse had different parenting philosophies. Inclusion of the mother’s spouse/partner to participate in the intervention would likely increase the chance that both caregivers learn the positive and negative skills, as well as how to use them at home. This would hopefully increase consistent use of positive parenting skills at home, leading to sustained reductions in child behavioral issues. Thus, second caregivers were invited to participate in the study. The mother was considered the primary caregiver for the study and was the individual in which decisions were based on for the single-case design (for example, if she met and established mastery of positive parenting skills, then the family would graduate from treatment). The spouse/partner data was secondary data that was used to compare findings. It was not be a requirement for the spouse/partner to participate in the study; however, it was strongly recommended that the spouse/partner at least attend the initial CDI Teach session during which they learn about the positive and negative parenting skills. Only one father (from Dyad 4) chose to participate in the CDI Teach session and the intervention. Pre-intervention. Prior to the start of the intervention, mothers received clear guidance on the goals and procedures of the present study and were given expected parent responsibilities. Over the course of the intervention, parent responsibilities included attending and participating in weekly sessions with their child, completing weekly ratings regarding their child’s externalizing behaviors, participating in DPICS observations, and completing daily homework assignments. Participants were assigned baseline conditions based on when they meet eligibility criteria (e.g., the first eligible dyad received 3-week baseline, the second eligible dyad received 82 4-week baseline, etc.). The purpose of assigning families to baseline conditions was for feasibility and ethical reasons. Given that families were reporting behavioral concerns, there was an ethical need to avoid a long delay of treatment; thus, the length of baseline conditions were determined apriori. Additionally, assigning families to different baselines guarded against threats to the research design should attrition have occurred. There was greater likelihood for attrition if families were in the baseline phase longer than the treatment phase. For example, Dyads 2, 3, and 4 would have remained in the baseline phase until Dyad 1 demonstrated an effect. The intervention phase would also not have begun for an individual dyad until a stable baseline was established. The introduction of the intervention after 3, 4, 5, and 6 weeks of baseline helped to reduce the likelihood of attrition compared to 3, 5, 7, and 9 weeks of baseline. Baseline stability for positive parenting skills and child behavior ratings were established. Given that negative parenting skills were a secondary research question, baseline stability was not established for negative parenting skills. Baseline. After the initial screening and pre-intervention procedures, the lead clinician conducted the baseline sessions, as outlined in the adapted PCIT session structure (see Appendix E). During these sessions, mothers first completed the ECBI to indicate their behavior concerns, and then participated in a five-minute DPICS observation with their child. The lead clinician watched the videotape of the DPICS observation and coded for child behaviors following the completion of each session. The number of baseline sessions was determined a priori so that the baseline phase only included three probe sessions for each family. A multiple probe design allowed for the a priori designation of baseline probe sessions, which may have helped reduce the likelihood of attrition. 83 A minimum of three data points is required to meet single-case research design standards with reservations (Kratochwill et al., 2014). It was expected that mothers and their children would exhibit stable patterns of behavior during baseline. For example, given that the child was displaying elevated behavior concerns, it was not expected that this behavior would improve or decrease over the course of three baseline sessions (which ranged from 3-6 weeks depending on the assigned baseline), but would remain the same. CDI Teach Didactic Session. After completing the final baseline DPICS observation during session three, mothers immediately received a didactic (i.e., informative lesson) in PRIDE (i.e., positive parenting) skills called the CDI Teach session. Specifically, the mother received a lesson in the PRIDE skills that included teaching, modeling, and role-playing the new skills. The lead clinician followed the CDI Teach script from the standard PCIT protocol (Eyberg & Funderburk, 2011) for the CDI Teach session. For Dyad 4, both the mother and father attended the CDIT Teach session. The following example demonstrates the CDI Teach process, using an adapted version of the CDI Teach script from the standard PCIT protocol (Eyberg & Funderburk, 2011, p. 20-24). The lead clinician will introduce a new skill: We’re going to talk about the special skills to use during the play sessions. We call these skills the PRIDE skills. P is for praise. Give your child labeled praise for positive behavior. Praise compliments your child on his behavior. There are two kinds of praise: labeled and unlabeled praise. Labeled praise is specific praise such as “I like how nicely you are sitting in your chair” or “You drew a beautiful picture!” Unlabeled praise such as “good” or “that’s great” or “nice job” is nonspecific. Labeled praise is more effective because it lets your child 84 know exactly what you like and it increase the behavior it describes. It also increases the child’s self-esteem and makes you and your child feel good! After the lead clinician finished teaching the remainder of the PRIDE skills, she took turns role-playing the skills with the mother. First, the clinician acted as the parent, while the mother acted as the child and played with the toys. The clinician used the PRIDE skills to help demonstrate how the mother could use them during play. Then, the mother acted as the parent, while the clinician took the role of the child. The mother practiced using the PRIDE skills (with the clinician acting as the child) and the clinician provided feedback. Role-playing between the clinician and the mother (without the child present) is consistent with the standard PCIT protocol. Treatment. Following baseline data collection and the CDI teach didactic session, mother-child dyads received six to eight treatment sessions of the modified PCIT intervention (see Appendix E for an outline of the modified structure). Treatment sessions took place via Telehealth and lasted approximately 60 minutes. Prior to intervention implementation, mothers were told that they would participate in three to eight treatment sessions, dependent on how often they practiced at home and demonstrated an improvement in parenting skills during treatment sessions. Prior to the beginning of every treatment session, the study clinician instructed the parent to put out three different toys that are considered “good toys” to practice PRIDE skills with according to the standard PCIT protocol (Eyberg & Funderburk, 2011). These toys included those such as blocks, Legos, Tinker Toys, paper and colored pencils, dolls, and Play Dough. Treatment sessions were broken into five parts, including 1) brief check-in with the parent; 2) ECBI; 3) DPICS observation, 4) live coaching, and 5) review of DPCIS data and homework. 85 ECBI. At the beginning of each treatment session, the mother filled out an ECBI form electronically and sent it back to the lead clinician via email. For Dyad 4, both the mother and father completed independent ECBIs at the beginning of each treatment session. Building Rapport. Following completion of the ECBI, the lead study clinician completed a 5-10 minute check in with the mother to discuss how things were going outside of parenting. Informal check-ins help build rapport among clinicians and parents (Eyberg & Funderburk, 2011). Review of Session Aims. Next, the lead clinician spent five minutes briefly reviewing the parenting skill(s) that the mother aimed to work on during that session. DPICS Observation. Then the mother and child completed a five-minute DPICS observation. The study clinician observed the mother-child interactions for five minutes and used the DPICS coding system to evaluate parenting skills. The lead clinician did not coach (i.e., provide suggestions to the mother or give feedback on her skills) during these five minutes of observation. The five minutes of observation time was introduced to parents as a time in which they should demonstrate all the skills they have learned. For Dyad 4, the mother and father both completed a five-minute DPCIS observation. Coaching. After the DPICS observation, the mother and child spent the next 20-25 minutes in play. This playtime was referred to as “special time” for the child. During this time, the lead clinician spent the remainder of the treatment session coaching the mother in her use of PRIDE skills. Coaching goals were set for individual sessions (e.g., increase the use of Labeled Praises, decrease the use of Questions, etc.) based on the DPICS observation and the number of each positive parenting skill the mother demonstrated during the observation. For Dyad 4, the mother and the father split the coaching time. In comparison to Mothers 1, 2, and 3, Mother 4 86 received half the amount of coaching time. During all treatment sessions and after the five-minute observations, mothers received in- vivo (i.e., in-person) coaching on their use of PRIDE during the session (referred to as “special time” to the child). Coaching occurred through a bug-in-the-ear device. The mother used headphones and the lead clinician called the mother’s cell phone and coached her through the phone. Common CDI coaching statements include labeled praises (e.g., nice imitating his play; good catching that question; great job setting an example of gentle play), gentle correctives (e.g., we want to reflect only when he’s talking nicely; you can just ignore that; maybe you could say what’s good about it), direct and indirect suggestions (e.g., try to label that; just build the same thing she’s building; can you reflect that?), and observations (e.g., he’s learning to take turns; you play with her so warmly; that sounds very genuine). Review. During the last five minutes of the treatment session, ECBI scores were plotted on a graph to progress monitor child externalizing behaviors. Additionally, the number of parenting skills (i.e., labeled praises, behavioral descriptions, reflections) that the lead clinician observed during the five-minute DPICS observations were plotted on a graph for each mother to progress monitor her parenting skills. These graphs were shared with the mother at the end of each session to show the increase in parenting skills and decrease in child externalizing behaviors. Lastly, the lead clinician assigned mothers homework of completing five minutes of “special time” (e.g., CDI) at home each day. Post-Session. Following the completion of the session, the lead clinician watched the videotape of the DPICS observation and coded for child externalizing behaviors. Child behavior observational data were not shared with the mother in order to avoid potential bias in the ECBI 87 ratings. It should be noted that child behavior observational data was used to corroborate parental ratings of child behavior but was not used to make decisions about treatment progress. Graduation. Participating mothers were required to meet the standard PCIT CDI mastery criteria (i.e., 10 labeled praises, 10 reflections, and 10 descriptions) in order to graduate from treatment (Eyberg & Funderburk, 2011) and move onto the maintenance phase of the study. Upon completion of all didactic and coaching sessions, families participated in a final DPICS observation and graduation session. Prior to the graduation ceremony, mothers completed an ECBI, DPICS observation, BASC-3, and survey about their satisfaction with treatment. It was expected that participating mothers would receive at least three coaching sessions, as it was possible for mothers to meet the mastery criteria after three treatment sessions. In standard PCIT, the average number of CDI sessions required to meet mastery in PRIDE skills is between five to six sessions (Eyberg & Funderburk, 2011; Harwood, O’Brien, & Eyberg, 2008). If mothers did not meet CDI mastery criteria after receiving eight coaching sessions and there was a stable rate of improvement in parenting skills or in child behavior problems, families graduated and moved to the maintenance phase. This occurred in the case of Dyad 2. If a mother did not meet CDI mastery criteria after receiving eight coaching sessions and there was not a stable rate of improvement in parenting skills or in child behavior problems, the mother was provided with a clinical referral to receive treatment for more severe behaviors. This occurred in the case of Dyad 4. Maintenance. After graduation from treatment, all mother-child dyads participated in one follow-up session. The purpose of the follow-up session was to examine whether the mother was continuing to meet CDI mastery criteria after treatment, indicating retention and maintenance of positive parenting skills. During the follow-up session, mothers first completed 88 an ECBI and then a DPICS observation. Follow-up sessions lasted no more than twenty minutes. However, if a mother was well-below meeting the mastery criteria (e.g., at the 50 percent mark and only giving five labeled praises, five descriptions, and five reflections) at the follow-up session, the lead clinician provided a 30-minute coaching session following the five minute observation to help improve the mother’s positive parenting skills. Post-intervention, Mother 4 received a consultation session to discuss child behavior problems, behavior charts, barriers to limit setting with Child 4, and how the parents could use the PRIDE skills with Child 4 while waiting to receive standard PCIT. The refresher coaching session followed the outline of the fourth CDI coaching session outline (refer to Appendix K for the integrity checklist). Data Analysis Two analytic approaches were used to evaluate changes in dependent variables for this study. First, visual analyses were used to assess the magnitude of change in observed parenting skills and in parent-reported child externalizing behaviors. If visual analyses demonstrated noticeable treatment effects across all four participants for a given variable (i.e., parenting skills as measured by the DPICS and externalizing behaviors as measured by ECBI scores), analysis involved effect size calculations. Visual Analysis Single-case researchers have traditionally used visual analysis of the data to determine if there is a functional relationship between an independent variable and an outcome variable and the strength of that relation (Kazdin, 2011; Kratochwill et al., 2013). In order to use visual analyses, the following assumptions must be met: (a) data must demonstrate a predictable and stable baseline trend for each participant, and (b) enough data points are needed to demonstrate change or stable improvements during the treatment phase. What Works Clearinghouse standards 89 for single-case design (Kratochwill et al., 2013) recommend a minimum of five data points per phase, although three data points is sufficient to meet standards with reservations. When these assumptions are met, visual analysis can be used to visually compare data between phases and across participants to determine whether treatment effect occurred. Six features are used for the visual analysis of within- and between-phase (i.e., comparing baseline and treatment phase) data patterns. These features include level, trend, variability, immediacy of effect, overlap of data between phases, and consistency of patterns within similar phases across multiple participants (Kratochwill et al., 2013). Level is the overall average of the outcome measures within a phase, while trend refers to the slope of the line that best fits the change in data. Variability is considered to be the visible range, variance, standard deviation or general scatter from the line that best fits the data. The level, trend, and variability of outcome measures were examined within each phase. Data patterns across each phase were examined through the consideration of the immediacy of the effect, overlap, and consistency of data. The immediacy of the effect refers to the change in level between the last three data points in one phase and the first three data points of the next. In other words, it represents the amount of time between the implementation of the independent variable (i.e., treatment) and visible change in the dependent variable. Kratochwill and colleagues (2013) recommend examining the change between the last three data points in the baseline phase and the first three data points in the treatment phase, as this provides a more convincing argument that the change is due to treatment. Overlap (Kratochwill et al., 2013) is the percentage of data from one phase that overlaps with data from another phase. The smaller the percentage of overlapping data points, the stronger the effect. Lastly, consistency of patterns within similar phases includes examining the data from all phases within the same condition 90 (e.g., comparing data from the treatment phase of each participant) to see if there are clear and stable patterns. A consistent trend makes it more plausible that the independent variable is functionally related to the observed change in the dependent variables. The lead clinician used visual analysis to review two graphs for each parent-child dyad (i.e., 10 graphs total) –one depicting the number of parenting skills from the DPICS and another depicting scores from the ECBI. Effect Size Although single-case design research has historically relied on visual analysis of the data to determine a functional relationship between the independent variable and the observed changes (Kazdin, 2011; Kratochwill et al., 2013), the use of quantitative approaches to confirm a functional relationship and to assess the magnitude of treatment effects is now also recommended (Kratochwill et al., 2013). If there was evidence of a functional relationship (i.e., there were three demonstrations of an experimental effect across participants), then an effect size was calculated. The reporting of effect sizes was intended to complement visual analysis by providing an evidence-based method that other experimental researchers are familiar with. The calculation of effect sizes in single-case design research serves as an additional source of information to help researchers further understand the size of the effect of single-case design studies. Often, studies using single-case design are not included in meta-analyses, as researchers are unsure how to interpret results from single-case design studies, how to combine results from different single-case design studies, or how to compare results from single-case design studies to results from other experimental methods (Shadish et al., 2015). Effect sizes allow researchers to interpret results from single-case designs using the same conventions they apply to other designs, as well as combine results from different single-case design studies (Shadish et al., 2015). The 91 ability to compare effect sizes is important given the need to systematically summarize findings from single-case design studies and compare findings to other studies (Kratochwill et al., 2013, Shadish et al., 2015). There are several effect size indices that can be utilized in single-case designs. Parker, Vannest, and Davis (2011) reviewed and compared nine potential non-parametric effect size indices using non-overlapping data between phases. Non-overlapping data are data points on a plotted graph that do not exceed the extreme (i.e., minimum or maximum) point identified in the other phase (Parker et al., 2011). The nine effect size indices included Extended Celeration Line (ECL), Percentage of Non-overlapping Data (PND), Percentage of All Non-overlapping Data (PAND), robust Pearson’s Phi (Phi), Percentage of Phase data Exceeding the Median (PEM), Robust Improvement Rate Difference (IRD), Non-overlap of All Pairs (NAP), and Tau indices. While these types of effect sizes are useful, not all can be included in evidence-based practice reviews that include between-group designs. Many parametric (e.g., Cohen’s d, Hedge’s g) and non-parametric effect size metrics (e.g., PAND, IRD, Tau-U) have been used in single-case design research; however, there is not wide-spread agreement about which effect size metric(s) are best for the evaluation of single- case design research (Gast & Ledford, 2014). More recently it has been recommended that a parametric effect size index and a non-parametric effect size index be calculated when doing single-case design research (Ross & Begeny, 2014). In this study, both parametric and non- parametric effect sizes were calculated including: PAND (non-parametric), Tau-U (non- parametric), and Hedge’s g (parametric). One non-overlap effect size index that has been used in single-case design for education studies is the percentage of all non-overlapping data (PAND; Parker, et al., 2007). The PAND 92 indicates non-overlap between baseline and intervention data but uses all data from both phases. It can be translated to Pearson’s Phi and Phi2 . For this study, the PAND will be calculated for parent and child data, as PAND does not require interpretation of data assumptions output (equal variance, homogeneity, and serial independence), calculations are less complex than ANOVA or regression analyses, and PAND calculations are recommended for meta-analyses (Parker et al., 2007). In order to compute PAND, the number of overlapping data points were identified for each individual and then divided by the total number of data points for that individual. This number was then subtracted by one and multiplied by 100 to calculate the PAND. Another type of non-overlapping effect size index is Tau-U (Parker et al., 2011). Tau-U combines non-overlap between phases with trend from the intervention phase and is one of two methods capable of controlling for a positive baseline trend (monotonic trend; Parker et al., 2011). Tau-U is less influenced by sample size, does not require data to meet statistical assumptions, provides accurate calculations when there are few data points, and has good statistical power (Parker et al., 2011). Due to these strengths, the Tau-U effect size was also used in this study. A Tau-U effect size statistic closer to zero is desirable, as that indicates that a lower percentage of the data did not overlap (i.e., showed improvement) between baseline and intervention phases (after controlling for baseline trend; Parker et al., 2011). Shadish and colleagues (2015) discussed three types of between-case effect size metrics. The benefit of between-case effect sizes is that they allow researchers to compare results from single-case design studies and combine single-case design results with those from other designs, such as between-group studies. This is desirable for meta-analyses. While the use of each type of effect size index has strengths and limitations, there is no hard and fast rule as to which effect size index to use. 93 One advantageous between-case effect size is Hedge’s g. Hedges and colleagues (2012, 2013) created this type of effect size for use with single-case, multiple baseline designs. Hedge’s g (Hedges et al., 2012, 2013) accounts for the fact that observations within cases cannot be assumed to be independent (i.e., it allows for the dependency), uses power analysis to help estimate the number of cases and observations that are needed to detect an anticipated effect size (at least three cases within a study are needed), and it assumes the normality of residuals. Additionally, Hedge’s g is the least complex between-case effect size to compute and it is often used with single-case design studies that have less than 20 participants. If visual analysis confirmed an observable treatment effect, effect sizes were calculated using Hedge’s g. Hedge’s g measures the effect size for the difference between means. In order to compute Hedge’s g, the mean of the treatment phase is subtracted from the mean of the baseline phase. This number is then divided by the pooled standard deviation. A large effect size is 0.80 or higher, while a moderate effect size is between 0.50 and 0.80, and a small effect is less than 0.20 (Shadish et al., 2015). Analysis by Research Question Research Question 1 Are there changes in parents’ parenting skills following implementation of the CDIT intervention? Research Question 1a: Are improvements in parents’ positive parenting skills (primary outcome variable) evident following implementation of the CDIT intervention? Research Question 1b: Are decreases in parents’ negative parenting skills evident following implementation of the CDIT intervention? The visual analysis and effect size approach described above was used to analyze parenting skills from the DPCIS observations. Frequency counts of each positive parenting skill 94 (i.e., labeled praise, reflections, behavioral descriptions) were gathered in five-minute observation periods at the onset of each treatment session. During the baseline phase, parents were expected to show low frequency counts on these parenting skills (i.e., labeled praise, reflections, behavioral descriptions). After implementation of the preventative PCIT intervention, an increase in the level and slope was predicted, indicating an increase in positive parenting skills. The PAND, Tau-U, and Hedge’s g calculations occurred only if visual analysis indicated a functional relationship between the introduction of treatment and changes in positive parenting skills over time. The number of parenting skills (i.e., labeled praises, behavioral descriptions, reflections) that the lead clinician observed doing the five-minute DPICS observations were plotted on a graph for each mother to progress monitor her parenting skills. The one father-child dyad was treated as a case study and case study analysis—including the use of descriptive and qualitative data—occurred. Research Question 2 Do children’s externalizing behaviors (defined as “any non-preferred behavior” including aggression, opposition, and noncompliance) decrease after implementation of the CDIT intervention? The visual analysis and effect size approach described above was also used to analyze ECBI scores (i.e., externalizing behavior) across baseline and treatment phases. During the baseline phase, children were expected to show ECBI scores in the at-risk range on the Intensity and Problem Behavior scales. Participating children were expected to receive scores between 115-131 on the Intensity Scale and less than 15 on the Problem Behavior Scale. After implementation of the intervention, a decrease in the level (i.e., mean) and slope (i.e., rate of 95 improvement) was predicted, indicating a decrease in problematic externalizing behavior. The calculation of effect sizes (PAND, Tau-U, and Hedge’s g) occurred only if visual analysis indicates a functional relationship between the introduction of treatment and changes in externalizing behavior over time. ECBI scores were plotted on a graph during each session for each individual child to progress monitor child externalizing behaviors. The lead clinician visually examined the rate of improvement (i.e., slope) and decrease in level (i.e., mean) of ECBI scores across baseline and intervention phases. Additionally, pre- and posttest BASC-3 T-scores (from the externalizing problems subscale scores) were used as a secondary measure to corroborate individual-level changes in behavioral concerns with the ECBI pre-post. Research Question 3 Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth? Research Question 3a: Can CDIT be implemented with integrity by a doctoral- level student (the study clinician) via Telehealth as indicated by high adherence to the treatment protocol? Research Question 3b: Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth as indicated by high quality of intervention delivery? Research Question 3c: Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth as indicated by high dosage (i.e., parent attendance) of the intervention? Research Question 3d: Can CDIT be implemented with integrity by a doctoral- level student (the study clinician) via Telehealth as indicated by high participant responsiveness to the intervention? Treatment integrity was analyzed across several dimensions: adherence, dosage, quality of intervention delivery, and participant responsiveness (Knoche et al., 2010). First, adherence to 96 the treatment protocol was assessed by calculating percentages of session components implemented by the study personnel. For a given item on session adherence checklists, a response of 0 indicated no attempt made or an unsuccessful attempt, while a response of 1 (or a checkmark) indicated an attempt made. At the end of each session, the number of completed tasks were summed to generate the number of tasks completed for that session. The sum was divided by the total number of required tasks and multiplied by 100, resulting in a percentage of adherence for each session. Post-treatment, adherence percentages from all sessions were generated for each of the four cases, as well as a combined average percentage across all participants. To assess dosage of the intervention, the number of full sessions that a participant attended were summed, then divided by the total number of possible sessions (e.g., 22) and multiplied by 100. This resulted in a percentage of dosage of the intervention. The higher the percentage, the greater the dosage of the intervention. Quality of intervention delivery was measured by a global rating of the clinician’s use of triadic and collaborative strategies during each intervention session using a 4-point Likert scale (1= low quality and 4 = high quality). To evaluate the average quality of overall intervention delivery, the clinician’s scores from each week were summed, then divided by the total possible score to calculate an average score. Additionally, an average quality of intervention delivery was also calculated per mother-child dyad. Additionally, participant engagement in the intervention was calculated by examining the participant’s homework completion. Homework was assigned after every treatment session and will be collected the following week, at the first session of the week. Each week, a percentage of homework completion was created. This was done by summing up the number of days the 97 participant completed his/her homework, divided by the total number of days the participant could have completed his/her homework, and multiplied by 100. An average homework completion score was taken by summing the weekly percentages of homework completion, divided by the total number of weeks, and multiplied by 100 (refer to Appendix N for an example of the homework sheets). Research Question 4 What are parent ratings of intervention satisfaction, which includes intervention acceptability and feasibility, after receiving the CDIT intervention? At the end of the graduation session, parents completed the Therapy Attitude Inventory (TAI) to assess treatment satisfaction and acceptability. Averages across the seven-item scores across all participants determined the overall level of caregiver satisfaction for the preventative PCIT intervention. An overall score of 28 or higher (highest score = 35) served as adequate treatment acceptability (Bagner et al., 2013; Berkovits et al., 2010). 98 CHAPTER IV. RESULTS Research Question 1 Are there changes in parents’ parenting skills following implementation of the CDIT intervention? Research Question 1a: Are improvements in parents’ positive parenting skills (i.e., Labeled Praises, Behavioral Descriptions, Reflections; primary outcome variable) evident following implementation of the CDIT intervention? Research Question 1b: Are decreases in parents’ negative parenting skills (i.e., Questions, Commands, Negative Talk) evident following implementation of the CDIT intervention? Visual analysis was used to determine whether a treatment effect occurred for the use of positive and negative parenting skills. The level, trend, variability, immediacy of effect, overlap of data between phases, and consistency of patterns within phases across participants were assessed. Visual analysis results for the DPICS positive and negative skills are summarized first (refer to Figures 1 and 2). It should be noted that, by the first intervention point, parents had only received the CDI Teach didactic and no coaching. By the second intervention point, parents had received one coaching session during which they received feedback on their parenting skills. Descriptive statistics are also provided in Tables 2 and 3. Positive Parenting Skills Dyadic Parent Interaction Coding System (DPICS) scores were analyzed via visual analysis and calculation of PAND, Tau-U, and Hedge’s g effect sizes from baseline to the completion of CDIT to assess the magnitude of behavior change. A large effect size was considered 0.80 or higher, while a moderate effect size was between 0.50 and 0.80, and a small effect was less than 0.20 (Shadish et al., 2015). The data obtained from the three statistical analyses for each effect size were examined to determine the presence of a treatment effect. A 99 treatment effect was considered to be present when: 1) visual analysis results indicated at least three demonstrations of a treatment effect and 2) the Tau-U effect size was statistically significant. When one of the two statistical analyses yielded significant results, partial evidence of treatment effect was considered to be demonstrated. No treatment effect was assumed when all statistical analyses yielded nonsignificant results. Positive Parenting Skills Summary. Visual analysis for positive parenting skills using the DPICS did provide partial evidence of a replicated treatment effect for increased positive parenting skills across most dyads. An effect, or a change in level, was considered immediate if there was a change in level when comparing the three data points in baseline to the first three intervention data points. All four mothers increased in their mean levels of Labeled Praises, Reflections, and Behavioral Descriptions from baseline to after intervention (refer to Table 2 for descriptive statistics). Visual analysis of DPICS data patterns suggests at least three demonstrations of a treatment effect (i.e., improvement in parenting skills) for all three positive parenting skills (i.e., Labeled Praises, Behavioral Descriptions, Reflections; refer to Table 10 for a complete summary). PAND and Tau-U analyses are described per individual dyad below. Hedge’s g analyses indicated a large effect for Behavioral Descriptions (BC-SMD = 1.92) and non-significant effects for Labeled Praises (BC-SMD = 0.93) and Reflections (BC-SMD = - 0.41). Dyad 1 Positive Parenting Skills. An immediate effect was observed for Mother 1’s use of Labeled Praises and Behavioral Descriptions. Intervention phase trends for Labeled Praises, Reflections, and Behavioral Descriptions were positive and in the direction of expected behavior change. Mother 1 demonstrated some variability in her use of Behavioral Descriptions. Analyses of data overlap across phases using PAND suggest moderate to large effect sizes (78% non- 100 overlap for Behavioral Descriptions, 100% non-overlap for Reflections, and 56% non-overlap for Labeled Praises; refer to Table 3). Tau-U analyses indicated moderate effect sizes for Mother 1’s use of Labeled Praises (Tau-U = 0.83, p = 0.05), large effect sizes for Behavioral Descriptions (Tau-U = 0.94, p = 0.02), and large effect sizes for Reflections (Tau-U = 1.00, p = 0.02). CDIT was considered effective in increasing Mother 1’s positive parenting skills. Dyad 2 Positive Parenting Skills. No immediate effects were observed for Mother 2’s use of Labeled Praises, Reflections, and Behavioral Descriptions. Intervention phase trends for Labeled Praises, Reflections, and Behavioral Descriptions were positive and in the direction of expected behavior change. Analyses of data overlap across phases using PAND suggest small to moderate effect sizes (45% non-overlap for Behavioral Descriptions, 36% non-overlap for Reflections, and 64% non-overlap for Labeled Praises; refer to Table 3). Tau-U analyses indicated moderate effect sizes for Mother 2’s use of Labeled Praises (Tau-U = 0.88, p = 0.03). Mother did not demonstrate statistically significant increases in her use of Behavioral Descriptions (Tau-U = 0.63, p = 0.13) or Reflections (Tau-U = 0.46, p = 0.26) over time. CDIT was considered partially effective in increasing Mother 2’s positive parenting skills. Dyad 3 Positive Parenting Skills. An immediate effect was observed for Mother 3’s use of Labeled Praises and Behavioral Descriptions. Intervention phase trends for Labeled Praises and Reflections were positive and in the direction of expected behavior change. Intervention phase trends for Behavioral Descriptions were negative and not in the direction of expected behavior change. Mother 3 demonstrated some variability in her use of Behavioral Descriptions. Analyses of data overlap across phases using PAND suggest moderate to large effect sizes (100% non-overlap for Behavioral Descriptions, 78% non-overlap for Reflections, and 100% non-overlap for Labeled Praises; refer to Table 3). Tau-U analyses indicated large effect sizes for 101 Mother 3’s use of Labeled Praises (Tau-U = 1.00, p = 0.02), large effect sizes for Behavioral Descriptions (Tau-U = 1.00, p = 0.02). Mother 3 did not demonstrate statistically significant increases in her use of Reflections (Tau-U = 0.56, p = 0.20). Dyad 4 Mother Positive Parenting Skills. An immediate effect was observed for Mother 4’s use of Labeled Praises. Intervention phase trends for Labeled Praises, Reflections, and Behavioral Descriptions were positive and in the direction of expected behavior change. Mother 4 demonstrated some variability in her use of Labeled Praises. Analyses of data overlap across phases using PAND suggest moderate to large effect sizes (100% non-overlap for Behavioral Descriptions, 63% non-overlap for Reflections, and 100% non-overlap for Labeled Praises; refer to Table 3). Tau-U analyses indicated large effect sizes for Mother 4’s use of Labeled Praises (Tau-U = 1.00, p = 0.01), large effect sizes for Behavioral Descriptions (Tau-U = 0.96, p = 0.01). Mother 4 did not demonstrate statistically significant increases in her use of Reflections (Tau-U = -0.04, p = 0.92). Dyad 4 Father Positive Parenting Skills. Although the father of Dyad 4 participated in the intervention, his data was not included in single-case design analyses. For the purposes of the study, Father 4’s data is reported as case study data to help inform future research and practice with CDIT. An immediate effect was observed for Father 4’s use of Labeled Praises and Behavioral Descriptions. Intervention phase trends for Reflections were positive and in the direction of expected behavior changes. However, a negative trend was observed for Labeled Praises and Behavioral Descriptions, which was not in the direction of expected behavior change. In regard to variability, Father 4 demonstrated some variability in his use of Labeled Praises and Behavioral Descriptions. Analyses of data overlap across phases using PAND 102 suggest moderate to large effect sizes (100% non-overlap Behavioral Descriptions, 100% non- overlap Labeled Praises, and 63% non-overlap Reflections). Figure 1. Positive Parenting Skills 103 Figure 2. Dyad 4 Father Positive Parenting Skills Table 2. Descriptive Statistics for Positive Parenting Skills Baseline Phase Intervention Phase Mean Range Mean Range Dyad 1 BD 0.00 0.00-1.00 10.17 1.00-19.00 RF 1.00 0.00-2.00 13.67 4.00-19.00 LP 1.00 0.00-2.00 12.50 1.00-21.00 Dyad 2 BD 0.00 0.00-1.00 3.75 0.00-8.00 RF 1.67 0.00-3.00 3.75 1.00-7.00 LP 0.00 0.00-0.00 4.13 0.00-8.00 Dyad 3 BD 0.00 0.00-0.00 9.00 5.00-16.00 RF 2.33 1.00-3.00 5.67 2.00-10.00 LP 1.00 0.00-2.00 9.50 6.00-12.00 Dyad 4 BD 0.00 0.00-1.00 3.63 1.00-8.00 RF 4.33 4.00-5.00 4.50 1.00-7.00 LP 0.00 0.00-1.00 5.38 2.00-9.00 Dyad 4^ BD 0.00 * 7.14 3.00-13.00 (Father) RF 3.00 * 5.71 1.00-11.00 LP 0.00 * 9.57 2.00-15.00 Note: BD = Behavioral Descriptions. RF = Reflections. LP = Labeled Praises *Dyad 4 Father only completed one baseline session 104 Table 3. Percentage of All Non-Overlapping Data for Positive Parenting Skills Dyad 1 Dyad 2 Dyad 3 Dyad 4 Dyad 4 (Father) BD 78% 45% 100% 82% 100% RF 100% 36% 78% 64% 63% LP 56% 64% 100% 100% 100% Note: BD = Behavioral Descriptions. RF = Reflections. LP = Labeled Praises Negative Parenting Skills Visual analysis for negative parenting skills using the DPICS (refer to Figure 3) did provide partial evidence of a replicated treatment effect for decreased negative parenting skills across all four dyads. An immediate effect, or a change in level, was observed when comparing the three data points in baseline and to the first three intervention data points. All four mothers decreased in their mean levels of questions and commands. Mothers 1, 3, and 4 did not verbalize Negative Talk prior to the start of the intervention. Visual analysis of DPICS data patterns suggests at least three demonstrations of a treatment effect (i.e., decrease in negative parenting behaviors) for Questions (refer to Table 10 for a complete summary of visual analysis results). PAND and Tau-U analyses are described per individual dyad below. Hedge’s g analyses indicated a large effect for Questions (BC-SMD = -2.86) and Commands (BC-SMD = -0.86) and a non-significant effect for Negative Talk (BC-SMD = -0.36). CDIT was considered effective in decreasing mothers’ use of Questions. Dyad 1 Negative Parenting Skills. For Questions, Mother 1 decreased her mean level from 24.33 at baseline to zero after intervention. For Commands Mother 1 decreased her mean level from 5.33 at baseline to zero after intervention. For Negative Talk, Mother 1 did not verbalize any Negative Talk at baseline or after intervention (refer to Table 4). Intervention phase trends for Negative Talk were neutral for Mother 1; she engaged in low rates of Negative Talk prior to the start of the intervention. A negative trend was observed for Mother 1’s use of 105 Questions and Commands, which was in the direction of the expected behavior change. In regard to variability, Mother 1 demonstrated less variability in her use of Commands and Negative Talk during the intervention phase. Analyses of data overlap across phases using PAND suggest negligible to large effect sizes for Mother 1 (100% non-overlap for Questions, 22% non-overlap for Commands, and no effect for Negative Talk), Tau-U analyses indicate a large effect size for Mother 1’s reduction of Questions over time (Tau-U = -1.00, p = 0.02). Since Mother 1 verbalized low rates of commands and negative talk across baseline and intervention conditions, she did not demonstrate a statistically significant decrease in the use of Commands and Negative Talk. CDIT was considered partially effective in decreasing Mother 1’s use of negative parenting behaviors. Dyad 2 Negative Parenting Skills. For Questions, Mother 2 decreased her mean level from 25.00 at baseline to 14.13 after intervention. For Commands, Mother 2 decreased her mean level from 8.33 at baseline to 4.75 after intervention. Mother 2 decreased her mean level from 3.33 at baseline to zero after intervention (refer to Table 4). An immediate effect was also demonstrated for Negative Talk for Mother 2. Mother 2 demonstrated a downward trend for Negative Talk, Questions, and Commands, which was in the direction of the expected behavior change. In regard to variability, Mother 2 demonstrated less variability in her use of Commands and Negative Talk during the intervention phase. Mothers 2 some variability in her use of Questions. Analyses of data overlap across phases using PAND suggest small to large effect sizes for Mother 2 (100% non-overlap for Questions, 73% non-overlap for Commands, and 18% non- overlap for Negative Talk). A large effect size was demonstrated only for Mother 2 in her reduction of Commands (Tau-U = -0.83, p = 0.04). Tau-U analyses indicated large effect sizes 106 for Mother 2’s reduction of Questions over time (Tau-U = -1.00; p = 0.01). CDIT was considered partially effective in decreasing Mother 2’s negative parenting behaviors. Dyad 3 Negative Parenting Skills. For Questions, Mother 3 decreased her mean level from 26.67 at baseline to 6.50 after intervention. For Commands, Mother 3 decreased her mean level from 3.67 at baseline to 1.17 after intervention. For Negative Talk, Mother 3 did not verbalize any Negative Talk at baseline or after intervention (refer to Table 4). Intervention phase trends for Negative Talk were neutral for Mother 3; she engaged in low rates of Negative Talk prior to the start of the intervention. Additionally, a negative trend was observed for Mother 3’s use of Questions and Commands, which was in the direction of the expected behavior change. In regard to variability, she demonstrated less variability in her use of Commands and Negative Talk during the intervention phase. Mother 3 demonstrated some variability in her use of Questions. Analyses of data overlap across phases using PAND suggest she demonstrated negligible to large effect sizes using PAND (100% Questions 78% Commands, and no effect for Negative Talk; refer to Table 5). Tau-U analyses indicate large effect sizes for Mother 3’s reduction of Questions over time (Tau-U = -1.00, p = 0.02). Since Mother 3 verbalized low rates of commands and negative talk across baseline and intervention conditions, she did not demonstrate a statistically significant decrease in the use of Commands and Negative Talk. CDIT was considered partially effective in decreasing Mother 3’s use of negative parenting behaviors. Dyad 4 Mother Negative Parenting Skills. For Questions, Mother 4 decreased her mean level from 26.00 at baseline to 3.17 after intervention. For Commands Mother 4 decreased her mean level from 2.67 at baseline to zero after intervention. For Negative Talk, Mother 4 did not verbalize any Negative Talk at baseline or after intervention. Intervention phase trends for 107 Negative Talk were neutral for Mother 4; she engaged in low rates of Negative Talk prior to the start of the intervention. Additionally, a negative trend was observed among Mother 4’s use of Questions and Commands, which was in the direction of the expected behavior change. In regard to variability, Mother 4 demonstrated less variability in her use of Commands and Negative Talk during the intervention phase. Analyses of data overlap across phases using PAND suggest negligible to large effect sizes for Mother 4 (100% non-overlap for Question, no effect for Commands, and no effect for Negative Talk). Tau-U analyses indicate large effect size for Mother 4’s reduction of Questions over time (Tau-U = -1.00, p = 0.01). Since Mother 4 verbalized low rates of commands and negative talk across baseline and intervention conditions, she did not demonstrate a statistically significant decrease in the use of Commands and Negative Talk. CDIT was considered partially effective in decreasing Mother 4’s negative parenting behaviors. Dyad 4 Father Negative Parenting Skills. An immediate effect was observed for Father 4’s use of Questions and Commands. Intervention phase trends for Questions and Commands were negative and in the direction of expected behavior changes. The intervention phase trend for Negative Talk was neutral, indicating that Father 4 was not verbalizing Negative Talk prior to the start of the intervention. In regard to variability, Father 4 did not demonstrate variability in his negative parenting behaviors; parenting behaviors consistently followed the trends. Analyses of data overlap across phases using PAND suggest large effect sizes (100% non-overlap Questions, 100% non-overlap Commands, and no effect for Negative Talk). 108 Figure 3. Negative Parenting Skills 109 Figure 4. Dyad 4 Father Negative Parenting Skills Table 4. Descriptive Statistics for Negative Parenting Skills Baseline Phase Intervention Phase Mean Range Mean Range Dyad 1 Questions 24.33 13.00-31.00 0.00 0.00-3.00 Commands 5.33 0.00-14.00 0.00 0.00-1.00 Negative Talk 0.00 0.00-0.00 0.00 0.00-0.00 Dyad 2 Questions 25.00 21.00-27.00 14.13 11.00-20.00 Commands 8.33 7.00-10.00 4.75 2.00-9.00 Negative Talk 3.33 1.00-7.00 0.00 0.00-2.00 Dyad 3 Questions 26.67 18.00-35.00 6.50 1.00-16.00 Commands 3.67 2.00-6.00 1.17 0.00-4.00 Negative Talk 0.00 0.00-1.00 0.00 0.00-1.00 Dyad 4 Questions 26.00 23.00-31.00 3.17 0.00-8.00 Commands 2.67 0.00-5.00 0.00 0.00-0.00 Negative Talk 0.00 0.00-1.00 0.00 0.00-0.00 Dyad 4 Questions 26 * 4.71 0.00-20.00 (Father) Commands 8 * 1.71 1.00-5.00 Negative Talk 0 * 1.00 0.00-7.00 *Dyad 4 Father only completed one baseline session Table 5. Percentage of All Non-Overlapping Data for Negative Parenting Skills Dyad 1 Dyad 2 Dyad 3 Dyad 4 Dyad 4 (Father) Questions 100% 100% 100% 100% 100% Commands 22% 73% 78% 0% 100% Negative Talk 0% 18% 0% 0% 0% Note: BD = Behavioral Descriptions. RF = Reflections. LP = Labeled Praises 110 Research Question 2 Do children’s externalizing behaviors (defined as “any non-preferred behavior” including aggression, opposition, and noncompliance) decrease following implementation of the CDIT intervention? Visual analysis was used to determine whether a treatment effect occurred for ratings of child behavior problems. The level, trend, variability, immediacy of effect, overlap of data between phases, and consistency of patterns within phases across participants were assessed. Only the Intensity Scale data, which is a parental indication of the severity of a child’s behavior, was utilized for visual analysis. The Problem Scale data is also reported as an indication of how parents’ perceptions of behavior issues changed over the course of the study (refer to Table 6). Visual analysis results for the ECBI ratings of child behavior are summarized first (refer to Figure 6). It should be noted that, by the first intervention point, parents had only received the CDI Teach didactic and no coaching. By the second intervention point, parents had received one coaching session during which they received feedback on their parenting skills. Descriptive statistics are also provided in Table 6. Pre-post BASC-3 scores were also examined to corroborate ECBI ratings (refer to Table 9). ECBI Ratings Visual analysis for child behavior problems using the ECBI did provide partial evidence of a replicated treatment effect for decreased child behavior problems across Children 1 and 3. An immediate effect, or a change in level, was observed when comparing the three data points in baseline and to the first three intervention data points. Stable patterns of behavior in parenting skills and child externalizing behavior scores were expected, and did occur, after three DPICs observations and three ECBI ratings with the exception of Child 2. Child 2 did experience a 111 slight decrease in behaviors across the three baseline sessions; however, the change was minimal, particularly across the last two baseline sessions, so the dyad did not remain in baseline. PAND and Tau-U analyses are described per individual dyad below. Hedge’s g analysis indicated a non-significant effect for child behavior changes (BC-SMD = -0.55). Despite only three children demonstrating a treatment effect for child behavior problems, mean levels of ECBI scores for all children decreased from the baseline to the intervention phase (refer to Table 6). A comparison of baseline and intervention phase levels indicate an improvement in ECBI levels over time for Children 1, 2, and 3, with ratings of Child 1’s and Child 3’s behavior problems falling below the sub-clinical range (i.e., in the normative range) upon completion of CDIT. Child 1. An immediate effect was demonstrated during the intervention phase across for Child 1. A negative trend, indicating a decrease in child behavior problems, was demonstrated during the intervention phase, which was in the direction of the expected behavior change. Child 1 also demonstrated consistent parent-reported behavior ratings. Ratings of Child 1’s behavior problems fell below the sub-clinical range (i.e., in the normative range) upon completion of CDIT. Analyses of data overlap across phases using PAND suggest large effect sizes for Child 1 (100% non-overlap; refer to Table 7). Tau-U analyses indicated large effect sizes in child behavior problems for Child 1 (Tau-U = -1.00, p = 0.02). CDIT was considered to be effective in decreasing Child 1’s parent-reported problem behaviors. Child 2. Child 2 demonstrated variability in his parent-reported behavior ratings during the intervention phase. Despite the variability, an overall negative trend in the direction of the expected behavior change during the intervention phase was demonstrated for Child 2. Ratings of Child 2’s behavior problems did fall below the sub-clinical range over the course of the 112 intervention; however, Mother 2 endorsed higher ECBI scores during the last two weeks of the intervention. This is likely due to extraneous factors, as discussed in the Discussion. At follow- up, Mother 2 did rate Child 2’s behavior problems in the normative range. Analyses of data overlap across phases using PAND suggest small effect sizes for Child 2 (37% non-overlap; refer to Table 7). Statistically significant changes in child behavior problems were not observed for Child 2 (Tau-U = -0.38, p = 0.36). CDIT was not considered to be effective in decreasing Child 2’s parent-reported problem behaviors. Child 3. A negative trend, indicating a decrease in child behavior problems, was demonstrated during the intervention phase for Child 3, and in the direction of the expected behavior change. Child 3 also demonstrated consistent parent-reported child behavior ratings. Ratings of Child 3’s behavior problems fell below the sub-clinical range (i.e., in the normative range) upon completion of CDIT. Analyses of data overlap across phases using PAND suggest large effect sizes for and Child 3 (100% non-overlap; refer to Table 7). Tau-U analyses indicated large effect sizes in child behavior problems for Child 3 (Tau-U = -1.00; p = 0.02). CDIT was considered to be effective in decreasing Child 3’s parent-reported problem behaviors. Child 4. An immediate effect was demonstrated during the intervention phase across for Child 4. However, she demonstrated variability in her parent-reported behavior ratings during the intervention phase. While Child 4 experienced a downward trend of child behavior problems over the first half of the intervention phase, child behavior problems increased overall; this is a trend in the opposite direction of the expected behavior change and was likely due to extraneous factors, as discussed in the next chapter (refer to Table 10 for a complete summary for visual analysis results). Child 4’s ratings of behavior problems fell in the at-risk range after completion of CDIT. Analyses of data overlap across phases using PAND suggest moderate effect sizes for 113 Child 4 (55% non-overlap; refer to Table 7). Statistically significant changes in child behavior problems were not observed for Child 4 (Tau-U = -0.58, p = 0.15). CDIT was not considered to be effective in decreasing Child 4’s parent-reported problem behaviors. Dyad 4 Father ECBI Ratings. An immediate effect, as well as a negative trend, was demonstrated during the intervention phase, indicating a decrease in child behavior problems. This was in the direction of expected behavior change. Although some variability was demonstrated in father-reported behavior ratings, the overall trend was still negative. Father 4’s ratings differed from Mother 4’s ratings of child behavior problems. According to Father 4, ratings of Child 4’s behavior problems did fall below the sub-clinical range over the course of the intervention, despite the slight uptick mid-intervention. Analyses of data overlap across phases using PAND suggest large effect sizes for (100% non-overlap). 114 Figure 5. Child Behavior Ratings 115 Figure 6. Dyad 4 Father Child Behavior Ratings Table 6. Descriptive Statistics for ECBI Baseline Phase Intervention Phase Mean Range Mean Range Dyad 1 Intensity 138.00 132.00-146.00 95.17* 67.00- 124.00 Problem 16.67 15.00-18.00 14.50 12.00- 16.00 Dyad 2 Intensity 126.67 108.00-154.00 109.13* 90.00- 125.00 Problem 12.00 9.00-18.00 8.25 5.00- 15.00 Dyad 3 Intensity 106.00 99.00-117.00 79.67* 59.00- 97.00 Problem 3.67 2.00-7.00 1.67 1.00- 2.00 Dyad 4 Intensity 133.67 130.00-141.00 126.25 111.00- 155.00 Problem 17.33 16.00-18.00 14.13 9.00- 19.00 Dyad 4^ Intensity 157.00 * 115 98.00- (Father) 127.00 Problem 19.00 * 9.29 5.00- 15.00 The Intensity Scale measures the severity of the child’s behavior and the Problem Scale measures the parental tolerance of the behavior. ^Dyad 4 Father only completed one baseline session * Indicates ratings at or below normative (“typical”) range of behavior 116 Table 7. Percentage of All Non-overlapping Data for ECBI Dyad 1 Dyad 2 Dyad 3 Dyad 4 Dyad 4 (Father) Intensity 100% 37% 100% 55% 100% Problem 56% 55% 33% 55% 100% Direct Child Behavioral Observation To corroborate parents’ ECBI ratings, a direct child behavioral observation via partial interval recording of specific operationalized behaviors was used (refer to Table 8). These observations were from the same 5-minute DPICS periods for each session. Based on the observations, Child 1 demonstrated argumentation, verbal aggression (e.g., yelling, screaming), and physical aggression (e.g., hitting mother) at the beginning of CDIT implementation (10-40% of intervals). Child 2 only demonstrated one incident (10%) of active dysregulation (e.g., hyperactivity), while Child 3 and Child 4 only demonstrated one incident (10%) of argumentation each. Table 8. Partial Interval Coding Child Behavioral Observation Child 1 Child 2 Child 3 Child 4 Child 4 Mother Father Baseline 1 20% A None None 10% A None Baseline 2 None None None None None Baseline 3 None None None None None CDI 1 40% VA None 10% A None None 10% PA CDI 2 None None None None None CDI 3 None 10% AD None None None CDI 4 None None None None None CDI 5 None None None None None CDI 6 None None None None 10% A CDI7 None None None 10% A None CDI 8 None None None None None Follow-Up None None None None None A = Argumentative, AD = Active Dysregulation, PA = Physical Aggression, VA = Verbal Aggression 117 BASC-3 Ratings Pre-intervention and post-intervention BASC-3 data were examined for all four dyads to corroborate ECBI changes of child behavior problems (refer to Table 9). Mothers 1 and 3 rated their child’s behavior within normative (i.e., average) ranges following the CDIT intervention in overall Externalizing Problems and in hyperactive and aggressive behaviors. Mother 2 rated her child’s behavior within the normative range in overall Externalizing Problems and in aggressive behaviors; however, she rated Child 2’s hyperactive behavior ratings in the at-risk range following CDIT, although there was a decrease from pre-intervention. Dyad 4 continued to rate her child’s behavior within the at-risk range following the CDIT intervention in Externalizing Problems and in hyperactive and aggressive behaviors, although there was a decrease in ratings from pre-intervention. Table 9. BASC-3 Externalizing Composite Pre-Post Scores Pre-Intervention Post-Intervention (T-score) (T-score) Dyad 1 Externalizing Problems 70 57 Hyperactivity 66 55 Aggression 70 57 Dyad 2 Externalizing Problems 70 56 Hyperactivity 69 63 Aggression 67 48 Dyad 3 Externalizing Problems 60 52 Hyperactivity 64 55 Aggression 54 48 Dyad 4* Externalizing Problems 67 66 Hyperactivity 63 61 Aggression 69 67 *Dyad 4 father did not complete the BASC-3 118 Summary of Results An overall summary of the visual analysis and nonparametric and parametric effect size calculations demonstrating dyads’ changes in positive and negative parenting skills and child behavior data can be found in Table 10. Table 10. Visual Analysis and Effect Size Calculations Results Level Immediate Trend Consistency Non- Tau-U Hedge’s Effect of Data Overlap Effect g Patterns Size Positive Parenting Skills LP n=4 n=3 n=4 n=3 56-100% Moderate NS (0.83- (0.93) 0.88; n = 2) Large (1.00; n = 2) BD n=4 n=4 n=3 n=2 45-100% Large Large (0.94- (1.92) 1.00; n = 3) RF n=4 n=2 n=4 n=4 36-100% Large NS (1.00; n = (-0.41) 1) Negative Parenting Skills Questions n = 4 n=4 n=4 n=2 100% Large Large (-1.00; n (-2.86) = 4) Commands n = 1 n=4 n=4 n=4 0-78% Moderate Large (0.83; n = (-0.86) 1) Negative n=1 n=1 n=1 n=4 0-18% NS NS Talk (-0.36) Behavior ECBI n=4 n=2 n=3 n=2 33-100% Large (- NS 1.00; n = (-0.55) 2) Note: LP = Labeled Praises; BD = Behavioral Descriptions; RF = Reflections; ECBI = Eyberg Child Behavior Inventory; NS = not significant 119 Research Question 3 Can CDIT be implemented with integrity by a doctoral-level student (the researcher and study clinician)? Research Question 3a: Can CDIT be implemented with integrity by a doctoral- level student (the study clinician) via Telehealth as indicated by high adherence to the treatment protocol? Research Question 3b: Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth as indicated by high quality of intervention delivery? Research Question 3c: Can CDIT be implemented with integrity by a doctoral-level student (the study clinician) via Telehealth as indicated by high dosage (i.e., parent attendance) of the intervention? Research Question 3d: Can CDIT be implemented with integrity by a doctoral- level student (the study clinician) via Telehealth as indicated by high participant responsiveness to the intervention? Adherence The study clinician’s self-reported ratings of adherence to session components indicated high adherence when CDIT was implemented via telehealth, as reflected in adherence ratings of 93.9% or higher across dyads and an average adherence rating of 97% across all four dyads (Table 11). These ratings are well above 80%, which Kratochwill and colleagues (2013) indicated is an acceptable percentage for treatment adherence. Inter-observer ratings across 30% of all four cases were highly consistent with clinician self-report ratings of adherence to session components (95% agreement). When adherence was less than 100%, it was due to not sharing the ECBI and DPICS graphs with the parent at the end of the session and/or not being able to provide a full 25-30 minutes of coaching time to the parent. Often times, not being able to share the graphs with the parent was due to a child requiring the parent’s complete attention (e.g., due to becoming emotionally dysregulated, interrupting frequently, and/or needing to complete a 120 bedtime routine), thereby ending the session. In regard to being unable to provide 25-30 minutes of coaching, this was due to the parent’s schedule (e.g., running late to the session, needing to leave early from a session, getting interrupted during a session), rather than due to the clinician not following protocol or being unable to complete a full session. These barriers to intervention implementation are discussed in the next section (Dosage) and chapter (Discussion). Table 11. Adherence to Treatment Protocol for CDIT CDIT Session Dyad 1 Dyad 2 Dyad 3 Dyad 4 Average 1 88% 88% 100% 100% 97% 2 100% 80% 90% 90% 3 100% 89% 100% 100% 4 88% 100% 100% 100% 5 100% 100% 100% 100% 6 100% 100% 100% 100% 7 N/A 100% N/A 95% 8 N/A 100% N/A 100% Average 96% 95% 98% 98% N/A = data not available, as the Dyad completed the intervention in 6 weeks Dosage All four dyads attended 100% of the CDIT sessions, indicating high dosage of the intervention. Session length varied between 45-60 minutes, depending on the dyad and the situation. For example, although 60 minutes was allotted for each session for every dyad, sometimes dyads were late to sessions and/or indicated a need to leave at the end of the allotted time frame, thereby decreasing the time spent in intervention. For Dyad 4, the 60-minute session was split between the mother and the father (i.e., each receiving 30 minutes of the session time); thus, they received less coaching time compared to Dyads 1, 2, and 3. Quality of Intervention Delivery The study clinician’s self-reported ratings of intervention quality indicated high quality implementation of CDIT, as reflected in intervention quality ratings of 3 or higher across 121 participants and an average quality rating of 3.97 (range of 1-4) across all four dyads (Table 12). Inter-observer ratings across 30% of all four cases were highly consistent with clinician self- reported ratings of intervention quality (96% agreement). Table 12. Quality of Intervention Delivery CDIT Session Dyad 1 Dyad 2 Dyad 3 Dyad 4 Overall Average (n = 4) 1 4 4 4 4 3.97 2 4 4 4 4 3 4 4 4 4 4 4 4 4 4 5 4 3 4 4 6 4 4 4 4 7 N/A 4 N/A 4 8 N/A 4 N/A 4 Dyad Average 4 3.88 4 4 N/A = data not available, as the Dyad completed the intervention in 6 weeks Participant Responsiveness Participant responsiveness, or engagement in the intervention, was calculated by examining participants’ homework completion. Average homework completion percentages ranged from 57% to 87%. Adequate homework completion is considered 70% or higher (McNeil & Hembree-Kigin, 2011). Mother 1 and Mother 4 achieved an adequate average homework completion percentage (87% and 80%, respectively). Mother 2 completed homework, on average, 63% of the time, while Mother 3 completed homework, on average, 57% of the time (Table 13). 122 Table 13. Participant Responsiveness: Homework Completion Week Week Week Week Week Week Week Week Weekly 1 2 3 4 5 6 7 8 Average Dyad 1 100% 71% 78% 86% 100% 86% N/A N/A 87% Dyad 2 57% 60% 20% 100% 100% 50% 67% 50% 63% Dyad 3 50% 83% 57% 25% 44% 83% N/A N/A 57% Dyad 4 55% 83% 100% 88% 71% 71% 75% 100% 80% Father 55% 83% 100% 88% 71% 71% 75% 100% 80% 4^ Overall Average 72% (n = 4) N/A = data not available, as the Dyad completed the intervention in 6 weeks Research Question 4 What are parent ratings of intervention satisfaction, which includes intervention acceptability and feasibility, after receiving the CDIT intervention? All four mothers indicated high satisfaction with CDIT and perceived the intervention to be acceptable and feasible overall (Average TAI score = 32). An overall TAI score of 28 or higher (highest possible score = 35) indicates adequate treatment acceptability (Bagner et al., 2013; Berkovits et al., 2010). Scores provided by each mother can be found in Table 14 and are described in more detail. Table 14. Parent-Rated Intervention Satisfaction, Acceptability, and Feasibility for CDIT Mother TAI Overall Score Dyad 1 33* Dyad 2 32* Dyad 3 30* Dyad 4 29* Average (n = 4) 31* *Indicates ratings at or above the cut-off for adequate acceptability (i.e., 28). Mother 1 had a TAI score of 33, indicating high intervention satisfaction. She rated all items with a 4 or 5. More specifically, she reported that she had learned several useful techniques in teaching her child new skills (rating = 4), felt she gets along with her child much better than 123 before (rating = 5), felt the major problems that her child presented with at home before the intervention started were greatly improved (rating = 5), she was very satisfied with the progress her child has made with his behavior (rating = 5), believed the intervention helped somewhat with other general personal or family problems not directly related to her child (rating = 4), felt that the intervention was very good in helping her improve the behaviors of her child (rating = 5), and she very much liked the intervention (rating = 5). Qualitatively, Mother 1 wrote “I am so grateful that I had this opportunity! My relationship with my child has improved so much and our home is much calmer than it was before. The aggression, tantrums, and hateful speech my child used regularly before PCIT is now a rarity. I consider myself so lucky to have had this intervention!” She also noted that “the intervention would have been difficult to access” at a clinical site and was glad it occurred via Telehealth. In regard to recommendations and/or future improvements, Mother 1 wrote that she “wished more parents could access this support for their families and children! I think in our community PCIT is pretty much provided to children with clinical diagnoses. However, parents and kids without disabilities struggle with behavior and relationships too. I hope that more PCIT providers take this approach like [the study clinician] did and provide the intervention to families who have children with at-risk behavior.” Mother 2 had a TAI score of 32, indicating high intervention satisfaction. She rated all items with a 4 or 5. More specifically, she reported that she learned very many useful techniques in teaching her child new skills (rating = 5), felt she gets along with her child much better than before (rating = 5), felt the major problems her child presented at home before the intervention started were somewhat improved (rating = 4), she was very satisfied with the progress her child has made with his behavior (rating = 5), believed the intervention helped somewhat with other 124 general personal or family problems not directly related to her child (rating = 4), felt that the intervention was good in helping her improve the behaviors of her child (rating = 4), and she very much liked the intervention (rating = 5). Mother 2 did not provide additional comments about the intervention or recommendations for the future. Mother 3 had a TAI score of 30, indicating adequate intervention satisfaction. She rated all items with a 4 or 5. More specifically, she reported that she has learned several useful techniques in teaching her child new skills (rating = 4), felt she gets along with her somewhat better than before (rating = 4), felt the major problems her child presented with at home before the intervention started were somewhat improved (rating = 4), she was very satisfied with the progress her child has made with her behavior (rating = 5), believed the intervention helped somewhat with other general personal or family problems not directly related to her child (rating = 4), felt that the intervention was very good in helping her improve the behaviors of her child (rating = 5), and she liked the intervention (rating = 4). Qualitatively, Mother 3 wrote “I feel like the strategies were great and having a set time to practice them was helpful and not overwhelming since it was meant to be short. In a crazy, busy world, it was nice to have something structuring me to take time out each day to play with intent and to build my relationship with my child.” Mother 4 had a TAI score of 29, indicating adequate intervention satisfaction. She rated all items with a 4 or 5, except for one item regarding her child’s behavior improvement. More specifically, she reported that she has learned several useful techniques for teaching her child new skills (rating = 4), felt she gets along with her somewhat better than before (rating = 4), felt the major problems her child presented with at home before the intervention were the same (rating = 3), she was somewhat satisfied with the progress her child has made with her behavior 125 (rating = 4), believed the intervention helped very much with other general personal or family problems not directly related to her child (rating = 5), felt that the intervention was good in helping her improve the behaviors of her child (rating = 4), and she liked the intervention very much (rating = 5). Qualitatively Mother 4 wrote, “I really appreciated all of the effort and time you gave to our family! I felt a better bond with [my daughter] during these sessions and I treasured that time. I know [my husband] felt the same, as he mentioned that often.” 126 CHAPTER V. DISCUSSION Purpose of Study Given that early identification of and intervention for disruptive behavior problems are associated with better mental health outcomes (Hess et al., 2012), there is a need to consider a preventative approach to service delivery. Due to the significant role that parents play in the healthy development of children, research on parent training interventions for the treatment of disruptive behaviors, and strong empirical support for prevention efforts, it is critical to develop and test feasible parent training prevention programs to reduce risk for long-term behavioral difficulties. One way to develop a preventive approach to service delivery and improve outcomes for at-risk children is to adapt an evidence-based parent training intervention designed for children with significant behavioral concerns. Parent-Child Interaction Therapy (PCIT) is one such intervention that has particularly strong empirical support in the treatment of child behavior problems. Furthermore, in some standard PCIT studies, Child Directed Interaction training (CDIT) alone has been shown to reduce behavior problems to below clinical levels for almost half of children who presented with a clinically significant behavior disorder prior to treatment (Bagner & Eyberg, 2007; Harwood, O’Brien, & Eyberg, 2009). Presently, there is limited research regarding preventive interventions for preschool-age children with emerging behavior problems (Berkovits et al., 2010). The purpose of this study was to contribute to the literature by investigating the efficacy of CDIT delivered via Telehealth as a brief, preventative, and accessible intervention for children with subclinical behavior concerns to help inform future research and service delivery. Using a multiple probe across participants single-case design with four mother-child dyads, this study examined changes in positive and negative parenting skills 127 and child behavior problems following CDIT implementation. Additionally, this study assessed treatment integrity across multiple dimensions and parental satisfaction. Parenting Skills Positive Parenting Skills As anticipated, mothers’ use of overall positive parenting skills, which included verbalizations of labeled praises, reflections, and behavioral descriptions, increased following implementation of CDIT. Although there was variability in the evidence of effects among individual participants across elements of visual analysis, all four mothers evidenced improvement in at least one parenting skill, while most mothers demonstrated improvement in two or more skills. CDIT was considered to be efficacious in increasing mothers’ use of specific labeled praises for Mothers 1, 2, 3, and 4 and for behavioral descriptions for Mothers 1, 2, and 4. CDIT was considered partially efficacious in increasing mothers’ use of reflections, as there were three demonstrations of an effect using visual analysis, but only significant results for Mother 1 using Tau-U analysis. These results are consistent with other studies that examined outcomes of CDIT (e.g., Bagner & Eyberg, 2007; Ginn et al., 2017; N’zi et al, 2016) and standard PCIT (e.g., Thomas & Zimmer-Gembeck, 2007) and found increases in positive parenting skills. To date, no studies have examined outcomes for each parenting skill individually. Although there is variability in the present study’s findings regarding positive parenting skills, this study adds to the literature by providing specific analyses for each individual parenting behavior. Currently, PCIT and CDIT studies provide effect sizes across all positive parenting behaviors (i.e., providing an overall effect size for Labeled Praises, Reflections, and Behavioral Descriptions). This study allowed for a more detailed examination of treatment and 128 non-treatment responses. Mothers demonstrated the greatest improvement in their use of Labeled Praises across baseline and intervention phases. Labeled praises are specific verbalizations that reinforce the child’s behavior and/or compliment his/her product (e.g., drawing, block tower, etc.) and let the child know exactly what the parent likes. Compared to Reflections and Behavioral Descriptions, it may have been more natural for mothers to “find the good” in their children and praise for appropriate behaviors. It may also have been easier for mothers to learn how to give Labeled Praises, compared to other parenting skills. For example, Labeled Praises can include specific compliments for good behavior, as well as compliments for children’s products (e.g., artwork, towers, etc.). The use of compliments is likely more familiar to mothers than reflecting (imitating) the child’s statements without asking questions (i.e., using Reflections) or describing what the child is physically doing (i.e., using Behavioral Descriptions). When examining changes in the use of Labeled Praises between all four mothers individually, Tau-U effect sizes revealed moderate to large effect sizes for Labeled Praises (0.83- 1.00). However, when examining changes among all four mothers together, Hedge’s g analysis revealed a non-significant effect for Labeled Praises. Differences in effect sizes may demonstrate the variability in mothers’ use of specific parenting skills, indicating larger effects for certain parents (e.g., Mothers 1 and 3) compared to others (e.g., Mothers 2 and 4). In the case of labeled praises, Mothers 1 and 3 demonstrated larger changes in their mean levels, while Mothers 2 and 4 demonstrated smaller, albeit marked, improvements in their mean levels. Differences in effect sizes may also demonstrate the limitations of using non-parametric versus parametric effect sizes in single-case research. Presently, there are several effect size indices that can be utilized in single-case design research; however, there is not wide-spread agreement about which effect size 129 metric(s) are the best for the evaluation of single-case design studies (Gast & Ledford, 2014). While non-parametric effect size metrics can allow for more detailed analyses within cases, parametric effect sizes allow for analysis between cases and lead to more generalizability of results. All four mothers demonstrated improvements in their mean levels of Behavioral Descriptions, although mean levels of change across baseline and intervention phases were smaller in comparison to mothers’ use of Labeled Praises. When examining changes in the use of behavioral descriptions across all four mothers, individually, Tau-U analyses revealed large effect sizes for Mothers 1, 3, and 4 (0.94 -1.00). Mother 2 did not demonstrate a statistically significant increase in the use of Behavioral Descriptions over time but showed improvement over time in her usage. When examining changes among all four mothers together, Hedge’s g analyses indicated a large effect for Behavioral Descriptions (1.92). These results may have been due to mothers’ lack of Behavioral Descriptions during baseline (i.e., no mothers used them pre- intervention) and the mean level of increase during intervention. Although all four mothers demonstrated improvements in their use of Reflections when considering trend, variability, changes in mean levels, and lack of overlap in baseline and intervention data, it was the most difficult skill for mothers to master. In CDIT, mothers are encouraged to replace asking questions (which is considered a negative parenting behavior in PCIT) with reflective statements. Questions ask for an answer from the child (i.e., taking the lead away from the child). They can also ask for information, be hidden commands, take over the lead of the conversation rather than follow it, can sometimes suggest disapproval, and can suggest that the parent is not really listening to the child (Eyberg & Funderburk, 2011). However, it is more natural for mothers to ask their children questions to learn more about their day, the things 130 the child is creating or building, or to learn more about the child in general. Thus, it may take longer for mothers to learn to use Reflections, as opposed to asking questions. Smaller levels of mean changes from baseline to intervention were apparent across all mothers for their use of Reflections. When examining changes in the use of Reflections between all four mothers individually, only Mother 1 demonstrated a large effect (1.00). Mothers 2, 3, and 4 did not demonstrate significant increases in their use of reflections. When examining changes among all four mothers together, Hedge’s g analysis also revealed a non-significant effect for Reflections. A few reasons have been hypothesized as to why significant changes in Reflections did not occur for Mothers 2, 3, and 4. First, mothers’ use of positive parenting skills during CDIT sessions was measured in five-minute increments at the beginning of each session. Each observation was only a “snapshot in time” and mothers’ practice of positive parenting skills at home may not have been completely captured in the observations. Additionally, if a child did not talk much during the DPICS, there was less of an opportunity for the mother to reflect the child’s statements. This may have explained why statistically significant changes were not seen for Mother 3, as Child 3 was quiet and was often observed to be focused on the task she was engaged in without spontaneously talking aloud. It may have been the case that during Special Time (i.e., at-home practice of the parenting skills that served as intervention “homework”), Child 3 was more talkative with her mother, providing Mother 3 more opportunities to use reflections; this situation would not be captured during the 5-minute DPICS. Mother 1 may have demonstrated the greatest level of change due to her practice of positive parenting skills at home, as she demonstrated the highest percentage of homework completion (discussed later). Her improvement may have also been due to external factors, such as her inherent ability to quickly learn and implement the skills taught. 131 Mother 1 was trained as a school psychologist, so it may be the case that she has some training or familiarity with the positive parenting skills. Different coaching goals among mothers may also explain variability in specific positive parenting skills. For example, instead of asking Child 3 questions (which is considered a negative parenting skill), Mother 3 focused on using labeled praises and behavioral descriptions. For Mother 2, CDIT coaching goals frequently focused on decreasing her use of questions. While coaching goals also included replacing questions with reflecting statements, it was easier for Mother 2 to focus on one goal for each session. Additionally, although Mother 2 was able to use fewer questions, replacing asking questions with reflections may have felt unnatural for her, which led to less frequent verbalizations. In regard to Mother 4’s use of reflections, a significant change in her use of them may not have been observed, as Mother 4 was using reflections with her child prior to CDIT implementation and her overall mean level of Reflections was not much different after intervention than at baseline. It should be noted again that parents had only received the CDI Teach didactic, but had not been coached in their parenting skills, by the first intervention DPICS (i.e., at the first intervention session). By the second intervention point, parents had received one coaching session during which they received feedback on their parenting skills. In regard to trends, Mother 1 demonstrated an immediate increase in her use of Labeled Praises after receiving coaching. No trends were immediately shown for Mother 1’s negative parenting behaviors after receiving coaching. Mothers 3 and 4 demonstrated an immediate increase in their use of Labeled Praises and Reflections and an immediate decrease in their use of questions after receiving coaching. No trends were immediately shown for Mother 2’s positive parenting skills, although she did demonstrate an immediate decrease in her use of Commands after receiving coaching. 132 Summary of Positive Parenting Skill Changes The inability to meet mastery in positive parenting skills for Mothers 2 and 4 could be attributed to the time necessary to develop mastery of positive parenting skills. Due to situational circumstances, Mother 2 had less opportunity to practice using the parenting skills at home compared to the other three dyads. Dyad 2 was together every other week, meaning that Mother 2 had one full week without the ability to practice the skills with her child (i.e., implement Special Time). Additionally, due to multiple illnesses in the family and the necessity of taking care of herself and others, Mother 2 also had less time to practice with her child while he was in the home. The stress of these environmental factors may also have contributed to slower gains in positive parenting skills (Chen & Fortensen, 2015). Dyad 4 also experienced environmental stressors that may explain why mastery in positive parenting skills was not achieved. Additionally, dosage of the intervention (i.e., 60 minutes) was split evenly between Mother and Father 4. Mother 4 reported that Child 4 was transitioning to a different preschool, which was causing stress, anxiety, and outbursts of externalizing behaviors. Although Mother 4 reported doing Special Time almost every day, she also noted that she and Father 4 were often tired due to their own stress in managing the behavior, which may have contributed to lower quality parent-child interactions and a more difficult time implementing (i.e., practicing) positive parenting skills. Mother 4 and Father 4 also reported differences in parenting philosophies and disciplinary techniques. Consequently, it may be that Mother 4’s beliefs regarding positive parenting practices and their ability to improve child outcomes may have led to lower scores. Mother 4 noted difficulty with acquiring positive parenting skills, despite having reported familiarity with the concepts due to her own background in counseling. It may be the case that Mother 4’s parenting values did not align with CDIT, as 133 she noted her desire to attend to her child and her child’s feelings and shared that it was difficult to implement planned ignoring, as she felt “mean.” A consultation session was held with Mother 4 to address barriers to parenting skill acquisition and discuss appropriate behavior charts, consequences, and rewards at home. While both parents demonstrated improvement in their positive parenting skills after introduction of the intervention, Father 4 utilized more positive parenting skills, overall, across sessions during CDIT compared to Mother 4. He utilized a greater number of skills at the beginning of treatment and gradually declined in the frequency over the course of the intervention. However, Father 4 was closer to meeting mastery criteria compared to Mother 4. Despite slower improvement in her overall use of parenting skills, Mother 4 had steady improvement in her use of them. Despite the lack of mastery for positive parenting skills for Mothers 2 and 4 and insignificant changes in Tau-U effect sizes for Mothers’ 2, 3, and 4 use of reflections, mean levels of all positive parenting skills increased over the course of the intervention for all four dyads. Thus, there is some support that CDIT was beneficial in improving mothers’ overall use of positive parenting skills. Other studies examining the use of CDIT have found similar results in that improvements in positive parenting skills and child behavior problems were demonstrated despite parents’ lack of meeting mastery criteria. Ginn and colleagues (2017) reported only 50% of parents met CDIT skill mastery, yet higher levels of positive parenting skills and lower levels of child behavior problems were seen at posttreatment. Maintenance of these positive changes were also seen at 6-week follow-up. N’zi and colleagues (2016) also reported only 71% of parents met mastery for CDIT skills, but still saw positive changes in parent and child behaviors after 8 weeks of treatment and at 6-month follow-up. Findings from these studies and the present study suggest that meeting mastery criteria for positive parenting skills may not be needed in 134 order to effect behavioral change in children. Findings from Thomas and Zimmer-Gembeck (2017) support this hypothesis. They found that time-limited, non-mastery of standard PCIT had better outcomes on child externalizing behaviors as opposed to time-unlimited mastery of skills. Thus, dosage of a brief intervention such as CDIT may be enough to prevent later behavioral problems from developing in children. Compared to other parenting programs such as PMTO, Incredible Years, and Triple P, CDIT shows promise in improving parent outcomes. For example, PMTO studies only cite small to medium effect sizes in improving positive parenting practices (Dishion et al., 2016). Meta- analyses of Incredible Years also indicate small mean effect sizes for improvements in parenting behaviors (Menting et al., 2013), while Triple P demonstrated medium effect sizes (Sanders et al., 2014). These findings provide more support for the use of a brief intervention such as CDIT, as its outcomes are comparable to more costly and time-consuming parent training programs. Negative Parenting Skills Also as expected, mothers’ use of negative parenting skills decreased following implementation of CDIT. These results are consistent with other studies that examined outcomes of CDIT (e.g., Bagner & Eyberg, 2007; Ginn et al., 2017; N’zi et al, 2016). It should be noted that negative parenting behaviors, overall, were low at baseline; thus the changes in negative parenting behaviors may not be fully attributed to the intervention. The biggest change was in mothers’ use of questions, while smaller rates of improvement were seen in mothers’ use of commands and negative talk. Mothers 1, 3, and 4 met mastery for negative parenting skills (i.e., demonstrating less than 3 total verbalizations for ALL negative parenting skills during the DPICS). Although Mother 2 did not meet mastery for negative parenting behaviors, she did demonstrate an overall decrease in her use of negative parenting behaviors. Analysis of DPICS 135 negative parenting skills revealed a decrease in mean verbalizations of questions and commands for all dyads and in mean verbalizations of negative talk for Mother 2 from baseline to intervention. Visual analysis yielded four demonstrations of an effect for questions, but only one demonstration of an effect for commands and negative talk. In regard to Father 4, he also met mastery for negative parenting behaviors and decreased his use of Questions and Commands following implementation of CDIT. Nonparametric effect sizes (Tau-U) were calculated for all skills among all four dyads. Large effect sizes were found for mothers’ decrease in questions (-1.00) across all four dyads. A medium effect was demonstrated for Mother 2’s reduction in commands (-0.83). No significant effects were observed for mothers’ use of negative talk. This was likely due to mothers’ low rates of negative talk with their children prior to CDIT implementation. For example, Mother 1, 3, and 4 were not observed to use any negative talk with their children during baseline. While Mother 2 was observed to verbalize some negative talk during baseline, the mean number of verbalizations was low (i.e., below 4). Throughout intervention, Mothers 1, 3, and 4 did use negative talk and Mother 2 reduced her verbalizations of negative talk so her mean level was zero. Using parametric effect sizes (Hedge’s g) to calculate differences across all four cases, there was a statistically significant, large effect for changes in Questions and Commands. Although implementation of CDIT was not the reason for mothers’ lack of negative talk, it is positive that mothers were not using negative talk initially and that they were able to maintain this skill throughout intervention. Low levels of negative talk may be indicative of a more positive parent-child relationship between participating dyads and, consequently, have resulted in less significant behavior problems. High levels of maternal negativity towards children has been correlated with externalizing behavior problems in children (Cole et al., 2003). 136 Similarly, all four mothers had low verbalizations of commands prior to CDIT implementation and reduced their verbalizations of commands during intervention so their mean levels were zero. This explains the lack of significant change across CDIT implementation. Mother 2 demonstrated the highest level of commands during baseline but reduced her use across intervention and demonstrated a significant change. Another explanation for this study’s findings regarding negative parenting practices include the level of child behavior concerns that were reported pre-intervention. Given that all four children demonstrated subclinical levels of behavior prior to CDIT implementation, it may be that mothers did not need to give commands as frequently and were not accustomed to using negative talk, as the children were not exhibiting more severe levels of behavior and fewer concerns. Social Learning Theory (Bandura, 1973) suggests that cyclical patterns of negativity between the parent and the child develop over time from repeated, parent-child interactions. For example, the cycle would start by a child misbehaving, followed by a parent scolding the child, which results in an increase in the child’s negative behavior. The reinforcement of the child’s misbehavior (i.e., by giving attention to a negative behavior) creates a cycle where parent-child interactions become more difficult to manage, ending to further escalation of problematic behaviors over time. Therefore, the cycle of negativity between the parent and the child may be less pervasive when behavior problems are emerging, as opposed to when they are clinically significant. Additionally, given that there were few child behavioral concerns exhibited during treatment sessions (discussed later), there were less opportunities for mothers to use more corrective techniques that may have led to verbalizations of negative talk and/or commands. For all four dyads, there was a neutral or decreasing trend in the use of overall negative parenting skills, indicating an improvement. After implementation of CDIT, Mother 2 did continue to 137 demonstrate negative parenting behaviors and did not meet the mastery criteria; however, she showed marked improvement and reduced her use of them. In sum, CDIT was considered to be efficacious in decreasing mothers’ use of questions for Mothers 1, 2, 3, and 4 and in decreasing Mother 2’s use of commands. CDIT was not considered to be efficacious in decreasing mothers’ use of negative talk. These results are consistent with the literature on CDIT (e.g., Ginn et al., 2017, N’zi et al., 2016). For example, Ginn and colleagues (2017) reported that parents who received CDIT demonstrated significantly more positive parenting skills and less negative parenting skills. This finding is also demonstrated in the present study. Child Behavior As hypothesized, child behavior ratings decreased following implementation of CDIT, with the exception of Child 4. However, it should be noted that Child 4’s behaviors did decrease as rated by the father. Overall results of this study are consistent with other studies (Berkovtis et al., 2010; Ginn et al., 2017; N’zi et al., 2016) Analysis of ECBI ratings revealed a decrease in mean ECBI scores from baseline to intervention for all four children. Visual analysis yielded three demonstrations of an effect for a reduction in child behavior problems. Nonparametric effect sizes (Tau-U) and parametric effect sizes (Hedge’s g) were also calculated among all four dyads. Large effects were found for Child 1 and Child 3 (-1.00), which are similar to the effect sizes found in prior studies of standard PCIT (e.g., Cooley et al., 2014; Eyberg, Nelson, & Boggs, 2008; Ward et al., 2016). Across all four dyads, Hedge’s g calculations did not yield a statistically significant effect size. Findings from Tau-U analyses provide support for the use of a brief intervention in improving child behavior outcomes, as results of the present study are comparable to results standard PCIT studies. 138 ECBI ratings of Child 1’s and Child 3’s behavior problems also fell in the normative range upon CDIT completion. Child behavior problems continued to be rated in the elevated range for Child 2 and Child 4; however, at follow-up, Mother 2 rated Child 2’s behavior problems in the normative range. In sum, CDIT was considered to be effective in decreasing child behavior problems for Child 1 and Child 3 and partially effective in decreasing child behavior problems for Child 2 and Child 4. Child 4 was demonstrating improvements in her behavior at the beginning of the intervention. At follow-up, parent ratings indicated that her behavior was starting to improve again. These findings are significant, as it is difficult to effect behavior change in children whose ratings are closer to normative levels. Behavior findings from the present study are supported by previous research. Ginn and colleagues (2017) completed CDIT with 15 families with children with ASD. Pre-intervention, children had mean ratings in the clinically significant range (M=134.67). At post-treatment, children’s mean ratings of behavior problems fell in the normative range upon CDIT completion (M=101.20). Similarly, Berkovits and colleagues (2010) examined the use of a brief, prevention focused intervention (two sessions of Child-Direction Interaction and two sessions of Parent- Directed Interaction) with families with children with elevated (mild) levels of behavior concerns. Pre-intervention, children had mean ratings in the normative range (M=107.71). Post- intervention, children had even lower mean ratings in the normative range (M=87.40). Although Berkovits et al. (2010) did not implement CDIT, findings from this study also support that the use of a brief, preventative approach can be effective in decreasing emerging behavior problems. Compared to other parenting programs such as PMTO, Incredible Years, and Triple P, CDIT also shows promise in improving child outcomes. PMTO, Incredible Years, and Triple P cite small to medium effect sizes in improving child externalizing behaviors and emotional 139 concerns, post-intervention (Dishion et al., 2016; Menting et al., 2013; Sanders et al., 2014). Additionally, these multicomponent programs require intervention implementation with a set dosage to effect behavior change, which requires financial resources and parent commitment. In contrast CDIT is brief, thereby reducing financial costs and time commitment for families, while also teaching evidence-based parenting skills and providing in-vivo coaching to parents. Thus, there is a relative advantage of using CDIT compared to other parenting programs, particularly as a preventative intervention. To corroborate ECBI ratings, observations of child behavior were done during the 5- minute DPICS. Although some problem behaviors were observed (e.g., hitting, screaming, crying, argumentation, restlessness), child observation data did not match parent reports of child behavior problems. This may be an indication of how parental tolerance affects parental perception of child behavior problems. It may also be that the frequency and severity of the reported child behavior problems were not adequately captured in the five-minute observation period. Additionally, some children (e.g., Child 1, Child 2, and Child 4) exhibited behaviors such as argumentation, restlessness, screaming, and hitting during sessions, but not during the 5- minute observation period. Since the DPICS observation occurs at the beginning of each session, mothers and children may not be “warmed up” to play; thus, children may be less likely to display externalizing behaviors right at the start of the session. It may also be the case that during the DPICS, children remember that the clinician is present during the session and want to appear more well-behaved for other people, as the clinician would have greeted the parent and child only a few minutes before the DPCIS observation began. Given that the DPICS is an observation of the parent-child interactions and parents’ parenting style, coaching cannot occur before the observation as to not influence the parents’ behavior. 140 Post-intervention BASC-3 scores indicated that some children’s externalizing behavior problems (i.e., overall externalizing problems, hyperactivity, and aggression) fell within the normative (i.e., average) ranges following CDIT intervention, with the exception of Child 2 and Child 4. Child 2’s ratings of hyperactive behavior still fell in the at-risk range following CDIT, although they decreased from pre- to post-intervention. Mother 4 continued to rate her child’s behavior within the at-risk range following the CDIT intervention in Externalizing Problems, hyperactivity, and aggressive behaviors, although there was a decrease in ratings from pre- intervention. As noted previously, Child 2 and Child 4 experienced extraneous factors that likely contributed to mothers’ higher ratings of child behavior problems. Given the familial stressors in Child 2’s family towards the end of the intervention (e.g., mother and grandmother falling ill) and the frequent changes in household every week, Child 2’s slight increase in mother-rated child behavior problems would be expected. Despite these situational stressors, Child 2’s mean levels of behavior problems decreased over the course of the intervention. Child 4 also experienced an uptick in parent-reported child behavior problems from both the mother’s and father’s perspectives that coincided with a change in her preschool. However, three weeks after the school change, parents reported decreasing levels of behavior problems, supporting the hypothesis that the behavior problems were due to a change in environment. While Father 4 initially rated Child 4’s behaviors as more problematic prior to the start of CDIT, Mother 4 consistently rated Child 4’s behaviors as more severe over the course of the intervention. After implementation of CDIT, Father 4 rated Child 4’s behaviors in the normative range of behavior, while Mother 4 rated her behaviors in the at-risk range. Different parent ratings may indicate different parental tolerance levels (i.e., one parent may better cope with 141 child behavior problems). Alternatively, different child behavior ratings may also be due to who spends more time with the child. In this case, Mother 4 was home with Child 4 more often than Father 4; thus, it is likely that she had more opportunities to observe externalizing behaviors and would explain the higher ratings from her perspective. Given the increase in parent-reports of child externalizing behavior, Dyad 4 was referred for full PCIT. Findings from N’zi and colleagues (2016) indicated that two families were also given a referral for full PCIT following CDIT implementation. Thus, although CDIT may be beneficial for addressing emerging child behavior problems, more intensive intervention may be needed for children exhibited higher levels of behavior problems. Summary of Child Behavior Outcomes Even though child behavior outcomes varied across participants, three children demonstrated overall improvement in their problematic behaviors. While only Child 1 and Child 3 demonstrated significant changes in their levels of behavior, Child 2 exhibited improvement in his behavior problems by follow-up. Prior studies of CDIT indicated maintenance of child behavior outcomes at 6-week, 3-month, and 6-month follow-up (Ginn et al., 2017; N’zi et al., 2016). Thus, it is possible that the positive behavior changes exhibited in the three child participants will be maintained, providing further support for the use of CDIT in improving child behavior outcomes over a short period of time. Intervention Fidelity Adherence As expected, the study clinician implemented CDIT with high treatment adherence, with an average overall adherence percentage of 97% across all four dyads, consistent with previous findings (e.g., Ginn et al., 2017, N’Zi et al., 2016). An overall adherence percentage of 80% or 142 higher was needed to achieve the minimum rating required for adequate treatment adherence (Kratochwill et al., 2013). CDIT followed the manualized PCIT protocol (Eyberg & Funderburk, 2011), which increased the likelihood that it would be carried out as intended (Perepletchikova & Kazdin, 2005). These findings suggest that CDIT can be implemented via telehealth while still maintaining high levels of adherence to the protocol, despite barriers that can be attributed to treatment delivered via telehealth. Dosage Dosage of the intervention was evaluated by examining treatment attendance. All four dyads attended 100% of CDIT sessions with no attrition, indicating high dosage of the intervention. It was hypothesized that participants would attend 90-100% of all sessions. These results are consistent with previous studies (Ginn et al, 2017; N’zi and colleagues, 2016). Although there was a month between the end of the intervention and Dyad 2’s follow-up session due to illness, Dyad 2 still rescheduled the post-treatment session and attended all intervention sessions. High treatment attendance may be indicative of participants’ satisfaction with the intervention, as well as due to the brevity of the intervention. Quality of Intervention Delivery Independent ratings by the study clinician and inter-rater observations by the research assistant across 30% of all sessions indicated high quality implementation of CDIT, as indicated by an average overall score of 3.97 (average score needed to be 3 or higher out of a possible score of 4). This indicates that the study clinician was able to use triadic and collaborative strategies throughout CDIT and while implementing the intervention via telehealth. This is an important finding that supports the future use of CDIT, as well as demonstrates that collaboration and the use of triadic strategies is possible via telehealth. The use of triadic strategies is 143 important for parent training interventions (Eyberg & Funderburk, 2011). The use of triadic strategies may also explain participants’ high adherence to treatment. The use of responsive coaching techniques during intervention and engagement with parents in problem-solving has been shown to predict higher level of skill acquisition and higher levels of treatment adherence (Barnett et al., 2015). Participant Responsiveness As hypothesized, homework completion rates were above 50% for all four dyads (average weekly range = 57% - 87%). According to the PCIT manual, homework rates should aim to be above 70% (Eyberg & Funderburk, 2011). Dyads 1 and 4 achieved rates over 70%, while Dyads 2 and 3 achieved rates between 50-65%. Findings from this study were consistent with Ginn and colleagues (2017) and N’zi and colleagues (2016) who reported homework completion rates above 50%. Despite lower homework completion rates, all four dyads demonstrated improvements in their parenting skills and decreases in their negative parenting behaviors. Additionally, three children demonstrated improvements in their problem behaviors. Based on these findings, completion of daily homework may not be necessary in order to achieve improvements in parenting skills and decreases in negative child behaviors. Stokes and colleagues (2016) found that parent report of homework completion rate was not related to changes in child disruptive behavior, although a higher rate of homework completion was predicative of parental mastery of skills in fewer sessions. Ros and colleagues (2017) found that higher levels of homework completion were associated with decreases in negative parenting practices and higher levels of confidence for families that also had high treatment knowledge gains. This indicates that homework completion may be beneficial, but limited knowledge of appropriate skills to use during home practice may lead to diminished outcomes (Ros et al., 144 2017). Thus, the rate of skill acquisition and quality of intervention implementation may be more important. Future research is needed to closely examine the relation between homework completion, mastery of parenting skills, and reductions of child problem behaviors. Satisfaction As hypothesized, all four mothers indicated high treatment acceptability and reported high satisfaction with CDIT. This result is consistent with results from standard PCIT studies (e.g., Bagner et al., Berkovits et al., 2010; Bjørseth & Wichstrøm, 2016), as prior CDIT studies did not examine caregiver satisfaction directly. All four mothers had ratings above 30 on the TAI, indicating high acceptability of the intervention. Generally, all mothers reported they learned several or many useful techniques and believed their children’s behavior problems had greatly or somewhat improved as a result of the intervention. All mothers reported their relationships with their children had very much improved since the beginning of the study. They also all indicated satisfaction with the progress made in their children’s general behaviors and indicated that they liked the intervention very much. Lastly, they reported that the intervention helped somewhat with other general personal or family problems not directly related to their child. Parental satisfaction with treatment has been shown to contribute towards the maintenance of parenting skills (Brestan, Jacob, Rayfield, & Eyberg, 1999). Thus, higher satisfaction ratings may be more likely to lead to positive parenting skill acquisition and adherence to treatment. Two mothers qualitatively noted that they wished an intervention like CDIT would be available for more parents, as CDIT helped improve their parenting behaviors and their relationships with their children. One mother also noted that the implementation via telehealth would make the intervention more accessible to parents. This mother also noted that parents with 145 children without disabilities could benefit from an intervention such as CDIT, as children without clinical diagnoses do not often receive treatment. High parent satisfaction with CDIT is significant, as the intervention was much shorter compared to PCIT and implemented with a new population (i.e., children with subclinical behaviors). Participating mothers still rated CDIT as a highly useful intervention and felt they received useful parenting techniques that helped improve the parent-child relationship and improve child behavior problems. As noted previously, the finding that child behavior problems decreased and that parents noticed this change is significant, as it is more difficult to effect behavior change for children who do not demonstrate a severe level of behavior problems. Additionally, CDIT was implemented via telehealth compared to previous studies where CDIT was implemented in-person. Despite the change in context, mothers still reported high levels of acceptability with CDIT, providing support for the use of CDIT via telehealth in the future. Limitations and Implications for Future Research A few limitations should be noted about the present study. First, due to the small sample size of the study (N = 4 dyads), inferences about treatment effects are limited. Due to the global pandemic and the need to modify study components, recruitment was suspended for 2 months. Eligibility criteria was extended by one standard deviation on the BASC-3 to include children with slightly lower and slightly higher ratings of externalizing behavior problems due to slow study recruitment. However, smaller sample sizes with single-case designs are common. Additionally, the nature of single-case design allows for a more in-depth analysis of treatment and non-treatment responses using a small number of participants. Although inferences about treatment effects are limited, within-case effect size indices (e.g., Tau-U) allow for the examination of specific behaviors. 146 Second, the generalizability of results is limited in scope, as the study sample was not homogenous in nature. While the primary parent was mothers (not fathers) and children were between the ages of 3-5 with no clinical diagnoses of behavior problems, the study sample was somewhat diverse. For example, participating children were between the ages 3-5 and included two male and two female child participants. Mothers were between the ages of 30-40. Three mothers were married and obtained higher education degrees in education, psychology, and counseling, while one was a single parent with some higher education. Two mothers had other children in the home. While the second parent was always invited to participate in the intervention, only one father chose to do so, which may have further complicated the study’s findings due to inconsistent parenting behaviors being used in the home. Replication of the current study with a more homogenous and a larger sample is recommended to increase the generalizability of the results. For example, the current single-case design study may be replicated with male children who are five years-old, then with female children who are five years-old, then with male children who are four years-old, etc. to determine the efficacy of the intervention with certain populations. Additionally, future studies should consider replication of the current study with fathers as the primary parent, as well as with different populations of children (e.g., all female children, all male children, children of the same age, etc.). To increase the generalizability of the study’s results more quickly, a waitlist control group design could be used to replicate the study with male and female participants ages 4-5 years old. With funding available, a randomized control trial (RCT) could also be implemented with male and female participants ages 4-5 years old to increase generalizability of the study’s findings and make the intervention more widely available. 147 Another limitation of the study includes the a priori selection of baseline points and intervention points. Although a multiple probe design does allow for a priori designation of baseline probe sessions to help reduce the likelihood of attrition and threat to history, it can prevent participants from establishing stable baselines. The number of baseline and maximum number of treatment sessions were pre-established due to the nature of the study (e.g., as this was a dissertation study, it was necessary to establish a completion timeline). In standard PCIT, mothers need to achieve mastery in order to move into the next phase of treatment. For this study, while the goal was for mothers to meet mastery, it was not necessary in order to end the intervention. In this study, two mothers did not meet mastery prior to the end of the intervention. However, the study clinician used clinical judgment and provided a follow-up “refresher session” with coaching to these mothers, as well as referred one family for full PCIT. Future studies should also consider incorporating follow-up sessions more than one week after completion of the intervention to more accurately assess maintenance in changes in parenting skills over time. Another limitation of the study includes the participation of both parents for Dyad 4. It is possible that Mother 4 did not meet mastery criteria due to a different dosage of the intervention. Mother 4 received only 30 minutes of intervention compared to the other mothers who received 45 minutes of intervention, thus receiving less dosage of the intervention compared to the other participants. Mother 4 had less opportunity to practice her parenting skills and to receive coaching on them, which may have contributed to her lack of skill mastery. Furthermore, the child received CDIT from both parents whereas the other children only received CDIT from their mothers. Although the purpose of including the mothers’ spouses/partners in the intervention was to increase consistency among parenting behaviors in order to decrease child behavior 148 problems, it is possible that the participation of both parents affected Child 4’s behavior differently compared to other participating mother-child dyads. An additional limitation is that reports of child behavior problems were based exclusively on mothers’ reports. While one father participated in the study, his data was not used to make decisions to continue and/or end the intervention. Other caretakers (e.g., fathers, grandparents, babysitters) and teachers have varied experiences with and observations of children with emerging behavior problems; thus, reports from different people about the child’s behaviors may have provided further information about the impact of CDIT on children’s behavior problems in varied contexts. Further limitations of the study include natural maturation of the children during their participation in CDIT. Events in children’s and their parents’ lives occurred concurrently with the intervention, which may have contributed to treatment effects identified and/or have contributed to a lack of a treatment effect. For example, child participants may have experienced reductions in their behavior problems due to being exposed to appropriate behaviors modeled by peers and teachers if they attended a preschool and/or daycare. Alternatively, changes in the child’s environment (e.g., preschool and/or daycare transitions, changes in the home environment) may have contributed to a lack of a treatment effect. Differences in mothers’ and fathers’ tolerance of child behavior and/or handling of child behavior problems may have also affected a treatment effect. For example, if parents are inconsistent in how they treat child behavior (e.g., giving a different number of warnings, providing different consequences, taking varied disciplinary action, or taking away privileges versus ignoring problematic behavior, praising for good behavior, etc.), the child may be confused and have difficulty learning what is and is not appropriate behavior at home. The use of a concurrent single-case design would have 149 allowed for more control of maturation effects; however, there appeared to be a lack of maturation effects among dyads while they were in baseline compared to intervention. Finally, the global pandemic was also considered a limitation of the study. Recent studies have shown that quarantine alone has been a stressful situation for parents due to the balance of personal life, work, caring for and educating children, and for some, a lack of resources (Spinelli et al., 2020). The pandemic has also affected children’s behavioral and emotional functioning, which has also been mediated by parents’ individual and dyadic stress (Brown et al., 2020; Spinelli et al., 2020). Thus, there are higher levels of need from parents and children, making it more difficult to intervene, implement research studies, and affect behavior change as quickly as before the pandemic. Parents may also have lower levels of motivation and/or energy to complete homework or practice using parenting skills due to the stress of the pandemic. Future research is needed to replicate findings of the current study, as well as extend them. In a public health approach to service delivery, multi-tiered, comprehensive services are provided in a systematic way. Presently, there is a gap in the literature for targeted secondary services. Findings from the present study indicate promise for the use of CDIT as a Tier 2 (targeted) intervention to prevent the development of more severe behaviors later on. While this study addresses a gap in the literature, more research on CDIT as a potential Tier 2 intervention is needed. Preventative efforts need to increase in order to reduce the need for more intensive services so that the service delivery system is not overwhelmed at the tertiary level (Kessler et al., 2005). In addition to replication of this study with different parent and child populations, future research is needed to examine the use of CDIT across different contexts and in different formats. For example, researchers may examine group format CDIT (e.g., one practitioner with 3-5 families) delivered in-person and also delivered via telehealth, as a group format allows 150 exponentially more families to be served, and a remote setting may decrease barriers to service delivery. To date, group PCIT has been examined with parent-child dyads and shows evidence of improving positive parenting skills and decreasing child problem behaviors (e.g., Foley et al., 2016; Nieter et al., 2013). Given that some mothers naturally spoke more often to their child compared to other mothers—thereby increasing their total number of verbalizations of positive praise, reflections, and behavioral descriptions and/or questions, commands, and negative talk—future research may also consider the use of ratios to examine the portion of positive and negative parental utterances for each parent. This would provide additional information on the use of a parent’s positive versus negative parenting behaviors and would control for parents who are not as talkative as other parents. Additionally, more research is needed to examine CDIT intervention fidelity with different providers (e.g., school psychologist, social workers, licensed mental health clinicians, etc.) who have not received formal PCIT training. In the present study, another limitation was that the clinician was a graduate student who had received PCIT training from a Level 1 trainer, had experience delivering standard PCIT, and was highly motivated to complete the intervention. Studies examining CDIT fidelity with different types of clinicians would help determine whether formal training is needed in order to deliver CDIT, thus informing future service delivery practices. Lessons Learned from Telehealth Service Delivery Although there were challenges to implementing an intervention via Telehealth and during a global pandemic, there were several benefits. The biggest benefit seemed to be that the use of Telehealth increased accessibility to care for families. Given that families did not have to 151 pick their child up from home, school, or daycare and then travel to a clinic, it increased their availability to participate in the intervention. This also meant that families could schedule the intervention at a more convenient time for their child and themselves (e.g., after the child had a nap, after snack time, not too late before bedtime). Additionally, if a family needed to reschedule an intervention session due to another conflict, it was easy to find another time to meet. This may have been the reason why families completed all sessions and received 100% of the intervention dosage. Comer and colleagues (2017) also reported that I-PCIT was associated with high engagement and treatment satisfaction and reduced perceived barriers to participation. Another benefit included the implementation of the intervention in the families’ homes. This provided a more naturalistic environment for the mother and the child and increased the likelihood of observing child behavior problems, as the child would be more comfortable in his/her home environment. Additionally, the child was able to use his/her own toys during CDIT, which may have increased the child’s willingness to participate in the intervention and seem more like daily Special Time. Lastly, mothers were able to receive coaching in a more naturalistic environment and in real time. This was particularly beneficial for Dyad 1 when Child 1 had behavioral outburst. For example, on a couple occasions, Child 1 had a behavioral outburst during the intervention session and ran out of the designated room. Mother 1 was able to follow him in the house and receive coaching in how to use positive parenting skills to re-engage the child in the session and/or to use planned ignoring. This allowed the mother to receive real-time coaching to address a typical child behavior scenario whereas she would not have had the opportunity in clinic based PCIT. Findings from the current study also provide support for the use of telehealth, especially during the global pandemic. CDIT was implemented with fidelity in a virtual space, with high 152 levels of adherence to the protocol by the study clinician, strong participant engagement in the intervention, and zero percent attrition rates. Findings from this study are consistent with one prior study of internet-delivered PCIT (I-PCIT), which found that the rate of posttreatment response for child behavior problems was higher for children in I-PCIT than in standard PCIT (Comer et al., 2017). While research on I-PCIT is new and limited, this study adds to the present literature and demonstrates the benefits of remote service delivery. Although more studies are needed to examine the use of telehealth services and outcomes from interventions delivered via telehealth, this study provides promising support for the potential use of telehealth and demonstrates how it increases access to mental health care. For example, the use of telehealth allows for services to be providing to a broader population, including those with a lack of transportation, families living in remote or rural regions, families with less access to quality care, families who may move due to jobs (e.g., military), or those who are hesitant to attend therapy in person due to the stigma of mental health, etc. and ensures that the provision of services can continue in an emergency situation, such as a global pandemic. Alternatively, consideration must be given to the resources needed to participate in CDIT via telehealth. For example, internet, webcam technology, and Smartphones can be costly expenses, which can affect the ability to participate in internet delivered CDIT. Clinicians may consider how to adapt CDIT to address these barriers, including the use of different contexts. For example, instead of staying in the home, parent-child dyads may use a private room in a public library so they can access internet and webcam technology. Implications for Practice The findings of the current study suggest that CDIT delivered via telehealth can be an effective intervention to increase mothers’ positive parenting skills, decrease negative parenting 153 behaviors, and decrease children’s disruptive behaviors using evidence-based practices. In the current study, CDIT was somewhat effective in improving children’s subclinical levels of behavior based on mothers’ ratings (ECBI). By the end of the intervention, three mothers reported their child’s overall externalizing behaviors, including aggression and hyperactivity, were no different from typically developing children of their age and gender based on the BASC- 3. In addition, all mothers reported high levels of satisfaction, improvement in the parent-child relationship, and their children’s behavior problems. Although this study is a pilot, these data provide support for the use of CDIT as a preventative approach (i.e., early intervention) to improve behavior outcomes of preschool-aged children with emerging behavior problems through enhancement of the parent-child relationship and improving parenting skills (McNeil & Hembree-Kigin, 2011). The use of collaborative and triadic strategies to teach and coach parents during CDIT consist of best practices in parent training that are endorsed by other parent training and interaction therapy protocols (e.g., Barnett et al., 2017; Eyberg 1988; Shanely & Niec, 2010). Practitioners’ use of triadic strategies may help to increase the use of positive reinforcement, reflective statements, and behavioral descriptions and decrease the use of questions, commands, and negative talk among parents of children with emerging behavior problems. More specifically, these strategies may be used by a variety of personnel who work closely with children and parents, including school psychologists, teachers, pediatricians, coaches, and child psychologists, to reduce child behavior problems. CDIT strategies may also be utilized by a variety of practitioners and personnel who work directly with children and families who may not have the physical facilities or level of training required to provide standard PCIT in their practice. 154 Many CDIT strategies have foundations in attachment and social learning principals to teach authoritative parenting. Clinicians trained in counseling or psychology fields have likely received education in these theories and understand the concepts of positive reinforcement, labeled praises, and reflective statements, as well as the importance of fostering positive parent- child relationships. Practitioners may employ the use of CDIT strategies without PCIT training, and model how to utilize CDIT skills for parents; however, as noted previously, more research is necessary in order to determine whether formal training in PCIT is still needed to implement CDIT. Even if formal training is still required for CDIT, school districts may consider investing $4000-$4700 in training one or two school psychologists so that they may implement CDIT with families within the district. If research also supports the use of group format CDIT, this would allow for a greater number of individuals to receive services in a shorter amount of time, thus operating under a public health approach. Although a seemingly costly investment for districts, implementing CDIT may drastically decrease the number of behavior referrals, behavior plans, and Individualized Education Programs (IEPs), as well as the number of staff needed to support students with externalizing behaviors, thereby significantly decreasing future costs to the school system. Given that CDIT is short, feasible, and shows promise for its delivery via telehealth, school psychologists specifically may consider incorporating the use of CDIT as a Tier 2 prevention strategy in the service delivery model. For example, to reach a broader audience, school psychologists may provide mini workshops to parents of preschool children to explain the importance of positive parent-child interactions, teach positive parenting practices, and discuss how these practices can be applied in the home. Given that the largest changes were seen in mothers’ use of labeled praises, school psychologists may consider focusing on teaching parents 155 to use labeled praises, as it would be expected that this parenting behavior would improve more quickly compared to parents’ use of reflections or behavior descriptions. While it would still be beneficial to introduce all three positive parenting skills to parents, it may take more time for parents to attain reflections and behavior descriptions. The workshop could be provided as a mini-series through a child development center, school district, parenting organization, library, or pediatric office. Another option may be for the school psychologist to implement CDIT with parents of children who are currently exhibiting behavioral problems, including hyperactivity, aggression, and defiance. School psychologists working with early childhood populations may provide individual consultation and/or brief coaching sessions to parents to help parents practice positive parenting skills. During consultation sessions, school psychologists may engage in an open discussion with parents on parenting practices and discuss CDIT strategies in order to reduce the risk of the development of more severe child behaviors in the future. It is important to note that three out of the four participating children in the present study demonstrated parent-rated behaviors in the normative range after CDIT implementation, despite not all parents demonstrating significant improvements in all parenting behaviors. Thus, it may be the case that it is not necessary for all parenting behaviors to change in order to see improvements in child behavior problems. School psychologists may consider targeting certain behaviors to teach parents (e.g., increasing the use of labeled praises, while decreasing the use of questions and commands) during workshops or individual consultation sessions. Practitioners should also consider how other factors such as parental stress, parental motivation to change, disruptions to and/or changes in the child’s environment, or child internalizing factors may influence how successful CDIT is with parent-child dyads. 156 PCIT has been modified for school settings (referred to as Teacher-Child Interaction Therapy, or TCIT) to improve the teacher-child relationship and decrease child behavior problems in the classroom and has also shown positive results when used with one teacher-child dyad (McIntosh et al., 2000), with one teacher and a classroom (Filcheck et al., 2004), and when used as a prevention program in Head Start Classrooms (Gershenson et al., 2010; Tiano & McNeil 2006). School psychologists may also consider the use of TCDIT with teachers who have children exhibiting problematic behaviors in the classroom. Since the core elements of PCIT are retained in TCIT, it is possible that TDIT may provide the same benefits as CDIT, although future research is needed to examine this hypothesis. This study contributed to the literature by examining the use of CDIT with children who exhibit subclinical levels of behavior. Findings from this study suggest that CDIT was able to reduce some child disruptive behaviors significantly, even to the point of decreasing behaviors to a normative range. This has implications for parental levels of stress, as well as clinical practices. Presently, a child typically has to hold a diagnosis of a disruptive behavior disorder in order to receive standard PCIT treatment. However, if practitioners consider providing a brief, feasible intervention such as CDIT to parents with children with emerging behavior problems, this could significantly improve both parent and child behaviors, as well as reduce the need for more intensive services later on. This preventative approach to the provision of services will also ensure that the service delivery system is able to provide adequate supports and intervention to families without becoming overwhelmed and costly to the individuals and the greater public. Although there are many parenting interventions that can be used (e.g., PMTO, Incredible Years, Triple P) to address child behavior problems and prevent more severe problems later on, findings from the present study demonstrate comparable parent and child outcomes to such programs, 157 while also providing in-vivo coaching to parents and teaching commonly used positive parenting practices. Thus, consideration for the use of a brief, less costly intervention like CDIT should be given to help improve child outcomes and increase access to services. Conclusion Research is needed to identify evidence-based preventative interventions for the purpose of providing services early to prevent more severe problem behaviors from developing later on. If untreated, preschool children with emerging behavior problems may undergo more functional impairment later in life compared to children who receive early intervention. One way to improve child behavior problems is through parent-training interventions. Given the gap in the literature with support for treating emerging behavior problems, the current study sought to determine whether a modified version of PCIT (CDIT) could be used as a feasible and acceptable parent training intervention for parents and their children who exhibit subclinical levels of behavior. Results of the study demonstrated support for the use of CDIT as a preventative intervention. Following implementation of CDIT delivered via telehealth, mothers’ positive parenting skills improved, while child behavior problems decreased. Future research is needed to identify whether CDIT can be used with a variety of populations, including slightly older children and/or with different caregivers, as well as in different formats across different contexts (e.g., group remote setting, group in-person setting, clinic-based, school-based, etc.). 158 APPENDICES 159 Appendix A CDI Mastery Criteria Table 15. CDI Mastery Criteria Skill Examples Behavioral Description Child is stacking blocks Parent: You are making a tall tower. Describe what the child is doing Child placed the horse in the barn. Parent: You put the horse in the barn. Child comes to sit next to parent. Parent: You are sitting beside me. Reflection Child: I drew a house. Parent: You did draw a house. Repeat or paraphrase what the child says Child: I love to play with the magnet tiles. Parent: You are having fun with the magnet tiles. Child: What are we having for dinner? Parent: You are wondering what we are going to eat for dinner. Labeled Praise Child hands a car to the parent. Parent: Thank you for sharing! Specifically state what you like about what the child is Child is putting crayons in the box. doing or saying Parent: I love how gently you are putting the crayons away. Child draws a heart. Parent: You drew a very pretty heart! 160 Appendix B PCIT Trainer Levels Table 16. PCIT Trainer Levels Type of Level of Training Qualifications Training Components Training Level I Within Program Trainer 1)Have met all the 1) 8 hours of initial Trainer (i.e., can teach/supervise requirements to be a Level 1 training with a ($1500- individuals at their own certified PCIT Therapist Master Trainer or Level 1700) program or agency) II Trainer 2) Served as the PCIT therapist for 4 PCIT cases 2) Supervision and to graduation criteria and consultation at least have been the primary once a month from a therapist on 3 of those Master Trainer or Level cases. II Trainer over the course of 12 months 3) Been the primary therapist or supervisor on 3) Supervise at least one at least two PCIT cases therapist-in-training and within the previous 2 have the trainee years at the time of successfully graduate application. one case (trainee is the primary therapist) while under the supervision of a Master Trainer or Level II Trainer 4) Ensure the supervision sessions with a therapist-in- training are observed by a Master Trainer or Level II Trainer in real time or through video recording 5) Successful completion of skill review (feedback given by the Master Trainer or Level II Trainer) 161 Table 16 (cont’d) 6) Certification as a Level I Trainer is renewable every 2 years. Six hours of PCIT continuing education credits are required. Level II Within Geographic Region 1)Have met all the 1) Have experience as a Trainer Trainer (i.e., can requirements to be a trainer for at least 10 teach/supervise professional certified PCIT Therapist new therapists who are or graduate-level eligible for certification individuals internal and 2) Have met all the as a PCIT therapist. external to their own requirements to be a program or agency within Level 1 Trainer 2) Have experience their geographic region) training and supervising 3) Served as the primary therapists as a trainer therapist for a minimum for at least two Basic of 20 PCIT cases to Trainings conducted by graduation criteria two separate Master Trainers 4) Documented history of at least seven years of 3) Co-lead one Basic active involvement in the Training consultation PCIT community call series with a Certified Master Trainer 5) At least three PCIT or Level II Trainer for publications or 12 months presentations at the state, national, or international 4) Co-lead one Basic PCIT conferences Training consultation call series under consultation of a Master Trainer or Level II Trainer 5) Experience training Level 1 applicants as a training assistant for at least one Initial Level 1 Trainer Training 6) Experience co- leading one cohort of Level 1 trainer calls 162 Table 16 (cont’d) with a Level II or Master Trainer 7) Provide PCIT supervision and consultation OR direct service to at least 5 PCIT cases a year 8) Arrange a site visitor with a Master Trainer or Level II Trainer ($1000 honorarium plus travel expenses) 9) Engage in monthly consultation calls with a Master Trainer 10) After the first year of practice, consult with a Master Trainer once every six months 11) Certification as a Level II Trainer is renewable every 2 years. 12 hours of PCIT continuing education credits and consultation with a Master Trainer once every six months are required Master National and International Requirements to be Requirements to be Trainer Trainers (i.e., can teach/ announced on the PCIT announced on the PCIT supervise professional or International Website International Website graduate-level clinicians in national and international mental health service agencies) Note: Developed based on the PCIT International training requirements (PCIT International, updated 7/25/2017) 163 Appendix C PRIDE Skills Handout Table 17. Pride Skills Handout PRIDE SKILLS Purpose EXAMPLES Praise for appropriate • Increases desired behavior • Thank you for behavior • Lets the child know what showing me your you like tower. • Makes both the parent and • I like the way you child feel good! are playing gently with the dolls. Reflect appropriate talk • Lets the child lead the • Child: This is a conversation castle. • Shows the child you are Parent: Yes, that is listening a castle. • Improves the child’s speech • Child: I like to play • Increases verbal with the trains. communication Parent: You like playing with the trains. Imitate appropriate play • Lets the child lead the play • Child (makes a • Shows approval of the house with the child’s choice of play blocks) • Teaches the child how to Parent: I’m going play with others to make a house too. Describe appropriate • Allows child to lead play • You’re coloring behavior • Shows interest neatly inside the • Teaches concepts lines. • Models speech and • You put the tiger on vocabulary top of the train. • Holds the child’s attention to the task Enjoy the interaction • Models appropriate • I’m having so much positive emotions fun playing with • Demonstrates interest in you! the child • Warm touches • Strengthens the parent- • Smiling child relationship • Laughing Note: Adapted from the PCIT Protocol (Eyberg & Funderburk, 2011). 164 Appendix D Toys for CDI Handout Suggested Toys for CDI • Creative, constructive toys o Letter blocks o Legos, Duplos o Tinker toys o Magnetic tiles o Lincoln logs o Mr. and Mrs. Potato Head o Crayons and paper o Chalkboard and colored chalk o Foam blocks o Play sets (e.g., farms, houses, toy foods) Toys to Avoid during CDI • Toys that encourage rough play (e.g., balls, bats, airplanes) • Toys that lead to aggressive play (e.g., toy guns, toy swords, toy soldiers, super-hero figures) • Toys that could get out of hand and require limit setting (e.g., paints, markers, bubbles, scissors, toy hammer) • Toys that have pre-set rules (e.g., board games, card games) • Toys that discourage conversation (e.g., books, computer games) • Pretend talk toys that lead the caregiver or child to imagine they are someone else, like puppets or costumes 165 Appendix E PCIT, Standard and Modified Versions Table 18. PCIT, Standard and Modified Versions Standard PCIT Modified PCIT (~15-20 weeks, 15-20 treatment sessions (9 weeks, 9 treatment sessions total) total) Session 1 Pretreatment DPICS Baseline Phase (Pretreatment) CDI Phase Session 1 Pretreatment DPICS Code parent-child interactions for praise, reflections, behavioral descriptions, questions, commands, and negative talk. Session 2 Intro to Treatment + Session 2 Pretreatment (Parent Only) CDI Teach DPICS Code parent-child interactions for praise, reflections, behavioral descriptions, questions, commands, and negative talk. Session 3 CDI Coach 1 Session 3 Pretreatment (Coding, Coach) DPICS CDI Teach Code parent-child interactions for praise, reflections, behavioral descriptions, questions, commands, and negative talk. Give an overview of PCIT, introduce CDI, demonstrate and practice PRIDE skills, and give CDI homework. Session 4 CDI Coach 2 Treatment Phase (Coding, Coach) + Review Mastery Criteria Session 5 CDI Coach 3 Session 4 CDI Coach 1 (Coding, Coach) Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on behavioral descriptions. 166 Table 18 (cont’d) Session 6+ CDI Coach 4 Session 5 CDI Coach 2 (Coding, Coach) Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on reflections and avoiding questions. PDI Phase Session 6 CDI Coach 3 Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on labeled praises. Session 7 PDI Teach Session 7 CDI Coach (Parent Only) (Rules for Effective 4 Commands, Time- Out Procedure) Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on whichever PRIDE skill is the weakest. Session 8 PDI Coach 1 Session 8 CDI Coach (Coach in play) + 5 PDI Homework (in play) Code parent-child interactions for PRIDE skills and coach parent(s) with a focus on whichever PRIDE skill is the weakest. Session 9 PDI Coach 2 (Code Session 9 Post CDI, Coach in play) treatment DPICS + Graduation Code parent-child interactions for praise, reflections, behavioral descriptions, questions, commands, and negative talk. Graduation ceremony with PCIT graduation certificate. Session 10 PDI Coach 3 Maintenance Phase (Code CDI, Code PDI, Coach in play and cleanup) Session 11 PDI Coach 4 Session 10 Follow-up (Coach) + Review DPICS Running Commands Code parent-child interactions for praise, reflections, behavioral descriptions, questions, commands, and negative talk. One follow-up coaching session (as-needed) 167 Table 18 (cont’d) Session PDI Coach 5 + 12 Establish House Rules Session PDI Coach 6 (Code 13 CDI, Code PDI, Coach in public outing) Session PDI Coach 7 (Code 14+ CDI, Code PDI) Session Post-treatment 15 DPCIS + (Post- Graduation treatment) Note: General outline was created using the PCIT Protocol (Eyberg & Funderburk, 2011). 168 Appendix F Treatment Integrity Checklist for Pre-Treatment Assessments Table 19. Pre-Treatment Integrity Checklist Item Y NA N Administer ECBI to mother Instruct parent to set up room with 5 sets of toys Greet mother and child Give instructions for Child-Led play situation (verbatim) After warm-up, give prompt to continue Child-Led play Code CDI for exactly 5 minutes Ask the mother if the situation was typical Give the mother brief, supportive feedback about the observation Graph ECBI and CDI skill data TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = # of Ys /Y+ N’s ___________ =__________% Length of session = _____________ 169 Appendix G Treatment Integrity Checklist for CDI Teach Session Table 20. CDIT Teach Integrity Checklist Item Y NA N Greet mother Discuss expectations for treatment Give overview of PCIT Explain the structure of therapy sessions Give overview of CDI Explain the importance of CDI Explain the basic rule for mothers in CDI (follow your child’s lead) Avoid Commands Avoid Questions Avoid Criticism Engage mother in recalling the “don't skills” P stands for Praise: Give your child labeled praise for positive behavior R stands for Reflect: Reflect your child’s appropriate talk I stands for Imitate: Imitate your child’s appropriate play D stands for Describe: Describe the positive things your child is doing E stands for Enjoy: Enjoy special time (CDI) with your child Engage mother in recalling the PRIDE skills Ignore inappropriate behavior as long as it is not aggressive or destructive Describe how to combine ignoring with the “DO” skills Explain that if a negative behavior cannot be ignored, the mother must stop the play Role-play CDI with the mother Describe the kinds of toys to use and avoid during CDI Ask mother specifically what toys she will use Explain how to set up and end the CDI play session at home Explain the importance for practicing CDI for 5 minutes every day Ask the mother to decide what time of day, and what room in her house, they will use for their daily practice Give mother CDI handout and suggested toys for CDI handout Give homework sheets to mother TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = # of Ys /Y+ N’s___________ =__________% 170 Appendix H Treatment Integrity Checklist for CDI Coach 1 Table 21. CDI Coach 1 Integrity Checklist Item Y NA N Administer ECBI to mother Greet mother and child Spend a few minutes addressing mother’s stressors unrelated to the child’s behavior Code mother and child in CDI for 5 minutes Give mother feedback on skills and set goals for coaching Coach mother with child for about 30 minutes Graph ECBI and CDI skill data Introduce ECBI graph and CDI skills progress graph to mother Give homework sheets and discuss what to emphasize TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = Integrity = # of Ys /Y+ N’s ___________ =__________% Length of session = _____________ 171 Appendix I Treatment Integrity Checklist for CDI Coach 2 Table 22. CDI Coach 2 Integrity Checklist Item Y NA N Administer ECBI to mother Greet mother and child Spend a few minutes addressing mother’s stressors unrelated to the child’s behavior Code mother and child in CDI for 5 minutes Give mother feedback on skills and set goals for coaching Coach mother with child for about 30 minutes Graph ECBI and CDI skill data Review ECBI graph and CDI skills progress graph with mother Describe the mastery criteria for CDI Give homework sheets and discuss what to emphasize TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = Integrity = # of Ys /Y+ N’s ___________ =__________% Length of session = _____________ 172 Appendix J Treatment Integrity Checklist for CDI Coach 3 Table 23. CDI Coach 3 Integrity Checklist Item Y NA N Administer ECBI to mother Greet mother and child Spend a few minutes addressing mother’s stressors unrelated to the child’s behavior Code mother and child in CDI for 5 minutes Give mother feedback on skills and set goals for coaching Coach mother with child for about 30 minutes Graph ECBI and CDI skill data Review ECBI graph and CDI skills progress graph with mother If mother does meet CDI mastery criteria…. Discuss transition from treatment and graduation If mother does not meet CDI mastery criteria… Review ECBI and CDI skills graph and discuss what skills to work on Give homework sheets and discuss what to emphasize TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = Integrity = # of Ys /Y+ N’s ___________ =__________% Length of session = ____________ 173 Appendix K Treatment Integrity Checklist for CDI Coach 4 and Beyond Table 24. CDI Coach 4 and Beyond Integrity Checklist Item Y NA N Administer ECBI to mother Greet mother and child Spend a few minutes addressing mother’s stressors unrelated to the child’s behavior Code mother and child in CDI for 5 minutes Give mother feedback on skills and set goals for coaching Coach mother with child for about 30 minutes Graph ECBI and CDI skill data Review ECBI graph and CDI skills progress graph with mother If mother does meet CDI mastery criteria…. Discuss transition from treatment and graduation If mother does not meet CDI mastery criteria… Review ECBI and CDI skills graph and discuss what skills to work on Give homework sheets and discuss what to emphasize TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = Integrity = # of Ys /Y+ N’s ___________ =__________% Length of session = _____________ 174 Appendix L Treatment Integrity Checklist for Post-Treatment and Graduation Session Table 25. Post-Treatment and Graduation Session Integrity Checklist Item Y NA N Administer ECBI to mother Greet mother and child Instruct mother to set up room with 5 sets of toys Give instructions for Child-Led play situation (verbatim) After warm-up, give prompt to continue Child-Led play Code CDI for exactly 5 minutes Graph ECBI and CDI skill data Compare pre and post-treatment skills in CDI with mother Review ECBI graph and CDI skills progress graph with mother Schedule a follow-up session Give child praise for accomplishments in treatment Give mother a certificate of completion and congratulations and convey confidence in her skills TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = Integrity = # of Ys /Y+ N’s ___________ =__________% Length of session = _____________ 175 Appendix M Treatment Integrity Checklist for Follow-Up Session Table 26. Follow-Up Session Integrity Checklist Item Y NA N Administer ECBI to mother Greet mother and child Instruct mother to set up room with 5 sets of toys Give instructions for Child-Led play situation (verbatim) After warm-up, give prompt to continue Child-Led play Code CDI for exactly 5 minutes Graph this po and CDI skill data Review ECBI graph and CDI skills progress graph with mother Give praise to the mother and child for their hard work in CDI Ask mother if she has any questions going forward Provide mother with resources if problems come up, but convey confidence in her skills TOTALS Quality of Intervention Delivery: 1 2 3 4 Note: Checklist was created using the PCIT Protocol (Eyberg & Funderburk, 2011). Therapist Comments about Session: Integrity Checker Comments about Session: Integrity = Integrity = # of Ys /Y+ N’s ___________ =__________% Length of session = _____________ 176 Appendix N CDI Homework Sheet Table 27. CDI Homework Sheet Date Did you spend 5 Activities Problems or Questions in minutes in Special Special Time? Time today? Yes No Monday __________ Tuesday __________ Wednesday __________ Thursday __________ Friday __________ Saturday __________ Sunday __________ Note: Homework sheet is from the PCIT Protocol (Eyberg & Funderburk, 2011). 177 Appendix O DPCIS In-Session CDI Coding Sheet Child’s Name _____________________ Date ________________ TREATMENT SESSION (CIRCLE ONE) Table 28. DPICS Treatment Session PRE-TREATMENT PRE-TREATMENT PRE-TREATMENT CDI COACH #1 DPICS #1 DPICS #2 DPICS #3 (CDI Teach) CDI COACH #2 CDI COACH #3 CDI COACH #4 CDI COACH #5 CDI COACH #6 CDI COACH #7 CDI COACH #8 POST- TREATMENT DPICS (Graduation) FOLLOW-UP Table 29. DPICS Categories POSITIVE TALLY CODES TOTAL MASTERY CATEGORIES NEUTRAL TALK BEHAVIORAL 10 DESCRIPTIONS REFLECTION 10 LABELED PRAISE 10 UNLABELED PRAISE NEGATIVE TALLY CODES TOTAL MASTERY CATEGORIES QUESTION 0 COMMAND 0 NEGATIVE TALK 0 Note: Coding sheet from the PCIT Protocol (Eyberg & Funderburk, 2011). 178 Appendix P Child Behavior Observation In-Session CDI Coding Sheet Child’s Name _____________________ Date ________________ TREATMENT SESSION (CIRCLE ONE) Table 30. Child Behavior Observation Treatment Session PRE-TREATMENT PRE-TREATMENT PRE-TREATMENT CDI COACH #1 DPICS #1 DPICS #2 DPICS #3 (CDI Teach) CDI COACH #2 CDI COACH #3 CDI COACH #4 CDI COACH #5 CDI COACH #6 CDI COACH #7 CDI COACH #8 POST- TREATMENT DPICS (Graduation) FOLLOW-UP • N= Noncompliance/Refusal (does not respond to a parent’s request or command within five seconds) • A= Argumentation (argues with parent) • VA= Verbal aggression (e.g., yelling, screaming, using a harsh tone of voice, whining) • PA= Displays physical aggression (e.g., hits, kicks, bites parent and/or hits, kicks, or throws toys) • PD = Passive dysregulation passive dysregulation (e.g., dawdles, is careless with toys, is slow in completing a task) • AD = Active dysregulation (e.g., overactive or restless, short attention span, easily distracted) Naturally frustrating or difficult toys: Legos, blocks, Lincoln Logs, puzzles, Latches boards, beads, bristle blocks Table 31. Child Behavior Observation Coding 00-00:30 0:31-1:00 1:01-1:30 1:31-2:00 2:01-2:30 2:31-3:00 3:01-3:30 3:31-4:00 4:01-4:30 4:31-5:00 Additional Notes: 179 REFERENCES 180 REFERENCES Abrahamse, M. E., Junger, M., Chavannes, E., Coleman, F. G., Boer, F., & Lindauer, R. L. (2012). Parent–child interaction therapy for preschool children with disruptive behavior problems in the Netherlands. Child And Adolescent Psychiatry And Mental Health, 6, 6- 24. doi:10.1186/1753-2000-6-24 Abrahamse, M. E., Junger, M., Leijten, P. H. O., Lindeboom, R., Boer, F., & Lindauer, R. J. L. (2015). Psychometric properties of the Dutch Eyberg Child Behavior Inventory (ECBI) in a community sample and a multi-ethnic clinical sample. Journal of Psychopathology and Behavioral Assessment, 37, 679-691. doi: 10.1007/s10862-015-9482-1. Ainsworth, M. D. S., & Bowlby, J. (1991). An ethological approach to personality development. American Psychologist, 46, 333-341. http://dx.doi.org/10.1037/0003-066X.46.4.333 Akin, B. A., Testa, M. F., McDonald, T. P., Melz, H., Blase, K. A., & Barclay, A. (2014). Formative evaluation of an evidence-based intervention to reduce long-term foster care: Assessing readiness for summative evaluation. Journal of Public Child Welfare, 8, 354- 374. doi: 10.1080/15548732.2014.939250 Axberg, U., Johansso, H.J., & Broberg, A.G. (2008). Parents’ description of conduct problems in their children: A test of the Eyberg Child Behavior Inventory (ECBI) in a Swedishsample aged 3–10. Scandinavian Journal of Psychology, 49(6), 497–505. doi:10.1111/j.1467- 9450.2008.00670.x Bagner, D.M., & Eyberg. S.M. (2007). Parent-child interaction therapy for disruptive behavior in children with mental retardation: A randomized controlled trial. Journal of Clinical Child Adolescent Psychology, 36(3), 418-429. doi: 10.1080/15374410701448448 Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215. http://dx.doi.org/10.1037/0033295X.84.2.191 Barkley, R.A. (2010). Deficient emotional self-regulation is a core symptom of ADHD. Journal of ADHD and related disorders, 1, 5-37. Barkley, R.A. (2013). Defiant children: A clinician’s manual for assessment and parent training (3rd ed.). New York: Guildford Press. Barnett, M. L., Niec, L. N., Peer, S. O., Jent, J. F., Weinstein, A., Gisbert, P., & Simpson, G. (2017). Successful therapist–parent coaching: How in vivo feedback relates to parent engagement in parent–child interaction therapy. Journal of Clinical Child and Adolescent Psychology, 46, 895-902. http://dx.doi.org/10.1080/15374416.2015.1063428 181 Baumrind, D. (1991). The influence of parenting style on adolescent competence and substance use. The Journal of Early Adolescence, 11, 56-95. doi: 10.1177/0272431691111004 Bellanti, C.J., & Bierman, K.L. (2000). Disentangling the impact of low cognitive ability and inattention on social behavior and peer relationships. Journal of Clinical Child Psychology, 29, 66-75. doi: 10.1207/S15374424jccp29017 Belsky, J., Vandell, D., Burchinal, M., Clarke-Stewart, K., McCartney, K., & Owen, M. (2007). Are there long-term effects of early child care? Child Development, 78, 681-701. doi: 10.1111/j.1467-8624.2007.01021.x Benoit D. (2004). Infant-parent attachment: Definition, types, antecedents, measurement and outcome. Paediatrics & Child Health, 9, 541–545. Berkovits, M.D., O’Brien, K.A., Carter, C.G., & Eyberg, S.M. (2010). Early identification and intervention for behavior problems in primary care: A comparison of two abbreviated versions of parent-child interaction therapy. Journal of Behavior Therapy, 41, 357-387. doi: 10.1016/j.beth.2009.11.002 Bessmer, J.L. (1998). The dyadic parent-child interaction coding system II (DPICS II): Reliability and validity (Doctoral dissertation). Retrieved from Dissertation Abstractions International: Section B: The Sciences and Engineering (3961). Bjørseth A., & Wichstrøm, L. (2016). Effectiveness of parent-child interaction therapy (PCIT) in the treatment of young children’s behavior problems. A randomized controlled study. Plos One, 11(9), e0159845.10.1371/journal.pone.0159845 Blizzard, A.M., Barroso, N.E., Ramos, F.G., Graziano, P.A., & Bagner, D.M. (2017). Behavioral training in infancy: What about the parent-infant relationship? Journal of Clinical Child & Adolescent Psychology 00(00), 1-13. 10.1080/15374416.2017.1310045. Boggs, S.R, Eyberg, S., Reynolds, L.A. (1990). Concurrent validity of the Eyberg Child Behavior Inventory. Journal of Clinical Child Psychology, 19(1), 75–78. doi: 10.1207/s15374424jccp1901_9 Bowlby, J. (1969). Attachment. Attachment and loss: Vol. 1. Loss. New York: Basic Books Brestan, E. V., Eyberg, S. M., Boggs, S.R., & Algina, J. (1997). Parent–child interaction therapy: Parents' perceptions of untreated siblings. Child & Family Behavior Therapy, 19, 13- 28.doi: 10.1300/J019v19n03_02 Budd, K. S., Hella, B., Bae, H., Meyerson, D. A., & Watkin, S. C. (2011). Delivering Paren Child Interaction Therapy in an urban community clinic. Cognitive And Behavioral Practice, 18(4), 502-514. doi:10.1016/j.cbpra.2010.12.002 182 Bufferd, S.J., Dougherty, L.R., Carlson, G.A., Rose, S., & Klein, D.N. (2012). Psychiatric disorders in preschoolers: Continuity from ages 3 to 6. American Journal of Psychiatry, 169, 1157-1165. doi: 10.1176/appi,ajp.2012.12020268 Butler, A.M. (2011). Cross-racial measurement equivalence of the Eyberg Child Behavior Inventory factors among low-income young African American and Non-Latino White children. Assessment, 20(4), 484–495. doi: 10.1177/1073191111431341 Campbell, S.B., Shaw, D.S., & Gilliom, M. (2000). Early externalizing behavior problems: Toddlers and preschoolers at risk for later maladjustment. Development and Psychopathology, 12, 467-488. doi: 10.1017/S0954579400003114 Chacko, A., Fabiano, G.A., Doctorproff, G.L., & Forston, B. (2018). Engaging fathers in effective parenting for preschool children using shared book reading: A randomized controlled trial. Journal of Clinical Child & Adolescent Psychology, 47, 79-93. doi: 10.1080/15374416.2016.1266648 Chaffin, M., Silovsky, J.F., Funderburk, B., Valle, L., Brestan, E.V…..& Bonner, B.L. (2004). Parent-child interaction therapy with physically abusive parents: Efficacy for reducing future abuse reports. Journal Of Consulting And Clinical Psychology, 72, 500-510. doi: 10.1037/0022-006X.72.3.500 Chorpita, B. F., Daleiden, E. L., Ebesutani, C., Young, J., Becker, K. D., Nakamura, B. J., . . Starace, N. (2011). Evidence based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18(2), 154-172. http://dx.doi.org/10.1111/j.1468-2850.2011.01247.x Cohen, M. (1998). The monetary value of saving a high-risk youth. Journal of Quantitative Criminology, 14, 5-33. doi: 10.1023/A:1023092324459 Cohen, M. A., & Piquero, A. R. (2009). New evidence on the monetary value of saving a high risk youth. Journal of Quantitative Criminology, 25, 25-49. http://dx.doi.org/10.1007/s10940-008-9057-3 Cole, P., Teti, L., Zahn-Waxler, C. (2003). Mutual emotion regulation and the stability of conduct problems between preschool and early school age. Development and Psychopathology, 15, 1-18. doi: 10.1017/S0954579403000014 Colvin, A., Eyberg, S.M., & Adams, C.D. (1999). Restandardization of the Eyberg Child Behavior Inventory. Gainesville, FL: University of Florida, Child Study Laboratory. Comer, J. S., Chow, C., Chan, P. T., Cooper-Vince, C., & Wilson, L. A. (2012). Psychosocial treatment efficacy for disruptive behavior problems in very young children: a meta analytic examination. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 26–36. doi:10.1016/j.jaac.2012.10.001 183 Cooley, M.E., Veldorale-Griffin, A., Petren, R.E., & Mullis, A.K. (2014). Parent-child interaction therapy: A meta-analysis of child behavior outcomes and parent stress. Journal of Family Social Work, 17(3), 191-208. doi: 10.1080/10522158.2014.888696 Cooley-Strickland, M., Quille, T.J., Griffin, R.S., Stuart, E.A., Bradshaw, C.P., & Furr-Holden, D. (2009). Community violence and youth: Affect, behavior, substance use, and academics. Clinical Child and Family Psychology Review, 12, 127-156. doi: 10.1107/s10567009-0051-6 Cotter, A. (2016). Psychometric properties of the dyadic parent-child interaction coding system (DPICS): Investigating updated versions across diagnostic subgroups. Unpublished master’s thesis, Auburn University, AL Cummings, E. M., & Davies, P. T. (2010). The Guilford series on social and emotional development. Marital conflict and children: An emotional security perspective. New York, NY, US: Guilford Press Danko, C. M., Garbacz, L. L., & Budd, K. S. (2016). Outcomes of parent–child interaction therapy in an urban community clinic: A comparison of treatment completers and dropouts. Children and Youth Services Review, 60, 42-51. http://dx.doi.org/10.1016/j.childyouth.2015.11.007 Darling, N., & Steinberg, L. (1993). Parenting style as context: An integrative model. Psychological Bulletin, 113, 487-496. http://dx.doi.org/10.1037/00332909.113.3.487 Davidov, M., & Grusec, J.E. (2006). Untangling the links of parental responsiveness to distress and warmth to child outcomes. Child Development, 77, 44-58. doi: 10.1111/j.1467-8624.2006.00855.x Davidson, T. M., & Cardemil, E. V. (2009). Parent-child communication and parental involvement in Latino adolescents. The Journal of Early Adolescence, 29, 99-121. http://dx.doi.org/10.1177/0272431608324480 DeGarmo, D.S., & Forgatch, M.S. (2007). Efficacy of parent training for stepfathers: From playful spectator and polite stranger to effective stepfathering. Parenting: Science and Practice, 7, 1–25. https://doi.org/10.1080/15295190701665631 DeGarmo, D.S., Patterson, G.R., & Forgatch, M.S. (2004). How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science, 5, 73–89. https://doi.org/10.1023/B:PREV.0000023078.30191.e0 Deković, M., Asscher, J.J., Hermanns, J., Reitz, E., Prinzie, P., & van den Akker, A. (2010). Tracing changes in families who participated in the home-start parenting program: 184 Parental sense of competence as mechanism of change. Prevention Science, 11, 263- 274. https://doi.org/10.1007/s11121-009-0166-5 Dishion, T., Forgatch, M., Chamberlain, P., & Pelham, W.E. (2016). The Oregon model of behavior family therapy: From intervention design to promoting large-scale system change. Behavior Therapy, 47, 812-837. doi: 10.1016/j.beth.2016.02.002 Egger, H.L., & Angold, A. (2006). Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry, 47, 313-337. doi: 10.1111/j.1469-7610.2006.01618.x Eisenstadt, T.H., Eyberg, S., McNeil, C.B., Newcomb, K., & Funderburk, B. (1993). Parent child interaction therapy with behavior problem children: Relative effectiveness of two stages and overall treatment outcome. Journal of Clinical Child Psychology, 22, 42-51. doi: 10.1207/s15374424jccp2201_4 Etkin, R. G., Koss, K. J., Cummings, E. M., & Davies, P. T. (2014). The differential impact of parental warmth on externalizing problems among triangulated adolescents. The Journal of Genetic Psychology, 175, 118–133. doi:10.1080/00221325.2013.813437 Evans, S., Sibley, M., Serpell, Z. (2008). Changes in caregiver strain over time in young adolescents with ADHD: The role of oppositional and delinquent behavior. Journal of Attention Disorders, 12, 516-524. doi: 10.1177/1087054708322987 Eyberg, S.M. (1988). Parent-child interaction therapy. Child & Family Behavior Therapy, 10, 33-46. doi:10.1300/J019v10n01_04 Eyberg, S.M., & Funderburk, B.W. (2011). Parent-child interaction therapy protocol Gainesville, FL: PCIT International. Eyberg, S.M., & Matarazzo, R.G. (1980). Training parents as therapists: A comparison between individual parent–child interaction training and parent group didactic training. Journal of Clinical Psychology, 36(2), 492-499. doi: 10. 1002.jclp.6.120360218 Eyberg, S.M., Nelson, M.M., & Boggs, S.R. (2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child Adolescent Psychology, 37, 215-237. doi: 10.1080/15374410701820117 Eyberg, S.M., Nelson, M.M., Ginn, N.C., Bhuiyan, N., & Boggs, S.R. (2013). Dyadic parent child interaction coding system (DPICS) comprehensive manual for research and training 4th edition. Gainesville, FL: PCIT International Eyberg, S.M., & Pincus, D.B. (1999). Eyberg Child Behavior Inventory and Sutter-Eyberg Behavior Inventory-Revised: Professional Manual. Odessa, FL: Psychological Assessment Resources. 185 Fearon, R.P., Bakermans-Kranenburg, M.J., van IJzendoorn, M.H., Lapsely, A., & Roisman, G.I. (2010). The significance of insecure attachment and disorganization in the development of children’s externalizing behavior. A meta-analytic study. Child Development, 8, 435- 456. doi: 10.1111/j.1467-8624.2009.01405.x Forgatch, M. S., & DeGarmo, D. S. (1999). Parenting through change: An effective prevention program for single mothers. Journal of Consulting and Clinical Psychology, 67, 711-724. http://dx.doi.org/10.1037/0022-006X.67.5.711 Forgatch, M.S., DeGarmo, D.S., & Beldavs, Z. (2005). An efficacious theory-based intervention for stepfamilies. Behavior Therapy, 36, 357–365. https://doi.org/10.1016/S00057894(05)80117-0 Forgatch, M.S., & Kjobli, J. (2016). Parent management training-Oregon model: Adapting intervention with rigorous research. Family Process Institute, 55, 500-513. doi: 10.1111/famp.12224 Forgatch, M. S., & Patterson, G. R. (2010). Parent management training—Oregon model: An intervention for antisocial behavior in children and adolescents. In J. R. Weisz & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 159- 177). New York, NY, US: The Guilford Press Forgatch, M.S., Patterson, G.R., DeGarmo, D.S., & Beldavs, Z.G. (2009). Testing the Oregon delinquency model with nine-year follow-up of the Oregon divorce study. Development and Psychopathology, 21, 637–660. https://doi.org/10.1017/S0954579409000340 Foster, E. M., & Jones, D. E. (2005). The high costs of aggression: Public expenditures resulting from conduct disorder. American Journal of Public Health, 95, 1767–1772. doi:10.2105/AJPH.2004.061424 Fox, L., Dunlap, G., Hemmeter, M. L., Joseph, G. E. and Strain, P. S. 2003. The teaching pyramid: A model for supporting social competence and preventing challenging behavior in young children. Young Children, 58, 48–52 Funderburk, B.W., Eyberg, S.M., Rich, B.A., & Behar, L. (2003). Further psychometric evaluation of the Eyberg and Behar rating scales for parents and teachers of preschoolers. Early Education and Development, 14, 67-81. doi: 10.1207/s15566935eed1401_5 Gast, D. L. & Ledford, J.R. (2014). Single case research methodology. (3rd ed.). New York: NY: Routledge. Ginn, N., Clionsky, L., Eyberg, S., Warner-Metzger, C., & Abner, J. (2017). Child-directed interaction training for young children with autism spectrum disorders: Parent and child 186 outcomes. Journal of Clinical Child & Adolescent Psychology, 47, 101-109. doi: 10.1080/15374416.2015.1015135 Goldfine, M. Wagner, S.M., Branstetter, S., & Mcneil, C. (2008). Parent-child interaction therapy: An examination of cost-effectiveness. Journal of Early and Intensive Behavior Intervention. 5(1). doi: 10.1037/h0100414 Gonzales, N. A., Dumka, L. E., Millsap, R. E., Gottschall, A., McClain, D. B., Wong, J. J., . . . Kim, S. Y. (2012). Randomized trial of a broad preventive intervention for Mexican American adolescents. Journal of Consulting and Clinical Psychology, 80, 1-16. http://dx.doi.org/10.1037/a0026063 Granic, I., & Patterson, G. R. (2006). Toward a comprehensive model of antisocial development: A dynamic systems approach. Psychological Review, 113, 101-131. http://dx.doi.org/10.1037/0033-295X.113.1.101 Harvey, E.A., Metcalfe, L.A., Herbert, S.D., & Fanton, J.H. (2011). The role of family experiences and ADHD in early development of oppositional defiant disorder. Journal of Consulting and Clinical Psychology, 79, 784-795. doi: 10.1037/a0025672 Harwood, M.D., O’Brien, K.A., & Eyberg, S.M. (2009). Mental health services for preschool children in primary care: A survey of maternal attitudes and beliefs. Journal of Pediatric Psychology, 34, 760-780. doi: 10.1093/jpepsy/jsn128 Henderson, A. T., & Mapp, K. L. (2002). A new wave of evidence: The impact of school, family, and community connections on student achievement. Austin, TX: Southwest Educational Development Laboratory Hess, R. S., Short, R. J., & Hazel, C. E. (2012). School-based practice in action series. Comprehensive children's mental health services in schools and communities: A publichealth problem-solving model. New York, NY, US: Routledge/Taylor & Francis Group. Hong, Y.R., & Park, J.S. (2012). Impact of attachment, temperament, and parenting on human development. Korean Journal of Pediatrics, 55, 449-454. doi: 10.3345/kjp.2012.55.12.449 Hood K., & Eyberg SM. (2003) Outcomes of parent-child interaction therapy: Mothers’ reports of maintenance three to six years after treatment. Journal of Clinical Child Adolescent Psychology, 32(3), 419-429. doi: 10.1207/S15374424JCCP3203_10 Horner, R.H., Sugai, G., Todd, A.W., & Lewis-Palmer, T. (2005). School-wide positive behavior support. In L. Bambara & L. Kern (Eds.), Individualized supports for students with problem behaviors: Designing positive behavior plans (pp. 359-390). New York:Guildford Press. 187 Hukkelberg, S. S., & Ogden, T. (2013). Working alliance and treatment fidelity as predictors of externalizing problem behaviors in parent management training. Journal of Consulting and Clinical Psychology, 81(6), 1010-1020. http://dx.doi.org/10.1037/a0033825 Hutchings, J., Gardner, F., Bywater, T., Daley, D., Whitaker, C., Jones, K., … Edwards, R. T. (2007). Parenting intervention in Sure Start services for children at risk of developing conduct disorder: Pragmatic randomised controlled trial. BMJ (Clinical research ed.), 334(7595), 678. doi:10.1136/bmj.39126.620799.55 Insel, T.R. (2008). Assessing the economic costs of serious mental illness. American Journal of Psychiatry, 165, 663-665. doi: 10.1176/appi.ajp.2008.08030366 Kaminski, J.W., & Claussen, A.H. (2017). Evidence base update for psychosocial treatments for disruptive behaviors in children. Journal of Clinical Child Adolescent Psychology, 46, 477-499. doi: 10.1080/15374416.2017.1310044 Kazdin, A.E. (2011). Single-case research designs. (2nd ed.). New York, NY: Oxford University Press. Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K.R., & Walters, E.E. (2005). Lifetime prevalence and age-of-onset distribution of DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry 62(6), 593-602. doi: 10.1001/archpsyc.62.6.593 Knoche, L. L., Sheridan, S. M., Edwards, C. P., & Osborn, A. Q. (2010). Implementation of a relationship-based school readiness intervention: A multidimensional approach to fidelity measurement for early childhood. Early Childhood Research Quarterly, 25, 299- 313. https://doi.org/10.1016/j.ecresq.2009.05.003 Kratochwill, T.R., Hitchcock, J.H., & Horner, R.H. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26-38. doi: 10.1177/07419322512452794 Kroneman, L.M., Hipwell, A.E., Loeber, R., Koot, H.M., & Pardini, D.A. (2011). Contextual risk factors as predictors of disruptive behavior disorder trajectories in girls: The moderating effect of callous-unemotional features. Journal of Child Psychology and Psychiatry, 52, 167-175. doi: 10.1111/j.1469-7610.2010.02300.x. Lanier, P., Kohl, P.L., Benz, J., Swinger, D., & Drake, B. (2014). Preventing maltreatment with a community-based implementation of parent-child interaction therapy. Journal of Child Family Studies, 23, 449-460. doi: 10.1007/s10826-012-9708-8 Larsson, B., Fossum, S., Clifford, G., Drugli, M.B., Handegard, B.H., & Morch, W.T. (2009). Treatment of oppositional defiant and conduct problems in young Norwegian children: Results of a randomized controlled trial. European Child Adolescent Psychiatry, 18, 42- 52. doi: 10.1007/s00787-008-0702-z 188 Latimer, K.K., Wilson, P.P., Kemp, J.J., Thompson, L.L, Sim, F.F., Gillberg, C.C., Puckering, C., & Minnis, H. (2012). Disruptive behaviour disorders: A systematic review of environmental antenatal and early years risk factors. Child: Care, Health and Development, 38(5), 611-628. doi: 10.1111/j.1365-2214.2012.01366.x Lavigne, J.V., Gouze, K.R., Hopkins, J., Bryant, F.B., & LeBailly, S.A. (2012). A multi-domain model of risk factors for ODD symptoms in a community sample of 4-year-olds. Journal of Abnormal Child Psychology, 40, 741-757. doi: 10.1007/s10802-011-9603-6 Lezin, N., Rolleri, I.A., Bean, S., & Taylor, J. (2004). Parent-child connectedness: Implications for research, interventions, and positive impacts on adolescent health. Santa Cruz, CA: ETR Associates Lieneman, C.C., Brabson, L.A., Highlander, A.,Wallace, N.M., & McNeil, C.B. (2017). Parent child interaction therapy: Current perspectives. Psychology Research and Behavior Management, 10, 239-256. doi: 10.2147/PRBM.S91200 Loukas, A., Suizzo, M., & Prelow, H. (2007). Examining resource and protective factors in the adjustment of Latino youth in low income families: What role does maternal acculturation play?. Journal of Youth and Adolescence, 36, 489-501. doi: 10.1007/s10964-006-9124-8 Lyon, A. R., & Budd, K.S. (2010). A community mental health implementation of Parent-Child Interaction Therapy (PCIT). Journal of Child and Family Studies, 19(5), 654-668. doi: 10.1007/s10826-010-9353-z Lyons-Ruth, K., Dutra, L., Schuder, M. R., & Bianchi, I. (2006). From infant attachment disorganization to adult dissociation: Relational adaptations or traumatic experiences?. The Psychiatric Clinics of North America, 29, 63–68. doi:10.1016/j.psc.2005.10.011 Mash, E.J. & Barkley, R.A. (2014). Child psychopathology, third edition. New York: Guildford Publications. Masse, J. J., McNeil, C. B., Wagner, S. M., & Chorney, D. B. (2007). Parent-child interaction therapy and high functioning autism: A conceptual overview. Journal of Early and Intensive Behavior Intervention, 4(4), 714-735. http://dx.doi.org/10.1037/h0100402 Maughan, D. R., Christiansen, E., Jenson, W. R., Olympia, D., & Clark, E. (2005). Behavioral parent training as a treatment for externalizing behaviors and disruptive behavior disorders: A meta-analysis. School Psychology Review, 34, 267-286 McCabe, K.M., Hough, R., Wood, P.A., Yeh, M. (2001). Childhood and adolescent onset conduct disorder: A test of the developmental taxonomy. Journal of Abnormal Child Psychology, 29, 305-316 189 McCollum, J. Gooler, F., Appl, D., & Yates T. (2001). PIWI: Enhancing parent-child interaction as a foundation for early intervention. Infants and Young Children, 14, 34-45. doi: 10.1097/00001163-200114010-00007 McCollum, J., & Yates, T. (1994). Dyad as focus, triad as means: A family-centered approach to supporting parent-child interactions. Infants and Young Children, 6, 54-63. doi: 10.1097/00001163-199404000-00008 McDonald, D., Dodson, M.C., Rosenfield, D., & Jouriles, E.N. (2011). Effects of a parenting intervention on features of psychopathy in children. Journal of Abnormal Child Psychology, 39, 1013-1023. doi: 10.1007/s10802-011-9512-9 McIntyre, L.L. (2008). Adapting Webster-Stratton’s incredible years parent training for children with developmental delay: Findings from a treatment group only study. Journal of Intellectual Disability Research, 52, 1176-1192. doi: 10.1111/j.1365-2788.2008.01108.x. McNeil, C.B. & Hembree-Kigin, T.L. (2011). Parent-child interaction therapy 2nd Edition. New York, NY: Springer. Menting, A.T., Orobio de Castro, B., & Matthys, W. (2013). Effectiveness of the Incredible Years parent training to modify disruptive and prosocial child behavior: A meta-analytic review. Clinical Psychology Review, 33, 901-913. doi: 10.1016/j.cpr.2013.07.006 Muratori, P., Lochman, J.E., Lai, E., Milone, A., Nocentini, A., Pisano, S., Righini, E., & Masi, G. (2016). Which dimension of parenting predicts the change of callous unemotional traits in children with disruptive behavior disorder? Comprehensive Psychiatry, 69, 202- 210. doi: 10.1016/j.comppsych.2016.06.002 National Academies of Sciences, Engineering, and Medicine (2016). Parenting matters: Supporting parents of children ages 0-8. Breiner, H., Ford, M., & Gadsden, V.L.(Ed.). Washington (DC): National Academies Press. Retried from” https://www.ncbi.nlm.nih.gov/books/NBK402020/ Nieter, L., Thornberry, T. R., & Brestan-Knight, E. (2013). The effectiveness of group parent child interaction therapy with community families. Journal of Child and Family Studies, 22(4), 490–501. doi:10.1007/s10826-012-9601-5 Nixon, R.D., Sweeney, L., Erickson, D.B., & Touyz, S.W. (2003) Parent-child interaction therapy: a comparison of standard and abbreviated treatments for oppositional defiant preschoolers. Journal of Consulting & Clinical Psychology, 71(2), 251-260. doi: 10.1037/022-006X.71.2.251 Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of triple-P positive parenting program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical Child and Family Psychology Review, 11, 114 -144. doi: 10.1007/s10567-008-0033-0 190 N’zi, A.M., Stevens, M.L., & Eyberg, S.M. (2016). Child directed interaction training for young children in kinship care: A pilot study. Child Abuse and Neglect, 55, 81-91. doi: 10.1016/j.chiabu.2016.03.001 Ogden, T., & Hagen, K.A. (2008). Treatment effectiveness of parent management training in Norway: A randomized controlled trial of children with conduct problems. Journal of Consulting and Clinical Psychology, 76, 607-621. doi: 10.1037/0022-006X.76.4.607 Ogundele M. O. (2018). Behavioural and emotional disorders in childhood: A brief overview for pediatricians. World Journal of Clinical Pediatrics, 7, 9–26. doi:10.5409/wjcp.v7.i1.9 Parker, R.I., Vannest, K.J., & Davis, J.L. (2011). Effect size in single-case research: A review of nine overlap techniques. Behavior Modification, 35, 303-322. doi: 10.1177/0145445511399147 Pasaluch, D.S., Dadds, M.R., Hawes, D.J., & Brennan, J. (2011). Do callous-unemotional traits moderate the relative importance of parental coercion versus warmth in child conduct problems? An observational study. Journal of Child Psychology and Psychiatry, 52, 1308-1315. doi: 10.1111/j.1469-7610.2011.02435.x. Patterson, G. R. (2002). Etiology and treatment of child and adolescent antisocial behavior. The Behavior Analyst Today, 3, 133-144. http://dx.doi.org/10.1037/h0099971 Perepletchikova, F., & Kazdin, A. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Journal of Clinical Science, 12, 365-383. doi: 10.1093/clipsy.bpi045 Ros, R., Graziano, P.A., & Hart, K.C. (2017). Parental homework completion and treatment knowledge during group parent-child interaction therapy. Journal of Early Intervention, 39, 299-320. doi: 10.1177/1053815117718491 Ross, S.G., & Begeny, J.C. (2014). Single-case effect size calculation: Comparing regression and non-parametric approaches across previously published reading intervention data sets. Journal of School Psychology, 52(4), 419-431. doi: 10.1016/j.jsp.2014.06.003 Rueger, S. Y., Katz, R. L., Risser, H. J., & Lovejoy, M. C. (2011). Relations between parental affect and parenting behaviors: A meta-analytic review. Parenting, 11, 1- 33. https://doi.org/10.1080/15295192.2011.539503 Sanders, M.R. (2003). Triple P – positive parenting program: A population approach to promoting competent parenting. Australian e-Journal for the Advancement of Mental Health, 2, 127-143. doi: 10.5172/jamh.2.3.127 191 Sanders, M.R. (2008). Triple P-positive parenting program as a public health approach to strengthening parenting. Journal of Family Psychology, 22, 506-517. doi: 10.1037/08933200.22.3.506 Sanders, M.R., Kirby, J.N., Tellegen, C.L., & Day, J.J. (2014). The triple P-positive parenting program: A systematic review and meta-analysis of a multi-level system of parenting support. Clinical Psychology Review, 34(4), 337-357. doi: 10. 1016/j.cpr.2014.04.003 Schuhmann, E.M., Foote, R.C., Eyberg, S.M., Boggs, S.R., & Algina, J. (1998) Efficacy of parent-child interaction therapy: Interim report of a randomized trial with short-term maintenance. Journal of Clinical Child Psychology, 27(1), 34-45. doi: 10.1207/s15374424jccp2701_4 Shadish, W., Hedges, L., Horner, R., & Odom, S. (2015). The role of between-case effect size in conducting, interpreting, and summarizing single-case design research. National Center for Education Research, 1-109 Shanley, J.R., & Niec, L.N. (2010). Coaching parents to change: The impact of in vivo feedback on parent’s acquisition of skills. Journal of Clinical Child & Adolescent Psychology, 39, 282–287. doi: 10.1080/15374410903532627 Sheridan, S. M., Knoche, L. L., Edwards, C. P., Kupzyk, K. A., Clarke, B. L., & Kim, E. M. (2014). Efficacy of the Getting Ready Intervention and the role of parental depression. Early Education and Development, 25, 746–769. doi:10.1080/10409289.2014.862146 Sheridan, S.M., Marvin, C., Knoche, L., & Edwards, C.P. (2008). Getting Ready: Promoting school readiness through a relationship-based partnership model. In Innocenti, M., Guest Editor, Early Childhood Services, Special Issue on Young Children’s Relationships, 2(3), 149-172 Smith, J. D., Dishion, T. J., Shaw, D. S., Wilson, M. N., Winter, C. C., & Patterson, G. R. (2014). Coercive family process and early-onset conduct problems from age 2 to school entry. Development and Psychopathology, 26, 917–932. doi:10.1017/S0954579414000169 Stormshak, E.A., Bierman, K.L., McMahon, R.J., & Lengua, L.J. (2000). Parenting practices and child disruptive behavior problems in early elementary school. Journal of Clinical Child Psychology, 29, 17-29. doi: 10.1207/S15374424jccp29013 Thijssen, J., Vink, G., Muris, P., & de Ruiter, C. (2017). The effectiveness of parent management training—Oregon model in clinically referred children with externalizing behavior problems in the Netherlands. Child Psychiatry and Human Development, 48, 136-150. doi: 10.1007/s10578-016-0660-5 192 Thomas, R., & Zimmer-Gembeck, M.J. (2007). Behavioral outcomes of parent-child interaction therapy and triple P—Positive parenting program: A review and meta-analysis. Journal of Abnormal Child Psychology, 35(3), 475-495. doi: 10.1007/s10802-007-9104-9 Tremblay, R. E. (2010). Developmental origins of disruptive behaviour problems: The “original sin” hypothesis, epigenetics and their consequences for prevention. Journal of Child Psychology and Psychiatry, 51, 341-367. doi.org/10.1111/j.14697610.2010.02211.x van Dijk, A., Poorthuis, A., & Malti, T. (2017). Psychological processes in young bullies versus bully-victims. Aggressive Behavior, 43, 430-439. https://doi.org/10.1002/ab.21701 Vitaro, F., Brendgen, M., Larose, S., & Trembaly, R. E. (2005). Kindergarten disruptive behaviors, protective factors, and educational achievement by early adulthood. Journal of Educational Psychology, 97, 617-629. http://dx.doi.org/10.1037/0022-0663.97.4.617 Wakschlag, L.S., Estabrook, R., Petitclerc, A., Henry, D., Burns, J.L., Perlman, S.B., Voss, J.L., Pine, D.S., Leibenluft, E., & Briggs-Gowan, M.L. (2015). Clinical implications of a dimensional approach: The normal: Abnormal spectrum of early irritability. Journal of the American Academy of Child and Adolescent Psychiatry, 54, 626-634. doi: 10.1016/j.jaac.2015.05.016 Waller, R., Gardner, F., Hyde, L.W., Shaw, D.S., Dishion ,T.J., & Wilson, M.N. (2012). Do harsh and positive parenting predict parent reports of deceitful-callous behavior in early childhood? Journal of Child Psychology and Psychiatry, 53, 946–953. doi: 10.1007/s10802-014-9965-7 Walton, A., & Flouri, E. (2010). Contextual risk, maternal parenting and adolescent externalizing behaviour problems: The role of emotion regulation. Child Care Health Development, 36, 275-284. doi: 10.1111/j.1365-2214/2009.01065.x Ward, M.A., Theule, J. & Cheung, K. (2016). Parent-child interaction therapy for chil disruptive behaviour disorders: A meta-analysis. Child & Youth Care Forum, 45(5), 675-690. doi: 10.1007/s10566-016-9350-5 Waters, E., & Cummings, M. (2000). A secure base from which to explore close relationships. Child Development, 71 164-172. doi: 10.1111/1467-8624.00130 Webster-Stratton, C. (2005). The Incredible Years: A training series for the prevention and treatment of conduct problems in young children. In E. D. Hibbs & P. S. Jensen (Eds.), Psychosocial treatments for child and adolescent disorders: Empirically based strategies for clinical practice (pp. 507-555). Washington, DC, US: American Psychological Association. 193 Webster-Stratton, C. H., & Reid, M. J. (2010). The incredible years program for children from infancy to pre-adolescence: Prevention and treatment of behavior problems. In R. C. Murrihy, A. D. Kidman, & T. H. Ollendick (Eds.), Clinical Handbook of Assessing and Treating Conduct Problems in Youth (pp. 117-138). New York, NY, US: Springer Science + Business Media. http://dx.doi.org/10.1007/978-1-4419-6297-3_5 Webster-Stratton, C., Reid, J., & Hammond, M. (2001). Social skills and problem-solving training for children with early-onset conduct problems: Who benefits? Journal of Child Psychology and Psychiatry, 42, 943-952 Webster-Stratton, C., Rinaldi, J., & Jamila, M. R. (2011). Long-term outcomes of Incredible Years parenting program: Predictors of adolescent adjustment. Child and Adolescent Mental Health, 16, 38–46. doi:10.1111/j.1475-3588.2010.00576.x 194