USING A RESEARCH-COMMUNITY PARTNERSHIP AND ROGERS’ DIFFUSION OF INNOVATIONS THEORY TO UNDERSTAND THE FIT OF AN EVIDENCE-BASED, PARENT-MEDIATED INTERVENTION FOR ASD IN THE MEDICAID SYSTEM By Katherine Pickard A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology - Doctor of Philosophy 2017 ABSTRACT USING A RESEARCH-COMMUNITY PARTNERSHIP AND ROGERS’ DIFFUSION OF INNOVATIONS THEORY TO UNDERSTAND THE FIT OF AN EVIDENCE-BASED, PARENT-MEDIATED INTERVENTION FOR ASD IN THE MEDICAID SYSTEM By Katherine Pickard Service use disparities within the autism spectrum disorder (ASD) field are particularly relevant for parent-mediated interventions, and may suggest a lack of fit between these interventions and the needs of underserved community settings. The present project attempted to understand and address these disparities through two studies. In the first study, Rogers’ Diffusion of Innovations theory was used to guide a research-community partnership aimed at understanding the perceived compatibility, complexity, and relative advantage, as well as the structural barriers to using an evidence-based, parent-mediated intervention for ASD (i.e. Project ImPACT) within a Medicaid system. Three focus groups were conducted with 16 Medicaideligible parents, and 3 focus groups were conducted with 16 ASD providers working within a Medicaid system. Across all groups, parents and providers reported general interest in using Project ImPACT but emphasized the need to: 1) reduce the complexity of written materials; 2) better support families in practicing the intervention within their pre-existing routines; 3) allow for a more flexible program delivery; 4) involve the extended family; and 5) ensure a strong parent-therapist alliance. These findings were used to propose an adapted delivery model for Project ImPACT in collaboration with the same community partner. After doing so, the primary aim of the second study was to determine whether these adaptations influenced parents’ and providers’ perception of program attributes specific to Roger’s theory and their intent to use the program. Eighty-two parents and eighty providers from a variety of socioeconomic backgrounds participated in the second study. Participants were randomized so that they watched a presentation of either the original or adapted Project ImPACT program. After watching the presentation, participants rated the following: 1) demographic information; 2) Project ImPACT attributes (i.e. compatibility, complexity, and relative advantage); 3) perceived structural barriers; and 4) intent to use the program. Two-way, between-subjects ANOVAs were run to examine the impact of program type (original vs. adapted), participant SES (i.e. Medicaid vs. non-Medicaid), and their interaction on perceptions of program attributes, perceived structural barriers, and intent to use the program. Results showed a significant main effect of SES and a marginal main effect of program type on parents’ report of perceived structural barriers to using Project ImPACT. There was no main effect of program type or SES on parents’ ratings of program attributes or intent to use. However, there was a significant interaction between SES and program type such that Medicaid-eligible parents rated the adapted program more favorably. Although Medicaid-eligible parents also tended to report greater intent to use the adapted program, this interaction was marginally significant. For providers, there was no main effect of program type, caseload SES, or their interaction on ratings of program attributes or intent to use. However, qualitative follow-up for both parents and providers demonstrated important differences in how participants discussed the strengths and weaknesses of the original versus adapted program. Results from the present study suggest that research-community partnerships may be beneficial in increasing the fit and relevance of parent-mediated interventions for ASD within low SES settings. Importantly, Roger’s Diffusion of Innovations theory may be a framework that can be used by researchers to guide and test adaptations to parent-mediated interventions for use within a variety of community settings. This dissertation is dedicated to my parents, Michael and Sally Pickard, my brother Christopher Pickard, and my fiancée, William Vann. I feel immensely grateful for their endless love and support. It is also dedicated to the motivated, supportive and thoughtful families and providers who partnered and participated in this research project. Their insight and guidance made this project an incredible and rewarding experience. iv ACKNOWLEDGEMENTS I would like to thank my mentor, Brooke Ingersoll, for her guidance, support, and encouragement during my dissertation project and throughout my time in graduate school. I would also like to thank my colleagues, particularly Nikki Bonter, Natalie Berger, Allison Wainer, and Karis Casagrande for their support. Finally, I am incredibly grateful to have had the opportunity to work with such motivated, supportive and thoughtful parents and providers of children with ASD. v TABLE OF CONTENTS LIST OF TABLES…………………………………………………………………………… viii LIST OF FIGURES…………………………………………………………………..…….… ix INTRODUCTION……………………………………………………………………………. 1 Background and Significance………………………………………………………….. 1 Utility of Rogers’ Diffusion of Innovations Theory………………………...………… 3 STUDY 1 ……………………………….………………………………………………...….. Introduction……………………………………………………………………………. Study Goals and Overview…………...……………….………..…………….…. Method…………………….…………………….…………………………………..… Community Partners and Procedures…...………………………………………. Parent-Mediated Intervention of Interest….…………………..……………….. Experimental Design and Study Procedure…….………………...…………….. Participants………………………………………..……………..……..… Measures…………………..….…….…………………………………….. Sociodemographic information…………….……………………… Focus groups…………..…………………………………………... Data Analysis…………………………………………………………….. Results……………………………………………………………………………….… Perceptions of Project ImPACT Compatibility with Beliefs and Lifestyle…….. Perceptions of Project ImPACT Complexity………...…………………………. Relative Advantage of Project ImPACT………………………………………... Perceived Structural Barriers to Project ImPACT Participation……………..…. Suggestions to Improve Project ImPACT Fit….…………………………..……. Discussion….………………….…………………………………………….………… Implications……………….………..…………………………………………… Limitations…………….………………………………………............................ Future Directions…………………………………………….………………….. Conclusion..……………………………………….……………….….……..….. 7 7 8 9 9 10 11 11 15 15 15 16 17 18 20 21 21 22 26 27 29 30 31 STUDY 2…………………….……………………………………………………….…….… Introduction……………………………………………………………………………. Parent-Mediated Intervention of Interest…...…….………………….................. Summary of Adaptations to Project ImPACT…………………………………... Testing the Impact of Adaptations Using Theory of Planned Behavior……..…. Method…………………………….…………………………………………………… Participating Community Agencies and Recruitment……...……………...……. 33 33 33 34 39 42 42 vi Experimental Design and Study Procedure…….……..…………………..…….. Study 2 Participants………………………………………………….....… Video Stimuli…….……………………………………………...……..… Measures………………………………………………………………..… Parent demographics………………………………………………. Child demographics………………………………………………... Parenting stress…………………………………………………….. Parent depression…………………………………………………... Perceived structural barriers………………………………………. Provider demographics…………………………………………….. Provider clientele…………………………………………………... Implementation leadership………………………………………… Provider general attitudes………………………………………….. Perceived workplace barriers……………………………………… Project ImPACT attributes………………………………………… Intent to use………………………………………………………... Open-ended feedback……………………………………………… Data Analysis…….…………………………………………………………...… Results……….……………………………….………………….…………………….. Parent Ratings of Program Attributes..……………………...……………….…. Parent Ratings of Perceived Structural Barriers……….……………..…………. Parent Ratings of Intent to Use…………………….…………….……………… Parent Qualitative Feedback……………………………………..……………… Provider Ratings of Program Attributes….………..…..………….…………….. Provider Ratings of Intent to Use……….………………………...…………….. Provider Qualitative Feedback………………………………………..………… Discussion…………………………………………………………….……………….. Implications……………………………………………………………..………. Limitations……………………………………………………………….……… Future Directions………………………………………………………..………. 42 43 46 46 46 47 47 47 48 48 48 48 49 49 49 51 51 53 54 54 55 56 59 64 65 67 73 74 78 80 CONCLUDING REMARKS………………………………………………………………… 82 APPENDICES……………………………………..………...………………………………. Appendix A Script for Study 1 Video of Original Program.……..……………...……. Appendix B Focus Group Interview: Parents…………………….………..…….……. Appendix C Focus Group Interview: Providers.….….………………………......……. Appendix D Example of Coded Transcript Excerpt………………………………..…. Appendix E Example of Content Analytic Table…….……..………………………… Appendix F Script for Study 2 Video of Original Program …….….….……………… Appendix G Script for Study 2 Video of Adapted Program …………………….......... 83 84 88 90 91 92 93 96 REFERENCES……………...…………………….………………………………………..… 99 vii LIST OF TABLES Table 1. Study 1 Parent Characteristics ……………….……………...………………………... 13 Table 2. Study1 Provider Characteristics………………………………………………….......... 14 Table 3. Frequency of primary pattern codes as they map onto Rogers’ theory….……………. 25 Table 4. Summary of Adaptations to Project ImPACT…..…………………………………….. 38 Table 5. Study 2 Parent Characteristics……….…...………………………………………..….. 44 Table 6. Study 2 Provider Characteristics…….…………………………...……………………. 45 Table 7. List of Study 2 Measures…………………………….………………………………... 52 Table 8. Parent Results From Two-Way, Between-Subjects ANOVAs………………………... 58 Table 9. Percent of Parents who Reported Each Theme by Program Type…..…….…...……… 63 Table 10. Provider Results From Two-Way, Between-Subjects ANOVAs …………………… 66 Table 11. Percent of Providers who Reported Each Theme by Program Type ………..………. 72 Table 12. List of Codes as they Relate to Site and Rogers’ Theory……………………………. 92 viii LIST OF FIGURES Figure 1. Parent ratings of program attributes by program type and Medicaid status………….. 55 Figure 2 .Parent ratings of structural barriers by program type and Medicaid status.……...….. 56 Figure 3. Parent ratings of intent to use by program type and Medicaid status...….………...… 57 Figure 4. Provider ratings of program attributes by program type and caseload SES……......… 64 Figure 5. Provider ratings of intent to use by program type and caseload SES………………… 65 ix INTRODUCTION Background and Significance Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by the presence of social communication challenges in addition to restricted, repetitive, and stereotyped patterns of interest (American Psychiatric Association, 2013). The rising prevalence of ASD over the past two decades (Zablotsky et al., 2015) has been coupled with promising research to demonstrate the efficacy of early intervention services on the short- and long-term outcomes of individuals with ASD and their families (Odom et al., 2010; Virués-Ortega, 2010; Wong et al., 2015). Specifically, early and intensive intervention services for ASD have been shown to improve the social communication skills, adaptive skills, educational placement, and social functioning for many children with ASD (Virués-Ortega, 2010; Wong et al., 2015). Despite the promise of these findings, families of children with ASD continue to report relatively large amounts of service access barriers and unmet service needs (Chiri & Warfield, 2012; Hodgetts, Zwaigenbaum, & Nichols, 2015; Pickard & Ingersoll, 2015). Parent-mediated interventions are one type of intervention service for ASD, and are one possible means to address the reported barriers and unmet service needs of families of young children with ASD (Stahmer & Pellecchia, 2015). Parent-mediated interventions are distinct from direct and more intensive early intervention services in that they aim to both: 1) actively support parents in delivering intervention strategies to their children within a variety of home routines; and 2) support child social communication and/or behavioral development through parents’ use of intervention strategies within the home setting. By supporting parents in learning intervention strategies, parent-mediated interventions have the potential to increase the quantity 1 of early intervention that a child is able to receive with less need for active and direct involvement from a therapist. Given their multi-pronged approach to treatment and growing evidence base, parentmediated interventions are considered to be a critical component of early intervention services for ASD (Wong et al., 2015). Research has shown that parents are able to learn evidence-based, intervention strategies at fidelity and are able to use these strategies to improve their child’s social communication skills (Aldred, Green, & Adams, 2004; Ingersoll & Wainer, 2013; Kasari et al., 2014; Kasari et al., 2015; Rogers et al., 2014; Wetherby et al., 2014). Parent-mediated interventions also provide benefits to the parents who participate in them by improving parent self-efficacy and empowerment, and by reducing parental stress and depression (Estes et al., 2014; McConnachie & Diggle, 2007; Pickard et al., 2016; Tonge et al., 2006). These additional benefits are crucial given the high rates of stress and depression in parents of children with ASD (Hayes & Watson, 2013; Karst & Van Hecke, 2012). Despite the notable benefits associated with participation in parent-mediated intervention programs, research has demonstrated that these interventions are underutilized in community settings, with less than 25% of parents reporting that they use this type of intervention for their child with ASD (Hume, Bellini, & Pratt, 2005; Thomas, Morrissey, & McLaurin, 2007). Moreover, parent-mediated interventions are no exception to the service use disparities that have been noted throughout the ASD field (Mandell, Novak & Zubritcky, 2005; Patten et al., 2012; Nguyen et al., 2016). In fact, in a recent study, we found that lower socioeconomic status (SES) parents of a child with ASD were more likely than higher SES parents to spontaneously report that parent-mediated interventions were an unmet health need (Pickard & Ingersoll, 2015). This unmet health need is particularly problematic given the fact that lower SES families often receive 2 ASD diagnoses later (Mandell et al., 2009). Although parent-mediated interventions could be one way for these families to make up early intervention time that is lost from a later diagnosis, the limited reported access to these services makes it challenging to do this. Given these disparities and the increase in ASD prevalence rates (Zablotsky et al., 2015), there is great need to improve the translation of evidence-based, parent-mediated interventions for ASD into underserved community settings (Stahmer & Pellecchia, 2015). Only two studies to date have specifically examined the effectiveness of an evidencebased, parent-mediated intervention for ASD in an underserved, low-SES, Medicaid population. Although both studies demonstrated benefits of parent participation on child social communication outcomes (Carr et al., 2015; Carr & Lord, 2016; Kasari et al., 2014), they also reported high levels of parent attrition, even when supports were provided to reduce the transportation and time requirements of the intervention for parents (Carr & Lord, 2016). Qualitative feedback from the parents who participated in one of the studies indicated that scheduling difficulty and a hectic home environment were both barriers to using the intervention (Carr & Lord, 2016). However, there was no assessment of parent perceptions of the parentmediated intervention program or other structural barriers to participation for the high percentage of parents who dropped out of the study. The results of these studies indicate that low-SES families can benefit from parent-mediated intervention when they do participate, but also suggest that many, as yet undetermined, variables may negatively impact parents’ ability to effectively engage in this type of intervention. Utility of Rogers’ Diffusion of Innovations Theory Although not yet fully understood, one possible explanation for the high rates of attrition in parent-mediated intervention programs is a lack of fit between parent-mediated interventions 3 and the needs and preferences of parents and providers in low-SES community settings. Given this possibility, researchers have recently argued for the need to both understand and improve the fit of exiting parent-mediated interventions within these settings (Divan et al., 2015; Ratto et al., 2016; Stahmer et al., 2016; Stahmer & Pellecchia, 2015). Although there are many conceptual theories that can guide the collaborative understanding and adaptation of parent-mediated intervention programs, Rogers’ Diffusion of Innovations theory was chosen to guide the present two studies given its extensive use within the dissemination and implementation field to explain the uptake and use of health interventions (Cooke, Thompson, & Schnurr, 2015; Greenhalgh et al., 2004; Nilsen, 2015; Rogers, 2002; Rogers, 2003). Rogers’ theory notes that the uptake and use of health interventions is influenced by the perceived attributes of the intervention within a given community setting. Although Rogers’ theory specifies many attributes that are important to consider as interventions are disseminated into the community, those attributes most often associated with increased intervention use include the intervention’s compatibility with the community and/or practice setting, its overall complexity (i.e. ease of learning), and its relative advantage when compared to the other services presently being used in the setting (Cook et al., 2015; Dingfelder & Mandell, 2010; Moore & Benbasat, 1996; Rogers, 2003; Rogers, 2003; Taylor & Todd, 1995; Tornatzky & Klein, 1982). Importantly, these attributes have been shown to explain between 49 and 87% of the variance in how quickly an intervention is disseminated into community settings (Rogers, 1995). The importance and utility of Rogers’ theory has been substantiated by research both within and outside of the ASD field. Outside of the ASD field, research suggests that parents’ perception of intervention attributes and barriers predicts their engagement in parent-mediated 4 interventions, independent of their socioeconomic background (Axford et al., 2012; Kazdin, Holland, & Crowley, 1997; Nock & Kazdin, 2005; Reyno & McGrath, 2006). These perceived attributes are equally critical for providers and have been shown to predict providers’ intent to use and actual use of novel health practices, above and beyond other important factors such as their own background and organizational climate (Henggler et al., 2008; Knudson & Roman, 2015; Sanni et al., 2013; Scott et al., 2008; Shapiro, Prinz, & Sanders, 2011). For example, physicians’ perception of the compatibility of a novel health practice predicted their intention to use and actual adoption of the novel health practice (Scott et al., 2008). Within the ASD field, research studies have both argued for the utility of Rogers’ Diffusion of Innovations theory to understand the fit of existing ASD services in community settings (Dingfelder & Mandell, 2011), and have also actively utilized Rogers’ theory to adapt evidence-based ASD interventions for better fit within a culturally diverse and underserved community setting (Ratto et al., 2016). For example, Ratto et al. (2016) used a community-based partnership with Rogers’ Diffusion of Innovation theory as a guiding framework to successfully adapt both an executive functioning intervention for school-aged children with ASD, as well as a parent and teacher-mediated behavior management intervention. Taken together, this research suggest that Rogers’ theory may be ideal in providing a clearly operationalized framework to both understand and enhance the fit for existing evidence-based, parent-mediated interventions, thereby improving their fit and use within low-SES community settings. The present paper presents two studies, both of which were grounded in Rogers’ Diffusion of Innovations theory (Rogers, 2003). In the first study, Rogers’ theory was used as a guiding framework within the context of a research-community partnership in the Medicaid system. The primary goal of this first study was to understand parents’ and providers’ perception 5 of the compatibility, complexity, and relative advantage of an existing evidence-based, parentmediated intervention, Project ImPACT (Ingersoll & Dvortcak, 2010), as a way to understand specific variables likely to influence Project ImPACT’s fit and use within this Medicaid system. In the second study, Rogers’ theory was again used to propose a series of adaptations to Project ImPACT in collaboration with the same Medicaid system and then to test the impact of these adaptations. Specific goals of this second study were to understand whether adaptations to Project ImPACT improved parents’ and providers’ perceptions of the program’s compatibility, complexity and relative advantage, as well as their intent to use the program, both within and outside of the Medicaid system. 6 STUDY 1 Introduction As previously noted, few studies have examined the effectiveness of evidence-based, parent-mediated interventions for ASD within Medicaid systems, and those that have done so have reported relatively high levels of parent attrition (Carr et al., 2015; Carr & Lord, 2016; Kasari et al., 2014). Although limited in number, the results of these initial studies may reflect a general trend in the development and dissemination of evidence-based practices, where interventions are largely developed independent of research-community partnerships, a type of collaboration with community stakeholders that can be used to address a variety of shared goals (Barrera, Castro, & Steiker, 2011; Brookman-Frazee et al., 2012; Cooke & Kilmer, 2012; Drahota et al., 2016; Glasgow, Lichtenstein, & Marcus, 2003; Mejia, Ulph, & Calam, 2016; Trickett, 2011; Wandersman, 2003). The absence of diverse community partnerships and collaboration early on in the intervention development process impedes the translation of highquality interventions, like parent-mediated interventions for ASD, into community settings (Trickett, 2001). Given that many ASD interventions have been developed in more rigorous research settings with families that tend to be more educated and resourced than families in the general population (Dingfelder & Mandell, 2011; Robertson et al., 2017), there is now a need to retroactively engage the community around adapting existing evidence-based practices to address apparent gaps as they are being moved out into the community (Barrera, Castro, & Steiker, 2011). Although neither ideal nor efficient, this process ensures the fit, acceptability and relevance of parent-mediated interventions within the community settings to which they are being translated. 7 In fact, many research studies outside of the ASD field have demonstrated the benefit of using community partnerships to retroactively adapt existing parent-mediated interventions in underserved settings (Barrera & Castro, 2006; Baumann et al., 2015). A recent meta-analysis of these studies within the disruptive behavior field suggested that culturally adapted parentmediated intervention programs appear to be equally efficacious but have greater perceived compatibility and parent retention rates when compared to their non-adapted counterparts, with effect sizes reaching d=0.45 (Baumann et al., 2015; Griner & Smith, 2006). This research suggests that using community-research partnership to understand and subsequently adapt the delivery of evidence-based, parent-mediated interventions may be one strategy to increase their fit, relevance and use within underserved and diverse community settings. Study Goals and Overview The goal of the present study was to understand the fit of an existing evidence-based, parent-mediated intervention for ASD, Project ImPACT (Ingersoll & Dvortcsak, 2010), within a low-SES, underserved system. Throughout this study, low-SES was conceptualized as 133% of the United States Federal Poverty Line, or the point at which a family can qualify for Medicaid insurance by family income. Medicaid insurance is one of the largest payers for health care in the United States and is specifically designed to serve families of low-income status. Within the state of Michigan, 2.29 million individuals are covered by Medicaid insurance in some capacity (Medicaid.gov, 2016). Families and providers within the Medicaid system were actively recruited and included in this study given the previous research to demonstrate the difficulty translating evidence-based parent-mediated interventions to families receiving services within this specific system of care (Carr et al., 2015; Carr & Lord, 2016; Kasari et al., 2014). 8 Although the parent-mediated intervention of interest, Project ImPACT, was initially developed in collaboration with parents and providers in community settings (Ingersoll & Dvortcsak, 2010), these initial stakeholders did not represent diverse socioeconomic backgrounds. This development process has ultimately limited the fit of Project ImPACT in the Medicaid system, and has underscored the need to collaboratively adapt the program to increase its ability to serve families from lower SES backgrounds. To understand the fit of Project ImPACT in the Medicaid system, research-community partnerships were formed with several Medicaid ASD service agencies. By forming these partnerships and using Rogers’ Diffusion of Innovations theory as a guiding framework (Rogers, 1995), the primary aims of this study were to understand: 1) the compatibility, complexity, and relative advantage of Project ImPACT when used within a Medicaid system; 2) the perceived structural barriers to the program’s use within the Medicaid system; and 3) adaptations that parents and providers felt would enhance Project ImPACT’s fit and use within the Medicaid system. Method Community Partners and Procedures This study was conducted in collaboration with three community agencies that provide a variety of mental health services through Detroit Wayne Mental Health Authority (DWMHA), a publicly funded health system in the metro-Detroit area. Each of the three agency partners provide a range of services to Medicaid eligible families, but are unique in having a developmental disability branch of the agency. Partnerships with these agencies were established after a number of ASD providers attended a Project ImPACT workshop sponsored by DWMHA. During this workshop it was evident from provider input that, while there was interest in using 9 Project ImPACT, the format and delivery of Project ImPACT was not a good fit for the families served within this system. This initial concern prompted a set of collaborative discussions with the DWMHA partnering agencies around methods that could be used to adapt Project ImPACT. Once the study protocol was finalized and approved by the DWMHA review board, interested providers were able to enroll in the research study through email with the primary investigator. Providers who worked with families of children with ASD provided parents with a recruitment flier outlining the goal of the parent focus groups. Parents who voiced interest were able to call, email or text the primary investigator to express interest in attending. From there, the primary research investigator was able to reach out to parents to answer any questions and to support parents in being able to attend the focus groups (i.e. by providing child care, transportation and a light meal). All participants were compensated $25 for their participation one of the focus groups and were able to access the Project ImPACT program, if interested, following their participation. Parent-Mediated Intervention of Interest Project ImPACT is an evidence-based, parent-mediated intervention that teaches parents to promote their child’s social communication skills (Ingersoll & Dvortcsak, 2010). Project ImPACT teaches parents to use a blend of developmental and naturalistic behavioral intervention techniques across a variety of daily routines in order to enhance their child’s social engagement, language, imitation, and play skills. Project ImPACT can be delivered in an individual or group format, and is geared towards children with ASD with an expressive language level between 6 and 42 months. In addition to the support for the specific intervention techniques used in Project ImPACT (Schreibman et al., 2015), an open trial and several controlled studies have demonstrated the efficacy of Project ImPACT for increasing parent use of the intervention 10 strategies, as well as child social communication skills (Ingersoll et al., 2016; Ingersoll & Wainer, 2013; Stadnick et al., 2015). Both qualitatively and quantitatively, participating parents have reported that they view their child more positively and that they feel more empowered to promote their child’s development after participating in the program (Pickard et al., 2016). The original Project ImPACT program was designed to be delivered with parents and providers meeting twice each week for one hour over 12 weeks. In each session, parents receive: 1) didactic instruction in intervention strategies; 2) modeling of the intervention techniques by the therapist; 3) live coaching from the therapist while practicing the strategies with their child; and 4) homework to practice the strategies at home with their child 20-30 minutes each day. Experimental Design and Study Procedure Focus group methodology was used to collect qualitative data regarding the fit of Project ImPACT within the Medicaid system, using Roger’s Diffusion of Innovations theory as a guiding framework. Following data collection, Rapid Evaluation Assessment Methodology (REAM; McNall & Foster-Fishman, 2007) was used in combination with Miles and Huberman’s approach to qualitative analysis (1994) to analyze the focus group results. Participants. Sixteen parents of a child with a current ASD diagnosis participated in one of three parent-only focus groups. All parents were Medicaid-eligible based on income (i.e. family income below 133% of federal poverty line; Healthcare.gov, 2015) and had at least one child between the ages of 18 months and 6 years (M=4.50; SD=0.73) who was receiving services through the ASD Medicaid Benefit in Michigan. Parents were an average age of 34.81 years (SD=7.07) and were 87.50% female. Eighty-one percent of participating parents endorsed a household income and family size that indicated that they fell below the federal poverty line 11 based on the Department of Health and Human Services 2016 guidelines (U.S. Department of Health and Human Services, 2016). Sixteen professionals who provided direct and parentmediated interventions to families of children with ASD participated in one of three provideronly focus groups. All professionals provided services through the ASD Medicaid Benefit and were 93.80% female. All participant demographic information is displayed in Tables 1 and 2. 12 Table 1. Study 1 Parent Characteristics Characteristic Percent (N=16) Parent Gender (% female) Age in years Number of household members Medicaid by income At or below poverty line 100.00% 87.50% Marital status Married/Living with Partner Single, Widowed, Divorced 43.80% 56.20% Ethnicity White/Caucasian Black/African American Hispanic/Latino Middle Eastern Asian/Pacific Islander Biracial/Other 12.50% 62.50% 0% 6.30% 0% 18.80% Education Less than High School High School Degree Some College/Specialized Training Bachelor’s Degree 12.50% 56.30% 18.80% 12.50% Average Annual Household Income <$10,000 $10,001-$15,000 $15,001-$20,000 $20,001-$30,000 37.50% 6.30% 31.30% 25.00% Mean (SD) 87.50% 34.81 (7.07) 4.00 (1.71) Child with ASD Age in years Gender (% male) 4.50 (0.73) 77.80% 13 Table 2. Study 1 Provider Characteristics Characteristic Percent (N=16) Parent Gender (% female) Age in years Mean (SD) 92.50% 30.69 (9.90) Ethnicity White/Caucasian Black/African American Hispanic/Latino Middle Eastern Asian/Pacific Islander Biracial/Other 43.75% 43.75% 0.00% 6.25% 0.00% 6.25% Education High School Degree Some College/Specialized Training Bachelor’s Degree Graduate Degree 18.75% 25.00% 0% 56.25% Occupation Behavior Technician Family Supports Coordinator Social Worker BCBA Early Childhood Therapist Psychologist 25.00% 12.50% 6.25% 12.50% 37.50% 6.25% Years experience with children with ASD Number of children with ASD on caseload 1 child 2-3 children with ASD 4-6 children with ASD 7-10 children with ASD 11-15 children with ASD 15+ children with ASD 4.69 (3.75) 12.50% 18.80% 12.50% 25.00% 12.50% 18.80% 14 Measures Sociodemographic information. Parents provided basic demographic information including parent gender, age, number of household members, marital status, ethnicity, highest educational degree obtained, and average household income. Additionally, parents provided the age and gender of their child with ASD. Providers reported similar demographic information including their gender, age, ethnicity, highest educational degree obtained, occupation, years of experience working with ASD, and number of ASD clients on their caseload. Focus groups. Focus groups were structured to include either all providers or all parents and ranged in size from having 4 to 9 individuals within each group. All focus groups were led by the primary investigator, in addition to at least one parent advocate or agency leader, who was known and respected by the participating parents and providers. This advocate was able to introduce the primary investigator and help with moderating any discussion as needed. Parent groups included parents of same-age children with ASD, all who were receiving Medicaid services through the same agency. Provider groups included providers working within the same community agency. All participants were provided with informed consent prior to their participation in the focus groups. During this time, the focus group leader asked questions regarding participants’ past research experience, if any, walked participants’ through the consent form, and answered questions. Participants in each focus group viewed a standardized, fifteen-minute video-based presentation outlining the format, goals, and requirements of the Project ImPACT program. The written script for this video is displayed in Appendix A. 15 Following the presentation, participating parents and providers participated in a semistructured focus group that lasted between 25-35 minutes for parents (M=28.91) and 30-51 minutes for providers (M=37.27). Each focus group opened with general perceptions about the presented Project ImPACT program, and then moved to specific sets of questions that mapped directly onto Roger’s theory. These questions regarded perceptions of Project ImPACT’s: 1) compatibility with the participants’ values, lifestyle, and/or service delivery system; 2) overall complexity of the program and its written material; and 3) relative advantage compared to current intervention use; as well as 4) the structural barriers to intervention participation; and 5) modifications that would enhance the fit of Project ImPACT within their community setting. Each section of the interview began with broad questions regarding parents’ and providers’ perceptions. Depending on initial responses to perceptions of the program, narrower probes for each interview section were provided to guide discussion. The semi-structured interview protocol for parents and providers are displayed in Appendices B and C. Following group participation, all parents and providers were given access to the Project ImPACT curriculum. Data Analysis The first portion of qualitative data analysis was completed using rapid evaluation and assessment methodology (REAM; McNall & Foster-Fishman, 2007). As part of REAM, two primary investigators, both trained in qualitative analysis, met at the completion of each focus group to discuss the central themes that emerged from the interview and additional questions that could guide further data collection. This process was repeated until no new themes emerged. The initial themes that emerged during REAM were used to facilitate Miles and Huberman’s (1994) approach to qualitative analysis (Miles & Huberman, 1994), which allows for qualitative data to be analyzed from within a given framework. In the first phase of data analysis, focus group 16 interviews were de-identified and reviewed by the two investigators to condense the data into analyzable, “first-cycle” codes (Miles & Huberman, 1994). During this process the initial codes that emerged during REAM were validated and any additional codes missed during REAM were added. Reliability was assessed after each investigator independently coded the transcripts, and was calculated as percent agreement on two of the six transcripts (i.e. roughly 30%; Boyatzis, 1998). Analysis indicated good reliability, with agreement on 81.17% of first-cycle codes. In the second phase of data analysis, investigators used both single case and contentanalytic summary tables to summarize the frequency of first cycle codes across each of the different focus groups, and to group them into overarching pattern codes (Miles & Huberman, 1994). The creation of these matrices was facilitated by the use of Dedoose software. During the final phase of data analysis, pattern codes were summarized across all of the parent and provider focus groups, and subsequently mapped onto one of each of the five key constructs of interest (i.e. compatibility, complexity, relative advantage, structural barriers and suggested modifications; Miles & Huberman, 1994). To ensure that these final primary themes were valid representations of the focus group data, they were re-presented to the original participating parents and providers in a second meeting about 5 months following the initial data collection. During this time, parents and providers had the opportunity to voice any concerns about the accuracy of the construct tables and other themes that were missed along the way. Examples of a coded transcript and content analytic summary table are displayed in Appendices D and E. Results Overarching pattern codes and their frequency across parents and providers are summarized in Table 3. 17 Perceptions of Project ImPACT Compatibility with Beliefs and Lifestyle In general, both parents and providers expressed initial positive perceptions of the Project ImPACT program. All of the parents reported that the social communication goals of Project ImPACT were important and consistent with their child’s needs. For many participants, emphasis was also placed on the similarity of the goals of Project ImPACT with other beneficial and effective services that they were currently receiving and/or delivering. Given these positive perceptions, many providers and even more parents reported that they would be motivated to try the program. “The goals are huge. I mean most of the time as support coordinators, parents say their goals mostly are learning how to interact with their child. That’s normally the primary thing, ‘if they could just tell me what’s going on I wouldn’t be so stressed, if I could just play with him, he doesn’t like me,’ those are really their main goals.” - Provider “It was the same goals that I had with [my son] and it’s exactly what I went through so it [the program] is basically on point.” - Parent Despite initial positive perceptions, unanimous concerns were voiced regarding the time intensity of the program. Many parents indicated that they had multiple jobs and chaotic home environments that would make it difficult to set aside 20-30 minutes each day to practice the Project ImPACT intervention strategies with their child with ASD. “I’m sure they feel the same, anything to help our kids, 24 hours a day for our kids. But […] the only thing for me personally, from our situation, we have, my son has a twin, and then we have a 20 month old, to set aside 20 minutes a day just with [child receiving services] is impossible.” – Parent “It’s just so hard. I mean you got a lot of your families, if they’re working, are working 2 and 3 jobs […]. It’s just a different dynamic, its survival. They don’t have time for extras.” – Provider Given their time constraints and living situations, many families and providers reported that teaching within daily routines was one aspect of Project ImPACT that made it compatible 18 with a family’s lifestyle in this particular community. Providers emphasized that many families did not have toys, making it difficult to ask parents to set aside playtime. On the other hand, parents and providers both underscored the benefits of being able to practice the intervention within pre-existing daily routines. “You tell them the 30 hours a week they’re gonna laugh at you and tell you ‘I’m not interested anymore’. But the reality is, when you tell them ‘oh if you just do it during lunch time or snack time’ they’re gonna do it and it’s gonna add up the same.” – Provider “Maybe not a 30 minute block but 10 minutes at bath time, 15 minutes during dinner, you know.” - Parent Finally, both parents and providers emphasized that the more rigid structure of delivering Project ImPACT (i.e. meeting twice per week for one hour) was not consistent with the way that services were typically delivered to families within the Medicaid system. This theme was particularly salient for providers, who reported their concerns more frequently than parents. For example, parents emphasized that there were instances in which they would be unable to complete a lesson, and that flexibility and the ability repeat those lessons was essential. Similarly, providers reported that the ability to repeat lessons and check in with parents more than twice per week would make the program more compatible with the manner in which they delivered services. “A lot of times our therapist, he gives us a lot of stuff to take home. Especially when we’re doing an intervention with my son and trying to snuff out a bad behavior and we take notes […] ourselves, a lot of times we’re successful doing it and sometimes you just can’t do it. So it needs to be flexible for it to work.” – Parent “Maybe shorter but more frequent check-ins, because um a week’s a really long time in their world, a lot can happen in a week. So if I had more frequent check-ins I might be more likely to stay on top of those suggestions.” – Provider 19 Perceptions of Project ImPACT Complexity Regarding the complexity of Project ImPACT, both parents and providers emphasized that the quantity and quality of written language throughout the parent manual was intimidating and prohibitory of parent learning and engagement. As can be seen in Table 3, providers reported this concern more frequently than parents. While discussing the written language of the program, providers often underscored the low literacy levels within their community, and the inability of parents to learn solely through reading and writing. On the other hand, all participants reported that the video examples, modeling, and role-play embedded in the Project ImPACT program should be emphasized, as parents were more likely to learn from these teaching techniques. “Fancy language, homework, reading keep it as minimal. And also what are their skillsets? Like we follow the child’s lead, we follow the parent’s lead. Literally everything you do for the child you have to do for the parent [because] oftentimes the skill sets aren’t gonna be much different.” – Provider “Visuals, with them I can know exactly how to do it. Reading I wouldn’t know how exactly to do it [use the intervention techniques].” – Parent “I saw a video once of a therapy and how to implement it and it seemed pretty straightforward, and it was short like this, 15-20 minutes, and from what I saw it makes a lot of sense. When you see a therapist do it, it makes it a lot easier.” - Parent An additional concern related to the complexity of Project ImPACT regarded the need to teach parents 14 different intervention techniques within a 12-week program. Providers, but not parents, often reported that the shear number of techniques and their terminology would likely overwhelm and disengage parents. They indicated that grouping and/or prioritizing these techniques might be one strategy to keep parents engaged and learning. “I would definitely minimize the number of techniques; but keeping in mind if they’re evidence based and they do work, maybe you can lump some into groups so instead of 14 you have four groups […]. It’s not so overwhelming to the parent.” – Provider 20 Relative Advantage of Project ImPACT When asked how Project ImPACT compared to other services that parents and providers were currently using, many participants indicated that they did not currently deliver or receive formalized parent-mediated intervention within their system. Despite perceiving the Project ImPACT program positively, only one clear advantage was voiced by parents and providers. This was the ability to deliver Project ImPACT in a group format. All participants emphasized the ability of a group format to simultaneously boost parent social support while also teaching them important skills to promote their child’s social communication development. Despite this clear advantage, many providers highlighted the structural barriers that would likely prevent the delivery of Project ImPACT in the group format. “When he first started school it was parent meetings I went to. Now they never have parent meetings for other parents to interact, like why not? I would like that.” - Parent “I would say 4 out of 5 of my families don’t have transportation right now so I think they would benefit significantly from a group, but if there is not transportation for that group then it’s not going to happen." – Provider Additionally related to relative advantage, providers frequently highlighted the lack of incentive for billing parent-mediated interventions in the Medicaid system. These providers emphasized that billing was not cost-effective when compared to billing for other direct, fullyreimbursable services. “It would be [billed under] the family training code – which it is a reimbursable service, but when you’re looking at it I mean if we fully wanted to encompass it, it wouldn’t balance out. It definitely wouldn’t financially be in our benefit.” – Provider Perceived Structural Barriers to Project ImPACT Participation In general, providers tended to report structural barriers to program participation more often than parents. These reported barriers were related to unstructured and chaotic home 21 environments, the importance of meeting parents’ basic needs prior to their participation in Project ImPACT, and the need to translate the program into Spanish and Arabic. Both parents and providers emphasized that it would be difficult to set up and manage the home environment so that parents could practice the intervention techniques with their child undisturbed by other family members. Providers also reported that parents’ sometimes dangerous home environment would impede their learning and ability to practice the intervention techniques. “We live with my husband’s parents. My husband’s parents, especially his father, he thinks things should be different - he’s very loud he doesn’t, you know, think before he speaks and sometimes I have to move our therapy sessions to another location because it’s just crazy.” – Parent “A lot of the homes I go to they can’t go play outside, it’s very dangerous, and there are also four or five kids so it’s like can we really get mom and kiddo time? Because you physically sometimes can’t leave the other kids.” – Provider Equally as prohibitive as the home environment was the fact that many families within the Medicaid system were reported to have their basic needs unmet. Providers reported this theme more frequently than parents, and were quick to emphasize the inability to engage parents who were struggling to find housing, food, and jobs to support their family. “She [a provider] has one family who’s moved twice already [and] they’ve been here 90 days, yeah so they’ll tell you how important those things [basic needs] are. Furniture, I think we’ve done 12 referrals. And her caseload is small only 35 people so yeah she is constantly on that referral bandwagon and if she doesn’t meet their basic needs she can’t meet the clinical needs.” – Provider “The other thing is recently we’ve had two families in homeless shelters so providing services with that is challenging because we’re really at the will of the shelter. Both of them were domestic violence situations so they don’t want visitors there. So that put a hold on our service we might stop, start, but reengage, but you’re really starting all over again.” – Provider. Suggestions to Improve Project ImPACT Fit Both parents and providers frequently made suggestions to add strategies and components to Project ImPACT that were not presently included in the program, but that would 22 enhance its overall fit within the Medicaid system. Participants unanimously reported that the program would be more likely to win initial parent engagement by emphasizing that the practice time could be spread out throughout the course of the day. “I think for our situation, you know, when you have kids teachable moments are always there, and I think if you can make this where the moments come up and you can use it at that time, a little bit here a little bit there, make it easier. It would be great if I could set aside the time and sit down with him and do it, and I would do that when possible but otherwise you have to use the moments when you can.” – Parent “I always tell them do two minutes here and two minutes there. The reality is it might add up to 30 minutes. […] I never tell them the total amount of time because I know in [another intervention] were supposed to tell them the 30 hours a week. You tell them the 30 hours a week, they’re gonna laugh at you and tell you I’m not interested anymore. But the reality is when you tell them oh if you just do it during lunch time or snack time they’re gonna do it and its gonna add up the same.” – Provider Parents more often than providers underscored that Project ImPACT would also benefit from strategies to help parents seek out emotional support from their family and other individuals in their life. Many parents reported feeling isolated and alone when attempting to promote their child’s development. Parents and providers reported that strategies to gain support from their family members would greatly enhance the ability of the program to meet their needs. For parents, this also meant the ability to train both immediate family members and older siblings to deliver the Project ImPACT intervention techniques. “We need a hotline for parents [of a child] with autism for when their kids are having a break down. Cause some days I’m just crying and can’t get myself together. That sounds like a good idea, a hotline where you could call and get some advice real quick.” - Parent “My mother. I really want them [my family] involved but they wouldn’t make the time for it.” – Parent “That would be [helpful], yeah, in fact, our therapist started having us bring our daughter to get her to play with him and generalize things. I think that’s very important.” - Parent In addition to increased family involvement, both parents and providers unanimously voiced that Project ImPACT would benefit from the inclusion of strategies to help parents 23 manage their child’s problem behavior. Providers emphasized that reducing behavior problems was often a primary goal for parents. Similarly, when asked about goals that were important to them, parents reported that aggressive behavior was a pressing concern. Finally, both parents and providers reported the need for Project ImPACT to incorporate strategies to build collaboration, communication and empathy across parents and professionals. For some providers, this emphasis centered on facilitating communication and engagement around the importance of parent training programs like Project ImPACT. “And it tends to sometimes overwhelm the families. Where it’s like ‘I just want to cut down the hours on everything,’ and they don’t realize the importance. It falls back on them and their stress versus focusing on this the best for [the child].” - Provider However, for both parents and providers a key piece of building the therapeutic relationship and gaining parent engagement was using basic empathy and empowerment-based strategies when working with families. Many parents and some providers emphasized that the lack of empathy and respect for parents’ perspectives was disheartening and counterproductive to engaging parents in beneficial parent training programs. Participating parents emphasized that they would disengage from the program if the therapist did not use a strength-based and collaborative approach to empower them. “So making sure especially on the therapist end that they understand. ‘He has this and this and this,’ and like really? We don’t look at it as something right and wrong.” – Parent “Yeah the parent’s the expert, that’s what I lead into is ‘you’re the expert we’re here to help you in modeling and molding what you know.’ And I think when the parent’s feel that there’s a sense of ownership there’s a sense of pride and then you know they’ll say, ‘Did I do that right?’ It always comes back to the clinician anyways but making them believe and letting them know they are the expert of their child.” – Provider 24 Table 3. Frequency of primary pattern codes as they map onto Rogers’ theory Parents Providers Compatibility of Project ImPACT with beliefs and lifestyle General positive perceptions of the program goals Required practice time is too time consuming Practicing intervention in daily routines is compatible with family lifestyle Program length and delivery is too rigid 24 7 5 11 14 9 8 24 Complexity of written materials and intervention techniques All reading materials (i.e. manual; homework) are too complex Too many intervention techniques are taught within a short program Writing requirements not feasible for many families Visual learning should be emphasized (i.e. videos, observation, practice) 17 0 6 10 28 9 14 9 Relative advantage of Project ImPACT compared to current practice Option for group support Non-incentive for insurance reimbursement 11 0 7 7 Structural barriers to intervention participation Difficulty managing home environment (siblings, organization, distractions, danger) Basic needs not being met (mental health, housing, food, toys, etc.) Lack of program translation in other languages 5 16 0 11 28 10 Suggested additions to enhance Project ImPACT Practice time needing to be spread throughout the day Strategies for family and sibling support/implementation Need for strong parent-professional communication and collaboration Challenging behavior problems should be able to be addressed if needed Daily living skills (i.e. feeding, potty training, dressing) should be addressed 19 20 11 9 11 17 10 11 7 6 25 Discussion The current study used a research-community partnership within a Medicaid system in order to better understand parents’ and providers’ perceptions of Project ImPACT, an evidencebased, parent-mediated intervention for ASD. Focus group methodology was used to facilitate parent and provider discussion around program attributes specific to Rogers’ Diffusion of Innovations theory (Rogers, 2003), as well as discussion around the structural barriers to program participation, and suggestions to increase the program’s fit within the Medicaid system. In general, parents voiced that the Project ImPACT goals were compatible with their own goals for their child with ASD. Additionally, all participants emphasized positive perceptions regarding their ability to practice the Project ImPACT intervention techniques within their preexisting daily routines. Despite these positive views, all parents and providers reported concerns regarding the time intensity of Project ImPACT, the program’s relatively rigid delivery format, and the complexity of the written materials. Although parents and providers highlighted the relative advantage of being able to deliver Project ImPACT in a group format, many providers underscored the structural barriers that would ultimately prevent this type of delivery in the Medicaid system. Outside of program attributes, both parents and providers reported structural barriers to program participation, including chaotic home environments and the lack of basic needs being met. Both parents and providers offered a number of suggestions to address their concerns. These included restructuring practice time, teaching in a more visual manner, addressing problem behavior, and including strategies to involve family members and enhance the parent-professional alliance. 26 Implications The process of retroactively engaging the community around the fit of existing evidencebased practices has been argued to be particularly relevant for parent-mediated interventions in the ASD field (Divan et al., 2015; Stahmer & Pellecchia, 2015). This study attempted to address this need through a research-community partnership formed within a Medicaid system, with mutual goals centering on enhancing the fit of Project ImPACT for use within this system. Although not captured within the recorded focus group data, parents and providers often expressed gratitude for being given a voice in this research study, as they reported that they are often forgotten. The tendency for research studies to be structured in a manner that excludes lower SES and ethnic minority families has frequently been cited in the ASD and developmental disability fields (Hilton et al., 2010; Pierce et al., 2014; Robertson et al., 2017; Tek & Landa, 2012), and is a crucial gap that has impeded the generalizability of recent research findings. Importantly, the insight gained from the partnerships around the fit of Project ImPACT within the Medicaid community clearly supports the recent push for the use of community partnerships to address service access disparities within the ASD field (Stahmer & Pellecchia, 2015). Three important implications stem from the present research study. First, consistent with research outside of the ASD field, parents and providers reported a number of perceived structural barriers that would likely prevent participation in Project ImPACT. Perceived structural barriers have been noted to predict engagement in parent-mediated intervention, above and beyond SES (Kazdin & Holland, 1997), suggesting that these barriers will be crucial to address in order to successfully use the program within the Medicaid system. In fact, parentmediated programs that have incorporated explicit components to help parents problem solve 27 around structural barriers have been noted to promote greater treatment adherence (Nock & Kazdin, 2005). In addition to structural barriers, many parents and providers also highlighted concerns regarding attributes specific to Project ImPACT that would be influential in their decision to adopt and use the program. These attributes mapped on to Rogers’ Diffusion of Innovations theory (Rogers, 2003), and suggest that they are crucial attributes to address within the intervention given their strong influence on program fit and use within the system. Importantly, the attributes highlighted by parents and providers were similar to those reported in a previous study within the ASD field that used community partnerships to adapt a parent-mediated program for use in South Asia (Divan et al., 2015). Feedback from parents and providers in South Asia centered on adapting the program language, incorporating video feedback, the desire to include the extended family, and the importance of the therapeutic alliance (Divan et al., 2015). This suggests that there may be specific aspects of parent-mediated programs, in general, that should be considered and evaluated in collaboration with community partners during the intervention development process. A final implication of the present study relates to the importance of collaboration and partnership between parents and professionals in parent-mediated intervention programs. A handful of participating providers reported that educating Medicaid-eligible parents on the importance of parent-mediated interventions was necessary to enhance their engagement and participation in the program. On the other hand, the majority of parents and a handful of providers underscored the need for genuine collaboration between parents and providers, and empathic listening on the part of the provider as a means to enhance parent engagement in the intervention. Participating parents emphasized that they would disengage from the program if the 28 therapist did not use a strength-based and collaborative approach to empower them. This qualitative report is consistent with a number of research studies within and outside of the ASD field that demonstrate the importance of parent-therapist alliance in predicting retention in treatment programs and long-term parent mental health outcomes (Curits & Singh, 1996; Dunst & Dempsey, 2007). Limitations There are several important limitations to consider within this study. Most prominent of the limitations is likely the generalizability of the study findings. The study results are drawn from a small number of focus groups and participating parents from within a specific community setting. Although the focus groups were generally representative of each agency’s demographics, the groups may have consisted of more motivated parents given their active choice to participate in the focus groups. Taken together, the study results may not generalize to other low-resourced parents outside of the metro-Detroit area. Additionally, parent and provider focus groups were run separately. The primary reason for this separation was due to suggestions from our community partners that combined focus groups might result in power dynamics that would make it more difficult for parents to express their experience and opinions. However, the lack of multiple stakeholder roles within the focus groups may have limited conversation between parents and providers that could have lent to more nuanced qualitative data. Two final study limitations relate to the framework that we chose to guide the adaptation of Project ImPACT. Although Rogers’ theory is widely used in the dissemination and implementation field to address intervention uptake and use, other frameworks, such as the Ecological Validity Model, have been used to guide adaptations geared towards specific cultural groups and may also have been appropriate for use in this study (Barrera & Catro, 2006; Bernal, 29 Bonilla, & Bellido, 1995; Nilsen, 2015). A final limitation to consider is the manner in which parents and providers were asked about the relative advantage of Project ImPACT as compared to other services that they were currently receiving and/or delivering. Participants were asked to discuss how Project ImPACT might be similar and/or different from their current intervention use/delivery. However, no follow-up questions were used to probe about the possible advantages or disadvantages of any differences noted between Project ImPACT and other intervention programs. Similarly, the question of the relative advantage of Project ImPACT was somewhat misleading. Research to date does not support the idea that parent-mediated intervention programs, like Project ImPACT, should replace direct intervention (Bibby et al., 2002; Weatherby & Woods, 2006). Therefore, the relative advantage of Project ImPACT should have been considered alongside other parent-mediated programs or the absence of parent-mediated programs, rather than the direct services that they would likely never replace. Future Directions Given recent calls within the ASD field (Stahmer & Pellecchia, 2015), future directions include making adaptations to Project ImPACT based on parent and provider input from the present study. These changes will be made in continual collaboration with the partnering Medicaid agencies. Although it will be important to consider the tension between fidelity and fit when making these adaptations (Parra-Cardona et al., 2012), the program attributes and suggestions that would likely be the target of change are specific to the manner in which Project ImPACT is delivered (i.e. the adaptable periphery; Damschroder et al., 2009), rather than the core components of the intervention itself. This suggests that any adaptations made to Project ImPACT on the basis of these partnerships will retain the overall fidelity of the program. After making these changes, eventual aims are to understand the impact of the adaptations on parents’ 30 and providers’ ratings of program attributes (i.e. compatibility, complexity, relative advantage) and intent to use the program both within and outside of the Medicaid system. Despite the importance of adapting Project ImPACT to meet the needs of under-resourced Medicaid settings, this study ultimately highlights the inefficiency of the intervention development process within and outside of the ASD field. It will be absolutely critical that future research incorporate diverse community input when developing novel health interventions. Engaging the community early on in this process may prevent the need to retroactively adapt interventions, which is costly, inefficient and prohibitive of meeting a larger public health need (Sandler et al., 2005). Conclusion In summary, the findings from this study suggest that parent attrition and disengagement in parent-mediated programs is not necessarily related to a lack of motivation by parents with low SES to participate. Rather, the finding suggests that parents are often motivated to participate in parent-mediated programs but that program attributes, perceived structural barriers, and parenttherapist alliance are all crucial aspects of parent-mediated programs that likely influence parents’ choice to enroll and engage in the program. Research within the ASD field has often examined the role of either program attributes, structural barriers or parent-professional alliance in isolation. Although all equally crucial aspects of parent engagement and adoption of parentmediated interventions for their child, the present study emphasizes the need to consider these factors in combination in order to ensure that a given parent-mediated intervention fits with the values and lifestyles of under-resourced communities. Importantly, the use of such partnerships serves as the starting point to systematically adapting these programs in collaboration with 31 community settings in order to address the service use disparities that have been documented throughout the ASD field. 32 STUDY 2 Introduction As more ASD researchers aim to effectively translate parent-mediated interventions to families from culturally diverse and underserved backgrounds, research studies within the ASD field have begun to use research-community partnerships to adapt parent-mediated intervention programs for use within these settings (Divan et al., 2015; Drahota et al., 2016; Pickard & Ingersoll, 2016; Rahman et al., 2016; Ratto et al., 2016; Stahmer et al., 2017). These studies have represented important early steps within the ASD field to adapt the format and delivery of parent-mediated intervention programs to address concern regarding the intensity and/or fit of the programs within low-SES, Medicaid settings (Carr & Lord, 2016; Kasari et al., 2014; Stahmer & Pellecchia, 2015). Although important to adapt evidence-based, parent-mediated intervention programs to meet the needs of low-SES and culturally diverse settings, it is equally important to understand the impact of these adaptations on program use and efficacy. With this in mind, the present study aims to present a series of adaptations made to a parent-mediated intervention program for ASD (i.e. Project ImPACT; Ingersoll & Dvortcsak, 2010) in collaboration with Medicaid-based community agencies, and to highlight the impact of these adaptations on parents’ and providers’ ratings of program attributes and intent to use the program across a variety of community settings. Parent-Mediated Intervention of Interest Project ImPACT is an evidence-based, parent-mediated intervention that teaches parents to promote their child’s social communication development through the use of naturalistic, developmental, and behavioral intervention strategies. Core strategies taught within the Project ImPACT curriculum include: 1) following your child’s lead; 2) adjusting your communication; 33 3) creating opportunities for your child to initiate; 4) teaching your child new skills; and 5) pacing the interaction (Ingersoll & Dvortcsak, 2010). Project ImPACT can be delivered in an individual or group format, and is geared towards children with an expressive language level between 6 and 42 months. The original Project ImPACT program was designed to be delivered with parents and providers meeting twice each week for one hour over 12 weeks. In each session, parents receive: 1) didactic instruction in intervention strategies; 2) modeling of the intervention techniques by the therapist; 3) live coaching from the therapist while practicing the strategies with their child; and 4) homework to practice the strategies at home with their child for 20-30 minutes each day. Research studies examining the efficacy of Project ImPACT have demonstrated that parent participation in the program results in parent learning of the intervention strategies, improvements in child social communication skills, and improvements in parent empowerment and self-efficacy (Ingersoll et al., 2016; Ingersoll & Wainer, 2013; Pickard et al., 2016; Stadnick et al., 2015). Summary of Adaptations to Project ImPACT All adaptations to Project ImPACT were grounded in a research-community partnership with families and providers within Detroit Wayne Mental Health Authority (Pickard, Kilgore, & Ingersoll, 2016), a Medicaid service delivery system in metro-Detroit. After better understanding the perceived compatibility, complexity, and relative advantage of Project ImPACT, as well as the structural barriers to its use within DWMHA, adaptations to Project ImPACT were made in continued collaboration with DWMHA and the intervention developers. Important in this process was ensuring the balance between fidelity and fit; that is retaining the core components of the intervention while making changes to its adaptable periphery in a manner that was able to address the concerns and suggestions raised in our partnership (Greenhagh et al., 2004). 34 To strike this balance, early on, it was important to identify the core components of Project ImPACT alongside the intervention developers. This identification process was guided by past research demonstrating the time intensity at which it was necessary to deliver Project ImPACT (Ingersoll & Wainer, 2013; Stadnick et al., 2015), as well as structured rating scales that are used to measure the program’s fidelity of implementation (Ingersoll & Dvortcsak, 2010). As part of this review process, it was clear that the delivery of Project ImPACT could be reduced from two hours per week to one hour per week while still retaining the program’s efficacy (Ingersoll & Wainer, 2013). Additionally, review of the fidelity checklists demonstrated that at least one specific intervention technique was required to be taught to parents within each of the five core fidelity domains (i.e. following the child’s lead, adjusting communication, etc.). Finally, for providers, core components of Project ImPACT also included key parent coaching strategies included in the program’s parent coaching fidelity checklist (Ingersoll & Dvortcsak, 2010). These domains included: 1) review of intervention techniques with parents; 2) modeling of the intervention techniques while explaining their impact on the child’s behavior; 3) parent practice and coaching around use of the intervention techniques with their child; 4) problem solving around parents’ use of the intervention techniques; and 5) support and planning around how parents can practice the intervention strategies at home with their child. After establishing the core components of Project ImPACT, a second step in the adaptation process was better understanding how suggested changes from our community partners might alter these components. However, much of the feedback that had been given in our partnership was that, overall, the Project ImPACT strategies and goals were crucial and important to parents and providers (Pickard, Kilgore, & Ingersoll, 2016). Therefore, all of the suggested adaptations to the program were not geared towards reducing the time intensity of the 35 program to less than one hour per week, or towards eliminating any of the identified core components of the program. Final adaptations to Project ImPACT, made in collaboration with DWMHA, are summarized within the context of Roger’s Diffusion of Innovations theory (Rogers, 2003) in Table 4. To improve the compatibility of Project ImPACT with family values and lifestyle, the “practice time” required of the program was changed so that it could be spread throughout the day (i.e. rather than in set chunks of 20-30 minutes), and during routines that parents were already completing with their child (i.e. meal time and bath time, rather than structured play). Improved program flexibility was addressed by emphasizing that the program could be delivered 1-2 hours per week with somewhat flexible lesson content and the opportunity to repeat lessons as needed. To reduce the complexity of the program material, all written language was changed to read at an 8th grade level. Additionally, lesson structures were adapted so that the verbal explanation of the intervention techniques was reduced and accompanied by visual, video, and bullet pointed descriptors of the technique. Finally, to reduce program complexity, the bulk of teaching was adapted to focus on parent observation of the therapist and parent practice of the intervention technique. At the end of each lesson, the parent and therapist completed a visual “practice plan” together (i.e. rather than written homework) in which the parent and therapist could collaboratively select a couple of times in which the parent could practice the intervention techniques with their child at home. Finally, to address participants’ reported structural barriers to program participation and suggestions for program enhancement, the first lesson of the program was restructured to include: 1) specific strategies for the therapist to build alliance with the parent; 2) additional problem solving and support around any anticipated barriers to program engagement and 36 success; and 3) discussion around helping the parent to get their “team on board,” including that of grandparents, spouses, friends and older children. Finally, the adapted Project ImPACT program included a new, optional module that included evidence-based strategies to support parents in managing challenging behavior. 37 Table 4. Summary of Adaptations to Project ImPACT Theme Compatibility with Values and Lifestyle Required practice was too time intense Need for practice in routines; difficulty with practice in play Program length and delivery too rigid Primary Adaptation Emphasis on practice time occurring 5-10 minutes across pre-existing routines. Additional videos and emphasis on practicing within daily routines throughout the program. Emphasis on the teaching content with flexibility around the type of delivery (i.e. ability to repeat lessons as needed). Complexity of Learning All reading materials too complex All written materials bullet-pointed and reduced to read at an 8th grade level. Too many intervention techniques Emphasis placed on teaching prioritized intervention strategies in each lesson. Writing requirements not feasible for many families Written homework eliminated and replaced with visual practice plan. Visual learning should be emphasized Teaching through video review, modeling and live coaching emphasized. Relative Advantage Compared to Current Practice Option for group support This option remained as a possible delivery format. Structural Barriers to Participation Difficulty managing environment Additional emphasis and problem solving around home environment. Basic needs not being met Case managers are included as collaborators within the program. Suggestions for Program Enhancement Practice time spread throughout the day Emphasis on practice time as 5-10 minutes spread throughout the day. Strategies for family and sibling implementation More explicit conversation around how to include other family members. Need for strong parent professional partnerships Challenging behavior should be addressed The addition of specific strategies for therapists to build alliance. Added lesson that included strategies to address behavior problems. 38 Testing the Impact of Adaptations Using Theory of Planned Behavior After making adaptations to Project ImPACT, the present study aimed to better understand the impact of these adaptations through the use of two theories. First, given that the initial adaptations to Project ImPACT were grounded in Rogers’ Diffusion of Innovations theory (Rogers, 1995; Rogers, 2003), one way to test the impact of the program adaptations was to understand whether parents from low-SES, Medicaid backgrounds rated the adapted program attributes (i.e. compatibility, complexity, and relative advantage) more favorably than those of the original program. Broad attitudes towards evidence-based practices and more specific attitudes towards program attributes based on Rogers’ theory have been shown to serve as barriers and facilitators to the uptake and use of mental health practices (Barnett et al., 2017; Cook, Thomposon, & Schnurr, 2015; Henggler et al., 2008). Taken together, this research suggests that more positive perceptions of the adapted Project ImPACT program could lead to greater uptake and use of the program for low-SES families. This finding would be pertinent given the high rates of attrition and reported disengagement of low-SES, Medicaid-eligible families when evidence-based, parent-mediated programs have been delivered in Medicaid settings (Carr & Lord, 2016; Kasari et al., 2014; Stahmer & Pellecchia, 2015). In addition to understanding possible changes in parents’ and providers’ perceptions, a second goal of this study was to approximate the impact of the program adaptations on Project ImPACT use. To do this, Theory of Planned Behavior (TPB; Azjen, 1991) was used to test the impact of the adaptations on parents’ and providers’ intention to use Project ImPACT. Intentions, a key component of TPB, are thought to encompass the motivational factors that drive behavior (Azjen, 1991; Sheeran, 2002; Webb & Sheeran, 2006), and have been shown to be associated with and predictive of both the adoption and sustained use health behaviors across a 39 variety of fields (Eccles, 2006). For example, in several meta-analyses, individuals’ intention to perform health behaviors such as hand-washing, contraceptive use, exercise, HIV protection, and smoking cessation were shown to be associated with the initial adoption of the specific health behavior with correlations ranging from r=0.35 to r=0.56, and variances explained ranging from R2=0.15 to R2=0.44 (Armitage & Conner, 2001; Eccles, 2006; Godin et al., 2008; Schwarzer et al., 2007; Scott et al., 2008). Within the mental health field, recent studies have also used TPB to demonstrate that providers’ intentions to use evidence-based practices are predictive of their initial adoption of that practice, mediating the relationship between key organizational factors and use (Williams et al., 2016). Although many studies have used intentions to predict only the initial adoption of a health behavior, others have shown that the same intentions are also associated with the sustained use of health behaviors (Liao & Lu, 2008; Moore & Benbasat, 1996; Quine, Rutter, & Arnold, 2001), with adoption and sustained use being associated at r=0.36 (Williams et al., 2016). Although they are more highly associated with self-reported use of health behavior, intentions have also been shown to predict the objective use of health behavior with similar but smaller effects (Eccles, 2006; Godin et al., 2008). In addition to predicting health behavior at a single time point, other studies using TPB have manipulated individuals’ intentions to assess the effect of these changes on the use of health behaviors. A meta-analysis of 47 of these studies found that average changes in intention of d=0.66 led to changes in both self-reported and actual behavior with effect sizes averaging at d=0.36 (Webb & Sheeran, 2006). For example, in one such study, school-aged children were randomized to an intervention aimed at increasing helmet use. As part of this study, participants’ beliefs, intentions and behavior were measured prior to and after the intervention. Results demonstrated that children who received the helmeting intervention (i.e. persuasive messaging) 40 demonstrated more positive beliefs about helmets and greater intention to wear helmets following the intervention, and were more likely to wear helmets immediately following the intervention and at 5-month follow-up (Quine, Rutter, & Arnold, 2001). Similar findings have been found as they relate to screening for cervical cancer, with brief planning-based interventions associated with increased intention and use of cervical cancer screenings (Sheeran & Orbell, 2000). Taken together, these studies suggest that behavioral intentions are an appropriate proxy for actual behavior, and that meaningful changes in intention could support behavior change. Therefore, prior to trialing Project ImPACT within low-SES, Medicaid settings, TPB can be used to understand whether parents’ and providers’ intentions to use Project ImPACT differ across the original and adapted programs, thereby representing meaningful differences in how the two programs might be used. A much larger and more diverse sample of parents and providers was used in the present study as a way to understand whether the impact of program adaptations were specific to parents from low-SES, Medicaid backgrounds or universal to parents of all SES backgrounds. By using both Rogers’ Diffusion of Innovations theory and Theory of Planned Behavior, the current study had specific aims to: 1) examine whether parents’ and providers’ perception of Project ImPACT program attributes (i.e. compatibility, complexity, and relative advantage; Rogers, 1995) differed across the original and adapted programs; 2) examine whether parents’ and providers’ intent to use Project ImPACT differed across the original and adapted programs; and 3) examine differences in how parents and providers qualitatively discussed the strengths and limitations of the original versus adapted Project ImPACT programs. 41 Method Participating Community Agencies and Recruitment Study 2 was conducted in collaboration with DWMHA (i.e. series of agencies in our initial community partnership; Pickard, Kilgore, & Ingersoll, 2016) and the Wardcliff Abilities Center, which is the developmental disabilities branch of Community Mental Health of Clinton, Eaton and Ingham Counties. Both partnering agencies provide services to families of young children with ASD who qualify for Medicaid insurance based on their family income. In addition to partnering with both Medicaid agencies, Study 2 was also open to families from both higher and lower SES backgrounds through local ASD list serves throughout the state of Michigan. Experimental Design and Study Procedure All parents and providers were able to participate in Study 2 remotely if they had access to the internet and/or a computer, smart phone, or tablet. Interested participants without access to a computer and/or internet were provided with a laptop, tablet, or computer through the Medicaid agency where they received services. Participating parents and providers were randomized so that they either watched a 13minute, video-based presentation of the original Project ImPACT program, or the same 13minute, video-based presentation of the adapted Project ImPACT program. Videos included key information and visuals about the format, structure, and requirements of the relevant program. After watching the presentation, participating parents and providers completed an online questionnaire that assessed: 1) a range of demographic information; 2) perceptions of Project ImPACT attributes; 3) intent to use Project ImPACT; and 4) other relevant mental health and organizational characteristics. 42 Study 2 Participants Eighty-two parents of a child with an ASD diagnosis and 80 ASD community providers participated in Study 2. All participating parents had at least one child between the ages of 18 months and 8 years (M=4.98; SD=1.78), and all participating providers worked with children with ASD within the same age range. Fifty-four percent of participating parents were Medicaideligible based on income (i.e. family income below 133% of federal poverty line) and 42.20% had household incomes and household sizes that placed them below the federal poverty line (i.e. family income and size below 100% of the federal poverty line; U.S. Department of Health and Human Services, 2016). Parents were an average age of 33.78 years (SD=5.78) and were 79.40% female. Participating providers were from a variety of professional backgrounds, were 81.30% female, and were 86.30% White/European American. 52.40% of participating providers reported serving families who were Medicaid-eligible based on income. All participant demographic information, by study group, is presented in Tables 5 and 6. 43 Table 5. Study 2 Parent Characteristics Characteristic Gender (% female) Age in years Number of household members Medicaid by income At or below poverty line Group Adapted Original Percent (N=40) Mean (SD) Percent (N=42) Mean (SD) 78.80% 80.20% 34.13 (5.62) 33.57 (6.19) 4.24 (1.20) 4.44 (1.16) 48.80% 59.10% 36.60% 45.50% Marital status Married/Living with Partner Single, Widowed, Divorced 75.60% 24.40% 81.80% 18.20% Ethnicity White/Caucasian Black/African American Hispanic/Latino Middle Eastern Asian/Pacific Islander Biracial/Other 56.10% 14.60% 9.80% 7.30% 7.30% 4.90% 54.50% 22.70% 9.10% 4.50% 2.30% 6.80% Education Less than High School High School Degree Some Specialized Training Bachelor’s Degree Graduate Degree 7.30% 14.60% 34.10% 31.70% 12.20% 9.10% 9.10% 31.80% 36.40% 13.60% Annual Household Income <$10,000 $10,001-$15,000 $15,001-$20,000 $20,001-$25,000 $20,001-$25,000 $25,001-30,000 $30,001-$35,000 $35,001-$40,000 $40,001-$45,000 >$45,001 2.40% 12.20% 4.90% 7.30% 2.40% 12.20% 7.30% 2.40% 7.30% 41.60% 9.10% 11.40% 2.30% 9.10% 6.80% 4.50% 9.10% 4.50% 11.40% 31.80% Child with ASD Age in years 4.71 (1.73) 44 5.33 (1.92) Table 6. Study 2 Provider Characteristics Group Characteristic Original Percent (N=42) Gender (% female) Adapted Mean (SD) Percent (N=38) 78.90% 83.30% 37.13 (12.32) Age in years 34.79 (8.85) Medicaid families 41.70% 56.80% Ethnicity White/Caucasian Black/African American Hispanic/Latino Middle Eastern Asian/Pacific Islander Biracial/Other 88.10% 4.80% 7.10% 0% 0% 0% 84.20% 2.60% 5.30% 5.30% 2.60% Education High School Degree Some College/Specialized Training Bachelor’s Degree Graduate Degree (MA) Graduate Degree (PhD/MD) 7.10% 4.10% 19.00% 57.10% 11.90% 5.30% 7.90% 21.10% 63.20% 2.60% Occupation Speech Language Pathologist Special Education Teaching Social Worker Early Interventionist Behavior Technician/BCBA Psychologist Occupational Therapist 14.30% 11.90% 4.80% 7.10% 8.90% 14.30% 14.30% 15.80% 13.20% 15.80% 10.50% 13.20% 10.50% 18.40% Years experience with children with ASD < 1 years 1-3 years 4-6 years 7-10 years >10 years 7.10% 31.00% 19.00% 9.50% 33.30% 10.50% 23.70% 34.20% 23.70% 7.90% 45 Mean (SD) Video Stimuli Videos of the original and adapted Project ImPACT program were designed to be identical in format, visuals, length and the majority of word content. The primary differences between the two videos were in descriptions of the program delivery related to the major program adaptations noted in Table 4. The original and adapted video scripts, with major differences, are displayed in Appendices E and F. Both videos were: 1) 13 minutes, 30 seconds in length; 2) 1600 words in length; and 3) were dictated at roughly an eighth grade reading level as measured by the Flesch-Kincaid Grade Level. In addition to formatting, it was crucial to determine that the two videos were rated equally on measures of visual appeal and viewer comprehension. To do this, 108 undergraduate college students participated in a brief online study in which they were randomly assigned to watch either the original or adapted Project ImPACT video, following which they rated the video using a seven-point likert scale on the following dimensions: 1) clarity of the presentation material; 2) visual appeal; 3) general interest in video content; 4) confusion about the material presented; 5) attention/engagement in the presentation; and 6) whether the material was presented clearly. Results from one-way ANOVAs highlighted no group differences in video ratings (all p values > 0.40; all Cohen’s d values < 0.08). Measures All measures used in Study 2 are summarized in Table 7. Parent demographics. Parent demographic information was assessed using measures of parent age, gender, racial/ethnic background, highest educational degree obtained, marital status, annual household 46 income, occupation, partner occupation if applicable, primary language spoken, health insurance status, and number of household members. Child demographics. Child demographic information was assessed using measures of child age and gender. In addition, the Autism Behavior Checklist (ABC; Krug, Arick, & Almond, 1990) was used as a measure of child ASD symptom severity. The ABC consists of 57 questions that ask about behaviors that are specific to ASD, as well as other general behavior problems. Total scores were summed with higher total scores indicating more severe ASD symptoms. Cronbach’s alpha for the ABC demonstrated strong internal reliability for the present study (α=0.88). Parenting stress. The Parenting Stress Index-Short Form (PSI-SF; Abdin, 1995) was used as a measure of parenting stress. The PSI-SF is a 36-item self-report questionnaire that is frequently used within the ASD field to reflect the amount of stress that parents experience in their relationship with their child. Parents indicated the degree to which they agreed with each of the questionnaire statements using a 5-point likert scale. A high “Total Score” on the PSI-SF indicates significant stress in the parent-child relationship. The PSI-SF demonstrated strong internal reliability in the current study (α=0.85). Parent depression. The Center for Epidemiological Studies Depression Scale (CES-D; Radloff, 1977) is a 20-item self-report measure that asks parents’ to quantify their experience of depressive symptoms over the prior week using a 4-point likert scale. Higher total scores on the CES-D reflect higher levels of depressive symptoms, with a cutoff score of 16 indicating clinical levels 47 of depressive symptoms over the prior week. The CES-D demonstrated strong internal reliability in the current study (α=0.92). Perceived structural barriers. The 20-item “stressors and obstacles scale” of the Barriers to Treatment Participation Scale (BTPS; Kazdin et al., 1997) was used to assess perceived barriers to participating in Project ImPACT. The BTPS is a 44-item checklist of perceived barriers to participation in parent-mediated intervention programs. Parents rated each of the 20 external stressors and obstacles using a 5-point likert scale. Higher total scores reflect a greater number of structural barriers likely to impede engagement in parent-mediated intervention programs. The BTPS demonstrated strong internal reliability in the current study (α=0.86). Provider demographics. Provider demographic information was assessed using measures of provider age, racial/ethnic background, highest educational degree obtained, professional role, and practice setting. Provider clientele. Information about providers’ ASD clientele was assessed using measures of average client age, years of experience working with children with ASD, number of children with ASD on caseload, insurance status of clients, and average income of families on caseload. Implementation leadership. The Implementation Leadership Scale (ILS; Aarons, Ehrhart; & Farahnak; 2014) is a 12item scale that assesses an organization’s leadership capacity noted to influence the adoption of health innovations. Participating providers rated how much they agreed with each of the 12-items about their organizational leader using a 5-point likert scale. Total scores were summed to reflect 48 greater organizational leadership. The ILS demonstrated strong internal reliability in the current study (α=0.93). Provider general attitudes. The Evidence-Based Practices Attitude Scale (EBPAS; Aarons, 2004) is a 15-item questionnaire that assesses providers’ attitudes towards the adoption of manualized EBPs. The EBPAS is a measure that has been frequently used as a predictor of the likelihood that providers will adopt novel, evidence-based health practices. Participating providers rated each of the 15items using a 5-point likert scale. Total scores were summed with greater scores representing greater openness to trying novel, manualized treatments. Cronbach’s alpha for the EBPAS was α=0.81. Perceived workplace barriers. Providers’ perceived barriers to using Project ImPACT was assessed using a 19-item barriers checklist that has been used to examine structural barriers to provider use of the Triple P - Positive Parenting Program (Turner & Sanders, 2006). The Triple P barriers checklist was modified so that the items were specific to Project ImPACT. Providers rated each of the 19 barriers using a 5-point likert scale. Barriers on the scale are related to: 1) workplace characteristics; 2) issues related to provider/program fit; and 3) challenges in program management. This measure had strong internal reliability (α=0.87) in the present study. Project ImPACT attributes. Parents and providers perception of Project ImPACT attributes was assessed using Moore and Benbasat’s (1991) measure of an innovation’s attributes specific to Roger’s theory (Lewis et al., 2015; Moore & Benbasat, 1991). The original measure contained 41 items that asked about attributes specific to Roger’s theory (i.e. compatibility, complexity and relative advantage). For 49 the purpose of this study, the original measure was shortened and modified so that items were specific to Project ImPACT. The 14-item short form included 5 items each specific to Project ImPACT compatibility and complexity, and 4 items specific to Project ImPACT relative advantage. Participants rated how much they agreed with each item using a 7-point likert scale, with maximum scores equaling 98. The modified measure had strong internal reliability for both participating parents (α=0.91) and providers (α=0.94). In order to determine the preliminary psychometric properties of the modified Project ImPACT Attributes scale, a principal component analysis (PCA) was conducted on the 14 items separately for parents and providers. Oblique, promax rotations were used given the underlying assumption that the three hypothesized factors (i.e. compatibility, complexity, and relative advantage) might correlate. The Kaiser-Meyer-Olkin measure demonstrated that the sample size for both parents and providers was adequate to perform factor analysis; KMO=0.84; 0.87 respectively. Additionally, Bartlett’s test of sphericity indicated that correlations amongst items were sufficiently large for both parent ratings, X2=608.19, p<0.001, and provider ratings, X2=846.68, p<0.001. Initial analyses from parent item ratings indicated that four components had eigenvalues greater than 1 that explained 70.74% of the variance in total Project ImPACT attitude scores. However, after rotation, nearly all items loaded distinctly onto the first factor, with only a handful of items loading onto the remaining three in a non-specific manner. Similarly, for providers, only two components had eigenvalues greater than 1 that explained 65.58% of the variance in total Project ImPACT attitude scores. After rotation, the majority of items loaded onto the first factor, with two items loading minimally (<0.45) onto the second factor. Given these findings, all subsequent analyses used total scores on the Project ImPACT Attributes scale, rather than scores from each of the hypothesized sub-scales. 50 Intent to use. Parents’ and providers’ intent to use Project ImPACT was assessed using the question, “How likely is it that you would use the Project ImPACT program if it were made available to you?” Participants were asked to give a percent (ranging from 0-100) likelihood that they would use the program (Scott et al., 2008). Open-ended feedback. All participants responded to the following open-ended questions, “what are some things that you like about the Project ImPACT program?” and, “what are any concerns that you have about the Project ImPACT program?" 51 Table 7. List of Study 2 Measures Construct Measure Participant Parent Measures Parent Demographics Child Demographics Parent Mental Health Parent Structural Barriers Age Race/Ethnicity, Spoken Language SES: Income, Education, Household Size, Occupation Child Age ASD Symptom Severity: ABC (Krug et al., 1991) Parenting Stress: PSI-SF (Abdin, 1995) Depressive Symptoms: CES-D (Radloff, 1977) BTPS (Kazdin et al., 1997) Parent Parent Parent Parent Parent Parent Parent Parent Provider Measures Implementation Climate General Attitudes Age Race/Ethnicity; Education Professional Role; Years Experience, Clientele ILS (Aarons, Erhart; & Farahnak; 2014) EBPAS (Aarons, 2004) Provider Provider Provider Provider Provider Provider Workplace Barriers Triple P Barriers Checklist (Turner & Sanders, 2006) Provider Modified Attributes Scale (Moore & Benbasat, 1991) Project ImPACT Intent to Use Scale Positive and Negative Program Feedback Parent, Provider Parent, Provider Parent, Provider Provider Demographics All Participants Project ImPACT Attributes Intent to use Project ImPACT Qualitative Feedback 52 Data Analysis Independent sample t-tests were used to examine possible group differences in parent age, parent income, parenting stress, parent depressive symptoms, and child ASD symptom severity across parents in the original and adapted video groups. No statistically significant group differences were observed, with all p values > 0.53. Similarly, for providers, no statistically significant differences were observed across groups regarding provider age, education level, experience working with children with ASD, reported implementation climate, and general attitudes towards evidence-based practices, with all p values > 0.49. Given these results, no variables were co-varied out in subsequent analyses. Additional bivariate correlations were run to examine the relationship between key parent and provider variables. For parents, intent to use was significantly correlated with program attribute ratings (r=0.60), perceived structural barriers (r=-0.29), and child symptom severity (r=0.22). Parents’ perceived structural barriers were also associated with ratings of program attributes (r=-0.27) and parent income (r=0.39). For providers, similar relationships were noted, with intent to use significantly associated with perceived program attributes (r=0.67), workplace barriers (r=-0.33), and provider education (r=0.23). To determine whether adaptations to Project ImPACT influenced parents’ and providers’ perceptions of program attributes (i.e. total score of program compatibility, complexity, and relative advantage), perceived structural barriers to program use, and their intent to use the program, two-way, between-subjects ANOVAs were run to examine the impact of program type (original vs. adapted), parent income status (i.e. qualifying versus not qualifying for Medicaid insurance), and their interaction on perceptions of program attributes and intent to use either program. In addition to quantitative analysis, qualitative methods were used to examine 53 differences across participants’ open-ended responses regarding the strengths and weaknesses of the original versus adapted Project ImPACT programs. Specifically, Miles and Huberman’s (1994) approach to qualitative analysis was used to analyze open-ended responses within the same framework that was used to analyze focus group data in Study 1 (Pickard, Kilgore, & Ingersoll, 2016). Prior to analyzing data, all responses were uploaded into Dedoose and categorized by whether the response was from a parent versus provider, and from a participant in the original versus adapted video group. After doing so, “first-cycle” codes were categorized as they related to: 1) program compatibility; 2) program complexity; 3) relative advantage, 4) perceived structural barriers; and 5) other positive or negative comments related to the program. Reliability was ensured by using consensus coding, in which two investigators collaboratively coded each response, both blind to participant groupings. Relational and variational sampling was used to compare the percent of parents and providers who reported codes across the original and adapted programs as a means to compliment the quantitative data analysis (Boyatzis, 1998). Results Parent Ratings of Program Attributes A two-way, between subjects ANOVA was run to examine the impact of program type (i.e. original versus adapted), parent SES (i.e. qualifying for Medicaid by income), and their interaction on parents’ total ratings of the perceived compatibility, complexity and relative advantage of Project ImPACT (i.e. summed across items on Roger’s attributes scale). Results from the two-way ANOVA are displayed in Table 8 and plotted in Figure 1. Total attribute scores had equal variance across groups, F(3,78)=0.40, p=0.75. There was no main effect of program type or Medicaid status on parents’ ratings of Project ImPACT’s total compatibility, complexity, and relative advantage. However, there was a significant interaction, F(1,78)=4.48, 54 p=0.03, ηp2=0.05, such that Medicaid-eligible parents rated the attributes of the adapted program more favorably than parents from higher SES backgrounds (d=0.56, p=0.04). Although it also appeared that higher SES parents rated the original program more favorably than Medicaid families, this follow-up comparison was not significantly significant (d=0.42; p=0.13). In general, power for the analysis was low (power=0.55), and indicated that a sample size of N=120 would be necessary to detect the noted effect with power=0.80. Figure 1. Ratings of Program Attributes (Range 56-98) Parent ratings of program attributes by program type and Medicaid status 90 85 80 Medicaid Parents 75 Non-Medicaid Parents 70 65 Original Project ImPACT Program Adapted Project ImPACT Program Parent Ratings of Perceived Structural Barriers A two-way, between subjects ANOVA was run to examine the impact of program type, parent SES (i.e. Medicaid status), and their interaction on parents’ total perceived structural barriers to using Project ImPACT (i.e. total scores on the BTPS). Results are displayed in Table 8 and plotted in Figure 2. Total perceived structural barriers scores had equal variance across groups, F(3,78)=1.78, p=0.16. There was a significant main effect of parent SES, F(1,78)=15.74, 55 p<0.001, ηp2=0.17, such that parents from low SES background reported significantly fewer perceived structural barriers to using Project ImPACT. Additionally, there was a marginal main effect of program type, F(1,78)=2.58, p=0.10, ηp2=0.03, such that parents across both SES groups tended to rate the adapted program as having fewer structural barriers than the original program. There was no interaction between parent SES and program type. Figure 2. Perceived Structural Barriers (Range: 18-64) Parent ratings of structural barriers by program type and Medicaid status 65 60 55 50 45 40 Medicaid Parents 35 Non-Medicaid Parents 30 25 20 Original Project ImPACT Program Adapted Project ImPACT Program Parent Ratings of Intent to Use A two-way, between subjects ANOVA was run to examine the impact of program type, parent SES (i.e. Medicaid status), and their interaction on parents’ intent to use Project ImPACT. Parents’ ratings of intent to use were negatively skewed such that parents across all groups reported relatively high intent to use Project ImPACT (M=85.60; SD=15.13). Given this, a square root transformation was used to normalize the distribution of intent to use ratings. Results from a two-way ANOVA demonstrated a marginally significant effect of parent income, such that Medicaid-eligible parents tended to report higher intent to use across both programs, 56 F(1,78)=2.97, p=0.08, ηp2=0.04. Additionally, there was a marginally significant interaction, such that parents from low SES, Medicaid backgrounds reported greater intent to use the adapted Project ImPACT program than higher SES parents, F(1,78)=2.43, p=0.11, ηp2=0.03. Power for the analysis was low for both effects (power=0.45), and indicated that a sample size of N=150 would be necessary to detect the noted effects with power=0.80. Figure 3. Ratings of Intent to Use (Range" 44100) Parent ratings of intent to use by program type and Medicaid status 100 95 90 85 80 75 Medicaid Parents 70 Non-Medicaid Parents 65 60 55 50 Original Project ImPACT Program Adapted Project ImPACT Program 57 Table 8. Parent Results From Two-Way, Between-Subjects ANOVAs Total Attributes Medicaid Parents Original Adapted Mean (SD) Mean (SD) 73.60 (8.75) 79.44 (10.42) Structural Barriers 37.65 (6.43) 33.84 (11.53) Intent to Use 84.35 (13.80) 89.32 (12.49) Outcome Measure Non-Medicaid Parents Original Adapted Mean (SD) Mean (SD) 77.35 (8.91) 74.41(8.47) 45.50 (10.13) 42.58 (7.67) 84.95 (19.71) 80.88 (15.32) 58 2.58 (0.03) Effects SES F(ηp2) 0.10 (0.00) 15.74 (0.17)** 0.02 (0.00) 2.97 (0.04) Program F(ηp2) 0.49 (0.01) Program x SES F(ηp2) 4.48 (0.05)* 0.05 (0.00) 2.43 (0.03) Parent Qualitative Feedback Participating parents in both program groups responded to two open-ended questions regarding aspects of Project ImPACT that they liked and aspects that concerned them. In order to be consistent with the framework that was used to adapt the Project ImPACT program (Pickard, Kilgore, & Ingersoll, 2016), parent responses were categorized into the following domains: 1) aspects of the program that were compatible or not compatible with parents’ beliefs, values, and lifestyle; 2) comments or concerns related to the complexity of the program material; 3) the relative advantage or disadvantage of Project ImPACT compared to other services presently being used; 4) perceived structural barriers to using Project ImPACT; and 5) other positive or negative aspects of the program that did not fall into one of the pre-existing domains. Parent responses, by program type, are displayed in Table 9. Many parents in both groups spontaneously reported a number of aspects of Project ImPACT that made it compatible with their beliefs, values and lifestyle. Parents in the original and adapted Project ImPACT groups reported equally liking that the program attempts to empower parents by supporting them in using a set of strategies to better engage and teach their child. [I like] that it would teach me how to play with my child with Autism. I am very much like that mom shown in the first video who struggles to play with her child. – Original parent The focus is to help parents gain knowledge on how to enter the child's world and communicate with them. Everyone benefits, especially the child. – Adapted parent I like the parent teaching component. It's something that parents can feel empowered to do and do when the therapists are not around....It's a way for the parent to connect to their child as well. – Adapted parent 59 Similarly, parents reported with about equal frequency that both the original and adapted Project ImPACT programs were compatible with their lifestyle given the naturalistic focus of the intervention. In particular, parents voiced liking that the program could be embedded into the activities and routines that they were already doing with their child. [The program is] simple, easy to learn, easy implementation, does not require a lot of time, can be done anywhere. – Original parent It looks as if Project ImPACT really helps parents to get more involved with helping their child to learn new social skills. I think that it would fit right into our lives and that it would be of great benefit to me and my children. We're really struggling now with some communication and behavioral issues, and I would definitely welcome an opportunity to work with this project. It looks as if it would make our lives easier and our time together more productive. – Adapted parent Although a percentage of parents in both Project ImPACT groups reported that the program goals were important to them, parents who viewed the adapted program were more likely to indicate that the social communication and behavioral goals were strengths of the program. Finally, regarding program compatibility, parents who watched the adapted Project ImPACT program were both more likely to indicate that the required practice time was feasible and less likely to voice concern about setting aside time to practice the learned intervention techniques at home. With 3 kids, it will/would be almost impossible to set aside 20-30 minutes each day for just 1 child. It would be even harder for me because my husband is usually at work and it is me by myself with all 3 kids at night. – Original parent I like that you can integrate [the program] into your daily routine without having to set aside a lot of extra time to do the therapy. I also think it's helpful to have a visual reminders and homework on one page. – Adapted parent In their open-ended responses, many parents reported themes regarding the complexity of the Project ImPACT program. At least a quarter of parents in both groups reported that the intervention techniques were taught in a way that would be relatively easy to learn, and few 60 parents spontaneously reported concerns regarding their ability to learn the program. However, across responses, parents who viewed the adapted Project ImPACT program were more likely to comment on the ease of learning the program and less likely (i.e. did not at all) to report concern regarding the program’s complexity. I like its simplicity. – Original parent It seems fairly easy to learn. I also like that there are visual reminders and practice plans – Adapted parent I’m worried about how long it would take me to learn the techniques. – Original parent A handful of parents in both the original and adapted Project ImPACT groups spontaneously voiced the relative advantage of Project ImPACT when compared to the other services that they were currently using for their child with ASD, often referencing that the program would make the services that they were receiving better. However, when asked what concerned them about the program, parents in both groups reported perceived structural barriers to using Project ImPACT. A small number of parents in both groups voiced concern that scheduling the program appointments would be difficult due their family’s hectic schedule of work, outside therapy, and other activities. Additionally, a number of parents in both program groups raised concerns over the possible cost of the program, emphasizing that it would be difficult to access the program if it were not covered by insurance. [I’m concerned about] the possible cost to parents if it's not covered by private insurance. This is similar to a program that came to me and my son prior to him being enrolled in government insurance and I had to save money to pay for it. – Original parent Cost! Is this covered by my insurance? – Adapted parent 61 Finally, in their open-ended responses, parents mentioned several other themes that did not directly map onto the Roger’s attributes or perceived structural barriers to program participation. For example, when asked about their concerns for the program, over a quarter of parents in the original video group and well over half of parents in the adapted group reported that they had no concerns regarding the Project ImPACT program. Additionally, 1 to 2 families in each group emphasized that the program would only be beneficial when delivered by highquality therapists who were empathic and well-trained in the program content. Finally, parents in the adapted video group spontaneously reported that they felt strategies to involve siblings and the extended family in the program was greatly beneficial. Like with any program we've done my concern is the quality of the therapists. – Original parent I like that it involves both the child and the parent. I think it would also be easy for other children to help too, which is important. – Adapted parent 62 Table 9. Percent of Parents who Reported Each Theme by Program Type. Positive Comments Original Adapted Compatibility of Project ImPACT Positive perception of program goals Parent learning and empowerment Required practice time is feasible Naturalistic practice in daily routines Original Adapted 2.70% 27.03% 5.26% 15.79% Challenging to learn techniques 8.10% 0% Structural Barriers Location Scheduling barriers Cost/Insurance Coverage 2.70% 2.70% 10.81% 2.63% 2.63% 5.26% Other No concerns about program Quality of therapists 32.43% 2.70% 63.16% 2.63% Compatibility of Project ImPACT 8.10% 45.95% 2.70% 18.92% 21.05% 42.11% 7.89% 15.79% Complexity of Project ImPACT Ease of learning techniques Manual appropriate for parent learning 24.31% 0% 36.84% 2.63% Relative Advantage Would enhance other services 2.70% 5.26% Other Strategies for sibling involvement Concerns 0% 5.26% 63 Age limitation of program Required practice time not feasible Complexity of Project ImPACT Provider Ratings of Program Attributes A two-way, between subjects ANOVA was run to examine the impact of program type, caseload Medicaid status (i.e. Medicaid vs. non-Medicaid caseload), and their interaction on providers’ total ratings of the perceived compatibility, complexity and relative advantage of Project ImPACT (i.e. summed across items on Rogers’ scale). Results from the two-way ANOVA are displayed in Table 10 and plotted in Figure 4. Total attribute scores had equal variance across groups, F(3,76)=0.44, p=0.72. There was no main effect of program type or caseload SES on providers’ ratings of Project ImPACT’s total compatibility, complexity, and relative advantage. There was also no interaction effect between program type and caseload SES, F(1,76)=0.70, p=0.41, ηp2=0.01. Figure 4. Provider ratings of program attributes by program type and caseload SES Ratings of Program Attributes (Range 100 95 90 85 80 Medicaid Caseload 75 Non-Medicaid Caseload 70 65 60 Original Project ImPACT Program Adapted Project ImPACT Program 64 Provider Ratings of Intent to Use A two-way, between subjects ANOVA was run to examine the impact of program type, caseload SES (i.e. family Medicaid status), and their interaction on providers’ total ratings of their intent to use Project ImPACT. Results from the two-way ANOVA are displayed in Table 10 and plotted in Figure 5. Total intent to use scores had equal variance across groups, F(3,76)=1.94, p=0.13. There was no main effect of program type or caseload SES on providers’ ratings of their intent to use Project ImPACT. There was also no significant interaction between providers’ reported caseload SES and program type, F(1,76)=1.61, p=0.21, ηp2=0.02. Figure 5. Ratings of Intent to Use (Range: 47-100) Provider ratings of intent to use by program type and caseload SES 100 95 90 85 80 75 70 65 60 55 50 Medicaid Caseload Non-Medicaid Caseload Unadapted Project ImPACT Program Adapted Project ImPACT Program 65 Table 10. Provider Results From Two-Way, Between-Subjects ANOVAs Outcome Measure Total Attributes Intent to Use Providers: Medicaid Caseload Original Adapted Mean (SD) Mean (SD) 79.38 79.76 (13.49) (10.86) 84.72 89.10 (18.48) (11.36) Providers: Non-Medicaid Caseload Original Adapted Mean (SD) Mean (SD) 75.76 79.96 (12.21) (11.83) 86.18 89.58 (11.86) (12.38) 66 Effects Program F(ηp2) 0.49 (0.00) 0.03 (0.00) SES F(ηp2) Program x SES F(ηp2) 0.39 (0.00) 0.70 (0.01) 0.10 (0.00) 1.61 (0.02) Provider Qualitative Feedback Participating providers in both groups responded to two open-ended questions regarding aspects of Project ImPACT that they liked and that concerned them. Like parents, provider responses were grouped into the following domains: 1) aspects of the program that were compatible or not compatible with their beliefs, values, and service delivery model; 2) comments or concerns related to the complexity of the program material; 3) the relative advantage or disadvantage of delivering Project ImPACT as compared to other services; 4) structural barriers or concerns related to using Project ImPACT with families; and 5) other positive or negative aspects of the program that did not fall into one of the pre-existing domains. Provider responses, by program type, are displayed in Table 11. Providers in both program groups indicated that a number of aspects of Project ImPACT made the program compatible with their beliefs, values, and service delivery model. These aspects included active parent involvement, the social communication goals targeted by the program, and the naturalistic and behavioral focus of the intervention. Providers in the original and adapted Project ImPACT groups reported equally that a strength of the program was that it actively involved parents by supporting them in using a set of strategies to better engage and teach their child. Providers often emphasized that this type of program delivery was associated with parent empowerment. I like the way the program is set up and how parents are given tools to help increase interaction with their children with autism. – Original provider It shows that there is not an expert needed to implement. It is empowering to parents. – Adapted provider I like that the parents get to be involved in learning strategies, that way the interventionist is not the only person working with the child and the parent can be happier with the results they see knowing that they helped play a part in their child's learning.– Adapted provider 67 Similarly, about a quarter of providers in both Project ImPACT groups spontaneously voiced the importance of the program’s emphasis on improving the core social communication deficits of ASD in a manner that was naturalistic and that could be embedded into the daily routines that parents were already doing with their child. The main thing is the connection with parents and the natural environment and around natural day-to-day activities. What are we trying to accomplish if not helping kids with ASD connect with the people around them and have a more rich experience in their dayto-day life? – Original provider I enjoy that Project ImPACT is parent-mediated, provides coaching to caregivers and is easily embedded into daily routines without requiring specific stimuli for implementation. – Original provider Well, it’s great that the project includes helping kids with essential components like social-communication skills during daily routines and activities and places where they learn naturally. – Adapted provider Providers in both the original and adapted Project ImPACT groups referenced the feasibility of having parents practice intervention techniques as part of the program. However, providers who watched the adapted Project ImPACT program were more likely to emphasize the feasibility and less likely to report concern regarding the practice requirements for parents. Similarly, providers in the adapted group were less likely to report concern regarding the fit of Project ImPACT within their service delivery model. I like that the written homework was removed. I think it's a lot more accessible to our parents now. I also really appreciate that it can be used during routines, which is really accessible to the busy families we work with. –Adapted parent It does not appear overbearing in terms of time commitment from either the parent or provider side. – Adapted provider Can be done in the home, on the family's own time - Don't need to buy lots of materials (inexpensive). – Adapted provider I’m concerned about getting the time commitment from parents to have those meaningful interactions with their child for 20-30 minutes/day. – Original provider 68 I do not have enough time in my schedule to work with parents twice a week for 60-90 minutes. I am school-based and provide in-home services for children 0-3 years old for 60 minutes per week. – Original provider Similar differences occurred regarding providers’ report of the program complexity. Although providers in both program groups spontaneously reported that the program would be relatively straightforward for parents to use, providers who viewed the adapted program were more likely to voice positive perceptions of Project ImPACT’s complexity. Specifically, these providers voiced more positive perceptions regarding the ease of parents learning the intervention techniques, the number of techniques taught to parents, and positive perceptions of the program manual for teaching parents. Providers in the adapted group were also less likely to report that the manual was too complex for parents to learn from it. I like that it [the manual] has visuals for the parents and is very systematic so that parents can more easily follow and understand things step by step – Adapted provider I like the parent coaching and the simplistic lesson plans. It seems very straightforward and easy to follow. The educational levels of most of the parents of my clients are low which can be challenging. It seems Project ImPACT took this into consideration. – Adapted provider [I’m concerned about] the accessibility of the written materials/homework for families with low literacy. – Original provider [I’m concerned about] the ability for some of my cognitively impaired parents or those with multiple children to do the homework. – Original provider Several providers in the adapted Project ImPACT group reported the relative advantage of the program when compared to the other services that they were currently delivering. These providers noted that the program was both inexpensive for families and would make the services that they were delivering better. 69 Our current model seeks to encourage children to engage with objects (attention span/non-preferred tasks, plays skills, etc). I have long felt that children with ASD need to start engaging LESS with objects and more with caregivers. – Adapted provider When asked what concerned them about the program, a couple of providers in the original Project ImPACT group noted that transportation, difficulty managing the home environment, and the cost of program manuals for families were all perceived structural barriers to using Project ImPACT. Importantly, a larger percentage of providers in both program groups reported that a significant barrier to using Project ImPACT was the time and cost associated with becoming trained in the intervention. I am concerned about the time it would take to learn a new program. – Original provider As a provider at a CMH [Community Mental Health], a barrier we have is paying for training. I believe that the cost is necessary and completely understandable, but difficult to swing with a very tight budget. – Adapted provider Unsure if I will be able to commit the time to become trained – Adapted provider Finally, in their open-ended responses, providers reported several other themes that did not directly map onto the Roger’s attributes or perceived structural barriers to Project ImPACT use. Specifically, when asked about their concerns about Project ImPACT, providers who watched the adapted program were more likely to report a lack of concern. Although a couple of providers in each group reported that a strength of Project ImPACT was the emphasis on collaboration across parents and providers, nearly a third of providers in each group reported that a major concern would the difficulty engaging parents in the program material. Some parents aren't engaged and don't want to participate in therapy. – Original provider In some lower SES communities, it would likely be harder to get full parent participation due to their [parent] schedules, priorities, and beliefs. – Original parent 70 Poor attendance/inconsistency of the families I support (frequent cancellations or noshows), which would limit the ability to move through the curriculum and keep parents invested. – Adapted provider It can be rather difficult to get underprivileged families involved. – Adapted provider 71 Table 11. Percent of Providers who Reported Each Theme by Program Type. Positive Comments Compatibility of Project ImPACT Positive perception of program goals Parent learning and empowerment Required practice time is feasible Naturalistic practice in daily routines Complexity of Project ImPACT Ease of learning techniques Feasible number of techniques taught Manual appropriate for parent learning Relative Advantage Better than other services Other Strategies for sibling involvement Parent-professional collaboration Original Adapted 26.32% 68.42% 0% 34.21% 23.53% 61.76% 8.82% 26.47% 13.16% 0% 0% 0% 0% 2.63% 35.19% 2.94% 17.65% 8.82% 2.94% 2.94% 72 Concerns Compatibility of Project ImPACT Goals not appropriate for all children Doesn’t fit into service delivery model Practice time not feasible for parents Original Adapted 0% 5.26% 34.21% 5.88% 0% 5.88% Complexity of Project ImPACT Not trained in parent training Manual too complex for parent learning 0% 10.53% 2.94% 0% Structural Barriers Difficulty managing home environment Difficulty getting trained in program Transportation Cost of manuals for families 2.63% 7.89% 2.63% 5.26% 0% 11.76% 0% 0% Other None Hard to engage parents 23.68% 31.58% 32.35% 29.41% Discussion The primary goal of the current study was to understand the impact of adaptations made to an evidence-based, parent-mediated intervention, Project ImPACT, in collaboration with families and providers in a Medicaid system. To do this, parents’ and providers’ ratings of Project ImPACT attributes, perceived structural barriers, and intent to use the program were compared based on the specific program that participants viewed (i.e. original versus adapted), participant SES (i.e. qualifying for Medicaid insurance versus not), and their interaction. Following quantitative analyses, qualitative analysis was used to better understand differences in parents’ and providers’ open-ended feedback regarding either the original or adapted Project ImPACT program. For providers specifically, there was no main effect of program type or caseload SES on ratings of program attributes or intent to use. Although there was a trend in that providers with Medicaid caseloads reported greater intent to use for the adapted Project ImPACT program, this interaction was not statistically significant. For parents, more noticeable quantitative differences were observed. Specifically, there was a significant interaction by which lower SES parents rated the attributes of the adapted Project ImPACT program more favorably. Although parents with Medicaid insurance also tended to report greater intent to use across both programs and the adapted program specifically, both of these findings were underpowered and not significant. Finally, there was a main effect of SES and a marginal main effect of program type such that parents with Medicaid insurance reported substantially fewer perceived barriers across both programs, and all parents, regardless of SES, tended to report fewer perceived barriers to participating in the adapted Project ImPACT program. 73 Open-ended responses were used to supplement and triangulate quantitative comparisons (Palinkas et al., 2011). By using Rogers’ Diffusion of Innovations theory (Rogers, 2003) as a framework for qualitative analysis, it was possible to understand how participant responses reflected the specific program attributes and feedback that had guided the initial adaptations to Project ImPACT (Pickard, Kilgore, & Ingersoll, 2016). In their responses, the majority of parents and providers reported that both the original and adapted Project ImPACT programs were compatible with their lifestyle and/or style of clinical work. Many participants indicated that the naturalistic focus of Project ImPACT and its emphasis on empowering parents through explicit teaching were strengths of the program. Parents who viewed the adapted program, specifically, were more likely to indicate that the required practice time of the program was feasible, that the program materials would facilitate learning, and that the explicit inclusion of other family members was important to them. Similar group differences were observed for providers, who were less likely to report concern regarding the program’s practice time and overall complexity when they viewed the adapted program. Finally, for both the original and adapted programs, continued concern was reported regarding insurance coverage, therapist quality, and the challenges of engaging parents from lower SES backgrounds. Implications Within the ASD field, there has been recent emphasis on using research-community partnerships to improve the fit and relevance of evidence-based practices, including parentmediated interventions, for use within a variety of culturally diverse and underserved community settings (Brookman-Frazee et al., 2012; Divan et al., 2015; Drahota et al., 2012; Drahota et al., 2016; Drmic et al., 2017; Ratto et al., 2016; Stahmer et al., 2017; Stahmer & Pellecchia, 2015; Wright et al., 2014). In fact, research from other behavioral health fields has demonstrated that 74 community-partnered research aimed at adapting parent-mediated interventions can result in higher rates of parent satisfaction and retention when these programs are delivered across a variety of community settings (Baumann et al., 2015; Griner & Smith, 2006). Despite emerging use of community-partnered research to adapted parent-mediated interventions within the ASD field (Divan et al., 2015; Ratto et al., 2016; Stahmer et al., 2016; Wright et al., 2014), only a small handful of studies have yet to be published on the impact of these adaptations (Drmic, Alijunied, & Reaven, 2017; Rahman et al., 2016). The current study used Rogers’ Diffusion of Innovations theory (Rogers, 2003) and Theory of Planned Behavior (Ajzen, 1991; Eccles, 2006; Webb & Sheeran, 2006) as an intermediate stage in which the impact of adaptations to Project ImPACT could be preliminarily tested, while also collecting continued feedback regarding the program adaptations. Results from this study highlighted small but noticeable differences in the way that participants quantitatively and qualitatively evaluated the original versus adapted Project ImPACT programs. Quantitatively, these differences appeared to be most robust for parents with Medicaid insurance, who tended to rate the adapted program more favorably. Although small, these findings may have implications regarding the generalizability of adapted intervention programs. For example, given that the parent sample in the present study included low-SES, Medicaid families both from within and outside our initial partnership, our findings may suggest that the positive impact of the program adaptations can generalize to other low-SES families outside of metro-Detroit. On the other hand, the interaction itself also suggests that the effects of the program adaptations were only meaningful for low-SES parents. The lack of universal improvements in parent ratings of the adapted Project ImPACT program is important and may highlight one limitation of adapting evidence-based practices for specific populations. 75 While quantitative results demonstrated small but meaningful differences in parents’ ratings of the two programs, qualitative results expanded these findings. In the absence of specific probes, participants’ open-ended feedback indicated group differences in perceptions of program attributes that were reflective of the initial adaptations to Project ImPACT (Pickard, Kilgore, & Ingersoll, 2016). Parents and providers qualitatively reported that the adapted program was both more compatible and less complex than the original Project ImPACT program. The consistent differences found across methodologies are important, as these differences in perceptions and attitudes (i.e. rooted in Roger’s attributes) have been shown to facilitate the adoption of novel health innovations (Henggler et al., 2008; Rogers, 2003; Scott et al., 2008). Therefore, when implemented in low-SES community settings specifically, the results of the present study suggest that the adapted Project ImPACT program may have higher rates of adoption than the original program. This is a crucial implication given the noted rates of attrition when parent-mediated intervention programs for ASD have been implemented within lower SES settings (Kasari et al., 2014). One important finding in the present study was related to differences in parents’ perceived barriers to Project ImPACT participation, with all parents reporting fewer perceived barriers for the adapted Project ImPACT program, and parents from low SES backgrounds reporting substantially fewer perceived barriers regardless of program type. Although small, the marginal main effect of program type is promising, as it suggests that the majority of parents tend to perceive fewer barriers to participating in the adapted program. Given that perceived structural barriers predict parent-mediated program engagement (Kazdin & Holland, 1997; Nock & Kazdin, 2005), this finding would suggest better adoption and use of the adapted program across parents. On the other hand, the finding that low-SES parents reported fewer barriers 76 across both programs contradicts the attrition rates noted within the ASD field (Kasari et al., 2014; Stahmer & Pellecchia, 2015), and the well-documented association between SES and perceived barriers to treatment use (Kazdin & Holland, 1997). This unexpected finding may be related to our conceptualization of low-SES as qualifying for Medicaid insurance. The recent passage of ASD Medicaid waivers across a variety of states, including Michigan, has greatly improved access to ASD services for families with Medicaid insurance (Leslie et al., 2017). Therefore, the smaller number of perceived barriers reported by low-SES families could be reflective of insurance status, rather than SES per se. An alternate explanation for this finding may be the experimental design used to assess perceived barriers to program use. Given that lowSES parents tend report that parent-mediated interventions are an unmet health need (Pickard & Ingersoll, 2015), their report of fewer perceived barriers may reflect a greater desire to access a program like Project ImPACT. Without engaging in the program, it may be challenging to assess structural barriers above and beyond a perceived need for the program. Finally, in addition to highlighting ways in which parents and providers viewed the adapted Project ImPACT program more positively, the current study also re-emphasized two key concerns related to using Project ImPACT within the Medicaid system. The first concern was rooted in the insurance coverage for Project ImPACT, with parents reporting that paying out of pocket for the program would be prohibitive. In general, this concern is consistent with other research within the ASD field that has demonstrated that cost is one barrier to participating in parent-mediated intervention programs (Carr et al., 2015; Kasari et al., 2014). Unfortunately, the qualitative analyses did not examine responses by participant SES, so it was not possible to know whether insurance type may have influenced parents’ report of this barrier. Recent research has demonstrated that families with Medicaid insurance have greater access to ASD services and 77 fewer out-of-pocket expenses (Leslie et al., 2017; Zhang & Baranek, 2016). Therefore, it may be possible that families with private insurance or no insurance were more likely to report insurance-related concerns for the program. Regardless, these findings highlight the continued difficulty for some families in being able to access insurance coverage for ASD services. A second notable barrier reported by many providers regarded the challenge of engaging low-SES parents in parent-mediated intervention programs. Although some providers attributed this difficulty to structural barriers outside of the family’s control, others explicitly referenced this difficulty as being rooted in parents’ motivation. Concern about parent engagement and motivation had been reported by many Medicaid providers during our initial community partnership, despite partnering parents’ emphasis on being motivated to use a program like Project ImPACT (Pickard, Kilgore, & Ingersoll, 2016). In the present study, parents across both programs also reported high intent to use Project ImPACT in the face of providers’ concern. These findings re-emphasize the importance of embedding both motivational strategies to support providers in engaging parents, in addition to strategies for providers to build strong, strength-based alliances with parents (Brookman-Frazee, 2004; Curits & Singh, 1996; Dunst & Dempsey, 2007; Stahmer & Pellecchia, 2015). Limitations There are several important limitations within this study. The first is related to the experimental manipulation used to test the impact of program adaptations and the sample it was used within. Although the program videos were designed to highlight key differences in program attributes, a large proportion of the videos were also identical given that the core components of the intervention were unchanged in the adaptation process. Moreover, given that parents and providers did not participate in the Project ImPACT program, all ratings were based on their 78 perceived experience of the program attributes. Taken together, the small effects that we observed may reflect the way in which the experimental videos were designed, as well as the hypothetical nature of participant “engagement” in the program. Another limitation that may have driven the smaller observed effects was the sample demographics for this second study. Given that the initial adaptations made to Project ImPACT were geared towards parents and providers within a Medicaid system, low-SES parents were conceptualized as those qualifying for Medicaid insurance in the present study. However, this Medicaid sample was generally more resourced and White than that of the sample for which Project ImPACT was initially adapted (Pickard, Kilgore, & Ingersoll, 2016). Therefore, despite showing that the adaptations impacted parents’ ratings of program attributes and their intent to use the program across all Medicaid parents, the smaller effects may also be reflective of a more resourced sample. In fact, when a small secondary analysis was run to test the effects of the adapted program for a subset of parents with equivalent demographics to that of the initial sample, the effect of program adaptations increased substantially for both ratings of program attributes (d=0.96) and intent to use the program (d=0.54). This finding is crucial to consider and may, again, reflect the more targeted impact of the program adaptations. Two final limitations relate to the scale that was used to measure parents’ and providers’ ratings of Roger’s attributes. Although the scale used in the present study was adapted from a well-cited measure, the subscales reported in the original measure (i.e. compatibility, complexity, and relative advantage; Moore & Benbasat, 1991) were not confirmed in the present data set. Instead, many of the items loaded onto a single scale and, thus, scores were summed across all items to represent a “general attributes” scale. Although representative of parents’ and providers’ attitudes towards the program broadly, it was not possible to quantitatively understand specific 79 attributes on which the two programs may have differed. Use of a more psychometrically-sound scale would have permitted this more nuanced quantitative analysis. Finally, only three of Rogers’ attributes were examined within the present study, while two others were not (i.e. trialability and observability). The choice to focus on compatibility, complexity, and relative advantage was based on the focus of our initial community partnership (Pickard, et al., 2016), as well as research to demonstrate that these three attributes are most associated with program adoption and use (Dingfelder & Mandell, 2010; Talor & Todd, 1995; Tornatzky & Klein, 1982). However, use of an all-encompassing scale, such as that developed by Cooke, Thompson, & Schnurr (2015), would allow for a better understanding of how the two programs differed across a greater number of attributes. Future Directions Future directions include continued data collection in order to increase the power to detect the reported quantitative effects. Given that the findings from the present study were promising but also limited by the experimental manipulation, future directions also include piloting the adapted Project ImPACT program within the Medicaid system as a way to better understand the perceived attributes and structural barriers to actual program participation and use. An essential piece of piloting the adapted program would also be collecting both initial effectiveness data in addition to data regarding attitudes towards specific program attributes and other key variables noted to impact the eventual dissemination and implementation of Project ImPACT within the Medicaid system (Curran et al., 2012). Finally, the present study highlighted the utility of Theory of Planned Behavior by demonstrating small differences in parents and providers intentions to use the two Project ImPACT programs. However, this study could not fully test Theory of Planned Behavior (i.e. 80 TPB; Ajzen, 1991) by showing that participant intentions to using Project ImPACT predicted their actual uptake and use of the program. Future research should examine TPB in full by examining whether individual intentions indeed mediate the relationship between key variables (i.e. individual attitudes, perceived social norms and self-efficacy; Ajzen, 1991), and use of Project ImPACT. By fully understanding TPB and the variables that predict intention and use of Project ImPACT, it will be possible to understand specific constructs that should be adapted and/or enhanced to increase intentions and use of the program. 81 CONCLUDING REMARKS In summary, high rates of attrition in parent-mediated intervention programs within the ASD field have underscored the need to engage underserved communities around improving the relevance and fit of these programs (Stahmer & Pellecchia, 2015). Although more ASD researchers are using research-community partnerships to improve the fit of a variety of evidence-based practices for culturally diverse and underserved community settings (Divan et al., 2015; Ratto et al., 2016; Stahmer et al., 2017), the present two studies are some of the first to both understand and attempt to address these issues utilizing the same theoretical framework. Taken together, both of these studies suggest that Rogers’ Diffusion of Innovations theory can be used to both reliably adapt parent-mediated intervention programs, in addition to estimating the preliminary impact of these adaptations on rates of program adoption and use. Outside of the reported group differences in this study, the universally high ratings of intent to use and qualitative report of the importance of interventions like Project ImPACT are crucial, and emphasize the need to better engage community partners in order to more efficiently disseminate parent-mediated programs to parents across a variety of community settings. 82 APPENDICES 83 Appendix A Script for Study 1 Video of Original Program Today you will be learning about Project ImPACT, an evidence-based intervention developed for young children with autism. Project ImPACT stands for “Improving Parents as Communication Teachers.” Project ImPACT is a parent-mediated intervention, which means that the parents are taught to use intervention strategies to improve their child’s development. Why use parent-mediated interventions like Project ImPACT for children with autism? Parent-mediated interventions, like Project ImPACT, are an evidence-based practice for children with autism. This means that research has shown that parent-mediated interventions are effective for improving outcomes for children with ASD and their families. What children with autism learn from their family can be more important than what they learn in school or in a clinic. Research has shown that parents are able to learn intervention strategies like the ones used in Project ImPACT to teach their child with autism important social communication skills. Because parents use these strategies throughout their child’s day, their child receives more hours of intervention over the course of the week within the routines that matter most. This increased amount of intervention within family routines helps children with autism develop better social engagement, better language, better play and better imitations skills. Finally, parents who use interventions like Project ImPACT with their child often feel less stressed, more optimistic and feel like they are better able to help their child. This is a video example of one child who received the Project ImPACT intervention. This is a video of the child at the beginning of the Project ImPACT program, at the end of the Project ImPACT program, and a few years after the Project ImPACT program was complete. What is Project ImPACT? There are 4 important aspects of Project ImPACT: First, Project ImPACT is naturalistic. This means that it is used during play and other activities that are part of the family’s daily routine. Naturalistic interventions, like Project ImPACT, lead to better generalization and maintenance of skills, meaning that a child is more likely to use learned skills across settings and remember these skills over time. These types of intervention are also better for teaching spontaneous skills, or skills that a child can use on their own, without help from a parent or therapist. Also, parents usually prefer naturalistic interventions like Project ImPACT because they lead to more positive family interactions. The second important aspect of Project ImPACT is that typical development guides treatment goals. Project ImPACT teaches social communication skills. All children learn these skills in 84 the roughly the same order, whether or not they have autism. Since children are better able to learn skills that fit where they are in their development, Project ImPACT uses the child’s current developmental level as a guide to help parents choose meaningful social communication goals for their child. What are social communication skills? The Project ImPACT program targets four sets of social communication skills that children with autism often struggle to learn. Each of these skill sets is important for improving children’s overall social communication development. The first set of skills is social engagement. Children with autism often have difficulty remaining actively engaged with others, and sharing their emotions using eye contact and facial expressions. Project ImPACT helps the child stay engaged and interact more effectively with their parents. The second set of skills is Language. Children with autism often have difficulty understanding and using verbal and nonverbal language to communicate with others. Project ImPACT helps improve the child’s ability to understand and flexibly communicate with their parents using verbal and nonverbal language. The third set of skills is imitation. Children with autism tend to have difficulty imitating other people’s vocalizations, body movements, gestures, actions with objects, and language. Project ImPACT helps improve the child’s ability to imitate their parents spontaneously during play. The fourth and final set of skills that are targeted by the Project ImPACT program is play skills Children with autism often have difficulty playing with toys in an appropriate way. Project ImPACT helps increase the child’s interest in toys as well as ability come up with new and different ways to play. Let’s go back to talking more about what is Project ImPACT”. The third important aspect of Project ImPACT is that social communication develops within emotionally-rich interactions with responsive caregivers. Parents who are more responsive to their child with autism are able to teach their child more social communication skills. Interventions like Project ImPACT that increase parent responsiveness lead to improvements in children’s social interactions. The last important thing to know about Project ImPACT is that teaching techniques are based on Applied Behavior Analysis or ABA. Intervention techniques that are grounded in ABA, like Project ImPACT, are effective for teaching a variety of skills to children with autism. In particular, naturalistic ABA interventions like Project ImPACT can improve children’s joint attention, engagement, play, language, and imitation. Therefore, Project ImPACT teaches parents ABA techniques to help their child learn new social communication skills. 85 What will you be required to do if you participate in Project ImPACT? While parents are learning Project ImPACT, they will be asked to set aside 20-30 minutes each day to practice the intervention one-on-one with their child with autism. This will require parents to structure at least one area of their home so that there are minimal distractions and they can practice the intervention undisturbed with their child. Parents can practice Project ImPACT with their child in many different daily routines, like playtime, mealtime, story time, bath time, and a dressing routine. The Project ImPACT program is 12 weeks long. During these 12 weeks, parents and therapists meet twice per week for 60-90 minutes. During these sessions, the parent, the therapist, and the child with ASD are present. One parent is required to be at each session, but other caregivers are also able to join as needed. These sessions can occur within a home or clinic-based setting. Parents are provided with a written Project ImPACT manual, which contains all of the written materials and homework for the Project ImPACT program. What are the Project ImPACT sessions like? During the first few sessions, the therapist will explain the program to the parent and answer any of their questions. Within these first few sessions, the parent and therapist work together to set goals for the child. Setting goals for a child is an important part of the Project ImPACT program. In order to set appropriate goals for a child, the parent and therapist first determine the child’s current skill level using the Social Communication Checklist. The social communication checklist helps the parent and therapist compare notes and make sure they are one the same page about the social communication skills that the child already has, and the skills that the child should learn next. After doing this, parents and therapists work together to set goals for the child in social engagement, language, imitation, and play. Children with autism usually have deficits in these four areas, which means that they are important skills to teach. Once the parent and therapist set goals, the therapist will begin to teach parents each of the Project ImPACT intervention strategies. The intervention strategies are spread out so that parents learn one to two intervention strategies per week. When teaching begins, parents are asked to read about each intervention strategy before they meet with the therapist. These readings are taken from the parent manual. During the sessions, the therapist will first explain the foals of the session to the parent. They will next review the homework that the parent had completed before the session. After reviewing the homework, the therapist will explain the intervention strategy to the parent. The therapist will then show the parent how to use the intervention strategy, by modeling it with the child. Next, the therapist will help the parent practice the intervention strategy with their child. The therapist will give the parent both positive and constructive feedback regarding their use of the intervention technique. After practicing, the therapist answers any of the parent’s questions and assign them homework. 86 This homework requires that parents practice the intervention strategy for 20-30 minutes each day with their child at home. After practicing, parents complete a set of written questions that ask about their time practicing the intervention. This homework is reviewed during the next session with the therapist. This is a brief example of what a Project ImPACT coaching session could look like. During the program, parents will learn how to use the Project ImPACT intervention techniques across a number of different routines with their child. By the end of the program, these intervention techniques should be a natural part of a family’s life. 87 Appendix B Focus Group Interview: Parents 1.) From what you saw in the presentation, what stood out to you about the Project ImPACT program? What did you like? What did you dislike? 2.) What concerns or barriers do you foresee in using the Project ImPACT program? Probe 1: How does the length of the program feel to you? (If concerns are raised) What could we do to address this? Probe 2: How do you feel about the time commitment of the program (i.e. 20-30 minutes per day of practice)? (If concerns are raised) What could we do to address this? Probe 3: Would you have any challenges making sure that you have time to practice the intervention with your child? Probe 4: Would you have any challenges related to other children or family in the home? (If so) What could we do to address this? Probe 5: What other challenges could you see coming up if you needed to use the program? 3.) What are some other services that you are currently using for your child? How does Project ImPACT compare to these services? Probe 1: Is it better or worse than these other services? Probe 2: Would Project ImPACT make the services you are using better or worse? 4.) What are your thoughts about the goals of Project ImPACT? How do they fit or not fit with the goals that you have for your child and your family? Probe 1: Are there goals that you feel are missing from the program? What about goals that are part of the program but feel unnecessary? Probe 2: How would you feel about learning the intervention techniques yourself? Would you feel comfortable being your child’s therapist? 5.) How easy do you think the Project ImPACT intervention would be to learn? Probe 1: How easy or hard would Project ImPACT be to learn? Why? Probe 2: What would make Project ImPACT easier to learn? What is your learning style? How could you learn the intervention techniques best? Probe 3: Would you feel comfortable being coached by a therapist? Who would be a good coach to you? 6.) How useful do you think the Project ImPACT program is for your child and their needs? Do you think the program would help your child? Probe 1: If so, how might it? If not, why not? 7.) From what you saw in the presentation, would this be a program that you would want to try? Probe 1: Why or why not? 88 8.) What changes might you want to see made to the Project ImPACT program in order to make the program better for you? Relate back to the concerns brought up. 89 Appendix C Focus Group Interview: Providers 1.) From what you saw in the presentation, what stood out to you about the Project ImPACT program? What did you like? What did you dislike? 2.) What concerns or barriers do you foresee in using the Project ImPACT program? Probe 1: How does the length of the program feel to you? Probe 2: Are there any concerns about the time commitment required of parents (i.e. 30 60 minutes per week)? Probe 3: What challenges might arise within the home setting? Related to other family members? Probe 4: What challenges might come up related to scheduling with families? 3.) What are some of the other services that you are currently delivering? How does Project ImPACT compare to these other services that you typically provide? Probe 1: Does the program seem better or worse than the other services that you are currently providing to children with autism? Why? Probe 2: Would using Project ImPACT alongside your other services make them better or worse? 4.) What are your thoughts about the goals of Project ImPACT? How do they fit or not fit with the goals that you have for your clients? Probe 1: Do you believe that parents should be trained to use intervention? Probe 2: Are the goals relevant for the children on your caseload? Probe 3: Are the children on your caseload good fits for the program? Probe 4: Are there goals that you feel are missing rom the program or goals that are part of the program that you feel are unnecessary? 5.) How simple and easy do you think the Project ImPACT intervention will be to learn? Probe 1: Do you think parents would be able to learn the intervention techniques? Probe 2: Do you think they would want to learn the intervention techniques? Probe 3: What do you think would be the best way to teach parents? Probe 4: How do you feel about training parents to learn the intervention techniques? Would that be easy or hard? Do you have background in parent training? 6.) How useful do you think the Project ImPACT program is for the children and families on your caseload? Do you think the program would help the children and families on your caseload? 7.) From what you saw in the presentation, would this be a program that you would want to try? 8.) What changes might you want to see made to Project ImPACT in order to make the program better for you? 90 Appendix D Example of Coded Transcript Excerpt Both comments together coded as “practice time not feasible” Both comments together coded as “materials too complex for parent learning” All comments coded as: “difficulty managing home environment” 91 Appendix E Example of Content Analytic Table Table 12. List of Codes as they Relate to Site and Rogers’ Theory Provider Site 1 Provider Site 2 Provider Site 3 Compatibility Practice time spread (5); Practice time too intense (5); Preference for routines (6); Like program (1); Want to try program (2); Practice time spread (1); Practice time too intense (2); Preference for routines (3); Like existing goals (3); Like program (2); Want to try program (5); Meet more frequently (2); Program longer than 12 weeks (2); Practice time spread (4); Preference for routines (1); Want to try program (2); Like existing goals (1); Meet more frequently (1); Program longer than 12 weeks (1) Complexity Homework too challenging (7); Language of program too complex (4); Need for more visuals (3); Need for observation (2) Homework too challenging (4); Language of program too complex (13); Prioritize techniques (4); Need for more visuals (1); Need for observation (1); Parent learning through coaching (1) Homework too challenging (1); Language of program too complex (3); Parent learning through coaching (1); Need for observation (1) Relative Advantage Option for group format (2); Insurance (2) Insurance coverage (5); Option for group format (3) Option for group support (2); Structural Barriers Resources-other (5); Mental health resources (3); Toys within home (1); Room without distractions (3); Manage other siblings (2); Translation (5) Managing siblings (1); Lack of toys/materials (5); Room without distractions (2); Resources-other (7);Translation (3) Resources-other (4); Lack of toys/materials (2); Room without distractions (2); Translation (2) Suggestions Behavior problems (4); Sibling implementation (2); Daily living skills (2); Psychoeducation/Motivation (8) Family support (2); Sibling implementation (3); Need for therapist alliance (3); Psychoeducation/Motivation (1) Behavior problems (3); Sibling implementation (3); Need for therapist alliance (2); Psychoeducation/Motivation (1); Daily living skills (2); Potty training (2); Family implementation (1) 92 Appendix F Script for Study 2 Video of Original Program Today you will be learning about Project ImPACT, an intervention for young children with autism and their parents. Project ImPACT stands for “Improving Parents as Communication Teachers.” Project ImPACT is a parent-mediated intervention, which means that the parents are taught to use intervention strategies to improve their child’s social communication skills. The Project ImPACT program is “evidence-based,” which means it has been researched and found to help children improve their social communication skills. Many teachers and therapists use the Project ImPACT techniques when they work with children with autism, and research shows that parents can learn them just as well to help their child. How Will Project ImPACT help parents and children with autism? There are lots of great reasons to learn and use Project ImPACT:  When parents learn the Project ImPACT techniques, they can teach their child throughout the day. Their child will get to learn during important activities like mealtime, bath time, and bedtime. Parents can help their child use their skills where they matter most and this will help the child remember them better.  Also, when parents use the Project ImPACT techniques with their child, their child will get more hours of intervention. Research shows that more hours of intervention can help a child have better language, play and engagement, and fewer behavior problems.  Another great thing is that parents who participate in programs like Project ImPACT say that they feel less stressed, have more family time, and have more hope. Parents who have participated in this program tell us that it has helped their children communicate and has made interacting with them more fun. Next you will see a video of a parent playing with their child with autism. This is a video of the parent and child playing before participating in Project ImPACT and while they were being trained to use the Project ImPACT intervention techniques. Notice the difference in how the mom is playing with the child, and the how the child is better able to communicate and enjoy playing near her. What is Project ImPACT? There are 4 important aspects of Project ImPACT: First, Project ImPACT is naturalistic. This means that it is used during play and other activities that are part of the family’s daily routine. Naturalistic interventions, like Project ImPACT, lead to better generalization and maintenance of skills, meaning that a child is more likely to use learned skills across settings and remember these skills over time. (CLICK) These types of intervention are also better for teaching spontaneous skills, or skills that a child can use on their own, without help from a parent or therapist. Also, parents usually prefer naturalistic interventions like Project ImPACT because they lead to more positive family interactions. The second important aspect of Project ImPACT is that typical development guides treatment goals. Project ImPACT teaches social communication skills. All children learn these skills in the 93 roughly the same order, whether or not they have autism. Since children are better able to learn skills that fit where they are in their development, Project ImPACT uses the child’s current developmental level as a guide to help parents choose meaningful social communication goals for their child. What are the Project ImPACT goals? Project ImPACT teaches four main social communication skills, all of which are important building blocks for learning:  The first is Social engagement: Children share their emotions and attention with others through eye contact, body language, gestures, and words. When a child is engaged, they are more likely to learn. The starting point of Project ImPACT is to help increase a child’s social engagement with their parent.  The second goal of Project ImPACT is Communication: Children communicate using nonverbal skills, like eye contact and gestures, before they use verbal language, such as words and sentences. Project ImPACT will help parents teach their child to communicate better using gestures, words, or sentences, depending on where their child is at.  The third goal of Project ImPACT is Imitation: Children use imitation to show interest in other people and to learn new skills. This program will help children imitate during play to improve their social engagement, play skills, and gestures.  The final goal of Project ImPACT is Play: Children develop many skills during play, such as language, problem solving, imagination, perspective-taking, and motor skills. This program will help parents teach their child how to play more creatively. Let’s go back to talking more about the 4 core aspects of Project ImPACT. The third important aspect of Project ImPACT is that children learn social communication from emotionally-rich interactions with parents. Parents who are more responsive to their child with autism are able to teach their child more social communication skills. Project ImPACT increases parent responsiveness as a means to improve their child’s social interactions. The last important thing to know about Project ImPACT is that teaching techniques are based on Applied Behavior Analysis or ABA. Intervention techniques that are grounded in ABA, like Project ImPACT, are effective for teaching a variety of skills to children with autism. In particular, naturalistic ABA interventions like Project ImPACT can improve children’s social engagement, language, imitation and play. Therefore, Project ImPACT teaches parents ABA techniques to help their child learn new social communication skills. What will parents and therapists do if they use Project ImPACT? Therapists will be required to support and coach parents in their learning and use of the intervention techniques. While parents are learning Project ImPACT, therapists will ask them to set aside 20-30 minutes each day to practice the intervention one-on-one with their child with autism. This will require parents to structure at least one area of their home so that there are minimal distractions and they can practice the intervention undisturbed with their child. Parents can practice Project ImPACT with their child in many different daily routines, like playtime, mealtime, story time, bath time, or a dressing routine. 94 The Project ImPACT program is 12 lessons. During the Project ImPACT program, parents and therapists meet twice per week for 60-90 minutes. During their sessions together, the parent, the therapist, and the child with autism are present. The same parent is required to be at each session, but other caregivers are also able to join as needed. The Project ImPACT sessions can occur within a home, at a clinic, or in a school. Parents are provided with a written Project ImPACT manual, shown here, which contains all of the written materials and homework for the Project ImPACT program. Therapists will also work from a corresponding manual that helps them support parents in learning Project ImPACT. What are the Project ImPACT sessions like? During the first few sessions, the therapist will explain the program to the parent and answer any of their questions. During this session, the parent and therapist work together to set goals for the child. Setting goals for a child is an important part of the Project ImPACT program. In order to set appropriate goals for a child, the parent and therapist first decide the child’s current skill level. The parent and therapist compare notes and make sure they are one the same page about the skills that the child already has, and the skills that the child should learn next. After doing this, the parent and therapist work together to set goals for the child in social engagement, communication, imitation, and play. Once the parent and therapist set goals, the therapist will begin to teach parents each of the 14 Project ImPACT intervention strategies. The intervention strategies are spread out so that parents learn one to two intervention strategies per week. When teaching begins, parents are asked to read about each intervention strategy before they meet with the therapist. These readings are taken from the parent manual and look like this. Pause. During the sessions, the therapist will first explain the goals of the session to the parent and answer any of their questions. They will next review the homework that the parent had completed before the session. After reviewing the homework, the therapist will explain the intervention strategy to the parent. The therapist will then show the parent how to use the intervention technique, by demonstrating it with their child. Next, the therapist will help the parent practice the intervention technique with their child. The therapist will give the parent both positive and constructive feedback. After practicing, the therapist will answer any of the parent’s questions and assign them homework, like that shown here. This homework requires that parents practice the intervention strategy for 20-30 minutes each day with their child at home. After practicing, parents complete a set of written questions that ask about their time practicing the intervention, and that ask them to reflect on what went well and what was more challenging. This homework is reviewed during the next session with the therapist. Next you will see an example of what a Project ImPACT coaching session could look like if it were done in a clinic. To quickly end, when using Project ImPACT therapists teach parents intervention techniques to improve their child’s social engagement, communication, imitation, and play. The program is 12 lessons long and teaches parents 14 different intervention techniques to use with their child across lots of different activities. By the end of the program, these intervention techniques should be a natural part of a family’s life. Thank you for taking the time to learn about Project ImPACT, we appreciate your help! 95 Appendix G Script for Study 2 Video of Adapted Program Today you will be learning about Project ImPACT, an intervention for young children with autism and their parents. Project ImPACT stands for “Improving Parents as Communication Teachers.” Project ImPACT does not teach parenting skills. Instead, it teaches parents to use special techniques to help their child develop social engagement, language, imitation, and play skills. These skills together are called social communication skills. The Project ImPACT program is “evidence-based,” which means it has been researched and found to help children improve their social communication skills. Many teachers and therapists use the Project ImPACT techniques when they work with children with autism, and research shows that parents can learn them just as well to help their child. How Will Project ImPACT help parents and children with autism? There are lots of great reasons to learn and use Project ImPACT  When parents learn the Project ImPACT techniques, they can teach their child throughout the day. Their child will get to learn during important activities like mealtime, bath time, and bedtime. Parents can help their child use their skills where they matter most and this will help the child remember them better.  Also, when parents use the Project ImPACT techniques with their child, their child will get more hours of intervention. Research shows that more hours of intervention can help a child have better language, play and engagement, and fewer behavior problems.  Another great thing is that parents who participate in programs like Project ImPACT say that they feel less stressed, have more family time, and have more hope. Parents who have participated in this program tell us that it has helped their children communicate and has made interacting with them more fun.  Finally, parents can also teach the Project ImPACT techniques to other family members, such as grandparents and siblings, so they can support you and your child too. Now you will see a video of a parent having a snack with their child with autism. This is a video of the parent and child before participating in Project ImPACT and while they were learning to use the Project ImPACT intervention techniques. Notice the difference in how the mom is interacting with the child, and the how the child is better able to communicate and enjoy being near her. What are the Project ImPACT goals? Project ImPACT teaches four main social communication skills, all of which are important building blocks for learning:  The first is social engagement: Children share their emotions and attention with others through eye contact, body language, gestures, and words. When a child is engaged, they are more likely to learn. The starting point of Project ImPACT is to help increase a child’s social engagement with their parent.  The second goal of Project ImPACT is communication: Children communicate using nonverbal skills, like eye contact and gestures, before they use verbal language, such as 96    words and sentences. Project ImPACT will help parents teach their child to communicate better using gestures, words, or sentences, depending on where their child is at. The third goal of Project ImPACT is Imitation: Children use imitation to show interest in other people and to learn new skills. This program will help children imitate during play to improve their social engagement, play skills, and gestures. The final goal of Project ImPACT is Play: Children develop many skills during play, such as language, problem solving, imagination, perspective-taking, and motor skills. This program will help parents teach their child how to play more creatively. Another important thing to know about Project ImPACT is that challenging behavior can be a goal if needed. Many children with autism have behavior problems such as tantrums and aggression that happen because they have trouble communicating. As their communication skills improve, their behavior problems often get better. However, some parents may find it helpful to learn some strategies to help their children with more serious behavior problems. What will parents and therapists do if they use Project ImPACT? Therapists support parents to help them learn and use the Project ImPACT techniques in a way that is best for the family. This requires that therapists are supportive and that they focus on a family’s strengths, that way they can build a strong relationship with families. While they are learning Project ImPACT, therapists will ask parents to set aside a few minutes here and there across their day to practice the intervention strategies with their child. Therapists will support parents and help them find the best times and locations for them to practice with as little distraction as possible. Parents can chose to practice Project ImPACT with their child in playtime or in routines that they already do like mealtime, story time, bath time, or a dressing routine. The Project ImPACT program is a 12 lesson program. During the Project ImPACT program, parents and therapists usually meet one or two times per week for about one hour. Usually, one lesson is taught each week, but some lessons can be repeated if needed During their sessions together, the parent, the therapist, and the child with autism are present. The same parent should be at each session, but other caregivers are also able to join as needed. In fact, the therapist will help parents get other family and siblings involved if they are interested The Project ImPACT sessions can occur at home, at a clinic, or in a school. Parents are provided with the materials they will need, including the Project ImPACT workbook, shown here, which uses pictures and visuals to help the parents learn. Therapists will also work from a corresponding manual that helps them support parents in learning Project ImPACT. What are the Project ImPACT sessions like? During the first session, the therapist will explain the program, and will get to know the parent and what’s important to them and their family. If parents need it, the therapist will be able to help the parents find other important resources, and can help the parents figure out how set up their home so that they are able to get the most out of the program. During the first session, the parent and therapist work together to set goals for the child. Setting goals for a child is an important part of the Project ImPACT program and is a team effort with the parent and therapist sharing their expertise with each other. During this time, the parent and therapist compare notes and make sure they are one the same page about the skills that the child already has, and the skills that the child should learn next. After doing this, the parent and therapist work together to set goals for 97 the child in social engagement, communication, imitation, and play. Once the parent and therapist set goals, the therapist will begin to teach parents each of the 5 main Project ImPACT strategies, shown here. For each of the main Project ImPACT strategies, therapists will help parents pick one or more specific techniques to use that work best for the parent and child. Parents learn about 1-2 techniques per lesson, and these techniques can be repeated if needed. When teaching begins, parents are asked to look at an overview of the lesson before they meet with the therapist. These overviews are visual and look like this. During each session, the therapist will first explain the goals of the session to the parent and answer any of their questions or concerns. They will next review and problem solve around how practice went for the parent the week before. After reviewing the practice, the therapist will explain the intervention technique to the parent. Parents are able to follow along with a visual worksheet in their parent workbook The therapist will then show the parent how to use the intervention technique, by demonstrating it with the child. They will ask the parent if they have any questions and can show them extra video if needed Next, the therapist will help the parent practice the intervention technique with their child. The therapist will give the parent positive feedback and feedback that will also help them use the technique better. After practicing, the therapist will answer any of the parent’s questions. The parent and therapist will work together to make a visual practice plan for parents over the next week at home, like that shown here. This practice plan helps parents pick some times and activities that work best for them to practice the techniques with their child over the next week. The therapist helps the parent to set alarm reminders on their phone if needed, and asks parents to think about how practice is going for them while they are doing it. This practice time is reviewed during the next session with the therapist. Now you will see an example of what a Project ImPACT coaching session could look like if it were done in a clinic. To quickly end, Project ImPACT teaches parents special techniques to teach their children social engagement, communication, imitation, play, and behavior too if needed. The program is flexible, can be used during the routines that parents already do with their child, and can help parents get other family members on board too. All teaching uses visuals to support parents, and therapists really get to know parents to make sure the program is the best fit with their goals and family. When a child gets many chances to use a skill during the day, they are more likely to use it in new situations and on their own. With the help of the therapist, parents will learn how to use the techniques in a way that works best for their family. By the end of the program, many parents report that the techniques have become part of their everyday lives. Thank you for taking the time to learn about Project ImPACT, we appreciate your help! 98 REFERENCES 99 REFERENCES Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6, 61-74. Aarons, G.A., Erhart, M.G., Farahnak, L.R. (2014). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9, 45-54. Abdin, R. R. (1990). Parenting Stress Index:(Short Form). Charlottesville, VA: Pediatric Psychology Press. Aldred, C., Green, J., & Adams, C. (2004). A new social communication intervention for children with autism: pilot randomised controlled treatment study suggesting effectiveness. Journal of Child Psychology and Psychiatry, 45, 1420-1430. Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing. Armitage, C. J., & Conner, M. (2001). Efficacy of the theory of planned behaviour: A meta analytic review. British Journal of Social Psychology, 40, 471-499. Axford, N., Lehtonen, M., Kaoukji, D., Tobin, K., & Berry, V. (2012). Engaging parents in parenting programs: Lessons from research and practice. Child and Youth Services Review, 34, 2061-2071. Baker-Ericzen, M., Hurlburt, M., Brookman-Frazee, L., Jenkins, M., & Hough, R.L. (2010). Comparing child, parent and family characteristics in usual care and EST research samples for children with disruptive behavior disorders. Journal of Emotional and Behavioral Disorders, 18, 82-99. Barnett, M., Brookman-Frazee, L., Regan, J., Saifan, D., Stadnick, N., & Lau, A. (2017). How intervention and implementation characteristics relate to community therapists’ attitudes toward evidence-based practices: A mixed methods study. Administration and Policy in Mental Health. Barrera, M., & Castro, F. G. (2006). A heuristic framework for the cultural adaptation of interventions. Clinical Psychology: Science and Practice, 13, 311-316. 100 Barrera, M.M., Castro, F.G., & Steiker, L.K.H. (2011). A critical analysis of approaches to the development of preventative interventions for subcultural groups. American Journal of Community Psychology, 48, 439-454. Baumann, A.A., et al. (2015). Cultural adaptation and implementation of evidence-based parent training: A systematic review and critique of guiding evidence. Children and Youth Services Review, 53, 113-120. Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research: Issues for cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology, 23, 67– 82. Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Sage. Brookman-Frazee, L., Stahmer, A.C., Lewis, K., Feder, J.D., & Reed, S. (2012). Building a research-community collaborative to improve community care for infants and toddlers at risk for autism spectrum disorders. Journal of Community Psychology, 40, 715-734. Carr, T., Shih, W., Lawton, K., Lord, C., King, B., & Kasari, C. (2015). The relationship between treatment attendance, adherence, and outcome in a caregiver-mediated intervention for low-resourced families of young children with autism spectrum disorder. Autism, 1362361315598634. Carr, T. & Lord, C. (2016). A pilot study promoting participation of families with limited resources in early autism intervention. Research in Autism Spectrum Disorders, 25, 87 96. Cook, J.M., Thompson, R., & Schnurr, P.P (2015). Perceived characteristics of intervention scale: Development and psychometric properties. Assessment, 22, 704-714. Cooke, J.R. & Kilmer, R.P. (2012). Systems of care: Partnerships for community psychology. American Journal of Community Psychology, 49, 393-403. Coolican, J., Smith, I. M., & Bryson, S. E. (2010). Brief parent training in pivotal response treatment for preschoolers with autism. Journal of Child Psychology and Psychiatry, 51, 1321-1330. Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50, 217-226. Curtis, W. J., & Singh, N. N. (1996). Family involvement and empowerment in mental health service provision for children with emotional and behavioral disorders. Journal of Child and Family Studies, 5, 503-517. 101 Davis, R., Campbell, R., Hildon, Z., Hobbs, L., & Michie, S. (2015). Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychology Review, 9, 323-344. Dingfelder, H.E. & Mandell, D.S. (2011). Bridging the research-to-practice gap in autism intervention: An application of diffusion of innovation theory. Journal of Autism and Developmental Disorders, 41, 597-609. Divan, G. et al. (2015). Adapting an evidence-based intervention for autism spectrum disorder for scaling up in resource-constrained settings: The development of the PASS intervention in South Asia. Global Health Action, 8, 27278. Drahota, A., Meza, R. D., Brikho, B., Naaf, M., Estabillo, J. A., Gomez, E. D., et al. (2016). Community‐ academic partnerships: A systematic review of the state of the literature and recommendations for future research. The Milbank Quarterly, 94, 163-214. Drmic, I. E., Aljunied, M., & Reaven, J. (2017). Feasibility, Acceptability and Preliminary Treatment Outcomes in a School-Based CBT Intervention Program for Adolescents with ASD and Anxiety in Singapore. Journal of Autism and Developmental Disorders, 1-21. Eccles, M. P., Hrisos, S., Francis, J., Kaner, E. F., Dickinson, H. O., Beyer, F., & Johnston, M. (2006). Do self-reported intentions predict clinicians' behaviour: a systematic review. Implementation Science, 1, 28. Estes, A. et al. (2014). The impact of parent-delivered intervention on parents of very young children with autism. Journal of Autism and Developmental Disorders, 44, 353-365. Glasgow, R.E., Lichtenstein, E., & Marcus, A.C. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health, 8, 1261-1267. Griner, D., & Smith, T. B. (2006). Culturally adapted mental health intervention: A metaanalytic review. Psychotherapy: Theory, Practice, Research, Training, 43, 531-548. Hayes, S.A. & Watson, S.L. (2013). The impact of parenting stress: A meta-analysis of studies comparing the experience of parenting stress in parents of children with and without autism spectrum disorder. Journal of Autism and Developmental Disorders, 43, 629-642. Henggeler, S. W., Chapman, J. E., Rowland, M. D., Halliday-Boykins, C. A., Randall, J., Shackelford, J., & Schoenwald, S. K. (2008). Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. Journal of Consulting and Clinical Psychology, 76, 556. Hilton et al. (2010). Brief report: Under-representation of African Africans in autism genetic research: A rationale for inclusion of subjects representing diverse family structures. Journal of Autism and Developmental Disorders, 40, 633-639. 102 Hodgetts, S., Zwaigenbaum, L., & Nichols, D. (2015). Profile and predictors of service needs for families of children with autism spectrum disorder. Autism, 19, 673-683. Hume, K., Bellini, S., & Pratt, C. (2005). The usage and perceived outcomes of early intervention and early childhood programs for young children with autism spectrum disorder. Topics in Early Childhood Special Education, 25, 195-207. Ingersoll, B., & Dvortcsak, A. (2010). Teaching social communication: A practitioner’s guide to parent training for children with autism. New York: Guilford Press. Ingersoll, B., & Wainer, A. (2013). Initial efficacy of Project ImPACT: A parent-mediated social communication intervention for young children with ASD. Journal of Autism and Developmental Disorders, 43, 2943-2952. Ingersoll, B., Wainer, A.L., Berger, N.I., Pickard, K.E., & Bonter, N. (2016). Comparison of a self-directed and therapist-assisted telehealth parent-mediated intervention for children with ASD: A pilot RCT. Journal of Autism and Developmental Disorders, 46, 2275 2284. Karst, J.S. & Van Hecke, A.V. (2012). Parent and family impact of autism spectrum disorder: A review and proposed model for intervention evaluation. Clinical Child and Family Psychology Review, 15, 247-277. Kasari, C. et al. (2014). Caregiver-mediated intervention for low-resourced preschoolers with autism: An RCT. Pediatrics, 134, e72-e79. Kazdin, A.E., Holland, L., & Crowley, M. (1997). Family experience of barriers to treatment and premature termination from child therapy. Journal of Consulting and Clinical Psychology, 65, 453-463. Kazdin, A. E., Holland, L., Crowley, M., & Breton, S. (1997). Barriers to treatment participation scale: Evaluation and validation in the context of child outpatient treatment. Journal of Child Psychology and Psychiatry, 38, 1051-1062. Knudson, H.K., & Roman, P.R. (2015). Innovation attributes and adoption decisions: Perspectives from leaders of a national sample of addiction treatment organizations. Journal of Substance Abuse Treatment, 47, 1-7. Leslie, D. L., Iskandarani, K., Dick, A. W., Mandell, D. S., Yu, H., Velott, D., et al. (2017). The effects of Medicaid home and community-based services waivers on unmet needs among children with Autism Spectrum Disorder. Medical Care, 55, 57-63. Lewis, C.C., Fischer, S., Weiner, B.J., Stanick, C., Kim, M., & Martinez, R.G. (2015). Outcomes for implementation science: An enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science, 10, 155-171. 103 Liao, H. & Lu, H. (2008). The role of experience and innovation characteristics in the adoption and continued use of e-learning websites. Computers & Education, 51, 1405-1416. Maglione, M. A., Gans, D., Das, L., Timbie, J., & Kasari, C. (2012). Nonmedical interventions for children with ASD: Recommended guidelines and further research needs. Pediatrics, 130, S169-S178. Mandell, D.S., Novak, M.M., & Zubritsky, C.D. (2005). Factors associated with age of diagnosis among children with autism spectrum disorders. Pediatrics, 116, 1480-1486. McConachie, H., & Diggle, T. (2007). Parent implemented early intervention for young children with autism spectrum disorder: a systematic review. Journal of Evaluation in Clinical Practice, 13, 120-129. Medicaid.gov (2016). Medicaid and CHIP in Michigan. Retrieved at: https://www.medicaid.gov/medicaid/by-state/stateprofile.html?state=michigan Mejia, A., Ulph, F., & Calam, R. (2016). Exploration of mechanisms behind changes after participation in a parenting intervention: A qualitative study in a low-resource setting. American Journal of Community Psychology, 0, 1-9. Merton, R. K. (1987). The focused interview and focus groups: Continuities and discontinuities. Public Opinion Quarterly, 550-566. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Sage. Moore, G.C., & Benbasat, I. (1991). Development of an instrument to measure the perceived characteristics of adopting information technology innovation. Information Systems Research, 2, 192-222. Moore, G. C., & Benbasat, I. (1996). Integrating diffusion of innovations and theory of reasoned action models to predict utilization of information technology by end-users. In Diffusion and Adoption of Information Technology (pp. 132-146). Springer, US. Nguyen, C.T., Krakowiak, P., Hansen, R., Hertz-Picciotto, I., & Angkustsiri, K. (2016). Sociodemographic disparities in intervention service utilization in families of children with autism spectrum disorder. Journal of Autism and Developmental Disorders, Online First September 17, 2016. Nilsen, P. (2015). Making sense of implementation theories, models, and frameworks. Implementation Science, 10, 1-13. 104 Nock, M.K. & Kazdin, A.E. (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology, 5, 872-879. Odom, S.L., Collet-Klingenberg, L., Rogers, S.L., & Hatton, D.D. (2010). Evidence-based practices in interventions for children and youth with autism spectrum disorders. Preventing School Failure, 54, 275-282. Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38, 44-53. Palmer-Wackerly, A.L., Krok, J., Dailey, P.M., Kight, L., & Krieger, J.L. (2014). Community engagement as a process and an outcome of developing culturally grounded health communication interventions: An example from the DECIDE project. American Journal of Community Psychology, 53, 261-274. Pellicano, E., Dinsmore, A., & Charman, T. (2014). What should autism research focus upon? Community views and priorities from the United Kingdom. Autism, 18, 756-770. Patten, E., Baranek, G.T., Watson, L.R., & Schultz, B. (2012). Characteristics influencing intervention choices in autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 20, 1-9. Pickard, K., Wainer, A., Bailey, K., & Ingersoll, B. (2016). A mixed-method evaluation of the feasibility and acceptability of a telehealth-based parent-mediated intervention for children with ASD. Autism. Pickard & Ingersoll (2015). Quality versus quantity: The role of SES on parent-reported service knowledge, service use, unmet service needs and barriers to service use. Autism, 1362361315569745. Pickard, K. E., Kilgore, A. N., & Ingersoll, B. R. (2016). Using Community Partnerships to Better Understand the Barriers to Using an Evidence‐ Based, Parent‐ Mediated Intervention for Autism Spectrum Disorder in a Medicaid System. American Journal of Community Psychology, 57, 391-403. Pierce, N. P., O’Reilly, M. F., Sorrells, A. M., Fragale, C. L., White, P. J., Aguilar, J. M., & Cole, H. A. (2014). Ethnicity reporting practices for empirical research in three autism related journals. Journal of Autism and Developmental Disorders, 44, 1507-1519. Quine, L., Rutter, D. R., & Arnold, L. (2001). Persuading school‐ age cyclists to use safety helmets: Effectiveness of an intervention based on the Theory of Planned Behaviour. British Journal of Health Psychology, 6, 327-345. 105 Radloff, L. S. (1977). The CES-D scale a self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385-401. Rahman, A., Divan, G., Hamdani, S. U., Vajaratkar, V., Taylor, C., Leadbitter, K., et al. (2016). Effectiveness of the parent-mediated intervention for children with autism spectrum disorder in south Asia in India and Pakistan (PASS): A randomised controlled trial. The Lancet Psychiatry, 3, 128-136. Ratto, A. B., Anthony, B. J., Pugliese, C., Mendez, R., Safer-Lichtenstein, J., Dudley, K., et al. (2016). Lessons learned: Engaging culturally diverse families in neurodevelopmental disorders intervention research. Autism, 1362361316650394. Reichow, B., Volkmar, F. R., & Cicchetti, D. V. (2008). Development of the evaluative method or evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders, 38, 1311-1319. Robertson, R.E. Sobeck, E.E., Wynkoop, K., & Schwartz, R. (2017). Participant diversity in special education research: Parent-implemented behavior interventions for children with autism. Remedial and Special Education, 1-13. Rogers, E.M. (2003). Diffusion of innovations (5th ed.). New York: Free Press. Rogers, E. M. (2002). Diffusion of preventive innovations. Addictive Behaviors, 27, 989-993. Rogers, E.M. (1995). Diffusion of Innovations (4th ed.). New York: Free Press. Rogers, S. J., Vismara, L., Wagner, A. L., McCormick, C., Young, G., & Ozonoff, S. (2014). Autism treatment in the first year of life: a pilot study of infant start, a parent implemented intervention for symptomatic infants. Journal of Autism and Developmental Disorders, 44, 2981-2995. Sandler, I., Ostrom, A., Bitner, M. J., Ayers, T. S., Wolchik, S., & Daniels, V. S. (2005). Developing effective prevention services for the real world: A prevention service development model. American Journal of Community Psychology, 35, 127-142. Schoenwald, S.K. & Hoagwood, K. (2001). Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services, 52, 1190-1197. Schreibman, L., et al. (2015). Naturalistic developmental behavioral interventions: Empirically validated treatments for autism spectrum disorder. Journal of Autism and Developmental Disorders, 45, 2411-2428. Schwarzer, R., Schüz, B., Ziegelmann, J. P., Lippke, S., Luszczynska, A., & Scholz, U. (2007). Adoption and maintenance of four health behaviors: Theory-guided longitudinal studies on dental flossing, seat belt use, dietary behavior, and physical activity. Annals of Behavioral Medicine, 33, 156-166. 106 Scott, S.D., Plotnikoff, R.C., Karunamuni, N., Bize, R., & Rodgers, W. (2008). Factors influencing the adoption of an innovation: An examination of the uptake of the Canadian Heart Health Kit (HHK). Implementation Science, 3, 41-49. Sheeran, P. (2002). Intention—behavior relations: A conceptual and empirical review. European Review of Social Psychology, 12, 1-36. Sheeran, P., & Orbell, S. (2000). Using implementation intentions to increase attendance for cervical cancer screening. Health Psychology, 19, 283-289. Smith, T., Scahill, L., Dawson, G., Guthrie, D., Lord, C., Odom, S., Rogers, S., & Wagner, A. (2007). Designing research studies on psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 37, 354-366. Srinath, S., & Jacob, P. (2016). Challenges in parent-mediated training in autism spectrum disorder. The Lancet Psychiatry, 32, 93-95. Stadnick, N. A., Drahota, A., & Brookman-Frazee, L. (2013). Parent perspectives of an evidence-based intervention for children with autism served in community mental health clinics. Journal of Child and Family Studies, 22, 414-422. Stadnick, N.A., Stahmer, A., & Brookman-Frazee, L. (2015). Preliminary effectiveness of Project ImPACT: A parent-mediated intervention for children with autism spectrum disorder delivered in a community program. Journal of Autism and Developmental Disorders, 45, 2092-2104. Stahmer, A. C., Aranbarri, A., Drahota, A., & Rieth, S. (2017). Toward a more collaborative research culture: Extending translational science from research to community and back again. Autism, 21, 259-261. Stahmer, A. C., Brookman-Frazee, L., Rieth, S. R., Stoner, J. T., Feder, J. D., Searcy, K., & Wang, T. (2016). Parent perceptions of an adapted evidence-based practice for toddlers with autism in a community setting. Autism, 1-14. Stahmer, A. C., & Pellecchia, M. (2015). Moving towards a more ecologically valid model of parent-implemented interventions in autism. Autism, 19, 259-261. Tek, S., & Landa, R. J. (2012). Differences in autism symptoms between minority and non minority toddlers. Journal of Autism and Developmental Disorders, 42, 1967-1973. Thomas, K.C., Morrissey, J.P., & McLaurin, C. (2007). Use of autism-related services by families and children. Journal of Autism and Developmental Disorders, 37, 818-829. Tonge, B., Brereton, A., Kiomall, M., Mackinnon, A., King, N., & Rinehart, N. (2006). Effects on parental mental health of an education and skills training program for parents of young 107 children with autism: a randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 45, 561-569. Tornatzky, L. G., & Klein, K. J. (1982). Innovation characteristics and innovation adoption implementation: A meta-analysis of findings. IEEE Transactions on Engineering Management, 1, 28-45. Trickett, E. J. (2011). From “water boiling in a Peruvian town” to “letting them die”: Culture, community intervention, and the metabolic balance between patience and zeal. American Journal of Community Psychology, 47, 58-68. Turner, K. & Sanders, M. (2006). Dissemination of evidence-based parenting and family support strategies: Learning from the Triple P-Positive Parenting Program system approach. Aggression and Violent Behavior, 11, 176-193. US Department of Health and Human Services. 2016 HHS Poverty Guidelines. Available at: https://www.uscis.gov/sites/default/files/files/form/i-864p.pdf. Accessed March 27, 2016. Virués-Ortega, J. (2010). Applied behavior analytic intervention for autism in early childhood: Meta-analysis, meta regressions and dose-response meta-analysis of multiple outcomes. Clinical Psychology Review, 30, 387-399. Wandersman, A. (2003). Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology, 41, 171181. Webb, T. L., & Sheeran, P. (2006). Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132, 249. Wetherby, A. M., Guthrie, W., Woods, J., Schatschneider, C., Holland, R. D., Morgan, L., & Lord, C. (2014). Parent-implemented social intervention for toddlers with autism: an RCT. Pediatrics, 134, 1084-1093. Williams, N. J., Glisson, C., Hemmelgarn, A., & Green, P. (2016). Mechanisms of change in the ARC Organizational Strategy: Increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Administration and Policy in Mental Health and Mental Health Services Research, 1-15. Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A., Kucharczyk, S., et al. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951-1966. Zablotsky, B., Black, L. I., Maenner, M. J., Schieve, L. A., & Blumberg, S. J. (2015). Estimated Prevalence of Autism and Other Developmental Disabilities Following Questionnaire 108 Changes in the 2014 National Health Interview Survey. National Health Statistics Reports, 87, 1-21. Zhang, W., & Baranek, G. (2016). The impact of insurance coverage types on access to and utilization of health services for US children with autism. Psychiatric Services, 67, 908 911. 109