COMMUNITY MENTAL HEALTH PROVIDERS’ USE OF PARENT TRAINING WITH MEDICAID-ENROLLED FAMILIES OF CHILDREN WITH AUTISM: A MIXED METHODS STUDY By Diondra Kathryn Straiton A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Psychology – Master of Arts 2019 ABSTRACT COMMUNITY MENTAL HEALTH PROVIDERS’ USE OF PARENT TRAINING WITH MEDICAID-ENROLLED FAMILIES OF CHILDREN WITH AUTISM: A MIXED METHODS STUDY By Diondra Kathryn Straiton Parent training, in which providers train parents to address their child’s maladaptive behavior or skill development, is an underutilized evidenced-based treatment for autism spectrum disorder (ASD). This mixed methods project examined the use of parent training as part of the Michigan Medicaid Autism Benefit, which covers applied behavior analysis (ABA) services for Medicaid-enrolled children with ASD under age 21 in Michigan. Descriptive statistics and multiple regression were used to analyze Medicaid claims data for 879 children and survey data from 97 ABA providers who service Medicaid-enrolled children with ASD. Content analysis was used to analyze open-ended survey items and phenomenological analysis was used to analyze interviews from a subset of 13 providers. Results demonstrated that: a) frequency of parent training encounters was very low, with evidence to suggest that Hispanic/Latino children receive fewer encounters; b) ABA providers’ conceptualization of parent training is inconsistent with the literature; c) providers report using evidence-based parent training strategies at a moderate-to-high level on the survey, but infrequently mention strategies in interviews; d) providers use sessions for other purposes; e) providers report having limited related training; f) providers report numerous barriers and facilitators which are related to their reported extensiveness of parent training; and g) parent training and family-centered care are significantly related. Implications for increasing use of parent training for this population are discussed. ACKNOWLEDGEMENTS I am extremely grateful to everyone who made this work possible. My incredible advisor, Brooke Ingersoll, has been a true mentor and has dedicated so much of her time and energy to this study and my overall growth as a scholar. Other members of my lab were also a great source of mentorship, particularly Karís Casagrande and Kyle Frost. I also could not have completed this work without support from my family – especially my parents Eileen Foley and Brian Straiton; my sisters Keira, Maia, and Brianna Straiton; and my grandmother Eileen Jenks and her sister Kathleen Petrausch. Additionally, I received so much support from my close friends, especially Olivia Furman, Lauren Kawulicz, Kaitlyn Ramirez, and Mollie Goldblum. My family and friends all spoil me with their kindness and love – thank you! Finally, this research was supported by the Blue Cross Blue Shield of Michigan Foundation Student Award Program (003701.SAP). iii TABLE OF CONTENTS LIST OF TABLES ........................................................................................................................ vi LIST OF FIGURES ..................................................................................................................... vii INTRODUCTION ..........................................................................................................................1 Parent Training for Children with ASD ........................................................................................3 Parent Training in Michigan ........................................................................................................5 Factors That Influence the Use of Parent Training ......................................................................7 Present Study .............................................................................................................................10 METHODS ...................................................................................................................................12 Design ........................................................................................................................................12 Medicaid Claims .......................................................................................................................12 Provider Survey ..........................................................................................................................14 Provider Interviews ...................................................................................................................20 Data Analysis ............................................................................................................................21 Data Integration .........................................................................................................................24 RESULTS .....................................................................................................................................26 Rates and Frequency of Parent Training Encounters .................................................................26 Format and Content of Parent Training Sessions ......................................................................27 Conceptualization of Parent Training .......................................................................................29 Evidence-Based Strategies Used in Parent Training Sessions ..................................................32 Other Strategies Used in Parent Training Sessions ...................................................................35 Client Characteristics Associated with Receipt of Parent Training ..........................................37 Provider Characteristics Associated with Parent Training ........................................................38 Barriers to the Use of Parent Training ......................................................................................40 Facilitators to the Use of Parent Training .................................................................................44 Relationship Between Parent Training Extensiveness and Family-Centered Care ...................48 DISCUSSION ................................................................................................................................49 Rates and Frequency of Parent Training Encounters .................................................................49 Strategy Use in Parent Training Sessions .................................................................................51 Format and Content of Parent Training Sessions ......................................................................53 Client Characteristics Associated with Receipt of Parent Training ..........................................54 Barriers to Parent Training Use .................................................................................................55 Facilitators to Parent Training Use ............................................................................................57 Relationship Between FCC and Parent Training Extensiveness ...............................................57 Limitations and Future Directions .............................................................................................58 APPENDICES ..............................................................................................................................60 APPENDIX A: Parent Training Definition Codebook .............................................................61 APPENDIX B: Interview Guide ...............................................................................................64 iv APPENDIX C: Provider Interview Codebook ...........................................................................67 REFERENCES .............................................................................................................................71 v LIST OF TABLES Table 1. Medicaid Claims Participant Demographic Information ............................................13 Table 2. Provider Demographic Information for the Survey and Interviews ............................15 Table 3. Research Questions and Data Sources ........................................................................24 Table 4. Frequency of Parent Training Encounters in Medicaid Claims and Provider Survey 27 Table 5. Joint Display of Parent Training Session Format and Content ...................................28 Table 6. Provider Definitions of Parent Training from Provider Survey ..................................30 Table 7. Joint Display of Evidence-Based Strategies Used by Providers in Parent Training Sessions .....................................................................................................................................34 Table 8. Other Strategies Used by Providers in Parent Training Sessions as Reported in Interviews ..................................................................................................................................36 Table 9. Hierarchical Regression Model of Demographic Variables and Number of ABA Encounters Predicting Number of Parent Training Encounters ................................................38 Table 10. Relationships Between Parent Training Extensiveness and Provider Demographic, Training, and Caseload Variables from the Survey ..................................................................39 Table 11. Correlations Between Barriers and Parent Training Extensiveness ...........................40 Table 12. Hierarchical Regression Model of Demographic Variables, Training Experiences, and Barriers Predicting Parent Training Extensiveness ............................................................42 Table 13. Barriers Reported in Interviews ................................................................................43 Table 14. Correlations Between Facilitators and Parent Training Extensiveness .....................44 Table 15. Hierarchical Regression Model of Demographic Variables, Training Experiences, and Facilitators Predicting Parent Training Extensiveness ........................................................46 Table 16. Facilitators Reported in Interviews ...........................................................................47 Table 17. Correlation Between Family-Centered Care and Parent Training Extensiveness .....48 vi LIST OF FIGURES Figure 1. Total Number of Overall and Evidence-Based Strategies Mentioned .......................32 Figure 2. Differences in the Frequencies of Evidence-Based Strategy Use ..............................33 Figure 3. Training Experiences Related to Parent Training ......................................................39 vii INTRODUCTION Autism spectrum disorder (ASD) is a developmental disorder characterized by pervasive deficits in social communication and a pattern of restrictive and repetitive behaviors (American Psychiatric Association, 2013). Children with ASD experience a high level of service needs compared to children with other special healthcare needs (CSHCN; Kogan et al., 2008; Montes, Halterman, & Magyar, 2009). In addition, their families often experience significant difficulties in accessing services for their children (Montes et al., 2009). For example, one large, nationally representative study of 40,256 parents of CSHCN found that children with ASD are 3.39 times more likely to have difficulties obtaining community and school services and 2.65 times more likely to be dissatisfied with the services that they have received than parents of other CSHCN, even after controlling for household demographics and insurance status (Montes et al., 2009). Hence, difficulties in accessing quality care is of critical importance for families of children with ASD. Children with ASD from traditionally underserved backgrounds (e.g., racial or ethnic minority, language minority, low socioeconomic status backgrounds) experience additional diagnostic and service access disparities (Fountain, King, & Bearman, 2011; Kalkbrenner et al., 2011; Liptak et al., 2008; Mandell et al., 2009; Mello, Urbano, Goldman, & Hodapp, 2016; Montes et al., 2009; Suppo & Floyd, 2012; Thomas, Ellis, McLaurin, Daniels, & Morrissey, 2007; Thomas et al., 2012). For instance, one survey-based study of caregivers in Pennsylvania found that children with ASD from low socioeconomic status (SES) backgrounds received an ASD diagnosis an average of 0.9 years later than children with ASD from middle- and upper- SES backgrounds, and children from rural regions received a diagnosis an average of 0.4 years later than children from urban regions (Mandell, Novak, & Zubritsky, 2005). Moreover, children 1 from Black, Latino, and other racial and ethnic minority backgrounds have a significantly lower likelihood of receiving an ASD diagnosis compared to White children, despite exhibiting clinical profiles consistent with ASD (Mandell et al., 2009). Another study found that Latino children experience an approximate one year delay in receiving an ASD diagnosis compared to White children (Magaña, Lopez, Aguinaga, & Morton, 2013). Access to ASD-related interventions is dependent on a medical diagnosis for nearly all public and private insurance; thus, a delay in diagnosis for a child also results in a delay in intervention services. Indeed, racial and ethnic disparities exist for behavioral health service use for individuals with developmental disabilities, even when controlling for need, age, and Medicaid enrollment (Harrington & Kang, 2008). This can be particularly grave for traditionally underserved families who experience multiple barriers to service access. For example, traditionally underserved families of children with ASD report needing to travel long distances to larger or more affluent cities for services, low household income, limited child care options, and having to give up their jobs to manage their children’s care as significant barriers to service access (Stahmer et al., 2019). Research has found that traditionally underserved families of children with ASD desire collaborative and strength-based strategies from their children’s health care teams (Pickard, Kilgore, & Ingersoll, 2016). These types of collaborative strategies are encompassed in family- centered care (FCC), an approach to service delivery that emphasizes collaborative health care decision-making (Kuo et al., 2012). FCC has been recognized by numerous state, federal, and academic organizations as a critical component in maximizing treatment outcomes for children with special healthcare needs (Kuo et al., 2012), and it has been found to improve access to care, child outcomes, and family functioning for families of children with special healthcare needs 2 (Dunst & Trivette, 2009; Kuhlthau et al., 2011; Kuo et al., 2012). Indeed, integrating FCC in the treatment of children with special healthcare needs may be particularly critical when considering strategies to improve child and family outcomes (Montes et al., 2009). Increasing FCC in the treatment of children with ASD may be particularly important, as children with ASD are significantly less likely to receive FCC than children with other emotional, developmental, or behavioral problems and children with other special healthcare needs (Kogan et al., 2008). Parent Training for Children with ASD Service options like parent training interventions can be leveraged to increase service access and involvement for families of CSHCN because they promote strong family-provider partnerships associated with family-centered care. Parent training is an intervention approach in which providers train parents to serve as agents of behavior change, with the child as the direct beneficiary of treatment (Bearss et al., 2015). Because the literature on parent training typically utilizes the term “parent” to refer to all types of caregivers, the term “parent” has been used herein for consistency, with recognition that this term is being used to refer to any biological, legal, familial or non-familial primary caregiver. Additionally, in the autism field, the term parent training has been used to describe varied types of interventions including parent support programs (such as care coordination and psychoeducation programs), and parent-mediated interventions, resulting in some confusion about what parent training actually means (Bearss et al., 2015). However, research suggests that parent implemented programs result in improved child outcomes while parent support programs do not (Kasari, Gulsrud, Paparella, Hellemann, & Berry, 2015). Parent training is an effective treatment option for teaching social communication (Ingersoll, Wainer, Berger, Pickard, & Bonter, 2016; Kasari, Gulsrud, Wong, Kwon, & Locke, 3 2010), decreasing disruptive behavior (Bearss et al., 2015), and improving adaptive skills in children with ASD (Scahill et al., 2016), with the majority of treatment research focused within the context of early intensive behavioral intervention with preschool-aged children with ASD (Matson, Mahan, & Matson, 2009). Parent training has a number of advantages in the treatment of children with ASD. First, parent training can increase the dosage of treatment that a child can receive, as the parent can continue implementing the intervention even outside of the time that the clinician spends providing direct services. In addition, parent training can promote skill generalization because the parent has more flexibility than most clinicians do in implementing the intervention in multiple contexts (Koegel, Schreibman, Britten, Burke, & O’Neill, 1982). Furthermore, parent training can lead to broader improvement in family functioning, as it has been shown to increase parent self-efficacy (Karst & Van Hecke, 2012) and reduce parental stress (Iadarola et al., 2018; Ingersoll et al., 2016; Koegel, Bimbela, & Schreibman, 1996). Given these benefits, parent training is considered a best practice in the treatment of children with ASD (National Autism Center, 2015; Wong et al., 2015) and therefore is of particular interest in studying how to increase parents’ engagement in their child’s ASD-related services to improve child and family outcomes. Parent training is uniquely positioned to fit the treatment needs of traditionally underserved families of children with ASD and has the potential to decrease many barriers related to service access for those families. For example, parent training may require less time with clinicians to produce child gains than clinician-implemented interventions alone (Koegel et al., 1982), and allows for increased flexibility of the availability of service delivery for parents (Koegel, Bimbela, & Schreibman, 1996; Koegel, Symon, & Koegel, 2002; Rocha, Schreibman, & Stahmer, 2007), which can be especially beneficial for families from economically 4 disadvantaged backgrounds with busy work schedules that may not be able to attend all of their child’s therapy appointments. This increased flexibility of service delivery is also of great benefit for families of children with ASD living in rural areas, who must travel great distances to access behavior support services such as applied behavior analysis (ABA); one study found that families from rural areas traveled 35.7 miles on average to access behavior support services, compared to a mean of 14.14 miles for non-rural families, which was a statistically significant difference (Mello et al., 2016). However, despite research and best practice guidelines supporting the use of parent training for treating children with ASD, it is underutilized in community settings in which the majority of children receive their services (Hume et al., 2005; Thomas, Morrissey, & McLaurin, 2007). This is despite the fact that parent training is often an intervention that is highly desired by families; one study found that while only 21% of parents reported receiving parent training, 78.2% of the parents that did receive parent training cited it as an efficacious intervention that contributed to their child’s growth, with parent training as the intervention that was most frequently endorsed as efficacious out of a list of 21 interventions (Hume et al., 2005). Parent Training in Michigan In the State of Michigan, policies were recently developed to better promote FCC and the implementation of parent training. Beginning in 2012, the Medicaid Autism Benefit for Behavioral Health Treatment (BHT) was created for the state’s community mental health (CMH) system under the auspices of the Michigan Department of Health and Human Services (MDHHS). After this legislation was passed in 2012, economically disadvantaged families of children with ASD in Michigan were able to receive intensive ABA services. This legislation includes parent training delivered by a Board Certified Behavior Analyst, Board Certified 5 Assistant Behavior Analyst, or another “qualified behavioral health professional” with a master’s degree (e.g. limited licensed psychologist, licensed clinical social worker). Working within the system of CMH agencies in Michigan is an ideal environment for studying the use of parent training for children with ASD from underserved backgrounds. First, Medicaid eligibility in Michigan requires a household income cutoff that is at least 133% of the federal poverty level; therefore, Medicaid-eligible families in Michigan are economically disadvantaged. Second, since the creation of the Medicaid Autism Benefit, CMH agencies in Michigan have been the primary mechanism for ABA service delivery for children served by Medicaid, including parent training. Thus, CMH agencies are an ideal environment to observe current parent training practices in community settings in Michigan, as the majority of parent training for Medicaid-eligible families is provided through CMH agencies, and there is a large source of funding allocated through the Medicaid Autism Benefit to provide parent training. Third, one of the 13 goals for 2018 set forth by the Michigan Department of Health and Human Services for Medicaid-eligible families of children with ASD was to increase the amount of eligible families of children with ASD that receive parent training at least once every three months to 50%, yet a recent audit found that only 37% of eligible families receive parent training at least once every three months (B. Groom, personal communication, February 7, 2017). Further, only 17% of those clients received parent training at least once per month, despite providers’ authorization to implement parent training more frequently (B. Groom, personal communication, November 3, 2017). As most evidence-based parent training interventions for children with ASD are delivered at least once per week (e.g., Mahoney & Perales, 2005), there is concern regarding the quality of parent training these families received. As evidence-based parent training models have more frequent sessions than what has been reflected in recent audits, 6 this may suggest that the penetration and intensity of parent training encounters is low and thus quality of parent training may be poor. The first step in addressing the penetration and quality of parent training in this system is to understand to what extent it is currently being used in this context and the reasons why it may or may not be occurring. Stakeholders at both the regional and state level are invested in better understanding current parent training practices and working to increase parent training utilization for traditionally underserved families. Factors That Influence the Use of Parent Training Provider use of evidence-based practices in mental health is influenced by a number of family-, provider-, and organization-level factors, with client outcomes improving when factors at each of these levels are addressed in provider training and support (Beidas & Kendall, 2010). Studying provider perceptions of barriers and facilitators to the use of a particular evidence- based intervention practice like parent training can provide critical information about entry points for interventions to increase the use of that practice. Family-level factors that may influence the use of parent training within the Medicaid Autism Benefit include logistical challenges like childcare and transportation, stigma related to autism, parents’ limited understanding of ASD, and a perception that parents are unmotivated to participate in their treatment (Stahmer et al., 2019). In qualitative studies, providers that work with traditionally underserved families often report difficulty engaging parents in services due to parents’ undervaluing of services for ASD or parent being unmotivated to participate in their child’s treatment (Fleming, Sawyer, & Campbell, 2011). Yet parents infrequently report that they are uninterested or unmotivated to participate in their child’s services (Stahmer et al., 2019; Fleming, Sawyer, & Campbell, 2011; Pickard et al., 2016). These perceptions of family engagement coupled with logistical challenges experienced by traditionally underserved families 7 of children with ASD (e.g., financial strain, limited transportation) are likely to influence families’ receipt of parent training. Moreover, Aarons (2005) has theorized that providers’ personal characteristics (e.g. demographics, personal values) as well as organizational factors (e.g. organizational support and climate) influence providers’ attitudes towards the implementation of an evidence-based practice. These attitudes are then theorized to influence providers’ behavioral intentions and self- efficacy, which in turn influence acceptance and fidelity to the implementation of an evidence- based practice (Aarons, 2005). Family-, provider-, and organizational-level barriers and facilitators to parent training for children with ASD are not frequently reported in the literature to date. Examples from one widely recognized parent training program for children at-risk for behavioral concerns, the Triple-P parenting program, have been identified at the family-, provider-, and organization- level. Identified family-level barriers include feeling that clients need higher level of Triple P than providers were trained to provide and feeling that the program was not appropriate for clients’ presenting problem (Sanders, Prinz, & Shapiro, 2009). Provider-level barriers include limited knowledge and skills in providing behavioral family intervention and Triple P clashing with one’s theoretical orientation (Sanders et al., 2009). Identified organization-level barriers include unavailability of overtime or compensated time for after-hours appointments, needing to provide appointments after regular hours that clashed with other commitments, insufficient access to consultation or supervision regarding the program, providers’ lack of recognition by colleagues for utilizing the program, and difficulty coordinating with other providers involved with the family (Sanders et al., 2009; Shapiro, Prinz, & Sanders, 2012). 8 A number of family-, provider- and organizational-level facilitators to Triple P use have also been identified. Of the family-level facilitators that were measured, receiving positive feedback from parents regarding the program, believing that Triple P produces change in children and families, and seeing observable change in children or families that received Triple P were both related to use of Triple P (Sanders et al., 2009; Shapiro et al., 2012). Provider-level facilitators that were related to increased use included: providers’ heightened knowledge and skills in implementing behavioral family interventions; feeling confident in providing consultation to parents; consulting with other Triple P providers; viewing supervision, consultation, and case discussions regarding Triple P to be helpful; and believing that supervision, consultations, and case discussions regarding Triple P were helpful (Sanders et al., 2009, 2009). Organizational-level facilitators included limited workplace barriers to using Triple P, being able to easily incorporate Triple P into one’s job activities, and workplace support of Triple P (Sanders et al., 2009; Shapiro et al., 2012). We anticipate that similar barriers and facilitators may be associated with current use of parent training as an overall service type provided in the CMH system (as opposed the specific manualized parent training program Triple P). Focusing on provider perspectives is of particular interest, with one recent study suggesting that only provider-level variables (and not family- or organization-level variables) predicted use of evidence-based practices in community mental health agencies (Beidas et al., 2017). By understanding providers’ perspectives of barriers and facilitators related to parent training, future work designing provider training initiatives and/or implementation interventions to help improve the quality of parent training in community settings can be better tailored to the needs of providers in the CMH system in Michigan. 9 Present Study The present study uses mixed methods in an effort to understand the current landscape of community providers’ use of parent training for children with ASD served by CMH agencies in the mid-Michigan region, with a specific focus on barriers and facilitators to parent training use that have been supported in the literature to date. This study utilized Medicaid billing data of parent training encounters at 12 CMH agencies across 22 counties in the mid-Michigan region, survey data from 97 CMH providers in that region, and qualitative data from follow-up interviews with a subset of 13 providers in order to unpack the reasons why the current utilization of parent training is so poor in this system despite the infrastructure in place to promote it. While the barriers and facilitators we have identified have been examined in relation to the implementation of particular health innovations (i.e., the uptake of a certain innovation in a specific practice setting, whereas here the innovation is already taking place), we hypothesize that these same variables will also be relevant to understand the current use of evidence-based practices. A convergent QUAN + qual complimentarity design (as described in Palinkas et al., 2011) was employed by integrating parent-training related Medicaid billing claims, provider survey data, and provider interview data to characterize the following: a) how parent training services are currently being delivered as part of the Medicaid Autism Benefit, b) client characteristics that are most associated with parent training utilization, c) provider characteristics that are most associated with parent training utilization, d) providers’ perspectives of barriers and facilitators in using parent training with traditionally underserved families, and e) the extent to which providers’ use of evidence-based parent training practices is associated with their use of family-centered care. 10 While the present study was largely exploratory in nature, we hypothesized that a) Medicaid billing claims and provider survey data would demonstrate a low rate of parent training usage with clients, with fewer parent training encounters than recommended in evidence-based curricula (i.e., at least once per week; Mahoney & Perales, 2005); b) providers would primarily use parent training in one-on-one sessions and would use it to cover a wide variety of skill areas; c) providers would not define parent training in the same way as the academic literature and would conceptualize it as parent support programs rather than parent implemented programs (Bearss et al., 2015), with limited coaching strategies involved; d) providers would report limited use of evidence-based coaching strategies and would also report using other strategies during sessions; e) Medicaid billing claims would demonstrate that children of racial or ethnic minority status and children from rural areas would receive less encounters of parent training; f) providers with smaller caseloads and more training experiences related to parent training would use parent training more often; g) providers’ endorsement of family-, provider-, and organization-level barriers would be associated with less self-reported parent training strategies; h) providers’ endorsement of family-, provider-, and organization-level facilitators would be associated with greater parent training strategy use; and i) providers’ use of parent training would be associated with their overall level FCC. 11 Design METHODS A convergent mixed-method design, in which Medicaid billing claims, the results from a provider survey, and interviews regarding providers’ experiences with parent training was utilized to integrate quantitative and qualitative data to characterize the parent training practices of CMH providers across Michigan. Particular emphasis was placed on providers’ perceived barriers and facilitators to parent training utilization. Content analysis was utilized to characterize providers’ definitions of parent training, with differentiation between providers’ descriptions of best practices from the literature (Bearss et al., 2015) and other strategies that they described as part of a parent training intervention. Phenomenological analysis was used to analyze interviews with providers and unpack results from the quantitative survey and billing claims. Constructs from both data sets overlapped, with qualitative data building upon quantitative data and providing a more in-depth understanding of provider perspectives regarding the limited use of parent training in Michigan’s CMH system, despite the infrastructure in place to support it. Medicaid Claims Participants. Deidentifed Medicaid claims for the time period of October 2017 through March 2018 was obtained from one of 13 regional Prepaid Inpatient Health Plans (PIHP) contracted in Michigan. This PIHP oversees all services provided by 12 CMH agencies across 22 counties in the mid-Michigan region. Claims data for all clients with a documented ASD diagnosis modifier on file and Current Procedural Terminology (CPT) codes billed to the Behavioral Health Treatment funds (the funds allocated for the provision of ABA services from 12 the Medicaid Autism Benefit) were analyzed for the 879 individuals who under 21 years old. See Table 1 for participant demographic information. Table 1. Medicaid Claims Participant Demographic Information (N = 879) Age (years) Gender Male Race White Black Asian American Indian/Alaskan Native Missing Ethnicity n - % - 697 79.3% 624 71.0% 107 12.2% 5 0 0.6% 0% 11 1.3% Hispanic/Latino 95 10.8% M Range 7.39 1-20 - - - - - - - - - - - - - - Claims data. The total number of ABA encounters for each eligible child were extracted for the six month time period of October 2017 through March 2018. Additionally, the total number one-on-one and group parent training encounters received by each eligible child were extracted. For those who received at least one encounter of parent training, we calculated the rate of parent training encounters out of total ABA encounters, expressed as a percentage. An encounter was defined as one session of parent training that was at least 15 minutes long, with 13 recommended session lengths of 45 minutes for parent training encounters. Client demographic information in the claims data included the child’s gender, age, race, and ethnicity. Additionally, we calculated the urbanicity of each regional CMH agency by dividing the rural population by the total population for all counties serviced by the CMH agency using 2010 U.S. Census data (U.S. Census Bureau, 2010). Provider Survey Pilot testing. First, a pilot survey was emailed to the 34 agency leaders who supervise ASD services within 1 PIHP in Michigan from which we recruited. Twenty-one agency leaders (62%) provided feedback on survey language and formatting. The final draft of the survey was then emailed to providers at three PIHPs in Michigan and conducted on Qualtrics, an online software used for data compilation and analysis (www.qualtrics.com). Providers were asked to report on their practices during the previous six months. Participants. Providers were recruited by email using contact information provided by an administrator at the PIHP that they worked for. Survey participants included 97 community providers who provided services through the Medicaid Autism Benefit and were employed by private agencies that were contracted by CMH agencies in Michigan or for a CMH agency itself. Three PIHP administrators helped to recruit participants via email. Two of these community partners shared all email addresses that they had on record of their individual ABA providers and Autism Coordinators at each CMH agency in their region. The third PIHP administrator only shared email addresses for the Autism Coordinators at each CMH agency in their region and did not share contact information for individual ABA providers. Because the survey link was sent via email and it was an anonymous survey, it is not possible to know the exact number of providers who received the survey link. We are aware that at least 294 providers received the 14 survey link, indicating that the upper bound of the response rate was 33% (there may have been other providers who received the link who we cannot track). Participants included 58 Board Certified Behavior Analysts (BCBAs), 5 Board Certified Assistant Behavior Analysts (BCaBAs), and 34 other “Qualified Behavioral Health Professionals” with master’s-level degrees (e.g. limited licensed psychologists). To be included, providers had to have at least one client on their caseload who was: a) diagnosed with ASD; b) under the age of 21; and c) utilized ASD-related services using the Medicaid Autism Benefit. All participants were provided with informed consent prior to their participation in the study and received a $5 Amazon gift card for participation. See Table 2 for provider demographic information from the survey. Table 2. Provider Demographic Information for the Survey and Interviews Provider Survey (N = 97) Provider Interviews (n = 13) % M Range n % M Range 36.23 22-64 37.00 28-59 Age (years) Gender Female n - - 79 81.4% Race/Ethnicity White Black Asian American Indian/ Alaskan Native 84 86.6% 2 1 3 2.1% 1.0% 3.1% Hispanic/Latino 3 3.1% - 15 - - - - - - - 9 69% 10 77% 0 0 1 0% 0% 8% - - - - - 1 8% - - - - - - - - - - - - - Table 2 (cont’d) Prefer Not to Answer/ 2 2.0% Missing Role BCBA BCaBA Social Worker Psychologist 58 59.8% 5 1 2 5.2% 1.0% 2.1% Other QBHP 26 26.8% Other Prefer Not to Answer/ Missing 4 1 4.1% 1.0% Certification BCBA BcaBA None 58 59.8% 5 5.2% 34 35.1% Primary Location of Service Client’s home 41 42.3% Agency-Based School/ 47 48.5% Center - - - - - - - - - - - - - Other 9 9.3% - 16 - - - - - - - - - - - - - - 0 0% 8 0 0 1 4 0 0 6 0 7 6 7 62% 0% 0% 8% 31% 0% 0% 46% 0% 54% 46% 54% - - - - - - - - - - - - - 0 0% - - - - - - - - - - - - - - - Note: QBHP stands for Qualified Behavioral Health Provider; *Incomputable due to some missing data used to compute parent training extensiveness Measures. Provider demographic information. Providers reported on their gender, age, race and ethnicity, educational attainment, disciplinary background, professional role, professional certifications, employment setting (CMH agency, contracted agency, or private practice), caseload, hours of overall service provision within the Medicaid Autism Benefit, and years of experience in working with clients with ASD and in using parent training. Number of training experiences related to parent training. Providers endorsed whether or not they had received any of the following training experiences related to providing parent training: taking a course related to working with families, participating in a parent training workshop, observing parent training modeled during an internship or outside of internship, receiving supervision in parent training, participating in self-guided learning about parent training, being trained in a specific parent training intervention approach, or other training experience. A total number of parent training experiences was computed by summing all training experiences and was utilized in subsequent analyses. Frequency of parent training use. Participants utilized a 5-point Likert scale to measure the frequency providers reported using parent training. Providers indicated the number of encounters per month of parent training that they delivered to a typical client [No encounters per month (1); More than 8 encounters per month (5)]. Quality of parent training use. Quality was measured using a 5-point Likert scale to describe frequency of use of the following seven evidence-based parent training strategies (Barton & Fettig, 2013) that providers used with a typical client [Not at all (1); Very much (5)]: 17 (1) collaborative goal-setting with parents, (2) modeling with video or live demonstration of intervention techniques, (3) providing time for parent practice, (4) providing feedback to parents, (5) planning or reflecting on parents’ use of intervention techniques at home, (6) providing written materials or manuals to support parent learning, and (7) problem-solving barriers to parents’ use of the intervention techniques. Ratings for each strategy were averaged to produce a measure of the overall quality of parent training. Conceptualization of parent training. Providers provided written responses to the following open-ended item in the survey: Please describe what the term “family training” means to you. Format of parent training encounters. Providers indicated the format in which they deliver parent training: one-on-one with a family or in a group format with multiple families. Providers were able to mark all options that applied. Content of parent training encounters. Providers indicated the skill areas that they targeted in their parent training sessions from a list of 5 possible content areas: principles of applied behavior analysis, including behavior management; communication skills; play skills; social interaction skills; self-care skills; and educational/academic support. Providers could also write in additional content areas if needed. Providers also indicated whether or not they use manualized parent training interventions with their clients served through the Medicaid Autism Benefit. Barriers to parent training use. Providers rated 13 common barriers to their use of parent training on a 5-point Likert scale [Not an obstacle (1); Extreme Obstacle (5)]. Items were adapted from the Triple P Barriers to Program Use Scale (Sanders et al., 2009; Shapiro et al., 2012) and the Implementation Climate Scale (Ehrhart, Aarons, & Farahnak, 2014), and were 18 selected to represent common barriers to the use of parent training in community settings (Stahmer et al., 2019). The scale included family-, provider-, and organizational-level barriers such as “It is difficult to engage families in a family training session” (family-level barrier); “Family training is not consistent with my theoretical orientation or preferred treatment approach” (provider-level barrier); and “I do not receive recognition for providing family training at my agency” (organization-level barrier). The family-level barriers subscale had moderate internal consistency (Cronbach’s alpha = 0.68), while the provider-level barriers subscale (Cronbach’s alpha = 0.77) and organization-level barrier subscale (Cronbach’s alpha = 0.82) had good internal consistency. Facilitators to parent training use. A facilitators scale of 10 items with a 5-point Likert scale was created by adapting items from the Triple-P Positive Parenting Program Facilitators to Program Use Scale (Shapiro et al., 2012), the Implementation Climate Scale (Ehrhart et al., 2014), and variables from the Theory of Planned Behavior (Ajzen, 1985). The facilitatiors scale included family-, provider-, and organizational-level facilitators from each of the three measures and contained items such as “The feedback from caregivers regarding family training is positive” (family-level facilitator; adapted from Facilitators to Program Use Scale); “Other providers I know use family training in their work with children with ASD” (provider-level facilitator; adapted from Theory of Planned Behavior); and “My agency provides training opportunities for family training (workshops, seminars, materials)” (organization-level facilitator; adapted from Implementation Climate Scale). The family-level facilitators subscale (Cronbach’s alpha = 0.79) and organization-level facilitators subscale (Cronbach’s alpha = 0.82) had good internal consistency. The provider-level facilitators subscale had poor internal consistency (Cronbach’s alpha = 0.55) and thus was not utilized in subsequent analyses. 19 Measure of Process of Care for Service Providers (MPOC- SP). The MPOC-SP is a 27- item self-assessment measure with a 7-point Likert scale. The MPOC-SP was developed to asses FCC in pediatric service providers and has excellent validity (Woodside, Rosenbaum, King, & King, 2001). The MPOC-SP has four scales: Showing Interpersonal Sensitivity, Providing General Information, Communicating Specific Information, and Treating People Respectfully. Scores for each item were averaged to provide an average score of FCC used in subsequent analyses. Provider Interviews Participants. All providers who completed the survey were offered an additional opportunity for a follow-up phone interview and an additional $20 Amazon gift card. A subset of 13 providers from 4 PIHPs participated in follow-up interviews. Participating providers’ survey responses regardinging their parent training extensiveness were examined to determine their representativeness. Tertials were determined and providers were accordingly categorized as having low, mid, and high levels of parent training extensiveness. See Table 2 for provider demographic information from the survey. Interviews format and content. Interviews were conducted in a semi-structured format developed based on reported barriers and facilitators to intervention use by providers working in community settings (e.g., child welfare) that have been identified in the literature (Aarons, 2005; Sanders et al., 2009; Shapiro et al., 2012). Additional questions were added to the interview guide using an iterative process that included information discussed by previous participants. All questions were further refined after each interview to ensure that the questions were relevant to providers in CMH agencies in Michigan. See Appendix B for the interview guide. We were interested in providers’ use of best practices for parent training as identified in the literature (e.g., 20 Barton & Fettig, 2013); therefore, we read providers a definition of evidence-based parent training before conducting the interview. Interview questions were related to: a) content and structure of their parent training sessions; b) barriers and facilitators to parent training utilization; c) billing practices for parent training sessions paid for by the Medicaid Autism Benefit; and d) training experiences related to using parent training interventions with families. Data Analysis Medicaid billing data. Descriptive statistics were used to characterize the percent of eligible children (Medicaid-enrolled children with ASD under 21 years old who received ABA services through one PIHP in the mid-Michigan region) who received at least one parent training encounter over 6 months. The average frequency of parent training encounters received over the 6 month period was calculated for the full sample and for children who received at least 1 encounter of parent training. Bivariate analyses were used to examine whether any child demographic variables (age, gender, race, and ethnicity, urbanicity) were related to the frequency of parent training encounters for clients with ASD served by the Medicaid Autism Benefit. For categorical data (race), a one-way ANOVA was used. For dichotomous data (gender, ethnicity), independent sample t-tests were used. For continuous data (child age, urbanicity), Pearson’s r was used. These analyses were exploratory; thus, variables that were found to be significantly related to frequency of parent training were entered into a multiple regression to examine the variables that were the strongest predictors of parent training extensiveness. Provider survey. Quantitative analyses. Descriptive statistics were used to characterize providers’ frequency of parent training use, quality of parent training, content of parent training sessions, and format of parent training sessions. Additionally, descriptive statistics were used to analyze 21 reported manual use and professional training experiences related to parent training. We created a parent training extensiveness variable to describe the extent to which providers used high- quality parent training with their clients enrolled in the Medicaid Autism Benefit. Parent training extensiveness was calculated by z-scoring the values for parent training frequency and quality and then summing both z-scores. Bivariate correlations were used to examine the relationship between parent training extensiveness and provider demographics, professional training experiences, and barriers and facilitators of parent training use. Each of the barriers subscales and the family- and agency-level facilitators subscales were used in subsequent analyses. Barrier and facilitator subscales that were significantly associated with parent training extensiveness (p < .05), were then entered into separate multiple regression models predicting parent training extensiveness. Because no demographic variables were related to FCC, a hierarchical regression model was not appropriate; thus, a Pearson’s correlation was used to examine the relationship between parent training extensiveness and FCC. Qualitative analyses. Content analysis was employed to identify common themes across providers’ responses to an open-ended survey item regarding their conceptualization of parent training. Both inductive and deductive approaches to qualitative content analysis were employed. Four codes were derived deductively from a taxonomy of parent training interventions for ASD that was developed from Bearss and colleagues (2015): care coordination, psychoeducation, parent mediated interventions for core symptoms of ASD, and parent mediated interventions for maladaptive behaviors. Open coding was also used to note additional themes present in the data. Inductive themes included: modeling of intervention strategies, checking in with families, treatment development and/or planning, unspecified efforts to support intervention strategy use, social-emotional support of family members, stress management, and providing opportunities for 22 caregiver practice and/or providing feedback to caregivers. Both deductive and inductive codes were integrated into a codebook (see Appendix A). Coders included one graduate student with experience in qualitative research and three highly trained undergraduate research assistants. Providers’ definitions of parent training were copied verbatim from the survey and uploaded to Dedoose 7.5.9, an online program for analyzing qualitative data. In the training phase, coders jointly developed a codebook that incorporated themes they noted in a subset of definitions. The codebook was operationalized and then utilized to check for its fit with a larger subset of definitions. Then, novice coders (highly trained undergraduate research assistants) took code application tests using the Dedoose program to determine their reliability with the expert coder (the author). Training was completed when novice coders reached a Cronbach’s alpha of 0.70 or higher agreement with the expert coder for three code application tests in a row. After establishing reliability, all definitions were independently analyzed by one coder. Twenty-one percent of definitions were double-coded by an additional coder and inter-rater reliability was determined for those definitions. The overall Cohen’s Kappa for all codes was 0.61, with a range of 0.28 to 1.00. Interviews. An initial interview guide (see Appendix B) was created to further unpack providers’ experiences in using parent training with clients enrolled in the Medicaid Autism Benefit. Interview questions were further refined after analyzing preliminary survey data and following discussions with 1 PIHP administrator who oversees ABA services for the region in which most participants worked. After 13 interviews, saturation was met; no additional themes were discussed that had not been presented in the preceding interviews. Interviews were transcribed verbatim by highly trained undergraduate research assistants and transcripts were then uploaded to Dedoose 7.5.9. 23 Qualitative analyses. Coders utilized phenomenological analysis (Moustakas, 1994) to examine the 13 interviews. Using horizontalization (Moustakas, 1994), each transcriber and the author analyzed a subset of transcripts and extracted significant statements. Clusters of meaning were developed from these significant statements and refined into themes using an iterative process in which the coding team discussed themes noted across interviews and refined the list of themes into 14 codes (see Appendix C). Broad codes related to session content and strategy use in session were deductively subcoded by the author using the codes developed in the definition codebook and the skill areas used in the survey. Before coding all interviews, interrater reliability (IRR) was established when novice coders reached a Cronbach’s alpha of 0.70 or higher across all codes with the expert coder for at least one code application test that incorporated at least 2 excerpts for each code. All interview transcripts were double-coded by 2 independent raters and any disagreements were resolved using consensus coding. Data Integration Quantitative data from the Medicaid billing data was compared with providers’ self- reported use of parent training in the provider survey. Qualitative data elaborated on trends seen in the Medicaid billing data and providers’ self-reported parent training practices in the survey. Data was analyzed in parallel, with results from the preliminary analysis of survey data integrated into the interview guide as interviews were conducted. Table 3. Research Questions and Data Sources Research Question What is the rate and frequency of parent training in CMH agencies in mid-Michigan? Data Source Medicaid claims Provider survey Items Rate of eligible children who received PT; average number of PT encounters per month for children who received PT Providers’ self-reported rate and frequency of utilization of PT 24 Providers’ endorsements of evidence-based PT strategies and curricula; providers’ descriptions of PT session content and format Open ended items regarding providers’ PT practices Open-ended item probing for providers’ definition of PT Ratings of evidence-based strategies Strategies mentioned when responding to item about activities completed in PT sessions Client’s age, gender, race, ethnicity, and urbanicity Provider survey Provider interviews Provider survey Provider survey Provider interviews Medicaid claims Table 3 (cont’d) What is the content and format of CMH providers’ parent training sessions? How do CMH providers conceptualize parent training? Which strategies do providers use in parent training sessions? Which client (child) characteristics are most associated with parent training use? Which provider characteristics are most associated with parent training use? What are providers’ perceptions of the barriers and facilitators to parent training use? Provider survey data Providers’ demographic variables (e.g., educational attainment, past training experiences); variables related to providers’ caseload Provider survey Provider interviews Barriers scale; facilitators scale Open-ended items regarding providers’ perceptions of barriers and facilitators to PT use To what extent is parent training associated with family-centered care? Provider survey Note: PT refers to parent training MPOC-SP scale and PT extensiveness variable 25 RESULTS Rates and Frequency of Parent Training Encounters Medicaid claims data from mid-Michigan and data from the provider survey were analyzed to determine the rates and frequency of parent training encounters provided to Medicaid-enrolled children with ASD in Michigan. Descriptive statistics were used to analyze both sources of data. Medicaid claims. Descriptive statistics were utilized to determine the frequency at which the client in the Mid-Michigan region received parent training from October 2017 through March 2018. Eight hundred and seventy-nine children received services through the Medicaid Autism Benefit during that time period (see Table 1 for demographic information). Of these children, only 55.1% (n = 484) received at least one parent training encounter over the 6 month time period. Furthermore, the frequency of encounters that most children received was not consistent with evidence-based practice (Mahoney & Perales, 2005). The average number of parent training encounters each child received was 1.50 (SD = 2.29; range 0-19) during that time period; which represents only 3.2% of all ABA encounters (M = 89.67, SD = 71.84) they received. The average number of parent training encounters for children who received at least one encounter was 2.73 (SD = 2.48), which corresponds to less than 1 session every other month (M = .46, SD = 0.41). Further, only 2.7% of children in the sample received at least 8 encounters of parent training, a frequency consistent with lower-intensity evidence-based parent training models (e.g., Hanen’s “More Than Words” Intervention; Carter et al., 2011). No children received 20 or more encounters of parent training, a frequency seen in higher-intensity models (e.g., Project ImPACT, Early Start Denver Model). 26 Provider survey. In contrast to the Medicaid claims data, 95.9% of surveyed providers reported providing parent training to the average client at least once in the past 6 months, with the majority of providers (74.2%) reporting providing 1-2 encounters per month to the average client. See Table 4 for a comparison of frequency of parent training encounters as seen in the Medicaid claims data and the provider survey. Table 4. Frequency of Parent Training Encounters in Medicaid Claims and Provider Survey Data Source N Client Receipt of PT Over 6 months # of PT Encounters per Month Provider Survey 97 95.9% report providing at least 1 No encounters/month 2.1% encounter to the average client 1-2 encounters/month 74.2% 3-4 encounters/month 17.5% 5-8 encounters/month > 8 encounters/month 0% 1.0% Medicaid Claims 879 55.1% of clients received at least 1 M = .46 encounters/month for encounter those who received PT Meta-Inferences • Few children receive parent training and those that do receive fewer sessions than what is recommended in the literature. • Providers report providing more parent training than what is reflected in the claims data. o Providers may not have a conceptualization of parent training that is consistent with the literature. ▪ Qualitative data from definitions provided in the survey supports this hypothesis. See Table 4. o Providers may not be billing for parent training when they are providing it. ▪ Some providers reported in interviews that they do not enter their own billing codes and instead submit case notes to a billing office. o Providers may be influenced by social desirability when completing the self-report survey. Format and Content of Parent Training Sessions Format. In the survey, providers primarily reported using parent training in individual, one-on-one sessions (94.8%), with 9.3% of providers reporting using group models of parent training with multiple families. Of the children that received at least one parent training 27 encounter in the Medicaid claims data, 98.3% of children received individual one-on-one encounters and 3.4% of children received group parent training encounters. Content. On the survey, providers reported covering a range of content in parent training sessions, including principles of applied behavior analysis (90.7%), communication skills (87.6%), self-care skills (85.6%), social skills (84.5%), educational/academic support (58.8%), and other content (8.2%). Addressing maladaptive behavior was the most frequently described category that providers wrote in for other session content (3.1%). In interviews, providers also spontaneously mentioned covering a range of content but did not mention covering educational/academic skills. See Table 5 for a joint display of session format and content. Thirty- eight percent of providers reported on the survey that they have used a manualized intervention during parent training sessions. Table 5. Joint Display of Parent Training Session Format and Content Session Format Primarily Used Survey (N = 97) Medicaid Claims Individual One-On-One Sessions Group Sessions 94.8% 9.3% Session Content (N = 879) 98.3% 3.4% Provider Survey (N = 97) Provider Interviews (n = 10 providers who mentioned skill areas) Skill Area % Representational Quote Percent Who Mentioned Principles of Applied Behavior Analysis 90.7% Obviously when I am treating a behavior I want 80% to know what the function of that behavior so that I can create an effective plan to tackle some of that. So what I’m doing in parent training is teaching the families how they can better recognize what some of those functions are so 28 Table 5 (cont’d) Self-Care Skills 85.6% they can adapt some of their behavior when they’re interacting with the kid so they can continue the plan even when me and my staff aren’t there. It could be a lot of different things, it could be teaching them to implement, like self-care hygiene tasks at home or we will teach the kids to comply with the teeth brushing routine here. And then take that home and have the parents implement that as well. 70% Communication Skills 87.6% When we’re working on a new communication 60% strategy or introducing a new communication strategy is another time you are definitely working with parents… 84.5% We deal with the concept of sexuality and appropriate social relationships, significant others. n/a 58.8% 30% 0% Social Skills Educational/Academic Skills Conceptualization of Parent Training Content analysis was used to analyze providers’ definitions of parent training as written in the provider survey. Providers defined parent training before responding to any items related to evidence-based parent training practices. See Table 6 for representative quotations of themes present in provider definitions of parent training. Most providers mentioned between 1 to 3 strategies in their definition of parent training (see Figure 1), with an average of 2.31 strategies mentioned. Overwhelmingly, providers failed to mention key evidence-based strategies of parent training, such as modeling of intervention techniques and providing practice opportunities and feedback to parents. Unspecified efforts to support intervention strategy use (42%) were frequently described by providers; these were vague descriptions of efforts to “teach” or “train” parents that were provided without mentioning specific methods for doing so in a session (e.g., not mentioning how modeling was used during these teaching efforts). Providers also frequently 29 described providing psychoeducation (51%) to parents during sessions and checking in about child progress (39%). Evidence-based parent training models typically include a combination of psychoeducation, modeling, and practice with feedback (e.g., Ingersoll & Dvortcsak, 2009; Rogers, Dawson, & Vismara, 2012); thus, we were interested in how many providers mentioned those strategies. Only 13% of providers mentioned modeling and only 12% of providers mentioned practice and/or feedback, two critical components of evidence-based parent training models that are associated with improved child outcomes (Ruppert, Machalicek, Hansen, Raulston, & Frantz, 2016; Wyatt Kaminski, Valle, Filene, & Boyle, 2008). Fifty-five percent of providers mentioned at least 1 of the 3 evidence-based strategies (psychoeducation, modeling, and practice and/or feedback), with the majority of those providers mentioning only one of the evidence-based strategies (see Figure 1). The average number of evidence-based strategies mentioned across all definitions was 0.75. Only 5 providers mentioned all three evidence-based strategies (5% of total sample). Further exploration of the content analysis indicated that of the 38 providers who mentioned only one evidence-based strategy, 95% of providers mentioned psychoeducation as that strategy. Additionally, 69% of providers that mentioned any of the evidence-based strategies included psychoeducation in their definition. Table 6. Provider Definitions of Parent Training from Provider Survey (N = 97) Theme Representative Quotation Psychoeducation Educate those involved as primary caregivers for the child on ASD [and] family dynamics that come with having a child who had ASD. Percent Who Mentioned 51% 30 Table 6 (cont’d) Unspecified Efforts to Support Intervention Strategy Use Checking In Treatment Development/Planning Social-Emotional Support and Empowerment Modeling Parent-Mediated Intervention for Maladaptive Behaviors Practice and Feedback Stress Management Care Coordination Provide information and recommendations for completing skilled therapeutic recommendations within the home environment. Providing feedback on child’s progress… Listen to the caregivers [sic] concerns and aspirations for their child and plan goals based on their input while simultaneously tailoring training to correspond with the goals. Extensive training in BIPs [Behavior Intervention Plans] if needed. Empowering parents to be in control and take control of their child’s treatment and prognosis. Give them to tools to one day “fire us” (in a good way). Make them feel they can do it without us when the time comes, because they are doing more than they even know. Family training focuses first on the important topics for the family/caregivers. This can include modeling… …instructing and modeling how to implement strategies to address challenging behaviors. Supporting them with implementation to address their concerns, issues and needs, using modeling and practice and feedback. Assisting parents with specific concerns and giving them the tools to solve problems that may arise in the future. … sharing information about other possible services. Parent-Mediated Intervention for Core Symptoms of ASD … a plan is made to work with the family in a way to promote functional/meaningful social interactions with the child that is being worked with in ABA. 42% 39% 25% 16% 13% 16% 12% 9% 5% 3% 31 Figure 1. Total Number of Overall and Evidence-Based Strategies Mentioned 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Total Number of Strategies (out of 11) Total Number of Evidence-Based Strategies 0 Strategies 4 Strategies 1 Strategy 5 Strategies 2 Strategies 6 Strategies (out of 3) 3 Strategies Evidence-Based Strategies Used in Parent Training Sessions Provider survey. Providers indicated the frequency at which they use a variety of evidence-based strategies during their parent training sessions on a Likert scale [Not at All (1); Very Much (5)]. They reported most frequently working with parents to problem-solve issues around the implementation of intervention strategies (M = 4.29, SD = .79), collaborative goal- setting (M = 4.24, SD = .91), providing feedback on parents’ use of strategies, and planning and reflecting on parents’ use of intervention strategies at home (M = 4.11, SD = .90). Modeling intervention strategies (M = 2.82, SD = 1.27), providing time for parent practice (M = 3.26, SD = 1.18), and providing written materials (M = 3.71, SD = 1.09) were less frequently endorsed in the 32 survey. To determine whether strategies were reportedly used at significantly different frequencies, a repeated measures ANOVA was conducted. The ANOVA indicated that there were significant differences among the reported frequencies of strategy use, F(6, 546) = 43.75, MSE = 29.28, p = < .001. Pairwise comparisons indicated that modeling, time for parent practice, and providing written materials were all significantly different from every other strategy; however, frequencies for problem-solving issues around the implementation of intervention strategies at home, collaborative goal-setting, providing feedback on parents’ use of strategies, and planning and reflecting on parents’ use of intervention strategies at home were all not significantly different from each other. See Figure 2 for reported frequencies of each strategy. Figure 2. Differences in the Frequencies of Evidence-Based Strategy Use Note: Strategies bracketed with * are significantly different from all other strategies. Strategies bracketed with + are not significantly different from the other strategies within that bracket. 33 Interviews. After listening to a definition of evidence-based parent training that included a definition of collaboration, modeling, practicing, giving feedback, planning and reflecting on intervention strategy use, written information, and problem solving barriers to strategy implementation (see Appendix B). Providers were asked to report on the activities that they typically do in a parent training session. Surprisingly, in contrast to the survey results, the strategies most often spontaneously described by providers included modeling (69%) and time for parent practice (69%). Providers less frequently spontaneously mentioned the other evidence- based strategies. Interestingly, collaborative goal-setting was described least often, despite having been one of the most highly rated strategies on the survey (M = 4.24, SD = 0.91). See Table 7 for a joint display of evidence-based strategies used by providers. Table 7. Joint Display of Evidence-Based Strategies Used by Providers in Parent Training Sessions Provider Survey (N = 97) Provider Interviews (n = 13) Evidence-Based M SD Representative Quotation Percent Who Mentioned 4.24 0.91 …the goals get updated and then it gets 8% decided what goals as to what we want to work on next, so having them involved in that, I think you get that buy in and their credit to see data on their own behavior. I definitely want to model it so they see what is happening so they can imitate that. 69% 69% 3.26 1.18 … I can be with the parent doing a family training to show them what’s going on with the client and then have them then step into the session and try out what we’ve been working on. 4.13 .89 Feedback is a big part of that. If possible, I like 46% the kid to be there so we can practice that and give feedback and I can help correct anything that is going on or give them reinforcement on what is going well with what they’re doing and 34 Modeling 2.82 1.27 Strategy Collaborative Goal-Setting Time for Parent Practice Feedback to Parents Regarding Intervention Strategies Table 7 (cont’d) just continue it from there to be able to maintain the skills. Planning and Reflecting on Intervention Strategy Use At Home Providing Written Materials Collaboratively Problem-Solving Barriers to Strategy Use at Home 4.11 .90 … we do modeling, we do problem solving. 31% We look at some of the reflective kind of behaviors, what did I do, what should I have done? 3.71 1.09 And often times providing just a handout so 31% they have something to refer back to. It can be like providing them visual schedules and helping them learn to utilize those. 4.29 .79 …we can go home with them and have them do it in a natural environment and help them troubleshoot problems there. 23% Meta-Inferences • Providers frequently reported using evidence-based strategies in the survey. However, aside from modeling and providing time for parent practice, the majority of providers did not mention these strategies spontaneously in interviews. o The quality of sessions is unclear. • Providers self-reported a high use of collaborative goal-setting on the survey, but goal- setting that was described in interviews was usually provider-directed. o Providers may not be self-aware of the lack of collaboration in their goal-setting. • On the survey, providers reported limited use of modeling during session, but the majority of interviewed providers mentioned modeling during their sessions. o Giving providers the definition of evidence-based practice before the interviews may have been primed them to describe exemplar sessions rather than typical practice. Other Strategies Used in Parent Training Sessions Providers also mentioned 4 other strategies during parent training sessions: checking in about child progress, treatment development or planning, social-emotional support of family members, and psychoeducation. While psychoeducation is a common component of evidence- based parent training models, it is not significantly related to child outcomes when controlling for intervention design and other evidence-based strategies that may be used in conjuction with it (Wyatt Kaminski et al., 2008); thus, it can be considered a common component of evidence- 35 based interventions one that is necessary but not sufficient for improving outcomes. Four of the 13 providers mentioned psychoeducation as a strategy they used in parent training sessions (31%). No interviewed providers mentioned using psychoeducation alone, with the the most frequently reported co-occuring strategies being modeling (23% of interviewed providers mentioned both; n = 3), time for practice (15% of interviewed providers; n = 2), and providing feedback (15% of interviewed providers; n = 2). Moreover, though checking in about child progress and treatment development and planning are key components of quality treatment management, they are not strategies that are related to learning processes seen in parent training sessions. Similarly, although social- emotional support of parents is related to improved service delivery for children with ASD, it is not in and of itself an evidence-based component of parent training for children with ASD (Bearss et al., 2015). See Table 8 for representational quotations of these additional strategies. Table 8. Other Strategies Used by Providers in Parent Training Sessions as Reported in Interviews (n = 13) Strategy Representational Quotation Checking In Psychoeducation Treatment Development/ Planning Social-emotional Support of Parents A lot of clinicians do just meet with the family and kind of check in, and say you know “how are things going” and kind of more just a discussion of current progress instead of building upon what they’ve already talked about and putting things into practice. I set specific goals that we try to meet for those trainings. So it could be training them in the principles of ABA. And that might involve kind of like a [sic] instruction as well as videos. …discussing future goals… That I don’t know besides empathetic listening that you can go to some kind of trainings on that, you have to really know that Percent Who Mentioned 38% 31% 23% 8% 36 Table 8 (cont’d) that’s a part of it. Allowing the families to express their fears and express their dissatisfaction you know with the system. Client Characteristics Associated with Receipt of Parent Training Medicaid claims. We examined the relationship between client demographic variables and number of parent training encounters received. Age at service was associated with number of parent training encounters, r = -.07, p = .03, such that younger children received more parent training encounters than older children. Hispanic/Latino ethnicity was also significantly associated with number of parent training encounters, t = -2.56, df = 867, p = .01, such that Hispanic/Latino children (M = 1.07, SD = 1.66) received fewer parent training encounters than non-Hispanic/Latino children (M =1.56, SD =2.35). Race [F(4,863) = .428, MSE = 4563.90, p = .79], gender (t = -1.61, df = 877, p = .11), and urbanicity (r = .000, p = .997) were all not related to the number of parent training encounters children received. A multiple regression was conducted with number of parent training encounters as the dependent variable and child ethnicity and age as the independent variables. Ethnicity was effect coded (1 = Hispanic, -1 = non-Hispanic), and age was mean centered. The model was significant (R2 = .01, F(2, 866) = 4.53, p = .01), with younger age predicting more parent training encounters (β = -.08, p = .023) and Hispanic/Latino status predicting fewer encounters (β = -.07, p = .04). To examine whether these findings were specific to parent training encounters or related to the number of overall ABA encounters the child received, we ran a second multiple regression controlling for number of ABA encounters. The second model was also significant and explained an additional 9% of variance in number of parent training encounters, (R2 = .10, F(3, 865) = 37 32.71, p < .001). When we controlled for number of ABA encounters in the second model, age (p = .71) no longer explained unique variance in parent training extensiveness. Hispanic/Latino ethnicity was marginally significant (β = -.06, p = .06), suggesting a trend toward Hispanic/Latino ethnicity predicting fewer encounters even when controlling for number of ABA encounters. See Table 9 for hierarchical regression model results. Table 9. Hierarchical Regression Model of Demographic Variables and Number of ABA Encounters Predicting Number of Parent Training Encounters N = 868 Variable Age Model 1 Model 2 β t β t -.08 -2.29** -.01 -.38 Table 9 (cont’d) Hispanic/Latino Ethnicity -.07 -2.08** -.06 -1.88* Number of ABA Encounters .31 9.39*** R2 F *p < .06, **p < .05, ***p < .001 .01 4.53* .10 32.71*** Provider Characteristics Associated with Parent Training Provider survey. The relationships between parent training extensiveness and a number of provider demographic, training, and caseload variables were examined using Pearson’s correlations for interval data, Spearman’s rho for ordinal data, and one-way ANOVA for categorical data. See Table 10 for bivariate analyses and Figure 3 for training experiences. Of these variables, number of training experiences related to parent training (r = .33, p = .001) and years of experience in working with children with ASD (rs = .24, p = .02) were associated with parent training extensiveness (all other ps > .05). 38 Figure 3. Training Experiences Related to Parent Training Observed During Practicum/Internship Engaged in Self-Guided Learning Received Supervision Observed Outside of Practicum/Internship Attended Workshop Taken Course Formal Training in Specific Intervention Approach Other Training Experience 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Table 10. Relationships Between Parent Training Extensiveness and Provider Demographic, Training, and Caseload Variables from the Survey Variable Age Gender r, F, or rs r = -.04 p .71 F = .50, df = 31 .98 Racial or Ethnic Minority Status F = .44, df = 31 .99 Years of Experience in Working with Children with ASD Years of Experience in Providing Parent Training rs = .24 rs = .15 .02 .16 Certification Type F = .81, df = 31 .73 Location of Service Provision F = 1.12, df = 31 .35 Number of Related Training Experiences r = .33 .001 Highest Level of Education Average Age of Clients on Caseload rs = -.01 rs = -.03 .97 .74 39 Table 10 (cont’d) Average Number of Clients Enrolled in Medicaid Autism rs = .14 .18 Benefit Hours Per Week Spent Seeing Clients in Medicaid Autism rs = .17 .12 Benefit Barriers to the Use of Parent Training Provider survey. A number of family-, provider-, and organization-level barriers were measured. See Table 11 for the correlations between specific barriers and parent training extensiveness. Believing that parent training was not appropriate for most clients (p = . 396) and parent training not being consistent with the provider’s theoretical orientation (p = .275) were the only barriers that were not significantly associated with parent training extensiveness. Table 11. Correlations Between Barriers and Parent Training Extensiveness (N = 92) Barriers Mean (SD) r p Family-Level Barriers Subscale (α = .68) 2.39 (.66) -.40 < .001 Difficult to Engage Families 3.06 (1.02) -.42 < .001 Families Are Not Interested in Participating 2.81 (1.08) -.27 .009 Clients Have Not Made Progress 2.26 (.96) -.26 .011 Not Appropriate for Most Clients 1.41 (.77) -.09 .396 Provider-Level Subscale (α = .77) 1.82 (.82) -.38 < .001 Not Consistent with Theoretical Orientation 1.35 (.74) -.12 .275 Not Enough Skills or Knowledge 2.19 (1.27) -.41 < .001 40 Table 11 (cont’d) Difficulty Applying to Clients’ Needs 1.90 (1.09) -.28 .007 Difficulty Tailoring to Clients and Their Needs 1.86 (1.06) -.32 .002 Organization-Level Subscale (α = .82) 2.44 (1.02) -.42 < .001 Lack of Agency Recognition 2.46 (1.28) -.26 .011 Lack of Access to Agency-Based Supervision 2.48 (1.32) -.38 < .001 Lack of Agency-Based Training 2.88 (1.49) -.42 < .001 Agency Priorities Do Not Include Parent Training 1.98 (1.19) -.26 .012 Sessions Conflict with Other Responsibilities 2.39 (1.35) -.29 .005 We conducted a hierarchical linear regression to determine which level of barriers explained unique variance in parent training extensiveness after controlling for significantly related provider demographic and training variables. In the first model, the only significantly associated provider demographic variable (years of experience in working with children with ASD) was entered, with the model explaining 5% in variance of parent training extensiveness. In the second model, the number of related training experiences was added, explaining an additional 8% of variance in parent training extensiveness. In the final model, the family-, provider-, and organization-level barriers were added and explained an additional 17% of variance. The final model was significant [F(5,86) = 7.32, p < .001)] and explained 30% of variance in parent training extensiveness. Family-level barriers explained unique variance in the final model (β = -.23, t = -2.29, p = .03). See Table 12 for the hierarchical regression model. 41 Table 12. Hierarchical Regression Model of Demographic Variables, Training Experiences, and Barriers Predicting Parent Training Extensiveness N = 92 Variable Model 1 Model 2 Model 3 β t β t Β t Years of experience with clients with ASD Number of training experiences Family-level barriers Provider-level barriers Organization-level barriers R2 F *p<.05; **p < .01, ***p < .001 .23 2.22* .16 1.56 .09 0.91 .29 2.85** .14 1.45 -.23 -2.29* -.17 -1.62 -.18 -1.57 .05 4.94* .13 .29 6.74** 7.32*** Interviews. During interviews, providers described barriers to using parent training for children enrolled in the Medicaid Autism Benefit. Horizontalization was utilized to identify significant statements in the transcripts and develop clusters of meaning that were refined into themes. The following 5 barrier themes emerged: a perception of limited family engagement and interest in parent training, family stressors and/or cultural differences from the provider, logistical barriers (e.g., scheduling concerns, lack of materials), lack of agency support for using parent training and/or using parent training is not an agency norm, and having limited training (pre-service and in-service) in how to provide parent training. The majority of providers mentioned each of these types of barriers (range: 69%-100%), suggesting that these are essential 42 barriers to the phenomenon of providing parent training to families of children with ASD through the Medicaid Autism Benefit. These barrier themes may be key areas to target in supporting the use of parent training in this context. See Table 13 for representational quotations of barriers and the percent of providers that mentioned each category of barriers. Table 13. Barriers Reported in Interviews (n = 13) Strategy Representational Quotation Limited Family Engagement and Interest in Parent Training Family Stressors and/or Cultural Differences from Provider Logistical Barriers Limited Agency Support and/or Using Parent Training is Not an Agency Norm Limited Training in Using Parent Training I think clinicians sometimes get worried that the parents will get overwhelmed with a lot of this information. Especially you know some of our Medicaid parents will view us more as a babysitter rather than something they have to actually do, so they kind of want us to fix their kid without having to actually do things and that can be tricky. Um usually if it’s a single parent or um you know one parent is um is in charge, usually the mother in charge of the care and well-being of the individual in question, um you get the uh the father usually is out of the home for a significant amount of time for financial strain or doesn’t have the same kind of nurturing approach, so it’s usually overburden like uh the parent, the mother is overburdened. There's location. So I work with families that are in very rural areas, so it's difficult to get out there all the time and to be honest they don't necessarily have any decent cell service or Wi-Fi service available either so getting new parent training with those families is often very difficult. I feel that my agency is probably indifferent to family training. …now being in the field and having different companies really push for this and require this and yet those coming out of school – it’s very intimidating having had little training in that area. Percent Who Mentioned 92% 69% 100% 85% 85% 43 Facilitators to the Use of Parent Training Provider survey. A number of family-, provider-, and organization-level facilitators were measured. All facilitators were significantly related to parent training extensiveness except for believing that parent training can be tailored to clients’ needs (p = .138). See Table 14 for the correlations between facilitators and parent training extensiveness. Due to low internal consistency of the provider-level facilitator subscale, the individual items were used rather than the overall subscale. Table 14. Correlations Between Facilitators and Parent Training Extensiveness (N = 92) Facilitators Mean (SD) R P Family-Level Facilitators Subscale (α = .79) 4.43 (.65) .38 < .001 Can Be Tailored to Clients’ Needs 4.69 (.65) .156 .138 Feedback from Parents is Positive 4.15 (.89) .38 < .001 Produces Positive Changes in Clients and Families 4.44 (.76) .39 < .001 Provider-Level Facilitators Subscale (α = .55)* Belief That It is Part of a Comprehensive Intervention 4.24 (.64) .21 .05 Program for Clients Have the Necessary Skills 3.98 (1.0) .45 < .001 Belief That Peers Also Provide It 4.02 (.96) .29 .005 Organization-Level Facilitators Subscale (α = .82) 3.75 (.89) .40 < .001 Agency Encourages Delivery 4.35 (.83) .25 .014 Agency Staff Provide Support 3.57 (1.29) .32 .002 Agency Provides Training Opportunities 2.84 (1.36) .43 < .001 44 Table 14 (cont’d) Agency Staff Believe Parent Training is Important 4.26 (.83) .25 .017 To determine which levels of facilitators explained unique variance in parent training extensiveness, we conducted a hierarchical linear regression that controlled for significantly related provider demographic and training variables. Because the provider-level facilitator subscale had poor internal consistency, the individual items were used in the hierarchical regression rather than the overall subscale. In the first model, years of experience in working with children with ASD was entered, with the model explaining 5% in variance of parent training extensiveness. In the second model, number of related training experiences was added, explaining an additional 8% of variance in parent training extensiveness. In the final model, the family-, and organization-level facilitator subscales and each of the 3 individual provider-level facilitators were entered, explaining an additional 17% of variance. The final model was significant [F(7,84) = 5.20, p < .001)] and explained 30% of variance in parent training extensiveness. In the final model, there were no unique predictors of parent training extensiveness, but number of related training experiences (p = .057), and having the necessary skills to do parent training (p = .063) were marginally statistically significant predictors. See Table 15 for the hierarchical regression model. 45 Table 15. Hierarchical Regression Model of Demographic Variables, Training Experiences, and Facilitators Predicting Parent Training Extensiveness N = 92 Variable Model 1 Model 2 Model 3 β t β t β t Years of experience with clients with ASD Number of training experiences Family-level facilitators Belief that it is a part of a comprehensive intervention program Having the necessary skills Belief that peers also do it Organization-level facilitators .23 2.22** .16 1.56 .01 0.11 .29 2.85*** .19 1.93* .05 .41 .09 .90 .24 1.89* .09 .89 .17 1.47 R2 F .05 .13 .30 4.94** 6.74*** 5.20**** *p<.07; **p < .05, ***p < .01, ****p < .001 Interviews. During interviews, providers reported on facilitators to use of parent training with their clients enrolled in the Medicaid Autism Benefit. Horizontalization was utilized to identify significant statements in the transcripts and develop clusters of meaning that were refined into themes. The following 4 facilitator themes emerged: high family engagement and interest in parent training, logistical facilitators (e.g., group format to increase number of clients that receive parent training, parent training manuals supplied by agency), agency support for 46 using parent training and/or using parent training is an agency norm, and having some training in how to provide parent training. If providers made a comment about wishing or hoping for a form of support that they did not currently have, this was also coded as a facilitator. See Table 16 for representational quotations and the percent of providers who mentioned each type of facilitator. All providers mentioned logistical facilitators and agency support and/or norms as facilitators, indicating that providers unanimously agreed that organization-level and logistical facilitators would be helpful in promoting parent training use in this context. Table 16. Facilitators Reported in Interviews (n = 13) Strategy Representational Quotation High Family Engagement and Interest in Parent Training But the other part is parents that are just looking to really help their child as opposed to just a quick fix. You know those parents that understand that autism is not a quick fix, they benefit more from parent training because they're not just looking for something that they can buy or create and just make everything better. Sometimes for some families it's just different personality types conflict. That's the way their life is, they want everything in a quick pass. We spend a lot of time explaining that this isn't going to be fixed right away. Percent Who Mentioned 54% Logistical Facilitators We have a small family training binder that the BCBAs at my clinic have access to. There are some handouts in there but there weren’t a lot, so we’re working to expand that and so when I have a topic I have found appropriate handouts for or created them for a family training, I’ve added that. I think they think it is very important and is a priority. I think they want to get it for as most families as possible and if the families are interested. Yeah at my university, a couple of professors had created a center for autism so we got practical experience from doing that for at least a semester, most of us it was a couple of years so we got to participate in Agency Support and Using Parent Training is an Agency Norm Training in Using Parent Training 100% 100% 23% 47 Table 16 (cont’d) a pop-up model and family training model so I felt well prepared for that. Relationship Between Parent Training Extensiveness and Family-Centered Care None of the provider demographic or training variables were related to level of FCC. Thus, a Pearson’s correlation was used to determine the association between parent training extensiveness and FCC. Parent training extensiveness and level of FCC were significantly correlated, r = .37, p < .001, with a moderate effect size. This suggests that a providers’ use of parent training is related to their level of collaborative care in general. See Table 17 for descriptive statistics the correlation between FCC and parent training extensiveness. Table 17. Correlation Between Family-Centered Care and Parent Training Extensiveness (N = 92) Mean (SD) r p Level of Family-Centered Care 5.17 (1.18) .38 < .001 48 DISCUSSION This study contributes to the field by utilizing mixed methods and multiple data sources to portray a more comprehensive depiction of the use of parent training for Medicaid-enrolled families of children with ASD served by community mental health agencies. Rates and Frequency of Parent Training Encounters As seen in the Medicaid claims data, only 55.1% of eligible children received parent training encounters over the 6 month time period. Those that did receive parent training received very few sessions overall (M = .46 encounters per month), which is not consistent with most evidence-based manualized parent training interventions (Ingersoll & Dvortcsak, 2009; Kasari et al., 2010; Rogers et al., 2012; Scahill et al., 2016). A discrepancy emerged in regards to the frequency of encounters seen in Medicaid claims data and provider self-report on the survey. Providers reported providing parent training to their clients much more frequently than what is reflected in the claims data. There are several potential explanations for this discrepancy. First, providers may be providing this service at frequencies that are not captured by the Medicaid claims data due to ways in which parent training encounters are billed. Preliminary analysis of interview excerpts related to the billing process suggests that many providers do not submit their own billing data into their agency’s electronic medical records and instead submit case notes to staff at a centralized billing office which then determines which codes to submit for reimbursement. Thus, it is possible that providers are providing parent training with their clients at a higher frequency than what is reported in the claims data, but yet it is not frequently billed for because of clerical errors made by billing office staff. It is also possible that providers believe that they are providing parent training, but may not be engaging in activities that are consistent with evidence-based parent training. This is not surprising, given that 85% of providers reported 49 that they had limited training in how to provide parent training. Thus, staff in the billing office may not not feel that the parent training billing code is appropriate based on descriptions of the service in the casenotes. In support of this possibility, the content analysis of provider definitions of parent training from the survey (Table 6) suggest that many providers may not be aware of evidence-based parent training strategies, and thus may provide descriptions in casenotes that are not consistent with parent training (e.g., checking in about child progress). Our preliminary analysis of interview excerpts related to the billing process supports this hypothesis; staff in the billing office may not submit parent training billing codes if the casenotes are not in alignment with the parent training billing codes. However, future research should examine this hypothesis more thoroughly. Social desirability, the tendency for people to present themselves as favorably as possible (King & Bruner, 2000), commonly influences survey responses (Phillips & Clancy, 1972). Thus, it is also possible that providers were influenced by social desirability when completing the survey and overreported how much they use parent training with their clients. As the Michigan Department of Health and Human Services has notified CMH agencies that the State goal was to have each child enrolled in the Medicaid Autism Benefit to receive at least 2 parent training encounters per quarter (3 months), providers had sufficient motivation to overreport their use of parent training. Taken together, although it is quite possible that providers are delivering parent training to their families more frequently than the Medicaid data suggest, the likelihood that most families are receiving parent training at a level of intensity to promote mastery in this context is low. 50 Strategy Use in Parent Training Sessions Providers reported using evidence-based strategies fairly frequently when provided with a Likert rating scale in the survey. However, the majority of providers who were interviewed did not mention some of these strategies spontaneously even after being primed at the beginning of the interview to discuss evidence-based parent training practices. In interviews, the majority of providers spontaneously mentioned modeling intervention strategies (69%) and providing time for parent practice (69%). Given that practicing intervention strategies has been shown to be the most important strategy to produce desired child outcomes (Wyatt Kaminski et al., 2008), this is encouraging. However, it is important to note that providers were asked at the beginning of the interview to reflect on parent training sessions that included time for parent practice (among other evidence-based strategies including collaboration, modeling, giving feedback, planning and reflecting on intervention strategy use, written information, and problem solving barriers to strategy implementation). Providers also mentioned 4 other strategies during interviews, in addition to the evidence-based strategies that we identified in the literature: checking in, treatment development/planning, social-emotional support of family members, and psychoeducation (see Table 8). Qualitative data from provider definitions of parent training and interviews converged, suggesting that providers use a variety of strategies during parent training sessions that are not associated with improved child outcomes. Training should address these misconceptions and bolster providers’ knowledge of evidence-based strategies to use during parent training sessions. Taken together, these results may indicate that providers do not discrimate between strategies that are evidence-based and those that are not. Providers may not be aware of the strategies that are most effective in improving child outcomes. Alternatively, 51 providers may use evidence-based strategies during routine care, but may not conceptualize it as parent training. While providers mentioned evidence-based parent training strategies more frequently in interviews, they likely had been primed to reflect on their practices with evidence-based parent training after hearing a definition that is consistent with the literature. Thus, they may have reported on exemplar parent training sessions rather than usual practice. Our rationale for providing this definition of evidence-based practice was to expand our knowledge of providers’ experiences in using evidence-based parent training models. However, the definitions provided on the survey may provide a more accurate depiction of usual practice as providers were not influenced by priming and were not prompted to describe their use of specific evidence-based strategies (as seen in the Likert scale ratings for evidence-based strategies in the survey); instead, providers were able to spontaneously mention strategies that they felt were central to parent training. Additionally, content analysis of providers’ definitions of parent training indicated that only 55% of providers mentioned evidence-based strategies in their definitions of parent training: modeling, practice and/or feedback, or psychoeducation. Psychoeducation was frequently mentioned by the full sample (51% of providers) and by 69% of the subset of 52 providers who mentioned any of those three evidence-based strategies in their definition of parent training. Most providers who mentioned an evidence-based strategy in their definition mentioned only one of the evidence-based strategies (see Figure 1), with 95% of those that mentioned only one evidence-based strategy mentioning psychoeducation. These results suggest that providers’ conceptualization of parent training frequently includes psychoeducation, yet psychoeducation was infrequently mentioned in conjunction with modeling and parent practice, which are more 52 closely aligned with adult learning and have been shown to result in improved child outcomes (Wyatt Kaminski et al., 2008). This suggests that providers are using psychoeducation differently from how it is used in evidence-based models. Providers may not be aware that psychoeducation is a necessary but not sufficient component of evidence-based parent training models and should be used in conjunction with modeling and parent practice. One key evidence-based strategy, collaborative goal-setting, was highly endorsed on the survey but infrequently mentioned spontaneously in interviews. Analysis of qualitative data from interviews suggested that when providers mentioned goal-setting, they typically described it in a provider-directed rather than collaborative manner. Indeed, when providers described goals in the context of parent training sessions, they typically described them in terms of carrying over goals that had already been identified by the provider in the child’s treatment plan. Developing a collaborative partnership between providers and parents has been shown to result in improved child outcomes (Brookman-Frazee & Koegel, 2004), with collaborative goal-setting being a crucial component of that partnership (Bailey, 1987; Moes & Frea, 2002). Collaborative goal- setting may be an important area to address to improve the quality of parent training sessions in this context. Format and Content of Parent Training Sessions Providers reported using parent training to cover a wide range of content. Medicaid claims data and provider self-report on the survey indicated that most providers use individual, one-on-one sessions of parent training. However, providers reported in interviews that group models would be a very helpful logistical facilitator to parent training use. One provider also reported a desire to be able to provide telehealth models of parent training: 53 “…the counties that I work in do not allow us to do tele-anything. We can't give tele- information or tele-parent training and I wanted to, especially in those rural areas in the winter time, it can be difficult to get to. It's dirt roads, really hilly and they're still, I'm close to [City Name] and [City Name], then I am, you know, the middle of the state where it's not plowed and not taken care of as nice. So with those families, tele-parent training would be, I don't want tele-supervision, but tele-parent training would be a huge benefit for them as well as myself.” While telehealth is currently an option for the delivery of parent training encounters in this system, billing claims data demonstrated that only 10 children (1%) received parent training via telehealth, and all of those children were from the same CMH agency. Telehealth models of parent training for children with ASD have demonstrated improved child and family outcomes, particularly when they include therapist assistance in working through the content (Ingersoll et al., 2016; Vismara, McCormick, Young, Nadhan, & Monlux, 2013). This could also be a potential facilitator to parent training use in the Medicaid Autism Benefit, especially for families that live in remote rural areas or who have a number of logistical barriers that make scheduling parent training sessions difficult. Client Characteristics Associated with Receipt of Parent Training Age and Hispanic/Latino ethnicity were significantly correlated with number of parent training encounters, with younger children receiving more encounters and Hispanic/Latino children receiving less. However, when we controlled for overall number of ABA services, age was no longer significant and Hispanic/Latino ethnicity became marginally significant. This suggests that while there may be some cultural differences and/or language barriers that influence Hispanic/Latino children’s receipt of parent training, Hispanic/Latino children’s 54 limited receipt of parent training may be better explained by their limited receipt of ABA encounters in general. This is consistent with previous studies that have found that Hispanic/Latino children with ASD receive significantly fewer ABA services than children of other ethnicities (Magaña et al., 2013). No other studies to our knowledge have examined racial and ethnic disparities in receipt of parent training. The present study demonstrates that there is some evidence to suggest a disparity in receipt of parent training for children for Hispanic/Latino children. Future research should further examine this finding in a larger sample with more racial and ethnic diversity. Additionally, future research should use multilevel modeling to parse out the effects of ethnicity in the context of nested data such as these Medicaid claims data (child within CMH agency). Barriers to Parent Training Use All measured barriers were associated with parent training extensiveness except for the beliefs that parent training is not consistent with one’s theoretical orientation and that parent training is not appropriate for most clients (see Table 9). This is unsurprising given that all providers need to have a background in ABA in order to be eligible to provide parent training under the Medicaid Autism Benefit. Barriers identified in previous literature regarding the use of the Triple P parent training program for children with challenging behavior seem to be appropriate for providers working with families of children with ASD as well. Adding the number of training experiences related to parent training and barrier subscales increased the amount of variance explained by the model. In the final model, only family-level barriers explained unique variance after controlling for years of experience in working with children with ASD, number of training experiences related to parent training, and provider- and organization-level barriers. This suggests that a perception of limited family engagement and 55 interest in parent training is a particularly pervasive barrier to use of parent training in this context. Qualitative data also corroborated this finding; 92% of providers reported that limited family engagement and interest in parent training was a barrier to their use of parent training. Future research should explore parent perspectives of their interest in parent training and engagement styles with providers to determine if this is an accurate observation made by providers or whether providers may have negative biases towards clients of lower-income backgrounds that are enrolled in the Medicaid Autism Benefit. Promoting a more positive perception of families enrolled in the Medicaid Autism Benefit and their interest in parent training may be one way to increase parent training use. Training providers in evidence-based strategies to engage families could also help to promote positive interactions between families and providers, as this has been demonstrated to increased parent participation in child therapy sessions and improved attendance (Haine-Schlagel, Martinez, Roesch, Bustos, & Janicki, 2018). Providers also noted many logistical barriers such as difficulty scheduling sessions, lack of readily available materials or manuals to use, large caseloads, transportation difficulties, and time restrictions on parent training encounter length. They also described limited training as a large barrier to parent training use; 85% of interviewed providers described having limited training in how to provide parent training and very few providers on the survey endorsed receiving structured pre-service training opportunities (e.g., taking a course about parent training) or training in a specific intervention approach (e.g., Project ImPACT, Early Start Denver Model). These results suggest that providers have not received adequate training on how to provide this service and should receive additional training at the pre-service and in-service levels as the number of training experiences are related to parent training extensiveness. Training efforts should also include supervision, as best practice in increasing providers’ use of an 56 evidence-based practice incorporates ongoing support strategies like supervision in addition to training (Beidas, Edmunds, Marcus, & Kendall, 2012; Herschell et al., 2010). Facilitators to Parent Training Use All measured facilitators on the survey were significantly correlated with parent training extensiveness except for the belief that parent training can be tailored to clients’ needs. Providers described in interviews that they had difficulty engaging with this population and infrequently engaged in collaborative goal-setting, two difficulties which may have negatively influenced their beliefs about how parent training can be individualized with sufficient buy-in from their clients. Results suggest that factors identified to be facilitators to the Triple P parent training program for children with challenging behavior also appear to be relevant for working with children with ASD. Additionally, increasing agency support may also increase use of parent training. In interviews, providers noted many ways in which agencies could be more supportive of their use of parent training, including providing materials or manuals to use, reducing competing demands on providers’ time, and conveying that parent training was an agency priority. To increase agency support, agencies could consider implementing a specific manualized parent training intervention and providing all needed materials for that intervention, thereby increasing logistical facilitators and decreasing unbillable time in which providers are expected to gather and/or create session materials. Agencies could also use incentives to promote providers’ use of parent training (e.g., financial incentives, public recognition). Relationship Between FCC and Parent Training Extensiveness Parent training and FCC were significantly correlated. This suggests that increasing use of parent training may also increase level of FCC. The Autism State Plan of Michigan 57 emphasizes collaborative care as a priority at the Michigan Department of Health and Human Services, and this positive correlation suggests that increasing use of parent training may also increase collaborative care in general. Indeed, promoting the use of parent training as a means to increase the overall level FCC that children with ASD receive would address state priorities as well as overall treatment quality for families of children with ASD, as FCC results in improved child and family outcomes (Dunst & Trivette, 2009; Kuhlthau et al., 2011; Kuo et al., 2012). This may be of particular importance for families of children with ASD in this context, who are already less likely to receive FCC in general than families of children with other emotional, developmental, or behavioral problems and children with other special healthcare needs (Kogan et al., 2008). Future research should examine the relationship between the use of parent training and FCC over time to determine how they may influence each other. Limitations and Future Directions Mixed methods were employed to provide a more complete picture of parent training in this system than the illustration that would have been accomplished by using quanitative or qualitative methods alone. The use of multiple data sources and analytic techniques provided a richer understanding of the current context of parent training provision in the Medicaid Autism Benefit. Overall, the results of the present study suggest that the quality of parent training sessions is unclear at best and likely limited. While we were not able to directly observe parent training sessions and measure quality in-vivo, our self-report measure of parent training extensiveness included quality (by measuring evidence-based parent training strategies) and frequency, which is likely a more accurate depiction of parent training practices than measuring either construct alone. Future research should include observational studies in which quality of parent training sessions can be directly observed and quantified. 58 Additionally, while our sample size for interviews was relatively small (n = 13), we did reach saturation after completing 13 interviews; no additional themes were described that had not already been described by at least one other provider. Triangulation of the quantitative and qualitative data helps to confirm much of our qualitative findings, with most quantitative results converging with qualitative results, suggesting that our findings are valid. However, future research should include more qualitative data to replicate these findings. While we had a large sample size for the Medicaid claims data (N = 879), the sample had a limited number children of Asian and American Indian/Alaskan Native races and may have limited our capacity to detect racial disparities in receipt of parent training. Future research should include more diverse samples and use multilevel modeling to analyze the nested data (child nested within CMH). The present study focused on providers’ perspectives on barriers and facilitators to the use of parent training, but using provider perspectives alone to characterize service use will not provide a complete depiction of the current landscape. Parent perspectives should also be measured as parents are the recipient of the service and can shed light on their level of interest in parent training and own barriers and facilitators to participating in parent training sessions. The perspectives of agency leaders would likely also contribute to a fuller understanding of the barriers and facilitators to parent training use in the Medicaid Autism Benefit. Future research should explore these perspectives, particularly with families from under-resourced backgrounds. This study demonstrates convergent evidence regarding community providers’ use of parent training for Medicaid-enrolled families of children with ASD. To our knowledge, it is the first study to investigate the use of parent training with underserved families of ASD using multiple data sources. 59 APPENDICES 60 APPENDIX A Parent Training Definition Codebook 61 Modeling Checking in Treatment development/planning Care coordination Unspecified Efforts to Support Intervention Strategy Use Social-emotional support Stress management Clinician demonstrates how to teach the child new skills or change the child's behavior. Any specific mention of modeling. Clinician answers questions from the family members and addresses any concerns that they have about the child; clinician gives family member verbal feedback about how the child is progressing in treatment. Clinician meets with family members to develop or adjust the treatment plan -- can be based on the family's goals and needs and can be done using a collaborative goal-setting process. The provider connects families to services and bridges gaps in family’s access to services. This is implemented either by the clinician meeting with the family to discuss service options and how to navigate the service system OR having the clinician directly contact and collaborate with other service providers who work with the child. Clinician explains how family should use the intervention strategies in naturalistic settings (e.g., home, grocery store rather than at school or clinic) but does not specifically model/demonstrate how to use the strategies. This is a broad description of supporting/helping the family to use the intervention strategies in some unspecified way. Clinician helps the family member to feel competent and in control of their child's future, giving the family encouragement and counseling to feel that they can be strong and support their child; motivating the family member to be actively involved in the child's care; any mention of improving family’s confidence in skills. This also includes helping the family deal with their struggles with child’s disability. The provider helps the family members to address external barriers to treatment (e.g., childcare, paying rent, other children with disabilities, many siblings present). 62 Psychoeducation Parent-mediated intervention for core symptoms of ASD Parent-mediated intervention for maladaptive behaviors Practice and feedback Educating the family about information related to their child’s condition or services. This can include ASD generally, information about efficacy and safety of interventions, treatment options, or ABA principles. Clinician trains the caregiver to implement a(n) intervention(s) to address the child's social interaction, communication, imitation, and/or play skills. Clinician trains the caregiver to implement a(n) intervention(s) to address the child's clinically significant disruptive behavior such as tantrums, aggression, noncompliance with routine demands, self-injury, property destruction, hyperactivity, food refusal, sleep disturbance, toileting problems, and elopement. The provider has the caregiver practice intervention strategies and/or provides feedback on intervention strategy use. Role play is considered practice. 63 APPENDIX B Interview Guide 64 Thank you so much for participating in today’s focus group to help us learn about your experiences with providing family training to your clients that receive services through the Michigan Medicaid “Autism Benefit” or Behavioral Health Treatment (BHT) services. Only de- identified information, or information without your name or other personal information tied to it, will be shared with your agency. The information that you share today will not be used to evaluate you in any way; we are just trying to understand, from a research perspective, how providers feel about providing family training with their clients. Do you have any questions before we begin? We are interested in understanding more about your opinions and practices with a particular kind family training, which is used with a variety of child health issues. The goal is to help parents feel more confident and skilled at helping their children. This type of family training is defined as a type of intervention for children with social-emotional and behavioral disorders, where the provider trains family members (like parents or other extended family members) to help change the child’s behavior. While family training looks different across therapies and providers, common components of a good program include: • Collaboration: Working with families to develop goals and monitor progress as a team • Modeling: Using video or live demonstration to show techniques to families • Practicing: Providing time for parent practice in sessions with role-play or live practice • Giving feedback: Provide parents with supportive feedback regarding their use of strategies • Planning and reflection: Supporting parents in planning/reflection on use of strategies at home • Written information: Providing written materials or manuals to support parent learning Problem solving: Working with parents to solve issues around implementation Please use this definition when answering these questions. 1. Have you ever provided family training with one of your clients billing with the Medicaid Autism Benefit? a. If yes, please tell us about what that has been like. i. What are your general impressions about family training? ii. What types of families do think family training is most appropriate for? iii. What types of activities do you do with families who you provide family training to? 65 iv. What makes it easier to provide family training for these clients? v. How do you bill for family training through the Medicaid Autism Benefit? vi. What don’t you like about providing family training? 2. What would you change to make providing family training more relevant and/or easy to do with your clients? 3. You’ve been asked by your agency to do many things with your clients (e.g. family training, direct service). What are some of these tasks, and how do you decide which ones to prioritize in a session? 4. Tell me about how you think your agency feels towards family training. a. In what ways are they supportive of you providing family training? b. In what ways are they not supportive? c. Tell me about whether you believe other providers at your agency provide family training, and why they may or may not be providing it. 5. To what extent did your pre-service training experiences in your graduate program prepare you to provide family training? 6. What else would you like to add that we haven’t asked you? Thank you so much for your time. We really appreciate it! 66 APPENDIX C Provider Interview Codebook 67 Barriers: Limited family engagement & interest in parent training Barriers: Logistical barriers to parent training Barrier: Limited agency support & perceived norms Barrier: Limited training in parent training Barrier: Family stressors or cultural differences Evidence-based strategies used in parent training sessions Other strategies used in parent training sessions The provider perceives limited motivation from caregivers and/or receives negative feedback from caregivers regarding the use of parent training. The provider notes that the family has a “fix it” mentality and desires provider to work with child to “fix” them without family input. The provider describes that the family focuses on the child’s challenging behavior and/or skill deficits without wanting to be involved to address it. The provider notes difficulty in providing parent training due to logistical concerns like location (clinic/home/school), caseload, scheduling (including schedule of provider or family), waitlist, limited/no manuals or materials, difficulty with transportation, etc. The provider describes that their agency does not prioritize parent training as a service OR puts too many competing demands on providers OR the provider does not believe that their peers are using parent training. The provider may also note that their agency does not provide adequate compensation, incentives, or materials. The provider describes having little or no training on how to provide parent training either pre-service (in graduate programs) or in-service (while practicing currently). The provider notes that the family has major financial concerns (including working multiple jobs or long hours because of financial concerns), multiple children with special needs, lives in remote/rural area, has different views on parenting due to cultural factors, many distractors in the home, poor mental health of caregivers, etc. and that those factors create barriers to using parent training. The provider describes modeling or live video demonstration or observation (e.g., one-way mirror) and practice and/or feedback. The provider describes checking in, treatment development/planning, care coordination, social-emotional support, stress management, psychoeducation, or does not describe 68 Skill areas for parent training sessions Billing process Facilitator: Agency support & perceived norms Parent training as a requirement for families Facilitator: High family engagement and interest in parent training Facilitator: Logistical facilitators for parent training use Facilitator: Training in parent training practice and feedback while describing parent-mediated interventions for core symptoms or maladaptive behavior. The provider mentions the content that they target during parent training sessions: play skills, imitation skills, communication skills, academic skills, coping skills to reduce challenging behavior, etc. The provider describes the process for documenting parent training session content and how their agency bills for parent training sessions. The provider describes agency support and incentives, including agency recognition for doing parent training, rewards, a sense that the agency values parent training, etc. This can also include the provider perceiving that peers at their agency are also doing parent training. The provider notes that the agency already has a requirement for families to receive parent training occasionally OR the provider wishes a requirement existed so that families would be more likely to want to participate in parent training. The provider perceives high motivation and/or interest from caregivers to do parent training or receives positive feedback about parent training sessions from family members. The provider mentions that families trust them and are actively engaged in sessions. The provider describes how location (clinic/home/school), timing of sessions (e.g., at drop-off or pickup), caseload variables (e.g., smaller caseload), group parent training formats, parent training manuals/materials, etc. help them to provide parent training to their clients. The provider has been trained in a manualized parent training intervention or in specific approaches to working with families in parent training sessions (e.g., behavioral skills training approach). This training can occur in pre-service (graduate program) or in-service (since starting their practice). 69 Positive impressions of parent training as an intervention Negative impression of parent training as an intervention Description of which families parent training is most appropriate for The provider notes positive aspects of parent training as an intervention, including improved skill generalization, skill maintenance, and overall family quality of life. The provider feels that parent training is not a service that benefits families or the provider has other negative feelings towards parent training as an intervention. The provider notes qualities of families who they believe parent training is most and least appropriate for (e.g., most appropriate for highly educated caregivers). 70 REFERENCES 71 REFERENCES Aarons, G. A. (2005). Measuring Provider Attitudes Toward Evidence-Based Practice: Consideration of Organizational Context and Individual Differences. Child and Adolescent Psychiatric Clinics of North America, 14(2), 255–viii. https://doi.org/10.1016/j.chc.2004.04.008 Ajzen, I. (1985). From Intentions to Actions: A Theory of Planned Behavior. In Action Control (pp. 11–39). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-69746-3_2 American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Pub. Bailey, D. B. (1987). Collaborative Goal-Setting with Families: Resolving Differences in Values and Priorities for Services. Topics in Early Childhood Special Education, 7(2), 59–71. https://doi.org/10.1177/027112148700700207 Barton, E. E., & Fettig, A. (2013). Parent-Implemented Interventions for Young Children With Disabilities: A Review of Fidelity Features. Journal of Early Intervention, 35(2), 194– 219. https://doi.org/10.1177/1053815113504625 Bearss, K., Burrell, T. L., Stewart, L., & Scahill, L. (2015). Parent Training in Autism Spectrum Disorder: What’s in a Name? Clinical Child and Family Psychology Review, 18(2), 170– 182. https://doi.org/10.1007/s10567-015-0179-5 Beidas, R. S., Edmunds, J. M., Marcus, S. C., & Kendall, P. C. (2012). Training and Consultation to Promote Implementation of an Empirically Supported Treatment: A Randomized Trial. Psychiatric Services, 63(7), 660–665. https://doi.org/10.1176/appi.ps.201100401 Beidas, R. S., & Kendall, P. C. (2010). Training Therapists in Evidence-Based Practice: A Critical Review of Studies From a Systems-Contextual Perspective. Clinical Psychology : A Publication of the Division of Clinical Psychology of the American Psychological Association, 17(1), 1–30. https://doi.org/10.1111/j.1468-2850.2009.01187.x Beidas, R., Skriner, L., Adams, D., Wolk, C. B., Stewart, R. E., Becker-Haimes, E., … Marcus, S. C. (2017). The relationship between consumer, clinician, and organizational characteristics and use of evidence-based and non-evidence-based therapy strategies in a public mental health system. Behaviour Research and Therapy, 99, 1–10. https://doi.org/10.1016/j.brat.2017.08.011 Brookman-Frazee, L., & Koegel, R. L. (2004). Using Parent/Clinician Partnerships in Parent Education Programs for Children with Autism. Journal of Positive Behavior Interventions, 6(4), 195–213. https://doi.org/10.1177/10983007040060040201 72 Carter, A. S., Messinger, D. S., Stone, W. L., Celimli, S., Nahmias, A. S., & Yoder, P. (2011). A randomized controlled trial of Hanen’s ‘More Than Words’ in toddlers with early autism symptoms. Journal of Child Psychology and Psychiatry, 52(7), 741–752. https://doi.org/10.1111/j.1469-7610.2011.02395.x Dunst, C. J., & Trivette, C. M. (2009). Meta-Analytic Structural Equation Modeling of the Influences of Family-Centered Care on Parent and Child Psychological Health [Research article]. https://doi.org/10.1155/2009/576840 Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157. https://doi.org/10.1186/s13012- 014-0157-1 Fleming, J. L., Sawyer, L. B., & Campbell, P. H. (2011). Early Intervention Providers’ Perspectives About Implementing Participation-Based Practices. Topics in Early Childhood Special Education, 30(4), 233–244. https://doi.org/10.1177/0271121410371986 Fountain, C., King, M. D., & Bearman, P. S. (2011). Age of diagnosis for autism: individual and community factors across 10 birth cohorts. Journal of Epidemiology and Community Health; London, 65(6), 503. http://dx.doi.org.proxy2.cl.msu.edu/10.1136/jech.2009.104588 Haine-Schlagel, R., Martinez, J. I., Roesch, S. C., Bustos, C. E., & Janicki, C. (2018). Randomized Trial of the Parent And Caregiver Active Participation Toolkit for Child Mental Health Treatment. Journal of Clinical Child & Adolescent Psychology, 47(sup1), S150–S160. https://doi.org/10.1080/15374416.2016.1183497 Harrington, C., & Kang, T. (2008). Disparities in service utilization and expenditures for individuals with developmental disabilities. Disability and Health Journal, 1(4), 184– 195. https://doi.org/10.1016/j.dhjo.2008.05.004 Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. https://doi.org/10.1016/j.cpr.2010.02.005 Hume, K., Bellini, S., & Pratt, C. (2005). The Usage and Perceived Outcomes of Early Intervention and Early Childhood Programs for Young Children With Autism Spectrum Disorder. Topics in Early Childhood Special Education, 25(4), 195–207. https://doi.org/10.1177/02711214050250040101 Iadarola, S., Levato, L., Harrison, B., Smith, T., Lecavalier, L., Johnson, C., … Scahill, L. (2018). Teaching Parents Behavioral Strategies for Autism Spectrum Disorder (ASD): 73 Effects on Stress, Strain, and Competence. Journal of Autism and Developmental Disorders, 48(4), 1031–1040. https://doi.org/10.1007/s10803-017-3339-2 Ingersoll, B., & Dvortcsak, A. (2009). Teaching Social Communication to Children with Autism: A Practitioner’s Guide to Parent Training and a Manual for Parents. Guilford Press. Ingersoll, B., Wainer, A. L., Berger, N. I., Pickard, K. E., & Bonter, N. (2016). Comparison of a Self-Directed and Therapist-Assisted Telehealth Parent-Mediated Intervention for Children with ASD: A Pilot RCT. Journal of Autism and Developmental Disorders, 46(7), 2275–2284. https://doi.org/10.1007/s10803-016-2755-z Kalkbrenner, A. E., Daniels, J. L., Emch, M., Morrissey, J., Poole, C., & Chen, J.-C. (2011). Geographic Access to Health Services and Diagnosis with an Autism Spectrum Disorder. Annals of Epidemiology, 21(4), 304–310. https://doi.org/10.1016/j.annepidem.2010.11.010 Karst, J. S., & Van Hecke, A. V. (2012). Parent and Family Impact of Autism Spectrum Disorders: A Review and Proposed Model for Intervention Evaluation. Clinical Child and Family Psychology Review, 15(3), 247–277. https://doi.org/10.1007/s10567-012- 0119-6 Kasari, C., Gulsrud, A. C., Wong, C., Kwon, S., & Locke, J. (2010). Randomized Controlled Caregiver Mediated Joint Engagement Intervention for Toddlers with Autism. Journal of Autism and Developmental Disorders; New York, 40(9), 1045–1056. http://dx.doi.org.proxy2.cl.msu.edu/10.1007/s10803-010-0955-5 Kasari, C., Gulsrud, A., Paparella, T., Hellemann, G., & Berry, K. (2015). Randomized comparative efficacy study of parent-mediated interventions for toddlers with autism. Journal of Consulting and Clinical Psychology, 83(3), 554–563. https://doi.org/10.1037/a0039080 King, M. F., & Bruner, G. C. (2000). Social desirability bias: A neglected aspect of validity testing. Psychology & Marketing, 17(2), 79–103. https://doi.org/10.1002/(SICI)1520- 6793(200002)17:2<79::AID-MAR2>3.0.CO;2-0 Koegel, R. L., Bimbela, A., & Schreibman, L. (1996). Collateral effects of parent training on family interactions. Journal of Autism and Developmental Disorders, 26(3), 347–359. https://doi.org/10.1007/BF02172479 Koegel, R. L., Symon, J. B., & Koegel, L. K. (2002). Parent Education for Families of Children with Autism Living in Geographically Distant Areas. Journal of Positive Behavior Interventions, 4(2), 88–103. https://doi.org/10.1177/109830070200400204 Kogan, M. D., Strickland, B. B., Blumberg, S. J., Singh, G. K., Perrin, J. M., & Dyck, P. C. van. (2008). A National Profile of the Health Care Experiences and Family Impact of Autism 74 Spectrum Disorder Among Children in the United States, 2005–2006. Pediatrics, 122(6), e1149–e1158. https://doi.org/10.1542/peds.2008-1057 Kuhlthau, K. A., Bloom, S., Cleave, J. V., Knapp, A. A., Romm, D., Klatka, K., … Perrin, J. M. (2011). Evidence for Family-Centered Care for Children With Special Health Care Needs: A Systematic Review. Academic Pediatrics, 11(2), 136-143.e8. https://doi.org/10.1016/j.acap.2010.12.014 Kuo, D. Z., Houtrow, A. J., Arango, P., Kuhlthau, K. A., Simmons, J. M., & Neff, J. M. (2012). Family-Centered Care: Current Applications and Future Directions in Pediatric Health Care. Maternal and Child Health Journal, 16(2), 297–305. https://doi.org/10.1007/s10995-011-0751-7 Liptak, G. S., Benzoni, L. B., Mruzek, D. W., Nolan, K. W., Thingvoll, M. A., Wade, C. M., & Fryer, G. E. (2008). Disparities in Diagnosis and Access to Health Services for Children with Autism: Data from the National Survey of Children??s Health: Journal of Developmental & Behavioral Pediatrics, 29(3), 152–160. https://doi.org/10.1097/DBP.0b013e318165c7a0 Magaña, S., Lopez, K., Aguinaga, A., & Morton, H. (2013). Access to Diagnosis and Treatment Services Among Latino Children With Autism Spectrum Disorders. Intellectual and Developmental Disabilities; Washington, 51(3), 141–153. Mahoney, G., & Perales, F. (2005). Relationship-Focused Early Intervention With Children With Pervasive Developmental Disorders and Other Disabilities: A Comparative Study. Journal of Developmental & Behavioral Pediatrics, 26(2), 77. Mandell, D. S., Novak, M. M., & Zubritsky, C. D. (2005). Factors Associated With Age of Diagnosis Among Children With Autism Spectrum Disorders. Pediatrics, 116(6), 1480– 1486. https://doi.org/10.1542/peds.2005-0185 Mandell, D. S., Wiggins, L. D., Carpenter, L. A., Daniels, J., DiGuiseppi, C., Durkin, M. S., … Kirby, R. S. (2009). Racial/Ethnic Disparities in the Identification of Children With Autism Spectrum Disorders. American Journal of Public Health, 99(3), 493–498. https://doi.org/10.2105/AJPH.2007.131243 Matson, M. L., Mahan, S., & Matson, J. L. (2009). Parent training: A review of methods for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 3(4), 868–875. https://doi.org/10.1016/j.rasd.2009.02.003 Mello, M. P., Urbano, R. C., Goldman, S. E., & Hodapp, R. M. (2016). Services for Children with Autism Spectrum Disorder: Comparing Rural and Non-Rural Communities. Education and Training in Autism and Developmental Disabilities; Arlington, 51(4), 355–365. 75 Moes, D. R., & Frea, W. D. (2002). Contextualized Behavioral Support in Early Intervention for Children with Autism and Their Families. Journal of Autism and Developmental Disorders, 32(6), 519–533. https://doi.org/10.1023/A:1021298729297 Montes, G., Halterman, J. S., & Magyar, C. I. (2009). Access to and Satisfaction With School and Community Health Services for US Children With ASD. Pediatrics, 124(Supplement 4), S407–S413. https://doi.org/10.1542/peds.2009-1255L Moustakas, C. (1994). Phenomenological Research Methods. SAGE. National Autism Center. (2015). Findings and conclusions: National standards project, phase 2. Randolph, MA: Author. Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed Method Designs in Implementation Research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z Phillips, D. L., & Clancy, K. J. (1972). Some Effects of “Social Desirability” in Survey Studies. American Journal of Sociology, 77(5), 921–940. Pickard, K. E., Kilgore, A. N., & Ingersoll, B. R. (2016). Using community partnerships to better understand the barriers to using an evidence-based, parent-mediated intervention for autism spectrum disorder in a Medicaid system. American Journal of Community Psychology, 57(3–4), 391–403. http://dx.doi.org.proxy2.cl.msu.edu/10.1002/ajcp.12050 Rocha, M. L., Schreibman, L., & Stahmer, A. C. (2007). Effectiveness of Training Parents to Teach Joint Attention in Children With Autism. Journal of Early Intervention, 29(2), 154–172. https://doi.org/10.1177/105381510702900207 Rogers, S. J., Dawson, G., & Vismara, L. A. (2012). An Early Start for Your Child with Autism: Using Everyday Activities to Help Kids Connect, Communicate, and Learn. Guilford Press. Ruppert, T., Machalicek, W., Hansen, S. G., Raulston, T., & Frantz, R. (2016). Training Parents to Implement Early Interventions for Children with Autism Spectrum Disorders. In R. Lang, T. B. Hancock, & N. N. Singh (Eds.), Early Intervention for Young Children with Autism Spectrum Disorder (pp. 219–256). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-30925-5_8 Sanders, M. R., Prinz, R. J., & Shapiro, C. J. (2009). Predicting Utilization of Evidence-Based Parenting Interventions with Organizational, Service-Provider and Client Variables. Administration and Policy in Mental Health and Mental Health Services Research, 36(2), 133–143. https://doi.org/10.1007/s10488-009-0205-3 76 Scahill, L., Bearss, K., Lecavalier, L., Smith, T., Swiezy, N., Aman, M. G., … Johnson, C. (2016). Effect of Parent Training on Adaptive Behavior in Children With Autism Spectrum Disorder and Disruptive Behavior: Results of a Randomized Trial. Journal of the American Academy of Child & Adolescent Psychiatry, 55(7), 602-609.e3. https://doi.org/10.1016/j.jaac.2016.05.001 Shapiro, C. J., Prinz, R. J., & Sanders, M. R. (2012). Facilitators and Barriers to Implementation of an Evidence-Based Parenting Intervention to Prevent Child Maltreatment: The Triple P-Positive Parenting Program. Child Maltreatment, 17(1), 86–95. https://doi.org/10.1177/1077559511424774 Suppo, J., & Floyd, K. (2012). Parent Training for Families who have Children with Autism: A Review of the Literature. Rural Special Education Quarterly, 31(2), 12–26. https://doi.org/10.1177/875687051203100203 Stahmer, A. C., Vejnoska, S., Iadarola, S., Straiton, D., Segovia, F. R., Luelmo, P., … Kasari, C. (2019). Caregiver Voices: Cross-Cultural Input on Improving Access to Autism Services. Journal of Racial and Ethnic Health Disparities. https://doi.org/10.1007/s40615-019- 00575-y Thomas, K. C., Ellis, A. R., McLaurin, C., Daniels, J., & Morrissey, J. P. (2007). Access to Care for Autism-Related Services. Journal of Autism and Developmental Disorders, 37(10), 1902–1912. https://doi.org/10.1007/s10803-006-0323-7 Thomas, K. C., Morrissey, J. P., & McLaurin, C. (2007). Use of Autism-Related Services by Families and Children. Journal of Autism and Developmental Disorders, 37(5), 818–829. https://doi.org/10.1007/s10803-006-0208-9 Thomas, P., Zahorodny, W., Peng, B., Kim, S., Jani, N., Halperin, W., & Brimacombe, M. (2012). The association of autism diagnosis with socioeconomic status. Autism, 16(2), 201–213. https://doi.org/10.1177/1362361311413397 U.S. Census Bureau (2010). Urban and rural classification and urban area criteria. Retrieved from https://www.census.gov/geo/reference/ua/urban-rural-2010.html. Vismara, L. A., McCormick, C., Young, G. S., Nadhan, A., & Monlux, K. (2013). Preliminary Findings of a Telehealth Approach to Parent Training in Autism. Journal of Autism and Developmental Disorders, 43(12), 2953–2969. https://doi.org/10.1007/s10803-013-1841- 8 Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A., Kucharczyk, S., … Schultz, T. R. (2015). Evidence-Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder: A Comprehensive Review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. https://doi.org/10.1007/s10803-014-2351-z 77 Woodside, J. M., Rosenbaum, P. L., King, S. M., & King, G. A. (2001). Family-Centered Service: Developing and Validating a Self-Assessment Tool for Pediatric Service Providers. Children’s Health Care, 30(3), 237–252. https://doi.org/10.1207/S15326888CHC3003_5 Wyatt Kaminski, J., Valle, L. A., Filene, J. H., & Boyle, C. L. (2008). A Meta-analytic Review of Components Associated with Parent Training Program Effectiveness. Journal of Abnormal Child Psychology, 36(4), 567–589. https://doi.org/10.1007/s10802-007-9201-9 78