USING AN INTERNET-BASED TRAINING PROGRAM TO DISSEMINATE NATURALISTIC BEHAVIORAL TECHNIQUES TO INDIVIDUALS WORKING WITH YOUNG CHILDREN WITH AUTISM By Allison L. Wainer A THESIS Submitted to Michigan State University in partial fulfillment of the requirement for the degree of MASTER OF ARTS Psychology 2011 ABSTRACT USING AN INTERNET-BASED TRAINING PROGRAM TO DISSEMINATE NATURALISTIC BEHAVIORAL TECHNIQUES TO INDIVIDUALS WORKING WITH YOUNG CHILDREN WITH AUTISM By Allison L. Wainer There is an identified need for the adaptation of training in evidence-based interventions to non-traditional service delivery methods, particularly for individuals working with children with autism spectrum disorders (ASD). Internet-based instructional formats have been shown to be an effective means of dissemination of intervention training for various clinical populations. As such, an internet-delivered intervention training program was created to introduce therapists and parents to reciprocal imitation training, a naturalistic behavioral intervention that has been shown to increase imitation on objects and gestures in young children with autism. Two separate multiple-baseline design studies were conducted to assess the impact of this internet-based training program on changes in therapist (study 1) and parent (study 2) knowledge and behavior, and changes in child behavior. Additionally, data examining the acceptability of the program were collected. Therapists and parents improved their knowledge and use of the intervention strategies in response to the internet-based training program. However, some individuals required additional live coaching in order to reach fidelity of implementation. The children improved their rates of imitation, but changes in rates of child supported joint attention were not observed. Results from this study suggest that an internet-delivered training program may be an effective method for disseminating evidence-based practices to individuals working with children with ASD. TABLE OF CONTENTS LIST OF TABLES ...........................................................................................................v LIST OF FIGURES .........................................................................................................vi INTRODUCTION ...........................................................................................................1 Distance Learning Programs ................................................................................1 Distance Learning Programs for ASD Providers .................................................2 Parent Training for Families of Children with ASD ............................................4 Parent Training and Distance Learning Programs ...............................................5 Parent Training and Self-Directed Distance Learning Programs ........................6 Self-Directed Distance Learning Program for Parents of Children with ASD ....9 Purpose of this Research ......................................................................................10 STUDY 1 .........................................................................................................................13 Method .................................................................................................................13 Participants ...............................................................................................13 Settings and Materials ..............................................................................14 Training Program .....................................................................................14 Experimental Design and Procedure ........................................................16 Dependent Measures ................................................................................16 Inter-Observer Reliability ........................................................................18 Results ..................................................................................................................18 Program Utilization..................................................................................18 Therapist Knowledge of RIT ...................................................................19 Fidelity of Implementation ......................................................................19 Child Imitation .........................................................................................19 Child Supported Joint Attention ..............................................................20 Summary ..................................................................................................20 STUDY 2 .........................................................................................................................21 Method .................................................................................................................21 Participants ...............................................................................................21 Settings and Materials ..............................................................................23 Training Program .....................................................................................23 Experimental Design and Procedure ........................................................23 Dependent Measures ................................................................................24 Inter-Observer Reliability ........................................................................24 Results ..................................................................................................................25 Program Utilization ...................................................................................25 Parent Knowledge of RIT..........................................................................25 Fidelity of Implementation ........................................................................25 Child Imitation ..........................................................................................26 Child Supported Joint Attention ................................................................26 Treatment Acceptability ............................................................................26 iii DISCUSSION ..................................................................................................................28 Limitations ...........................................................................................................32 Summary ..............................................................................................................33 REFERENCES ................................................................................................................43 iv LIST OF TABLES Table 1. Child participant characteristics.........................................................................34 Table 2. Behavioral definitions ........................................................................................35 Table 3. Average treatment acceptability ratings by BIRS scale.....................................36 v LIST OF FIGURES Figure 1. Therapist-child interaction: Fidelity of implementation...................................37 Figure 2. Therapist-child interaction: Child imitation rates.............................................38 Figure 3. Therapist-child interaction: Child supported joint attention ............................39 Figure 4. Parent-child interaction: Parent fidelity of implementation .............................40 Figure 5. Parent-child interaction: Child imitation rates .................................................41 Figure 6. Parent-child interaction: Child supported joint attention .................................42 vi INTRODUCTION Recent epidemiological reports suggest that the number of children diagnosed with autism spectrum disorders (ASD) is rising, with as many as 1 out of every 150 children receiving this diagnosis (Centers for Disease Control and Prevention, 2007). While the rate of the diagnosis has increased, there has not been a corresponding growth in the dissemination of evidenced-based interventions for children with autism. This, in turn, has engendered a serviceneed discrepancy for children with autism and their families (Sperry, Whaley, Shaw, & Brame, 1999; Stahmer & Gist, 2001; Symon, 2005). Additionally, recommendations from the National Research Council state that children with autism should receive specialized services for at least 25 hours a week, 12 months per year (National Research Council, 2001). Given this populations’ need for intensive intervention and the growing number of children requiring such services, an expansion in the availability of, and access to, evidence-based treatments is essential. The National Research Council’s (2001) report on educating children with autism concluded that many individuals working with children with ASD do not receive sufficient instruction in evidence-based intervention techniques. Barriers associated with training therapists in evidence-based intervention techniques include limited monetary resources, significant time demands, and problems with the portability of intervention from the research laboratory to existing clinical settings (Kazdin, 2004). As such, it is necessary to consider training models in which these barriers can be overcome in time- and cost-effective ways. Distance Learning Programs The use of computer and internet technology can help address, and surmount, many of the challenges associated with traditional service delivery models by granting remote access to evidence-based practices (Symon, 2001). The percentage of US households with a computer has 1 grown dramatically over the past two decades from 22% in 1990, to 58% in 2000 to 78% in 2008 (U.S. Census Bureau, 2009). As of 2007, 71% of US households had access to the internet and nearly 83% of adults were able to access the internet from home, work, or elsewhere (U.S. Census Bureau, 2009). Public libraries and internet cafes provide additional access to computers and the internet. Given the extent of computer and internet access, computer-delivered service models seem to be a promising alternative delivery method. There are numerous benefits associated with using computer and internet-based technology to disseminate evidence-based practices, including the potential for intervention to be accessed from anywhere at any time. Computerized instruction allows for the individualization of training to meet the specific needs of an individual, while keeping the instruction highly standardized and maintaining fidelity of program implementation (Hollon et al., 2002; Mandel, Bigelow, & Lutzker, 1998). Training via computers and the internet also favors an exciting and interactive learning experience, allowing for the combination of many instructional formats including graphics, animation, video, and audio. This format makes it possible for the learner to directly interact with the instructional content; hypothetical situations, vignettes and practice exercises can be transformed from words on a page into rich media forms such as video or animation which then can be used to develop and test participant knowledge (Weingardt, 2004). Distance Learning Programs for ASD Providers Initial research has indicated that technology-based instruction is an effective means for disseminating knowledge to various populations (Hollon et al., 2002). Computer and internetdelivered programs have been utilized to train professionals in a variety of health-related settings, including clinicians who work with individuals with ASD (Benjamin et al., 2008; Granpeesheh, Tarbox, Dixon, Peters, Thompson, & Kenzer, 2010; Weingardt, Cucciare, Bellotti 2 & Lai, 2009). Granpeesheh and colleagues (2010) compared the effectiveness of an eLearning program with traditional live didactic training to teach new behavioral therapists the principles and procedures involved with Applied Behavior Analysis (ABA). The eLearning program consisted of several training modules with animated slides, audio recordings, and video demonstrations; after completing the modules, participants attended a two-hour follow-up discussion with an expert therapist to address questions about the teaching techniques. Results from this study indicate that participants in both training groups significantly increased their knowledge about ABA principles and procedures, with those in the traditional training group demonstrating slightly more gains than those in the eLearning condition (Granpeesheh et al., 2010). These results suggest that a computerized training program can be an effective information delivery system; however, the support from expert trainers seemed to produce additional learning benefits. Importantly, is unclear how these gains in knowledge would translate to the ability to correctly implement the ABA techniques. Some professional training programs have incorporated computerized content with direct expert coaching. One recent study assessed the effectiveness of a DVD-delivered training program for community-based therapists working with children with autism and their families (Vismara, Young, Stahmer, Griffith, & Rogers, 2009). After participating in the digital training, therapists’ implementation of intervention techniques improved significantly, suggesting the potential for success of this type of education delivery model. It is important to note that the majority of therapists required didactic instruction and team supervision from a professional in order to achieve fidelity of implementation, suggesting the utility of expert support in such programs (Vismara et al., 2009). 3 Parent Training for Families of Children with ASD Although the use of technology-based training formats can increase service providers’ access to training in evidence-based intervention techniques, this is not necessarily sufficient to ensure that children with ASD will be provided with more, or enough, access to evidence-based intervention. Even when armed with knowledge of evidence-based techniques, therapists can only spend so much time working directly with a given child. As such, it is critical to consider training other key individuals, such as parents, in evidence-based intervention techniques. Parent training is an especially cost-effective and ecologically valid way to bolster the amount of intervention a child receives. With respect to autism intervention, parent training and family involvement in intervention has been cited as a fundamental component of effective intervention programs (National Research Council, 2001). A growing body of literature suggests that parents can be successfully trained in techniques to improve the quality of parent-child interactions (Koegel, Bimbela, & Schreibman, 1996; Symon, 2001; Mahoney & Perales, 2003), to produce gains in language skills (Charlop & Trasowech, 1991; Rogers et al., 2006), imitation skills (Ingersoll & Gergans, 2007), joint attention and joint engagement (Drew et al., 2002) and appropriate play skills (Stahmer, 1995), and to decrease problem behavior (Moes & Frea, 2002) in young children with autism. There are additional benefits to parent training, including increases in generalization and maintenance of child skill, a reduction in parent stress, and increases in family leisure time (Koegel, et al., 1996; Koegel, Schreibman, Britten, Burke & O’Neill, 1982). However, there are several barriers associated with accessing parent training programs via traditional clinic-based service delivery models. Limited financial resources, limited transportation, lack of child care, and geographic isolation have all been cited as obstacles 4 individuals face when trying to enroll in clinic-based intervention or parent training programs (Stahmer & Gist, 2001; Symon, 2001; Taylor, et al., 2008). Moreover, lengthy waitlists and extensive time commitments involved in various programs make enrollment in services and continued participation difficult for many individuals (Stahmer & Gist, 2001). As such, there is an identified need for the adaptation of parent training in evidence-based interventions to nontraditional service delivery models (Feil et al., 2008). Parent Training and Distance Learning Programs Similar to the therapist training programs described above, service delivery models integrating computer and internet-delivered training with active professional support have also been used to train parents in evidence-based practices (Taylor et al., 2008; Feil et al., 2008). Taylor and colleagues (2008) developed a service delivery model, blending computerized and internet-based informational content with personal coaching, to introduce parents to the Incredible Years Parenting Program, a psychoeducational intervention aimed at decreasing behavior problems in young children. The creators utilized a number of media formats including videos, texts and graphics presented on screen, along with optional audio lectures through a secure website. Participants watched video vignettes and answered questions similar to those that a live facilitator would ask in a group training session. Coaches communicated with parent participants through in-person home visits, regularly scheduled phone calls and electronic messages via a message board on the secure website. After participating in this type of training program, parents reported high levels of goal achievement and were quite satisfied with the program and training received. Moreover, attrition rates for this program were rather low relative to clinic-based training programs, with 76% of participants completing the entire training and 82% of participants completing at least half of the program (Taylor et al., 2008). 5 While a delivery model such as the one described above may be effective for distributing information about evidence-based practices, it still requires live, in-person contact between participants and coaches. This may be difficult for families living in remote areas or for whom scheduling home visits is a problem. Additionally, there are acute costs associated with supporting personnel salary and travel reimbursement. Recognizing these barriers, Feil and colleagues (2008) developed a service delivery model that required fewer personnel, limited travel and was based solely on the use of remote technology. The interactive program InfantNet, used The Playing and Learning Strategies program (PALS) to teach at-risk parents adaptive parenting approaches that promote healthy development in their young children. Parents were presented with instructional content in the form of videos and narrated text and were subsequently asked to videotape themselves practicing the newly learned techniques with their child. The video interactions were automatically uploaded to a secure web server where coaches viewed them remotely. Coaches then provided feedback and reinforcement via weekly scheduled phone calls. Results from initial pilot case studies indicate that parents were able to use, and learn from, this type of intervention delivery system and reported high satisfaction with the overall program (Feil et al., 2008). Parent Training and Self-Directed Distance Learning Programs This combination of technology and remote professional contact seems to be a promising avenue for alternative models of service delivery. However, devising and implementing such a program requires ample resources in order to support a system as comprehensive as this. It may be that some parent training curricula can still be effectively delivered in other, less resource intensive, formats, such as self-directed learning programs. Self-directed learning programs capitalize on individuals’ desire to learn and manage their own education (Knowles, 1975), and 6 have been successfully used to teach parents a variety of skills and behaviors focused on behavior management (Webster-Stratton, Hollinsworth, & Kolpacoff, 1989; Connell, Sanders & Markie-Dadds, 1997), adaptive parenting techniques (Lagges & Gordon, 1999; Bert, Farris, & Borkowski, 2008), and strategies for reducing child tantrums (Endo, Sloane, Hawkes, Mcloughlin, & Jenson, 1991). Self-directed learning programs have also been used to effectively teach adults with disabilities, including training parents with intellectual disabilities in basic parenting skills (Wade, Llewellyn, & Matthews, 2008). These types of learning programs are typically brief, relatively low-cost, can be completed in participants’ homes and have the potential to be translated into computerized formats relatively easily (Bert et al., 2008; Nefdt, Koegel, Singer & Gerber, 2009). A growing number of studies have begun to explore the use of computer technology to distribute self-directed learning programs (Hollon et al., 2002, Kacir & Gordon, 1999, Pacifici, Delaney, White, Cummings, & Nelson, 2005). Through a self-directed distance learning format, a greater number of people can gain access to standardized instruction in evidence based practices in a relatively straightforward and affordable way. For example, Kacir and Gordon (1999) adapted the parent training program, Parenting Adolescents Wisely, into a brief selfdirected interactive laserdisc program in order to instruct parents in rural Appalachia in adaptive parenting strategies. The laserdisc program presented videos of various child behavior problems, asked parents how they would react, and delivered direct feedback based on the parent response. The computerized Parenting Adolescents Wisely program has been shown to effectively reduce child problem behaviors and improve parent knowledge and use of adaptive parenting skills (Kacir & Gordon, 1999). Similarly, MacKenzie and Hilgedick (1999) created the ComputerAssisted Parenting Program (CAPP) aimed at teaching parents effective behavior management 7 techniques. This program utilized a number of media formats including computer simulation, written instruction, and multiple-choice questions with immediate corrective feedback to develop and test parent knowledge about strategies for managing disruptive behaviors. Participation in this study resulted in increases in parents’ use of behavioral management strategies and parental involvement (MacKenzie & Hilgedick, 1999). Self-directed distance learning programs have been shown to be effective for distributing parent training curricula focused on behavior management and general adaptive parenting. It is less clear whether this service delivery format is effective for parent training programs focused on techniques for teaching new skills to other high-needs populations, such as children with ASD. Because children with ASD experience profound deficits in certain areas, such as language, nonverbal communication and cognitive abilities, many interventions for these children often revolve around developing specific skills or sets of skills (National Research Council, 2001). The type and complexity of skill-building intervention techniques, especially those aimed at increasing the frequency of certain behaviors, are likely to differ from the techniques involved in interventions aimed at reducing the presence of unwanted behaviors. Moreover, there are unique barriers involved in working with young children with ASD, especially with regard to the difficulties they experience communicating with others (National Research Council, 2001). These unique circumstances may make it challenging for parents of children with ASD to learn intervention techniques without active support from a professional. As such, it is essential to examine the effectiveness of computer and internet-delivered selfdirected training programs for teaching parents of children with ASD skill-building intervention techniques. 8 Self-Directed Distance Learning Program for Parents of Children with ASD To date, only one published study has empirically evaluated the effectiveness of computerized self-directed training programs to teach skill-building intervention techniques to parents of children with autism. Nefdt and colleagues (2009) created a self-directed learning program for parents of young children with autism which presented motivational techniques from pivotal response training (PRT) aimed at increasing children’s social communication, primarily their verbal language. The program consisted of 14 training modules presented via DVD with an accompanying paper-based parent manual. Within each module, information was presented via text and audio lecture and short video examples of each technique were provided. Parents completed short quizzes to check for comprehension at the conclusion of each module. Additionally, at the conclusion of the program, parents participated in an interactive learning task where they assessed others’ abilities to implement PRT techniques. Results indicated that parents were generally willing to complete such a program and those who completed it were able to implement PRT strategies with fidelity, provided more language opportunities for their children and displayed greater confidence when interacting with their children. Additionally, children of these parents showed an increase in their use of verbal language (Nefdt, et al., 2009). In this study, both parents and their young children with autism were able to learn and benefit from parental participation in a self-directed distance learning program without support and guidance from professional coaches. The work done by Nefdt and colleagues is the first study to suggest that techniques from an evidence-based skill-building intervention for young children with autism can be successfully adapted into a remotely-delivered parent training program. The study utilized a pre-recorded DVD to deliver instruction; alternatively, the use of an internet-based delivery system for parent 9 training has promising implications, especially with regards to keeping instruction systematic yet individualized. Moreover, use of the internet allows parents to have immediate remote interactions with the instructional content, coaches and other parents. The internet also allows parents’ use of the online program to be tracked, which can help ensure study standardization and offer insight into the way in which parents use these types of programs. By tracking participant use, important information about how parents approach such programs, such as completing the program all at once versus stretching it out over days and weeks, can be acquired. Purpose of this Research Children with autism exhibit delays in areas other than verbal language and as such, it is critical to assess the effectiveness of remote parent training programs for introducing inventions that focus on building additional skills in these young children. One realm in which children with autism exhibit significant deficits is in imitation, particularly in social contexts (Ingersoll, 2008). Social imitation appears to be related to a number of other, more advanced, socialcommunication skills, such as verbal language, play, and joint attention in young children with autism (see Ingersoll, 2008 for review). As such, interventions focused on teaching imitation skills, especially in a social context, seem to be promising for the development of more sophisticated social-cognitive and social-communication skills (Ingersoll & Schreibman, 2006; Ingersoll, 2007). Given the considerations described above, the goal of the current study was to evaluate the use, feasibility, and preliminary efficacy of an internet-based training program to teach an evidence-based imitation intervention. The current study was the first to evaluate an internet-based, self-directed distance learning program aimed at introducing skill-building intervention techniques. A new internetbased training program was developed to teach techniques from reciprocal imitation training 10 (RIT; Ingersoll & Schreibman, 2006), a naturalistic behavioral intervention aimed at improving spontaneous imitation in young children with autism. Previous research has indicated that both undergraduate therapists and parents can learn to effectively implement RIT when trained by expert therapists in a lab setting (Ingersoll, 2010; Ingersoll & Gergans, 2007; Ingersoll & Schreibman, 2006). Ingersoll and Gergans (2007) demonstrated that parents were able to increase their use of intervention strategies in a lab setting, as well as during play interactions with their children at home. Additionally, children in the study increased their imitation with objects during lab sessions and were able to generalize object imitation skills to their home environments. The manualized RIT curriculum, in conjunction with the success of parent training in RIT, especially in parents’ ability to generalize RIT skills to the home setting, makes this an appropriate parent training intervention to adapt to a computerized delivery model. It is important to note, however, that as part of the clinic based training, parents received one-on-one coaching and direct feedback from an expert trainer over the course of ten weeks. It is not yet clear if such intensive support and feedback are necessary for parents to learn RIT techniques. Given the unknown variables associated with the utilization of such innovative technology, the use, feasibility, and efficacy of the internet-based training program were first examined with a sample of undergraduate research assistants who were training as therapists for an intervention study. In a second study, these use, feasibility, and effectiveness of this program were evaluated with parents of young children with ASD. The goal of these two studies was to assess the degree to which therapists-in-training and parents could learn about and effectively implement RIT after engaging in an internet-based self-directed training program. Additional goals of these studies were to assess the impact of the training on child behavior, specifically 11 child imitation skills and supported joint attention, and to evaluate the acceptability of this training approach. 12 STUDY 1 Method Participants Participants for this study were six female undergraduate students at a large mid-western 1 university. These therapists-in-training were new research assistants in a laboratory specializing in the study of autism interventions. All of the therapists had previous experience working with children. Three had previous experience working with individuals with ASD; however, only one had experience with young children with ASD. None of the therapists had been formally trained in any autism intervention techniques, nor exposed to any video or live-demonstrations of RIT, prior to beginning this program. Five young children with ASD were recruited to interact with the therapists to examine their ability to implement the intervention with a child before and after training. Each child interacted with the same therapist during baseline and post-training. All children had been previously involved in studies in the research lab and met DSM-IV criteria for autism as well as the cutoff for autism spectrum disorder on the Autism Diagnostic Observation Schedule (Lord, Rutter, DiLavore, & Risi, 2002). Additionally, their parents completed the Developmental Profile-3 (DP-3; Alpern, 2007) to provide an estimate of their cognitive and communication age and the Social Responsiveness Scale (SRS; Constantino et al., 2003) to obtain a measure of autism severity (see Table 1). Settings and Materials 1 Because the undergraduate participants had not received prior training, they were called therapists-in-training. However, for clarity and easy of communication, they will be referred to as “therapists” throughout the remainder of the text. 13 Therapists completed the online training program on computers in their homes or in the research lab. All baseline and post-training sessions were conducted and recorded in a treatment room at the research lab. Five pairs of developmentally-appropriate toys from the research lab were provided for each session. Training Program Program Delivery Platform. The self-directed distance learning program was delivered via the password-protected secure web server, ANGEL Learning Management Suite (ANGEL LMS). ANGEL LMS is an online suite of teaching and learning management tools that are designed for efficient and easy management and distribution of educational information. Users of ANGEL LMS are able to create media-rich online courses and can share course content with other approved users. ANGEL LMS can support assessments and quizzes, PowerPoint presentations, PDFs, links to external websites, discussion rooms and live computer chats, music and images. Additionally, ANGEL LMS, in conjunction with StoreMedia, a secure passwordprotected server for publishing audio and video media to the web, is able support large video files. Therapists were assigned a username and password for access to the ANGEL course. Instructional Content. The distance learning program was developed to introduce individuals to RIT, an evidence-based intervention aimed at increasing imitation in young children with autism (Ingersoll, 2010; Ingersoll & Schreibman, 2006). RIT techniques presented included contingent imitation, linguistic mapping, modeling, prompting, and providing natural reinforcement for imitation (Table 2, adapted from Ingersoll & Gergans, 2007). Instructional content was presented in five short training modules. The first training module, Introduction to Reciprocal Imitation Training, presented an introduction to RIT and imitation, provided background information about naturalistic behavioral intervention and 14 offered the rationale for training in evidence-based techniques. The second training module, Setting up the Home for Success, introduced ways to limit environmental distractions and prepare for a successful RIT session. The three final training modules, which covered techniques specific to RIT, were Contingent Imitation, Linguistic Mapping, and Teaching Object Imitation. Information for each module provided the foundation for techniques presented in the subsequent modules. Instructional content was presented through a combination of PowerPoint slideshows and video clips of RIT techniques. Important text was presented on the screen, while concurrent audio lectures provided more thorough explanations of the concepts. During each slideshow, video clips demonstrating specific RIT techniques were viewed. At the end of the Contingent Imitation and Linguistic Mapping training modules, three to five longer video clips, combining the techniques discussed in the module, were presented. At the conclusion of the final module, Teaching Object Imitation, five, 5-minute videos of full RIT sessions were displayed. Additionally, a PDF of the training manual was available for participants to view and print, if desired. Instructional content was presented in the order described above because a familiarity with, and understanding of, contingent imitation and linguistic mapping is crucial for learning the steps involved with teaching object imitation and correctly implementing full RIT sessions. The first four modules took between 4 and 12 minutes to view, while the final module, Teaching Object Imitation, took 40 minutes to view. The extended length of the last module was due to the full session RIT video examples at the conclusion of the presentation. After viewing each training module, the therapists were asked to complete short online quizzes to assess their comprehension of the module. Additionally, after viewing the final three 15 training modules, the therapists were asked to engage in short interactive learning tasks. These interactive learning tasks required participants to judge videotaped adult-child interactions for accurate use of RIT techniques. After completing the quiz and rating the videos, participants were able to view the correct responses and were provided with the rationale for the answers. Experimental Design and Procedure A single-subject, multiple-baseline design was conducted across therapist-child dyads (Hersen & Barlow, 1976), with one child interacting with two therapists. Dyads were randomly assigned to baseline periods of 2, 3, 4, 5, 6 or 7 sessions. After baseline sessions, therapists completed the self-paced online program. Upon completion of the program, therapists were filmed implementing RIT. The amount of time between the final baseline session and first posttraining session was 19-40 days (M = 29 days). Baseline. During baseline sessions, therapists were asked to play with a child in the same way they usually play with children for 10 minutes. These sessions were video recorded. Training. During the training phase, therapists were asked to work through the online training program over the subsequent two weeks. They completed each of the training modules with the corresponding comprehension checks and interactive learning tasks. Post-Training. During post-training, therapists were filmed implementing three, 10minute RIT sessions. If the therapist was unable to achieve fidelity of implementation after the third post-training session, 30 minutes of live demonstration and instruction from an expert coach was provided. After the in-person coaching session, therapists were filmed for two additional sessions. Dependent Measures 16 Program Utilization. Frequency and duration of therapist utilization of the internet-based training program was tracked on ANGEL LMS. Therapist knowledge of RIT. Therapist knowledge of RIT and naturalistic behavioral intervention techniques was assessed with a brief online multiple choice exam and an interactive learning task. The interactive learning task required the therapists to view seven, 5-minute video clips of adult-child interactions and to rate the clips for fidelity of RIT techniques. Therapists completed the same knowledge quiz and interactive learning task before beginning the first training module and upon completion of the final training module. Fidelity of Implementation of RIT. To evaluate correct implementation of RIT, trained research assistants scored the videotaped therapist-child interactions for fidelity. Observers rated the participants from one (low) to five (high) on contingent imitation, linguistic mapping, and teaching object imitation (including modeled actions, prompting and praise) using the RIT fidelity form (Ingersoll & Lalonde, 2010; Appendix). An average rating of 4 or above (80%) was considered implementing the intervention with fidelity. Child Imitation. Previous research has demonstrated that RIT is an effective intervention for developing imitation skills in young children with ASD (Ingersoll, 2010; Ingersoll & Gergans, 2007; Ingersoll & Schreibman, 2006, Meyer & Ingersoll, 2011). However, the majority of research in this area has examined the impact of this intervention as implemented over several weeks or months. Less is known about the effects of RIT when implemented over a shorter period of time. To evaluate changes in child imitation performance during this brief intervention, the overall rate of imitation for each therapist-child interaction was recorded. Observers recorded the number of times a child imitated an action modeled by the therapist. Rate per minute of imitation was calculated. 17 Child Supported Joint Attention. RIT involves the use of techniques such as contingent imitation and linguistic mapping in order to increase child engagement and joint attention during play. Previous research has suggested that RIT leads to increases in joint attention skills (Ingersoll & Schreibman, 2006; Meyer & Ingersoll, 2011). As such, it was expected that the use of these strategies would be related to increases in supported joint attention in the child. To rate supported joint attention, trained research assistants used interval scoring to code the child’s behavior during the videotaped interaction. Every 10-seconds the observer coded the child’s behavior for the presence or absence of supported joint attention (e.g., moving attention from an object to the adult). See Table 2 for Behavioral Definitions. Inter-Observer Reliability Inter-observer reliability was obtained for 25% of the observational measures by trained research assistants blind to the therapists point in training. Pearson’s r was used to calculate reliability on measures using ordinal data. The correlation for fidelity of implementation was .94 and for child imitation rate was .98. Follow-up t-tests indicated no significant differences between raters for any of the measures. Cohen’s Kappa was used to calculate reliability for the measures using interval data. Reliability for supported joint attention was .75. Results Program Utilization All therapists enrolled in the study completed the internet-based program. The amount of time between initial access to the program and completion of the post-training assessments ranged from 5-36 days (M = 14 days). During this time, the therapists logged on to the program between 4-12 different times (M = 8). All of the therapists viewed the training modules in the correct order. Five of the therapists completed all components of the internet-based program, 18 while one therapist completed everything except the interactive learning tasks after training modules three and four. Therapist Knowledge of RIT Paired t-tests were conducted to test for significant differences between therapist scores on the RIT knowledge quiz and interactive learning tasks from pre- to post-training. Results indicated that the therapists did significantly better on the RIT knowledge quiz at post-training (M = 90.83%, SD = 5.85%) than they did at pre-training (M = 68.33%, SD = 14.38%), t(5) = 5.89, p < .05. There was an improvement in participant scores on the interactive learning task from pre- training (M = 80%, SD = 9.7%) to post-training (M = 88%, SD = 7.4%); however, this difference was not significant, t(5)= 2.39, n.s. Fidelity of Implementation of RIT All therapists improved their implementation of RIT techniques from baseline to posttraining (see Figure 1). Kim, Becky, and Erin were able to implement RIT techniques with fidelity after using the internet-based training program. The child initially paired with Natalie experienced significant disruptions to his routine and subsequent behavioral issues over the course of training, particularly during the post-training sessions (sessions 5-7). However, when Natalie was paired with a different child (sessions 8-10) she was able to achieve fidelity of implementation of RIT without additional support and coaching. Although Alex and Hannah improved their use of the intervention techniques with the use of the internet-based program, they did not initially achieve fidelity of implementation. However, after a 30-minute coaching session, they were able reach fidelity of implementation across two additional post-training sessions. Child Imitation 19 Across therapist-child dyads, all of the children displayed low levels of imitation during baseline sessions. After the therapists engaged in the internet-based training, each child’s rate of imitation increased with the therapist with whom they interacted. Jake and Dean, the children paired with Alex and Hannah, both increased their rate of imitation further after their therapist received additional coaching and achieved fidelity of implementation (see Figure 2). Child Supported Joint Attention During baseline, all children demonstrated low to moderate rates of supported joint attention. After the therapists completed the training program, Dean demonstrated increases in his rate of supported joint attention. However, changes in supported joint attention rates from baseline to post-training were not observed for any of the other children (see Figure 3). Summary Undergraduate therapists showed substantial increases their knowledge and implementation of RIT techniques from baseline to post-training sessions. Additionally, all the children increased their rates of imitation, although, with the exception of Dean, they did not improve their rates of supported joint attention. These results provide initial evidence for the efficacy of this training model. However, it is important to note that prior to the first baseline session, many of the therapists had never interacted with a child with ASD, especially in a treatment setting. Moreover, the therapists did not meet the participating children until baseline and thus were unfamiliar with the children’s idiosyncrasies. Given these circumstances, it is not surprising that baseline scores were low and that such substantial increases from baseline to posttraining were observed for many of the participants across dependent measures. It is important to evaluate whether these same improvements would be observed in interactions with intervention providers who had more experience and familiarity with the children. Given the evidence for 20 parents as effective intervention providers, one way to perform this evaluation is to examine the outcomes of this internet-based training program with parents of young children with ASD. STUDY 2 Method Participants Three young children with ASD and their mothers participated. The children were all diagnosed by a professional using DSM-IV criteria. At intake, parents completed the Social Communication Questionnaire (SCQ; Berument, Rutter, Lord, Pickles, & Bailey; Rutter, Bailey, & Lord, 2003) to obtain a measure of autism severity and the DP-3 (Alpern, 2007) to assess child developmental level. Additionally, parents completed the Parenting Stress Index-Short Form (PSI-SF; Abidin, 1995) to evaluate current parenting stress, with the thought that increased levels of stress may have an impact on learning (Robbins, Dunlap, & Plienis, 1991). Families were recruited via word of mouth through the research lab and flyers at local referral agencies and community organizations. Parents who had already participated in a formal naturalistic behavioral parent training program were excluded from participation in the current study. Jonathan was 5 years, 9 months at intake. He had a cognitive age of 16 months on the DP-3 Cognitive Scale and a communication age of 6 months on the DP-3 Communication Scale. Jonathan scored a 22 on the SCQ, which is above the cutoff for suggested autism. Jonathan lived with his mother, father, and younger brother who was suspected of having ASD. Jonathan’s mother, Jamie, had a graduate degree and worked in the mental health field. Jamie’s score on the PSI-SF was 106, suggesting that she was experiencing clinically elevated levels of stress at intake. She had previously received training in relationship development intervention (RDI; 21 Gutstein & Sheely, 2002), a developmental intervention for children with autism. She had not received training in any naturalistic behavioral intervention techniques prior to the current study. Rick was 7 years, 4 months at intake. He had a cognitive age of 47 months on the DP-3 Cognitive Scale and a communication age of 26 months on the DP-3 Communication Scale. Rick scored a 34 on the SCQ, which is above the cutoff for suggested autism. Rick lived with his mother and father. Rick and his family moved to the US when he was 3 years old. Although both the language of origin and English were spoken in the home, all of Rick’s education and intervention work was in English. Rick’s mother, Jill, had a graduate degree and was currently staying at home to raise Rick. Jill received a score of 117 on the PSI-SF, suggesting that she was experiencing clinically elevated levels of stress at intake. Two years prior to participation in the current study, Jill had participated in a three-month intensive parent training program in structured ABA techniques through an area autism center. She had never been trained in any naturalistic intervention techniques prior to the current study. Gary was 2 years, 2 months at intake. He had a cognitive age of 24 months on the DP-3 Cognitive Scale and a communication age of 16 months on the DP-3 Communication Scale. Gary scored an 8 on the SCQ. Although this score is below the SCQ cutoff for suspected ASD, the developers of this instrument suggest using interpretive caution when using it with children under four years of age. As such, Gary was also administered Module One of the ADOS (Lord et al., 2002), on which he was above the cutoff for an autism spectrum disorder. Gary lived with his mother and father. Gary’s mother, Tina, attended some college and worked as an executive assistant. Tina’s score on the PSI-SF was 64, suggesting that her level of stress was not clinically elevated at intake. Tina had not received any parent training prior to participation in the current study. 22 Settings and Materials Parents completed the online parent training program on their own computers. Additionally, all baseline and post-training sessions were completed in the participants’ homes and were videotaped by the investigator. The investigator brought pairs of toys for the parents to use during these sessions; however, the participants were also able to use their own toys if they wished. Training Program Study 2 utilized the same training program, program delivery platform, and instructional content as in Study 1. In addition to the procedures described above, parents were instructed to practice techniques presented in each training module with their child before moving on to the next module of the program. Experimental Design and Procedure A single-subject, multiple-baseline design was conducted across parent-child dyads (Hersen & Barlow, 1976). Dyads were randomly assigned to baseline periods of 3, 5, or 6 sessions. After the baseline sessions, participants completed the self-paced, online program. Upon completion of the program, parents were filmed implementing the intervention techniques with their child in their home. The amount of time between the final baseline session and first post-training session was 23-36 days (M = 30 days). Baseline. During baseline sessions, parents were asked to play with their child as they normally do for 10 minutes. Training. Study 2 utilized the same training procedures as described in Study 1. 23 Post-Training. During post-training, parents were filmed implementing three 10-minute RIT sessions in their homes. If the parent was unable to achieve fidelity of implementation after the third post-training session, 30 minutes of live demonstration and instruction from an expert coach was provided. After the in-person coaching session, parents were filmed implementing RIT for an additional 10-minute session. After finishing the online training program, parents completed a treatment acceptability questionnaire to assess program utility and parent satisfaction with the intervention. Dependent Measures Study 2 utilized the same dependent measures as Study 1 including program utilization, participant knowledge of RIT, fidelity of implementation of RIT, child imitation, and child supported joint attention. Treatment Acceptability. Parent participants were also asked to complete a modified version of the Behavioral Intervention Rating Scale (BIRS; Elliott & Trueting, 1991) at posttreatment to evaluate the feasibility, acceptability, and effectiveness of the service-delivery model and intervention program. The BIRS is a well-validated measure that asks individuals to endorse items that assess the acceptability of a treatment’s procedures as well as the treatment’s perceived effectiveness on a 6-point scale, ranging from one (strongly disagree) to three (neutral) to six (strongly agree). For the purposes of this study, the BIRS was modified to better reflect the goals of the current intervention (i.e., acquisition of imitation skills). Parents were also asked to rate three additional items that assessed the usability of the program using the same rating scale as the BIRS (See Table 3 for additional items). Lastly, parents were asked to indicate benefits and limitations of the intervention in an open-ended format. Inter-Observer Reliability 24 Reliability was calculated on 25% of the observational measures. Pearson’s r was used to calculate reliability on measures using scale data. The correlation for fidelity of implementation was 0.99 and for child imitation rate was also 0.99. Follow-up t-tests indicated no significant differences between raters for any of the measures. Cohen’s Kappa was used to calculate reliability for the measures using interval data. Reliability for supported joint attention was .66. Results Program Utilization The amount of time between initial access to the program and completion of the posttraining assessments ranged from 1- 46 days (M = 22 days). During this time, the parents logged on to the program between 2-13 times (M = 9). Each parent viewed the training modules in the correct order. Two of the parents completed all components of the internet-based program, while one parent completed all activities except the interactive learning tasks after training modules three and four. Parent Knowledge of RIT Parents’ average score on the RIT knowledge quiz increased from 70% (range = 60%-80%) at pre-training to 87% (range = 75%-95%) at post-training. Parents’ average score on the Interactive Learning Tasks also increased from 67% (range = 51%-82%) at pre-training to 78% (range = 71%-86%) at post-training. Given the small sample size, inferential statistics were not conducted. Fidelity of Implementation During baseline, all mothers occasionally used some RIT techniques, particularly contingent imitation and linguistic mapping. However, they were not using these techniques consistently, nor were they implementing the RIT imitation training procedure. All three mothers, however, 25 improved their implementation of RIT techniques after completing the internet-based training program (see Figure 4). Jamie and Jill achieved fidelity of implementation based solely on their use of the internet-based training program. Tina’s use of the techniques approached fidelity after engaging in the internet-based training; however, she required an additional live coaching and demonstration session before achieving full fidelity of implementation. Child Imitation All children showed a stable and low rate of imitation during baseline. After their parents completed the training program, Jonathan and Rick showed a substantial increase in their rate of imitation (see Figure 5), while Gary showed a small increase in imitation. Once Gary’s mother Tina received coaching in the imitation training procedure involved with RIT and achieved fidelity of implementation, his rate of imitation increased dramatically. Child Supported Joint Attention During baseline, Jonathan showed stable and low to moderate rates of supported joint attention. Rick demonstrated moderate to high levels of supported joint attention during baseline; however these rates were variable across sessions. With the exception of his final baseline session, Gary generally showed stable and low rates of supported joint attention throughout baseline. After their parents completed the training program, all three children generally maintained baseline rates of supported joint attention, suggesting that the training did not influence this behavior. Treatment Acceptability All three parents responded favorably on the modified BIRS (Table 3). Parents were also asked to answer open-ended questions about the benefits and limitations of the service delivery model and intervention. When asked about the benefits of the training program, parents 26 responded that the information presented was helpful, the techniques were easy to use, and that both the parents and children had fun during RIT sessions. In response to a question about the limitations of the program, parents stated that they wished there had been more video examples to help them generate ideas for modeling appropriate play actions for imitation. Additionally, one parent stated that her internet connection was slow and as such some of the videos took longer to load. Interestingly, although Jamie and Jill were able to achieve fidelity of implementation based only on the use of the internet-based program, both indicated that they would have liked additional coaching or feedback when learning to use the intervention techniques. 27 DISCUSSION Given the growing discrepancy between the need for intervention and available services for individuals with ASD, the development of alternative service delivery formats is critical. Thus, the goal of the current study was to evaluate the feasibility and preliminary efficacy of an internet-based training program for introducing evidence-based intervention techniques to individuals working with children with ASD, including parents. The current study found that individuals working with young children with ASD were able to utilize an internet-based training program to learn about and correctly implement reciprocal imitation training. Both undergraduate therapists and parents demonstrated gains in knowledge about RIT and naturalistic behavioral intervention techniques after engaging in the training program. Additionally, all adults demonstrated improvements in their use of RIT from baseline to posttraining, with six out of the nine participants achieving fidelity of implementation without any additional training opportunities. The other three participants were able to implement RIT with fidelity after receiving additional support in the form of a half-hour live coaching session. During this brief intervention training period, all children increased their rates of imitation during filmed interactions. However, changes in child supported joint attention were not observed. Finally, parents reported high satisfaction with the service-delivery format and intervention program. Together these findings lend support to the contention that remotely-delivered programs can be used to disseminate training to parents and providers working with children with ASD (Nefdt et al., 2009). The current study provides some evidence for the efficacy and acceptability of computerized self-directed learning programs for teaching evidence-based skill-building intervention techniques. However, results from the current study also suggest that self-directed training programs may not provide sufficient training for many individuals interested in learning 28 about and utilizing evidence-based intervention techniques. One third of the participants required additional live demonstration, feedback, and support before achieving fidelity of implementation. Moreover, parents who did not receive this coaching indicated that additional support and feedback would have been beneficial. This finding is consistent with previous literature suggesting that coaching, feedback, and “on-the-job” problem solving is particularly important for maximizing the effectiveness of training programs, especially those introducing evidencebased intervention techniques (e.g., Feil et al., 2008; Thomson, Martin, Arnal, Fazzio, & Yu, 2009; van Oorsouw, Embregts, Bosman, & Jahoda, 2009). One of the benefits of internet-based training formats is that the addition of remote coaching and feedback components can be easily integrated into the service-delivery system. Baharav and Reiser (2010) recently utilized streaming internet technology to provide live feedback and coaching to parents implementing in-home speech and language therapy. Results from the pilot study suggested that the remote coaching was both feasible and effective; they found that child gains achieved in traditional therapy settings could be maintained and improved when the parent received remote live feedback and supervision from an expert therapist (Baharav & Reiser, 2010). Although previous research on parent training in RIT involved 10 weeks of coaching sessions (Ingersoll & Gergans, 2007), participants in the current study were able to achieve fidelity of implementation after one 30-minute coaching session. Thus, there is evidence to suggest that minimal amounts of additional support would be sufficient to learn the correct use of these basic intervention techniques. Therefore, the utilization of an internet-based training program with additional coaching available would likely produce a time-savings effect relative to traditional clinic-based training models and other more coaching-intensive distance training programs (e.g., Feil, 2008). As such, future research should consider the feasibility, acceptability, 29 and effectiveness of internet-based training formats that deliver systematic instruction with a minimal, yet sufficient, amount of remote feedback from an expert coach. Given the brevity of this training program, significant changes in child skill, such as imitation, were not necessarily expected. Nonetheless, across both studies, children increased their rates of imitation. This finding suggests that by utilizing the imitation training procedures taught in the internet-based program, adults were able to elicit more imitative behaviors from the children. As such, results from the current study lend support to the growing body of literature indicating that RIT is an effective intervention for increasing imitation skills in children with ASD (Ingersoll, 2010; Ingersoll & Gergans, 2007; Ingersoll & Schreibman, 2006, Meyer & Ingersoll, 2011). Yet, the impact of this internet-based training program on the long-term development of child imitation skills, particularly during interactions outside of RIT, is unknown. Previous research has demonstrated that parents’ use of RIT over a 10 week period of time led to increases in children’s spontaneous imitation, and that these skills generalized to different interaction settings (Ingersoll & Gergans, 2007). Thus, there is evidence to suggest that the use of the internet-based training program, and subsequent prolonged implementation of RIT, would lead to generalizable gains in imitation skills. Future research should consider exploring these longer-term outcomes. Previous research has indicated that the use of RIT techniques, particularly contingent imitation, is effective for increasing coordinated joint attention (Ingersoll & Schreibman, 2006; Lewy & Dawson, 1992). In the current studies, increases in supported joint attention were not observed, despite the fact that the adults increased their use of the RIT techniques. In previous studies assessing the relationship between RIT and joint attention, adults were instructed to act in a contrived way (e.g., model an action for imitation once per minute, but do not make any other 30 social initiations) during baseline or comparison sessions in order to standardize these interactions. During the current studies, however, adults were instructed to interact with the child as usual; it is possible that the adult participants, particularly parents, were already using other effective strategies to elicit supported joint attention from the children in usual play interactions. Thus, the lack of observed changes in supported joint attention may have been a result of the ways in which the adults were engaging with the children prior to use of the internet-based training program. It is also possible that improvements in supported joint attention are only evident after implementing RIT for longer periods of time. In support of this possibility, Meyer and Ingersoll (2011) noted that the initial use of RIT techniques, especially contingent imitation, can be overwhelming and frustrating for some children with ASD, leading to initial decreases in supported joint attention followed by increases over time. Finally, although the adults in the current studies achieved fidelity of implementation, it is possible that additional coaching may have been necessary in order to improve other aspects of child behavior including supported joint attention. Additional research on the longer-term use of RIT would be beneficial for elucidating the complexities of the relationship between the use of RIT techniques and changes in supported joint attention. Importantly, parent participants indicated that this service-delivery model was both useable and acceptable. Additionally, they indicated that the RIT intervention techniques were effective for teaching their children imitation and other social-communication skills. Although the parents in the current study demonstrated universally positive responses to this program, it is likely that differences in participant variables such as current life stressors, demographics, access to technology, and experience with technology would impact how an individual perceives the effectiveness and social validity of the program. Previous research has demonstrated an inverse 31 relationship between parental stress and the amount of progress made by children in a parent training program (Robbins, Dunlap, & Plienis, 1991). Yet in the current study all parents, including those who reported clinically significant levels of stress at intake, were able to complete the program, utilize the techniques correctly, and elicit changes in behavior from their children. It is important to note that parental stress was not assessed as an outcome variable in the current study; future research should consider monitoring parental stress during the use of such a training program to clarify the relationship between stress levels and program utilization, perceived acceptability, and effectiveness. Interestingly, one family initially enrolled in the current study had to withdrawal prior to completion because of the birth of a new child. It is clear that the current family situation made the allocation of time to the parent training program challenging; however, the family also indicated that they were not particularly familiar or adept with using technology. As such, it is possible that inexperience and lack of efficacy with the service delivery model further impeded the family’s ability to continue their participation. Given that the ultimate goal of this research is to increase access to evidence-based intervention, it will be essential for future research to examine the ways these variables influence the use, feasibility, acceptability, and effectiveness of such a service-delivery model. Limitations There are several limitations to the current study. First, as discussed above, the duration of the study was relatively brief. It is unknown whether the observed changes in adult and child behavior would be maintained over time. In particular, it is unclear whether parents would maintain such high fidelity when utilizing RIT in the home, particularly without any opportunities for feedback and problem-solving. Additionally, in order for individuals to participate in the current study, continual access to a computer and the internet was necessary. 32 As such, it is possible that the use of such a program may not be as feasible or acceptable for individuals with limited access to these resources. Future research should explore the effectiveness of this internet-based training program when utilized in the public domain in locations such as libraries, community agencies, schools, and hospitals. Because the program was standardized such that all participants viewed the same training modules and same video examples, individualization of the program to meet a given family’s specific needs was not possible. In particular, much of the lecture and video examples provided in the training program were focused on using these techniques with children who had less advanced social-communication skills. As such, it would be important for future program development and research to consider the individual needs of families by adding materials appropriate for a variety of developmental levels and providing the opportunity to receive individualized feedback and support from an expert trainer. Finally, the procedures involved in RIT are relatively simple and teach about prompting for only one type of skill. It is unknown whether a more complex intervention program, targeting the development of multiple skill domains, would be able to be effectively taught via a self-directed, internet-based training program. Summary This study provides initial evidence for the efficacy of a self-directed, internet-based distance learning program to disseminate training in evidence-based skill building intervention techniques for young children with ASD. Such an approach has the potential to significantly increase access to evidence-based intervention services for many individuals with ASD at minimal cost. Nonetheless, a more supportive and interactive training program, providing additional feedback and coaching, may be particularly beneficial for some consumers. 33 Table 1 Child Participant Characteristics Child Study 1 Jake Tim Andy Zak Dean Study 2 Jonathan Rick Gary 1 Chronological Age (months) Cognitive Age (months) Communication 1 Age (months) 66 35 66 40 74 29 29 29 34 41 20 18 26 22 46 69 88 26 16 47 24 Autism Severity 2 3 SRS SCQ 6 26 16 1 Developmental Profile-3 2 ≥60 consistent with ASD diagnosis; ≥76 consistent with autism diagnosis 3 ≥15 consistent with ASD diagnosis 34 92 56 81 79 60 22 34 8 Table 2 Behavioral Definitions RIT Components Contingent Imitation Following the child’s lead and imitating the child’s actions with toys, as well as imitating the child’s gestures/body movements and vocalizations. Linguistic Mapping The use of simple, repetitive language around the child’s focus of attention to describe objects and action. Imitation Training The correct use of all three imitation training strategies within a single trial. Modeling Actions Modeling an action and descriptive verbal marker with a toy related to the child’s play. Prompting Using physical guidance, a verbal command, or gesture to encourage the child to imitate the modeled action if the child does not spontaneously imitate after the third model. Reinforcement Providing the child with praise and continued access to the toys after both spontaneous and prompted imitation. Child Behaviors Imitation The child imitates the adult’s model of an action with a toy or a gesture within 10-seconds of the model. The imitation may be spontaneous or completed with the assistance of a verbal command, gestural prompt, or physical prompt. Supported Joint Attention The adult and child are actively involved in the same object, are in physical proximity to one another, and the child’s behavior is being affected by the adult’s presence, actions, or verbalizations. The child’s behavior must show evidence of joint awareness at least once every 10-seconds. 35 Table 3 Average Treatment Acceptability Ratings by BIRS scale Scales Mean (Range) Program Acceptability (12 items) 6.00 (6.00-6.00) Program Effectiveness (8 items) 5.38 (4.00-6.00) Program Usability (3 items, see below) 6.00 (6.00-6.00) The online format of the program was appropriate for learning the intervention strategies. The amount of training and support received was sufficient for me to learn the intervention strategies. The parent training materials were easy to understand. 1 = Strongly Disagree, 3 = Neutral, 6 = Strongly Agree 36 Figure 1. Therapist-Child Interaction: Fidelity of Implementation Baseline Post-Training Post-Coaching 5 4 3 2 1 Alex & Jake 1 2 3 4 5 6 7 5 4 3 2 1 Kim & Tim Fidelity Rating 1 2 3 4 5 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 5 4 3 2 1 10 Natalie & Andy* Becky & Zak 1 2 3 4 5 6 7 8 5 4 3 2 1 Erin & Tim 1 2 3 4 5 6 7 8 9 Hannah & Dean 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 11 Session *Sessions 5-7 were conducted with Andy, sessions 8-10 were conducted with Zak 37 12 Figure 2. Therapist-Child Interaction: Child Imitation Rates Baseline Post-Training Post-Coaching 0.5 Alex & Jake 0 1 2 3 4 5 6 7 0.5 Kim & Tim 0 1 2 3 4 5 6 Rate per minute 0.5 Natalie & Andy 0 1 2 3 4 5 6 *Video from session 4 was damaged and could not be coded for imitation rates. Video from session 5 was damaged half-way through. Data until 4 minutes 15 seconds is reported for session 5. 7 0.5 Becky & Zak* 0 1 2 3 4 5 6 7 8 0.5 Erin & Tim 0 1 2 3 4 5 6 7 8 9 0.5 Hannah & Dean 0 1 2 3 4 5 6 7 8 Session 38 9 10 11 12 Figure 3. Therapist-Child Interaction: Child Supported Joint Attention Baseline Post-Training Post-Coaching 0.5 Alex & Jake 0 1 2 3 4 5 6 7 0.5 Kim & Tim 0 1 2 3 4 5 6 Percent of intervals Natalie & Andy 0.5 0 1 2 3 4 5 6 7 Becky & Zak* 0.5 0 1 2 3 4 5 6 7 8 *Video from session 4 was damaged and could not be coded for supported joint attention. Video from session 5 was damaged half-way through. Data until 4 minutes 15 seconds is reported for session 5. 0.5 Erin & Tim 0 1 2 3 4 5 6 7 8 9 Hannah & Dean 0.8 0.6 0.4 0.2 0 1 2 3 4 5 6 7 Session 39 8 9 10 11 12 Figure 4. Parent-Child Interaction: Parent Fidelity of Implementation Post-Training Baseline Post-Coaching 5 4 3 2 Jamie & Jonathan 1 Fidelity Rating 1 2 3 4 5 5 4 3 2 Jill & Rick 1 1 2 3 4 5 6 7 5 4 3 2 Tina & Gary 1 1 2 3 4 5 6 Session 40 7 8 9 Figure 5. Parent-Child Interaction: Child Imitation Rates Baseline Post-Coaching Post-Training 1 0.8 0.6 0.4 0.2 Jamie and Jonathan 0 1 2 3 4 5 Rate per minute 1 0.8 0.6 0.4 0.2 Jill and Rick 0 1 2 3 4 5 6 7 1 0.8 0.6 0.4 0.2 Tina and Gary 0 1 2 3 4 5 6 Session 41 7 8 9 Figure 6. Parent-Child Interaction: Child Supported Joint Attention Baseline Post-Training Post-Coaching 1 0.8 0.6 0.4 0.2 Jamie & Jonathan 0 1 2 3 4 5 Percent of intervals 1 0.8 0.6 0.4 0.2 Jill & Rick 0 1 2 3 4 5 6 7 1 0.8 0.6 0.4 Tina & Gary 0.2 0 1 2 3 4 5 6 Session 42 7 8 9 REFERENCES 43 REFERENCES Abidin, R. R. (1995). Parenting Stress Index. Lutz, FL: Psychological Assessment Resources. Alpern, G.D. (2007). Developmental Profile 3. Los Angeles, CAL Western Psychological Services. Baharav, E., & Reiser, C. (2010). Using telepractice in parent training in early autism. Telemedicine and E-Health, 16, 727-731. Benjamin, S.E., Tate, D.F., Bandgiwala, S.I., Neelon, B.H., Ammerman, A.S., Dodds, J.M., Ward, D.S. (2008). Preparing child care health consultants to address childhood overweight: A randomized controlled trail comparing web to in-person training. Maternal and Child Health Journal, 12, 662-669. Bert, S.C., Farris, J., & Borkowski, J.G. (2008). Parent training: Implementation strategies for adventures in parenting. Journal of Primary Prevention, 29, 243-261. Berument, S., Rutter, M., Lord, C., Pickles, A., & Bailey, A. (1999). Autism screening questionnaire: Diagnostic validity. The British Journal of Psychiatry, 175, 444-451. Centers for Disease Control and Prevention. (2007). CDC Releases new data on autism spectrum disorders (ASDs) from multiple communities in the United States. Retrieved November 29, 2009, from http://www.cdc.gov/media/pressrel/2007/r070208.htm?s_cid=mediarel_r070208 Charlop, M.H., & Trasowech, J.E. (1991). Increasing autistic children’s daily spontaneous speech. Journal of Applied Behavior Analysis, 24, 747-761. Connell, S., Sanders, M.R., & Markie-Dadds, C. (1997). Self-directed behavioral family intervention for parents of oppositional children in rural and remote areas. Behavior Modification, 21, 379-408. Constantino, J.N., Davis, S.A., Todd, R.D., Schindler, M.K., Gross, M.M., Brophy, S.L., … Reich, W. (2003). Validation of a brief quantitative measure of autistic traits: Comparison of the social responsiveness scale with the autism diagnostic interviewrevised. Journal of Autism and Developmental Disorders, 33, 427-433. Drew, A., Baird, G., Baron-Cohen, S., Cox, A., Slonims, V., Wheelwright, S., … Charman, T. (2002). A pilot randomised control of a parent training intervention for pre-school children with autism: Preliminary findings and methodological challenges. European Child & Adolescent Psychiatry, 11, 266-272. Elliott, S.N., & Treuting, M.V. (1991). The Behavior Intervention Rating Scale: Development and validation of a pretreatment acceptability and effectiveness measure. Journal of School Psychology, 29, 43-51. 44 Endo, G.T., Sloane, H.N., Hawkes, T.W., Mcloughlin,C., & Jenson, W.R. (1991). Reducing child tantrums though self-instructional parent training materials. School Psychology International, 12, 95-109. Feil, E.G., Baggett, K.M., Davis, B., Sheeber, L., Landry, S., Carta, J.J., & Buzhardt, J (2008). Expanding the reach of preventive interventions: Development of an internet-based training for parents of infants. Child Maltreatment, 13, 334-346. Granpeesheh, D., Tarbox, J., Dixon, D. R., Peters, C. A., Thompson, K., & Kenzer, A. (2010). Evaluation of an eLearning tool for training behavioral therapists in academic knowledge of applied behavior analysis. Research in Autism Spectrum Disorders, 4, 11-17. Hersen, M, & Barlow, D. H. (1976). Single case experimental designs: Strategies for studying behavior change. New York: Pergamon. Hollon, S.D., Muñon, R.F., Barlow, D.H., Beardslee, W.R., Bell, C.C, Bernal, G., … Sommers, D.(2002). Psychosocial intervention development for the prevention and treatment of depression: Promoting innovation and increasing access. Biological Psychiatry, 52, 610630. Ingersoll, B. (2007). Teaching imitation to children with autism: A focus on social reciprocity. The Journal of Speech-Language Pathology and Applied Behavior Analysis, 2, 269-277. Ingersoll, B. (2008). The social role of imitation in autism: Implications for the treatment of imitation deficits. Infants & Young Children, 21, 107-119. Ingersoll, B. (2010). Brief Report: Pilot randomized controlled trial of reciprocal imitation training for teaching elicited and spontaneous imitation to children with autism. Journal of Autism and Developmental Disorders, 40, 1154-1160. Ingersoll, B., & Gergans, S. (2007). The effect of a parent-implemented imitation intervention on spontaneous imitation skills in young children with autism. Research in Developmental Disabilities, 28, 163-175. Ingersoll, B., & LaLonde, K. (2010). The impact of object and gesture imitation training on language use in children with autism. Journal of Speech, Language, and Hearing Research. 53, 1040-1051. Ingersoll, B., & Schreibman, L. (2006). Teaching reciprocal imitation skills to young children with autism using a naturalistic behavioral approach: Effects on language, pretend play, and joint attention. Journal of Autism and Developmental Disorders, 36, 487-505. Kacir, C.D., & Gordon, D.A. (1999). Parenting adolescents wisely: The effectiveness of an interactive videodisk parent training program in appalachia. Child & Family Behavior Therapy, 21, 1-22. 45 Kazdin, A. E. (2004). Evidence-based treatments: Challenges and priorities for practice and research. Child and Adolescent Psychiatric Clinics of North America, 13, 923-940. Knowles, M.S. (1975). Self-directed learning: A guide for learners and teachers. New York: The Adult Education Co. Koegel, R.L., Bimbela, A., & Schreibman, L. (1996). Collateral effects of parent training on family interactions. Journal of Autism and Developmental Disabilities, 26, 347-359. Koegel, R.L., Schreibman, L., Britten, K.R., Burke, J.C., & O’Neill, R.E. (1982). A comparison of parent training to direct child treatment. In R.L. Koegel, A. Rincover, & A. L. Egel (Eds.), Educating and Understanding Autistic Children. San Diego, CA: College-Hill Press. Lord, C., Rutter, M., DiLavore, P.C., & Risi, S. (2002). Autism Diagnostic Observation Schedule. Los Angeles, CA: Western Psychological Services. Lagges, A.M., & Gordon, D.A. (1999). Use of an interactive laserdisc parent training program with teenage parents. Child & Family Behavior Therapy, 21, 19-37. Lewy, A.L., & Dawson, G. (1992). Social stimulation and joint attention in young autistic children. Journal of Abnormal Child Psychology, 20, 555-566. MacKenzie, E.P., & Hilgedick, J.M. (1999). The computer-assisted parenting program (CAPP): The use of a computerized behavioral parent training program as an educational tool. Child & Family Behavior Therapy, 21, 23-42. Mahoney, G., & Perales, F. (2003) Using relationship-focused intervention to enhance the socialemotional functioning of young children with autism spectrum disorders. Topics in Early Childhood Special Education, 23, 77-89. Mandel, U., Bigelow, K.M., & Lutzker, J.R. (1998). Using video to reduce home safety hazards with parents reported for child abuse and neglect. Journal of Family Violence, 13, 147162. Meyer, K., & Ingersoll, B. (2011). Evaluation of a Sibling-Mediated Imitation Intervention for Young Children with Autism. Manuscript submitted for publication. Moes, D. R., & Frea, W. D. (2002). Contextualized behavioral support in early intervention for children with autism and their families. Journal of Autism and Developmental Disorders, 32, 519–533. National Research Council. (2001). Educating children with autism. Committee on Educational Interventions for Children with Autism. Catherine Lord and James P. McGee, eds. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. 46 Nefdt, N., Koegel, R., Singer, G., & Gerber, M. (2010). The use of a self-directed learning program to provide introductory training in pivotal response treatment to parents of children with autism. Journal of Positive Behavior Interventions, 12, 23-33. Pacifici, C., Delaney, R., White, L., Cummings, K., & Neloson, C. (2005). Foster parent college: Interactive multimedia training for foster parents. Social Work Research, 29, 243-251. Robbins, F. R., Dunlap, G., & Plienis, A.J. (1991). Family characteristics, family training, and the progress of young children with autism. Journal of Early Intervention, 15, 173-184. Rogers, S. J., Hayden, D., Hepburn, S., Charlifue-Smith, R., Hall, T., & Hayes, A. (2006). Teaching young children with autism useful speech: A pilot study of the Denver model and PROMPT intervention. Journal of Autism and Developmental Disorders, 36, 1007– 1024. Rutter, M., Bailey, A., & Lord,C. (2003). SCQ: The Social Communication Questionnaire. Manual. Western Psychological Services: Los Angeles, CA. Sperry, L.A., Whaley, K.T., Shaw, E., Brame, K. (1999). Services for young children with autism spectrum disorder: Voices of parents and providers. Infants & Young Children, 11, 17-33. Stahmer, A.C. (1995). Teaching symbolic play skills to children with autism using pivotal response training. Journal of Autism and Developmental Disorders, 25, 123–41. Stahmer, A.C., & Gist, K. (2001). The effects of an accelerated parent education program on technique mastery and child outcome. Journal of Positive Behavior Interventions, 3, 7582. Symon, J.B. (2001). Parent education for autism: Issues in providing services at a distance. Journal of Positive Behavior Interventions, 3, 160-174. Symon, J.B. (2005). Expanding interventions for children with autism: Parents as trainers. Journal of Positive Behavior Interventions, 7, 159-173. Taylor, T.K., Webster-Stratton, C., Feil, E.G., Broadbent, B., Widdop, C.S., & Severson, H.H. (2008). Computer-based intervention with coaching: An example using the incredible years program. Cognitive Behaviour Therapy, 37, 233-246. Thomson, K., Martin, G. L., Arnal, L., Fazzio, D., Yu, C.T. (2009). Instructing individuals to deliver discrete-trials teaching to children with autism spectrum disorders: A review. Research in Autism Spectrum Disorders, 3, 590-606. United States Census Bureau. (2009). Statistical Abstract of the United States: 2009. 47 Wade, C., Llewellyn, G., Matthews, J. (2008). Review of parent training interventions for parents with intellectual disability. Journal of Applied Research in Intellectual Disabilities, 21, 351-366. Webster-Stratton, C., Hollinsworth, T., & Kolpacoff, M. (1989). The long-term effectiveness and clinical significance of three cost-effective training programs for families with conductproblem children. Journal of Consulting and Clinical Psychology, 57, 550-553. Weingardt, K.R. (2004). The role of instructional design and technology in the dissemination of empirically supported, manual-based therapies. Clinical Psychology: Science and Practice, 11, 313-331. Weingardt, K.R., Cucciare, M.A., Bellotti, C., & Lai, W.P. (2009). A randomized trail comparing two models of web-based training in cognitive-behavioral therapies for substance abuse counselors. Journal of Substance Abuse Treatment, 37, 219-227. Van Oorsouw, W.M.W.J., Embregts, P.J.C.M., Bosman, A.M.T., & Jahoda, A. (2009). Training staff serving clients with intellectual disabilities: A meta-analysis of aspects determining effectiveness. Research in Developmental Disabilities, 30, 503-511. Vismara, L.A., Young, G.S., Stahmer, A.C., Griffith, E.M., & Rogers, S.J. (2009). Dissemination of evidence-based practice: Can we train therapists from a distance? Journal of Autism and Developmental Disorders, 39, 1636-1651. 48