VIDEO MODELING AND MATRIX TRAINING: EFFECTS ON ACQUISITION AND GENERALIZATION OF SOCIAL SKILLS BY ADOLESCENTS WITH AUTISM SPECTRUM DISORDER By Tiffany Ann Stauch A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Special Education—Doctor of Philosophy 2018 ABSTRACT VIDEO MODELING AND MATRIX TRAINING: EFFECTS ON ACQUISITION AND GENERALIZATION OF SOCIAL SKILLS BY ADOLESCENTS WITH AUTISM SPECTRUM DISORDER By Tiffany Ann Stauch The vast majority of research examining interventions for individuals with autism spectrum disorder (ASD) has focused almost exclusively on early intervention services and supports, with limited efforts going to support adolescents and adults with ASD (Wong et al., 2015). This is especially concerning as it has been estimated that approximately 50,000 individuals with ASD turn 18 years old each year in the United States alone (Shattuck et al., 2012b). Employment opportunities are often limited for those with disabilities, but individuals with ASD tend to have greater difficulty finding and sustaining employment than those with other disabilities, possibly due to a lack of appropriate social skills. The literature on vocational skills and work related social skills for adolescents and adults with ASD is limited and many of the published studies are considered to be of poor quality, making it difficult to make inferences or draw conclusions regarding the most effective vocational treatment approaches for adolescents and adults with ASD. The dissertation is comprised of three separate studies that examined the current issues surrounding the vocational skills literature for adolescents and adults with ASD. The first study reviewed all single case experimental design (SCED) studies from 1980-2017 that explicitly taught vocational or vocational social skills to adolescents or adults with ASD. Each study was evaluated by applying the Single-Case Design Technical Documentation from the What Works Clearinghouse (WWC; Kratochwill et al., 2010; 2013) to ensure it sufficiently met the methodical rigor for the selected design. A total of twenty-four studies met inclusion criteria and were categorized as video modeling, audio cueing, textual cueing, and additional intervention strategies. Half of the experiments met standards or met standards with reservations and only half of those experiments demonstrated strong or moderate evidence of effect. None of the interventions could be categorized as an EBP. The second study evaluated the effectiveness of video modeling to teach two adolescents with ASD two or three vocational and social skills in a simulated job setting. Applied Behavior Analysis (ABA) technicians implemented a video modeling intervention 1 to 3 times per week for approximately 2 hours each session. Videos for social skills were embedded into videos for vocational tasks. A multiple probe design across behaviors was used to evaluate the effectiveness of the intervention. Results indicate video modeling was effective for teaching vocational skills, although a functional relation was not demonstrated for one participant due to lack of sufficient opportunity for replication. Mixed results were observed for social skills. The third study utilized a multiple probe design across behaviors to evaluate the effectiveness of video modeling on the acquisition of vocational and social skills among 3 adolescents with ASD in a public high school. A school para-professional was trained to facilitate the intervention to teach participants 4 vocational tasks and 4 social skills necessary to operate a concession stand for employees of the school. Matrix training was used to facilitate generalization of the social skills across vocational tasks. Results indicate video modeling was effective for teaching vocational skills and participants were able to use most social skills across vocational tasks. Participants generalized vocational skills to a new setting with actual customers, but had difficulty performing social skills in the generalization setting. ACKNOWLEDGEMENTS First and foremost, I would like to thank my dissertation chair, Dr. Joshua Plavnick, for his continued support and guidance over the past six years. You have truly helped me to learn and grow as a researcher and behavior analyst. I am extremely appreciative of your hard work, critical feedback, and willingness to go above and beyond to make sure I felt supported along the way. I must also express my appreciation for the members of my dissertation committee, Dr. Bouck, Dr. Fisher, and Dr. Sung. Your guidance and input have made this research project even stronger. I would also like to thank my family. Although none of you would have likely picked me to be the first in this family to get a PhD, I am humbled by your genuine interest and support throughout this seemingly endless journey. Thank you to my parents, who gave me wings and taught me to soar. It is hard to put into words what your endless support, adoration, and encouragement has meant to me. Mom, thank you for guiding my career path and teaching me to go big or go home. You have been, and always will be, my hero. And Dad, if it weren’t for your thoughtfulness, wittiness, and easy-going personality, I wouldn’t be who I am today. Thank you both for teaching me that Kaids aren't quitters. To my husband, Matthew—thank you for your support, encouragement, and willingness to help out when I needed it the most. You are always there for me, no matter what. To my beautiful daughter, Emmaline Kaid—it was an honor to carry you as I carried out this intervention project. Thank you for being such a happy and easy-going baby, which made writing this dissertation just the slightest bit easier. I would also like to thank those who helped make this project happen. Thank you to Centria Healthcare for allowing me to use their office space to conduct part of this study and for iv being understanding even when a few holes were put in the walls. Thank you to Mr. Tom, who made “The Stable” a possibility through his willingness to learn and dedication to the students. Last, but certainly not least, a special thank you to my best friend and biggest cheerleader, Annie Gallagher. You have been instrumental to my success at Michigan State and I could not have completed this project without you by my side. You are truly an amazing friend and colleague and I am forever grateful for all you have done for me over the years. v TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES KEY TO ABBREVIATIONS CHAPTER 1 Introduction Employment and Adults with ASD Transition Planning Identification of Evidence Based Practices Teaching Vocational Skills with Video Modeling Promoting Generalization with Matrix Training Outline of Current Dissertation and Research Questions CHAPTER 2 Systematic Literature Review Vocational and Social Challenges for Adolescents with ASD Previous Vocational Skills Literature Reviews Previous applications of the WWC Standards to the Literature Purpose of Systematic Literature Review Method Search strategies Application of design standards Application of evidence standards Evidence based practice Coding of articles Categorization of experiments Coder agreement Video modeling experiments with strong evidence Video modeling experiments with moderate evidence Audio cueing experiments with strong evidence Textual cueing experiments with strong evidence Video modeling interventions Categorization of interventions Work related social skills Importance of teaching in employment settings and schools Design standards Evidence standards Study details Evidence based practices Results Discussion ix x xi 1 1 3 4 6 7 8 12 12 13 14 15 16 16 17 18 20 20 21 22 23 23 29 30 30 31 31 32 37 37 38 39 40 41 vi Limitations Conclusion CHAPTER 3 Pilot Study Procedures Challenges for Adolescents with ASD State of Vocational Skills Literature Video modeling and Vocational Skills Work Related Social Skills Purpose of Pilot Study Method Participants Facilitators Setting and materials Measurement of dependent variables Inter-observer agreement Experimental design Vocational and social skill pairing 1 Vocational and social skill pairing 2 Vocational and social skill pairing 3 Acquisition of vocational skills Acquisition of social skills Extension of the vocational skills literature Limitations Conclusion Creation of video clips Probe sessions Video modeling Video fading Procedural integrity Results Discussion CHAPTER 4 Research Study Transition Planning for Adolescents with ASD Work Related Social Skills Video Modeling Generalization Purpose of Study Method Participants Facilitator Setting Materials Measurement of dependent variables Inter-observer agreement (IOA) vii 42 44 45 45 46 46 48 50 50 50 52 53 54 56 56 57 58 59 60 60 61 63 65 66 66 70 71 72 73 74 76 77 77 78 79 81 82 82 82 84 84 85 85 89 Procedures Experimental design Creation of video clips Probe sessions Generalization probes Video modeling Video fading Maintenance probes Procedural integrity Social validity Matrix probes for social skills Acquisition of social skills Efficiency of instruction Limitations Conclusion Results Discussion 89 91 91 91 92 96 96 98 98 99 99 100 106 107 108 110 112 113 113 115 117 119 122 125 126 135 141 144 147 149 153 157 159 160 CHAPTER 5 Discussion Current State of the Vocational Skills Literature Evidence Based Practices for Vocational Skills Training Staff to use Evidence Based Practices Methods to Teach Vocational Skills and Work Related Social Skills Work Related Social Skills APPENDICES APPENDIX A WWC Design Standards Coding Form APPENDIX B WWC Evidence Standards Coding Form APPENDIX C WWC Study Characteristics Coding Form APPENDIX D Parent Informed Consent Form APPENDIX E Research Participant Assent Form APPENDIX F Baseline Procedural Integrity Checklist APPENDIX G Video Modeling Procedural Integrity Checklist APPENDIX H Data Collection Sheet APPENDIX I Social Validity Survey REFERENCES viii LIST OF TABLES Table 1. Results of Design Quality Analysis and Evidence Rating 26 Table 2. Summary of Studies that Met Standards or Met Standards with Reservations 33 55 59 63 87 88 95 Table 3. Task Analysis for Vocational Target Behaviors Table 4. Inter-observer Agreement (mean and range) Across Participants, Conditions, and Behaviors Table 5. Example of Social Behaviors Probed Across Vocational Tasks During Probe Sessions Table 6. Task Analysis for Vocational Target Behaviors Table 7. Operational Definitions of Social Skills with Example Scenarios Table 8. Example of Social Behaviors Probed Across Vocational Tasks During Probe Sessions ix LIST OF FIGURES Figure 1. Percentage of Trials Joseph Performed Vocational and Social Skills Figure 2. Percentage of Trials Anthony Performed Vocational and Social Skills Figure 3. Matrix for Vocational Social Skills across Vocational Tasks Figure 4. Percentage of Trials Dominic Performed Vocational and Social Skills Figure 5. Percentage of Trials Michael Performed Vocational and Social Skills Figure 6. Percentage of Trials Timothy Performed Vocational and Social Skills 69 70 93 104 105 106 x KEY TO ABBREVIATIONS ABA: Applied Behavior Analysis ASD: Autism Spectrum Disorder BCBA: Board Certified Behavior Analyst EBP: Evidence-based Practice ID: Intellectual Disability IDEA: Individuals with Disabilities Education Act IEP: Individualized Education Plan IOA: Inter-observer Agreement RCT: Randomized Control Trial SCED: Single Case Experimental Design WWC: What Works Clearinghouse xi CHAPTER 1 Introduction Autism spectrum disorder (ASD) is a lifelong neurological disorder characterized by difficulties with communication, social skills, and the presence of restricted or repetitive behaviors (American Psychiatric Association, 2013). Although individuals with ASD are typically diagnosed in childhood, deficits in communication and social skills persist into adolescence and adulthood, where these individuals continue to be negatively impacted both socially and occupationally (Plimley. 2007; Rao, Bidel, & Murray, 2008; Roux, Shattuck, Rast, Rava, & Anderson, 2015). Estimates suggest approximately 50,000 individuals with ASD turn 18 years old each year in the United States alone (Shattuck et al., 2012b) and for many of these individuals, school-based services will end. The majority of individuals with ASD will continue to need some sort of support throughout their adult lives (Buescher, Cidav, Knapp, Mandell, 2014). Employment opportunities are often limited for those with disabilities, but appear to be worse for individuals with ASD, who are often underemployed or unemployed at higher rates than those with other disabilities (Chen, Sung, & Pi, 2015). Individuals with ASD who do work often fail to hold regular employment due to behavioral problems or poor social skills, rather than an inability to perform the actual job tasks (Hurlbutt & Chambers, 2004; Müller, Schuler, Burton, & Yates, 2003; Müller, Schuler, & Yates, 2008). Employment and Adults with ASD Successful employment is often the end goal following high school or post-secondary education for many individuals with and without disabilities. The benefits of employment for individuals with ASD have been well documented in the literature and have been shown to improve quality of life (García-Villamisar, Wehman, & Navarro, 2002), increase independence 1 (Persson, 2000), increase personal dignity (García-Villamisar, Ross, & Wehman, 2000), and even improve cognitive performance (García-Villamisar & Hughes, 2007). Earning a meaningful wage allows individuals with disabilities financial freedom to support themselves and pursue personal interests without relying on government funding (Hendricks, 2010). Despite the many documented benefits of work, adults with ASD continue to struggle to find and maintain employment. An estimated 50 to 75% of adults with ASD are unemployed including higher functioning individuals and those who have attended post-secondary education (Howlin, Goode, & Hutton, 2004; Hurlbutt & Chalmers, 2004; Mawhood, Howlin, & Rutter, 2000; Roux et al., 2015). This trend appears to start in high school, where it has been documented that at least 35% of young adults with ASD never work or attend postsecondary education upon leaving high school (Shattuck et al., 2012b). The barriers to employment seem to be more complex for individuals with ASD than those with other disabilities, with the social demands of the workplace frequently cited as the most common obstacle to successful employment (Hurlbutt & Chambers, 2004; Müller et al., 2003). Social skills are arguably one of the most important skillsets for obtaining and maintaining employment for individuals with and without disabilities. The need for increased focus on social skills for individuals with disabilities has been well documented over the past 30 years (Agran, Hughes, Thoma, Scott, 2016). Unfortunately, many individuals with disabilities, especially those with ASD, still have difficulty maintaining employment due to a lack of appropriate social skills for the workplace. Work related social skills include any social skill related to obtaining or maintaining employment, such as appropriate interactions with customers, co-workers, and supervisors (Agran et al.). Interacting appropriately with others is complex and involves understanding and using verbal and non-verbal language, such as facial expressions and 2 body language. Even jobs composed of mundane tasks require at least some sort of social interaction on a regular basis. This means that even the most skilled employee must possess at least a basic level of social competency before entering the work force. Teaching and acquiring the full range of social skills required for the work place is difficult with at least 27 unique social skills identified as essential for entry-level employment, such as seeking clarification or assistance when needed, arriving on time, appropriate physical contact with others, responding to criticism, and following directions (Agran et al.). Unlike job tasks, which often have a set of clear expectations and ongoing training opportunities, work related social skills are not typically written down and employers often assume employees will come with the necessary social skills for the job (Butterworth & Strauch, 1994). Individuals with ASD typically require explicit training in social skills; however, adolescents and adults with ASD receive relatively few opportunities to learn and practice social skills. Transition Planning Schools play a crucial role in preparing youth with ASD for the transition to adulthood. Transition planning is described as a “results oriented process” beginning no later than age 16 that focuses on preparing individuals for post-school activities, such as post-secondary education, vocational education and employment through instruction, community experiences, and acquisition of daily living skills (IDEA, 2004). Given the low employment rates among adults with ASD, more needs to be done to better prepare youth with ASD for the workplace. Successful transition to adulthood requires thoughtful coordination of services between school, home, work, and community (Hendricks & Wehman, 2009). Several factors have been identified as important features of successful transition planning to better prepare adolescents with ASD for the transition to adulthood. First, exposure 3 to work experiences and job-related services, such as opportunities to explore various careers, vocational courses, career interest assessments, or job counseling should be incorporated into transition plans (Test et al., 2009). Providing multiple opportunities for adolescents with ASD to gain hands-on experience and career knowledge allows individuals to acquire job skills and practice social skills in a relatively safe environment. Second, social skills training focused on methods for generalizing skills should play a larger role in the education of adolescents with ASD (Hendricks & Wehman, 2009). Relatively little time is spent teaching social skills to adolescents with ASD (Reichow & Volkmar, 2010), despite the abundance of typically developing peers in high school settings. Teaching general social skills that are relevant to both school and work settings may increase the likelihood that these skills will generalize to relevant environments. Third, families of adolescents with ASD should be connected to a range of adult service providers, such as vocational rehabilitation agencies, independent living centers, and mental health services (Test, Smith, & Carter, 2014). Establishing these relationships while adolescents with ASD are still in school provides youth with ASD with potential tools for post- secondary success and can allow for a more seamless transition to adulthood. Identification of Evidence Based Practices There are currently a wide range of potential interventions for individuals with ASD, some of which are potentially harmful (i.e., chelation therapy), which is why the use of practices backed by research is important for this population. Evidence-based practice (EBP) refers to practices that have shown to be effective by peer-reviewed research (Odom et al., 2005; Cook, Tankersley, Cook, & Landrum, 2008). In special education, educators are required to use scientifically based practices to improve academic and functional skills of individuals with disabilities (IDEA, 2004). The use of EBPs to teach work related skills to individuals with ASD 4 is important given the limited time and resources available to teach these skills. Several recent literature reviews have examined the vocational skills literature for adolescents and adults with ASD to evaluate the quality of published studies and determine if any of the intervention strategies can be considered EBPs. Two reviews have applied the most recent protocol for evaluating Single Case Experimental Design (SCED) studies, the Single-Case Design Technical Documentation from the What Works Clearinghouse (WWC; Kratochwill et al., 2010; 2013) to the vocational skills literature (Anderson et al., 2016; Seaman & Cannella-Malone, 2016). Instead of rating each study based on a variety of criteria and assigning the study an overall score, each study evaluated using the WWC standards is critically screened to ensure it sufficiently meets the methodical rigor for the selected design. The WWC design applies the most rigorous evaluation of internal validity by focusing on design conventions and evidence standards to determine if a practice is evidence-based or not. Overall, researchers have concluded that the lack of published research studies focused on employment outcomes for adults with ASD, as well as the low methodological quality of these studies, makes it difficult to draw conclusions regarding effective vocational treatment approaches for this population (Anderson et al., 2016; Nicholas et al., 2015; Seaman & Cannella- Malone, 2016; Taylor et al., 2012; Walsh et al., 2014; Walsh et al., 2017; Westbrook et al., 2012; Westbrook et al., 2015). Only one review, conducted by Walsh and colleagues (2017), explicitly coded for work related social skills. However, this review was limited to only technology aided interventions, meaning studies that may have utilized a different intervention to teach work related social skills were excluded. Work related social skills are arguably more important than specific job skills; therefore, social skills should be examined as a part of the vocational skills literature. 5 Teaching Vocational Skills with Video Modeling Based on the vocational skills literature reviews, video modeling appears to be the most commonly used practice for teaching vocational skills to adolescents and adults with ASD. Video modeling is an EBP with substantial support and has been shown to be effective at increasing a variety of behaviors among individuals with ASD, such as social skills, communication, vocational skills, and daily living skills (Bellini & Peters, 2008). Video modeling can be delivered in various formats (i.e., video prompting, video self-modeling, and point of view modeling) but typically involves a video presentation of a person engaging in the target behavior and then the participant has an opportunity to demonstrate the desired skill (Bellini & Akullian, 2007). Video modeling may be effective for individuals with ASD for a several reasons. Video modeling plays to the strengths of individuals with ASD, who often have a preference for and strength in visually cued instruction (Quill, 1997). Due to the technological aspect of video modeling, extra visual and auditory stimuli can be removed and videos can be edited to focus on relevant stimuli (Corbett & Abdullah, 2005). Repeated viewings of the model are possible and the videos never change, giving it an advantage over live modeling by providing consistency and repetition (Bellini & Akullian, 2007). Video modeling may also be effective because the intervention can be implemented without much adult support (Kellums & Morningstar, 2012; Van Laarhoven, Van Laarhoven-Myers, & Zurita, 2007). This is especially important in employment settings for individuals with disabilities who are required to complete job tasks independently or with occasional support from a job coach. 6 Promoting Generalization with Matrix Training Generalization is the ability to use skills with new people, settings, behaviors, or materials beyond what he or she was taught (Stokes & Baer, 1977). Once individuals learn a new skill or behavior, generalization of these skills is often a goal of the intervention; however, individuals with ASD do not readily generalize learned skills (Townley-Cochran, Leaf, Taubman, & Leaf, 2015). Failure to generalize learned behaviors is often seen as one of the biggest barriers to the success of treatment (Vismara & Rogers, 2010). Treatment for individuals with ASD should include strategies to increase the likelihood that these individuals will generalize skills, such as teaching across sufficient examples, programming common stimuli, teaching loosely, and using intermittent reinforcement (Stokes & Baer). Since it is impractical to continue an intervention indefinitely, generalization of skills should be programmed for to promote the transfer to skills to the natural environment without the intervention in place. Matrix training is a proven technique that is both effective and efficient in training and generalizing skills to a wide range of individuals. In matrix training, the targets are systematically identified and components of the desired response combinations are arranged along two axes of a matrix. For example, a 2 by 2 matrix could be used to teach non-verb combinations, with two verbs listed along one axis and two nouns listed along the other axis. The targets are then trained along the diagonal of the matrix. For example, if a child is taught “roll ball” and “push car”, the responses “push ball” and “roll car” may emerge without direct training. When learners demonstrate correct responding to the untrained areas of the matrix without direct training, this is called recombinative generalization (Goldstein, 1983a). Matrix training allows for the most efficient instruction because skills are learned without being directly taught. Although matrix training has yet to be used with older individuals with ASD, it has been 7 used in combination with video modeling to teach and facilitate generalization of play skills to young children with ASD. Only one known study has combined video modeling and matrix training in order to teach three children with ASD to engage in novel play using three different play sets (MacManus, MacDonald, and Ahearn, 2015). Once participants had been trained on at least one play set, they were able to recombine actions and vocalizations with play sets that had not been previously trained, indicating that matrix training may offer a more systematic approach to increasing novel play. Matrix training has not yet been used to teach vocational or social behaviors, but could be used in a similar manner to MacManus and colleagues (2015), wherein the vocational tasks would take the place of the play sets and the social skills would replace the play behaviors. The diagonal targets would be selected for intervention because of the perceived usefulness of the social skill for the vocational task. After learning these combinations, the effects of recombinative generalization may be seen as participants perform each social behavior across each vocational task. Identifying effective and efficient methods for teaching and generalizing social skills is essential for teaching work related social skills given the limited time and resources that are devoted to social skills instruction in transition programs. Outline of the Current Dissertation and Research Questions The proposed dissertation is composed of five chapters on the topic of vocational and social skills for adolescents with ASD. Chapter 1 is the introduction, which contains an overview of the current issues surrounding the vocational skills literature for adolescents and adults with ASD. Chapters 2, 3, and 4 are stand-alone research studies written for publication in various journals. A brief overview of each manuscript is provided below, including information 8 on each journal and the specific research questions. Chapter 5 synthesizes the results of all three studies into a final discussion of the outcomes and implications for future research. Chapter 2 is a systematic review of the literature that examines the research that has already been conducted on this topic and outlines the next steps that need to be done to advance the vocational skills literature for individuals with ASD. Each study in the review was evaluated using the WWC design standards (Kratochwill et al., 2013) to ensure it sufficiently met the methodical rigor for the selected design. The WWC evidence standards were then applied to evaluate the quality of studies that met standards or met standards with reservations to determine if any practices could be considered EBPs for teaching vocational skills. Although several literature reviews have been recently published on this topic, flaws are present among previous reviews, such as the exclusion of SCED studies (Taylor et al., 2012; Westbrook et al., 2012); summarizing findings without evaluating the quality of the studies (Nicholas et al., 2015; Walsh et al., 2014) or determining if interventions could be EBPs (Seaman & Cannella-Malone, 2016); including studies not directly related to employment (Anderson et al., 2016); or only including technology aided interventions (Walsh et al., 2017). Although the WWC standards have recently been applied to the vocational skills literature, none have examined both vocational skills and work related social skills in school and employment settings. Given the multitude of reviews on this topic, an in-depth analysis of the vocational skills literature in relation to the most rigorous standards (i.e., WWC standards) is warranted in order to identify methodological strengths and weaknesses of published studies and identify directions for future research. The results of the literature review in Chapter 2 provide the background for a pilot study of a video modeling intervention to teach vocational and social 9 skills to adolescents with ASD. This literature review examined the following research questions: 1. When applying the WWC design standards to the published SCED studies to evaluate the quality of these studies, how many meet standards, meet standards with reservations, or do not meet these standards? 2. When applying the WWC evidence standards to studies that met standards or met standards with reservations, do these studies demonstrate strong, moderate, or no evidence of effect? 3. Is there enough evidence for any intervention to qualify as an evidence-based practice for teaching vocational or social skills to adolescents and adults with ASD? Chapter 3 is the manuscript for a pilot research study examining the effects of video modeling on the acquisition of vocational skills and social skills among adolescents with ASD. The pilot study utilized a multiple probe across behaviors design; however, the study ended prematurely partly due to issues with feasibility in a clinical setting, so there are not enough replications of effect to demonstrate a functional relation. In addition, the participants were not ideal for the intervention given the presence of maladaptive behaviors or seizures, which at times, prevented participants from attending the simulated work site. The study extends the current research by examining the effectiveness of one video to teach two distinct skills (i.e., vocational and social skills). The present study investigated the following research questions: 1. Does video modeling lead to the acquisition of vocational skills for adolescents with ASD when implemented in a vocational setting? 2. Does video modeling lead to the acquisition of social skills for adolescents with ASD when implemented in a vocational setting? 10 Chapter 4 is the third publishable paper in this dissertation that extends and provides an experimental analysis of the hypothesis formulated based on the pilot study. This research study utilized a multiple probe design across behaviors replicated across participants to teach three adolescents with ASD to operate a concession stand for adult employees of a high school. The current study extends the literature by examining both the effectiveness and efficiency of video modeling on the acquisition of vocational and social skills taught simultaneously among adolescents with ASD. Second, the study extended the literature on matrix training by examining its effectiveness in facilitating the generalization of social behaviors across vocational tasks. More specifically, the present study investigated the following research questions: 1. Does video modeling lead to the acquisition of vocational skills for adolescents with ASD when implemented in a vocational setting? 2. Does video modeling lead to the acquisition of vocational social skills for adolescents with ASD when implemented in a vocational setting? 3. Does matrix training facilitate the generalization of vocational social skills across vocational tasks? 4. Can participants generalize vocational and social skills to a new setting and with new people? 11 CHAPTER 2 Systematic Literature Review The vast majority of research examining interventions for individuals with autism spectrum disorder (ASD) has focused almost exclusively on early intervention or school age services and supports, with limited attention to adolescents and adults with ASD (Wong et al., 2015). This is particularly concerning given that in the United States, the majority of costs for individuals with ASD are related to adult services, which costs approximately $175-196 billion compared to the estimated $61-66 billion for children (Buescher et al., 2014). Over the next decade, estimates suggest half a million adolescents with ASD will graduate high school, with approximately 50,000 individuals with ASD turning 18 years old each year in the United States alone (Roux et al., 2015). The majority of these individuals will continue to need some support or additional services throughout their adult lives. Vocational and Social Challenges for Adolescents with ASD Although individuals with ASD are typically diagnosed in childhood, characteristics associated with the disorder, such as deficits in verbal and non-verbal communication or social skills, persist into adolescence and adulthood, where these individuals continue to be negatively impacted in almost all aspects of life, including employment, socializing, and living independently (Plimley. 2007; Rao et al., 2008; Roux et al., 2015; Seltzer et al., 2004). Employment opportunities are often limited for those with disabilities, but individuals with ASD tend to have greater difficulty finding and sustaining employment than individuals with other disabilities. Approximately 37% of young adults with ASD never had a job or received postgraduate education following high school, compared to an estimated 8% of young adults with either a learning disability, emotional impairment, or speech and language impairment 12 (Roux et al., 2015; Shattuck et al., 2012). Individuals with ASD who do work often fail to hold regular employment due to characteristics associated with the disability, such as behavioral problems or poor social skills, rather than an inability to perform the actual work tasks associated with their job (Hurlbutt & Chambers, 2004; Müller et al., 2003, Müller et al., 2008). Despite these poor outcomes, research has shown that individuals with ASD can be taught the vocational and social skills necessary for the workplace with appropriate supports, which can help individuals with ASD maintain employment (see Nicholas et al., 2015; Seaman & Cannella- Malone, 2016; Walsh et al., 2014; Walsh et al., 2017). Previous Vocational Skills Literature Reviews Several literature reviews have synthesized the vocational skills literature for adolescents and adults with ASD. Researchers have concluded that the overall lack of published research studies focused on employment outcomes for adults with ASD makes it difficult to draw conclusions regarding effective vocational treatment approaches (Bennett & Dukes, 2013; Taylor et al., 2012; Nicholas et al., 2015; Walsh et al., 2014; Westbrook et al., 2012). In addition, many of the published studies were found to be of low methodological quality and contained high levels of potential bias. Within the vocational skills literature reviews, there has been a lack of emphasis on work related social skills, with many reviews focusing solely on vocational skills for individuals with ASD. Only one study, conducted by Walsh and colleagues (2017), included work place social skills in their review of vocational skills for adolescents and adults with ASD. Researchers utilized Reichow’s (2011) evidence-based practice (EBP) criteria to evaluate the quality of 18 studies that utilized technology-aided interventions. Walsh and colleagues only found three studies that examined social skills; however, none of these studies utilized a SCED to explicitly 13 teach work related social skills to individuals with ASD. One study utilized a pre- post design to examine changes on social cognitive measures, another study examined interview skills, and the third study taught general social skills. In addition, since the review was only focused on technology-aided interventions, it is possible other studies that taught work related social skills to individuals with ASD were not included. Technology-aided interventions were categorized as a probable EBP, with nine studies receiving an adequate rating, one study received a strong rating, and eight studies received a weak rating. When each intervention type (i.e., video modeling, video feedback, audio cueing, and virtual reality) was evaluated separately, none met the criteria for EBP. Despite technology aided interventions being categorized as a probable EBP, almost half of the studies reviewed received a weak rating and only one study received a strong rating, which supports previous findings regarding the inadequate rigor of intervention studies to teach vocational skills to adolescents with ASD. Previous Applications of the WWC Standards to the Literature Two recent studies (Anderson et al., 2016; Seaman & Cannella-Malone, 2016) have applied the Single-Case Design Technical Documentation from the What Works Clearinghouse (WWC; Kratochwill et al., 2010; 2013) to the vocational skills literature to determine if any practices can be considered EBPs. Each study evaluated using the WWC standards is critically screened to ensure it sufficiently meets the methodical rigor for the selected design. A visual analysis protocol is then applied to studies that meet criteria or meet with reservations to categorize studies as demonstrating strong evidence, moderate evidence, or no evidence. The WWC design standards use objective measures and apply the most rigorous evaluation of internal validity by focusing on design conventions and evidence standards to determine if a practice is evidence-based or not. 14 Seaman and Cannella-Malone (2016), identified 15 SCED studies evaluating pre- employment skills, vocational skills, or job retention skills published between 2010 and 2015. The WWC standards were applied to the 15 studies and 60% met evidence standards or met evidence standards with reservations. Of those nine studies that met evidence standards with or without reservations, 67% (n = 6) demonstrated strong evidence of effect and 22% (n = 2) demonstrated moderate effect. Intervention strategies were not analyzed further to determine if any practices could be considered evidence-based. This review focused exclusively on intervention strategies to obtain employment or teach specific job skills; therefore, studies examining work related social skills were not included. Although researchers examined the strength of evidence of the included studies, they did not determine if any of the practices could be considered EBPs. Anderson and colleagues (2016) also applied the WWC standards to the vocational skills literature for adults with ASD. Studies that targeted any skill that was likely to increase employment opportunities, such as activities of daily living, reading fluency and comprehension, internet skills, and using a mobile phone, were included in the review, but studies conducted in school settings were excluded. Of the three studies that taught social skills, only one targeted work related social skills. Behavioral skills training, which had nine studies that met the WWC design standards with or without reservations, was concluded to be an EBP for teaching vocational related skills. However, only 44% of the behavioral skills training studies actually targeted vocational skills. Researchers did not give an in-depth analysis of the application of the WWC standards, which is critical to improve the quality of these studies in future research. Purpose of Systematic Literature Review Several groups of researchers have evaluated the vocational skills literature, with mixed 15 findings across studies (Anderson et al., 2016; Bennett & Dukes, 2013; Taylor et al., 2012; Nicholas et al., 2015; Seaman & Cannella-Malone, 2016; Walsh et al., 2014; Walsh et al., 2017; Westbrook et al., 2012). Although the WWC standards have recently been applied to the vocational skills literature, none have examined both vocational skills and work related social skills in school and employment settings. Therefore, the purpose of this systematic literature review was to (a) evaluate the methodological rigor of intervention studies examining methods for teaching vocational skills and work related social skills to adolescents with ASD, using the WWC design standards in order to determine studies that meet standards, meet standards with reservations, or do not meet standards; (b) evaluate studies that meet standards or meet standards with reservations to determine if there is strong evidence, moderate evidence, or no evidence of causal effects; and (c) determine if there is enough evidence for any intervention to qualify as an evidence based practice (EBP) for teaching vocational skills or work related social skills. Method Search strategies. This review was based on a systematic search of peer-reviewed research studies published and available from 1980-2017. The Pro-Quest online collection of databases, which contains 94 social sciences including ERIC and PsycINFO, was searched for entries containing any combination of the following search terms: (1) vocational OR occupational OR job OR employment, and (2) social, and (3) aut* OR ASD OR asperger. The abstracts of the articles containing the search terms were screened for the following inclusion criteria: the study (a) consisted of a researcher manipulated intervention that explicitly taught only vocational or vocational social skills, (b) included at least one participant diagnosed with ASD and (c) was published in an English language, peer-reviewed journal. Studies related to obtaining employment and naturally occurring services, such as supported employment, were 16 excluded from review. If a study included participants without ASD and that participant’s data could be separated from the rest, the participant with ASD was included, but those without ASD were excluded from review. In order to evaluate studies according to the WWC standards, only studies that focused solely on vocational or work related social skills social skills were included in this review, meaning studies that taught vocational skills in conjunction with other skills (i.e., daily living skills) were excluded. Work related social skills included studies that taught social skills in a vocational setting or taught social skills necessary to complete job tasks. To ensure all relevant studies were included in this review, references from the selected articles were also examined to identify potential studies not captured during the search. Application of design standards. The WWC single case research design standards (Kratochwill et al., 2013) were used in the current study to evaluate the methodological rigor of the vocational skills literature for adolescents and adults with ASD. A coding protocol for the design standards was created based on stage 2 of the WWC study review guide (Institute of Education Sciences, 2014) and was used to code all studies included in this review (see below). The design standards were applied to individual experiments within each study, meaning each experiment was evaluated separately. A single participant was considered an experiment in reversal/withdrawal designs, alternating treatments, changing criterion, multiple baseline across settings or behaviors, and multiple probe across settings or behaviors. Studies that utilized a multiple baseline or multiple probe design across participants were considered one experiment since one’s ability to make inferences regarding a functional relation were dependent on the performance of each participant. Each study was evaluated based on the design standards to determine if it met standards, met standards with reservations, or did not meet standards. Any 17 studies that did not meet design standards with or without reservations were not evaluated further. The first standard from the WWC design standards (Kratochwill et al., 2013) requires that the independent variable be systematically manipulated. This standard must be met or the study will not meet the design standards. The second standard requires interobserver agreement (IOA) to be documented for at least 20% of data points per condition and meet minimum acceptability levels of 0.80 on average if measured by percentage and .60 if IOA is measured using Cohen’s kappa (Kratochwill et al., 2013). To meet the third standard, the study must have a minimum of three demonstrations of an intervention effect at three different points in time (Kratochwill et al., 2013). In order to meet the WWC criteria for demonstrating an intervention effect, studies are required to document a certain number of data points per phase. For multiple baseline and multiple probe designs, all studies must have at least three cases with concurrent, overlapping baseline phases and graphs must show consistency across all cases regarding timing of sessions. In addition, the intervention phases must not overlap across cases, with baseline measurements occurring at or after the point in which one case finishes training. Multiple probe designs also require that each case have a probe point when another case either first receives treatment or reaches a predetermined mastery criterion. Application of evidence standards. Experiments that meet standards or meet standards with reservations were then further evaluated using the WWC standards for visual analysis to determine the strength of the functional relation between the independent and dependent variable (Kratochwill et al., 2013). A coding protocol based on stage 3 of the WWC study review guide visual analysis section (Institute of Education Sciences, 2014) was created to determine the evidence rating of studies that met standards with or without reservations. Visual analysis of 18 graphical displays of data is conducted to determine that the systematic manipulation of the independent variable is responsible for the observed change in the dependent variable, which should be demonstrated at least three different points in time throughout the study. Visual analysis of graphs involves four steps across six features (level, trend, variability, immediacy of effect, overlap, and consistency of data patterns). First, baseline data are analyzed to ensure a stable and predictable pattern of data, with variability that is sufficiently consistent, and a stable trend or a trend moving away from the therapeutic direction. Baseline data must also demonstrate a need for the intervention. If baseline data appears stable, the second step is to examine the within-phase data patterns to ensure the level, trend, and variability of each condition is consistent. The third step is to examine the data patterns between phases. In addition to a difference between level, trend, and variability between adjacent phases, there should also be an immediate effect and an absence of overlap between each phase and the next. For multiple baseline and multiple probe designs, data must also be compared vertically to ensure changes in data patterns are independent of one another. For alternating treatment designs, the overall mean levels for intervention and comparison conditions should demonstrate separation between the series. The last step is to determine whether there are at least three demonstrations of an effect based on visual analysis of baseline data and within- and between-phase comparisons. An effect, or functional relation, is evident when systematic manipulation of the independent variable is associated with a change in the dependent variable. Studies demonstrate strong evidence if there are at least three demonstrations of effects and no demonstrations of non-effects. Studies with moderate evidence demonstrate at least three effects but also at least one non-effect or a multiple baseline or multiple probe design that suggests changes in data patterns are not independent of one another. 19 Studies are rated as having no evidence if there are fewer than three effects at different points in time or there is no demonstration of a proposed concern evident in the baseline data. Evidence based practice. Once the design and evidence standards were applied to all SCED studies, the last step was to determine if there was enough evidence for the intervention to qualify as an EBP according to the WWC standards. To determine if any of the interventions qualify as an EBP, three criteria must be met including, (a) a minimum of five SCED experiments that meet standards or meet standards with reservations and demonstrate strong or moderate evidence of effect; (b) the experiments were conducted by at least three distinct research teams without overlapping authorship at three different locations, and (c) at least 20 combined cases or participants across papers (Kratochwill et al., 2013). If the ‘5-3-20’ rule is met, the evidence for an intervention is considered by calculating effect size. Coding of articles. A total of three separate coding protocols were used to evaluate the studies included in this review and were based on the WWC study review guide. All articles were first coded as either teaching vocational skills or work related social skills. All of the articles were then coded using the design standards found in stage 2 of the WWC study review guide (Institute of Education Sciences, 2014) to assess the quality of evidence to determine if studies met standards, met standards with reservations, or did not meet standards. Studies that did not meet design standards with or without reservations were not evaluated any further. The first author and a doctoral level graduate student with a master’s degree in special education coded the articles. The first author conducted a 2-hr initial training with the doctoral student to review the essentials of SCED research, an overview of the WWC design standards, and detailed information regarding each item on the protocol. The first author then demonstrated how to apply the protocol to SCED studies that were not included in this review. The graduate assistant 20 was then given an opportunity to apply the WWC design standards to a different study and was provided with feedback. The first author and graduate assistant then independently coded studies not included in this review until 100% agreement on all design standards on a single study was achieved. After all articles were coded using the designs standards protocol, only experiments that met standards or met standards with reservations were then coded using stage 3 of the WWC study review guide visual analysis section (Institute of Education Sciences, 2014) to determine the evidence rating. Training on the evidence standards protocol was conducted in the same manner as the design standards protocol. All experiments that met standards or met standards with reservations were coded by the first author according to stage 4 of the WWC study review guide to describe additional experiment details, including setting, research design, participant characteristics, baseline and intervention conditions, maintenance phases, independent variable, dependent variables, social validity, and implementation fidelity. Once all experiments were coded, each intervention method was evaluated using the EBP criteria to determine if any of the interventions could be considered an EBP for teaching vocational skills to adolescents and adults with ASD. Categorization of experiments. All experiments that met the inclusion criteria for this review were grouped by the intervention method used to teach vocational skills. Experiments were separated into four intervention categories: video modeling interventions, audio cueing interventions, textual cueing interventions, and additional intervention strategies. Video modeling experiments utilized video modeling or a variation of video modeling, such as video self-modeling, video prompting, or video rehearsal as the main independent variable. Audio cueing experiments utilized covert audio coaching or covert audio cueing, which consists of a 21 two-way radio system that allows the instructor to deliver prompts or feedback from afar so that only the individual being coached can hear, as the main independent variable. Textual cueing experiments used written or pictorial cues, such as written scripts or picture schedules, as the main independent variable. All remaining interventions with only one study per intervention type were categorized as additional intervention strategies. Studies that taught work related social skills were separated into one of the four categories based on the intervention used to teach the skill. Video modeling and audio cueing experiments were separated into two separate categories, rather than grouped together into the broad category of technology-aided interventions, for several reasons. In their review of technology aided interventions, Odom and colleagues (2015) identified seven distinct types of interventions that utilized technology, which categorized video modeling and audio cueing as separate interventions. In addition, the method of instruction varies greatly between interventions. Video modeling is a method of instruction in which the learner is shown a visual model of the skill and then provided with an opportunity to perform the skill (Plavnick, 2013), whereas audio cueing interventions are typically used to provide in the moment feedback to participants (Bennett et al., 2010). Although both interventions are considered types of technology aided interventions, video modeling is also categorized as its own evidence-based practice by the National Professional Development Center for ASD (Wong et al., 2015). Given that the goal of this review is to identify evidence-based practices for teaching vocational skills to adolescents and adults with ASD, the present review categorizes interventions in alignment with highly regarded evidence-based documentation (e.g., Wong et al.). 22 Coder agreement. As in Maggin and colleagues (2013), inter-rater reliability was calculated for all items on the design standards protocol. A point-by-point agreement was used to determine the percentage of agreements (total number of agreements/total number of agreements plus disagreements) on the design standards. Inter-rater reliability was calculated for 75% of included experiments. The average agreement across the seven items on the design standards protocol was 95% (range, 72 to 100%). For the additional criteria required for multiple probe and multiple baseline designs, the average agreement was 92% (range, 68 to 100%). There were three total discrepancies regarding whether an experiment met standards, met standards with reservations, or did not meet standards. To resolve these discrepancies and arrive at a final rating, both coders reviewed and discussed the design standard description and the study until 100% agreement was achieved. For the evidence standards, inter-rater reliability was calculated for the overall evidence rating for 75% of experiments that met standards or met standards with reservations. The average agreement for the overall evidence rating of experiments was 90%. Discrepancies occurred in determining whether experiments demonstrated strong, moderate, or no evidence of effect. These discrepancies were resolved in the same manner as described for the design standards. Results A total of 24 studies, consisting of 52 experiments, met the inclusion criteria for this review. Of the 52 total experiments, 32 were classified as video modeling interventions, six were categorized as audio-cueing interventions, six were considered textual cueing interventions, and eight fell into the category of additional intervention strategies. A total of four studies, consisting of 11 experiments, taught work related social skills to adolescents or adults with ASD. Since the 23 unit of analysis for SCEDs is at the individual case, rather than the study level, references will be made to the experiments, which represents the unit of analysis for individual cases. Design standards. All of the experiments were evaluated using the design standards. Table 1 summarizes the results of the application of the design standards for the studies included in this review. Approximately 50% of experiments either met standards (n = 11) or met standards with reservations (n = 15), and 50% of experiments (n = 26) did not meet standards. For the majority of the experiments that did not meet the standards, the primary issue was that the experiment did not meet the criteria for the minimum number of data points per phase (n = 18). Of the 11 experiments that taught work related social skills, less than 20% of experiments met standards (n = 2; Dotto-Fojut, Reeve, Townsend, & Progar., 2011; Rausa, Moore, & Anderson, 2016) and 82% of experiments did not meet standards (n = 9; Mackey & Nelson, 2015; Walsh, Holloway, & Lydon, 2017). A total of 32 experiments were categorized as video modeling interventions. The majority of video modeling experiments (72%) met standards (n =8; Allen, Wallace, Renes, Bowen, & Burke, 2010b; Bennett, Gutierrez, Honsberger, 2013; Cihak & Schrader, 2008; Rausa, Moore, & Anderson, 2016; Van Laarhoven, Van Laarhoven-Myers, & Zurita, 2007) or met standards with reservations (n =15; Alexander, Ayres, Smith, Shepley, Mataras, 2013; Bennett et al., 2013; English et al., 2017; Goh & Bambera, 2013; Kellums & Morningstar, 2012). A total of 28% of experiments did not meet standards due to inter-observer agreement not meeting the minimum threshold of 80% agreement (n =1; Allen et al., 2010a) or phases not having at least three data points per phase (n =8; Burke et al., 2013; Walsh, Holloway, & Lydon. 2017). A total of six experiments were categorized as audio cueing interventions. Of the six audio cueing experiments, 33% met standards (n =2; Bennett, Ramasamy, Honsberger, 2013a; 24 Bennett et al., 2013b) and none met standards with reservations. A total of 66% of audio cueing intervention experiments (n =4) did not meet standards due to a lack of IOA data collection (Bennett, 2010) and not having at least three data points per phase or not having three attempts to demonstrate an effect (Allen et al., 2012). Of the six textual cueing experiments, only one met standards (n =1; Dotto-Fojut et al., 2011). The majority of experiments (n =5) did not meet standards due to at least one phase with less than three data points (Burke, Andersen, Bowen, Howard, & Allen, 2010; Ganz & Sigafoos, 2005) or a combination of concerns, such as the experiment not including at least three attempts to demonstrate an intervention effect or collect IOA data (Robinson & Smith, 2010) and the intervention not meeting additional criteria for multiple baseline designs (White, Hoffmann, Hoch, Taylor, 2011). Of the eight additional intervention strategy experiments, none of the experiments met standards with or without reservations. This was due to at least one data phase with less than three data points (n =5; Dotson, Richman, Abby, Thompson, & Plotner, 2013), not meeting additional criteria for multiple probe designs (n =1; Lattimore, Parsons, & Reid, 2006), or a combination of design flaws (n =2; Mackey & Nelson, 2015). None of these studies were further evaluated using the evidence standards. 25 Table 1. Results of Design Quality Analysis and Evidence Rating Author Number of Experiments Manipulation of IV IOA Three attempts to demonstrate an effect Three data points in each phase Criteria for multiple baseline or multiple probe Results of evidence standards Meets Design Standards Allen et al. (2010b) Bennett et al. (2013a) 1 1 Bennett et al. (2013b) 1 Bennett, Gutierrez, & Honsberger (2013) Cihak & Schrader (2008) Dotto-Fojut et al. (2011) Rausa, Moore, & Anderson (2016) 1 4 1 1 Yes Yes Yes Yes Yes Yes Yes VanLaarhoven et al. (2007) Meets Design Standards with Reservations 1 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Alexender et al. (2013) 2 Yes Yes Yes Yes 26 Yes Yes No evidence Strong evidence Not applicable Strong evidence Not applicable No evidence Not applicable No evidence Yes Yes Yes Strong evidence Moderate evidence Strong evidence Yes, with reservations2 No evidence Table 1. (cont’d) Bennett, Gutierrez, & Honsberger (2013) English et al. (2017) Goh & Bambara (2013) 4 4 1 Kellems & Morningstar (2012) Does not Meet Design Standards 4 Allen et al. (2010a) Allen et al. (2012) Bennett (2010) Burke et al. (2010) Burke et al. (2013) Dotson et al. (2013) Ganz & Sigafoos (2005) Lattimore, Parsons, & Reid (2006) 1 3 1 2 1 5 1 1 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Not applicable No evidence Yes, with reservations1 Strong evidence Yes, with reservations1 Yes, with reservations2 No evidence Strong evidence No Yes Yes Not applicable Not Yes No No No Yes Yes Yes Yes Yes Yes Yes Yes applicable No (for 1 experiment) Not applicable Not applicable No No No No No Not applicable Not No3 No3 No1, 2 applicable Not applicable Not applicable Not applicable Not applicable Not Yes Yes Yes No3 applicable Not applicable 27 Table 1. (cont’d) Mackey & Nelson (2015) Robinson & Smith (2010) Walsh, Holloway, & Lydon (2017) White et al. (2011) 2 1 7 1 Yes Yes Yes Yes No No No No Yes Yes Yes Yes Yes Yes No Yes No3 Not applicable Not applicable Not No1, 2 No3 applicable Not applicable Not applicable Note. 1 = Fewer than 3 baseline data points at the start of each phase. 2 = Insufficient data points just prior to introduction of intervention. 3 = Probe data not vertically aligned according to standards. 28 Evidence standards. The WWC visual analysis protocol was applied to experiments within studies that met standards or met standards with reservations. Of the 26 experiments (12 studies) that met the design standards or met design standards with reservations, 50% of experiments demonstrated strong evidence (n =12) or moderate evidence (n =1) of a functional relation between the independent and dependent variable and 50% of the experiments provided no evidence of a functional relation. Table 1 summarizes the results of the evidence ratings for the studies included in this review. The two experiments that taught work related social skills demonstrated strong (Dotto-Fojut et al., 2011) or moderate evidence of effect (Rausa et al., 2016). Of the 23 video modeling experiments that met standards (with or without reservations), 39% demonstrated strong evidence of causal relation (n = 9; English et al., 2017; Kellums & Morningstar, 2012; Van Laarhoven et al., 2007), 4% demonstrated moderate evidence of effect (n = 1; Rausa et al., 2016), and 57% demonstrated no evidence. All video modeling experiments utilizing alternating treatment designs demonstrated no evidence due to continuous overlapping means between intervention and comparison conditions (n = 9; Bennett et al., 2013; Cihak & Schrader, 2008). For the remaining video modeling experiments, the most common reason experiments demonstrated no evidence was due to no effect between phases (n = 3; Alexander et al., 2013; Goh & Bambara, 2013) followed by variability within phases (n = 1; Allen et al., 2010b). Both audio cueing experiments (Bennett et al., 2013a; Bennett et al., 2013b) that met standards also demonstrated strong evidence of effect based on the WWC evidence standards. The one textual cueing intervention that met standards (Dotto-Fojut et al., 2011) also demonstrated strong evidence of effect. 29 Video modeling experiments with strong evidence. Several experiments utilized multiple probe designs to evaluate the effects of video modeling delivered through a portable media player on the acquisition of vocational skills among adolescents and adults with ASD. In four experiments, Kellums and Morningstar (2012) taught four young adults with ASD to independently use video modeling with written instructions on an iPod to complete vocational tasks within their respective jobs. Similarly, Van Laarhoven and colleagues (2007) utilized a pocket PC to teach one adolescent with Asperger’s to perform three different vocational tasks using video rehearsal and video feedback strategies. In both studies, participants were taught to use the devices independently. In another study, consisting of four experiments, English and colleagues (2017) used video modeling with video feedback to teach gardening skills to three adults with ASD at a commercial flower nursery. Across all three studies, the intervention was conducted in employment settings and the ways in which video modeling was delivered (e.g., via iPod or iPad) were found to be socially acceptable. All experiments across the three studies demonstrated stable baseline patterns, with no overlapping data points between baseline and intervention and a clear difference in level and trend between phases. Video modeling experiments with moderate evidence. Only one video modeling study, which taught work related social skills, demonstrated moderate evidence of effect. Rausa and colleagues (2016) used video modeling to teach one adult with ASD to answer customer phone calls for a floral company. Answering phone calls consisted of four different behaviors— listening, handling complaints, taking orders, and using professional speech. The intervention was effective at improving three out of the four social behaviors, with minimal differences between baseline and intervention for using professional speech. Results of the study 30 demonstrated video modeling can be effective for teaching social skills necessary to complete job tasks. Audio cueing experiments with strong evidence. In two separate experiments, Bennett and colleagues (2013a, 2013b) utilized multiple baseline across participants designs to examine the effects of covert audio coaching on the acquisition of photocopying skills (2013a) or folding T-shirts (2013b) among adolescents with ASD. An adult delivered supportive and corrective statements through a two-way radio with ear-buds while observing the participants from afar. In both experiments, all six participants acquired the vocational skill in only a few trials, maintained the skills at high levels of accuracy, and in one experiment, participants were able to fold T-shirts in a novel work setting. Although participants in Bennett and colleagues (2013a) demonstrated relatively high levels of performance during baseline, the purpose of the study was to increase current skills, rather than skill acquisition. Despite high performance during baseline, data patterns remained relatively stable with no overlapping data points between phases and an immediate increase upon implementation of the independent variable. Results of both experiments demonstrate that covert audio coaching can be effective and efficient in teaching vocational skills to adolescents with ASD. Textual cueing interventions with strong evidence. Dotto-Fojut and colleagues (2011) utilized a multiple baseline across participants design to teach four adolescents with ASD a work related social skill. Participants were taught to describe a problem and ask for assistance in contrived work situations when materials needed to complete the job were missing, broken, or mismatched using graduated guidance, scripts, and script fading. All participants were able to independently approach an instructor, describe a problem, and request assistance. In addition, participants were able to use the skill with un-trained stimuli and a new setting, discriminate 31 between scenarios when assistance was needed and was not needed, and maintained the skill after one month. Despite gaps in data collection during baseline, data patterns remain stable with an increase in performance within the first three sessions of intervention. During intervention, data remained relatively stable with some participants demonstrated a slight drop in performance when the script was first faded. Overall results of this study demonstrate that individuals with ASD can be taught work related social skills while completing vocational tasks. Study Details. Only studies that met standards or met standards with reservations were reviewed further to examine additional relevant details. Table 2 provides a summary of participant characteristics, setting, research design, independent variable, maintenance and generalization data, social validity, and target behaviors of each study that met standards or met standards with reservations. A total of 38 participants were included in the 12 studies that met standards or met standards with reservations (26 experiments), with an average age of 18.4 years (range: 12 years old to 28 years old). Ninety two percent (n= 36) of participants were male. Participant race was only reported in two of the studies (Bennett et al., 2013; Bennett et al., 2013b), with five participants described as Caucasian, two as African American, and one as Hispanic. Approximately 80% (n= 31) of participants had a diagnosis of ASD, 15% (n= 6) were diagnosed with Asperger’s syndrome, and 5% (n= 2) had PDD-NOS, although only two studies (Alexander et al., 2013; Cihak & Schrader, 2008) reported standardized scores based on autism assessments or rating scales. Half of the studies were conducted in a vocational setting, such as the participant’s place of employment, while the other half took place in a school setting, such as a teacher workroom, classroom, or conference room. 32 Table 2. Summary of Studies that Met Standards or Met Standards with Reservations Author Description of participants Video Modeling Interventions Alexander et al. (2013) Allen et al. (2010b) Bennett, Gutierrez, & Honsberger (2013) Cihak & Schrader (2008) 7 adolescents (6 male and 1 female) between the ages of 15 and 18 years diagnosed with ASD and ID 3 young adult males between the ages of 17 and 22 years diagnosed with PDD-NOS (2) or PDD-NOS and Asperger’s 5 adolescent males between the ages of 13 and 18 years diagnosed with ASD 4 young adult males between the ages of 16 and 21 years diagnosed with ASD and ID Setting Methods Target Behaviors Maintenance Generalization Social Results Vocational: sorting mail Yes for 3 participants Yes-Setting generalization validity No School: classroom and conference room Single case; multiple probe across participants Work: retail store Single case; multiple baseline across participants Vocational: wearing a WalkAround® costume and engaging customers School: teacher workroom Single case; alternating treatment Vocational: Photocopying, making labels, and sending a fax Yes Yes- Setting generalization Yes No No Yes School: classroom and teacher workroom Single case; alternating treatment Vocational: Photocopying and sending faxes or preparing first aid and food kits. Yes No Yes 5 out of the 7 participants mastered the targeted skill after watching video models, although 3 participants also needed error correction to sort mail with 100% accuracy. All participants were able to use the targeted skills after watching video models. All participants made significant gains in targeted behaviors, but no differences were noted between conditions. Overall, video self modeling and adult modeling were both effective and efficient in teaching vocational tasks across participants, although individual differences were noted. 33 Table 2. (cont’d) English et al. (2017) Goh & Bambara (2013) Kellems & Morningsta r (2012) 3 adult males between the ages of 18 and 23 years of age diagnosed with ASD or Asperger’s (2) 1 adult female 28 years of age diagnosed ASD and ID 4 young adult males between the ages of 16 and 22 years diagnosed with ASD (2) or Asperger’s (2) Work: greenhouse Single case; multiple probe across behaviors Vocational: Gardening skills Yes Yes Yes- Stimulus generalization for 2 participants All participants demonstrated an increase in performance on all gardening tasks. Yes No No Yes No Yes The participant reached mastery criteria for two of the job tasks and approached mastery for making photocopies. All participants demonstrated an increase in performance on all vocational tasks and used the iPod independently. Work: University building Single case; multiple probe across behaviors Vocational: Assembling conference packets, shred paper, and photocopying Single case; Multiple probe across behaviors Work: Bowling alley, museum, community center, or various locations for vending machine business Vocational: 3 tasks for each participant. Cleaning bathrooms, vacuuming, cleaning sidewalk, filling out order book, taking inventory, filling crate with items to restock, emptying the garbage, breaking down boxes, recycling cardboard, cleaning glass display case, polishing wood panels. Rausa, Moore, & Anderson (2016) 1 adult male 23 years of age diagnosed with ASD Work: flower and plant farm Single case, multiple baseline across behaviors Social: Four telephone behaviors—listening, taking orders, handling complaints, and professional speech Yes No Yes Video modeling was effective for teaching three out of behaviors. 34 Table 2. (cont’d) Work: Restaurant 1 adolescent male 18 years of age diagnosed with ASD Single case; multiple probe across behaviors Vocational: Rolling silverware, sorting and sanitizing silverware, and clocking in and out of the workplace Van Laarhoven, Van Laarhoven- Myers, & Zurita (2007) No No No Video modeling delivered via pocket PC was associated with a significant increase in independent responses across all three targeted tasks. Audio Cueing Interventions Bennett et al. (2013a) Bennett et al. (2013b) School: teacher workroom School: student lounge area Single case; multiple baseline across participants Single case; multiple baseline across participants 3 adolescent and adult males between the ages of 13 and 22 years diagnosed with ASD 3 adolescents (2 males and 1 female) between the ages of 15 and 18 years diagnosed with ASD Vocational: Photocopying Yes No No All participants mastered the skill. Vocational: Folding t- shirts Yes No No All participants were able to perform the targeted behavior with 100% accuracy. Textual Cueing Strategies Dotto-Fojut et al. (2011) 4 adolescent males between the ages of 12 and 13 years diagnosed with ASD School: classroom Single case; multiple baseline across participants Social: Describing a problem and asking for assistance Yes Yes- Setting and stimulus generalization No All participants acquired the targeted skills. 35 Almost all of the studies (92%) provided information related to the procedures utilized during baseline session. Data were collected until stable in 42% of the studies or for a set number of sessions in 16% of studies. Criteria for moving on to intervention from baseline was not reported in 42% of studies. All studies provided information regarding the content and delivery of the intervention. Most studies (67%) listed specific mastery criteria required during the intervention condition. Two studies also provided information on the mean session length during baseline and intervention (Allen et al., 2010b; Bennett et al., 2013b) and five studies provided information on the number of days data were collected per week (Alexander et al., 2013; English et al., 2017; Goh & Bambara, 2013; Rausa et al., 2016; VanLaarhoven et al., 2007). A total of 34 different target behaviors were taught across the 12 studies, with the majority of skills related to office tasks, such as making photocopies, sending faxes, sorting mail, making labels, assembling packets, or shredding paper. Of the 34 behaviors, five were work related social skills—asking for assistance, listening, handling complaints, handling orders, and professional speech—across two of the studies (Dotto-Fojut et al., 2011; Rausa et al., 2016). Approximately seventy percent of studies measured the dependent variable by calculating the percentage of correct steps completed independently, 16% measured the percentage of correct trials, one measured the percentage of correct responses, and one study used partial interval recording. Most studies (83%) conducted a maintenance phase, whereas only 33% of studies reported some type of generalization data were collected. A total of 75% of studies measured implementation fidelity across conditions and reported levels of fidelity. Only one study (Goh & Bambara, 2013) listed information on staff training or technical assistance provided to support 36 implementation of the intervention. Approximately 60% of studies reported social validity data that were collected through interviews or questionnaires. Evidence based practice. In order to determine if there was a sufficient level of replication for any intervention, studies that demonstrated strong or moderate evidence of effect were evaluated based on the WWC ‘5-3-20’ rule for determining EBPs (Kratochwill et al., 2013). For video modeling, there were four studies, consisting of nine experiments, that demonstrated strong or moderate evidence of effect. These four studies included nine participants and were conducted by three independent research teams. For audio cueing, there were two studies, consisting of two experiments that demonstrated strong evidence. These two studies included six participants and were conducted by one research team. For textual cueing interventions, there was only one study, consisting of one experiment and four participants, that demonstrated strong evidence of effect. According to the WWC standards, none of the interventions would be considered an evidence-based practice for teaching vocational or work related social skills. Discussion The purpose of the current systematic review was to evaluate the methodological rigor of studies targeting vocational skills and work related social skills to adolescents and adults with ASD using the WWC SCED standards and determine if any of the interventions could be considered an EBP. Currently, there are an insufficient number of studies in any one intervention category that meet the SCED design standards to be considered an EBP. Results of the present review are consistent with previous research on adolescents and adults with ASD, which found that the majority of studies are found to be of low methodological rigor (Odom et al., 2015; Seaman & Cannella-Malone, 2016; Walsh et al., 2014; Walsh et al., 2017; Westbrook et al., 2012). 37 None of the interventions had enough evidence to be considered an EBP based on the WWC SCED standards. This is particularly concerning given that all of studies included in the review were conducted in 2005 or later, which is the same time that standards for EBPs were first proposed for SCED research in special education (Horner et al., 2005). Only 50% of the 52 experiments included in this review met design standards or met standards with reservations, with 50% of those experiments (n = 13) demonstrating strong or moderate evidence of a functional relation between the dependent and independent variable. Video modeling was the largest intervention category, with a total of 23 experiments that met standards or met standards with reservations. These 23 experiments included 21 participants and were conducted across 10 studies and eight independent research teams. However, 57% of these experiments demonstrated no evidence of effect, meaning there were not enough studies that demonstrated strong or moderate evidence of effect to meet the 5-3-20 criteria for determining an EBP (Kratochwill et al., 2013). Video modeling interventions. Video modeling is a well established EBP for teaching a range of skills to individuals with ASD of all ages, but more research is needed for video modeling to be considered an EBP for teaching vocational skills to adolescents and adults with ASD. One possible explanation for video modeling not meeting the EBP standard is that two video modeling studies, consisting of nine experiments, utilized alternating treatment to compare types of video models (Bennett et al., 2013; Cihak & Schrader, 2008). Although these studies met the design standards and demonstrated that video modeling was effective for teaching vocational skills, there was little to no difference between the two interventions, meaning they did not demonstrate evidence of a functional relation. Since relatively few studies have examined methods for teaching vocational skills to individuals with ASD, it may have been more 38 beneficial if effective interventions were first identified before comparing variations of the same intervention. In addition, it may be unnecessary to make minor distinctions between types of video models (i.e., video prompting, video self-modeling, video modeling with voice over narration) and reference each type of video model as its own intervention. At least four specific types of video models, including basic, point-of-view, video prompting, and video self-modeling (Plavnick, 2013) have already been identified and categorized as video modeling. By referencing each of these types of video models as its own intervention, it may become more difficult to identify effective interventions and for practitioners to make decisions regarding treatment approaches. Instead, researchers may consider referring to any intervention that uses video recording to provide a visual model of the targeted behavior as simply video modeling. Categorization of interventions. Differences in inclusion criteria and categorization of interventions have resulted in inconsistencies in potential EBPs for teaching vocational skills. Only one practice, behavioral skills training, has been identified as an EBP to teach employment related skills to adults with ASD (Anderson et al., 2016). However, in their review, only six of the 18 studies actually taught a specific vocational skill and vocational studies taught in school settings were excluded. Studies that taught “employment-related skills”, such as reading comprehension and fluency, internet skills, activities of daily living, and answering a mobile phone were included, meaning potentially irrelevant employment skills were taught. Other reviews have concluded that technology aided interventions are generally effective for teaching a range of skills to adolescents and adults with ASD (Odom et al., 2015; Seaman & Cannella- Malone, 2016; Walsh et al., 2017). Technology aided interventions utilize a broad range of devices and instructional approaches, such as video modeling, covert audio coaching, visual prompts, specific training, speech generating devices, performance feedback, or self- 39 management. Categorizing any intervention that utilizes technology as the main instructional instrument into one intervention strategy is problematic. Including such a wide range of procedures and devices makes it difficult for researchers or practitioners to select effective interventions. Researchers should instead consider categorizing interventions by the specific procedures utilized. Work related social skills. The majority of skills that were taught in studies that met standards with or without reservations were rather basic office related tasks. Although the explicit teaching of vocational skills to individuals with ASD is needed, additional emphasis should be placed on teaching work related social skills. Within the past couple of years, there has been an increased focus on teaching work related social skills to individuals with ASD, with three out of the four vocational social skills studies included in this review published in 2015 or later. However, less than 20% of experiments examining work related social skills met standards (Dotto-Fojut et al., 2011; Rausa et al., 2016). While it is encouraging that more studies are examining methods to teach work related social skills to individuals with ASD, it is important that these studies are methodologically sound given the importance of social skills for obtaining and maintaining employment (Hurlbutt & Chambers, 2004; Müller et al., 2003, Müller et al., 2008). The two work related social skills studies targeted social skills directly related to job tasks, such as asking for assistance or skills necessary for interacting with customers over the phone. Social skills related to job tasks have been identified by employers as most important to teach because they allow for completion of required work tasks and lead to greater success in the workplace (Agran et al., 2016). Teaching vocational skills and social skills in combination with one another may be one way to better equip individuals with ASD with the range of skills needed 40 to maintain employment. Since video modeling is the most commonly used intervention to teach vocational skills and has been shown to be effective for teaching a range of social skills (Wang, Cui, & Parrila, 2011; Rausa et al., 2016), it is plausible that video modeling may be an effective method for teaching work related social skills. Importance of teaching in employment settings and schools. Given the difficulties among individuals with ASD to generalize skills to new settings, it is important to teach and assess employment related skills in actual employment settings. Among the studies that met standards or met standards with reservations, only 33% assessed generalization of skills and only half of the studies were conducted in actual employment sites. Many of the interventions that were implemented in actual work environments were found to be socially acceptable, meaning job coaches and employers had no problem with the handheld devices as long as they were used for job training purposes (Kellums & Morningstar, 2012). This provides further support for the feasibility and likelihood that these interventions can be effectively implemented in employment settings, which is essential given the difficulties individuals with ASD have in generalizing acquired skills to settings and situations in which they should be applied (Hume, Loftin, & Lantz, 2009). Studies conducted in school settings are also important given the poor employment outcomes for adolescents with ASD upon leaving high school. Although few studies examined generalization of skills to actual work settings, studies conducted in school settings are necessary to prepare adolescents with ASD for the transition to adulthood and prepare them for future employment opportunities. Exposure to work experiences, involvement in work study programs, and vocational course work in high school have been identified as predictors of successful employment for individuals with ASD (Test et al., 2009). 41 Limitations. There are several limitations in the present investigation that should be considered. First, unlike previous literature reviews on this topic, the studies in this review were limited to those that implemented SCED research designs. This means that randomized controlled trials or quasi-experimental studies that implemented vocational or work related social skills interventions were automatically excluded. This was done to apply the WWC design standards to SCED studies and evaluate all studies in a consistent manner. Future research could include both SCED and group designs in order to synthesize findings once more vocational skills research emerges. Before comparing SCED research to randomized control trials for vocational skills and social skills, future research should first address some of the concerns regarding the methodological rigor of SCED studies. The most common design issues among experiments that did not meet standards was a lack of data points in each phase. Drawing inferences from SCED research relies heavily on the trend or slope of the data, meaning the more data points within a phase, the easier it can be to determine a pattern of responding (Kratochwill et al., 2013). When phases have less than three data points, it can be difficult to draw conclusions regarding the pattern of data. Future researchers could address this methodological flaw by collecting data across additional sessions, which can strengthen the design by allowing for more conclusive patterns of responding among participants. Another limitation was that we took a more conservative approach to applying the WWC standards than previous researchers by excluding SCED studies that taught vocational skills to individuals with ASD if the data could not be separated from participants who did not have ASD or from skills that were not vocational. This means studies that implemented a multiple baseline or multiple probe design across participants who did not have ASD or across behaviors that were not vocational skills were commonly excluded. Although potential experiments were excluded, 42 applying the most stringent criteria ensured included studies were evaluated as the standards were designed. In the current study, approximately half of studies utilizing multiple baseline or multiple probe designs failed to meet design standards based on the additional WWC criteria. Multiple baseline designs are often more favorable in educational settings since the intervention does not have to be withdrawn and multiple probe designs are useful due to the periodic, rather than continuous data collection (Gast & Ledford, 2014). However, the reduced number of data points required for multiple probe designs has the potential to reduce the strength of the effect, especially if there is high variability in the data. In addition, not including enough data points within each phase or not vertically aligning the data across cases has the ability to affect the methodological rigor of these studies and make it difficult to determine if a functional relation is present. Considering the methodological flaws for these designs found in the current study, it would be beneficial for researchers to be familiar with the additional criteria for the WWC for multiple baseline or probe designs to ensure they meet standards or meet standards with reservations. Third, only the Pro-Quest database was searched for articles. Studies published in other databases may have been missed. In addition, studies that investigated vocational social skills but didn't explicitly use the term social skills, may have been missed. Although only one database was searched, the references of relevant studies were searched to identify articles that may have been missed in the initial search. Given the consistencies among SCED studies identified in previous vocational skills literature reviews (Anderson et al., 2016; Bennett & Dukes, 2013; Lounds-Taylor et al., 2012; Nicholas et al., 2015; Seaman & Cannella-Malone, 2016; Walsh et al., 2014; Walsh et al., 2017; Westbrook et al., 2012), it is likely that this review included the majority of studies examining vocational interventions for adolescents and adults 43 with ASD. Conclusion. Overall, the current research base for teaching vocational skills and work related social skills to adolescents and adults with ASD is limited in terms of both scope and quality of published studies. None of the current practices for teaching vocational skills, including video modeling, audio cueing, and textual cueing, can be considered an EBP. Only four published studies taught work related social skills, the majority of which did not meet standards. Additional research is needed in order to address the underemployment rates of adults with ASD and equip them with the full range of skills necessary for successful employment. Future research should consider the WWC standards for SCED research when designing interventions studies to ensure the research design meets evidence standards. 44 CHAPTER 3 Pilot Study Challenges for Adolescents with ASD Although individuals with autism spectrum disorder (ASD) are typically diagnosed in childhood, deficits in social skills persist into adolescence and adulthood, where these individuals continue to be negatively impacted both socially and occupationally (Plimley. 2007; Rao et al., 2008; Roux et al., 2015). The majority of costs associated with care for individuals with ASD are in adult services as they continue to need some sort of support throughout their adult lives (Buescher et al., 2014). Estimates suggest 50,000 individuals with ASD turn 18 years old each year in the United States alone (Shattuck et al., 2012b) and unfortunately, employment opportunities are often limited for those with disabilities. Individuals with ASD who do work often fail to hold regular employment due to poor social skills or behavioral problems, rather than an inability to perform the actual job tasks (Elksnin & Elksnin, 2001; Hurlbutt & Chambers, 2004; Müller et al., 2003, Müller et al., 2008). Researchers have examined employment rates among individuals with ASD and have found that those with ASD are often underemployed or unemployed at higher rates than those with other disabilities (Roux et al., 2015; Shattuck et al., 2012b). Shattuck and colleagues (2012) examined the National Longitudinal Transition Study-2 (NLTS2) data, a national study designed to provide nationally representative information about youth with disabilities, and found that approximately 35% of young adults with ASD had never worked or attended postsecondary education upon leaving high school, suggesting current school-based services for transition aged youth may be inadequate to meet the needs of these individuals. Explicit instruction in work and related social skills for adolescents with ASD could improve employment outcomes for this 45 group in the early transition period. State of Vocational Skills Literature Despite the poor vocational outcomes for adolescents and adults with ASD, there is relatively little research on the most effective interventions to teach the skills necessary for adolescents with ASD to succeed in the workplace. Several recent literature reviews have attempted to evaluate and synthesize the vocational skills literature for adolescents and adults with ASD (Nicholas et al., 2015; Walsh et al., 2014; Walsh et al., 2017; Westbrook et al., 2012). Based on these recent reviews, there appears to be considerable variability in the quality of published studies. The ways in which these studies are evaluated makes it difficult to draw conclusions regarding the most effective vocational treatment approaches for adults with ASD. Studies that implemented an intervention that utilized some form of technology, such as video modeling or audio cueing, appeared to have the greatest impact on vocational skills (Walsh et al. 2014; Walsh et al., 2017). Video Modeling and Vocational Skills Video modeling is an evidence-based practice that has been used to teach a wide range of skills to individuals with ASD of all ages. Video modeling can be delivered in various formats, but typically involves a video presentation of a person engaging in the target behavior followed by an opportunity to demonstrate the desired skill (Bellini & Akullian, 2007). Video modeling has been used to teach complex skills to individuals with developmental disabilities by breaking tasks down into their component parts, showing a video clip of one step, and then providing participants with an opportunity to complete the step before the next step is shown (see Banda, Dogoe, & Matuszny, 2011). Some researchers refer to videos of longer tasks that are task analyzed and broken into smaller steps as video prompting (Sigafoos et al., 2007). Since the 46 main instructional component of the intervention is the video, regardless of how it is viewed, references will simply be made to video modeling rather than making the distinction between the video modeling and video prompting. Breaking the video into smaller parts, as opposed to showing the entire video, is useful for participants with moderate or severe developmental disabilities, who may have difficulty attending to lengthy video clips with a series of steps (Sigafoos et al.). Video modeling is a highly effective procedure for teaching skills because it provides a step by step guide of how to perform the skill and allows for consistency, repetition, and feedback since videos can be replayed. Several studies have investigated the effectiveness of video modeling to teach vocational skills to adolescents with ASD. Kellums and Morningstar (2012) taught four young adults with ASD to independently use an iPod to view video models and complete three vocational tasks within their respective jobs. Each video was divided into small clips based on the task analysis for the skill. Prior to each step being shown, a brief written description (with voice over for two of the participants) was provided, followed by the model reciting a verbal description of the task as he completed it. Unlike many video modeling interventions, no additional components, such as prompts, reinforcement, or error correction were used throughout the study. All participants demonstrated an increase in percentage of correctly completed steps for each vocational task upon implementation of the video modeling intervention and maintained their performance on the first two vocational skills 30 days following the intervention. In addition, participants, job coaches, and employers found the iPod to be a socially acceptable way to teach new skills within employment settings. Similarly, Bennett and colleagues (2013) utilized an alternating treatment design to compare video modeling with and without narration on the acquisition of vocational skills 47 among five adolescents with ASD. Participants were taught three clerical skills—making photocopies, sending a fax, or making labels—each consisting of 12 steps, in the teacher workroom of the high school they attended. Each skill was taught using either video modeling with voice-over narration, video modeling without narration, or without the use of video modeling (control condition). Although three of the participants preferred video modeling with voice-over narration, all participants acquired vocational skills through both comparison conditions, indicating the effectiveness of video modeling as a method to teach vocational skills to adolescents with ASD. Work Related Social Skills Given the benefits of video modeling to teach work skills to adolescents with ASD, it may also be possible to teach work related social skills using video modeling. Vocational social skills consist of any social skill related to obtaining or keeping a job, such as interacting appropriately with customers, co-workers, and supervisors; following directions; asking for help; reporting problems; or accepting criticism (Agran et al., 2016; Elksnin & Elksnin, 2011). Deficits in social skills play a large role in low employment rates among those with disabilities (Phillips, Kaseroff, Fleming, & Huck, 2014). Successful employment relies heavily on one’s ability to utilize a variety of social skills with co-workers, consumers, and supervisors. Of the existing research designed to teach work related social skills to individuals with ASD, several limitations exist. Many vocational social skills studies rely on self-reports to measure success of the interventions. Although many studies have found that participants with ASD self-reported lower levels of anxiety or increased peer-relations and empathy post- intervention, the interventions did not measure whether it led to an increase in observable social skills (Hillier et al., 2007; Hillier, Fish, Cloppert, & Beversdorf, 2011; Kaboski et al., 2015). 48 Vocational social skills studies for individuals with ASD have utilized pre- and post-designs to broadly assess changes in social skills or other related skills on rating scales following an intervention (Kaboski et al.). Interventions relying on indirect assessments should be combined with more objective measures, such as those that require direct observation and frequent measurement of specific social skills. Only a couple of studies have explicitly taught and measured work related social skills to individuals with ASD. In one study conducted by Dotto-Fojut and colleagues (2011), researchers directly taught four adolescents with ASD to ask for assistance while they completed vocational tasks. Participants were taught to ask for help when a problem arose using graduated guidance, scripts, and script fading. Researcher used problem scenarios and typical scenarios to teach participants to discriminate between situations where help was or was not needed. All participants were able to ask for assistance without the use of scripts and generalized the skill to a new setting and to problem scenarios that were not explicitly taught. In a more comprehensive approach to teaching social skills in vocational settings, Walsh and colleagues (2017) used The Walker social skills curriculum: the ACCESS program (adolescent curriculum for communication and effective social skills; Walker et al., 1988) and video modeling to teach seven adults with ASD and ID 30 social behaviors across three social domains (i.e., peer-related, adult-related, and self-related social skills). A multiple probe design across participants was used to evaluate the effectiveness of the 20-week intervention and participants demonstrated an increase in the measured social skills in work settings across domains and demonstrated a decrease in maladaptive behaviors. While results of these studies are encouraging, additional research is needed to examine potential procedures and dependent measures to teach work related social skills to fully prepare individuals with ASD to obtain and maintain employment. 49 Purpose of Pilot Study The purpose of the preliminary pilot investigation was to examine the effects of video modeling on the acquisition of vocational skills and social skills among adolescents with ASD. To date, video modeling has not been used to teach work related social skills and has not been used to teach two distinct skills within the same video. The current study is a preliminary investigation intended to examine potential methods to effectively and efficiently teach vocational and social skills to adolescents with ASD by combining the skills into one video and teaching them simultaneously. More specifically, the present study investigated the following research questions: (1) Does video modeling lead to the acquisition of vocational skills for adolescents with ASD when implemented in a vocational setting? (2) Does video modeling lead to the acquisition of social skills for adolescents with ASD when implemented in a vocational setting? Method Participants. Researchers obtained approval from the institutional review board and recruited two adolescents with ASD, who participated in the study. Participants were recruited through a health care organization that provided in-home and center-based applied behavior analysis (ABA) therapy to individuals with ASD from ages 0 to 21. Participants in the study met the following inclusion criteria: (a) prior diagnosis of ASD from a licensed psychologist or psychiatrist outside the context of this study; (b) at least 14 years old prior to the start of the study; (c) did not engage in self-injurious behavior; and (d) during a brief screening prior to the study, attended to a television screen for at least 20 s, followed one-step directions, and spoke in sentences comprised of at least three words. Each participant received approximately 20 hr of individualized, one-on-one ABA therapy delivered by a trained ABA technician every week. 50 The first author, a Board Certified Behavior Analyst (BCBA), supervised ABA services for approximately 2 hr per week by providing ongoing training, coaching, and feedback to technicians as well as monitoring progress on goals and writing or modifying treatment plans. Participants received ABA therapy in their home, community, or at the health care agency. Joseph was an 18-year-old White male who was diagnosed with ASD, an intellectual disability, and epileptic seizures. He attended a post-secondary education program within his local school district for adults aged 18 to 26 diagnosed with a developmental disability. Joseph’s education focused primarily on preparing adults for semi-independent living, with frequent community outings and basic vocational training, which typically included simply assembly tasks, such as putting together pizza boxes or cleaning related tasks at various local businesses. Joseph communicated in full sentences and could engage in reciprocal communication with peers and adults. Joseph had a tendency to perseverate on preferred topics at work during times when he was expected to complete assigned tasks or answer questions and had to be frequently redirected to the task. Joseph suffered from epileptic seizures, which were often triggered when he was excited or angry. On several occasions, video modeling sessions were cancelled or cut short due to seizures. He also occasionally exhibited maladaptive behavior, including property destruction or verbal aggression (e.g., name calling or refusal). Property destruction only occurred on two occasions during video modeling sessions and the technician followed the procedures in the participant’s behavior plan protocol. Anthony was a 16-year-old Black male diagnosed with ASD. He attended a center-based special education school for elementary to high school students with emotional or behavioral disorders. He received speech therapy, occupational therapy, and social work services in addition to his educational programming and also participated in Special Olympics track and 51 field. About half-way through the study, Anthony was moved from the special education school to a special education classroom within his home district due to a decrease in maladaptive behaviors. Anthony communicated in full sentences and frequently initiated interactions with others. He could engage in brief conversations, but struggled with reciprocal communication if the topics were not centered around preferred interests or items. For example, he could talk at length about his favorite rapper, but when asked about events at school, he would typically provide limited details. Anthony had no previous vocational skills training. Anthony exhibited several maladaptive behaviors, including physical aggression towards others, verbal aggression (e.g., swearing or name calling), and property destruction. These behaviors were exhibited on several occasions during video modeling sessions, at which point, the technician followed the procedures in the participant’s behavior plan protocol. Although these behaviors were present before deciding to include the participant in the study, they occurred infrequently. When Anthony switched schools, there was an increase in frequency and magnitude of maladaptive behaviors across settings. Facilitators. Each participant’s ABA technician was the main facilitator for the video modeling intervention. Joseph’s first technician left for an extended vacation after the ninth session. Starting in session 10, Joseph had a different technician implement the intervention. Both of Joseph’s technicians were males, had 2 years of experience in providing ABA therapy to individuals with ASD, were both enrolled in courses to obtain their BCBA, and were registered behavior technicians. Anthony’s technician was a male, had 1 month of experience providing ABA therapy, and had not previously worked with individuals with ASD. The first author, a fourth year doctoral candidate in special education and BCBA, supervised implementation of the intervention. She was responsible for overseeing all aspects of the intervention including 52 training each ABA technician (see below), collecting procedural integrity and inter-observer agreement (IOA) data, developing scripts for target behaviors, creating video models, and monitoring participant progress on targeted behaviors. Technicians were trained to implement the intervention with at least 90% fidelity and collect data on targeted behaviors until reaching 90% agreement levels with the first author. Anthony’s, as well as Joseph’s first technician, were taught to implement the intervention and collect data during an initial 3-hour training using a behavioral skills training approach (e.g., Sarokoff & Sturmey, 2004; Sarokoff & Sturmey, 2008; Ward-Horner & Sturmey, 2012). The training included instruction, prompting, role-playing, and specific feedback on implementation accuracy based on a procedural integrity checklist until technicians reached a criterion of at least 90% accuracy during role play opportunities. Joseph’s second technician was trained to implement the intervention and collect data after observing the first author implement the intervention and was provided with in the moment coaching and feedback based on the procedural integrity checklist until he reached at least 90% accuracy of implementation, at which point he began independently implementing the intervention. Setting and materials. The study took place at the satellite office of the healthcare agency where participants received ABA services. The office had a large open work space, which was primarily used for training purposes, and was surrounded by three large shared offices, a conference room, and a kitchen. The open work space had three rows of tables facing a computer screen and whiteboard and had a lounge area with couches and chairs towards the back. Approximately 20 typically developing adults worked at the office each day, with up to 40 adults on training days. The concession stand was constructed in a corner of the open space by using two long tables, a mini-refrigerator, and storage bins for extra supplies. 53 A variety of instructional materials were used throughout the study. An Apple iPad 2 was used to display video modeling clips to participants and a Sony Camcorder (model-HDR XR 160) was occasionally used to record sessions. The facilitators used paper and pencil to collect data. Materials necessary to run the concession stand were also used throughout the study including a mini-refrigerator, food items (e.g., chips, granola bars, candy, gum, etc.), and preparation supplies (e.g., napkins, silverware, plates, measuring cups, etc.). Extra supplies were kept in storage bins with picture labels to indicate which items were in each drawer. Supplies to manage payments were also used in the study and included a register set up as a point of sale (POS) using the Square register application, Square stand, Apple iPad Air 2, and a cash box. When the Square register application was open, a picture menu of the concession stand items appeared. Participants could simply click on the picture of the item(s) that were ordered, click charge, and then select the cash received or swipe a credit card using the Square stand. Measurement of dependent variables. Three vocational and social skills were measured for Joseph and two vocational and social skills were measured for Anthony. Each vocational skill was paired with a complementary social skill based on the logic that they frequently occurred together (e.g., maintaining small talk was a social skill one needed when performing the vocational task of working the cash register). Vocational behaviors included: cleaning surfaces, restocking supplies, and working the cash register. Each vocational skill was task analyzed (see Table 3) and an operational definition is provided below. Each individual subtask was assessed for accuracy and scored as correct, incorrect, or not applicable. Any task-analyzed step that was completed correctly, regardless of the order, was scored as correct since order was not essential to the correct final outcome for each skill. The total correct responses on the task analysis during 54 a given session were divided by the total number of steps in the task analysis (minus any steps that were scored as n/a) and multiplied by 100 to obtain a percentage of correct steps performed for each session. Social behaviors included: accepting a compliment, accepting criticism, and maintaining small talk. These social behaviors were chosen after a review of the literature on vocational skills for individuals with disabilities, which identified these social skills as behaviors that were commonly taught or assessed in vocational settings (see Elksnin & Elksnin, 2011). For social behaviors, observers recorded the occurrence or nonoccurrence of targeted social skills following a programmed antecedent (see below). The total correct responses for the social skill were divided by the total response opportunities for that skill and multiplied by 100 to obtain a percentage. The targeted behaviors are defined in the pairings below. Table 3. Task Analysis for Vocational Target Behaviors Cleaning surfaces Restocking supplies Working the cash register 1. Gathers supplies; multi- 1. Checks supply levels for 1. Waits 5-10 s and surface spray cleaner and dish rag or paper towel 2. Removes all portable items from the counter area and throws away trash, if any 3. Sprays cleaner on countertop 4. Wipes clean with dish rag or paper towel 5. Checks counter for any missed spots or excess spray and wipes again if necessary 6. Shakes out rag over trash or throws paper towel in the trash item 2. Gathers additional items from storage for the item that is below 50% or below the fill 3. Does not gather supplies for the item that is above 50% or above the fill line 4. Items are neatly placed in containers (e.g., organized in same direction, stacked on top of one another or next to one another with nothing sticking out) to fill line or no more will fit 7. Returns portable items to 5. Returns extra items to original location on surface storage, if any 8. Returns cleaning items to 6. Draws a checkmark on then asks customer if he or she is ready to order 2. Touches icons for items ordered on iPad 3. Correctly repeats customer order 4. Tells customer price of order 5. Takes card or cash 6. Completes payment 7. Hands correct change or card back to customer storage supply list to indicate which item was restocked 55 Vocational and social skill pairing 1. The first pairing of vocational and social skills was cleaning surfaces and accepting a compliment. These behaviors were paired together because it was likely a technician or customer may compliment the participant based on his work performance. Similar to the skill described by White et al. (2011), cleaning surfaces had eight steps and included clearing off and wiping down tables (see Table 3). If there were no portable items to return to the original location, the skill was based on seven steps. The programmed antecedent for this behavior was whenever an adult told the participant to clean a specific work area. Accepting a compliment was defined as: when another person compliments the participant, he (a) thanks the person for the compliment and (b) makes a comment about the content of the compliment (Whang et al., 1984). For example, if a technician said to the participant, “You are very professionally dressed today!” The participant may say, “Thank you. I bought this outfit over the weekend.” The purpose of requiring participants to comment on the compliment, rather than simply thanking the person, was to prevent single word answers since the complete response was more likely to facilitate additional social interaction. Vocational and social skills pairing 2. Pairing two included restocking supplies and accepting criticism. These behaviors were paired together because errors can occur when restocking supplies and if an error did occur, a technician was likely to deliver corrective feedback or criticism. There were up to six steps required to restock items, which centered around refilling supplies, such as napkins, cups, plates, and food or drink items. If participants did not have extra supplies to return to storage, restocking supplies was based on five steps. Similar to White et al. (2011), restocking supplies was scored as correct if the participant added additional items until they reached the fill line and the participant did not add additional items when the items were at or above the fill line. The technician always told the participant to 56 restock two different supply items at a time, one that needed to be restocked and the other did not need to be restocked, which was also the programmed antecedent for this behavior. Table 3 provides the task list for restocking supplies. Accepting criticism was defined as: when another person criticizes the participant, he (a) acknowledges the criticism, apologizes for what he did incorrectly, or states he agrees with the problem, and (b) requests or provides a solution to prevent the problem from occurring in the future. For example, if a technician said to the participant, “You added too many cups and they spilled everywhere.” The participant may say, “I’m sorry for making a mess. Next time I will not add as many cups.” Vocational and social skill pairing 3. Pairing three included working the cash register and maintaining small talk. These skills were taught together because working the cash register involved interacting with customers and engaging in small talk is often seen as an important aspect of customer service. Working the cash register involved seven steps and included taking customer orders and managing payment. If the customer happened to approach the counter and immediately order, the skill was only based on six steps. The antecedent for this vocational task was an adult standing at the counter to place an order. Table 3 outlines all of the steps involved in working the cash register. Plavnick and colleagues (2013) operationally defined maintaining a conversation as responding to a comment or question from a peer by asking a follow-up question or emitting an open-ended comment. Similarly, small talk in the present study was defined as: when another person asks the participant a question or makes an open-ended comment, the participant (a) answers the question or acknowledges the statement and (b) asks the person an on-topic question or makes an on-topic comment that invites a response from the person (Plavnick et al.). For example, before ordering, a customer may say, “I can’t believe it’s already 4:00 p.m.” The participant may respond with, “Time has been going by quickly. What time do 57 you work until?” Inter-observer agreement (IOA). The primary observers and data collectors throughout the study were the ABA technicians, who met a criterion of 90% accuracy for data collection reliability with the first author. Technicians were trained to collect data on targeted behaviors by watching recorded examples and nonexamples of targeted behavior, performed by adult models who followed scripted scenarios and were created solely for training purposes; independently recording the occurrence or nonoccurrence of the behavior; discussing with the researcher how they scored the behavior; and resolving any discrepancies. IOA between facilitators and the researcher via live sessions was conducted for 64% and 68% of the sessions for Joseph and Anthony, respectively. The first author was the second observer and simultaneously though independently recorded dependent measures across randomly selected sessions that were evenly distributed across conditions, behaviors, and participants. A point-by-point agreement calculation (Cooper, Heron, & Heward, 2007) for each behavior was used to derive a percentage of agreement. For vocational skills, each operational step was scored as either an agreement or disagreement between the researcher and technician. For social behaviors, agreements or disagreements were scored for each trial. The percentage of agreements was obtained by dividing the number of agreements by the total number of agreements plus disagreements multiplied by 100. Mean IOA for each participant, behavior, and condition is summarized in Table 4. 58 Table 4. Inter-observer Agreement (mean and range) Across Participants, Conditions, and Behaviors Clean Surfaces Restocking Supplies Joseph Baseline Intervention 96% (88-100%) 96% (96-100%) Anthony 96% (88-100%) 96% (88-100%) Accept Compliment Joseph Anthony Baseline 92% (75-100%) 94% (75-100%) Intervention 100% 97% (75-100%) Baseline Intervention 95% 92-100%) 98% (92- 100%) 85% (83-100%) 92% (83-100%) Accept Criticism Baseline 100% 100% Intervention 98% (80-100%) 93% (67-100%) Cash Register Baseline Intervention 100% 95% (95-100%) n/a n/a Small Talk Baseline 100% n/a Intervention 100% n/a Experimental design. A multiple probe design with probe conditions (Gast & Ledford, 2014) across behaviors and replicated across participants was used to evaluate the effectiveness of video modeling for teaching the targeted vocational and social skills. In the multiple probe design with probe conditions, all behaviors are probed prior to implementation of the intervention and then again each time a targeted behavior reaches criterion. Probing all behaviors in this manner allows researchers to examine the extent to which any previously mastered behaviors occur without first displaying the video models and assesses whether the remaining untreated behaviors occur at levels similar to baseline. All social and vocational behaviors were initially probed under baseline conditions for all participants. Following baseline probes, video modeling was applied to the first pair of targeted behaviors for each participant until participants performed the vocational skill with at least 80% accuracy on each trial across two consecutive sessions. The same criterion was applied to all pairs of targeted behaviors. Mastery criteria was based on performance on vocational skills, rather than social skills to prevent participants from staying in the intervention condition for vocational skills longer than necessary. To limit the participants’ ability to learn skills simply by 59 observing one another, one participant was first taught to clean surfaces while the other was taught to restock supplies. Once participants met mastery criteria during the video modeling condition, all behaviors were probed under baseline conditions. A maintenance condition was included for some behaviors after the post video modeling probe. Procedures Creation of video clips. Video clips for each behavior pairing were created and streamed to an Apple iPad for use during video modeling sessions. Videos were filmed at the healthcare agency where the concession stand was set up. Three to five variations of each behavior pairing were made to promote stimulus generalization of the skill through the use of multiple exemplars across models, social partners, materials, and vocal statements (Plavnick et al., 2013; Stokes & Baer, 1977). Videos were 39 s to 1 min 38 s in length and included male and female adult models performing the components of the vocational and social skills. Although each skill had a similar number of steps, some steps took longer to complete (e.g., making change took longer than swiping a credit card or restocking popcorn kernels took longer than restocking bags of chips). Adult models were employees who worked for the health care agency, but were not necessarily directly involved in the study. Each video was broken down into chapters that corresponded with each individual subtask on the task analysis so that each step could be played individually during error correction procedures (described below). Scripts for each behavior pairing were created and models were instructed to follow the script during creation of the videos. For each behavior pairing, the social behavior was embedded within the vocational task; therefore, one video was made to depict both skills. For example, during a video for cleaning surfaces, an adult was shown wiping down a table. When the adult finished cleaning the table, a second adult walked over and told the person the table looked very clean. 60 The first adult smiled and said, “Thank you. I wanted it to look nice.” Probe sessions. The first three sessions of the current study functioned as a baseline to probe levels of targeted behaviors for each participant prior to implementing the intervention. Later probes functioned as post-training probes or additional baseline, depending on the behavior. It took approximately 2 hrs to conduct probe sessions, which included setting out concession stand materials, putting them away at the beginning and end of the session, and manipulating materials in order to contrive trials for each of the vocational behaviors (e.g., ensuring one item was full and one was low for restocking trials). Trials were conducted in a 1 on 1 manner; however, employees of the company were often present, either walking through the open space to access the kitchen or working at one of the tables, but they did not interfere with implementation of the intervention. During probe sessions, each participant’s ABA technician contrived two opportunities for the participant to perform each vocational task, for a total of four or six vocational trials for Anthony and Joseph, respectively. Trials for working the cash register were initiated whenever the first author approached the counter and ordered an item from the menu. The first author served as the consumer to emulate a real customer placing an order while ensuring accurate delivery of the discriminative stimulus (SD) and allowing the technician to collect data on the targeted behavior. Trials for cleaning surfaces and restocking supplies were initiated when a technician told the participant to clean the work area or restock supplies, but did not include information on how to perform any of the tasks. Participants were allowed to complete the job task until 15 s elapsed without a correct step performed or he verbally indicated he was done with the task. If the participant asked a question related to how to perform the job task, the technician made a general statement to the participant to continue with the job (e.g., “Just try your best”). Once a participant received two opportunities to perform one of the 61 vocational tasks, he was instructed to take a break while the other participant received two trials of a vocational skill. The participant who was on break sat at one of the tables or in the lounge area with his technician and played a game or used a tablet. Social skills were probed during trials for vocational skills and each participant received a total of two or three opportunities to perform each social skill during probe sessions. Joseph received one or two social skills probes during each vocational trial (see Table 5). For example, on the first trial for cleaning surfaces, he was given a compliment. On the second trial for cleaning surfaces, his ability to accept criticism and maintain small talk were probed. Anthony received one social skills probe during each vocational trial. Once a participant received two opportunities to perform a vocational task and one opportunity to perform each of the social skills, the participant was then given two opportunities to perform the next vocational task until all vocational and social skills were probed in combination with one another. Social skills were probed across vocational tasks, as opposed to just with the paired vocational skill, because the end goal was to assess whether participants could perform all social skills across all job tasks. Technicians followed scripted protocols that ensured the correct pairings occurred and the correct SD was delivered. No feedback, prompts, or error correction were provided to participants during probe sessions. 62 Table 5. Example of Social Behaviors Probed Across Vocational Tasks During Probe Sessions Trial Vocational skill Cleaning surfaces Trial 1 Maintain small talk Not probed Trial 2 Cleaning surfaces Trial 3 Restocking supplies Technician says, “Wow it has been busy today. How are you feeling?” Technician says, “Do you have any plans after work?” Trial 4 Restocking supplies Not probed Trial 5 Working the cash register Trial 6 Working the cash register Customer says, “Did you do anything fun over the weekend?” Not probed Accept criticism Technician says to participant, “The seating area is not very clean. There are used napkins everywhere.” Not probed Accept a compliment Technician says, “You are very professionally dressed today.” Not probed Not probed Technician says, “The supplies for customers should be full at all times.” Technician says, “You need to work quickly when using the cash register.” Not probed Technician says, “I appreciate how hard you are working today!” Not probed Not probed Technician says, “This table looks really clean!” Video modeling. After the initial baseline probe sessions, participants were taught the vocational and social skill pairings using video modeling. Intervention sessions took place 1 to 3 days a week and each session lasted approximately 2 hrs. Due to space constraints, only one participant could perform a vocational skill at a time. When participants were not working, they took a break by playing a game or using a tablet or worked on other programs that were a part of their ABA therapy, but were unrelated to the current study. A total of three to five trials of the vocational skill were conducted during each session. To initiate a trial, technicians contrived 63 opportunities for participants to practice each skill by engaging in the programmed antecedent behavior described in baseline probe sessions. The technician then told the participant to “watch the video” and then showed him the video segment associated with the step on the task analysis then immediately provided the participant with an opportunity to perform the step. Once the step was completed, the technician said, “watch this” or “look” and showed the participant the next video segment, which depicted the next step on the task analysis. If the participant made an error on any step in the task analysis, the technician scored the step as incorrect, interrupted the error, and re-played the video segment that demonstrated the specific step. The participant was immediately given another opportunity to complete the step. If after re-watching the video segment the participant still performed the step incorrectly, the participant was told what to do differently next time and asked to continue with the task. Technicians also created one opportunity for participants to demonstrate the paired social skill by engaging in the antecedent behavior during every vocational skills trial. The social skill was embedded into the video for vocational skills and occurred either at the beginning or end of the video. After the segment of the video that depicted the social skill was viewed, the technician paused the video and engaged in the same antecedent behavior for the social skill that was shown in the video. The participant was then expected to respond in a manner that aligned with the target behavior definition. Error correction procedures for social behaviors were conducted in a similar manner as for vocational tasks. If an error was made on a social skill, the technician said, “nice try,” told the participant to “watch the video,” and showed him the segment of the video that depicted the social skill. The technician then gave the same SD and the participant received another opportunity to demonstrate the social skill. If the participant performed the behavior correctly, the technician continued the video or had the participant take a 64 break if the social skill appeared at the end of the video. If the participant performed the behavior incorrectly, the technician told the participant how to respond differently next time. Video modeling was conducted until participants performed the vocational skill with at least 80% accuracy on each trial across two consecutive sessions. To prevent participants from staying in the intervention for the vocational skill for longer than necessary, participants were not required to master the paired social skill before moving on to the next vocational skill. The intervention was designed to teach both participants all three vocational skills to mastery; however, the intervention was terminated early for both participants. Joseph experienced an increase in frequency of epileptic seizures, which resulted in several injuries and extended hospitalizations. For the safety of the client and at the recommendation of his doctor, all outside services and therapies took place in the client’s home, and as a result, the intervention was discontinued. Anthony engaged in two instances of severe aggression towards the technician and property destruction, resulting in several thousand dollars in damages to the office building. Due to the severity of the property damage and for the safety of the employees of the company, the intervention was discontinued for Anthony. Video fading. Technicians administered one video fading probe for vocational and social skills at least once per week once participants met mastery criterion for the previously mastered skill. For instance, if a participant had previously mastered cleaning surfaces and was learning to restock supplies, he was asked to clean a counter for one trial, in addition to the trials for the current targeted skills. The technician then performed the antecedent behavior for the originally paired social skill during the maintenance trial for the vocational skill. If the participant did not perform the previously mastered vocational task with at least 80% accuracy or performed the social skill incorrectly, the participant was shown a video depicting both skills and 65 was given another opportunity to perform the vocational and social skill. Procedural integrity. The first author created a comprehensive checklist with 56 operationally defined items to assess the extent to which all components of the intervention were implemented correctly or incorrectly on a trial by trial basis. A separate procedural integrity checklist with 24 operationally defined items was used to assess fidelity during probe sessions. Procedural integrity data was collected for 55% of probe and intervention sessions by the first author to assess fidelity of implementation. The first author provided general performance feedback as well as specific feedback on how to contrive role-play scenarios for each of the behaviors on a weekly basis to technicians. If an item on the checklist was missed, performance feedback was given each time procedural integrity was assessed. The mean level of procedural integrity for Joseph’s technicians was 95% (range 92-100%) and 91% (range 82-100%) for baseline and intervention conditions, respectively. The mean level of procedural integrity for Anthony’s technician was 96% (range 96-100%) and 92% (range 86-97%) for baseline and intervention conditions, respectively. Results Results of video modeling on the acquisition of vocational skills for Joseph are displayed in Figure 1. Joseph demonstrated zero or near zero levels of responding on all vocational skills during initial baseline probe sessions, with an immediate increase in performance in targeted skills when the intervention was implemented. Responding for restocking supplies increased to a mean of 84% (range, 69 to 100%) when video modeling was applied. During the first post- training probe, Joseph restocked supplies with 78% accuracy and responding for the remaining untrained behaviors remained near initial baseline probe levels. When the second post-training probe was conducted, performance on restocking supplies dropped slightly (50%). During 66 maintenance sessions, Joseph restocked supplies with a mean of 93% (range, 80-100%) and no videos were shown during maintenance. Joseph demonstrated a rapid increase in cleaning surfaces once video modeling was administered, with a mean of 86% (range, 75 to 98%). During the post-training probe, Joseph’s responding remained high for cleaning surfaces (80%) and the untaught behavior remained low. Joseph cleaned surfaces with 88% accuracy during maintenance probes and no videos were shown. During baseline probes for working the cash register, Joseph performance remained at 0%. When the intervention was applied to working the cash register, mean responding was 72% (range, 50 to 91%) for the five sessions. Maintenance probes were not conducted for working the cash register. Results of video modeling on the acquisition of social skills for Joseph are displayed in Figure 1. Joseph did not demonstrate accepting criticism during initial baseline probes. When video modeling was applied to accepting criticism, responding varied with an overall mean of 40% (range, 0 to 80%). During the two post-training probes, Joseph accepted criticism with 67% and 33% accuracy. During maintenance sessions, Joseph accepted criticism with 100% accuracy. During baseline probes for accepting a compliment, Joseph demonstrated zero or near zero levels of the behavior. When video modeling was applied to accepting a compliment, responding increased to a mean of 83% (range, 67 to 100%). During the third probe condition, Joseph accepted a compliment 67% of the time. During maintenance for accepting a compliment, Joseph performed the skill with 100% accuracy. Joseph maintained small talk on one occasion during baseline probes. Responding increased to a mean of 82% (range, 50 to 100%) when video modeling was applied to maintaining small talk. No maintenance sessions were conducted for this behavior. Results of video modeling on the acquisition of the vocational skills of cleaning surfaces 67 and restocking supplies for Anthony are displayed in Figure 2. During initial baseline probes for cleaning, Anthony performed the skill with 0% accuracy. Anthony demonstrated a large magnitude of change and an increasing trend when video modeling was applied to cleaning surfaces with mean responding of 87% (range, 76 to 100%). During the first post-training probe, Anthony performed cleaning surfaces with a mean of 79% (range, 69 to 88%). During maintenance, Anthony cleaned surfaces with a mean of 86% (range, 69-100%), with one video shown during the first six sessions and no videos shown during the last two maintenance sessions. Anthony cleaned surfaces with 88% accuracy during the final post-training probe. Anthony demonstrated zero levels of responding during initial baseline probes for restocking. When video modeling was applied to restocking supplies, mean responding immediately increased to 82% (range, 53 to 100%). During the post-training probe, Anthony restocked supplies with 67% accuracy. Results of video modeling on the acquisition of social skills for Anthony are displayed in Figure 2. Anthony demonstrated zero levels of responding for accepting a compliment during initial baseline probes. When video modeling was applied to accepting a compliment, responding increased to a mean of 39% (range, 0 to 100%). During the first post-training probe for accepting a compliment, Anthony demonstrated the skill with 0% accuracy. During maintenance probes, Anthony accepted a compliment with a mean of 89%. In the final post- training probe, Anthony accepted a compliment with 100% accuracy. During baseline probes for accepting criticism, Anthony demonstrated the skill with 0% accuracy. Video modeling was then applied to accepting criticism and mean responding increased to 49% (range, 0 to 100%). During the post-training probe, Anthony accepted criticism with 50% accuracy. 68 Figure 1. Percentage of Trials Joseph Performed Vocational and Social Skills VM BL VM BL VM BL y l t n e d n e p e d n I d e t e l p m o C s p e t S f o e g a t n e c r e P l l a r e v O 100 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 Restocking supplies and accepting criticism 1 3 5 7 9 11 13 15 17 19 21 23 25 Cleaning surfaces and accepting a compliment 1 3 5 7 9 11 13 15 17 19 21 23 25 1 3 5 7 9 11 13 15 17 19 21 23 25 Sessions Cash register and small talk Percentage of steps completed independently across baseline (BL) and video modeling (VM) conditions. Closed circles represent vocational skills. Closed triangles represent social skills. Open circles and triangles represent maintenance probes. 69 Figure 2. Percentage of Trials Anthony Performed Vocational and Social Skills VM BL BL VM BL 100 80 60 40 20 0 100 80 60 40 20 0 y l t n e d n e p e d n I d e t e l p m o C s p e t S f o e g a t n e c r e P Cleaning surfaces and accepting a compliment 1 3 5 7 9 11 13 15 17 19 21 1 3 5 7 9 11 13 15 17 19 21 Sessions Restocking supplies and accepting criticism Percentage of steps completed independently across baseline (BL) and video modeling (VM) conditions. Closed circles represent vocational skills. Closed triangles represent social skills. Open circles and triangles represent maintenance probes. Arrow depicts when no videos were shown during video fading sessions. Discussion Results of the study indicate video modeling was associated with an increase in vocational and social skills among participants with ASD, suggesting video modeling may be an effective intervention for teaching various work related skills to adolescents with ASD in vocational settings. Both participants demonstrated an increase in performance when video modeling was applied to vocational skills. Participants also demonstrated an increase in social skills when taught within the same video, although these skills were acquired at a slower and more varied rate, possibly suggesting the need for additional teaching of vocational social skills. Although data are limited, results are consistent with previous video modeling interventions for 70 teaching vocational skills to adolescents and adults with ASD (Bereznak, Ayers, Mechling, & Alexander, 2012; Bennett et al., 2013; Burke et al., 2013; Kellums & Morningstar, 2012). Acquisition of vocational skills. Previous research has demonstrated that video modeling often leads to a rapid rate of acquisition of targeted behaviors, with future targeted behaviors often acquired faster than the first (Charlop-Christy, Le, & Freeman, 2000; Charlop- Christy & Daneshvar, 2003). Interestingly, in the current study cleaning surfaces and restocking supplies were acquired at the same rate for both participants despite these skills being taught in a different order. Both participants reached mastery criteria within five sessions for cleaning and 10 sessions for restocking supplies. It is possible the difference in complexity between restocking supplies and cleaning may have contributed to the difference in the rate of acquisition. Although restocking supplies had fewer steps in the task analysis, it required participants to make a discrimination between two items, one that was full and one that was not. Cleaning surfaces was more straightforward and each step could be easily depicted in the video. Tasks or steps in a task analysis that require greater attention to detail may benefit from additional teaching methods. Adding in voice over narration or embedding written instructions into video models may be ways to place added emphasis on certain aspects of targeted behaviors (Bennett et al., 2013; Kellums & Morningstar, 2012). Although voice over narration or written instructions combined with video modeling makes it difficult to determine which components are essential for skill acquisition and which are unnecessary, they may reduce the need for adult support and prompting. Using methods that potentially ease the task of fading adult-mediated support is an important aspect to consider when teaching skills in employment settings where independence is often expected. 71 Acquisition of social skills. Both participants demonstrated overall lower levels of performance among targeted social skills when compared to vocational skills. For Joseph, accepting criticism followed a similar response pattern as restocking supplies throughout video modeling sessions; however, overall levels for accepting criticism were much lower than the vocational skill. Joseph never accepted criticism with 100% accuracy during a video modeling session, but was able to perform the skill during maintenance sessions. Joseph was able to accept a compliment and maintain small talk with 100% accuracy for the first two sessions and then performance on both skills was more variable, possibly suggesting issues with motivation. Since no additional reinforcement was provided for correct responses, it is possible social interactions were not reinforcing for Joseph and were not enough to maintain the skill. Anthony demonstrated a different response pattern, with zero levels of responding during the first two sessions for both social skills with a gradual increase in performance before reaching 100% during the final session. There are a couple of possible explanations for the unique response patterns of the acquisition of social skills among participants. Given that individuals with ASD often require higher levels of support and greater intensity of interventions to master job skills and social skills (Wehman et al., 2016), participants would have likely benefited from more frequent intervention sessions as well as additional practice opportunities in order to master the social skills. Previous research on teaching work related social skills to adolescents with ASD demonstrated greater success in the acquisition of these skills. In Walsh and colleagues (2017), researchers used a comprehensive curriculum focused solely on teaching social skills in a work place setting. Dotto-Fojut and colleagues (2011) taught participants to independently complete all 12 vocational scenarios before teaching the social skill. In the current study, participants were 72 taught both skills simultaneously, with the SD for social skills embedded into trials for vocational skills. Participants were also taught the vocational skills to mastery, rather than the social skills to prevent participants from continuing in an intervention phase after mastering vocational skills for an extended period of time. Since participants reached mastery criteria of vocational skills within 10 sessions or fewer, the treatment intensity may not have been sufficient for participants to also acquire the social skills. Increasing treatment intensity or teaching vocational skills to mastery before teaching the social skills may have resulted in higher levels of performance among targeted social skills. Extension of the vocational skills literature. The study also extends the literature on teaching work related social skills to adolescents with ASD. To date, limited research has focused on methods to increase social skills in the workplace among individuals with disabilities despite the wealth of research that highlights the integral role of social skills for maintaining employment (Agran et al., 2016; Phillips et al., 2014). When video modeling was applied to social skills, both participants demonstrated an increase in responding, although there was not an immediate increase in performance as there was with vocational skills. The difference in responding between the vocational and social skills is not surprising given that social skills are often more complex and situations are more variable than with vocational skills, which tend to be more straightforward. Results of the present investigation are also consistent with research that indicates the most common reason individuals with ASD are fired from their job is because they don't have the social skills for the job, not because they cannot perform the job tasks (Elksnin & Elksnin, 2011; Hurlbutt & Chambers, 2004; Müller et al., 2003; Müller et al., 2008). Combining vocational skills and social skills in one video and teaching them simultaneously may be a more efficient method for helping participants acquire social skills in 73 the workplace. Often times, vocational rehabilitation agencies focus primarily on obtaining employment for individuals with disabilities and providing on the job training of vocational skills, rather than teaching social skills (Chen et al., 2015). Although securing employment is the primary purpose of vocational rehabilitation agencies, difficulties with social skills is a characteristic of individuals with ASD who, due to this difficulty, often need additional support or explicit instruction in social skills. When individuals with disabilities lack the social skills necessary for a particular job, the focus of intervention is often on changing the environment, such as a new job placement, rather than addressing potential social skills deficits (Phillips et al., 2014). Simply changing job placements is problematic for individuals with ASD when the social deficits have not been directly addressed as these difficulties are likely to continue in the next placement. Given that participants will need some type of on the job training to acquire the skills to perform the job, embedding the social skill directly into the job task may be one way to provide social skills training to individuals with disabilities. Policies and procedures are in place to support the development of job related skills among individuals with disabilities in job settings (Chen et al., 2015), but nothing requires the development of social skills, despite the research that indicates the need for it. Teaching vocational and social skills together, may be one way to address this concern. Limitations. There are several limitations within the present investigation that warrant further consideration. The first limitation was ending the study early for both participants due to maladaptive behavior or seizures. Although Joseph demonstrated evidence of effect across three pairs of behaviors, Anthony only received the intervention across two pairs of behaviors. Ending the study prematurely limited the ability to draw conclusions regarding the functional relation between the intervention and targeted behaviors. Additional replications of the effect of the 74 intervention across one or more behaviors would strengthen the results of the study. Future studies should follow the recommendations as outlined by Gast and Ledford (2014), which states that a multiple probe design be conducted across at least three tiers (i.e., behaviors) in order to demonstrate experimental control. A second limitation was the presence of maladaptive behaviors, which occasionally interfered with administration of the intervention and ultimately led to the study being discontinued for Anthony. It is possible both participants may have benefited from targeted therapy that focused on behaviors that hindered quality of life, such as coping and tolerance skills, self-management, or the reduction of maladaptive behaviors, before targeting vocational skills within a community setting. However, employment is known to play a role in quality of life in adults with ASD and even reduce maladaptive behaviors in individuals with ASD (Taylor, Smith, & Mailick, 2014). Despite the presence of maladaptive behaviors, participants still demonstrated mastery of vocational skills and showed improvements in social skills. Given that the majority of vocational interventions for adolescents with ASD tend to include higher functioning individuals who exhibit zero or low levels of maladaptive behavior (Walsh et al., 2017), researchers should consider how to address maladaptive behaviors in future vocational skills studies. Researchers may also consider investigating job opportunities based on participant strengths and interests, having them complete a job-related preference assessment, or allowing participants to sample various jobs before simply placing them into a job. A third limitation was variable performance across social skills. The current study was designed to be implemented 3 days a week, but was only implemented 1 or 2 days due to a variety of issues, such as scheduling, maladaptive behavior and seizures, and location of services. Despite the inconsistent performance, improvements in social skills were observed 75 from baseline to intervention. More consistent implementation of the intervention may have allowed for additional replications and better demonstration of effect for social skills. Future research should continue to examine efficient methods to teach social skills to adolescents with ASD that can be utilized in the workplace. Social skills research for vocational skills should also examine the benefits of utilizing a mastery model for teaching social skills, increasing the intensity of the intervention, or possibly teaching vocational and social skills separately to mastery and then combining them. Conclusion. The outcomes of the present study support previous research on the use of video modeling to teach vocational skills to adolescents with ASD. Video modeling also led to an increase in work related social skill for both participants. Vocational skills and social skills were taught in conjunction with one another, meaning one video was used to depict both skills. This contributes to the current vocational skills literature by demonstrating a potentially efficient manner for teaching skills necessary for the workplace. Overall, the present study presents preliminary data that suggests video modeling may be a feasible approach for teaching work related skills to adolescents with ASD. 76 CHAPTER 4 Research Study Transition Planning for Adolescents with ASD The purpose of special education services is to prepare individuals with disabilities for life after high school, including further education, employment, and independent living (Individuals with Disabilities Education Act, 2004). Transition planning is a formal process for identifying post-secondary goals and is an integral part of adolescents with disabilities individualized education plan (IEP), beginning no later than age 16. Transition planning takes into account each student’s needs, strengths, and preferences and focuses on preparing individuals for post-school activities, including vocational education and employment through instruction, community experiences, and acquisition of daily living skills (IDEA, 2004). Work experiences, especially those in community settings, provide students with hands on learning and on-the-job training opportunities, which helps better prepare them for future employment. Simulation training, which allows for additional instructional trials on work tasks in a separate setting (Lattimore, Parsons, & Reid, 2006; Lattimore, Parsons, & Riddle, 2008), has been used to support adults with disabilities in employment settings and may be one way to provide job- related experiences to adolescents with disabilities in high school settings. Despite the benefits of transition planning for some individuals with disabilities, many individuals with autism spectrum disorder (ASD) are not prepared to secure and maintain employment following secondary schooling. Individuals with ASD are often underemployed or unemployed at higher rates than those with other disabilities (Roux et al., 2015). Approximately 35% of young adults with ASD had never worked or attended postsecondary education upon leaving high school and more than half were completely disengaged from employment or 77 educational opportunities in the first 2 years post high school (Shattuck et al., 2012b). Those with ASD who are competitively employed typically work under 30 hr a week and in low paying jobs (Taylor & Seltzer, 2011). Work Related Social Skills Although the reasons for differences in post-secondary outcomes specifically among individuals with ASD are not fully known, deficits in appropriate social skills are a core characteristic of ASD and are said to play a critical role in low employment rates among individuals with disabilities (Phillips et al., 2014). Some evidence suggests individuals with ASD who are employed often struggle to maintain employment due to insufficient social skills rather than an inability to perform the required job duties (Elksnin & Elksnin, 2011; Hurlbutt & Chambers, 2004; Müller et al., 2003, Müller et al., 2008). Current school-based transition services may not be sufficient to fully prepare individuals with ASD for employment after graduation. Increased emphasis on developing vocational skills and work related social skills for adolescents with ASD could improve employment outcomes for these individuals. Employers often believe that individuals should come equipped with the basic social skills necessary for the job (Butterworth & Strauch, 1994). Work related social skills include any social skill related to obtaining or maintaining employment, such as appropriate interactions with customers, co-workers, and supervisors (Agran, Hughes, Thoma, Scott, 2016). Specific social skills that are frequently cited as critical in employment settings include seeking clarification or assistance when needed, arriving on time, appropriate physical contact with others, responding to criticism, and following directions (Agran et al.; Elksnin & Elksnin, 2011). Individuals with ASD who have not had opportunities to develop social skills enter the work force at a disadvantage in the competitive job market. Preparation for employment after high 78 school should include training in basic communication, interpersonal skills, and good work habits in addition to completing job related tasks in an effort to fully prepare individuals with ASD to obtain and maintain meaningful employment. Despite the importance of job related social skills, only a handful of studies have explicitly taught social skills to individuals with ASD in work settings. Dotto-Fojut and colleagues (2011) taught four adolescents with ASD to ask for assistance while completing vocational tasks. Using graduated guidance, scripts, and script fading, participants were taught to ask for help when a problem arose when materials needed to complete the job were missing, broken, or mismatched. Participants generalized the skill to a new setting and to untaught scenarios and were also able to discriminate between scenarios when assistance was needed and was not needed. Embedding key social skills directly into job training tasks may be one way to provide efficient social skills training to individuals with disabilities. In another study, Walsh and colleagues (2017) used a social skills curriculum in conjunction with video modeling to teach social communication skills to adults with ASD in a training room at a vocational rehabilitation center. A total of 30 social behaviors were targeted across three social domains (i.e., peer-related, adult-related, and self-related social skills) during the 20-week intervention. Participants demonstrated an increase in social skills across domains; however, social skills were not necessarily work-related and not assessed while participants performed job tasks. Video Modeling Video modeling is a potentially advantageous intervention for teaching a range of skills because it can readily depict the antecedents and consequences that correspond to the target behavior, which might help the observer learn what to do, as well as when and why to do it. In addition, extraneous visual and auditory stimuli can be removed to limit stimulus over-selectivity 79 (Ploog, 2010). Watching videos requires little adult interaction, potentially easing the task of fading adult-mediated support (Bellini & Akullian, 2007), an important aspect to consider when teaching skills in employment settings where independence is often expected. Video modeling, or variations of video modeling, such as video self-modeling or video prompting, have shown particular promise for teaching vocational skills in school and employment settings. Bennett and colleagues (2013) compared video prompting with and without voice over narration to teach five adolescents with ASD to make photocopies, send a fax, or make labels in the teacher workroom of the high school they attended. Similarly, Cihak and Schrader (2008) used video modeling and video self-modeling to teach vocational tasks (making copies and sending faxes or preparing first aid and food kits) to four adolescents with ASD. Results of these studies demonstrate the effectiveness of video modeling on the independent acquisition of chained vocational tasks among adolescents with ASD in school settings. Despite the positive outcomes of these studies, all of the tasks could be completed without requiring social interaction with others, further isolating individuals who already experience frequent social isolation. The effectiveness of video modeling for teaching a range of social skills, as well as vocational skills, suggests the intervention might be efficacious for teaching adolescents with ASD to engage in work related social skills while completing work tasks. In one such pilot study, Stauch and Plavnick (in preparation) used video modeling to teach two adolescents with ASD vocational and social skills in a simulated work setting. Participants were taught two vocational skills—cleaning counters and restocking supplies—while simultaneously engaging in a work related social skill—accepting a compliment or accepting criticism. Vocational and social skills were paired together and taught by showing participants one video sequence with both skills modeled in the video. Both participants mastered the vocational skills and showed 80 improvements in social skills; however additional replications of the effect would have strengthened the results of this study. In addition, generalization was not carefully planned for in this investigation. Generalization Planning for generalization should be a considered at the onset of the intervention since generalization of skills is a major goal of most interventions. Matrix training is a proven technique that is both effective and efficient in training and generalizing skills to a wide range of individuals. In matrix training, pairs of behaviors are taught together, such as play behaviors and verbal statements, in a systematic manner to allow behaviors from one set to combine with behaviors from another set without explicitly training all combinations (Axe & Sainato, 2010). It is hypothesized that through the systematic training of skills across the diagonals of the matrix, participants will be able to generalize the skills to untrained areas of the matrix. This outcome, termed recombinative generalization by Goldstein (1983a), allows for efficient instruction because skills are learned without being directly taught. Matrix training has been used in combination with video modeling and video-enhanced activity schedules to teach and facilitate generalization of play skills to young children with ASD (Dauphin, Kinney, Stromer, 2004; MacManus, MacDonald, & Ahearn, 2015). In both matrix training studies, participants were able to recombine actions and vocalizations with play sets that had not been previously trained. Matrix training has not yet been used to teach vocational or social behaviors, but could be used in a similar manner to the previously mentioned studies, wherein the vocational tasks would take the place of the play sets and the social skills would replace the play behaviors. The effects of recombinative generalization would be seen as participants perform each social behavior across each vocational task. 81 Purpose of Study The purpose of the current investigation was to extend the procedures utilized in Stauch and Plavnick (in preparation) to teach three adolescents with ASD four vocational skills and four social skills to run a concession stand for employees of a school. Matrix training was used to facilitate generalization of the four social skills across vocational tasks. More specifically, the present study investigated the following research questions: (1) Does video modeling lead to the acquisition of vocational skills for adolescents with ASD? (2) Does video modeling lead to the acquisition of social skills for adolescents with ASD when implemented in a simulated work setting? (3) Does matrix training facilitate the generalization of social skills across vocational tasks? (4) Can participants generalize vocational and social skills to a new setting and with new people? Method Participants. After receiving approval from the university institutional review board, researchers recruited a local high school to participate, described the study to special education teachers in the school, and asked them to refer adolescent students with ASD to participate in the research study. Three adolescents with ASD were referred and all were included. Teachers selected participants who currently performed their job training at the school, rather than at a community-based job site. All participants received a majority of their academic instruction in a self-contained special education classroom for individuals with developmental and intellectual disabilities who were working toward a certificate of completion rather than a high school diploma. Participants in the study met the following inclusion criteria: (a) prior diagnosis of ASD from a licensed psychologist or psychiatrist outside the context of this study; (b) at least 14 years old prior to the start of the study; (c) did not engage in self-injurious behavior or property 82 destruction; and (d) during a brief screening prior to the study, attended to a television screen for at least 20 s, followed one-step directions, and spoke in sentences comprised of at least three words. Dominic was a 14-year-old Greek male diagnosed with ASD. He received weekly speech and language services and bi-weekly social work services. Dominic displayed behaviors that interfered with job tasks at the community-based job site, such as riding on the wooden pallets and requiring frequent redirection from an adult. Dominic was observed to initiate with peers and adults around him, but had difficulty sustaining reciprocal conversations. He often perseverated on several topics and had difficulty talking about non-preferred interests. Michael was a 15-year-old White male with a diagnosis of ASD. He received bi-weekly speech and language services and weekly social work services. Michael was transferred to this high school from another high school within the district because of aggressive behaviors (i.e., hitting, kicking, and yelling) directed toward a peer. When he was transferred to his current school, he was assigned a one-to-one male aide to help him stay on task, maintain appropriate behaviors, and transition from class to class. Based on the severity of the behaviors at his previous school, Michael did not participate in community-based job training. Michael was observed to use sarcasm and humor with adults and peers and frequently told a variety of jokes, but he struggled to maintain personal space and use appropriate language. Timothy was a 15-year-old White male with ASD and 22Q Syndrome, which is a disorder caused by a small missing piece of the 22nd chromosome. He received bi-weekly occupational therapy and up to six times per month of social work services and speech and language services. Timothy displayed behaviors that interfered with job tasks at the community- based job site, such as playing in the kitty litter and removing tags from store merchandise. 83 Timothy was quiet, rarely made eye contact, and did not typically initiate with peers or adults. His vocal responses consisted primarily of one- to four-word utterances that were quiet and muffled, making his language difficult for others to comprehend. Facilitator. Michael’s one-on-one para-professional was the facilitator of the intervention for all three participants. The facilitator was a white male with a high school diploma and 2 years of experience as a para-educator for individuals with disabilities. A special education teacher, who held a Master’s degree in special education and an endorsement in ASD, was recruited to be a secondary observer and data-collector. The first author, a doctoral candidate in special education and Board Certified Behavior Analyst, provided training and coaching to the facilitator to implement the intervention with fidelity. The facilitator was taught to implement the intervention and collect data during an initial 3 hr training using a behavioral skills training approach (e.g., Sarokoff & Sturmey, 2004; Sarokoff & Sturmey, 2008; Ward- Horner & Sturmey, 2012) that included instruction, prompting, role-playing, and specific feedback on implementation accuracy based on a procedural integrity checklist until the facilitator reached a criterion of at least 90% during role play opportunities. Setting The training setting was a self-contained classroom for individuals with developmental disabilities. The majority of the study took place in the small kitchen located within the classroom, which had a refrigerator, stove, microwave, and a sink. A small rectangular table was placed at the entrance to the kitchen, which held the cash register for participants to practice taking orders. Concession stand supplies that were utilized throughout the study, such as food items or preparation supplies, were set out on the counters. Extra supplies and materials were stored in the refrigerator or in labeled cupboards. Participants set out concession stand materials 84 and put them away each day. The facilitator, participants, and secondary observers were the only ones in the classroom during intervention sessions. The generalization setting was the high school teachers’ lounge. The lounge had several circular tables with chairs for staff to eat lunch, two refrigerators, a microwave, a vending machine, storage cupboards, and a sink. The layout of the concession stand in the generalization setting was similar to that of the training setting with materials that were being used placed on counters and extra supplies stored in labeled cupboards. Materials requiring refrigeration were stored in the refrigerator. Materials. A variety of instructional materials were used throughout the study. An Apple iPad 2 was used to display video modeling clips to participants. The facilitator used paper and pencil to collect data. Materials necessary to run the concession stand were also used throughout the study including food items (e.g., chips, granola bars, candy, fruits, vegetables, etc.), preparation supplies (e.g., napkins, silverware, plates, measuring cups, etc.), a popcorn machine, refrigerator, and hardware to manage payments. A picture recipe book was also available at all times to participants for menu items that required preparation. To manage payments, the register was set up as a point of sale (POS) using the Square register application, Square stand, Apple iPad Air 2, and cash box. A picture menu of all the concession stand items appeared when the Square register application was open on the iPad. Participants could select the picture of the item(s) that were ordered, touch the charge button, and then select the amount of cash received or swipe a credit card using the Square stand. Measurement of dependent variables. Four vocational skills and four social skills were measured for Michael and Dominic, whereas Timothy was only taught three of the four skills. As in Stauch and Plavnick (in preparation), each vocational skill was paired with a 85 complementary social skill. Vocational target behaviors included: cleaning surfaces, restocking supplies, completing customer orders, and working the cash register. Each vocational skill was task analyzed (see Table 6). Each individual subtask was assessed for accuracy and scored as correct, incorrect, or not applicable. Any task-analyzed step that was completed correctly, regardless of the order, was scored as correct since order was not essential to the correct final outcome for each skill. The total correct tasks during a given session were divided by the total number correct and incorrect tasks on the task analysis across trials and multiplied by 100 to obtain a percentage of correct steps performed for each session. 86 Table 6. Task Analysis for Vocational Target Behaviors Cleaning surfaces Restocking supplies Completing Working the cash register When an adult stands at the counter to place an order, the participant: 1. Waits 5-10 s and asks customer if he or she is ready to order (n/a if customer immediately orders) 2. Touches icons for items ordered on iPad 3. Correctly repeats customer order 4. Correctly fills out customer order sheet 5. Tells customer price of order 6. Takes card or cash 7. Completes payment 8. Hands correct change or card back to customer When the facilitator tells the participant to clean, he: 1. Gathers supplies; multi-surface spray cleaner and dish rag or paper towel 2. Removes all portable items from the counter area and throws away trash, if any 3. Sprays cleaner on countertop 4. Wipes clean with dish rag or paper towel 5. Checks counter for any missed spots or excess spray and wipes again if necessary 6. Shakes out rag over trash or throws paper towel in the trash 7. Returns portable items to original location on surface, if any 8. Returns cleaning items to storage When the facilitator tells the participant to restock two different supplies, one that was full and one that needed restocked he: 7. Checks supply levels for item 8. Gathers additional items from storage for the item that is below 50% or below the fill customer orders When a customer order sheet is placed on the counter or someone verbally requests an order to be filled, the participant: 1. Correctly reads ordering sheet or repeats order 2. Gathers appropriate supplies needed to complete order for item 1 9. Does not gather supplies for the item that is above 50% or above the fill line 10. Items are neatly 3. Prepares drink or food item consistent with order (n/a if grab and go) 4. Gathers appropriate supplies for item 2 (if 2nd item ordered) 5. Prepares second item consistent with order (if 2nd item ordered or n/a if grab and go) 6. Hands drink or food items to customer 7. Cleans up and puts away food prep materials placed in containers (e.g., organized in same direction, stacked on top of one another or next to one another with nothing sticking out) to fill line or no more will fit 11. Returns extra items to storage, if any 12. Draws a checkmark on supply list to indicate which item was restocked 87 Social behaviors included: accepting a compliment, accepting criticism, fulfilling requests, and maintaining small talk. Table 7 provides operational definitions and examples for each of the social skills. For social behaviors, observers recorded the occurrence or nonoccurrence of targeted social skills following a programmed antecedent. The total correct responses for the social skill were divided by the total response opportunities for that skill and multiplied by 100 to obtain a percentage. Table 7. Operational Definitions of Social Skills with Example Scenarios Behavior Accepting a compliment Definition When another person compliments the participant, he (a) thanks the person for the compliment and (b) makes a comment about the content of the compliment. Accepting criticism Fulfilling requests Maintaining small talk When another person criticizes the participant’s work, he (a) acknowledges the criticism or apologizes for what he did incorrectly and (b) requests or provides a solution to prevent the problem from occurring in the future. When an adult asks a question or makes a request related to the participant’s job, the participant either (a) correctly answers the question; (b) vocally responds to indicate he heard the request and performs the correct behavior to complete the request, or if the participant does not know the answer; or (c) vocally responds to indicate he heard the request, seeks out the answer, and relays the correct information When another person asks the participant a question or makes an open-ended comment, the participant (a) answers the question or acknowledges the statement and (b) asks the person an on topic question or makes an on topic comment that invites a response from the person. 88 Example Scenario The facilitator says to the participant, “Wow, this table looks so clean!” The participant says, “Thank you. I tried my best to make it look nice.” The facilitator says to the participant, “You added too many popcorn kernels and made a mess.” The participant says, “I’m sorry for making a mess. Next time I will not add as many.” A customer asks, “What kind of soda do you have?” The participant says, “We have coke and diet coke.” A customer says, “I can’t believe it’s already 11:00 a.m.” The participant responds with, “Time has been going by quickly. What time is your lunch?” Inter-observer agreement (IOA). The primary facilitator and data collector throughout the study was the para-professional. The facilitator and special education teacher were trained to collect data on targeted behaviors until reaching 90% agreement levels with the first author. Training consisted of watching previously recorded examples and nonexamples of targeted behaviors performed by adult models, independently recording the occurrence or nonoccurrence of the behavior, discussing with the researcher how they scored the behavior, and solving any discrepancies. The first author or special education teacher were the secondary observers and simultaneously though independently recorded dependent measures across sessions that were evenly distributed across conditions, behaviors, and participants for at least 35% of sessions. A point-by-point agreement calculation (Cooper, Heron, & Heward, 2007) for each behavior was used to derive a percentage of agreement. For vocational skills, each operational step was scored as either an agreement or disagreement between the observer and facilitator. For social behaviors, agreements or disagreements were scored for each trial. The percentage of agreements was obtained by dividing the number of agreements by the total number of agreements plus disagreements multiplied by 100. For Dominic, mean IOA for all behaviors during baseline was 97% (range, 75% to 100%) and 97% (range, 66% to 100%) for intervention. For Michael, mean IOA for all behaviors was 95% (range, 75% to 100%) during baseline and 98% (range, 67% to 100%) during intervention. For Timothy, mean IOA for all behaviors during baseline was 98% (range, 75% to 100%) and 95% (range, 50% to 100%) during intervention. Experimental design. A multiple probe design with probe conditions (Gast & Ledford, 2014) across behaviors replicated across participants was used to evaluate the effectiveness of video modeling for teaching the targeted vocational and social skills and to assess the generative 89 transfer of social skills across the matrix. Probing all behaviors once a targeted behavior reaches criterion allows researchers to examine the extent to which the previously mastered behaviors occur without first displaying the video models and assesses whether the remaining untreated behaviors occur at levels similar to baseline. The intervention is administered sequentially across cases and experimental control is demonstrated if baseline levels of each case remain stable and changes in responding are replicated only when the intervention is applied (Horner & Baer, 1978). A functional relation is established when the effect is consistently and reliably replicated across cases. In the present study, a single participant represents a complete experiment with the opportunity to replicate an observed functional relation across remaining participants. All social and vocational behaviors were initially probed under baseline conditions for all participants. Following baseline and generalization probes, video modeling was then applied to the first pair of targeted behaviors for each participant until the participant performed the vocational skill with at least 80% accuracy on each trial across two consecutive sessions. Mastery criteria was based on performance on vocational skills, rather than social skills to prevent participants from staying in the intervention condition for vocational skills longer than necessary. Once participants met mastery criteria, all behaviors were probed under baseline conditions to assess whether trained behaviors improved without the intervention in effect and whether untrained behaviors were similar to baseline levels. Alternating between intervention and baseline conditions was conducted in this manner until all participants received video modeling for all target behaviors. A video fading and maintenance condition was included for mastered behaviors after the post intervention probe. After a final probe was conducted for all targeted behaviors, these skills were assessed in the generalization setting. 90 Procedures Creation of video clips. A total of 22 videos, ranging from 55 s to 3 min 34 s in length were developed and streamed to an Apple iPad 2 for viewing during video modeling sessions. Although each skill had a similar number of steps, completing customer orders took significantly more time than the remaining skills since following a recipe for food items requiring preparation was comprised of several smaller steps that were not captured in the task analysis. Five or six video exemplars for each vocational behavior were filmed in the training setting using female adult models, who followed scripts that depicted a range of scenarios participants may encounter while completing each job task. Each video was broken down into chapters that corresponded with each individual subtask on the task analysis so that each step could be played individually during error correction procedures. For each behavior pairing, the social behavior was embedded within the vocational task; therefore, one video was made to depict both skills. Probe sessions. The first three sessions of the current study functioned as a baseline to probe levels of targeted behaviors for each participant prior to implementation of the intervention. Later probes functioned as post-training probes or additional baseline, depending on the behavior. Probe sessions took approximately 2 hr and included setting out concession stand materials and putting them away at the beginning and end of the session and conducting trials for targeted skills. The first author, secondary observer, facilitator, and participants were the only people in the room during probe sessions. During probe sessions, each participant worked one-on-one with the facilitator for two trials, while the other two participants sat in the classroom area and quietly read or played on a tablet. Once a participant received two trials of the same vocational skill, he was then instructed to take a break, while another participant 91 received two trials of a vocational skill. This pattern continued until all participants received the required number of trials and all vocational and social behaviors were probed. During probe sessions, the facilitator contrived two opportunities for each participant to perform each vocational task. Trials for working the cash register were initiated whenever an adult approached the counter and looked at the menu. Trials for completing orders were initiated when a customer order sheet was filled out and placed on the counter and the facilitator told the participant an order was ready. Trials for cleaning surfaces and restocking supplies were initiated when the facilitator told the participant to clean the work area or restock supplies, but did not include any information on how to perform any of the tasks. Participants were allowed to complete the task until 15 s elapsed without a correct step performed or he verbally indicated he was done with the task. If the participant asked a question related to how to perform the job task, the facilitator made a general statement to the participant to continue with the job (e.g., “Do your best”). Matrix probes for social skills. Matrix training was used to assess generalization of learned social skills across vocational tasks. The four-by-four instructional matrix included the vocational skills along the y-axis and vocational social skills along the x-axis, yielding 16 combinations of social behaviors across vocational tasks, only four of which were directly taught (see Figure 3). Social skills trials were conducted across jobs because each of the social behaviors could be useful when interacting with other participants, customers, and the facilitator, regardless of the job he was performing. 92 Figure 3. Matrix for Vocational Social Skills across Vocational Tasks Accept instruction Accept a compliment Accept criticism Maintain small talk Train Train Train Train Work the cash register Complete customer orders Clean surfaces Restock supplies Social skills were assessed during trials for vocational tasks and each participant received a total of four opportunities to perform each social skill during probe sessions. During probe sessions, two social skills trials were conducted during one vocational trial so that each social behavior was assessed one time across all of the vocational behaviors (Table 8). Two of the four social behaviors were probed during the first vocational skills trial and the remaining two social behaviors were probed during the second vocational skills trial. Each social behavior was probed one time across each vocational task. Therefore, each participant received a total of four trials for each social behavior during each probe session. The facilitator followed scripted protocols to ensure the correct pairings and delivery of the correct discriminative stimulus (SD). Once a participant received two opportunities to perform a vocational task and one opportunity 93 to perform each of the social skills, he was then given an opportunity to perform the next vocational task until all jobs were probed and all social skills were probed across each vocational task. No feedback, prompts, or error correction was provided to participants during probe sessions. To assess mastery of each of the four social behaviors across each vocational task, a final probe session was conducted once all participants met mastery criteria for all behaviors and had received video modeling training on all vocational tasks. 94 Table 8. Example of Social Behaviors Probed Across Vocational Tasks During Probe Sessions Vocational Accept a compliment Accept criticism Fulfill requests Maintain small talk Facilitator says to participant, “This area is not very clean. There are used napkins everywhere.” Not probed Not probed Not probed Customer asks, “Where should I put my empty drink can?” Facilitator asks, “Wow it has been busy today. How are you feeling?” Not probed Not probed Facilitator asks, “Do you have any plans after work?” Not probed Not probed Facilitator asks, “Did you hear it’s supposed to be warm all weekend?” Customer asks, “Did you do anything fun over the weekend?” Facilitator asks, “When will the concession stand be open?” Customer says, “Can you add extra salt to the popcorn please?” Not probed Not probed Facilitator says, “The supplies for customers should be full at all times.” Not probed Customer says, “I didn’t receive everything I ordered.” Facilitator says, “You need to work quickly when using the cash register.” Not probed Customer asks, “Am I able to use my credit card to pay for my order?” Not probed Cleaning surfaces Facilitator says, “You are very professionally dressed today.” Cleaning surfaces Restocking supplies Restocking supplies Completing orders Completing orders Working the cash register Not probed Facilitator says, “I appreciate how hard you are working today!” Not probed Facilitator says, “Your shoes were a great choice to wear to work.” Not probed Not probed Working the cash register Facilitator says, “You’ve been trying really hard today.” 95 Generalization probes. Once baseline probes were completed, one generalization probe was conducted in the generalization setting with employees of the school, who were recruited to serve as customers. Upon arrival, the three volunteer customers were provided with a brief overview of the purpose of the day’s session and given a card that listed what to say and do while ordering from the concession stand. The cards were created to ensure the correct SDs were delivered and a range of potential customer situations were included. Generalization probe sessions were conducted in the same manner as baseline probe sessions. To assess generalization of skills to a new setting and with actual customers, a generalization probe was conducted once all participants met mastery criteria for all behaviors and had received video modeling training on all vocational tasks. Video modeling. After the initial baseline probe sessions and before participants were expected to use each skill in the generalization setting, participants were taught all vocational and social skill pairings in the training setting. Intervention sessions lasted approximately 2 hrs and were conducted 3 days a week. Sessions took place while the other students in the class were in the community at various job sites. Since there was only one facilitator, participants took turns performing job tasks. Each participant received one or two trials consecutively and was then told he could take a break until it was his turn again. As in probe sessions, participants on a break sat in the classroom area and read or played on a tablet. Participants rotated turns until each participant received three to five opportunities to perform the targeted vocational skill as well as any required maintenance or video fading trials for previously learned behaviors (see below). To initiate a trial, the facilitator contrived opportunities for the participant to practice each skill by engaging in the antecedent behavior described above. The facilitator told the 96 participant to “watch the video,” showed him the video segment that depicted the step on the task analysis, and then provided the participant with an opportunity to complete the step. Once the step was completed, participants were told to “watch” or “look” and shown the next video segment, which depicted the next step on the task analysis. If the participant made an error in the task analysis, the facilitator scored the step as incorrect, interrupted the error, and showed the participant the video segment that demonstrated the specific step. The participant was immediately given another opportunity to complete the step. If after watching the video segment the participant then performed the step correctly, he was shown the next video segment and expected to continue with the task analysis sequence. If after re-watching the video segment the participant still performed the step incorrectly, the participant was told what to do differently next time and asked to continue with the task. The facilitator also created one opportunity to demonstrate the paired social skill by engaging in the antecedent behavior during every vocational skills trial. The social skill was embedded into the video for the vocational task and occurred at the beginning or end of the video. After the segment of the video that depicted the social skill was shown, the facilitator paused the video and engaged in the same antecedent behavior that was shown in the video. The participant was expected to respond in a manner that aligned with the target behavior definition for that skill. The same error correction procedure that was used for the vocational task was followed for the social behavior. Video modeling continued until the participant performed the vocational skill with at least 80% accuracy on each trial across two consecutive sessions. To prevent participants from staying in the intervention for the vocational skill for longer than necessary, participants were not required to master the paired social skill before moving on to the next vocational skill. However, 97 error correction procedures were utilized if participants did not demonstrate the social skill correctly during video fading and maintenance probes (see below). Timothy experienced an increase in absences during the intervention for the third behavior pairing. Due to the increase in absences, there was not enough time left in the school year to teach the last behavior pairing. Video fading. During the first several sessions following mastery, videos were systematically faded. Video fading for previously mastered skills took place during video modeling sessions for the new behavior pairing. To fade the videos, participants were asked to perform the previously mastered vocational skill for two trials, with one video viewed during the first trial and no video on the second trial until both trials were performed with 80% or greater accuracy. Once 80% mastery was achieved, a video was shown on the second trial only if the participant performed the first trial with less than 80% accuracy. Once participants achieved 80% or greater accuracy on both trials and no videos, maintenance probes were periodically conducted for mastered skills. If an error occurred during video fading, the facilitator followed the error correction procedures as in video modeling sessions. A social skills probe for the previously mastered social skill was embedded into each vocational trial and error correction procedures were utilized if an error was made. Maintenance probes. Maintenance probes for vocational and social skills occurred once mastery criterion was met and videos were faded. To conduct maintenance probes, at least one time per week, the facilitator created one opportunity for the participant to perform a previously mastered vocational and social skill by engaging in the antecedent behavior. No videos were shown during maintenance probes. If a mistake was made on a social skill during maintenance probes, the facilitator told the participant what to do differently next time. No error correction was used for vocational skills. 98 Procedural integrity. A comprehensive checklist with 24 items for baseline or 56 operationally defined items for intervention sessions was used to assess the extent to which all components of the intervention were implemented correctly or incorrectly on a trial by trial basis. Each item on either checklist was assessed for accuracy and scored as observed, not observed, or n/a if it was not applicable on that trial. Procedural integrity data was collected for at least 35% of baseline and intervention sessions for each participant to assess fidelity of implementation and provide ongoing evaluative feedback to the facilitator. If an item on the checklist was missed, performance feedback was given each time procedural integrity was assessed. Mean procedural integrity across baseline and intervention sessions was 93% (range, 88 to 100%) for Dominic and Michael and 95% (range 92 to 100%) for Timothy. Social validity. The facilitator was asked to complete the Intervention Rating Profile (Martens, Witt, Elliot, & Darveauz, 1985) to indicate how satisfied he was with the goals, procedures, and outcomes following the completion of the intervention. The intervention rating profile consists of 15 questions rated using a 6-point Likert scale ranging from strongly disagree to strongly agree. The questionnaire was slightly adapted to focus on the video modeling intervention and vocational and social skills. Scores above 52.5 are considered to reflect acceptability of the intervention by the rater (VonBrock & Elliott, 1987). Overall, the facilitator found the intervention to be beneficial to students, liked the procedures utilized, thought it was a good way to teach new skills, and would recommend the intervention to others. The facilitator rated the intervention an 80, which is well above the acceptability level of 52.5. A total of eight items were marked as strongly agree, five as agree, one as slightly agree, and one as slightly disagree. The item marked slightly disagree was regarding whether the intervention was consistent was those the facilitator had used in classroom settings. 99 Results Results of video modeling on the acquisition of vocational skills for Dominic are displayed in Figure 4. Dominic demonstrated zero or near zero levels of responding on all vocational skills during initial baseline probe sessions, with an immediate increase in performance in targeted skills when the intervention was implemented. Responding for cleaning increased to a mean of 94% (range, 85 to 100%) when video modeling was applied. Dominic demonstrated an immediate increase in performance when video modeling was applied to restocking supplies, with a mean of 91% (range, 87 to 100%) across four sessions. When video modeling was applied to completing orders, Dominic demonstrated a steady increase in performance across the seven sessions with a mean of 85% (range, 65 to 100%). When the intervention was applied to working the cash register, a steady increase in performance was observed with mean responding of 83% (range, 63 to 92%) for the five sessions. All vocational behaviors were performed with 75% or greater accuracy during the final post-training probe. Following training, Dominic was able to clean, restock, complete orders, and work the cash register with 93%, 75%, 100%, and 93% accuracy in the generalization setting. Results of video modeling on the acquisition of social skills for Dominic are displayed in Figure 4. During initial baseline probe sessions, Dominic demonstrated zero or near zero levels of responding across all social skills. When video modeling was applied to accepting a compliment, responding was varied with an overall mean of 51% (range, 33 to 80%) across three sessions. Dominic accepted a compliment with 75% accuracy during the first two post-training probes and 50% accuracy during the third post-training probe. When video modeling was applied to accepting criticism, performance increased to 29% accuracy (range, 0 to 50%) across four sessions. During the first post-training probe, Dominic accepted criticism with 25% 100 accuracy, but this increased to 75% or higher for the remaining post-training probes. When video modeling was applied to fulfilling requests, performance increased to 66% accuracy (range, 33 to 100%) across seven sessions and increased to 75% or greater during post-training probes. When the intervention was applied to small talk, performance increased to 37% (range, 0 to 100%), but dropped to 25% accuracy during the post-training probe. Results of matrix training for the social skills are demonstrated during the final post-training probe. Out of a total 16 possible combinations, four social behaviors were trained and nine were acquired across the vocational tasks. Dominic did not acquire the social skill of maintaining small talk across any of the vocational tasks. Results of video modeling on the acquisition of vocational skills for Michael are shown in Figure 5. Michael demonstrated low levels of performance across vocational skills during initial baseline sessions and demonstrated a rapid increase in performance when video modeling was applied to targeted behaviors. During initial baseline probe sessions, Michael cleaned with a mean of 44% (range, 31 to 50%). When video modeling was applied to cleaning, Michael reached mastery criteria within two sessions and performance increased to an average of 95% (range, 93 to 97%). Similar results are seen for restocking, with a mean of 92% (range 75 to 100%) and mastery criterion achieved within three sessions. Michael completed orders with a mean of 87% (range, 65 to 100%) across 10 sessions. Initial baseline levels were higher for working the cash register, with a mean of 58% (range, 44 to 69%). When video modeling was applied to working the cash register, performance increased to a mean of 89% (range, 78 to 96%) and mastery criterion was achieved within three sessions. During the final post-training probe, Michael demonstrated all skills with 90% or greater accuracy. Generalization of vocational behaviors was observed with all skills performed above 80% accuracy. 101 Results of video modeling on the acquisition of social skills for Michael are shown in Figure 5. During initial baseline probes, Michael demonstrated each social skill with a mean of 33% or lower accuracy. Performance for accepting a compliment increased to a mean of 90% (range, 80 to 100%) when video modeling was applied. During all post-training probes, Michael accepted a compliment with 75% or greater accuracy. When video modeling was applied to accepting criticism, performance increased to a mean of 87% (range, 60 to 100%). Michael accepted criticism with 50% accuracy during the first post-training probe, but this increased to 75% or greater accuracy during future post-training probes. An increasing trend was observed during initial baseline sessions for fulfilling requests, with an overall mean of 38%. When video modeling was applied to fulfilling requests, an immediate increase in performance occurred (M = 97%), with only one session performed with less than 100% accuracy. When video modeling was applied to small talk, Michael achieved 100% accuracy across all three sessions. Results of matrix training are demonstrated during the final post-training probe, with all 16 possible combinations acquired, four of which were trained and 12 additional social behaviors acquired across vocational tasks. Generalization of social skills was observed with all skills performed with 75% or greater accuracy during the post-training generalization probe. Results of video modeling on the acquisition of vocational skills for Timothy are shown in Figure 6. Timothy demonstrated zero levels of responding for all three vocational skills during baseline. When the intervention was applied to cleaning surfaces, a steady increase in performance was observed with mean responding of 85% (range, 54 to 100%) across six sessions. Similar results are seen for restocking supplies, with an immediate increase in performance upon implementation of the intervention and mean responding of 87% (range 78 to 100%) across six sessions. An overall increasing trend was observed for completing orders, with 102 a slight drop in performance before mastery criterion was achieved in the final two sessions and a mean performance of 78% (range, 53 to 91%) across 10 sessions. All vocational skills were performed with 80% or greater accuracy during the final post-training probe. Generalization was observed, with zero levels of responding for vocational behaviors during the first generalization probe and final levels of 75%, 58%, an 83% for cleaning, restocking, and completing orders respectively. Results of video modeling on the acquisition of social skills for Timothy are shown in Figure 6. During baseline, Timothy demonstrated zero or near zero levels of responding across all social skills. When video modeling was applied to accepting a compliment, performance increased to a mean of 73% (range, 40 to 100%). An initial increasing trend was observed when video modeling was applied to accepting criticism, with performance declining in the final three video modeling sessions. Timothy accepted criticism with a mean of 69% (range, 25 to 100%). Performance on fulfilling requests was more variable, with an average of 69% during intervention sessions. Results of matrix training are demonstrated during the final post-training probe for social skills. A total of nine combinations of behaviors were possible for Timothy. Three social behaviors were trained, with an additional five acquired across vocational tasks. Some generalization of social skills was observed with performance of 25% or below during the initial probe and performance of 33%, 66%, and 100% for compliment, criticism, and fulfilling requests respectively during the final generalization probe. 103 Figure 4. Percentage of Trials Dominic Performed Vocational and Social Skills BL VM BL VM BL VM BL VM BL 100 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 y l t n e d n e p e d n I d e t e l p m o C s p e t S f o e g a t n e c r e P 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Cleaning surfaces and accept compliment Restocking supplies and accept criticism 1 3 5 7 9 11 13 15 17 19 21 23 25 27 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Completing orders and fulfill requests 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Cash register and small talk Percentage of steps completed independently across baseline (BL) and video modeling (VM) conditions for Sessions Dominic. Closed circles represent vocational skills. Closed triangles represent social skills. Open triangles represent social skills matrix training probes. Red circles and triangles depict generalization probes. 104 Figure 5. Percentage of Trials Michael Performed Vocational and Social Skills 100 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 y l t n e d n e p e d n I d e t e l p m o C s p e t S f o e g a t n e c r e P BL VM VM BL BL VM BL VM BL Cleaning surfaces and accept compliment 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Restocking supplies and accept criticism 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Completing orders and fulfill requests 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Cash register and small talk 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Percentage of steps completed independently across baseline (BL) and video modeling (VM) conditions for Sessions Michael. Closed circles represent vocational skills. Closed triangles represent social skills. Open triangles represent social skills matrix training probes. Red circles and triangles depict generalization probes. 105 Figure 6. Percentage of Trials Timothy Performed Vocational and Social Skills BL VM BL 100 VM BL VM BL 80 60 40 20 0 100 80 60 40 20 0 100 80 60 40 20 0 y l t n e d n e p e d n I d e t e l p m o C s p e t S f o e g a t n e c r e P Cleaning surfaces and accept compliment 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 Restocking supplies and accept criticism 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 Completing orders and fulfill requests 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 Sessions Percentage of steps completed independently across baseline (BL) and video modeling (VM) conditions for Timothy. Closed circles represent vocational skills. Closed triangles represent social skills. Open triangles represent social skills matrix training probes. Red circles and triangles depict generalization probes. Discussion Overall results of the study confirm video modeling was effective in teaching adolescents with ASD vocational skills and social skills in a simulated work setting. All participants acquired vocational tasks relatively quickly, reaching the mastery criterion within 10 sessions for 106 each skill. This is consistent with previous research utilizing video modeling to teach chained vocational tasks to adolescents with ASD, where participants acquired skills within 15 or fewer sessions (Bennett et al., 2013; Cihak & Schrader, 2008; Stauch & Plavnick, in preparation). Participants also demonstrated improvements across most social skills from baseline to the final post-training probe, indicating matrix training was effective for teaching some social skills across vocational tasks. Participants generalized vocational skills to a new setting and with actual customers and demonstrated an improvement in performance on social skills in the generalization setting. These outcomes replicate and extend previous research for teaching vocational and social skills to adolescents with ASD (Stauch & Plavnick, in preparation). Acquisition of social skills. The rate of acquisition and patterns of responding for social skills varied across participants. Michael acquired vocational and social skills at similar rates and demonstrated high levels of performance across all behavioral pairings. Yet Timothy reached 100% accuracy for accepting criticism, then demonstrated a steady decline in performance, possibly suggesting issues with motivation. Dominic rarely performed any of the social skills with 100% accuracy during video modeling sessions. It is possible that social skills interventions may need to be more individualized for participants and take into account social skill repertoires at pre-intervention. A direct assessment of work related social skills where the environment is arranged to contrive opportunities for participants to engage in a variety of social skills, may be one method to identify social deficits and the important prerequisite skills to teach in a given setting (Lerman, White, Grob, Laudont, 2017). Directly measuring a range of social skills in addition to job-specific skills may help prioritize skills and provide information regarding the level of intervention necessary before starting competitive employment. Given the varying response patterns and rates of acquisition, a participant like Michael may only need 107 training for a short period of time on a few specific skills whereas participants such as Timothy or Dominic may require more intensive social skills intervention on a range of social skills. Dominic’s results are consistent with results observed in Stauch and Plavnick (in preparation), where participants demonstrated lower levels of responding for social skills than vocational skills. Dominic’s performance on social skills typically increased during video fading and maintenance sessions, possibly indicating more trials or intervention sessions were necessary to acquire social skills. Another possible explanation for the increase in performance on social skills during video fading and maintenance conditions is due to the error correction procedures that were utilized. During error correction procedures for video modeling and fading conditions, the facilitator first showed another video, and then told the participant what to do differently next time if an error was made. During maintenance, no additional videos were shown, but verbal error correction was still provided for social skills. The facilitator typically repeated the same phrase during error correction procedures. For example, for accepting criticism, the facilitator often told participants to say, “Sorry about that, next time I will [enter improvement].” It is possible Dominic learned which response to give based on the verbal SD or tone of voice delivered during error correction rather than through the videos. Dominic did not show improvements in maintaining small talk from baseline to intervention; however, video fading and maintenance sessions were not conducted for this skill due to time constraints. Given the increase in performance for other social skills during video fading and maintenance, it is possible Dominic may have demonstrated higher levels of performance for maintaining small talk with more sessions. Efficiency of instruction. The use of video modeling to teach chained vocational tasks and social skills simultaneously appeared to be an efficient approach for teaching a range of 108 work related skills to participants. Efficiency of instruction is important in employment settings due to the limited time and resources available to teach work related skills, with the majority of time devoted to teaching specific job tasks, rather than social skills (Chen et al., 2015). By teaching two skills at once, we were able to maximize instructional time and ensure participants acquired the social skills necessary for their jobs without taking away from the time needed to learn job tasks. Although the present study did not investigate outcomes within an authentic work environment, the acquisition of job-specific social skills might improve sustained employment outcomes for individuals with ASD, who are more likely to be fired if they do not have the social skills for the job, even if they are able to perform the necessary job tasks (Elksnin & Elksnin, 2011; Hurlbutt & Chambers, 2004; Müller et al., 2003; Müller et al., 2008). Since difficulty with social interaction is a core characteristic of ASD, individuals with ASD often require explicit instruction to develop critical social skills. Embedding social skills training opportunities into job tasks may be one way to prepare individuals for the varied demands of competitive employment since these skills typically do not occur in isolation of one another. Video modeling combined with matrix training also appeared to enhance the efficiency of instructional procedures. Training along the diagonal of the matrix resulted in a possible 16 combinations of behaviors for Michael and Dominic and nine possible combinations for Timothy. Participants demonstrated high levels of responding during the final post-training probe. Overall, Michael acquired all 16 possible combinations of responses, 75% of which were accounted for by recombinative generalization. Dominic acquired 12 of the 16 possible combinations of behavior, 56% of which were due to recombinative generalization. Timothy acquired eight of the nine possible responses, 56% of which were acquired without direct training. Since four of these behaviors were taught (three for Timothy), the remaining 109 combinations of responses were acquired without explicit training. The effects of recombinative generalization observed in this study are similar to other studies utilizing matrix training and video modeling or video enhanced activity schedules (Dauphin et al., 2004; MacManus et al., 2015) and offer insights into potential methods to increase the efficiency of training and generalizing social skills among adolescents with ASD. Systematic instruction of social skills that does not require directly teaching all skills has the potential to be beneficial in employment settings given the range of skills needed to interact with a variety of people across contexts. Unlike previous vocational skills studies for adolescents with ASD, which did not require social interaction when teaching vocational tasks (Seaman & Cannella-Malone, 2016), participants in the current study were taught social skills while they completed all work tasks. This is important because most jobs require individuals to interact with customers, co-workers, or supervisors for at least some portion of the day, and doing so effectively plays a crucial role in individuals with disabilities ability to obtain and maintain employment (Phillips et al., 2014). Interactions with others may come in the form of accepting instructions from a supervisor, answering questions from customers, or engaging in small talk with co-workers during lunch or on breaks. Directly teaching every possible combination of social skill with each job task or across different individuals would be challenging and time consuming. Since individuals with ASD struggle to generalize learned skills (Koegel & Koegel, 1995), matrix training may be an efficient way to teach employees with ASD to use a range of social skills across contexts. Limitations. There are limitations of the present investigation that warrant further consideration. First, two of the three participants demonstrated variable performance across social skills during intervention sessions. Participants acquired vocational behaviors relatively 110 quickly, possibly not allowing enough trials or intervention sessions for social skills. Although participants did not master social skills during intervention sessions, participants showed gains from baseline to the final post-training probe, indicating the intervention was effective at improving social behaviors. Future research should examine methods to improve the acquisition of work related social skills, such as utilizing a mastery model for teaching social skills in work settings. Another potential limitation is the relevance of the social skills that were targeted. In the present investigation, the social skills were selected based on previous studies that identified the skills as important for the workplace (Agran et al., 2016; Elksnin & Elksnin, 2011; Philips et al., 2014). Each of the social skills were useful across each of the job tasks, but participants could complete each job without using the social skills. In employment settings, productivity and accuracy are likely initially more important than one’s ability to accept a compliment or engage in small talk. In the current study, the facilitator reported that participants frequently stopped working or completed tasks incorrectly when a problem could not be solved. Teaching participants social skills that may lead to increased success on the job, such as identifying work- related problems and asking a supervisor for assistance, may have been more beneficial in the current study. Future research should examine which work related social skills are most relevant to teach to individuals with disabilities. A third limitation is that there were several components of the intervention, and it is unknown which components were essential for skill acquisition and which were unnecessary. Error correction procedures were utilized in conjunction with video modeling, so it is possible that participants learned to perform some steps due to error correction rather than through videos. We presume that multiple components were necessary for optimal outcomes, though future 111 research could conduct a component analysis to determine which specific variable or variables are responsible for the observed effects. Last, since the intervention was delivered in a one-on-one format, participants were not always engaged in the intervention and sat idle for periods of time. When the facilitator delivered the intervention to one participant, the other two participants took a break by reading or using a tablet device until it was their turn again. More frequent breaks are a common accommodation for students with individualized education plans, so it is not that unusual for individuals with ASD to have short periods of work followed by a brief instructional break. However, future research may consider a group instructional approach (see Plavnick, Sam, Hume, & Odom, 2013; Plavnick, Kaid, and MacFarland, 2015), where the intervention is delivered to multiple participants at one time, as a way to reduce the amount of idle time for participants. Future research may also examine methods to train job coaches to deliver the intervention since they typically work one-on-one with individuals in employment settings. Conclusion. Results of the current investigation support previous research on the use of video modeling to teach vocational skills and work related social skills to adolescents with ASD in simulated work settings. Vocational skills and social skills were taught simultaneously, with one video being used to depict both skills. Matrix training was used to facilitate the generalization of social skills across vocational tasks. This contributes to the current vocational skills literature by demonstrating a potentially efficient manner for teaching skills in a simulated setting that can generalize to a workplace setting. Overall, the present study suggests video modeling combined with matrix training may be a feasible and efficient approach for teaching work related skills to adolescents with ASD. 112 CHAPTER 5 Discussion The three studies in this dissertation investigated the current state of the vocational skills literature and potential methods to teach vocational and work related social skills to adolescents with ASD. The systematic literature review in Chapter 2 evaluated studies that taught vocational skills or work related social skills to adolescents or adults with ASD in school or work settings. The review concluded that there are relatively few SCED studies targeting vocational skills or social skills and none of the practices can be considered evidence-based practices (EBPs). The pilot study in Chapter 3 examined the effectiveness of video modeling to teach vocational and social skills to adolescents with ASD in a simulated work setting. Both participants acquired the two or three vocational skills but demonstrated varied responding for social skills. Results of the pilot study informed the final SCED study in Chapter 4, where three adolescent participants were taught vocational and social skills in order to operate a concession stand for school employees. Video modeling was effective for teaching vocational skills and matrix training was effective for teaching some of the social skills across vocational tasks. In both studies, videos for social skills were embedded into videos for vocational tasks, demonstrating the efficiency of video modeling to teach chained vocational tasks and social skills simultaneously. Considering the research that has been conducted and the results of the current studies, implications for practice and recommendations for future research are provided. Current State of the Vocational Skills Literature At least seven vocational skills literature reviews for adolescents and adults with ASD have been conducted in the last five years (Anderson et al., 2016; Bennett & Dukes, 2013; Nicholas et al., 2015; Seaman & Cannella-Malone, 2016; Walsh et al., 2014; Walsh et al., 2017; 113 Westbrook et al., 2012). However, there are relatively few studies that have explicitly taught vocational skills or work related social skills when compared to the number of vocational skills literature reviews that have been conducted. The systematic literature review in Chapter 2 was designed to address the limitations of previous reviews by including vocational skills and work related social skills in both school and work settings for adolescents and adults with ASD. It is clear from all of the reviews that there is not enough research focused on teaching vocational skills to individuals with ASD and even fewer studies to teach work related social skills. In addition, many of the published studies are of poor methodological quality. Due to the overall poor quality and the lack of published studies, there are currently no EBPs to teach vocational skills or work related social skills to adolescents and adults with ASD. In order to address the current limitations of the vocational skills literature, a shift needs to be made from reviewing the vocational skills studies to contributing to the current research. More research needs to be done that explicitly examines methods to teach vocational and work related social skills to individuals with ASD. Studies should be designed according to the WWC SCED standards (Kratochwill et al., 2013) to ensure research studies meet the highest methodological rigor, which allows SCED research to be evaluated in a manner comparable to RCTs. Chapter 3 was a pilot study, meaning there were not enough demonstrations of effect for the study to meet the WWC standards with or without reservations. Chapter 4 would meet WWC design standards with reservations based on the number of probe points during baseline. In order to meet standards, the multiple probe design requires three initial baseline data points and at least three probe points for cases just prior to the introduction of the intervention. Since this would have taken considerably more time and it was likely participants would not 114 demonstrate significant increases in skills until the intervention was implemented, only one probe point was conducted just prior to the implementation of the intervention. Based on the WWC evidence standards (Kratochwill et al., 2013), there was strong evidence of effect for teaching vocational skills across the three experiments in Chapter 4. One of the experiments would also demonstrate strong evidence of effect for teaching work related social skills. Although all participants demonstrated an increase in social skills from baseline to the final post-training probe, given the amount of variability during the video modeling phase for social skills, two of the experiments would be categorized as demonstrating no evidence of effect. It is encouraging that the study meets standards with reservations and was effective for teaching vocational skills since the procedures used to teach work related social skills in Chapter 4 were novel. Future research should address some of the limitations in Chapter 4 by improving the procedures to teach work related social skills to adolescents with ASD in order to demonstrate evidence of effect between the independent variable and all dependent variables. Evidence Based Practices for Vocational Skills There are currently no EBPs for teaching vocational skills to individuals with ASD. In addition to designing studies according to the WWC standards, researchers should consider how interventions are categorized in order to aid practitioners in selecting relevant EBPs. Some reviews have concluded that technology aided interventions are generally effective for teaching a range of skills to adolescents and adults with ASD (Odom et al., 2015; Seaman & Cannella- Malone, 2016; Walsh et al., 2017). Technology-aided interventions includes any intervention that uses technology as the main instructional component. As technology continues to be more widely used and incorporated into regular routines, having an EBP focused solely on technology means that at some point, almost every intervention could be included in this category. There are 115 also possible unintended consequences associated with technology-aided interventions as an EBP. The current definition of technology aided interventions already utilizes a broad range of devices and instructional approaches, such as video modeling, covert audio coaching, visual prompts, specific training, speech generating devices, performance feedback, or self- management. Having such a wide range of both devices and instructional approaches means it is possible practitioners may believe they are using EBPs anytime they incorporate technology into their instruction. Having technology aided interventions as its own EBP makes it difficult to categorize interventions based on the actual instructional techniques used to teach skills, which is arguably more important than whether technology was incorporated or not. Within the vocational skills literature, the two most commonly used interventions—video modeling and covert audio cueing—both utilize technology as the main instructional component, but are very different interventions. Video modeling is a way to deliver instruction by providing a visual model of the skill, followed by an opportunity for the learner to perform the skill (Plavnick, 2013), whereas audio coaching is a method of delivering verbal instruction, prompts, or feedback (Bennett et al., 2010). With video modeling, videos can be replayed and never change, providing consistency and repetition (Bellini & Akullian, 2007), and can also be implemented without much adult support (Kellums & Morningstar, 2012; Van Laarhoven, Van Laarhoven-Myers, & Zurita, 2007). Covert audio coaching uses verbal prompting delivered via headset and requires an adult to be nearby at all times (Bennett et al.). By grouping these two interventions together, along with the many other instructional strategies that utilize technology, it becomes difficult to make informed decisions on the most effective practices for individuals with disabilities. Identifying these interventions based on their instructional components is more precise and will ultimately help practitioners select relevant EBPs. 116 Training Staff to use Evidence Based Practices In order to ensure adolescents and adults with ASD receive adequate training, it may be beneficial to first sufficiently train unskilled staff to use systematic instructional practices and EBPs. In Chapter 2, there were 12 vocational skills studies that met standards or met standards with reservations, 75% of which reported data regarding implementation fidelity. However, only one study (Goh & Bambara, 2013) provided information on staff training or technical assistance provided to support implementation of the intervention. Including information on how service providers are trained to implement interventions is important since para-professionals and job coaches are often responsible for providing vocational instruction to individuals with ASD. These individuals typically receive little specialized or formal training on EBPs (Carter, O’Rourke, Sisco, & Pelsue, 2009; Hall, Bose, Winsor, & Migliore, 2014). In Chapters 3 and 4, a training protocol was used to teach different service providers with a range of experiences and background knowledge to implement the video modeling intervention independently and with fidelity. In Chapter 3, three different behavior technicians were trained to implement the video modeling intervention to two individuals with ASD. Two of the behavior technicians were registered behavior technicians, which means they were required to complete a 40-hour training program and pass a competency assessment and written examination. These individuals were already familiar with common data collection procedures, measurement, assessment, and delivering instruction, as were used to receiving feedback on their performance. The third behavior technician only had 1 month of experience providing ABA therapy to individuals with ASD. Although the third behavior technician did not have as much experience implementing 117 interventions, he was exposed to many behavioral concepts during his initial training to become a technician. In Chapter 4, one para-professional was trained to implement the video modeling intervention with fidelity to three individuals with ASD. Although the para-professional had 2 years of experience as a one-on-one aide for an individual with ASD, he had very little experience implementing interventions independently or collecting data. The intervention implemented by the para-professional was more complex than the one carried out by the behavior technicians. The para-professional was required to implement the intervention across three participants, facilitate matrix training trials, systematically fade videos, conduct maintenance trials and generalization sessions, and collect data on eight different target behaviors. Since mastery criteria was based on individual performance, participants were often at different points in the intervention at different times. For example, in a single session, the para-professional may have been required to follow baseline procedures for one participant, implement the intervention for the third behavior pairing while fading videos for the second behavior pairing for another participant, and implement the intervention for the final behavior pairing and conduct maintenance probes for the third participant. Despite the differences in experiences and background knowledge, all facilitators were taught to implement the intervention with at least 90% accuracy during an initial 3-hour training. Facilitators were trained using a behavioral skills training approach (e.g., Sarokoff & Sturmey, 2004; Sarokoff & Sturmey, 2008; Ward-Horner & Sturmey, 2012), which included instruction, prompting, role-playing, and specific feedback on implementation accuracy during role play opportunities. This level of integrity was maintained throughout the intervention, with all facilitators averaging 92% accuracy or higher during baseline and intervention sessions across all 118 participants and studies. Maintaining high levels of fidelity across five participants and three facilitators is challenging given the differences in experience with behavioral concepts and implementation of EBPs. The results of both studies demonstrated that this may be a feasible intervention for implementation in public service settings, where providers may have less experience with EBPs. Just as using EBPs to teach individuals with ASD to learn new skills is important, utilizing empirically supported training methods to train service providers is essential to increase the likelihood that para-professionals and job coaches can implement effective interventions with fidelity. Since SCED research relies on clear descriptions of participants and procedures so that findings can be generalized and others can replicate the research, future studies should consider including descriptions of those responsible for carrying out the intervention as well as procedures used to train others to implement the intervention. Methods to Teach Vocational Skills and Work Related Social Skills Video modeling may be useful for teaching a range of skills necessary for employment. In Chapters 3 and 4, video modeling was effective in teaching multiple skills at once, increasing the efficiency of learned skills. Teaching social skills in conjunction with vocational skills is beneficial given the need for social skills in the workplace and the limited time that is devoted to teaching these skills. In supported employment, the initial job training is typically provided by the job coach, who uses behavioral strategies to teach new skills and build independence by fading supports over time which includes assessing job performance and teaching new skills. However, this initial training is often the most time intensive (Wehman et al., 2014). Video modeling may be a cost-effective intervention to support individuals with ASD because it has the potential to be incorporated into already existing practices and work place training programs. The procedures used to train the facilitators in Chapters 3 and 4 could easily be used to train job 119 coaches to carry out a similar video modeling intervention in an employment setting. Although videos were faded over time in both studies, it may have also been beneficial to fade out the facilitator and train participants to use the video modeling intervention independently. In order for individuals with ASD to be successful in competitive employment, they must be able to work independently without the use of job coaches or other individuals. Since video modeling can be delivered on a range of devices, including iPods, iPads, iPhones, personal digital assistants, and computers that are often readily accessible and socially valid, training participants to access videos and use them as needed would be a way to promote independence and reduce the need for adult delivered prompts. There are no other known studies that have used video modeling to measure two skills taught within the same video. Video modeling is a commonly used intervention for teaching a wide range of skills to individuals with ASD from pre-school aged to adults. Combining multiple skills into one video has the potential to be useful for teaching skills other than vocational and social skills. Although it is common for skills to initially be taught in isolation (i.e., discrete trial training), in more natural settings, individuals are typically expected to use multiple skills at one time. For example, in a classroom setting, a student needs to know academic skills, such as solving math problems or reading sight words, in addition to skills for success in the classroom, such as when and how to ask for assistance. Since both skills are necessary for the classroom setting, teaching these skills together may be a more efficient method to teach the range of skills a student may need to be successful in a general education classroom. Using one video to teach skills that go together may be one way to provide more opportunities for individuals to practice skills that are not typically taught (i.e., social skills) without taking away from instructional time for the skills that are typically taught. 120 Since teaching multiple skills in one video is a new concept, more research is needed to examine the most effective way for individuals to master both skills. In Chapters 3 and 4, mastery criterion was based on vocational skills to prevent participants from staying in an intervention phase for longer than necessary. The vocational skills were acquired relatively quickly, meaning participants may not have received enough teaching trials to also mastery social skills. However, due to the variability of social skills during the intervention phase for most participants, simply making mastery criterion based on their performance on social skills may not have been sufficient. Given the potential benefits to instructional methods that are both efficient and effective, future research should examine ways to use video modeling to teach multiple skills to mastery. Techniques commonly used to promote generalization were utilized in Chapter 4 and appeared to be more effective for generalizing vocational skills to a new setting and with new people when compared to social skills. Matrix training was effective for teaching some social behaviors across tasks, but it was not as effective for generalizing behaviors to a new setting. Matrix training procedures are complex and require a lot of pre-planning. In Chapter 4, baseline procedures had to be arranged so that each social behavior was paired with every vocational behavior at least once. This also limited the number of social skills trials with the paired vocational task, making it difficult to determine improvements in the social skill until future baseline probes. Given the amount of time and effort required to set up and measure matrix training trials, there may be more other methods for promoting generalization that are easier to implement. Multiple exemplar training (Stokes & Baer, 1977) provides the individual with multiple examples and practice with a variety of stimulus and response topographies. The video 121 equivalent of multiple exemplar training was used in Chapter 3 and 4 to promote generalization of vocational and social skills by including at least five variations of videos that included different materials, scenarios, and linguistic examples for each of the pairs of targeted behaviors. Another common technique, programming common stimuli (Stokes & Baer et al.), was used in Chapter 4 by including salient stimuli and materials in both the training and generalization setting. Although multiple variations of each social skill were used in the videos, Michael was the only participant who was able to consistently respond to any of the social skills using a variety of responses outside of the ones used in the video. Interestingly, participants could handle variations of vocational tasks in natural environment sessions that were not shown in videos, such as new combinations of food orders or items not practiced during intervention sessions. Since participants had more difficultly acquiring social skills, video modeling combined with daily instruction, more trials per session, and a mastery model of instruction (Plavnick et al., 2015; Stauch et al., 2018) may be a more effective method for teaching complex social skills. Once participants master these skills, methods to promote generalization of these skills may be more effective. Work Related Social Skills Recently, there has been an increased emphasis on teaching work related social skills to individuals with ASD, but many studies are of poor methodological quality. When considering procedures to teach work related social skills, researchers should look to studies that are designed according to the WWC standards. In Chapter 2, there were 11 experiments that taught social skills, but only two of these studies met the WWC standards. Dotto-Fojut and colleagues (2011) first taught participants how to complete the vocational tasks before teaching the work related social skill of asking for help. Rausa and colleagues (2016) used video modeling to teach 122 telephone behaviors, which were social skills necessary to complete the job task. Production related social skills have been identified by employers as most important to teach because they allow for completion of required work tasks and lead to greater success in the workplace (Agran et al., 2016). In Chapters 3 and 4, the social skills were selected based on previous studies that identified the skills as important for the workplace (Agran et al., 2016; Elksnin & Elksnin, 2011; Philips et al., 2014); however, the social skills were interaction based rather than production based. This means that the social skills were useful for the setting and job tasks, but participants could still complete each job task without using the social skills. Although several variables likely influenced the inconsistencies in performance across some of the social skills, it is possible that participants may have performed better on the social skills if they were required to complete job tasks. For example, if participants were unable to move on to the next step in the task analysis for the job skill without first completing the social skill correctly, they may have been more motivated to perform the social skill in order to complete the job task. In both studies, participants did not receive any additional reinforcement for engaging in the social skills and some of the skills may have been aversive (i.e., accepting criticism). Participants may not have been motivated to perform the skills since the main purpose of the social skills was to interact with their supervisor or customers. More research is needed to examine the most important work related social skills to teach. Many individuals with ASD do not receive adequate social skills training, let alone sufficient training for social skills specific for the workplace. It is unknown whether increased emphasis on training general social skills among individuals with ASD would improve one’s ability to learn work related social skills. Since there is some overlap between more general 123 social skills and work related social skills, it seems plausible that an increased knowledge of basic social skills would increase the likelihood that one could acquire work related social skills. However, it often requires a lot of explicit instruction with multiple opportunities to practice the skills in order for individuals with ASD to acquire basic social skills. The goals and outcomes outlined in each individual’s transition plan should play a larger role in the social skills selected for intervention. For example, if an individual is not interested in interacting with others, but wants a job, it may be more socially valid to teach the individual work related social skills. Due to the time required to teach social skills, it may make sense to just teach social skills that will allow individuals with ASD to obtain and maintain employment rather than social skills to obtain and maintain friends. Video modeling appears to be the most commonly used intervention to teach vocational skills to individuals with ASD. More research, designed according to the WWC standards, is needed to support this intervention as an EBP for teaching vocational skills. Results of the two experimental studies support previous research on the use of video modeling to teach vocational skills to adolescents with ASD in simulated work settings. In both studies, videos for social skills were embedded into videos for vocational tasks. All participants demonstrated an increase in social skills from baseline, although mixed results were observed across participants, indicating the need for more research to determine effective methods to teach work related social skills to individuals with ASD. The studies contribute to the current literature for teaching vocational and work related social skills by demonstrating a potentially efficient manner for teaching a range of skills necessary for the workplace. 124 APPENDICES 125 APPENDIX A WWC Design Standards Coding Form WWC Coding Form The coding manual was created by Maggin, Briesch, and Chafouleas (2013) and has been developed to assist with implementing the design standards developed by the What Works Clearinghouse (WWC) panel on evaluating single subject research (Kratochwill et al., 2010). The unit of analysis for the design standards is at the case, rather than study, level. That is, each of these design standards should be applied to individual cases within studies (e.g., individual students) resulting in as many design assessments as there are number of participants, with the exception of multiple baseline across participants design, which should be evaluated as one case. 1. Study Authors and year Enter author's last names and year 2. Title of Study 3. Study Design Include the type of experimental design used in the study (multiple baseline across participants, reversal, etc.). 4. Intervention Include the independent variable used in the study (video modeling, covert audio coaching, etc.) 126 5. Design Standard #1 (DS#1): The independent variable (i.e., the intervention) must be systematically manipulated, with the researcher determining when and how the independent variable conditions change. If this standard is not met, the study Does Not Meet Evidence Standards. Record “1” if the independent variable was systematically manipulated. Record “0” if the independent variable was not systematically manipulated. Mark only one oval. 0 1 2 7. Design Standard #2A (DS#2A): Each outcome variable must be measured systematically (i.e., repeatedly) over time by more than one assessor (i.e., IOA). Record “1” if inter-assessor agreement was reported. Record “0” if inter-assessor agreement was not reported. Mark only one oval. 0 1 2 8. Design Standard #2B (DS#2B): The study needs to collect inter-assessor agreement on at least twenty percent of the sessions. Record “1 if inter-assessor agreement was collected on at least 20% of sessions. Record “0” if inter- assessor agreement was not collected on at least 20% of sessions. Mark only one oval. 0 1 2 127 9. Design Standard #2C (DS#2C): The inter-assessor agreement must meet minimal thresholds. Inter-assessor agreement (commonly called interobserver agreement) must be documented on the basis of a statistical measure of assessor consistency. Although there are more than 20 statistical measures to represent inter-assessor agreement (see Berk, 1979; Suen & Ary, 1989), commonly used measures include percentage agreement (or proportional agreement) and Cohen’s kappa coefficient (Hartmann, Barrios, & Wood, 2004). According to Hartmann et al. (2004), minimum acceptable values of inter-assessor agreement range from 0.80 to 0.90 (on average) if measured by percentage agreement and at least 0.60 if measured by Cohen’s kappa. Regardless of the statistic, inter-assessor agreement must be assessed for each case on each outcome variable. If this standard is not met, or if inter-assessor agreement is not reported, the study Does Not Meet Evidence Standards. Record “1 if inter-assessor agreement did meet the minimum thresholds listed above. Record “0” if inter-assessor agreement did not meet the minimum thresholds listed above. Mark only one oval. 0 1 2 10. Design Standard #3 (DS#3): The study must include at least three attempts to demonstrate an intervention effect at three different points in time or with three different phase repetitions. Eligible phase contrasts must be between common phases (e.g., group contingency and baseline or group contingency and another intervention). An attempt to demonstrate a treatment effect refers explicitly to phase contrasts that are adjacent (e.g., AB). A minimum of three such contrasts must be present in the study to meet this standard. If this standard is not met, the study Does Not Meet Evidence Standards. Examples of designs meeting this standard include ABAB designs, multiple baseline designs with at least three baseline conditions, alternating/simultaneous treatment designs with either at least three alternating treatments compared with a baseline condition or two alternating treatments compared with each other, changing criterion designs with at least three different criteria, and more complex variants of these designs. Examples of designs not meeting this standard include AB, ABA, and BAB designs. * Record a “1” if the study included at least three attempts to demonstrate an intervention effect at three different points in time. Record a “0” if the study did not include at least three attempts to demonstrate an intervention effect at three different points in time. Mark only one oval. 0 1 2 128 11. Design Standard #4 (DS#4): For a phase to qualify as an attempt to demonstrate an effect, the phase must have a minimum of three data points. To Meet Standards a reversal /withdrawal (e.g., ABAB) design must have a minimum of four phases per case with at least 5 data points per phase. To Meet Standards with Reservations a reversal /withdrawal (e.g., ABAB) design must have a minimum of four phases per case with at least 3 data points per phase. Any phases based on fewer than three data points cannot be used to demonstrate existence or lack of an effect. To Meet Standards a multiple baseline design must have a minimum of six phases with at least 5 data points per phase. To Meet Standards with Reservations a multiple baseline design must have a minimum of six phases with at least 3 data points per phase. Any phases based on fewer than three data points cannot be used to demonstrate existence or lack of an effect. * Record a “2” if the case meets the standards for reversal or multiple baseline designs described above. Record a “1” if the case meets the standards with reservations for reversal or multiple baseline designs described above. Record a “0” if the case does not meet the standards with reservations for reversal or multiple baseline designs described above. Mark only one oval. 0 1 2 12. Design Standard Meet (DS_MEET): Review your responses on the following items to determine whether the study has met design standards, met design standards with reservations, or has not met design standards. If the study scored a 0 on questions 1, 2, 3, or 4, this study does not meet standards. If the study scored 1 on question 1-4, this study meets standards with reservations. If the study scored 1 on questions 1-3 and a 2 on question 4, the study meets design standards. Record a “2” if the case meets design standards. Record a “1” if the case meets design standards with reservations. Record a “0” if the case does not meet design standards. Mark only one oval. 0 1 2 13. Comments Enter comments related to why the study did not meet criteria, questions, or if you are unsure of any of your responses. Multiple Baseline Design Additional Criteria The following coding manual has been developed to assist with implementing the design standards developed by the What Works Clearinghouse (WWC) panel on evaluating single-subject research (Kratochwill et al., 2010). The unit of analysis for the design standards is at the case, rather than study, level. That is, each of these design standards should be applied to individual cases within studies (e.g., individual students) resulting 129 in as many design assessments as there are number of participants, with the exception of multiple baseline across participants design, which should be evaluated as one case. 1. Study Authors and year Enter author's last names and year 2. Title of Study Additional Criteria for Multiple Baseline Designs 3. Did all cases have baseline data before the intervention was administered to the first case (i.e., overlapping baselines)? There must be at least three cases with concurrent baseline phases. Mark only one oval. Yes No Uncertain 4. Do graphs present the timing of sessions consistently across all cases in the design (e.g., does Session 1 for all cases occur at the same time)? * Mark only one oval. Yes No Uncertain 130 5. If the design did not collect data during an intervention or training phase, is the design appropriate given the relevant research question? This design is appropriate if the intervention is expected to produce an effect when all of the intervention components have been implemented; it is inappropriate if the intervention is expected to produce immediate effects when the intervention is implemented. Mark only one oval. Yes No Uncertain Not applicable * Can you determine that the intervention or training phases for each case do not overlap? If so, when any case has a probe after completing training (i.e., the next probe after training or intervention ends), do the other cases still in the baseline phase have continuing baseline measurement at or after that point? Mark only one oval. Yes No Uncertain 5. Does the study meet all of the additional criteria for multiple baseline designs? Mark only one oval. Yes No Uncertain 6. Comments Enter comments related to why the study did not meet criteria, questions, or if you are unsure of any of your responses. 131 Multiple Probe Design additional criteria The following coding manual has been developed to assist with implementing the design standards developed by the What Works Clearinghouse (WWC) panel on evaluating single-subject research (Kratochwill et al., 2010). The unit of analysis for the design standards is at the case, rather than study, level. That is, each of these design standards should be applied to individual cases within studies (e.g., individual students) resulting in as many design assessments as there are number of participants, with the exception of multiple baseline across participants design, which should be evaluated as one case. 1. Study Authors and year Enter author's last names and year 2. Title of Study 6. Participant Name (n/a if MPD across participants) Design Standard 4 Review the two questions below to determine if the study does not meet criteria or meets with or without reservations 7. Do initial baseline sessions for each case overlap vertically? Within the first 3 baseline sessions, the design must include 3 consecutive probe points for each case to Meet WWC Single-Case Design Standards without Reservations or at least 1 probe point for each case to Meet WWC Single-Case Design Standards with Reservations. Mark only one oval. Yes, without reservations Yes, with reservations No Uncertain 132 8. Are probe points available just before introducing the independent variable? Within the 3 sessions just before introducing the independent variable, the design must include 3 consecutive probe points for each case to Meet WWC Single-Case Design Standards without Reservations or at least 1 probe point for each case to Meet WWC Single-Case Design Standards with Reservations. Mark only one oval. Yes, without reservations Yes, with reservations No Uncertain 9. Design Standard #4 (DS#4): Use the criteria from the two questions above to determine if the study meets or meets with reservations. Record a “2” if you answered, "Yes, without reservations" to the two questions above. Record a “1” if you answered "yes, with reservations" to at least 1 of the above questions. Record a 0 if you answered No to either of the questions above. Mark only one oval. 0 1 2 Additional Criteria for Multiple Probe Designs 10. Did all cases have baseline data before the intervention was administered to the first case (i.e., overlapping baselines)? There must be at least three cases with concurrent baseline phases. Mark only one oval. Yes No Uncertain 11. Do graphs present the timing of sessions consistently across all cases in the design (e.g., does Session 1 for all cases occur at the same time)? Mark only one oval. Yes No Uncertain 12. Can you determine that the intervention or training phases for each case do not overlap? If so, when any case has a probe after completing training (i.e., the next probe after training or intervention ends), do the other cases still in the baseline phase have continuing baseline measurement at or after that point? Mark only one oval. 133 Yes No Uncertain 13. Does each case not receiving the intervention have a probe point in a session in which another case either (1) first receives the intervention or (2) reaches the prespecified intervention criterion? (i.e., a predetermined performance level, such as three consecutive sessions with at least 80 percent of time on task or three consecutive probes with correct responses)? * Mark only one oval. Yes No Uncertain 14. Design Standard Meet (DS_MEET): Review your responses on the following items to determine whether the study has met design standards, met design standards with reservations, or has not met design standards. If the study scored a 0 on questions 1, 2, 3, or 4, this study does not meet standards. If the study scored 1 on question 1-4 and met the additional criteria, this study meets standards with reservations. If the study scored 1 on questions 1-3, 2 on question 4, and a yes on all the additional criteria for multiple probe designs, the study meets design standards. Record a “2” if the case meets design standards. Record a “1” if the case meets design standards with reservations. Record a “0” if the case does not meet design standards. Mark only one oval. 0 1 2 14. Comments Enter comments related to why the study did not meet criteria, questions, or if you are unsure of any of your responses. 134 APPENDIX B WWC Evidence Standards Coding Form The coding manual was created by Maggin, Briesch, and Chafouleas (2013). Evidence Standards- Visual analysis For use with all SCED cases that meet standards or meet standards with reservations. Authors and year Title of study Study Design Mark only one oval. Multiple probe design Multiple baseline design Changing criterion design Alternating treatment design Reversal/withdrawal Participant name (or # of participants with ASD) Design Rating Mark only one oval. 2=Meets standards 1=Meets standards with reservations Baseline Analysis Page 43 Do the data in the first baseline phase for each case document that the proposed concern is demonstrated? (e.g., low levels of performance) Mark only one oval. 135 Yes No Uncertain Do the data in the first baseline phase for each case document there is sufficient demonstration of a clearly defined baseline pattern that can be used to assess the effects of an intervention (e.g., variability is sufficiently consistent, stable trend or trend moving away from therapeutic direction). **For alternating treatment designs, select N/A. Mark only one oval. Yes No N/A Uncertain Baseline Data Rating. If the outcome data meet both of these conditions, select “Yes”. If not, select “No.” If outcome data do not meet these conditions, skip to the “Rate the Evidence” step below and select “No Evidence”. If the baseline data for any tier in a multiple baseline or multiple probe design meet these conditions, complete the remaining steps of the visual analysis. This is necessary to accurately determine the number of demonstrations of an effect and demonstration of a non-effect. Mark only one oval. Yes No Uncertain Baseline--Supporting information, concerns, or questions: Describe the data pattern or other evidence that supports your answer or list uncertainties or questions. Within Phase Analysis Compare consistency of data patterns (i.e., level, trend, and variability) within phases of the same condition. Are the data patterns in phases of the same condition consistent? **For alternating treatment designs select NA. Mark only one oval. Yes No N/A Uncertain 136 Within Phase--Supporting information, concerns, or questions: Describe the data pattern or other evidence that supports your answer or list uncertainties or questions. Between Phase Comparison Compare level, trend, and variability for adjacent phases. Are there differences in the level, trend, and variability of data between each phase and the next phase? **For alternating treatment designs, select N/A. Mark only one oval. Yes No N/A Compare overlap and immediacy of effect for adjacent phases. Is there an immediate effect and an absence of overlap between each phase and the next phase? Uncertain Mark only one oval. Yes No Uncertain For Multiple Baseline and Multiple Probe Designs Only Page 45. All other designs, skip these questions. MBD/MPD: Vertically compare each baseline series to determine if changes in data patterns in each tier are independent of one another. When one case either first receives the intervention or reaches a pre-specified condition, are the probes for other cases not receiving the intervention consistent in level and trend with their previous baseline points? Mark only one oval. Yes No Uncertain 137 MBD/MPD: Is the time in which an intervention effect is initially demonstrated with one series (e.g., the first five days following introduction of the intervention for participant 1) independent of changes in the data pattern over the same time frame in the other series of the design? Mark only one oval. Uncertain Yes No For Alternating Treatment Designs Only Page 46. All other designs, skip this question. AT design: Compare the overall mean levels for the intervention and comparison conditions. The overall mean should include all points, including outliers, for each condition. Do they clearly demonstrate a visual effect? Mark only one oval. No Yes Uncertain Between Phase Summary Between Phase--Supporting information, concerns, or questions: Describe the data pattern or other evidence that supports your answer or list uncertainties or questions. Rate the evidence Data points per phase. All phases considered must have a minimum number of data points. Studies with strong evidence must have at least five data points per phase; studies with moderate evidence must have 3 or 4 data points per phase; and studies with no evidence will have less than 3 data points per phase. Were there enough data points per phase? Mark only one oval. 0 -Less than three data points for any phase = No Evidence 1 - Three or Four data points for any phase = Moderate Evidence 138 2 - Five or more data points for all phases = Strong Evidence There must be at least three demonstrations of a treatment effect. Based on visual analysis questions above (e.g., within and between phase comparisons), were there at least three demonstrations of a treatment effect? Strong evidence: At least three effects and no non- effects. Moderate evidence: At least three effects, but also either (1) at least one example of a non-effect or (2) multiple baseline data or multiple probe data suggesting that an effect demonstrated in one series is associated with change in the data pattern for another series determining the overall evidence rating. No evidence: (1) Fewer than three effects at different points in time or (2) the baseline data do not demonstrate the proposed concern. Alternating treatment designs can only show strong or moderate evidence of a causal effect if there are no clear effects in the opposite direction and the overall means clearly demonstrate a visual effect. If there are clear effects in the opposite direction or the overall means do not clearly demonstrate a visual effect, there is no evidence of a causal effect. Mark only one oval. 0 = No evidence 1= Moderate evidence 2 = Strong evidence Provide the total number of demonstrations of an effect and demonstrations of a non-effect for each experiment, participant, or outcome. Record as number of effects: number of non- effects (e.g., 4:1). Leave blank if the baseline data do not demonstrate the proposed concern. The presence of non-effects can compromise the strength of a study. The WWC criteria require at least three demonstrations of a treatment effect to provide evidence. If there are no instance of non-effect then the study demonstrates strong evidence; if these three demonstrations are accompanied by one instance of non-effect then the study demonstrates moderate evidence; if the ratio of effects to non-effects is greater than 3:1 (e.g., 2:1) then the study provides no evidence. (e.g., You are allowed one mistake to not get dinged out). Overall Evidence. If a “0” was entered for any of the previous items, then the study demonstrates no evidence. If a “1” was entered for any of the previous items, then the study demonstrates moderate evidence.If all responses were “2” then the study demonstrates strong evidence. 1 = The ratio of effects to non-effects is less than or equal to 3:1. 0 = The ratio of effects to non-effects is greater than 3:1. 2 = No instances of non-effects Mark only one oval. Mark only one oval. 0 = No Evidence 1 = Moderate Evidence 2 = Strong Evidence 139 Overall Evidence--Supporting information, concerns, or questions: Describe the data pattern or other evidence that supports your answer or list uncertainties or questions. 140 APPENDIX C WWC Study Characteristics Coding Form Study Characteristics Authors and year Study Design Mark only one oval. Multiple probe design Multiple baseline design Reversal/withdrawal Changing criterion Alternating treatment * Sample size. Include the number of participants diagnosed with ASD. * Participant ages. * Participant diagnoses. * Participant races. * Participant gender. 141 Other relevant participant characteristics (e.g., additional diagnoses, descriptions) Setting. Describe the location(s) of the experiment (e.g., classrooms, warehouse, school, office building) Brief description of independent variable (e.g., intervention). It should include the length of the intervention and the dosage (including number of days, weeks, or months; number of sessions; and time/trials per session) as well as information about the content and delivery of the intervention. Brief description of baseline condition. It should include the number of days, weeks, or months; number of sessions; and time/trials per session. Include procedures used in the baseline condition. Describe any maintenance phases (after the intervention has ended), including the timing, condition, duration, and data trends. 142 Target behaviors. Describe each eligible outcome (dependent variable). If outcomes vary by case, specify which outcomes go with each case or SCD experiment. Indicate how each eligible outcome was collected and measured. Implementation fidelity. Describe the steps the researcher took to ensure fidelity in baseline and intervention procedures and the level obtained. Support for implementation. Indicate both the staff training and technical assistance conducted to support the implementation of the intervention. Other (social validity, other relevant information) 143 APPENDIX D Parent Informed Consent Form 8/10/16 Dear Parent or Guardian, We are conducting a research study to identify new ways to teach vocational skills and social skills to adolescents with autism. Many adults with ASD do not regularly work and struggle with social skills in work situations. We hope to provide information about how adolescents learn job related skills, interact with others, and how they can learn these skills more effectively. Your child was selected as a possible participant because he/she has been diagnosed with autism spectrum disorder (ASD) and receives job training as a part of his or her school programming. This letter is to request your permission for your child to be included in the study and for the results to be used as data in our study on the effectiveness of this procedure. By allowing your child to participate in this study, you are giving permission for your child’s teacher to provide information to researchers about the ways your child interacts with others and for observations of your child. Each observation and session will last approximately 1-2 hours and will occur 3 times a week for a period of approximately 3-4 months. The purpose of the observations is to assess how your child responds to a teaching procedure known as Video Modeling, where your child will be shown a video clip of a skill and then given an opportunity to perform the skill. Teachers and para-professionals are being trained to implement video modeling procedures. This teaching procedure is based on research demonstrating effective practices for individuals with autism. Your child will be given instructions in job skills needed to run a concession stand for school staff as well as the social skills necessary to interact with peers, supervisors, and customers. All data collected during this study will be kept in a locked and secure location. Access to data will only be given to the researchers and the MSU Institutional Review Board. This procedure is for research purposes and is free of charge and the results will be provided to you upon request. The potential benefits to your child for taking part in this study are that he or she may learn new vocational skills related to the food service industry and social skills for interacting with others. During the research activities, your child will have opportunities to interact with adult employees, researchers, and peers in a fun and safe manner. Additionally, attempts to increase job-related skills and social behaviors will be made. Allowing your child to participate will help us to improve this procedure to make it more helpful in the future. The potential risks for your child participating in this study include the possibility that engaging in a new activity may be distressing for them. However, this is not expected as the procedures are designed to be fun for your child. The only other known risks for your child have to do with privacy and confidentiality, which will be protected to the greatest extent allowable by law. The data for this project will be kept confidential unless there is a danger to yourself, your child, or others. Data will be stored in a locked file cabinet in a locked office or on encrypted computers. Members of the research team will be the only people to have access to data. The results of this study may be published or presented at professional meetings, but the identities of 144 all research participants will remain anonymous. You can indicate your permission for your child to participate by signing the letter and returning it to your child’s teacher. If, after you sign and return the letter, you change your mind, simply let your child’s teacher know and your child will not be asked to participate. There is no penalty for refusing to participate. With your consent, we also plan to video record research activities. Your confidentiality will be protected to the maximum extent allowable by law. All video recordings will be collected and stored in a locking file cabinet in a locked office in the College of Education at MSU or on encrypted computers or flash drives with password access for members of the research team only. If you consent, some of the activities will be recorded and reviewed at a later date by members of the research team only. You can indicate your permission for video recording by checking the appropriate boxes following the signature line. There is no penalty for not providing consent for recording procedures and your child can still participate in the study if you do not agree to recording. If you have any questions at any time please feel free to contact me by email at kaidtiff@msu.edu. If you have concerns or questions about this study, such as scientific issues, how to do any part of it, or to report an injury, please contact the researcher Tiffany Stauch: email kaidtiff@msu.edu, or by regular mail at Counseling, Educational Psychology, and Special Education Room 447 Erickson Hall Michigan State University East Lansing, MI 48824. Also feel free to contact the faculty member supervising this research project, Dr. Joshua Plavnick: (517) 432-8346, email plavnick@msu.edu, If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at Olds Hall, 408 West Circle Drive #207, MSU, East Lansing, MI 48824. Sincerely, Tiffany Stauch, MA, BCBA Doctoral Candidate Erickson Hall room 447 Counseling, Educational Psychology, and Special Education Michigan State University East Lansing, MI 48824 kaidtiff@msu.edu 145 PERMISSION FOR CHILD’S PARTICIPATION I consent to the participation of my child in the research project entitled “Video Modeling and Matrix Training: Teaching and Generalizing Social Skills across Vocational Tasks to Adolescents with Autism Spectrum Disorder”. I have read the attached letter and the project has been thoroughly explained to me. I acknowledge that I have had the opportunity to obtain additional information regarding the project and that any questions I have raised have been answered to my full satisfaction. I also authorize the release of records (IEP and educational assessments) to Tiffany Stauch for the purposes of the research study. I understand that my consent for the release of records is voluntary and I can withdraw my consent at any time in writing. Should I withdraw my consent, it does not apply to information that has already been provided under the prior consent for release. Furthermore, I understand that I am free to withdraw my consent at any time and to discontinue participation in the project without prejudice. Finally, I acknowledge that I have read the consent form. I sign it freely and voluntarily. A copy has been given to me. Child’s Name: ____________________________ Age: _________ Relationship to child: _______________________________ Signed: I agree to allow video recording of research activities: (Parent or guardian) Date: Yes No Initials____________ 146 APPENDIX E Research Participant Assent Form We are here because we are conducting a research study. Some important information about the research is listed below: Study Title: Video Modeling and Matrix Training: Effects on Acquisition and Generalization of Social Skills across Vocational Tasks by Adolescents with Autism Spectrum Disorder Researcher and Title: Tiffany Stauch, Special Education Doctoral Candidate Department and Institution: Dept. of Counseling, Educational Psychology, & Special Education at Michigan State University Address and Contact Information: 620 Farm Lane, Room 447; Erickson Hall; East Lansing, MI; 48824. Email: kaidtiff@msu.edu These are some things we want you to know about research studies: Your parent needs to give permission for you to be in this study. You do not have to be in this study if you don’t want to, even if your parent has already given permission. You may stop being in the study at any time. If you decide to stop, no one will be angry or upset with you. You were asked to be in this study because your teachers and parents thought it might help you make friends. Sometimes good things happen to people who take part in studies, and sometimes things happen that they may not like. We will tell you more about these things below. What are you being asked to do? The research will take place in a building where adults work. You will receive 1 on 1 therapy with an adult and you will work with 2 other peers. You will be taught how to run a concession stand to sell snack items to adults. You will work 2-3 days a week for 2-3 hours at a time. If you decide to participate, it will last about 4 months. What are the good things that might happen? You could learn new skills that could help you get a job. You could also learn ways to interact with others. What bad things could happen? It could be difficult to practice some of the new skills, but we don’t think any bad things will happen if you decide to be in the study. Who will be told about the things we learn about you? We will not tell anyone what you tell us or what we learn about you without your permission unless there is something that could be dangerous to you or someone else. Do you have to be part of the study? You don’t have to be in the study if you don’t want to. You can also stop participating at any time and nobody will be upset with you. Nothing bad will happen if you choose not to be in the study. 147 Does it cost anything to be in the study? It does not cost any money to be in this study. Do you get anything for being in the study? You will not get anything for being in the study. WHO SHOULD YOU ASK IF YOU HAVE QUESTIONS If you have questions you should talk to the person listed at the top of this form. If you have any other questions or concerns about your rights while you are in this study you may contact the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at Olds Hall, 408 West Circle Drive #207, MSU, East Lansing, MI 48824. IF YOU SIGN YOUR NAME BELOW, IT MEANS YOU AGREE TO TAKE PART IN THIS STUDY ________________________________________ Sign your name to be in the study ________________________________________ Print your name to be in the study ________________________________________ Signature of research team member obtaining assent Date Date 148 APPENDIX F Baseline Procedural Integrity Checklist Tech: _________________ Participant ID: _______________ Date: ____________________ Observer: ________________ Time: __________________ IOA score: ________________ Percent of Items Scored as + (total items marked + / total items – items marked as NA) PI score:_________________ KEY: + Happened; --Did not happen; n/a if not applicable PREPARATION FOR SESSIONS Environmental Arrangement 1. Data collection materials available and prepared • Data sheet, pen, clipboard • Participant name and date on data sheet 2. Teaching materials prepared and available • Lesson plan/procedures • Target behavior definitions • Checklists for restock • Cash register materials • Food/drink items set out/prepared • Change cheat sheet available BASELINE PROBE SESSIONS Cash Register 3. Trials were initiated by engaging in designated antecedent behavior (when customer approached counter) 4. Two social probes conducted during each vocational trial (each social skill probed 1 time) 5. Participant receive 2 opportunities to engage in vocational skill 6. No feedback/prompts provided for correct/incorrect responses 7. Discontinues trial according to criteria ((a) made continuous mistakes on a step for 15 s; (b) was non- responsive for 15 s on any step; or (c) verbally indicated he was done with the task Check Comments +/-/na Check Comments +/-/na IOA DATA COLLECTION Cash Register 9. Waits 5-10 s and then asks customer if he or she is ready T1 + or - T2 + or - Comments to order 10. Touches icons for items ordered on iPad 11. Correctly repeats customer order 12. Correctly fills out customer order sheet 13. Tells customer price of order 14. Takes card or cash 15. Completes payment 16. Hands correct change or card back to customer % of steps correct Maintains small talk When another person asks the participant a question or makes an open-ended comment, the participant (a) answers the question or acknowledges the statement and (b) asks the 149 person an on topic question or makes an on topic comment that invites a response from the person Accepting a compliment When another person compliments the participant, he (a) thanks the person for the compliment and (b) makes a comment about the content of the compliment. Accepting criticism When another person criticizes the participant’s work, he (a) apologizes for what he did incorrectly or states he agrees with the problem, and (b) requests or provides a solution to prevent the problem from occurring in the future. Fulfilling customer requests When an adult asks a question or makes a request related to the participant’s job, the participant will either (a) correctly answer the question; (b) vocally respond to indicate he heard the request and completes the request, or if the participant does not know the answer; (c) vocally responds to indicate he heard the request, seeks out the answer, and relays the correct information. BASELINE PROBE SESSIONS Complete orders 8. Trials were initiated by engaging in designated antecedent behavior (when customer approached counter) 9. Two social probes conducted during each vocational trial (each social skill probed 1 time) 10. Participant receive 2 opportunities to engage in vocational skill 11. No feedback/prompts provided for correct/incorrect responses 12. Discontinues trial according to criteria ((a) made continuous mistakes on a step for 15 s; (b) was non- responsive for 15 s on any step; or (c) verbally indicated he was done with the task Check Comments IOA DATA COLLECTION Complete Orders 1. Correctly reads ordering sheet 2. Gathers appropriate supplies needed to complete order T1 + or - T2 + or - Comments for item 1 3. Prepares first item consistent with order (n/a if grab and go item) 4. Gathers appropriate supplies for item 2 (or n/a) 5. Prepares second item consistent with order 6. Hands drink or food items to customer 7. Cleans up and pus away food prep materials % of steps correct Social Skills: 2 probed per vocational trial (see cash register IOA for definition) Maintains small talk Accepting a compliment Accepting criticism Fulfilling customer requests 150 BASELINE PROBE SESSIONS Cleaning Surfaces 13. Trials were initiated by engaging in designated antecedent behavior (told participant to clean work area) 14. Environment arranged so some steps need to be done (e.g trash on table; portable items on table) 15. Two social probes conducted during each vocational trial (each social skill probed 1 time) 16. Participant receive 2 opportunities to engage in skill 17. No feedback/prompts provided for correct/incorrect responses 18. Discontinues trial according to criteria ((a) made continuous mistakes on a step for 15 s; (b) was non- responsive for 15 s on any step; or (c) verbally indicated he was done with the task Check Comments IOA DATA COLLECTION Clean surfaces 1. Gathers supplies; multi-surface spray cleaner and dish T1 + or - T2 + or - Comments rag or paper towel 2. Removes all portable items from surface 3. Sprays cleaner on surface 4. Wipes clean with dish rag or paper towel 5. Checks counter for any missed spots or excess spray and wipes again if necessary 6. Wrings out rag or throws paper towel in the trash 7. Returns portable items to original location on surface 8. Returns cleaning items to storage % of steps correct Social Skills: 2 probed per vocational trial (see cash register IOA for definition) Maintains small talk Accepting a compliment Accepting criticism Fulfilling customer requests BASELINE PROBE SESSIONS Restock supplies 19. Trials were initiated by engaging in designated antecedent behavior (told participant to restock 2 supplies) 20. Environment arranged so one item needs restocked and the other does not 21. Two social probes conducted during each vocational trial (each social skill probed 1 time) 22. Participant receive 2 opportunities to engage in skill 23. No feedback/prompts provided for correct/incorrect responses 24. Discontinues trial according to criteria ((a) made continuous mistakes on a step for 15 s; (b) was non- responsive for 15 s on any step; or (c) verbally indicated he was done with the task Check Comments 151 IOA DATA COLLECTION Restock Supplies 1. Checks supply levels for items 2. Gathers additional items from storage for the item that is T1 + or - T2 + or - Comments below the fill line 3. Does not gather supplies for the item that is above the fill 4. line Items are neatly placed in containers (e.g., organized in same direction, stacked on top of one another or next to one another with nothing sticking out) to fill line 5. Returns any extra items to storage 6. Draws checkmarks on supply list to indicate which items were restocked % of steps correct Social Skills: 2 probed per vocational trial (see cash register IOA for definition) Maintains small talk Accepting a compliment Accepting criticism Fulfilling customer requests 152 APPENDIX G Video Modeling Procedural Integrity Checklist Tech: _________________ Participant ID: _______________ Observer: ________________ Time: __________________ Percent of Items Scored as + (total items marked + / total items – items marked as NA) _________________ KEY: + Happened; --Did not happen; n/a if not applicable Date: ____________________ Behavior pairing: ________________ PREPARATION FOR SESSIONS Environmental Arrangement 1. Ipad at least 25% charged 2. Data collection materials available and prepared • Data sheet, pen, clipboard • Participant id and date on data sheet 3. Teaching materials prepared and available • Lesson plan/procedures • Target behavior definitions • Checklists for restock • Cash register materials • Food/drink items set out/prepared VM TRAINING Trial #1 Vocational Skill: ___________________ 4. Gains attention when labeling behavior and instructs participant to watch video closely 5. Video presented step by step. Technician does not talk over video. 6. Behavior specific praise provided for watching segment of video at least once. 7. Trials initiated by engaging in designated antecedent behavior (VS 1/2: when customer approached counter; VS 3/4: when tech tells them to clean/restock) 8. Follows error correction procedure for incorrect responses (interrupt, show video segment, present trial again). If correct, moved on, if still incorrect, told what to do differently next time. Social Skill: _____________________ 9. Trial initiated by engaging in designated antecedent behavior during vocational trial 10. Follows error correction procedure for incorrect responses (same as above) VM TRAINING Trial #2 11. Gains attention when labeling behavior and instructs participant to watch video closely 12. Video clip presented step by step. Technician does not talk over video. 13. Behavior specific praise provided for watching segment of video at least once. 153 Check Comments +/-/na Check Comments +/-/na Put a check every time error correction procedure used. Check Comments 14. Shows different video clip from previous trial. 15. Trials initiated by engaging in designated antecedent behavior (VS 1/2: when customer approached counter; VS 3/4: when tech tells them to clean/restock) 16. Follows error correction procedure for incorrect responses (interrupt, show video segment, present trial again). If correct, moved on, if still incorrect, told what to do differently next time. 17. Uses scenarios/materials different from previous trial. Put a check every time error correction procedure used. Social Skill 18. Trial initiated by engaging in designated antecedent +/-/na Comments behavior during vocational trial 19. Varies scenario from previous trial 20. Follows error correction procedure for incorrect responses (same as above) VM TRAINING Trial #3 21. Gains attention when labeling behavior and instructs participant to watch video closely 22. Video clip presented step by step. Technician does not talk over video. 23. Behavior specific praise provided for watching segment of video at least once. 24. Shows different video clip from previous trial. 25. Trials initiated by engaging in designated antecedent behavior (VS 1/2: when customer approached counter; VS 3/4: when tech tells them to clean/restock) 26. Follows error correction procedure for incorrect responses (interrupt, show video segment, present trial again). If correct, moved on, if still incorrect, told what to do differently next time. 27. Uses scenarios/materials different from previous trial. Check Comments Put a check every time error correction procedure used. Social Skill 28. Trial initiated by engaging in designated antecedent +/-/na Comments behavior during vocational trial 29. Varies scenario from previous trial 30. Follows error correction procedure for incorrect responses (same as above) VM TRAINING Trial #4 31. Gains attention when labeling behavior and instructs participant to watch video closely 32. Video clip presented step by step. Technician does not talk over video. 33. Behavior specific praise provided for watching segment of video at least once. 34. Shows different video clip from previous trial. Check Comments 154 35. Trials initiated by engaging in designated antecedent behavior (VS 1/2: when customer approached counter; VS 3/4: when tech tells them to clean/restock) 36. Follows error correction procedure for incorrect responses (interrupt, show video segment, present trial again). If correct, moved on, if still incorrect, told what to do differently next time. 37. Uses scenarios/materials different from previous trial. Put a check every time error correction procedure used. Social Skill 38. Trial initiated by engaging in designated antecedent +/-/na Comments behavior during vocational trial 39. Varies scenario from previous trial 40. Follows error correction procedure for incorrect responses (same as above) VM TRAINING Trial #5 41. Gains attention when labeling behavior and instructs participant to watch video closely 42. Video clip presented step by step. Technician does not talk over video. 43. Behavior specific praise provided for watching segment of video at least once. 44. Shows different video clip from previous trial. 45. Trials initiated by engaging in designated antecedent behavior (VS 1/2: when customer approached counter; VS 3/4: when tech tells them to clean/restock) 46. Follows error correction procedure for incorrect responses (interrupt, show video segment, present trial again). If correct, moved on, if still incorrect, told what to do differently next time. 47. Uses scenarios/materials different from previous trial. Check Comments Put a check every time error correction procedure used. Social Skill 48. Trial initiated by engaging in designated antecedent +/-/na Comments behavior during vocational trial 49. Varies scenario from previous trial 50. Follows error correction procedure for incorrect responses (same as above) Maintenance-Vocational Skills and Social Skills Skills mastered: 51. Follows video fading procedures OR Conducts at least 1 +/-/na Comments maintenance probe for previously mastered vocational skills 1 time per week 52. Follows error correction procedures if participant performs skill below 80% for 2 sessions (shown video for skill before next session) 53. Conducts at least 1 maintenance probe for previously mastered social skills 1 time per week 155 Matrix Training-Social Skills Skills mastered: 54. Conducts at least 2 matrix probes for previously +/-/na Comments mastered social skills while participant performs current job task 55. Follows error correction procedure if participant performs skill incorrectly or if performance drops below 50% for 2 sessions 156 APPENDIX H Data Collection Sheet Working the Cash Register Condition: Video prompting Maintenance Generalization Working the Cash Register 17. Waits 5-10 s and then asks customer if he or she is ready to order T1 T2 T3 T4 T5 18. Touches icons for items ordered on iPad 19. Correctly repeats customer order 20. Correctly fills out customer order sheet 21. Tells customer price of order 22. Takes card or cash 23. Completes payment 24. Hands correct change or card back to customer % of steps correct **Maintains small talk** Fulfilling customer requests Accepting a compliment Accepting criticism Video shown? Completing Customer Orders Condition: Video prompting Maintenance Generalization T1 T2 T3 T4 T5 Completing Customer Orders 1. Correctly reads ordering sheet 2. Gathers appropriate supplies needed to complete order for item 1 3. Prepares first item consistent with order (n/a if grab and go) 4. Gathers appropriate supplies for item 2 (if 2nd item ordered) 5. Prepares second item consistent with order (if 2nd item ordered) (n/a if grab and go) 6. Hands drink or food items to customer 7. Cleans up and puts food prep materials away % of steps correct Maintains small talk **Fulfilling customer requests** Accepting a compliment Accepting criticism Video shown? 157 Condition: Cleaning Surfaces Video prompting Maintenance Generalization Cleaning Surfaces 1. Gathers supplies; multi-surface spray cleaner and dish rag or T1 T2 T3 T4 T5 paper towel 2. Removes all portable items from surface and throws away trash, if any 3. Sprays cleaner on surface 4. Wipes clean with dish rag or paper towel 5. Checks counter for any missed spots or excess spray and wipes again if necessary 6. Shakes out rag over trash or throws paper towel in the trash 7. Returns portable items to original location on surface 8. Returns cleaning items to storage % of steps correct Maintains small talk Fulfilling customer requests **Accepting a compliment** Accepting criticism Video shown? Condition: Restocking Supplies Video prompting Maintenance Generalization Restocking Supplies 1. Checks supply levels for items 2. Gathers additional items from storage for the item that is below 50% or below the fill line 3. Does not gather supplies for the item that is above 50% or above the fill line 4. Items are neatly placed in containers (e.g., organized in same direction, stacked on top of one another or next to one another with nothing sticking out) to fill line or until no more will fit 5. Returns any extra items to storage 6. Draws checkmarks on supply list to indicate which items were restocked % of steps correct Maintains small talk Fulfilling customer requests Accepting a compliment **Accepting criticism** Video shown? 158 T1 T2 T3 T4 T5 APPENDIX I Social Validity Survey Intervention Rating Profile The purpose of this questionnaire is to obtain information that will aid in the selection of classroom interventions. Teachers of children with autism will use these interventions to teach a variety of skills, including vocational and social skills. Please check the box which best describes your agreement or disagreement with each statement. Strongly Disagree Disagree Slightly Disagree Slightly Agree Agree Strongly Agree 1. This was an acceptable intervention for teaching vocational and social skills. 2. Most teachers would find this intervention appropriate for teaching new skills in addition to the ones described. 3. This intervention should prove effective in teaching vocational and social skills. 4. I would suggest the use of this intervention to other teachers. 5. The students’ vocational and social skill deficits were severe enough to warrant use of this intervention. 6. Most teachers would find this intervention suitable for teaching social or vocational skills. 7. I would be willing to use this intervention in the classroom setting. 8. This intervention would not result in negative side effects for the student. 9. This intervention would be appropriate for a variety of children. 10. This intervention is consistent with those I have used in classroom settings. 11. The intervention was a fair way to handle the students’ skill deficits. 12. This intervention was reasonable for the target behaviors described. 13. I liked the procedures used in this intervention. 14. This intervention was a good way to teach new skills to students. 15. Overall, this intervention was beneficial for the students. Comments: 159 REFERENCES 160 REFERENCES Anderson, A., Moore, D. W., Rausa, V. C., Finkelstein, S., Pearl, S., & Stevenson, M. (2017). A systematic review of interventions for adults with autism spectrum disorder to promote employment. Review Journal of Autism and Developmental Disorders, 4, 26-38. doi: 10.1007/s40489-016-0094-9 Agran, M., Hughes, C., Thoma, C. A., & Scott, L. A. (2016). Employment social skills: What skills are really valued? Career Development and Transition for Exceptional Individuals, 39, 111-120. doi: 10.1177/2165143414546741 Alexander, J. L., Ayres, K. M., Smith, K. A., Shepley, S. B., & Mataras, T. K. (2013). Using video modeling on an iPad to teach generalized matching on a sorting mail task to adolescents with autism. Research in Autism Spectrum Disorders, 7, 1346–1357. doi: 10.1016/j.rasd.2013.07.021 Allen, K. D., Burke, R. V., Howard, M. R., Wallace, D. P., & Bowen, S. L. (2012). Use of audio cuing to expand employment opportunities for adolescents with autism spectrum disorders and intellectual disabilities. Journal of Autism and Developmental Disorders, 42, 2410–2419. doi: 10.1007/s10803-012-1519-7 Allen, K. D., Wallace, D. P., Greene, D. J., Bowen, S. L., & Burke, R. V. (2010). Community- based vocational instruction using videotaped modeling for young adults with autism spectrum disorders performing in air-inflated mascots. Focus on Autism and Other Developmental Disabilities, 25, 186-192. doi: 10.1177/1088357610377318 Allen, K. D., Wallace, D. P., Renes, D., Bowen, S. L., & Burke, R. V. (2010). Use of video modeling to teach vocational skills to adolescents and young adults with autism spectrum disorders. Education & Treatment of Children, 33, 339–349. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing. Axe, J. B., & Sainato, D. M. (2010). Matrix training of preliteracy skills with preschoolers with autism. Journal of Applied Behavior Analysis, 43, 635–652. doi: 10.1901/jaba.2010.43- 635 Banda, D. R., Dogoe, M. S., & Matuszny, R. M. (2011). Review of video prompting studies with persons with developmental disabilities. Education and Training in Autism and Developmental Disabilities, 514-527. 161 Bellini, S., & Akullian, J. (2007). A meta-analysis of video modeling and video self- modeling interventions for children and adolescents with autism spectrum disorders. Exceptional Children, 73, 264-287. doi: 10.1177/001440290707300301 Bellini, S., & Peters, J. K. (2008). Social skills training for youth with autism spectrum disorders. Child and Adolescent Psychiatric Clinics of North America, 17, 857–873. doi: 10.1016/j.chc.2008.06.008 Bennett, K., Brady, M. P., Scott, J., Dukes, C., & Frain, M. (2010). The effects of covert audio coaching on the job performance of supported employees. Focus on Autism and Other Developmental Disabilities, 25, 173–185. doi: 10.1177/1088357610371636 Bennett, K. D. (2013). Improving vocational skills of students with disabilities. Teaching Exceptional Children, 46(2), 60–67. doi: 10.1177/004005991304600207 Bennett, K. D., Gutierrez, A., & Honsberger, T. (2013). A comparison of video prompting with and without voice-over narration on the clerical skills of adolescents with autism. Research in Autism Spectrum Disorders, 7, 1273–1281. doi: 10.1016/j.rasd.2013.07.013 Bennett, K. D., Ramasamy, R., & Honsberger, T. (2013a). Further examination of covert audio coaching on improving employment skills among secondary students with autism. Journal of Behavioral Education, 22, 103–119. doi: 10.1007/s10864-013-9168-2 Bennett, K. D., Ramasamy, R., & Honsberger, T. (2013b). The effects of covert audio coaching on teaching clerical skills to adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 43, 585-593. doi: 10.1007/s10803-012-1597-6 Bereznak, S., Ayres, K. M., Mechling, L. C., & Alexander, J. L. (2012). Video self-prompting and mobile technology to increase daily living and vocational independence for students with autism spectrum disorders. Journal of Developmental and Physical Disabilities, 24, 269–285. doi: 10.1007/s10882-012-9270-8 Butterworth Jr, J., & Strauch, J. D. (1994). The relationship between social competence and success in the competitive work place for persons with mental retardation. Education and Training in Mental Retardation and Developmental Disabilities, 29, 118-133. Buescher AS, Cidav Z, Knapp M, & Mandell DS. (2014). Costs of autism spectrum disorders in the united kingdom and the united states. JAMA Pediatrics, 168, 721–728. doi: 10.1001/jamapediatrics.2014.210 Burke, R. V., Allen, K. D., Howard, M. R., Downey, D., Matz, M. G., & Bowen, S. L. (2013). Tablet-based video modeling and prompting in the workplace for individuals with autism. Journal of Vocational Rehabilitation, 38, 1–14. doi:10.3233/JVR-120616 162 Carter, E., O'Rourke, L., Sisco, L. G., & Pelsue, D. (2009). Knowledge, responsibilities, and training needs of paraprofessionals in elementary and secondary schools. Remedial and Special Education, 30, 344-359. doi: 10.1177/0741932508324399 Charlop-Christy, M. H., Le, L., & Freeman, K. A. (2000). A Comparison of video modeling with in vivo modeling for teaching children with autism. Journal of Autism and Developmental Disorders, 30, 537–552. doi: 10.1023/A:1005635326276 Charlop-Christy, M. H. & Daneshvar, S. (2003). Using video modeling to teach perspective taking to children with autism. Journal of Positive Behavior Interventions, 5, 12-21. doi: 10.1177/10983007030050010101 Chen, J. L., Sung, C., & Pi, S. (2015). Vocational rehabilitation service patterns and outcomes for individuals with autism of different ages. Journal of Autism and Developmental Disorders, 45, 3015–3029. doi: 10.1007/s10803-015-2465-y Cihak, D. F., & Schrader, L. (2008). Does the model matter? Comparing video self-modeling and video adult modeling for task acquisition and maintenance by adolescents with autism spectrum disorders. Journal of Special Education Technology, 23(3), 9-20. doi: 10.1177/016264340802300302 Cook, B. G., Tankersley, M., Cook, L., & Landrum, T. J. (2008). Evidence-based practices in special education: Some practical considerations. Intervention in School and Clinic, 44, 69-75. doi: 10.1177/1053451208321452 Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis. (2nd ed.) Upper Saddle River, NJ: Pearson. Corbett, B. A., & Abdullah, M. (2005). Video modeling: why does it work for children with autism? The Journal of Early and Intensive Behavioral Intervention, 2, 2-8. doi: 10.1037/h0100294 Dauphin, M., Kinney, E. M., & Stromer, R. (2004). Using video-enhanced activity schedules and matrix training to teach sociodramatic play to a child with autism. Journal of Positive Behavior Interventions, 6, 238–250. doi: 10.1177/10983007040060040501 Dotson, W. H., Richman, D. M., Abby, L., Thompson, S., & Plotner, A. (2013). Teaching skills related to self-employment to adults with developmental disabilities: An analog analysis. Research in Developmental Disabilities, 34, 2336–2350. doi: 10.1016/j.ridd.2013.04.009 Dotto-Fojut, K. M., Reeve, K. F., Townsend, D. B., & Progar, P. R. (2011). Teaching adolescents with autism to describe a problem and request assistance during simulated vocational tasks. Research in Autism Spectrum Disorders, 5, 826–833. doi: 10.1016/j.rasd.2010.09.012 163 Elksnin, N., & Elksnin, L. K. (2011). Adolescents with disabilities: The need for occupational social skills training. Exceptionality, 9, 91-105. doi: 10.1080/09362835.2001.9666993 English, D. L., Gounden, S., Dagher, R. E., Chan, S. F., Furlonger, B. E., Anderson, A., & Moore, D. W. (2017). Effects of video modeling with video feedback on vocational skills of adults with autism spectrum disorder. Developmental neurorehabilitation, 20, 511- 524. doi: 10.1080/17518423.2017.1282051 Ganz, J. B., & Sigafoos, J. (2005). Self-monitoring: Are young adults with MR and autism able to utilize cognitive strategies independently? Education and Training in Developmental Disabilities, 40, 24–33. Gast, D. L. & Ledford, J. R. (2014). Single case research methodology: Applications in special education and behavioral sciences (2nd ed.). New York, NY: Routledge/Taylor & Francis Group. García‐Villamisar, D., Ross, D., & Wehman, P. (2000). Clinical differential analysis of persons with autism in a work setting: A follow-up study. Journal of Vocational Rehabilitation, 14, 183-185. García‐Villamisar, D., & Hughes, C. (2007). Supported employment improves cognitive performance in adults with autism. Journal of Intellectual Disability Research, 51, 142- 150. doi: 10.1111/j.1365-2788.2006.00854.x Goh, A. E., & Bambara, L. M. (2013). Video self-modeling: A job skills intervention with individuals with intellectual disability in employment settings. Education and Training in Autism and Developmental Disabilities, 48(1), 103–119. Goldstein, H. (1983). Recombinative generalization: Relationships between environmental conditions and the linguistic repertoires of language learners. Analysis and Intervention in Developmental Disabilities, 3, 279–293. doi. 10.1016/0270-4684(83)90002-2 Hall, A. C., Bose, J., Winsor, J. E., & Migliore, A. (2014). From research to practice: Knowledge translation in job development. Inclusion, 2, 65-79. doi: 10.111/jar.12077 Hendricks, D. R., & Wehman, P. (2009). Transition from school to adulthood for youth with autism spectrum disorders: Review and recommendations. Focus on Autism and Other Developmental Disabilities, 24, 77-88. doi: 10.1177/1088357608329827 Hendricks, D. (2010). Employment and adults with autism spectrum disorders: Challenges and strategies for success. Journal of Vocational Rehabilitation, 32, 125-134. doi: 10.3233/JVR-2010-0502 Hillier, A., Campbell, H., Mastriani, K., Izzo, M. V., Kool-Tucker, A. K., Cherry, L., & Beversdorf, D. Q. (2007). Two-year evaluation of a vocational support program for adults on the autism spectrum. Career Development for Exceptional Individuals, 30, 35-47. doi: 10.1177/08857288070300010501 164 Hillier, A. J., Fish, T., Siegel, J. H., & Beversdorf, D. Q. (2011). Social and vocational skills training reduces self-reported anxiety and depression among young adults on the autism spectrum. Journal of Developmental and Physical Disabilities, 23, 267-276. doi: 10.1007/s10882-011-9226-4 Horner, R. D., & Baer, D. M. (1978). Multiple‐probe technique: A variation of the multiple baseline. Journal of Applied Behavior Analysis, 11, 189-196. doi: 10.1901/jaba.1978.11- 189 Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179. doi: 10.1177/001440290507100203 Howlin, P., Goode, S., Hutton, J., & Rutter, M. (2004). Adult outcome for children with autism. Journal of Child Psychology and Psychiatry, 45, 212-229. doi: 10.1111/j.1469- 7610.2004.00215.x Hume, K., Loftin, R., & Lantz, J. (2009). Increasing independence in autism spectrum disorders: A review of three focused interventions. Journal of Autism and Developmental Disorders, 39, 1329-1338. doi: 10.1007/s10803-009-0751-2 Hurlbutt, K., & Chalmers, L. (2004). Employment and adults with Asperger syndrome. Focus on Autism and Other Developmental Disabilities, 19, 215–222. doi: 10.1177/10883576040190040301 Individuals with Disabilities Education Act, 20 U.S.C. § 1400 (2004) Institute of Education Sciences (IES). (2014). What Works Clearinghouse Procedures and Standards Handbook Version 3.0. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_ procedures_v3_0_standards_handbook.pdf. Kaboski, J. R., Diehl, J. J., Beriont, J., Crowell, C. R., Villano, M., Wier, K., & Tang, K. (2015). Brief report: A pilot summer robotics camp to reduce social anxiety and improve social/vocational skills in adolescents with ASD. Journal of autism and developmental disorders, 45, 3862-3869. doi: 10.1007/s10803-014-2153-3 Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed). New York: Oxford University Press. Kellems, R. O., & Morningstar, M. E. (2012). Using video modeling delivered through ipods to teach vocational tasks to young adults with autism spectrum disorders. Career Development and Transition for Exceptional Individuals, 35, 155–167. doi: 10.1177/0885728812443082 165 Koegel, R. L., & Koegel, L. K. E. (1995). Teaching children with autism: Strategies for initiating positive interactions and improving learning opportunities. Baltimore, MD, Paul H Brookes Publishing. Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. What works clearinghouse. Kratochwill, T. R., Hitchcock, J. H., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26-38. doi: 10.1177/0741932512452794 Lattimore, L. P., Parsons, M. B., & Reid, D. H. (2006). Enhancing job‐site training of supported workers with autism: a reemphasis on simulation. Journal of Applied Behavior Analysis, 39, 91-102. doi: 10.1901/jaba.2006.154-04 Lattimore, L. P., Parsons, M. B., & Reid, D. H. (2008). Simulation training of community job skills for adults with autism: A further analysis. Behavior Analysis in Practice, 1, 24-29. doi: 10.1007/BF03391717 Lerman, D. C., White, B., Grob, C., & Laudont, C. (2017). A Clinic-Based assessment for evaluating job-related social skills in adolescents and adults with autism. Behavior analysis in practice, 10, 323-336. doi: 10.1007/s40617-017-0177-9 Mackey, M., & Nelson, G. (2015). Twins with autism: utilising video feedback to improve job‐ related behaviours. British Journal of Special Education, 42, 390-410. doi: 10.1111/1467-8578.12107 MacManus, C., MacDonald, R., & Ahearn, W. H. (2015). Teaching and generalizing pretend play in children with autism using video modeling and matrix training. Behavioral Interventions, 30, 191–218. doi: 10.1002/bin.1406 Maggin, D. M., Briesch, A. M., & Chafouleas, S. M. (2013). An application of the what works clearinghouse standards for evaluating single-subject research synthesis of the self- management literature base. Remedial and Special Education, 34, 44–58. doi: 10.1177/0741932511435176 Martens, B. K., Witt, J. C., Elliott, S. N., & Darveaux, D. X. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional psychology: Research and practice, 16(2), 191-198. doi: 10.1037/0735-7028.16.2.191 Mawhood, L., Howlin, P., & Rutter, M. (2000). Autism and developmental receptive language disorder—A comparative follow-up in early adult life. I: Cognitive and language outcomes. The Journal of Child Psychology and Psychiatry and Allied Disciplines, 41, 547-559. doi: 10.1111/1469-7610.00642 166 Müller, E., Schuler, A., Burton, B. A., & Yates, G. B. (2003). Meeting the vocational support needs of individuals with Asperger syndrome and other autism spectrum disabilities. Journal of Vocational Rehabilitation, 18, 163–175. Müller, E., Schuler, A., & Yates, G. B. (2008). Social challenges and supports from the perspective of individuals with Asperger syndrome and other autism spectrum disabilities. Autism, 12, 173–190. doi: 10.1177/1362361307086664 National Autism Center (2015). Findings and conclusions: National standards project, phase 2. Randolph, MA. Nicholas, D. B., Attridge, M., Zwaigenbaum, L., & Clarke, M. (2015). Vocational support approaches in autism spectrum disorder: A synthesis review of the literature. Autism, 19, 235–245. doi:10.1177/1362361313516548 Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148. doi: 10.1177/001440290507100201 Odom, S. L., Thompson, J. L., Hedges, S., Boyd, B. A., Dykstra, J. R., Duda, M. A., ... & Bord, A. (2015). Technology-aided interventions and instruction for adolescents with autism spectrum disorder. Journal of autism and developmental disorders, 45, 3805-3819. doi: 10.1007/s10803-014-2320-6 Persson, B. (2000). Brief report: A longitudinal study of quality of life and independence among adult men with autism. Journal of Autism and Developmental Disorders, 30, 61-66. doi: 10.1023/A:1005464128544 Phillips, B. N., Kaseroff, A. A., Fleming, A. R., & Huck, G. E. (2014). Work-related social skills: Definitions and interventions in public vocational rehabilitation. Rehabilitation psychology, 59(4), 386. doi: 10.1037/rep0000011 Plavnick, J. B. (2013). Video modeling (VM) fact sheet. Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institute, The National Professional Development Center on Autism Spectrum Disorders. Plavnick, J. B., Sam, A. M., Hume, K., & Odom, S. L. (2013). Effects of video-based group instruction for adolescents with autism spectrum disorder. Exceptional Children, 80, 67- 83. doi: 10.1177/001440291308000103 Plavnick, J. B., Kaid, T., & MacFarland, M. C. (2015). Effects of a school-based social skills training program for adolescents with autism spectrum disorder and intellectual disability. Journal of Autism and Developmental Disorders, 45, 2674-2690. doi: 10.1007/s10803-015-2434-5 167 Plimley, L. A. (2007). A review of quality of life issues and people with autism spectrum disorders. British Journal of Learning Disabilities, 35, 205–213. doi: 10.1111/j.1468- 3156.2007.00448.x Ploog, B. O. (2010). Stimulus overselectivity four decades later: A review of the literature and its implications for current research in autism spectrum disorder. Journal of autism and developmental disorders, 40, 1332-1349. doi: 10.1007/s10803-010-0990-2 Quill, K. A. (1997). Instructional considerations for young children with autism: The rationale for visually cued instruction. Journal of Autism and Developmental Disorders, 27, 697– 714. doi: 10.1023/A:1025806900162 Rao, P. A., Beidel, D. C., & Murray, M. J. (2008). Social skills interventions for children with Asperger’s syndrome or high-functioning autism: A review and recommendations. Journal of Autism and Developmental Disorders, 38, 353–361. doi: 10.1007/s10803-007- 0402-4 Rausa, V. C., Moore, D. W., & Anderson, A. (2016). Use of video modelling to teach complex and meaningful job skills to an adult with autism spectrum disorder. Developmental neurorehabilitation, 19, 267-274. doi: 10.3109/17518423.2015.1008150 Reichow, B., Volkmar, F. R., & Cicchetti, D. V. (2008). Development of the evaluative method for evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders, 38, 1311–1319. doi: 10.1007/s10803-007-0517-7 Reichow, B., & Volkmar, F. R. (2010). Social skills interventions for individuals with autism: Evaluation for evidence-based practices within a best evidence synthesis framework. Journal of autism and developmental disorders, 40, 149-166. doi: 10.1007/s10803-009-0842-0 Robinson, K.A., & Smith, V. (2010). A specific vocational training program for an adolescent with autism. Developmental Disabilities Bulletin, 38, 93–109. Roux, A., Shattuck, P., Rast, J., & Anderson, K. (2015). National autism indicators report: Transition into young adulthood. Philadelphia, PA: Life Course Outcomes Research Program, A.J. Drexel Autism Institute, Drexel University. Sarokoff, R. A., & Sturmey, P. (2004). The effects of behavioral skills training on staff implementation of discrete‐trial teaching. Journal of Applied Behavior Analysis, 37, 535- 538. doi: 10.1901/jaba.2004.37-535 Sarokoff, R. A., & Sturmey, P. (2008). The effects of instructions, rehearsal, modeling, and feedback on acquisition and generalization of staff use of discrete trial teaching and student correct responses. Research in Autism Spectrum Disorders, 2, 125-136. doi: 10.1016.j.rasd.2007.04.002 168 Seaman, R. L., & Cannella-Malone, H. I. (2016). Vocational skills interventions for adults with autism spectrum disorder: a review of the literature. Journal of Developmental and Physical Disabilities, 28, 479-494. doi: 10.1007/s10882-016-9479-z Shattuck, P. T., Narendorf, S. C., Cooper, B., Sterzing, P. R., Wagner, M., & Taylor, J. L. (2012b). Postsecondary education and employment among youth with an autism spectrum disorder. Pediatrics, 129, 1042-1049. doi: 10.1542/peds.2011-2864 Sigafoos, J., O’Reilly, M., Cannella, H., Edrisinha, C., De La Cruz, B., Upadhyaya, M., ... & Young, D. (2007). Evaluation of a video prompting and fading procedure for teaching dish washing skills to adults with developmental disabilities. Journal of Behavioral Education, 16(2), 93-109. doi: 10.1007/s10864-006-9004-z Smith, J. D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17, 510-550. doi: 10.1037/a0029312 Stauch & Plavnick (in preparation). Teaching vocational and social skills to adolescents with autism spectrum disorder using video modeling: A pilot study. Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of applied behavior analysis, 10, 349-367. Tate, R. L., McDonald, S., Perdices, M., Togher, L., Schultz, R., & Savage, S. (2008). Rating the methodological quality of single-subject designs and n-of-1 trials: introducing the Single- Case Experimental Design (SCED) Scale. Neuropsychological Rehabilitation, 18, 385– 401. doi: 10.1080/09602010802009201 Taylor, J. L., & Seltzer, M. M. (2011). Employment and post-secondary educational activities for young adults with autism spectrum disorders during the transition to adulthood. Journal of autism and developmental disorders, 41, 566-574. doi: 10.1007/s10803-010-1070-3 Taylor, J., McPheeters, M. L., Sathe, N. A., Dove, D., Veenstra-VanderWeele, J., & Warren, Z. (2012). A systematic review of vocational interventions for young adults with autism spectrum disorders. Pediatrics, 130, 531–538. doi: 10.1542/peds.2012-0682 Taylor, J. L., Smith, L. E., & Mailick, M. R. (2014). Engagement in vocational activities promotes behavioral development for adults with autism spectrum disorders. Journal of Autism and Developmental Disorders, 44, 1447-1460. doi: 10.1007/s10803-013-2010-9 Test, D. W., Mazzotti, V. L., Mustian, A. L., Fowler, C. H., Kortering, L., & Kohler, P. (2009). Evidence-based secondary transition predictors for improving postschool outcomes for students with disabilities. Career Development for Exceptional Individuals, 32, 160–181. doi: 10.1177/0885728809346960 169 Test, D. W., Smith, L. E., & Carter, E. W. (2014). Equipping youth with autism spectrum disorders for adulthood: Promoting rigor, relevance, and relationships. Remedial and Special Education, 35, 80-90. doi: 10.1177/0741932513514857 Townley-Cochran, D., Leaf, J. B., Taubman, M., Leaf, R., & McEachin, J. (2015). Observational learning for students diagnosed with autism: A review paper. Review Journal of Autism and Developmental Disorders, 2, 262-272. doi: 10.1007/s40489-015-0050-0 Van Laarhoven, T., & Laarhoven-Myers, V. (2007). The effectiveness of using a pocket PC as a video modeling and feedback device for individuals with developmental disabilities in vocational settings. Assistive Technology Outcomes and Benefits, 4, 28-45. Van Laarhoven, T., Winiarski, L., Blood, E., & Chan, J. M. (2012). Maintaining vocational skills of individuals with autism and developmental disabilities through video modeling. Education and Training in Autism and Developmental Disabilities, 47, 447–461. Vismara, L. A., & Rogers, S. J. (2010). Behavioral treatments in autism spectrum disorder: What do we know? Annual Review of Clinical Psychology, 6, 447–468. doi: 10.1146/annurev.clinpsy.121208.131151 Von Brock, M. B., & Elliott, S. N. (1987). Influence of treatment effectiveness information on the acceptability of classroom interventions. Journal of School Psychology, 25, 131-144. doi: 10.1016/0022-4405(87)90022-7 Walsh, L., Lydon, S., & Healy, O. (2014). Employment and vocational skills among individuals with autism spectrum disorder: Predictors, impact, and interventions. Review Journal of Autism and Developmental Disorders, 1, 266–275. doi: 10.1007/s40489-014-0024-7 Walsh, E., Holloway, J., McCoy, A., & Lydon, H. (2017). Technology-aided interventions for employment skills in adults with autism spectrum disorder: a systematic review. Review Journal of Autism and Developmental Disorders, 4, 12-25. doi: 10.1007/s40489-016- 0093-x Ward‐Horner, J., & Sturmey, P. (2012). Component analysis of behavior skills training in functional analysis. Behavioral Interventions, 27(2), 75-92. doi: 10.1002/bin.1339 Wehman, P., Schall, C., Carr, S., Targett, P., West, M., & Cifu, G. (2014). Transition from school to adulthood for youth with autism spectrum disorder: What we know and what we need to know. Journal of Disability Policy Studies, 25(1), 30-40. doi: 10.1177/1044207313518071 Wehman, P., Brooke, V., Brooke, A. M., Ham, W., Schall, C., McDonough, J., ... & Avellone, L. (2016). Employment for adults with autism spectrum disorders: A retrospective review of a customized employment approach. Research in developmental disabilities, 53, 61-72. doi: 10.1016/j.ridd.2016.01.015 Wendel, E., Cawthon, S. W., Ge, J. J., & Beretvas, S. N. (2015). Alignment of single-case design (SCD) research with individuals who are deaf or hard of hearing with the what works 170 clearinghouse standards for SCD research. Journal of Deaf Studies and Deaf Education, 20, 103–114. doi: 10.1093/deafed/enu049 Westbrook, J. D., Fong, C. J., Nye, C., Williams, A., Wendt, O., & Cortopassi, T. (2015). Transition services for youth with autism: A systematic review. Research on Social Work Practice, 25, 10–20. doi: 10.1177/1049731514524836 Westbrook, J. D., Nye, C., Fong, C. J., Wan, J. T., Cortopassi, T., & Martin, F. H. (2012). Adult employment assistance services for persons with autism spectrum disorders effects on employment outcomes. St. Louis, United States: Federal Reserve Bank of St Louis. Whang, P. L., Fawcett, S. B., & Mathews, R. M. (1984). Teaching job-related social skills to learning disabled adolescents. Analysis and Intervention in Developmental Disabilities, 4, 29-38. doi: 10.1016/0270-4684(84)90016-8 White, E. R., Hoffmann, B., Hoch, H., & Taylor, B. A. (2011). Teaching teamwork to adolescents with autism: The cooperative use of activity schedules. Behavior Analysis in Practice, 4, 27–35. doi: 10.1007/BF03391772 Wang, S. Y., Cui, Y., & Parrila, R. (2011). Examining the effectiveness of peer-mediated and video-modeling social skills interventions for children with autism spectrum disorders: A meta-analysis in single-case research using HLM. Research in Autism Spectrum Disorders, 5, 562-569. doi: 10.1016.j.rasd.2010.06.023 Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A., Kucharczyk, S., … Schultz, T. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966. doi: 10.1007/s10803-014-2351-z 171