TRAINING JOB COACHES IN SYSTEMATIC METHODS OF INSTRUCTION By John D. Wenzel IV A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Applied Behavior Analysis—Master of Arts 2019 TRAINING JOB COACHES IN SYSTEMATIC METHODS OF INSTRUCTION ABSTRACT By John D. Wenzel IV Adults with intellectual and developmental disabilities (IDD) face high rates of unemployment, but supported employment models that incorporate job coaches to teach vocational skills on the jobsite improve employment outcomes. Job coaches, however, are not typically trained to implement systematic instructional practices. This study evaluated the effectiveness of a job coach training using a behavioral skills training (BST) with multiple exemplar training model to teach three systematic instructional methods (developing a task analysis, simultaneous prompting, and least-to-most prompting). A multiple probe across behaviors design was replicated across three participants and was evaluated using visual analysis. All participants demonstrated mastery of all three skills in simulated assessments following training and were able to successfully generalize these skills when teaching novel tasks to individuals with IDD. The study provides evidence that BST with multiple exemplar training can be used to teach newly hired job coaches with little experience working with individuals with IDD to implement systematic instructional strategies. Clinical implications and directions for future research are discussed. Keywords: Job coaches, least-to-most prompting, simultaneous prompting, staff training, task analysis TABLE OF CONTENTS LIST OF TABLES .................................................................................................................iv LIST OF FIGURES ...............................................................................................................v KEY TO SYMBOLS AND ABBREVIATIONS ..................................................................vi Introduction ............................................................................................................................1 Method ...................................................................................................................................6 Participants ........................................................................................................................6 Job Coach Participants. ..............................................................................................6 Actors. ........................................................................................................................6 Interns with IDD. .......................................................................................................6 Setting and Materials ........................................................................................................7 Dependent Variables and Measurement ...........................................................................8 Task analysis. .............................................................................................................8 Simultaneous prompting. ...........................................................................................9 Least-to-most prompting. ...........................................................................................10 Interobserver Reliability and Procedural Fidelity. .....................................................11 Experimental Design .........................................................................................................13 Procedures .........................................................................................................................13 Baseline procedures. .................................................................................................13 Intervention. ..............................................................................................................13 Generalization. ...........................................................................................................17 Social Validity .................................................................................................................17 Data Analysis ...................................................................................................................18 Results. ...................................................................................................................................20 Participant 1 .....................................................................................................................20 Participant 2 .....................................................................................................................21 Participant 3 .....................................................................................................................22 Social Validity .................................................................................................................22 Discussion. .............................................................................................................................25 Limitations and Future Research .....................................................................................28 Conclusion .......................................................................................................................30 APPENDIX ............................................................................................................................32 REFERENCES ......................................................................................................................38 iii LIST OF TABLES Table 1: List of Tasks Used in Training Sessions and Assessments. ....................................33 Table 2: Definitions of Instructional Strategies Provided to Job Coach Participants During All Assessments. ........................................................................................................34 iv LIST OF FIGURES Figure 1. Percentage of steps Participant 1 performed correctly across instructional strategies during probe, training, and generalization sessions. Note: P2 = probe 2; P3 = probe 3; P4 = probe 4; BST = behavior skills training... ...............................................35 Figure 2. Percentage of steps Participant 2 performed correctly across instructional strategies during probe, training, and generalization sessions. Note: P2 = probe 2; P3 = probe 3; P4 = probe 4; BST = behavior skills training.... ..............................................36 Figure 3. Percentage of steps Participant 3 performed correctly across instructional strategies during probe, training, and generalization sessions. Note: P2 = probe 2; P3 = probe 3; P4 = probe 4; BST = behavior skills training... ...............................................37 v KEY TO SYMBOLS AND ABBREVIATIONS Autism Spectrum Disorder Behavioral skills training Intellectual and developmental disabilities Interobserver agreement Participant 1 Participant 2 Participant 3 ASD BST IDD IOA P1 P2 P3 vi Introduction Adults with intellectual and developmental disabilities (IDD) often struggle to find and maintain employment. In the United States, compared to 68.3% of individuals without disabilities, the employment rate for adults with disabilities was 19.4% (Bureau of Labor and Statistics, 2019); only 26.3% of individuals with intellectual disability (Kraus et al., 2018) and 14% of individuals with autism spectrum disorder (ASD; Roux et al., 2017) were competitively employed in 2016. This means that while individuals with disabilities have higher chances of facing unemployment overall, the most vulnerable members of this population face the greatest hardships in finding work. In an effort to help individuals with IDD succeed in an employment setting, supported employment has been developed as an evidence-based method of support (Marshall et al., 2014). Early studies defined supported employment as employment where an individual with a disability is able to work as a paid employee in an inclusive setting with peers who do not have disabilities (Rusch & Hughes, 1989). To support this employment, individuals with IDD receive extra training, supervision, and accommodations by support personnel and coworkers; without these supports, these individuals would be unlikely to find or maintain employment. More recent studies on supported employment have identified four primary phases for achieving successful supported employment, including (1) creating a jobseeker profile, (2) looking for a job based on that profile, (3) direct on-site training that is faded out over time, and (4) longer-term supports like transportation arrangements for individuals who cannot drive (Wehman et al., 2012). Job coaches are an essential support provider within the supported employment model, as job coaches serve a variety of roles working directly with an individual with IDD to provide or facilitate the supports needed for the individual to be successful in the workplace (Parent, Unger, 1 Gibson, & Clements, 1994). These supports often include assessing the strengths and interests of an individual to help match them with a job, providing help in applying for and getting a job, direct training on particular job tasks, or providing assistance with off-jobsite issues like transportation or Social Security that might affect the individual’s ability to work. Direct support and initial jobsite training are often the most intensive aspects of job coaching. These often include (a) the use of behavioral strategies to modify job duties to better fit an individual’s abilities, (b) the use of jobsite supports like visual aids or assistive technologies to help facilitate skill acquisition, and (c) transfer of stimulus control to naturally occurring stimuli in the workplace so that support can be systematically faded (Wehman et al., 2014). The ultimate goal of job coaching is to facilitate skill acquisition and independence through effective jobsite support. While supported employment itself is an evidence-based practice, prior research has shown that job coaches rarely receive any meaningful degree of training in evidence-based instruction, nor do they receive proper oversight and support (Hall, Butterworth, Winsor, Kramer, Nye-Lengerman, & Timmons, 2018; Rogan & Held, 1999; Wehman et al., 2014). This lack of training and supervision can limit the ability of job coaches to provide effective support to individuals with disabilities, impacting skill acquisition and independence. To date, we have only been able to identify one study that examined the efficacy of a training to implement systematic instructional strategies on job coach performance. To address the need for better systematic training for job coaches, Brock and colleagues (2016) examined whether six job coaches could be taught to implement three systematic instructional practices by training and then measuring their implementation fidelity for task analysis, simultaneous prompting, and least-to-most prompting. A multiple baseline across 2 behaviors design was used, replicated across participants, to show that job coach implementation only improved for each strategy after implementation of the group training procedure. The job coach training sessions used procedures similar to behavioral skills training (BST), an evidence- based training model that includes didactic instruction on a target skill, instructor demonstration of the skill, trainee skill rehearsal, and feedback from the instructor on trainee performance (Miltenberger, 2012; Parsons, Rollyson, & Reid, 2012). The study participants demonstrated skill acquisition of each strategy in a simulated roleplay setting with two tasks and then demonstrated generalization of the behavior to a novel job task that they had not been trained to teach. The results showed that the participants implemented each strategy for the generalization task with significantly higher fidelity after the group training sessions, indicating that not only could the participants be effectively trained to implement systematic instruction, but that they could generalize these skills to untrained tasks. Implementation was only assessed during roleplay scenarios with actors without disabilities, however, and data were not collected for implementation with individuals with IDD. Additionally, job coaches only practiced implementation with two tasks and implementation was only assessed with a single generalization task. Given these limitations, it is unknown whether participants could generalize performance to additional vocational tasks and whether they could apply the skills with the same fidelity to supporting individuals with IDD. Despite the lack of research on teaching job coaches to implement systematic instructional strategies with individuals with IDD, previous studies on training paraeducators and other support personnel to implement systematic instruction and other behavioral interventions provide evidence that, if properly trained, support personnel can effectively implement these practices in a live environment. For example, Walker and Snell (2017) found that group training 3 workshops and weekly individual coaching sessions could train paraeducators to implement function-based interventions for challenging behavior with three students with ASD and intellectual disability. Paraeducators learned about functional behavior assessment, what function-based interventions are, and how to implement them through two hour-long workshops. Instructors described and modeled the strategies and provided opportunities for paraeducator participants to practice these skills through roleplay. Feedback was also provided on implementation during these workshops, as well as during weekly coaching sessions where participants and instructors reviewed the participants’ implementation with actual students. The paraeducator participants went from low percentages of implementation fidelity during baseline to meeting criteria after training was implemented. Further, students displayed a corresponding decrease in challenging behavior frequency after their paraeducators were trained on and implemented function-based interventions. In another study, Britton, Collins, Ault, and Bausch (2017) found that a constant time delay procedure was effective in teaching a paraeducator and a peer tutor to implement a simultaneous prompting procedure to teach a high school student with moderate intellectual disability to perform multiple academic and functional living tasks. During the baseline condition of this multiple baseline study, support personnel participants implemented the simultaneous prompting procedure with low fidelity. After constant time delay instruction, both participants showed an immediate increase in independent implementation fidelity. The student participant improved his performance for each task after his peer tutor and paraeducator began to learn and use the simultaneous prompting procedure as well. The body of research evidence suggests that support personnel, including job coaches, can be trained to effectively implement systematic instructional strategies. However, there is limited 4 evidence regarding how to effectively teach these skills in a manner that will generalize to untrained tasks and teaching individuals with IDD in a vocational setting. Brock et al. (2016) provided evidence that BST can be used to teach job coaches systematic instructional strategies, but did not assess implementation in a real setting with individuals with IDD. In this study, we aim to expand on the procedures of Brock et al. (2016) by combining similar training procedures with the design and non-simulated assessment of the paraeducator literature by assessing generalization of systematic instructional procedures learned in roleplay scenarios to teaching vocational tasks to individuals with IDD. We had two primary research questions. First, to what extent does BST affect implementation fidelity for job coaches using task analysis, simultaneous prompting, and least-to-most prompting methods to teach vocational skills during simulations? Second, how do these skills generalize to novel tasks via multiple exemplar training and instruction with individuals with IDD? 5 Participants Method Job Coach Participants. Three female undergraduate college students participated in this study. Each participant was involved with a peer mentoring program for young adults with IDD at a major midwestern university. All job coach participants met the following inclusion criteria: (1) expressed an interest in a career working with individuals with IDD; (2) had never been employed as a job coach; and (3) did not have prior training or experience with implementing task analysis, simultaneous prompting, or least-to-most prompting. All job coach participants had high school diplomas and were in their final year of pursuing a bachelor’s degree. Participant 1 was a 22-year-old White female pursuing a degree in Human Resources and Labor Relations. Participant 2 was a 22-year-old Black female pursuing a degree in Kinesiology. Participant 3 was a 21-year-old South Asian female pursuing a degree in Neuroscience. Participants received $20 after completion of training for each strategy ($60 total for training) and an additional $40 after completing the generalization probes, for a potential total of $100. Actors. Four actors were recruited to perform in simulations to assess job coach performance. Three actors were White female students (ages 21 to 29) pursuing a master’s degree in Applied Behavior Analysis. The fourth actor was a 24-year-old White male with a bachelor’s degree in Aerospace Engineering. Interns with IDD. Three student interns with IDD were recruited to assist with the generalization probes. These students were recruited from the intermediate school district’s one- year school-to-work transition program that was based at the university. All students attending the school-to-work transition program were invited to volunteer if they met the following inclusion criteria: (1) school-based eligibility for special education services (e.g., active 6 individualized education plan) under the category of intellectual disability or ASD; and (2) had not previously received systematic instruction on the generalization tasks (making a milkshake and bagging groceries). Among the six who volunteered to participate, three interns were randomly selected and then randomly assigned to a job coach participant. All three interns were White, 23 years old, and pursuing a special education certificate of completion. Intern 1 had a diagnosis of ASD and mild to moderate intellectual disability. Intern 2 had a diagnosis of Noonan syndrome and mild to moderate intellectual disability. Intern 3 had a diagnosis of ASD and mild to moderate intellectual disability. Setting and Materials Training sessions for job coach participants took place in a university conference room. During training sessions, materials included a computer with internet access, a video camera, and a slideshow presentation for each strategy. A sheet of paper and a pen were provided for use in creating a task analysis. Different materials were used for conducting the various tasks during training and simulation assessments with job coach participants. For example, for setting the table, materials included a placemat, a plate, a cup, a napkin, and silverware; for cleaning a table, materials included a table with items on it, a spray bottle of cleaning solution, and a rag. During generalization probes with interns with IDD, job coach participants were asked to teach the interns to make a milkshake and to bag groceries. For making a milkshake, necessary materials included a blender, milk, ice cream, measuring cups, and an ice cream scoop. For bagging groceries, necessary materials included a shopping bag, assorted cans, a box of crackers, a bag of chips, a loaf of bread, and a bag of candy. 7 Dependent Variables and Measurement The primary dependent variables were the percentage of steps implemented correctly by the job coach participants for the three selected systematic instructional strategies: task analysis, simultaneous prompting, and least-to-most prompting during both simulation assessments and implementation with interns with IDD (generalization). These instructional strategies were selected because of their versatility and the evidence that they are effective at teaching various vocational skills (Brock et al., 2016; Parsons, Reid, Green, & Browning, 2001). The three different strategies can be applied to different tasks as needed, and they provide a job coach with the tools to analyze a task, teach it to an untrained learner, and then fade prompts systematically in order to promote independence. All assessments of job coach participant performance were recorded and coded later. Task analysis. Task analysis “breaks down complex behaviors or skills into smaller steps that can be more easily taught” (Cooper, Heron, & Heward, 2007, p. 706). Task analyses typically take the form of sequentially ordered steps for the overall task arranged in a list. Prior to the start of this study, the first author generated a list of tasks and then wrote a task analysis for each task. All tasks could be broken down into 8-12 steps in a task analysis. A team of three Applied Behavior Analysis graduate students and one doctoral-level behavior analyst reviewed the steps for each task analysis in a group format, provided edits and suggestions, and came to a consensus on the order and number of steps included. The team then reviewed each task analysis again and agreed that all tasks were similar in complexity and difficulty. The final list of task analyses was used as the master coding sheet (see Table 1). To assess accurate implementation of the participants’ task analyses, a coder compared the master task analysis for each task to the task analysis generated by the job coach participant. 8 The task analysis was coded for whether the task analysis met the following general criteria: (1) each step was a broken-down part of the larger skill; (2) the steps were listed in an order where the task could be completed correctly; (3) the steps were written specifically enough to be followed correctly; (4) the steps were written concisely in direct language; and (5) extraneous steps were not included. Criterion 1 was scored for each individual step of the task analysis that matched the master task analysis, while criteria 2-5 were coded for the task analysis as a whole. Performance for each task analysis assessment was calculated by taking the number of steps that met these criteria and dividing by the total number of possible steps on the master task analysis for a percentage correct (see Appendix A). Simultaneous prompting. Simultaneous prompting is a strategy in which the job coach gives a prompt (i.e., a discriminative stimulus that occasions correct responding) immediately after a target stimulus (Gibson & Schuster, 1992). To assess accurate implementation of simultaneous prompting, the job coach participant was told to use a task analysis to implement the strategy with an actor (during the simulation assessment) or with an intern with IDD (during generalization assessment); the number of trials in each session was equal to the number of steps in the task analysis. During baseline and probe assessments, the job coach participant was instructed to use the task analysis she had generated during the task analysis baseline assessment to implement simultaneous prompting. This was done so that job coach participants were not provided with a model of a task analysis before beginning intervention on task analysis. During intervention assessments, a task analysis was provided by the instructor to assess implementation of simultaneous prompting. Implementation was recorded and coded for accuracy afterward. The target stimulus for the first step in the task was a task direction from the job coach participant, and for the rest of the task it was the completion of the previous step in the task analysis. Each 9 session consisted of the number of coded trials equal to the number of steps in the task analysis for that particular task, beginning when the job coach participant gave a task direction such as “Let’s set the table.” Each subsequent trial began as soon as the job coach participant provided reinforcement (or when the actor/intern completed the previous step in the task if no reinforcement was provided). For job coach participants, correct implementation involved: 1) giving an initial task direction; 2) providing a prompt within 1 s of the task direction or reinforcement/completion of the previous step (e.g. “set the napkin on the placemat); 3) providing immediate reinforcement in the form of vocal praise within 1 s after completion of each step (e.g. “great job”); and 4) providing specific reinforcement for completion of the entire task (e.g. “Great job, you set the table!”). Prompts were vocal or model, and the first prompt the job coach participant gave that was followed by a correct response was considered the successful prompt for all subsequent trials in that session (see Appendix B). The percent of correctly implemented trials was calculated by dividing the number of trials implemented correctly by the total number of trials on the task analysis for that session. Least-to-most prompting. Least-to-most prompting (also known as the system of least prompts) is the delivery of prompts from various levels in a predetermined hierarchy, starting with the least intrusive prompt and gradually working up to the most intrusive, if necessary (Alberto & Schofield, 1979). To assess implementation of least-to-most prompting, the number of trials in each session was equal to the number of steps in the task analysis. Each trial began identically to the simultaneous prompting procedure. Correct implementation involved delivering the task direction, followed by waiting 5-7 s for the actor/intern to respond after the target stimulus. No prompts or reinforcement was delivered during this time. If the actor/intern did not 10 respond or responded incorrectly, the job coach participant began with the lowest prompt in the hierarchy. In sequential order, the hierarchy included a gestural prompt toward the required item or items for that step (e.g. pointing to the milk), a vocal verbal prompt (e.g. “Pour the milk in the measuring cup up to the red line”), and a model prompt (e.g. saying “like this” and pretending to pour milk). If the actor/intern did not respond within 5-7 seconds of the prompt or if the actor/intern made an error, the job coach participant then moved on to the next highest prompt in the hierarchy and waited 5-7 seconds for the actor/intern to respond again. This process was repeated until the actor/intern responded correctly. When the actor/intern responded correctly to a prompted step, the job coach participant provided reinforcement in the form of specific vocal praise (e.g., “great job setting the napkin on the placemat”). Last, the job coach participant delivered specific reinforcement for the overall task (e.g. “Great, you set the table!”). The percent of correctly implemented trials was calculated by dividing the number of trials implemented correctly by the total number of trials on the task analysis for that session (see Appendix C). During simulation assessments for least-to-most prompting, actors were instructed to make a predetermined number of errors on predetermined steps in the task analysis. A random number generator was used to determine the number of steps in the task analysis where errors would occur (between two and five steps total), at which steps the errors would occur, and how many prompts would be required before the actor was to respond correctly. Interobserver Reliability and Procedural Fidelity. The first author served as the primary coder for participant task analyses and implementation videos and determined through visual analysis when participants were ready to progress through assessment and training. A reliability coder was trained by the first author during multiple sessions that detailed the study’s 11 measurement procedures. The reliability coder practiced with sample videos and task analyses for each strategy until 90% agreement with the first author was met and they were considered reliable. Interobserver agreement (IOA) data were collected for 34% of all videos and task analyses. At least 33% of assessments for each strategy in each phase (baseline, intervention, generalization) were included in the IOA assessment. Point-by-point IOA was 94% for task analysis, 95% for simultaneous prompting, and 91% for least-to-most prompting across observed sessions. Point-by-point IOA was calculated by taking the number of steps where coders agreed on a rating, dividing that number by the sum of the number of agreements and disagreements, and multiplying by 100 to obtain a percentage of agreement. The coder also completed procedural integrity checklists for 33% of the recorded videos to ensure the actors were following the guidelines specified by the first author (e.g. making the correct number of errors, not providing feedback or reinforcement to job coach participants). Actor procedural fidelity was 93% across observed sessions and was calculated by taking the number of steps actors implemented correctly, dividing by the total number of steps, and multiplying by 100 to obtain a percentage of steps implemented correctly. Finally, a coder collected procedural fidelity data on the training implemented by the first author for 33% of sessions for each strategy by phase (see Appendix E). Instruction was coded for a variety of criteria, including having all materials prepared, providing didactic instruction, providing demonstrations of the strategy, and providing time for participants to rehearse the strategy before assessment. Procedural fidelity for training was 100% across all observed sessions and was calculated by taking the number of criteria the instructor implemented correctly, dividing by the total number of criteria, and multiplying by 100 to obtain a percentage of steps implemented correctly. 12 Experimental Design The study used a multiple probe design across behaviors with probe conditions, replicated across three participants (Gast, Lloyd, & Ledford, 2014). This allowed the study to demonstrate experimental control if job coach implementation fidelity increased only when the training was implemented for each behavior. The job coach participants began the baseline probe for all strategies at the same time, and the introduction of the intervention was staggered across strategies beginning when each participant showed stable responding for the first strategy. Subsequent baseline probes for all three strategies were conducted when each participant met criterion of 85% of steps implemented correctly for the current strategy during training. Training began for the next strategy after one session of the baseline probe. Generalization probes with intern participants were conducted for each participant following completion of training on all strategies. Procedures Baseline procedures. In the baseline condition, job coach participants were provided with a brief definition of a strategy (see Table 2) and then asked to write a task analysis or implement the strategy with an actor to the best of their ability. Baseline probes were conducted for three sessions. No further training or instructions were provided. Intervention. The first author acted as instructor to provide training on the three instructional strategies to the job coach participants. The first author was a graduate student in Applied Behavior Analysis who had at least 7 years of experience as a job coach. Initially training was conducted in group sessions and training sessions were intended to be held three times each week. Although all three participants were able to attend the first three training sessions together, they had conflicting work and school schedules and struggled to find 13 consistent times when all three could attend a session together. As a result, initial group meetings were only held an average of one time per week. Since this would greatly elongate the time it would take to complete training and assessment for all three strategies, the training structure was changed to individual sessions after the third task analysis training session. Individual training sessions did not differ from the group sessions in any other way. After individual sessions were implemented, participants were able to progress through the training protocol at different rates. For example, Participants 1 and 3 met mastery criteria for task analysis after three sessions, but Participant 2 required five sessions before mastering the strategy and moving on to simultaneous prompting. Training consisted of BST with multiple exemplar training for each strategy separately. The first session for each of the three strategies was 2 hours, followed by 1-hour sessions for all subsequent trainings for that strategy. During the first session, the instructor gave a PowerPoint presentation where the strategy was defined and each step of implementation was described. The instructor then gave examples of how a job coach could implement the strategy and modeled the strategy with a sample task. Next, the job coach participants were asked to rehearse implementing the strategies by either writing a task analysis or practicing with an actor who roleplayed as an individual learning the task. The instructor gave brief in-the-moment feedback on performance to each participant as they implemented the strategy and provided more detailed feedback after each rehearsal. Job coach participants always rehearsed the strategy at least once but were given the opportunity to rehearse additional times, if desired. The subsequent 1-hour training sessions consisted of a brief didactic overview of the strategy, at least one example and model of the strategy, and additional opportunities for roleplay practice for the remainder of the training session. 14 To promote skill generalization, multiple exemplar training was embedded within BST, allowing the job coach participants to learn and practice each skill with a variety of stimuli that are reflective of those they might encounter in a natural setting, but which share certain characteristics that come to control a similar response for all the various stimuli (Cooper, Heron, & Heward, 2007; Greer, Yaun, & Gautreaux. 2005). Thus, to the extent possible, a new task was used for instruction and assessment during each session for a given strategy, but the same implementation steps were used for each exemplar. If a task was repeated, it was repeated either between task analysis and simultaneous prompting, or task analysis and least-to-most prompting; to avoid carryover effects, no tasks were repeated between probes or between the two prompting strategies. At the end of each session, participant implementation was assessed and recorded in the same manner as baseline. Percentage of steps performed accurately during this simulation was used as the dependent variable. Different tasks were used for each assessment to avoid practice effect and to allow participants to be assessed on the ability to generalize their training across novel tasks throughout the training process. For task analysis, the job coach participants were provided with a novel task and asked to write a task analysis detailing each step one would need to complete to perform the task. During each assessment, job coach participants were provided with the sheet of basic definitions for the strategy, a paper and pen to write their task analysis, and a short list of parameters for the task. This list consisted of the name of the task, brief descriptions or visuals of the task materials and their initial setup, and the same information for the conclusion of the task. For example, when asked to write a task analysis for setting the table, the materials used for the task were placed on the table (not set up) and the job coach participants were provided with a picture of what the materials should look like once setting the table had 15 been completed. After the job coach participant wrote the task analysis, the instructor read their task analysis aloud and provided reinforcement and/or corrective feedback. For simultaneous and least-to-most prompting, the job coach participants were asked to conduct a simulated training session with an actor. For these simulations, each job coach participant was paired with an actor who acted as though they were learning a task. Each job coach participant was asked to implement the current target procedure (i.e., simultaneous prompting or least-to-most prompting) to teach the actor to perform a vocational task. For simultaneous prompting, the job coach participant was asked to use simultaneous prompting to teach a skill to an individual who was learning it for the first time. For least-to-most prompting, the job coach participant was asked to use least-to-most prompting to fade out prompts for an individual who had been learning the task for a while. No social praise or reinforcement was provided from the actors during the simulation assessment itself. Each performance was recorded. Following completion of the simulation, the video was transferred to a laptop and the instructor and job coach participant watched the footage together while the instructor provided reinforcement and/or corrective feedback. The video was also later coded for accuracy of implementation to assess job coach participant skill acquisition. Training on one strategy continued until the job coach participant met mastery criterion of 85% or above on the simulation assessments for three sessions. Once mastery was met, all three strategies were probed again. Then the next strategy was introduced in a two-hour session and BST was implemented for that strategy until the job coach participant met mastery criterion and all three strategies were probed again. Then the last strategy was introduced in a two-hour session and BST was implemented until the job coach participant met mastery criterion. All three 16 strategies were probed one more time and then the job coach participant was moved to generalization. Generalization. Following training, generalization data were collected on job coach participant implementation of the instructional strategies with novel tasks with an individual with IDD. Generalization was assessed for each of the three strategies (task analysis, simultaneous prompting, and least-to-most prompting) by measuring job coach participant implementation fidelity for teaching an individual with IDD to make a milkshake and bag groceries. Specifically, to assess task analysis, the job coach participant was asked to write a task analysis for each task and the task analysis was scored as described for the dependent measure. The job coach participant was then directed to use the two prompting strategies to teach the intern how to complete each task. The two tasks were presented to each participant in a random order, and in each session the job coach implemented simultaneous prompting with the first task and least-to- most prompting with the second. The tasks used for simultaneous prompting and least-to-most prompting alternated each session, and the first task was chosen by flipping a coin. For example, Participant 3 began with bagging groceries for simultaneous prompting in the first session, so the order for bagging groceries across the three sessions was simultaneous prompting, least-to-most prompting, and then back to simultaneous prompting. Her order for making a milkshake was the inverse (e.g. least-to-most, simultaneous, least-to-most). Social Validity Social validity was collected from the job coach participants and from the interns with IDD. Social validity was assessed for job coach participants through questionnaires before and after training. Before training, job coaches were asked whether they had heard of each strategy, and then asked to what degree they believed they could implement the strategy. Participants 17 rated their ability to implement the strategies on a scale from 1 to 5 (1 = unable to implement the strategy correctly; 2 = unable to implement the strategy correctly most of the time; 3 = able to implement the strategy correctly sometimes; 4 = able to implement the strategy correctly most of the time; 5 = able to implement the strategy correctly all the time). After training, the second question was asked again, and compared to the pre-training scores to assess the degree to which the job coaches thought they had mastered the strategy. After the training, job coaches were also asked open-ended questions about their thoughts on the training sessions and procedures, and to what degree they found them acceptable and helpful. For the interns with IDD, a close-ended interview was conducted after the final generalization probe. The interns with IDD were presented with the following questions and asked to rate them on a 5-point scale (1=Poorly, 2=Fairly, 3=Alright, 4=Good, 5=Great): 1. How well did your job coach do when helping to teach you each task? 2. When you knew what to do next, how well did your job coach do letting you try steps on your own? Data Analysis Visual analysis was used to assess for a functional relation between the training and the performance of the three instructional strategies. Sessions were coded and data were graphically displayed following each intervention session to allow for ongoing evaluation of behavior change within and across instructional strategies (Wolery & Harris, 1982). Upon introduction of the training, changes in level and trend were assessed for each instructional strategy for each job coach participant to determine the effectiveness of the intervention. Additionally, replication of effects of the training was used to evaluate the internal validity of the training. Overall, demonstrations of effect were shown because job coach participants displayed low levels of 18 performance during baseline conditions with a non-accelerating trend, and performance immediately increased in level after the introduction of the training intervention for each strategy.. 19 Results A multiple probe experiment was conducted for each of the three participants across the three instructional strategies, providing a potential of nine opportunities to demonstrate an experimental effect. Using visual analysis of the data from assessments with actors and the generalization probes, nine demonstrations of experimental effect were shown. Participant 1 Results of the intervention on acquisition of the instructional strategies for Participant 1 (P1) are displayed in Figure 1. P1 displayed low levels of accurate performance on all instructional strategies during baseline probe sessions, with an immediate increase in performance in targeted skills when the intervention was implemented. For task analysis, P1’s performance during the first baseline assessment was 75%, before dropping to 50% for the two subsequent sessions. When P1 entered intervention for task analysis, she showed an immediate effect in her first intervention session and met mastery criterion within three intervention sessions, with scores of 100%, 90%, and 100%. For simultaneous prompting, P1’s baseline data were stable near 30%. When P1 entered intervention for simultaneous prompting, she showed an immediate effect in her first intervention session with a score of 100% and performance remained at that level. For least-to-most prompting, P1’s performance was somewhat variable during baseline as she scored 20% on the initial probe, 5% for the subsequent 3 probes and then 20% on her final probe. There was an immediate effect following the first intervention session, as she scored 92%, 90%, and 90%. P1 generalized all three strategies to working with an individual with IDD during her completed generalization probes. 20 Participant 2 Results of the intervention on acquisition of the instructional strategies for Participant 2 (P2) are displayed in Figure 2. P2 displayed low levels of accurate performance on all instructional strategies during baseline probe sessions, with an immediate increase in performance in targeted skills when the intervention was implemented. For task analysis, P2’s performance in the first baseline assessment was 67%, before dropping to 14% for the two subsequent sessions. When P2 entered intervention for task analysis, she showed an immediate effect in her first intervention session and met mastery criterion within five sessions, with scores of 79%, 100%, 57%, 100%, and 92%. For simultaneous prompting, P2’s baseline data were somewhat variable, with a slight downward trend from 25% to 5% in the first three baseline probes before rising back to 31% for the final baseline probe. When P2 entered intervention for simultaneous prompting, she showed an immediate effect in her first intervention session with a score of 94%. Performance maintained at this level and P2 met mastery criterion within three sessions. For least-to-most prompting, P2’s performance during the initial baseline probe was 24%, before increasing to around 30% and maintaining at that level for subsequent baseline probes. There was an immediate effect following the first intervention session, as she scored 83%. After a slight decrease to 72%, P2’s performance increased to 95% and maintained between 90% and 95% for subsequent sessions, meeting mastery criterion after five sessions. P2 generalized all three skills to working with an individual with IDD; with the exception of her first generalization probe for least-to-most prompting where her performance decreased to 74%, P2’s performance was between 85% and 100% for generalization probes for all three strategies. 21 Participant 3 Results of the intervention on acquisition of the instructional strategies for Participant 3 (P3) are displayed in Figure 3. P3 displayed low levels of accurate performance on all instructional strategies during baseline probe sessions, with an immediate increase in performance in targeted skills when the intervention was implemented. For task analysis, P3’s performance during the first baseline assessment was 83%, before dropping to 36% and 29% for the two subsequent sessions. When P3 entered intervention for task analysis, she showed an immediate effect in her first intervention session. She met mastery criterion within three intervention sessions, with scores of 86%, 100%, and 93%. For simultaneous prompting, P3’s baseline data were stable near 37%. When P3 entered intervention for simultaneous prompting, she showed an immediate effect in her first intervention session with a score of 100%. P3’s performance decreased slightly to 94% during the second intervention session before returning to 100% and maintaining at that level. For least-to-most prompting, P3’s performance was relatively stable during baseline at around 25%. There was an immediate effect following the first intervention session, as she scored 83% before increasing to 94% and maintaining between 90% and 100% for subsequent sessions and met mastery criterion after four sessions. P3 generalized all three skills to working with an individual with IDD; with the exception of her first generalization probe for least-to-most prompting where her performance decreased to 76%, P3’s performance was between 90% and 100% for generalization probes for all three strategies. Social Validity Before training, all three participants indicated that they had never heard of any of the target strategies. All three subjects rated their ability to implement simultaneous prompting and least-to-most prompting as 1 (unable to implement the strategy correctly), and P1 and P2 rated 22 their ability to implement task analysis as 1 as well. P3 rated her ability to implement task analysis as 2 (unable to implement the strategy correctly most of the time. After training, P1 and P3 both rated their ability to implement task analysis and simultaneous prompting as 5 (able to implement correctly all the time), and rated least-to-most prompting as 4 (able to implement the strategy correctly most of the time). P2 rated her ability to implement task analysis as 5, simultaneous prompting as 4, and least-to-most prompting as 3 (able to implement the strategy correctly sometimes). For the open-ended questions, participants indicated that the length of the overall training was appropriate. P1 and P2 indicated that the training lasted a long time (over a month), also stated that this was due to scheduling and the detail of the training. All three participants indicated that the training sessions themselves were an acceptable length. All participants indicated that the teaching presentations, instructor demonstrations, opportunities for rehearsal, and instructor feedback contributed positively to the training and were helpful for them to learn the target strategies. Participants also stated that they would likely use their knowledge of the target strategies in their future careers (employment law, occupational therapy, and neuroscience for P1, P2, and P3, respectively). The strategies were also viewed as being beneficial for individuals with IDD who might receive support from job coaches who use systematic instructional practices. Finally, participants expressed that they felt the training was appropriate for new job coaches, and that the training would have a high success rate for teaching newly hired job coaches with no prior experience. Intern participants were asked two brief questions following conclusion of generalization probes. All three rated their job coach’s helpfulness when learning the generalization tasks as 5 (great). When asked how well their job coach let them try steps on their 23 own during least-to-most prompting, all three intern participants rated their job coach’s performance as 5 (great). 24 Discussion Despite their important role in supported employment for individuals with IDD, job coaches rarely receive instruction in systematic instructional strategies (Rogan & Held, 1999; Wehman et al., 2014). The results of the current study indicate that BST with multiple exemplar training can be effective at training job coaches to implement systematic instructional strategies. All job coach participants showed an immediate and significant increase in performance during simulation assessments following the introduction of the intervention, and all three mastered each strategy within three to five intervention sessions. Experimental effects were seen across strategies and replicated across participants. During generalization sessions with interns with IDD, job coach participants showed generalization across all three strategies and were able to successfully implement the strategies to teach an intern with IDD. These outcomes replicate and extend upon previous job coach training literature (e.g., Brock et al., 2016) in a number of ways. First, the results indicate that a BST model can effectively teach job coaches with no prior experience or training how to implement systematic instructional strategies with fidelity. In Brock et al. (2016), job coach participants had between 3 and 30 years of experience providing job coaching. In the current study, none of the participants had any experience job coaching or providing systematic instruction, and all three participants acquired the instructional strategies. This indicates that not only can BST procedures similar to Brock et al. (2016) effectively train job coaches with years of experience, but that the procedures are appropriate for all levels of skill and experience. Based on these findings, similar procedures could be used by job coach employers to train newly hired staff on how to analyze a task, begin to teach a task with minimal learner errors, and systematically fade prompting to promote learner independence. 25 Second, individual training sessions can effectively teach job coaches to implement systematic instruction in a short amount of time. Though the current study began with group training, much like Brock et al. (2016), scheduling made these sessions infrequent. At this original pace, completing all three strategies would have taken months. After the first three task analysis training sessions, sessions were changed to an individual format to better accommodate participant schedules. After the change in format, training was able to progress much faster and more efficiently than previously, with participants meeting the instructor at least three times a week. Despite this change in format after teaching task analysis, there was no marked change in acquisition across the three strategies. Though the original goal of this study was to assess group training in order to develop a more affordable and practical training protocol for use by job coach employers, these results are still of practical use. There may be occasions when an employer only hires one new job coach at a particular point in time, and in cases like this a group training may not be an option. Developing individual training protocols for job coaches can also allow the training to move at an individualized pace; if most job coaches in a training group have mastered task analysis, but one has not, individual training sessions may be helpful in catching up the new trainee with the rest of the group. Third, multiple exemplar training allowed for the participants to successfully generalize use of the instructional strategies across multiple tasks. Brock et al. (2016) showed that job coaches could successfully acquire the three target strategies with two tasks during training sessions and generalize the use of these strategies to one untrained task (bagging groceries). In the current study, multiple exemplar training was used to promote generalization throughout the entire intervention. The instructor modeled a variety of different tasks during training sessions, participants rehearsed each strategy with a different task during each session, and a novel task 26 was used each time participants were assessed on their implementation fidelity for a given strategy, allowing for exposure to over 18 different tasks. All three participants mastered the three strategies and used them with a new task each session, indicating that these training procedures are effective at programming for generalization across tasks. These findings have important implications for organizations providing training to job coaches and show that practicing systematic instructional strategies with a variety of different stimuli and tasks can help improve job coaches’ ability to implement these strategies in various different settings. Finally, the current study highlights the importance of assessing generalization to working with individuals with IDD, as performance on least-to-most prompting decreased for two of the three job coach participants during their first generalization probe. Previous job coach training literature only assessed implementation in a simulated roleplay setting (Brock et al., 2016). Training sessions and assessments in the current study were conducted with actors, but participants were also assessed when implementing the target strategies with individuals with IDD. During generalization probes, P2 showed a slight decrease in performance on task analysis and simultaneous prompting but remained at or above mastery criterion for both strategies. P3’s performance for task analysis and simultaneous prompting during the generalization probes was similar to her level of fidelity in previous assessments with an actor. This indicates that job coach performance during roleplay assessments may be indicative of similar performance in a live setting with individuals with IDD. However, in some cases participant performance during generalization probes was more variable than roleplay performance, and not all participants maintained performance at mastery levels for all strategies during these probes. Both P2 and P3 showed a decrease in performance for least-to-most prompting during their first generalization session before performance improved and returned to mastery levels. Still, despite a slight 27 decrease in performance, all job coach participants remained well above baseline levels during all generalization assessments. Overall, the current study provides some initial evidence that job coaches can be quickly and effectively trained using a BST with multiple exemplar training model to implement systematic instructional strategies during simulated roleplay assessments; that they can generalize these strategies across various tasks and stimuli; and that they can generalize these strategies to instruction of individuals with IDD. These findings expand upon research by Brock et al. (2016); however, the current study used individual training, which may not be practical for organizations that provide and/or design job coach training. Still, given the lack of training that is currently being provided to most newly hired job coaches (Hall et al., 2018), these procedures and findings provide a feasible option for those who hire and train job coaches. Limitations and Future Research A number of limitations of the current study provide directions for future research. First, there were only three generalization probes used to assess implementation with individuals with IDD. These three sessions (with only two task analysis data points) can provide limited implications for the participants’ ability to generalize these strategies, but more data are needed to truly assess this generalizability. Further, generalization probe sessions took place in a controlled classroom environment with only two tasks. This is a much different setting than those where job coaches will actually be expected to implement these procedures. Job sites can be loud, busy, and chaotic; job coaches could be easily distracted while trying to provide instruction. Job coaches are also expected to be capable of adapting rapidly to new situations and may need to provide instruction with little notice in advance. Implementation of these systematic instructional strategies may not generalize so easily from a quiet, controlled room with ample 28 time to prepare, to a bustling cafeteria where an employee with IDD is expected to rapidly serve different items to each customer during the lunch rush. Future research should assess the ability of job coach participants to use their newly acquired skills in real employment settings. Additionally, generalization probes did not investigate skill acquisition of the interns with IDD. Although the job coaches were able to implement the strategies with high levels of fidelity in most of the generalization probes, intern learning and performance for the target task was not assessed. The ability to successfully implement systematic instructional strategies may not matter if those strategies do little or nothing to improve rates of skill acquisition for the individuals the job coaches aim to serve. Future research could compare skill acquisition for individuals with IDD when taught by a job coach who has not received training on systematic instruction to those who are taught by a job coach who has received such training. Research evidence that these strategies can be successfully applied in an employment setting and also improve skill acquisition or employment outcomes for individuals with IDD could be used to advocate for more widespread adoption of behaviorally-based training protocols for job coaches. Another limitation involves the process of developing tasks and task analyses for the training sessions. The data indicate that the selected tasks may not have been as similar in difficulty and complexity as intended. For the first baseline task, setting the table, all three participants performed significantly higher on developing a task analysis than they did for the other two baseline tasks. For P1 the difference between the first and second baseline sessions was 25 percentage points; for P2 the difference was 53 percentage points; and for P3 the difference was 37 percentage points. Performance on subsequent baseline probes generally aligned with performance from the second and third data points, indicating that the first session was an outlier. The table setting task may have been less difficult or complex than the 29 subsequent tasks, and it may have been more immediately clear which discrete steps needed to happen and in which order. This outlier may call into question the validity of the other data; there is a possibility that participant performance may have been affected by varying complexity and difficulty of the tasks, rather than performance being a function of only whether training had been received. Future research on job coach training could conduct a companion study to assess standardization of tasks and task analyses to ensure that results are truly indicative of job coach skill acquisition, rather than the difficulty of the assigned task. The stability of all other data, however, minimize these concerns. Finally, the current study only included three participants. While these participants came from a variety of ethnic backgrounds and fields of study, the small sample size reduces the ability to draw broader conclusions about the effectiveness of the training with job coaches from different backgrounds. Future research could conduct studies with more participants, as well as assessing both individuals who have experience job coaching and those who do not. Different training formats or protocols may be best suited to job coaches with varying levels of experience and investigating this question could lead to more efficient use of training resources. Conclusion Job coaches are a key part of the employment support process for individuals with IDD. One of the primary ways job coaches support these individuals is through direct instruction on job tasks. Little research has been conducted to investigate the efficacy of job coach training protocols for systematic instructional strategies, and no studies known to the current research team evaluate job coach implementation with individuals with IDD rather than with simulated roleplay scenarios. The current study provides evidence that a BST model incorporating multiple exemplar training can be used to effectively train new job coaches to implement task analysis, 30 simultaneous prompting, and least-to-most prompting across a variety of basic tasks. Additionally, it expands upon the previous literature by providing an assessment of job coach implementation with individuals with IDD to identify if strategies practiced in a simulated setting generalize to a live setting. The training protocol analyzed in the current study could have implications for job coaches and individuals with IDD at large. Individuals with IDD deserve the most appropriate and high-quality support available, and job coaches deserve to be given the appropriate tools to provide that support. Further evaluation of job coach training protocols could contribute to more widespread use of evidence-based training methods to teach systematic instructional strategies in a vocational setting, which in turn may improve the skills and confidence of job coaches and, most importantly, improve employment outcomes for the individuals with IDD that they serve. 31 APPENDIX 32 Table 1: List of Tasks Used in Training Sessions and Assessments Task name Setting the table Making packets Solving a math problem (with calculator) Putting on a jacket Making a sandwich Using a vending machine Making toast Cleaning a table Folding laundry Making cards Sending an email Hole-punching papers for a binder Cleaning a whiteboard Copying a document Vacuuming Renaming a document Making a milkshake Bagging groceries Number of steps in task analysis Phase(s) Used 8 10 Baseline probe Baseline probe 10 Baseline probe Model/practice only Task analysis, least-to-most prompting Model/practice only Task analysis, least-to-most prompting Task analysis, simultaneous prompting Task analysis, simultaneous prompting Task analysis, simultaneous prompting Baseline probe Baseline probe Least-to-most prompting Least-to-most prompting Least-to-most prompting Baseline probe Generalization probes Generalization probes 8 9 9 8 10 8 9 8 10 9 10 9 8 11 9 33 Table 2: Definitions of Instructional Strategies Provided to Job Coach Participants During All Assessments Strategy Task analysis Definition Breaks down complex behaviors or skills into smaller steps that can be more easily taught (Cooper, Heron, & Heward, 2007, p. 706). Simultaneous prompting The instructor gives a controlling prompt immediately after a task direction or completion of the previous step (Gibson & Schuster, 1992). Least-to-most prompting The delivery of prompts from various different levels in a predetermined hierarchy, starting with the least intrusive prompt and gradually working up to the most intrusive, if necessary (Alberto & Schofield, 1979). 34 Figure 1. Percentage of steps Participant 1 performed correctly across instructional strategies during probe, training, and generalization sessions. Note: P2 = probe 2; P3 = probe 3; P4 = probe 4; BST = behavior skills training. 35 Figure 2. Percentage of steps Participant 2 performed correctly across instructional strategies during probe, training, and generalization sessions. Note: P2 = probe 2; P3 = probe 3; P4 = probe 4; BST = behavior skills training. 36 Figure 3. Percentage of steps Participant 3 performed correctly across instructional strategies during probe, training, and generalization sessions. Note: P2 = probe 2; P3 = probe 3; P4 = probe 4; BST = behavior skills training. 37 REFERENCES 38 REFERENCES Alberto, P. S., & Schofield, P. (1979). An instructional interaction pattern for the severely handicapped. Teaching Exceptional Children, 12, 16-19. doi:10.1177/004005997901200105 Brock, M. E., Cannella-Malone, H. I., Schaefer, J. M., Page, E. J., Andzik, N. R., & Seaman, R. L. (2016). Efficacy of training job coaches to implement evidence-based instructional strategies. Journal of Vocational Rehabilitation, 45, 351-364. doi:10.3233/JVR-160835 Britton, N. S., Collins, B. C., Ault, M. J., & Bausch, M. E. (2017). Using a constant time delay procedure to teach support personnel to use a simultaneous prompting procedure. Focus on Autism and Other Developmental Disabilities, 32, 102-113. doi:10.1177/1088357615587505 Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River, NJ: Pearson. Gast, D. L., Lloyd, B. P., & Ledford, J. R. (2014). Multiple baseline and multiple probe designs. In Gast, D. L. & Ledford, J. R. (Eds.), Single case research methodology: Applications in special education and behavioral sciences (pp. 251- 296). New York, NY: Routledge. Gibson, A. N., & Schuster, J. W. (1992). The use of simultaneous prompting for teaching expressive word recognition to preschool children. Topics in Early Childhood Special Education, 12(2), 247-267. doi:10.1177/027112149201200208 Greer, R. D., Yaun, L., & Gautreaux, G. (2005). Novel dictation and intraverbal responses as a function of a multiple exemplar instructional history. The Analysis of Verbal Behavior, 21(1), 99-116. doi:10.1007/BF03393012 Hall, A. C., Butterworth, J., Winsor, J., Kramer, J., Nye-Lengerman, K., & Timmons, J. (2018). Building an evidence-based, holistic approach to advancing integrated employment. Research and Practice for Persons with Severe Disabilities, 43(3), 207- 218. Kraus, L., Lauer, E., Coleman, R., & Houtenville, A. (2018). 2017 Disability Statistics Annual Report. Durham, NH: University of New Hampshire. Retrieved from https://disabilitycompendium.org/sites/default/files/user- uploads/2017_AnnualReport_2017_FINAL.pdf Marshall, T., Goldberg, R. W., Braude, L., Dougherty, R. H., Daniels, A. S., Ghose, S. S., George, P., & Delphin-Rittmon, M. E. (2014). Supported employment: assessing the evidence. Psychiatric Services, 65(1), 16-23. doi:10.1176/appi.ps.201300262 Miltenberger, R. G. (2012). Behavior modification: Principles & procedures (5th ed.). Belmont, CA: Wadsworth. 39 Parsons, M. B., Reid, D. H., Green, C. W., & Browning, L. B. (2001). Reducing job coach assistance for supported workers with severe multiple disabilities: An alternative off- site/on-site model. Research in Developmental Disabilities, 22, 151-164. doi:10.1016/s0891-4222(01)00064-6 Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: a guide for practitioners. Behavior Analysis in Practice, 5(2), 2-11. doi:10.1007/BF03391819 Rogan, P. M., & Held, M. (1999). Paraprofessionals in job coach roles. Research and Practice for Persons with Severe Disabilities, 24, 273-280. doi:10.2511/rpsd.24.4.273 Roux, A. M., Rast, J. E., Anderson, K. A., & Shattuck, P. T. (2017). National autism indicators report: developmental disability services and outcomes in adulthood. Retrieved from Drexel University Life Course Outcomes website: https://drexel.edu/autismoutcomes/publications-and-reports/publications/National- Autism-Indicators-Report-Developmental-Disability-Services-and-Outcomes-in- Adulthood/ Rusch, F. R., & Hughes, C. (1989). Overview of supported employment. Journal of Applied Behavior Analysis, 22, 351-363. doi:10.1176/appi.ps.201300262 Schuster, J. W., Gast, D. L., Wolery, M., & Guiltinan, S. (1988). The effectiveness of a constant time-delay procedure to teach chained responses to adolescents with mental retardation. Journal of Applied Behavior Analysis, 21(2), 169-178. doi:10.1901/jaba.1988.21-169 U.S. Department of Labor, Bureau of Labor Statistics. (2019). Persons with a disability: labor force characteristics - 2018. Retrieved from https://www.bls.gov/news.release/pdf/disabl.pdf Walker, V. L., & Snell, M. E. (2017). Teaching paraprofessionals to implement function- based interventions. Focus on Autism and Other Developmental Disabilities, 32, 114- 123. doi:10.1177/1088357616673561 Wehman, P., Lau, S., Molinelli, A., Brooke, V., Thompson, K., Moore, C., & West, M. (2012). Supported employment for young adults with autism spectrum disorder: Preliminary data. Research and Practice for Persons with Severe Disabilities, 37, 160- 169. doi:10.2511/027494812804153606 Wehman, P. H., Schall, C. M., McDonough, J., Kregel, J., Brooke, V., Molinelli, A., . . . & Thiss, W. (2014). Competitive employment for youth with autism spectrum disorders: Early results from a randomized clinical trial. Journal of Autism and Developmental Disorders, 44, 487-500. doi:10.1007/s10803- 013-1892-x 40