TEACHING STAFF TO IMPLEMENT MAND TRAINING WITH CHILDREN WITH ASD THROUGH TELEHEALTH By Shelby R. Rosalik A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Applied Behavior Analysis – Master of Arts 2021 TEACHING STAFF TO IMPLEMENT MAND TRAINING WITH CHILDREN WITH ASD ABSTRACT THROUGH TELEHEALTH By Shelby R. Rosalik Telehealth services have increased substantially in the field of applied behavior analysis (ABA) since the start of the COVID-19 pandemic, though little research exists to empirically evaluate the efficacy of direct ABA telehealth treatment or more specifically, how behavior technicians can be trained to implement such treatment. The present investigation utilized a nonconcurrent multiple baseline design across participants to evaluate the use of an online behavioral skills training (BST) approach to teach behavior technicians to implement 20-min mand training sessions via telehealth with children diagnosed with autism spectrum disorder (ASD). The training phase of the study consisted of both role-play with feedback as well as feedback during sessions with the child participant. Results showed increases in behavior technician’s percentage of accurate implementation and rate of fully correct trials implemented following the training. Child participants also showed increased rates of independent mands following the BST implementation. Thus, BST may be an effective approach to teach behavior technicians to deliver mand training via telehealth. Key words: direct ABA telehealth treatment, behavioral skills training, mand training, autism spectrum disorder TABLE OF CONTENTS LIST OF TABLES ......................................................................................................................... iv LIST OF FIGURES .........................................................................................................................v Introduction ......................................................................................................................................1 Method .............................................................................................................................................7 Participants and Setting ...............................................................................................................7 Materials ......................................................................................................................................8 Dependent Measures .................................................................................................................10 Adult Participant Dependent Variables ...............................................................................10 Child Participant Dependent Variables ...............................................................................10 Interobserver Agreement .....................................................................................................12 Experimental Design .................................................................................................................13 Procedures .................................................................................................................................14 Baseline ...............................................................................................................................14 Training ...............................................................................................................................15 Post-training with Feedback ................................................................................................16 Maintenance ........................................................................................................................17 Procedural Integrity of Training Implementation......................................................................17 Results ............................................................................................................................................18 Adult Participants ................................................................................................................14 Child Participants ................................................................................................................22 Discussion ......................................................................................................................................26 REFERENCES ..............................................................................................................................32 iii LIST OF TABLES Table 1. Accurate Implementation of Mand Training Checklist used to measure adult participant rate of fully correct trials and percentage of correct steps implemented. ........................................9 Table 2. Script one of four utilized in role play sessions during the training phase. .....................16 iv LIST OF FIGURES Figure 1. Adult participant’s rate of fully correct mand training trials. Ella’s graph is indented to represent the nonconcurrent design. ..............................................................................................19 Figure 2. Adult participant’s percentage of mand training steps implemented correctly and duration of sessions in minutes. Ella’s graph is indented to represent the nonconcurrent design. The circles represent the adult participants’ percentage of steps implemented correctly depicted on the left-hand y-axis. While the triangles represent the session duration (in minutes) for the first 10 contrived mand trials implemented depicted on the right-hand y-axis. ............................21 Figure 3. Child participant’s rate of correct independent mands per minute. Gia’s graph is indented to represent the nonconcurrent design.............................................................................23 Figure 4. Child participant’s percentage of correct independent mands. Gia’s graph is indented to represent the nonconcurrent design ...............................................................................................25 v Introduction The Council of Autism Service Providers (CASP) defines telehealth as a method for practitioners to deliver health care services directly to patients though the use of telecommunication, such as ZoomTM, SkypeTM, etc. (CASP Organizational Guidelines & Standards, 2020). Telehealth has been utilized within the field of applied behavior analysis (ABA) in a variety of ways, such as providing remote supervision to staff who are actively delivering in-person treatment to clients and utilizing technology to deliver remote staff or parent trainings. A limited, but recently growing, body of research indicates that telehealth can potentially be used to deliver ABA services to children with ASD wherein both the child and provider interact merely through technology (Rodriguez, 2020). However, because telehealth delivery of ABA requires staff to engage in specific behaviors that may not be necessary during face-to-face therapy (e.g., screen sharing to occasion specific behavior of clients or deliver reinforcers), empirically informed staff training methods are needed. A majority of the literature in the area of telehealth has been focused on Board Certified Behavior Analysists® (BCBA®) consulting with parents through video conferencing platforms (Rodriguez, 2020). Until very recently, the field of ABA has not focused on direct telehealth service delivery to clients in which an ABA provider works with the client through the use of an online video platform with little to no involvement of the caregivers or anyone else physically in the child’s space (i.e., instructions and prompts are given to the client through the screen) (Ferguson et al., 2020; Pellegrino & DiGennaro Reed, 2020; Pollard, LeBlanc, Griffin, & Baker, 2021; Rodriguez, 2020). Due to limited research utilizing direct telehealth services, there is little guidance as to when direct telehealth treatment may be suitable for a particular client, nor mechanisms for ensuring effective delivery. Nonetheless, during the COVID-19 pandemic, the 1 field of ABA quickly transitioned to telehealth to maintain social distancing guidelines while sustaining services (Rodriguez, 2020). Although COVID-19 provided a sense of urgency for advancing telehealth services in ABA, the pandemic is only one of several reasons to expand direct telehealth services within the field. ABA practitioners could utilize direct telehealth services for clients who live in rural areas with limited treatment options, when clients are placed on waiting lists to receive therapy, when a client or staff member has a mild sickness such as the common cold in order to avoid compromising the health of others, when a parent cannot bring their child to therapy (e.g., transportation problems, extreme weather, etc.), or when the client’s home environment does not support in home services (e.g. very small living space, parents are not comfortable with having staff in their home, etc.). Additional research is needed to fully understand the potential for delivering direct telehealth services, as well as mechanisms for making such service feasible across a range of situations. As ABA providers troubleshoot how to safely maintain services for clients during the COVID-19 pandemic and insurance providers continue to approve direct telehealth services, there has been some guidance within the literature as to when direct telehealth therapy may be a suitable approach. Rodriguez (2020) designed a guide that may assist BCBAs® in determining when direct telehealth may be appropriate for a client based on a program modifications assessment and a telehealth model selection matrix. These tools created by Rodriguez (2020) allow BCBAs® to assess if both the client and their caregivers are prepared for telehealth services based on both of their individual skill repertoires (e.g., the client’s intensity of problem behavior, ability to attend to the screen, and schedule of reinforcement, the amount of caregiver coaching needed, etc.) (Rodriguez, 2020). According to the matrix, a client should receive direct 2 telehealth treatment without caregiver support when they exhibit a low amount of problem behavior; can tolerate a thin schedule of reinforcement; are able to consistently respond to model, within-stimulus, and vocal prompts; and can attend to the screen for a long enough duration for sufficient learning to occur. Whereas clients who show deficiencies in these areas may require more intensive treatment models (e.g., additional caregiver support, modified teaching procedures, caregiver coaching, or intensive consultation with a more experienced BCBA®) before utilizing direct telehealth services (Rodriguez, 2020). Although the Rodriguez (2020) decision-making model provides a mechanism to conduct an initial evaluation of direct telehealth feasibility, there are some limitations that will require additional research. First, the model does not provide guidance about whether a child could be gradually taught to engage in direct telehealth therapy, even if they do not initially demonstrate the hypothesized pre-requisite behaviors. Additionally, the telehealth model selection matrix does not account for how long telehealth sessions should last, which could be a potential limitation when using direct telehealth treatment. Rodriguez (2020) states that clients typically receive between 20 to 40 hr of ABA services per week; however, most clients likely would not be able to sit in front of a screen for several hours a day, leaving practitioners with little guidance as to how long each telehealth session should be or how to potentially increase session length over time. And finally, the matrix assumes that Behavior Technicians (BTs) are prepared to deliver treatment directly to the child via telehealth. But given the relative novelty and somewhat challenging logistics of direct telehealth treatment, there may be barriers for technicians in swiftly transitioning to telehealth without additional training. Therefore, more research is needed to better understand logistical aspects of telehealth sessions, as well as potential benefits that could be obtained from various session lengths. 3 Although there is relatively little empirical support regarding direct telehealth treatment for children with ASD, one recent study evaluated the effectiveness of such treatment for tact training via ZoomTM for six children with ASD (Ferguson et al., 2020). The child participants were placed in dyads and asked to tact the name of two superheroes and their power when shown an image of the superhero through the screen. The experimenters found that all six participants learned both the name and power of their target stimuli. Therefore, these results by Ferguson et al. (2020) suggest that tacts can be taught to children with ASD via telehealth during relatively brief telehealth sessions. Similar to brief tact training sessions, providers may be able to successfully teach children with ASD to vocally mand via brief telehealth treatment sessions. There may also be some benefit to introducing children with ASD to direct telehealth treatment through mand training prior to introducing other skills. Similar to a child beginning ABA therapy for the first time, pairing a virtual technician and training session with a variety of preferred stimuli, as in mand training, could lead to increased motivation for future telehealth sessions (Michael & Sunderg, 2001; Plavnick & Ferreri, 2012). Utilizing mand training when starting a client on telehealth could potentially help the technician both pair themselves with reinforcing stimuli as well as the activity of socially interacting with someone through a screen. However, we found no prior investigations of procedures for mand training via telehealth. Conditioning of virtual social interactions may also help teach children with ASD social skills in a variety of contexts, such as video conferencing with family members who live far away. Mand training also allows for the length of telehealth session to remain very brief while potentially providing benefit. Therefore, this approach may be beneficial for a wider range of clients than those hypothesized as ready for telehealth (Rodriguez, 2020), such as children who 4 only attend to the screen for a short period of time. With these potential benefits, manding may be the best skill to target when introducing a client to ABA therapy via telehealth. However, when mand training is implemented via telehealth, procedures vary from those utilized in person. The behavior technician implementing mand training is likely limited to utilizing reinforcers that can be shown to the child through a screen-sharing feature (e.g., YouTubeTM videos, the ZoomTM whiteboard feature, music, online games/activities, etc.). Other reinforcers such as social activities and more traditional tangible reinforcers are extremely limited in that the technician cannot physically interact with the child to deliver social reinforcers (e.g., tickles, hugs, etc.) or allow the child to physically interact with tangible items (e.g., toys, edibles, etc.). Since many reinforcers delivered during telehealth mand training sessions are likely shown through a screen-sharing feature, the technician must be extremely fluent in navigating the telecommunication platform in order to ensure they are delivering reinforcers effectively and efficiently. Thus, the technician must be able to quickly share their screen, navigate from one activity to the next, and troubleshoot any technical issues in order to keep the child engaged. Due to the aforementioned unique qualities of mand training via telehealth, it is likely that even technicians who are fluent in providing mand training in person may not be able to effectively apply these skills via a telehealth platform. Thus, it is important to develop and test training methods that prepare technicians to administer the intervention. In addition, as was the case for some providers during the COVID-19 pandemic, staff training might require remote sessions to mitigate potential health risks for all involved. Therefore, evidence-based staff training procedures that could be utilized through telehealth are needed to teach staff to implement mand training procedures accurately and effectively in order to maintain the child’s 5 attention throughout a telehealth session. A procedure that lends itself well to this approach is behavioral skills training (BST). BST incorporates performance and competency-based strategies through demonstrations/models of the target skill and trainee practice with feedback until mastery criteria is met (Parsons, Rollyson, & Reid, 2012). The primary purpose of the present investigation was to assess the extent to which an online BST model followed by performance feedback can lead to a change in adult participant’s improved delivery of mand training. Improved delivery was measured through both the rate of fully correct mand training trials implemented and the percentage of accurate implementation of mand training steps by adult participants during direct ABA treatment sessions via telehealth. A secondary purpose was to assess the extent to which an online BST model to teach staff how to implement mand training via telehealth leads to a change in manding emitted by child participants with ASD. This change in manding was measured by the rate and percentage of correct independent mands emitted by child participants. 6 Participants and Setting Method The adult participants in this study were two female behavior technicians who had prior experience working with children on the autism spectrum. Sasha, was a 22-year-old Black female who had approximately two years of experience delivering ABA treatment in person and no experience delivering ABA therapy via telehealth. Ella, was a 23-year-old white female who had three years of experience delivering ABA treatment in person and one month of experience delivering direct ABA therapy via telehealth. However, Ella had never received training regarding ABA therapy via a telehealth platform. Ella was also currently in her first year of a master’s program in ABA. Both participants were recruited through email by a staff directory of an ABA clinic affiliated with a midwestern university. Both adult participants also had experience working with the child participants in person prior to clinic shutdowns caused by COVID-19. The child participants were two children diagnosed with ASD. Both child participants had prior experience with mand training in face-to-face sessions and were able to vocally mand. Avery, was a three-year-old white female who had never received ABA therapy via telehealth. Gia, was a four-year-old white female who had about one month of experience with ABA therapy via telehealth prior to the study, but had no experience with mand training via telehealth during that time. The child participants were recruited through an email that was sent to each child’s parents by the experimenter after they were recommended for the study by their BCBA®. The entire study was conducted online through the use of video conference meetings set up by the adult participants on ZoomTM. 7 Materials The materials necessary for this study consisted of a computer or tablet with a webcam that the adult participants, child participants, and experimenter could each use in their own home during sessions; the ZoomTM app; highspeed internet access; a variety of free online videos, games, or activities; a mand training checklist created by the experimenter (seen in Figure 1); and a series of four role play scripts created by the experimenter (seen in Figure 2). 8 Table 1. Accurate Implementation of Mand Training Checklist used to measure adult participant rate of fully correct trials and percentage of correct steps implemented. 9 Dependent Measures Adult Participant Dependent Variables Two dependent variables were measured for behavior technicians: rate of fully correct trials and percentage of correct steps implemented during mand training sessions. The rate of fully correct trials implemented was calculated using a two-step process. First, the experimenter added the total number of completely correct trials from the implementation checklist (Figure 1). Second, the experimenter divided the total correct trials by the total minutes rounded to the nearest tenth that it took for the behavior technician to administer the first ten trials. The percentage of steps the adult participant implemented correctly was calculated by dividing the sum of steps completed correctly during the first ten contrived mand trials by the total number of applicable steps during the observation period. This number was then multiplied by 100 to equal a percentage. Child Participant Dependent Variables Prior to baseline, the experimenter conducted a brief interview with the child’s BCBA® in order to define a target mand topography for each child participant. The experimenter asked the BCBA® how long of an utterance the child typically emits prior to taking part in the study (e.g., one-word, two-words, etc.). The length of utterance was used to inform expectations regarding a ‘correct’ mand for each child. However, if participating in this study was the child participant’s first time participating in a direct telehealth treatment session, a one-word mand was automatically selected as the target utterance length to try and avoid setting task demands that were too difficult for the child. Such an approach is similar to how mands are taught in face- to-face therapy by starting with one word then slowly building longer utterances as the child masters the skill. Thus, Avery’s target topography was one-word mands (e.g., “play”, “heart”, 10 “dinosaur”) since she had no previous experience with ABA therapy via telehealth. Whereas Gia’s target topography was defined as two-words (e.g., “press play”, “get Daniel”, “drive trolley”) based on the recommendation by her BCBA®. However, if the child emitted a mand that was more words than their target topography it was still coded as a correct response. Child participants’ rate of independent mands emitted per minute and percentage of independent correct mands were measured to evaluate the impact of the training for adults on child-level outcomes. For both measures, only mands contrived by the adult participant were counted in order to strictly focus on the skillset of the adult; thus, if the child emitted a spontaneous mand it was not scored as part of the total number of mands. The rate of independent contrived mands emitted by the child participant per minute was calculated by dividing the total number of trials in which the child emitted a correct independent mand after the establishing operation (EO) was contrived by the adult by the length of the experimenter’s observation in minutes to equal a rate. The percentage of independent correct mands was calculated by dividing the total number of trials in which the child emitted a correct independent mand after the EO was contrived by the adult participant by the total number of trials observed by the experimenter. Although child mands were dependent on adult behavior, rate of mands was a useful performance indicator for this particular study. Rate of child mands provides information regarding the utility of the brief mand training session in terms of children emitting more mands in a specific time frame and experiencing the putative reinforcers following those mands. Percent of correct mands was also calculated because it offers a measure of impact on children that was not confounded by potential changes to adult behavior (as were rate of mands per minute). 11 Interobserver Agreement Interobserver agreement (IOA) was taken on a trial-by-trial basis, with the experimenter serving as the primary observer and a secondary observer assessing video recordings of ZoomTM sessions. The second observer measured adult and child behavior for a minimum of 33.3% of sessions, evenly distributed across all participants and conditions. Agreement was defined as trials in which both the experimenter and the second observer coded the adult participant’s implementation of a step on the checklist or the child’s response the same. A disagreement was defined as any time in which the experimenter and second observer’s codes for the adult participant’s implementation or child’s response was not the same. IOA was taken on 33.3% of dyad one’s baseline sessions, 33.3% of post-training sessions, and 40% of maintenance sessions. Mean agreement for Sasha’s responding was 100%, 87% (range, 84-90%), and 83%% (range, 82-83%) for baseline, post-training, and maintenance respectively. Mean agreement for Avery’s responding was 100%, 75% (range, 70-80%), and 85% (range, 80-90%) in baseline, post-training, and maintenance respectively. For dyad two, IOA was calculated for 40% of sessions during baseline, 50% of post-training, and 33.3% of maintenance. The mean agreement for Ella’s responding was 87% (range, 80-94%), 93% (range, 90-96%), and 92% for baseline, post-training, and maintenance respectively. Mean agreement for Gia’s responding was 75% (range, 70-80%), 90%, and 90% in baseline, post-training, and maintenance respectively. Additionally, IOA was calculated regarding the observation length of each telehealth session which was used to calculate the rate measures for both adult and child participants. The purpose of this was to ensure that the denominator used to calculate rate was a reliable measure of session length in seconds. To calculate percentage of agreement for session length, the 12 experimenter and observer’s observation times were rounded to the nearest tenth of a minute then converted to seconds by multiplying each observation length by 60. Observation lengths in seconds for the experimenter and observer were then converted to a ratio by dividing the smaller observation length by the larger observation length. This number was then rounded to the nearest hundredth. Lastly, the average percentage of agreement for these ratios was calculated by dividing the sum of the ratios by the total number of sessions which was then multiplied by 100 to equal a percentage. The average agreement between the experimenter and observer’s coding for observation times was 91.5%. Experimental Design This study utilized a nonconcurrent multiple baseline design across participants in order to measure the effect of the online training protocol on the dependent measures of adults and children. The nonconcurrent multiple baseline across participants allowed for the start of intervention for each dyad to be staggered to reduce the chance of extraneous variables controlling behavior. Each dyad participated in a minimum of three baseline sessions in order to establish a pretraining measure of performance. Once a stable trend in the rate of correct mand trials administered by the adult participant was established for dyad one, the adult participant entered the training phase. Once the adult participant met mastery criteria in the role play during the training phase (85% accurate implementation for three consecutive sessions), they entered a post-training phase in which they delivered mand training via telehealth to the child participant they were paired with before training. During the post-training phase, the adult was given feedback by the experimenter after every session until they met mastery criteria on the accurate implementation checklist (85% accurate implementation for three consecutive sessions). Once 13 mastery criteria was met, the adult participant continued mand training sessions with the child participant without feedback until a stable percentage of steps implemented correctly by the adult participant was established. Percentage of correct responding criteria was used instead of rate because the field usually relies on such measures to determine ‘mastery’ and we had no a priori expectations that rate needed to reach a certain level. Following the application of the intervention to dyad one, dyad two entered the baseline phase, which was scheduled for five sessions or until rate of correct trials stabilized, at which time the adult participant entered the training phase until meeting mastery criteria during role plays. Once mastery criteria were met, the dyad entered the post-training phase during which feedback was provided until mastery criteria on the accurate implementation checklist was met. Procedures Baseline Within the baseline condition, the adult participants were paired with a child participant and were informed of the child’s target topography for manding determined by the assessment described above as well as at least three preferred items that could be used as reinforcers which were gathered by a brief interview with the child’s caregiver. The experimenter also gave the adult participant the accurate implementation of mand training checklist so that each adult participant would have an idea of what they would be assessed on. However, the experimenter did not provide any additional information. If the adult participant asked a question that was not seen on the checklist, the experimenter simply told them to try their best. The experimenter then asked the adult participants to conduct a maximum of 20 min of mand training with the child during a direct telehealth treatment session while the experimenter observed. While observing the session, the experimenter completed the accurate implementation of mand training checklist 14 in order to determine the current level of the adult participant’s implementation of mand training via telehealth prior to receiving training. Training During the training phase, the experimenter presented a PowerPoint® training on implementing mand training via telehealth. During the presentation, the experimenter vocally explained each step of the mand training procedure: contriving a motivating operation (MO), providing the appropriate prompt, how to engage with the child, when and how to conduct error correction, and data collection procedures. Throughout the training, the experimenter also utilized various video models of role-played telehealth mand training sessions. Once the PowerPoint® portion of the training was completed, the experimenter role played telehealth mand training sessions with the adult participant during which the experimenter served as the child and followed a series of four scripts that the experimenter created (see Figure 2 for an example), and the adult participant served as the behavior technician implementing the intervention. After each role play scenario, the experimenter provided the adult participant with vocal and written feedback based on the accurate implementation checklist. The experimenter and adult participant continued to role play sessions until the adult participant met mastery criteria of 85% accurate implementation, based on the accurate implementation of mand training checklist, for three consecutive sessions. Sasha met mastery criteria after five role-play sessions and Ella met mastery after three sessions. An additional role play session was conducted for Ella after a two-week break from sessions due to a holiday break at the child’s clinic, during which mastery criteria was maintained. 15 Table 2. Script one of four utilized in role play sessions during the training phase. Post-Training with Feedback Once the adult participant met mastery criteria for the role plays during the training phase, the dyad entered a post-training phase identical to the baseline phase; however, the experimenter provided feedback both vocally and written until the adult participant reached mastery criteria with the child participant (three consecutive sessions with 85% accurate implementation). This feedback was provided in order to ensure the adult participant implemented procedures correctly when transitioning from the role play scenario with the experimenter to sessions with their child participant pair. 16 Maintenance Once adults met mastery criteria during the post-training and feedback phase, they entered a maintenance phase during which no feedback was provided. The experimenter’s role during maintenance was identical to baseline across a minimum of three sessions for each participant. Procedural Integrity of Training Implementation Procedural integrity was collected by a second observer for both the BST PowerPoint® and role play training sessions through the use of a PI checklist developed by the experimenter. This measure was included in order to ensure that the experimenter implemented the training procedures and role play as outlined in the methods. The second observer recorded a “+” if the experimenter implemented the step correctly, a “–” if the step of the procedure was implemented incorrectly, or an N/A if the step did not apply. Data were collected on 100% of the PowerPoint® training sessions and 44.4% of role play sessions. Mean procedural integrity was 100% for both PowerPoint® and role play training sessions. 17 Adult Participants Results Adult participants rate of correct mand trials are depicted in Figure 3. During baseline, Sasha had an average rate of 0.0 correct mand training trials per min. However, after receiving training, Sasha’s responding increased to an average rate of 1.4 (range, 0.5-2.3) correct trials per min within the post-training phase, which included feedback following each session. During maintenance sessions, Sasha had an average rate of 1.3 (range, 0.9-1.8) correct trials per min. Similarly, during baseline, Ella demonstrated an average rate of 0.1 (range, 0-0.3) correct trials per min. Her responding increased to an average rate of 1.4 (range, 0.5-1.9) correct trials per min in the post-training with feedback phase. Subsequently, during the three maintenance sessions, Ella’s average rate of responding was 1.2 (range, 0.7-1.5) correct trials per min. 18 Figure 1. Adult participant’s rate of fully correct mand training trials. Ella’s graph is indented to represent the nonconcurrent design. Adult participants percentage of mand training steps implemented correctly are depicted in Figure 4. During baseline, both adult participant’s percentage of steps implemented correctly fell below the mastery criteria of 85%. Sasha had an average of 2.3% (range, 0-7%) steps 19 implemented correctly. However, during the post-training with feedback condition, Sasha met mastery criteria after six sessions and demonstrated an average of 85.3% (range, 74-94%) steps implemented correctly. In the maintenance phase, Sasha’s average percentage of steps implemented correctly was 81.4% (range, 76%-88%), which was slightly below the mastery criteria for the training phase (85%). Ella had an average of 61.6% (range, 44-76%) of steps implemented correctly in the baseline condition. During the post-training with feedback condition, Ella met mastery after four sessions and showed an average of 88.3% (range, 65-98%) steps implemented correctly. This high percentage of responding continued during the maintenance phase with an average of 89% (range, 82-94%) steps implemented correctly, falling above the mastery criteria for the training phase (85%). 20 Figure 2. Adult participant’s percentage of mand training steps implemented correctly and duration of sessions in minutes. Ella’s graph is indented to represent the nonconcurrent design. The circles represent the adult participants’ percentage of steps implemented correctly depicted on the left-hand y-axis. While the triangles represent the session duration (in minutes) for the first 10 contrived mand trials implemented depicted on the right-hand y-axis. 21 Child Participants The primary dependent variable for child participants was rate of correct independent mands per minute. During the baseline phase, Avery emitted an average rate of 0.0 correct independent mands per min. However, once moved to the post-training phase, Avery’s rate of independent mands per min increased to an average rate of 0.6 (range, 0.2-1.0). Avery’s rate of independent mands then continued to increase with an average rate of 1.5 (range, 0.7-2.1) in the maintenance phase. Similarly, in baseline Gia had an average rate of 0.5 (range, 0.2-0.8) independent mands per min. Which increased to an average rate of 1.0 (range, 0.7-1.2) independent mands per min in the post-training phase. Gia’s level of responding then increased slightly in the maintenance phase with an average rate of 1.2 (range, 1.0-1.5) independent mands per min. 22 Figure 3. Child participant’s rate of correct independent mands per minute. Gia’s graph is indented to represent the nonconcurrent design. The secondary dependent measure for child participants was percentage of correct independent mands. During baseline, Avery had an average of 0% for percentage of correct 23 independent mands. However, Avery emitted an average of 22% (range, 10-30%) correct independent mands in the post-training phase and an average of 52% (range, 30-70%) correct independent mands in the maintenance phase. Whereas Gia had an average of 50% (range, 10- 80%) correct independent mands in baseline. Then in the post-training phase, she emitted an average of 53% (range, 40-80%) correct independent mands, which then increased to an average of 73% (range, 60-80%) in the maintenance phase. 24 Figure 4. Child participant’s percentage of correct independent mands. Gia’s graph is indented to represent the nonconcurrent design. 25 Discussion After receiving the PowerPoint® training, both adult participants met mastery criteria quickly within the role play phase. During the post-training with feedback condition, both adult participants displayed increases in their average rate of correct trials and an improved percentage of correct responding. Levels of responding were also maintained when feedback was removed. Therefore, the online BST approach used in the present investigation may assist behavior technicians learning to accurately implement mand training via telehealth. Such an outcome could be beneficial to children with ASD as it increases the number of learning opportunities they have during sessions and teaches them to mand for preferred stimuli while interacting with an adult on a video platform. The results of the present investigation align with previous research demonstrating that BST leads to rapid acquisition of mand training procedures among service providers (Nigro- Bruzzi & Sturmey, 2010; Suberman & Cividini-Motta, 2020). Similar to results observed by Nigro-Bruzzi and Sturmey, staff in the current study met mastery criteria during the BST condition relatively quickly. However, the current study extends previous research as staff were trained via a telehealth model in which the trainer carried out all instruction and role play sessions within a ZoomTM environment. The results provide a preliminary pathway for ABA providers to prepare staff for telehealth-based services delivered directly to children with ASD. Child participants also demonstrated an increase in rate of mands over the course of the study. Avery and Gia showed an average increase of 1.5 and 0.7 mands per min, respectively, from baseline to maintenance phases. Child participants also exhibited gains in percentage of independent correct mands over the course of the study. Avery demonstrated an average increase of 52% from baseline to maintenance whereas Gia demonstrated an average increase of about 26 23%. Interestingly, Gia showed an increasing trend of accurate independent mands during baseline, though her rate of mands per min was below 1.0 across all sessions. While percentage of mand accuracy increased slightly during post training phases, there was a more obvious increase in rate of mands per minute. Although percent correct is commonly used as a measure of verbal operants in the behavior analytic literature, rate offered another perspective of the potential impact of changes in adult behavior on a child’s level of responding. The present investigation is the first examination we know of to teach mands to children with ASD via direct telehealth. The results build on prior research examining direct telehealth for teaching tacts (Ferguson et al., 2020) and total task chaining (Pellegrino & DiGenarro-Reed, 2020) to children with ASD. Mand training in particular may be an ideal skill to utilize when introducing a child to direct telehealth treatment due to the potential of facilitating pairing between the child and behavior technician as well as conditioning social interactions via telehealth as a positive experience (Michael & Sunderg, 2001; Plavnick & Ferreri, 2012). For example, Plavnick and Ferreri (2012) found that face-to-face mand training increased non- targeted behaviors including orienting towards a speaker and compliance with instructions and led to lower levels of interfering behaviors. Although the present study did not examine collateral effects of mand training via telehealth, it might be the case that teaching mands promotes other prosocial skills and increases a child’s responsiveness to learning new operants via telehealth. Given the potential barriers to delivering ABA to children with ASD via telehealth, it is important to consider pathways that could enhance the overall process, such as beginning with mand training. Future research examining this process is extremely important given that some children may not be able to obtain in person treatment for a variety of reasons, 27 such as limited treatment options in rural areas, inability to receive in home services due to small living space, etc. Although benefits of the staff training were demonstrated for both the adult and child participants, it is important to note that both adult and child participant’s prior experience with direct telehealth treatment may have impacted their success. Prior to the current investigation, both Ella and Gia had separate brief experiences with direct ABA treatment via telehealth. This prior experience could potentially explain Ella’s higher percentage of accurate implementation and Gia’s increasing percentage of correct independent mands in baseline. However, even though Ella showed higher levels of accurate implementation prior to receiving training, her rate of fully correct trials remained near zero in baseline. Yet, after receiving training, Ella’s rate of fully correct trials immediately increased to an average rate of 1.0 independent mands per min in the post training phase. Therefore, although she was implementing mand training at a relatively high accuracy (average of 61.6%) in baseline, mand training trials were being implemented as a low rate, thus creating few learning opportunities for the child participant. The present investigation expands upon current recommendations about telehealth delivery of ABA services in a variety of ways. First, although the Rodriguez (2020) telehealth model selection matrix provides ABA practitioners with guidance as to what telehealth model may be suitable for a child, there is little guidance in the literature as to how long each direct telehealth session should be. As Rodriguez (2020) mentions, direct ABA services typically range from about 20 to 40 hr per week. However, it is unclear if telehealth services should be implemented with the same intensity due to potential barriers such as the length of time a child will sit in front of a screen. Thus, it might be beneficial to begin with shorter telehealth sessions and slowly increase the session length over time. In this study, both Avery and Gia remained 28 engaged for the full 20-min sessions. Additionally, after the study was completed, both child participants’ parents reported that they believed the session length was an appropriate duration for their child. Thus, 20-min mand training sessions via telehealth may be appropriate for children around three to four years old who have an established repertoire of vocal mands. However, more research in this area is needed in order to inform practitioners what session duration to start with when beginning to utilize direct telehealth treatment, if session duration can be increased over time, and if so to what extent. Another element of the Rodriguez (2020) telehealth model selection matrix asks providers to consider that some children may be able to interact with a behavior technician without caregiver facilitation, while others may require caregivers to deliver prompts, reinforcers, or manage problem behavior. In the present investigation, Avery was able to attend to the adult participant with minimal redirections provided by her mother, who sat with her in front of the screen throughout each session. Whereas Gia engaged with the adult participant with no parental facilitation once she was connected to ZoomTM. As Rodriguez (2020) mentions, due to limited caregiver availability, a telehealth model with little to no caregiver facilitation would be ideal. Strategies that promote independent engagement in telehealth sessions would therefore be a fruitful area for additional research as it could increase access to services for children with ASD. Although the current study provides more guidance regarding methods for training behavior technicians to conduct direct ABA therapy on a telehealth platform, several limitations exist. The nonconcurrent multiple baseline design presents threats to internal validity that were not controlled for as dyad one began the study several months prior to dyad two. To control for time as a confounding variable, future researchers should consider utilizing a concurrent 29 experimental design. The research design also limited conclusions that can be drawn from these results in that there were only two dyads of participants in this study. Future replications should ensure to recruit at least three dyads of participants. An additional limitation of this study was that each child participant had one session in which IOA fell below 80%, though it was never less than 75% agreement. Disagreements for child responding was often the result of many factors regarding the difficulty of coding a recorded streaming event. For instance, technical difficulties due to poor internet connection led to a variety of complications in coding child responses such as lagging in video or sound. Gia’s video also often cut out in some of the ZoomTM recordings making it very difficult to decipher vocal mands based on the audio alone. Some disagreements were also due to differences in the interpretation of the child responses based on poor articulation. Moreover, the use of a parent interview to determine putative reinforcers may have limited the strength of child participants’ establishing operations; thus, potentially negatively impacting child responding. Future studies should consider investigating ways to conduct child preference assessments via a telehealth platform. Lastly, although child participants displayed gains in both rate of correct independent mands and percentage of correct independent mands over the course of the study, interpretation of these data are limited. Due to the study design, it is not possible to conclude that child participant gains were a direct result of adult participant’s increased level of responding after receiving training. The rate measure for child participant responding may have been limited and possibly confounded by adult performance. For instance, online games/activities which require the child to mand in quick succession (e.g., manding for colors to be added to an online coloring page) could have led to a higher rate of mand trials vs. a YouTubeTM video which often had a 30 longer inter-trial interval (e.g., 30 s to 1 min) between mand trials. Thus, child’s rate of mands could have depended on how quickly the adult participant contrived each trial based on the particular reinforcer, which limits what can be concluded when analyzing the child participants’ rate of responding. Similarly for percentage of correct independent mands, child participants were often exposed to novel reinforcers each session and therefore would require additional prompting for some reinforcers while remaining independent with others. This could have impacted the child participant’s responding since adult participants were trained to prompt child participants’ response the first time novel activities were used. Thus, the child participants could have had the ability to mand independently the first time each novel activity was used but were not given the opportunity to do so. Future research should consider replication with children as the main participants rather than behavior technicians by altering the experimental design to remove any influence the behavior technicians’ responding could have had on the child participants’ responding. For instance, the rate of contrived mand trials could be held constant across each phase in order to better measure the child participants’ skill acquisition. Despite these limitations, the current investigation added to the limited research regarding the use of direct telehealth ABA therapy sessions with young children with ASD. Behavior technicians were able to quickly learn how to use a telecommunication platform to implement brief mand training sessions with increased integrity and rate of implementation as a result of a BST approach to staff training. In addition to adult participant gains, potential benefits for child participants were also observed in that both child participants displayed increased rates and percentages of correct independent mands emitted over the course of the study. Thus, a BST approach to staff training may be appropriate for ABA practitioners considering utilizing direct telehealth treatment with clients. 31 REFERENCES 32 REFERENCES Albert, K. M., Carbone, V. J., Murray, D. D., Hagerty, M., & Sweeney-Kerwin, E. J. (2012). Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd Increasing the mand repertoire of children with autism through the use of an interrupted chain procedure. Behavior Analysis in Practice, 5(2), 65-76. doi:10.1007/BF03391825 ed.). Pearson Education Inc. standards. training by staff and unprompted vocal mands by children. Journal of Applied Behavior Analysis, 43(4), 757-761. https://doi.org/10.1901/jaba.2010.43-757 Evaluating discrete trial teaching with instructive feedback delivered in a dyad arrangement via telehealth. Journal of Applied Behavior Analysis, 53(4), 1876-1888. doi:10.1002/jaba.773 Council of Autism Service Providers (CASP). (2020). Organizational guidelines and Ferguson, J. L., Majeski, M. J., McEachin, J., Leaf, R., Cihon, J. H., & Leaf, J. B. (2020). Nigro‐Bruzzi, D., & Sturmey, P. (2010). The effects of behavioral skills training on mand Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: A guide for Pellegrino, A. J., & DiGennaro Reed, F. D. (2020). Using telehealth to teach valued skills to adults with intellectual and developmental disabilities. Journal of Applied Behavior Analysis, 53(3), 1276-1289. doi:10.1002/jaba.734 Plavnick, J. B., & Ferreri, S. J. (2012). Collateral effects of mand training for children with Pollard, J. S., LeBlanc, L. A., Griffin, C. A., & Baker, J. M. (2021). The effects of transition to Rodriguez, K. A. (2020). Maintaining treatment integrity in the face of crisis: A treatment Suberman, R., & Cividini‐Motta, C. (2020). Teaching caregivers to implement mand technician‐delivered telehealth ABA treatment during the COVID‐19 crisis: A preliminary analysis. Journal of Applied Behavior Analysis, 54(1), 87-102. doi:10.1002/jaba.803 selection model for transitioning direct ABA services to telehealth. Behavior Analysis in Practice, 13(2), 1-298. doi:10.1007/s40617-020-00429-8 training using speech generating devices. Journal of Applied Behavior Analysis, 53(2), 1097-1110. https://doi.org/10.1002/jaba.630 practitioners. Behavior Analysis in Practice, 5(2), 2-11. doi:10.1007/BF03391819 autism. Research in Autism Spectrum Disorders, 6(4), 1366-1376. doi:10.1016/j.rasd.2012.05.008 33 Sundberg, M. L., & Michael, J. (2001). The benefits of Skinner’s analysis of verbal behavior for children with autism. Behavior Modification, 25(5), 698-724. doi:10.1177/0145445501255003 34