BEHAVIORAL ARTISTRY AND APPLIED BEHAVIOR ANALYSIS: ASSESSING THE EFFECTS OF A STAFF TRAINING PACKAGE ON ‘PERCEPTIVE SENSITIVITY’ LEVELS IN BEHAVIOR ANALYTIC TREATMENT By Rebecca Saur A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Applied Behavior Analysis – Master of Arts 2023 ABSTRACT Currently, resources surrounding the effects of behavioral artistry within the field of Applied Behavior Analysis (ABA) are very limited. The aim of this study was to examine the extent to which a staff training package was able to increase demonstrations of ‘perceptive sensitivity’, a domain of behavioral artistry, among behavior technicians (BT). Furthermore, this study assessed the accuracy of participant observations when identifying behaviors related to client disengagement. This study examined levels of ‘perceptive sensitivity’ using a concurrent multiple probe across participants design, in which participant latency to identification of client disengagement behaviors were examined. Results showed that all participants improved demonstrations of ‘perceptive sensitivity’ as well as accuracy of identifying client disengagement behaviors. Key words: behavioral artistry, perceptive sensitivity, staff training TABLE OF CONTENTS INTRODUCTION ...........................................................................................................................1 METHOD ........................................................................................................................................5 Participants and Setting ...............................................................................................................5 Materials ......................................................................................................................................5 Experimental Design ...................................................................................................................7 Dependent Measures ...................................................................................................................8 Procedures ...................................................................................................................................9 Procedural Fidelity ....................................................................................................................15 RESULTS ......................................................................................................................................16 Latency to Identify Disengagement ..........................................................................................16 Accuracy of Observations .........................................................................................................17 DISCUSSION ................................................................................................................................20 REFERENCES ..............................................................................................................................25 APPENDIX A: PROCEDURAL FIDELITY CHECKLISTS .......................................................27 APPENDIX B: RESULTS OF PARTICIPANT ACCURACY OF OBSERVATIONS ...............29 iii INTRODUCTION Autism Spectrum Disorder (ASD) is a developmental disability that primarily impacts a person’s social, behavioral, and communication skills (CDC, 2020). Applied behavior analysis (ABA) is an evidence-based practice for children with ASD (National Autism Center, 2015) and has been recommended as the treatment of choice for children with an ASD diagnosis (Eldevik et al., 2009). ABA’s emphasis on improving socially significant behaviors is therefore ideal in helping individuals with ASD live more fulfilling and meaningful lives (Cooper et al., 2019; Axelrod et al., 2012). Despite its promise, important concerns remain within the field of ABA. Recent calls for compassionate care, person-centered care, and empathy-based treatment have begun a shift in service provision, emphasizing the need for strong interpersonal skills within an ABA practitioner’s repertoire. In their 2018 article, Taylor and colleagues discuss the importance of interpersonal skills as demonstrated by Board Certified Behavior Analysts  (BCBA) via a survey provided to 95 parents of children with ASD. The survey addressed specific traits that a BCBA might exhibit when optimally demonstrating empathetic and active-listening skills, as well as their correlation with positive treatment relationships. Results of this study revealed that many BCBA interpersonal skills fall short of consumer expectations. For example, while 84.21% of parents agreed that “the behavior analyst cares about my child”, only 65.26% of parents agree that “the behavior analyst acknowledges when treatment is not working”, highlighting a potential lack of self-reflection in behavior analysts. Behavioral artistry refers to a practitioner’s interpersonal skills including but not limited to care, attentiveness, creativity, and concern for the client (Foxx, 1985; Callahan et al., 2019). The construct of behavioral artistry was introduced by Foxx (1985) where the initial qualities of 1 a behavioral artist were outlined before later elaboration by Callahan and colleagues in their 2019 article. Despite the introduction of behavioral artistry to ABA nearly four decades ago, there is only one published empirical study examining the topic (i.e., Callahan et al., 2019). Behavioral artistry consists of seven traits, each addressing a specific element of interpersonal skills that, according to Callahan and colleagues (2019), describe a well-rounded practitioner. These traits include doesn’t like to fail, likes people, has a sense of humor, has “perceptive sensitivity”, looks “for the pony”, is thick-skinned, and is “self-actualized” (Foxx, 1985; Callahan et al., 2019). Researchers propose that practitioners fall on a continuum spanning from what they referred to as ‘behavioral technologists’ to ‘behavioral artists’. A practitioner who engages in behaviors closely aligned with many of the aforementioned traits would likely be classified as a behavioral artist, while a practitioner who does not, would be classified as a behavioral technologist. In addition to their classification and identification of behavioral artistry traits, Callahan and colleagues (2019) also aimed to evaluate the preference of parents of children with ASD for descriptors of behavioral artistry. In this study, 86 parents of children with ASD were provided an online survey where descriptions of interpersonal traits were specified. For each trait, both a description aligning closely with behavioral artistry descriptors and its opposite (i.e., non- descriptors of behavioral artistry) were provided, and participants were asked to indicate which traits were more desirable for an ABA practitioner to exhibit. Results showed that 86.6% of participants preferred descriptions associated with behavioral artistry, indicating that the traits of behavioral artistry are preferred by the individuals who are likely to receive ABA services. It is important that we move forward with development in these areas, as they may lead to more positive interactions with clients, parents, and families, and may facilitate positive relationships. 2 Despite current demand for practitioners who embody behavioral artistry traits in the treatment of individuals with ASD (Callahan et al., 2019; Taylor et al., 2018), there is only one known experimental study that has examined teaching such skills to practitioners (Notarianni, 2022). In their study, Notarianni (2022) aimed to assess the specific trait ‘likes people’. In this study, adult participants were scored by instances of positivity toward their child counterpart. The purpose of this study was twofold: to examine the effects of behavioral skills training (BST) and video self-modeling (VSM) on the ‘likes people’ trait throughout in-vivo sessions of adult- child dyads, and to investigate the effects on child behavior post-behavioral artistry instruction. Throughout Notarianni’s (2022) study, adult participants were paired with a child diagnosed with ASD that was actively receiving intensive ABA services. Sessions were conducted within the treatment rooms where ABA services were provided. To assess the ‘likes people’ trait, adult participants were scored based on instances of positivity emitted toward the child, measured by occurrences of affectionate touch, vocal affirmations, or positive body language. Instances of positivity were recorded using a 5 s partial interval measure over 5 min sessions, meaning that the adult participant needed only to engage in one of the previously mentioned instances of positivity per interval to be considered to have engaged in positivity towards the child. Child behaviors were measured using three observable behaviors, including: remaining close to the adult participant, negative vocalizations, and responses to social initiations. Results revealed that all four adult participants demonstrated an increase in their engagement in behaviors that reflected the ‘likes people’ trait following their exposure to an intervention consisting of behavioral skills training and video self-modeling. Child participants did not demonstrate changes in behavior following the intervention for adult participants. The 3 author notes that this could be a result of already high behaviors noted in pre-training (e.g., a score of 98% in pre-training increasing to 100% post-intervention), or a delay to behavior change after training. Although the magnitude and maintenance of behavior change varied across adult participants, the results are promising as they suggest that traits of behavioral artistry are amenable to change. The present study aims to build on the relatively nascent prior literature on behavioral artistry (Callahan et al., 2022; Notarianni, 2019) by evaluating a second trait of behavioral artistry: ‘has perceptive sensitivity’. ‘Perceptive sensitivity’ is described as a practitioner’s ability to attend to behaviors that are small, subtle, and gradual in nature, and may be early indicators of client distress (Foxx, 1985; Callahan et al., 2019). Perceptive sensitivity is therefore imperative in an individualized approach to treatment, as enhanced levels of perceptive sensitivity may allow for more rapid identification of client precursor behaviors on behalf of behavior technicians (BT), thereby avoiding more significant behaviors that impede learning. The purpose of the present study is to examine the trait of perceptive sensitivity and assess the extent to which such a repertoire can be improved among behavior technicians. The study specifically examines the effects of a staff training package focused on perceptive sensitivity on participants’ latency to accurately identify instances of client disengagement behaviors. A successful study in this area will provide important context that may help to inform staff training procedures, as well as contribute to the existing literature focused on the construct of behavioral artistry. 4 METHOD Participants and Setting Participants in this study included three adult participants, Margaret, Nathan, and Ella. All participants were employed as BTs at an early intensive behavioral intervention (EIBI) treatment center affiliated with a Midwestern university. Participants were selected for this study based on recommendations from their supervising BCBA, as well as anecdotal observations of perceptive sensitivity levels by the researcher. Once participants were identified, their voluntary participation in this study was requested. Inclusion criteria included: currently employed as a BT at aforementioned EIBI treatment center and not currently enrolled in any higher education coursework in a related field. Sessions took place twice daily throughout the course of a typical treatment day (between 8:30 AM and 4:00 PM) and occurred in close proximity to the clinic. Sessions were conducted in a variety of settings including hallway vestibules, empty classrooms, office spaces, and empty treatment rooms. Locations were selected based on availability to ensure minimal interruptions to the session. Materials Materials included thirteen PowerPoint slideshows, each including three 1 min video demonstrations of client disengagement behaviors. These PowerPoint presentations were titled Session 001 – 013 with one presentation being presented to participants per session. All videos were edited to be the same length and were selected to depict similar indicators of disengagement behaviors so that identification would be of comparable complexity across sessions. Additionally, all videos were edited to be 1 min in length wherein the disengagement behavior began at 10 s. 5 All videos featured one client; a three-year-old male with an ASD diagnosis, currently receiving thirty hours of intensive 1:1 ABA therapy per week at the aforementioned ABA treatment center. This client was selected for his participation in video demonstrations of disengagement behaviors due to his ability to be coached to exhibit the necessary behaviors (seen in Table 1) at specified times. Featured alongside the client in every video was an instructor, played either by the researcher or a trained research assistant. All videos were set at the client’s instructional table and were designed to look like a typical session of discrete trial training. Videos were captured with a video camera and tripod. The client was taught to discriminate between two cards, each indicating a “serious” or “silly” response. When the “serious” card was presented by the client’s BT behind the researcher and out of view of the camera, the client was instructed to complete the task with a “calm body, quiet voice, and facing the researcher” (that is, the antithesis of disengagement for the purposes of this study). After ten seconds of appropriate responding by the client, the BT would change the card to “silly”. In the presence of the “silly” card, the client was instructed to engage in any disengagement behaviors while following through with task demands placed by the instructor. Performing task demands decreased the likelihood that compliance would be conflated with engagement by adult participants. Videos were collected in approximately 15 min segments and were uploaded to OneDrive to be edited and formatted into session presentations. See Figure 1 for a detailed depiction of the distribution of client videos throughout this study. A forty-four slide PowerPoint presentation was created for its use as a staff training package in the training phase of this study. This presentation consisted of five main sections: the instructional lesson, three discrimination training exercises, and an opportunity for feedback and questions. The videos featured in this presentation were identical in length and arrangement to 6 the ones of the session videos, with the exception of their use with multiple clients. That is, the staff training presentation featured video demonstrations of four clients. More information on this staff training package can be found under Procedures and in Figures 1 and 2. All sessions were recorded via Zoom, wherein both the participant and the viewing screen is visible. Sessions were then uploaded to a cloud drive, where they were accessed for data collection purposes over the course of the data collection period. Client video examples and non-examples Pre-training and Post- Training training Pre-training Post- training Discrimination training 3 novel 3 novel videos per videos per session session Section1 Section 2 Section 3 Section 4 2 novel video 2 novel video 2 novel video 3 novel video examples, 2 examples, 2 examples, 2 examples, 2 novel video novel video novel video novel video non- non- non- non- examples examples examples examples Figure 1 Distribution and Use of Video Examples Experimental Design This study was conducted using a concurrent multiple probe across participants design (Ledford & Gast, 2018). The multiple probe design was advantageous as it limited participants’ exposure to videos which helped to increase researcher efficiency by limiting quantity of necessary videos. Additionally, traits of behavioral artistry are believed to be non-reversible, and therefore a multiple probe design is ideal. All participants completed five pre-training sessions before the first participant moved into training, while all other participants remained in the pre-training phase. Post-training 7 sessions were conducted twice daily, approximately 3 to 4 hours apart, with probe sessions occurring every three to five sessions for participants in the pre-training phase. Once observable behavior change occurred for the first participant, the second participant was then moved into training. This same process was applied for all participants included in this study. Dependent Measures Latency to identify disengagement was the primary dependent measure for this study. Disengagement behaviors were defined as any instance in which the client engages in one or more of the following behaviors: altered body orientation, vocal stereotypy, and fidgeting. Table 1 provides specific definitions of each behavior, as well as examples and non-examples. Specifically, researchers measured the length of time between the onset of the first instance of disengagement and the time at which the participant identified that disengagement had occurred. Additionally, researchers evaluated participants’ accuracy of observations of disengagement. Following each indication of client disengagement by participants, researchers asked participants to explain why they identified disengagement. All responses were recorded and coded as correct, incorrect, or timed-out, based on the alignment between the participants’ reason for noting disengagement and the primary researcher’s coded instance of disengagement. For example, a participant response highlighting client non-compliance would result in an incorrect response, as compliance is not included as a behavioral indicator of disengagement. An incorrect response would result in a latency of 50 seconds, or the maximum latency for one video. An instance in which the participant did not choose to stop the video at any point resulted in a latency of 50 seconds. 8 Behavior Definition Example Non-example Altered body Any change in body Client turns Client coughs orientation orientation away from sideways in chair during session, and social partner for when presented with turns their head longer than 2 seconds a non-preferred task sideways to cough into their arm Vocal Any instance in which Client hums loudly Client exclaims, stereotypy the client engages in a when instructional “knees and toes!” as vocal behavior that is materials are a result of the social above a conversational presented partner singing, volume or lasts longer “head, shoulders…” than 2 seconds and/or is unrelated to instructional task Fidgeting Any repetitive gross Client begins to Client pumps fist in behaviors motor behavior that shake legs and flail excitement or lasts longer than 3 arms simultaneously celebration seconds, and/or is during teaching unrelated to session instructional task Table 1 Disengagement Behaviors: Definitions, Examples, and Non-examples Procedures Probe Sessions Prior to beginning each session, researchers identified a suitable space to conduct the session. These areas included: hallway vestibules, empty classrooms, office spaces, and empty treatment rooms. Importantly, areas were chosen to be free of other individuals so that interruptions during sessions were minimal. Participants were then invited to follow the researcher into the determined space to begin the session one at a time. Once in the appropriate location, the researcher presented the participant with a PowerPoint slideshow featuring three 1 min long videos of client behaviors. Importantly, each video featured one or more of the disengagement behaviors listed in Table 1 to demonstrate disengagement behaviors beginning at 10 s. The remaining 50 s routinely 9 featured gradual escalation in client behaviors such that they became more overt as the video progressed. This was done to evaluate participants’ ability to attend to early indicators of client behaviors that may be subtle, consistent with previous descriptions of perceptive sensitivity (Callahan et al., 2019). Before beginning the session, the researcher began recording via Zoom, using the “screen share” and “record” features. This allowed both the screen and participant responses to be recorded and uploaded for later use in data collection. Once recording, the researcher greeted the participant, and provided a brief introduction to the task consisting of the following statement: “I am going to show you a series of three videos. I would like you to stop me if at any point you notice that the client has become disengaged. If you choose to stop me, I will pause the video and ask why you chose to identify disengagement.” This introduction was consistent across all sessions. The first video was played until the participant signaled to stop the video. Upon participant’s signal to stop the video (i.e., “now”, “stop”, “stop the video”, or something similar), the researcher paused the video and said, “Okay, can you tell me why?”. The participant then shared their observation of client disengagement behaviors. The researcher provided a neutral response and moved on to the next video. The same was repeated for all three videos. After all three videos were shown, the researcher thanked the participant for their participation and told them that the session was complete. No reinforcement or feedback was provided. All recorded sessions were then uploaded to a cloud drive to be coded by researchers. Prior to the subsequent session, researchers recorded the latency to identify disengagement and accuracy of participant observations of disengagement for each video clip. The process described above was the same 10 for sessions occurring prior to perceptive sensitivity training (described below) and sessions following the training. Perceptive Sensitivity Training The training for the present study consisted of an instructional lesson, four separate discrimination training exercises, and an opportunity for final feedback and questions from the participant. See Figure 2 for a visual account of the staff training package components. The staff training package began with a 15 min instructional lesson which provided a brief overview of behavioral artistry, focused specifically on perceptive sensitivity. Perceptive sensitivity was defined per the Callahan et al. (2019) definition. The connection between perceptive sensitivity and disengagement was then made, detailing the importance of a practitioner’s ability to attend to client disengagement behaviors. Next, the researcher described the relevance of perceptive sensitivity to disengagement behaviors, and definitions, examples, and non-examples were shared. Finally, the researcher discussed the value of a practitioner’s ability to distinguish between client engagement behaviors versus mere compliance. In the first section of discrimination training, four videos of client behaviors were provided; two of these videos were examples of disengagement behaviors, and two videos were non-examples. The same was true for the first three sections of the training package. In this section, the researcher let each 1 min long video play in its entirety before classifying the video as an example or non-example. Throughout each video, the researcher narrated their thoughts (e.g., “I’m noticing that the client is beginning to engage in a gross motor behavior that is lasting longer than 2 s”) while the participant was instructed to watch attentively and interject if desired. In the next section of training, the researcher paused the video at every instance of disengagement and asked leading questions such as, “did you notice how the client turned their 11 body away?”. The participant’s attention was drawn to the specified example, and they were invited to share their observations. The researcher’s narration of disengagement allowed the participant to develop strategies for identifying client disengagement. Participants were provided social praise following correct responses and/or observations. During the third section of training, participants were instructed to attempt the exercise with support and feedback from the researcher. Feedback from the researcher was provided at the end of each video or as requested from participant. For example, if a participant asked, “would you pause it there because the client was shaking their legs?” the researcher was free to answer their question and provide an explanation. After the participant engaged with each video the researcher provided immediate feedback in the form of either correction or praise. If the participant incorrectly classified a video, an error correction procedure in which the researcher identified the error, explained the error, and then identified the correct response was used. The last section of the training was used as an ‘exit ticket’ for the training package. In this exercise, five videos featuring two non-examples and three examples of disengagement were shown and participants were instructed to sort all videos independ ently. The researcher played the video and allowed the video to play all the way through before asking the participant if they believed it was an example or non-example of disengagement. The researcher then recorded their answer before moving on to the next video. Participants were required to sort with a minimum of 80% accuracy prior to moving into the final feedback portion of the training. In the event that participants sorted with below 80% accuracy, they would return to the third section of the training and continue this way until 80% accuracy was met (seen in Figure 2). All participants achieved a minimum of 80% accuracy in their first attempt. Both Margaret and Ella received scores of 100% in their first attempt and Nathan received a score of 80%. 12 The final portion of the training included final feedback from the researcher as well as any questions from participants. In this section, participant scores from the final training section were shared, and any specific relevant feedback was provided (i.e., “you sorted four of five videos correctly, allow me to show you the one you missed.”) as well as social praise. Social praise was used as a reinforcer throughout training. Any questions from the participants were answered in this portion, including requests to view additional footage. All three participants requested additional exposure to videos to ensure understanding. In this case, the researcher returned to the first and second sections of the training and allowed participants to rewatch videos as requested. Yes Figure 2 Visual Representation of Staff Training Package Procedure 13 Maintenance While previously designed to include a maintenance component, maintenance data was only collected with one participant, Margaret. Maintenance sessions were cond ucted identically to probe sessions. Due to extenuating circumstances in which access to participants became limited, the maintenance phase of this study was eliminated. Interobserver Agreement Interobserver agreement (IOA) was collected for a minimum of 30% of all sessions for each participant. Sessions selected for IOA were chosen randomly to reduce any potential bias in data collection. IOA was collected by a research assistant who was trained on data collection for the purposes of this study. This training consisted of a PowerPoint slideshow which featured definitions, examples, and non-examples of disengagement behaviors, as well as video examples of participant sessions. Each dependent variable was defined in detail and data collection procedures were described for each. Prior to coding independently, the secondary observer was trained to 100% reliability on training videos provided by the researcher. IOA data for participant latency to identify disengagement was calculated by dividing the shorter latency by the longer latency and multiplying by 100. IOA data for participant accuracy of observations was calculated by dividing the total number of agreements by the sum of agreements and disagreements and multiplying by 100 (Ledford & Gast, 2018). Mean IOA for all participants’ latency to identify disengagement was 88.1% (range, 83%-91.3%). Mean agreement for Margaret’s latency to identify disengagement was 90%, and 83% for participant accuracy of observations. Mean agreement for Nathan’s latency to identify disengagement and accuracy of observation was 83% respectively. Finally, mean agreement for 14 Ella’s latency to identify disengagement was 91.3%, and 93% for Ella’s accuracy of observations. Procedural Fidelity Procedural fidelity data were collected on the extent to which the researcher accurately administered probe sessions using a checklist that was designed by the researcher. Fidelity data were collected across 33% of all probe sessions. This checklist can be found in Figure 4 in the Appendix. The mean procedural fidelity of probe sessions was 100%. Procedural fidelity of the researcher’s implementation of the training session as described was also collected. Data were collected on 33% of all training sessions. These data were collected using a checklist to evaluate the researcher’s accuracy of implementation of training. This checklist can be found in Figure 5 in the Appendix. Mean procedural fidelity of the intervention was 100%. 15 RESULTS Latency to Identify Disengagement Below, results for all three participants’ latency to identify disengagement are listed and described accordingly. See Figure 3 for a graphical depiction of participant responding. Margaret Margaret was the first participant to enter the training period. Her latency to identify disengagement across five pre-training sessions was 41 s (range, 25.33 s to 50 s). Training was implemented following pre-training sessions. Across eight post-training sessions, Margaret’s latency to identify disengagement reduced to 9.92 s (range, 1 s to 22.67 s). In her first session post-training, Margaret’s data exhibited a drop from a latency of 50 s in her fifth pre-training to a latency of 5 s. Her latency to identify disengagement increased slightly in the next session to 9.33 s but dropped down to her lowest latency of 1 s. In subsequent sessions her levels continued to remain low before an increase in the fifth session to a latency of 22.67 s. In maintenance sessions, a similar pattern of variability was observed, with data ranging from 2.67 s to 18.67 s. Nathan Nathan was the second participant to enter training. He completed five consecutive pre- training sessions, as well as a pre-training probe prior to entering training. Across six pre-training sessions, Nathan’s latency to identify disengagement was 32.33 s (range, 21.33 s to 50 s). Nathan’s latency to identify disengagement reduced to 4.19 s (range, 2.33 s to 7.3 s) across five post-training sessions. Nathan’s latency to identify disengagement remained consistently low across post-training sessions. Nathan’s pre-training scores were highly variable, ranging from 21.33 s to 50 s. However, in post-training, Nathan’s scores ranged of 2.33 s to 7 s, a much more consistent trend. 16 Ella Ella was the third and final participant to enter training. She entered training following five consecutive pre-training sessions and three pre-training probes. Across all eight pre-training sessions, Ella demonstrated an average latency to identify disengagement of 39.92 s (range, 28.33 s to 50 s). In six post-training sessions, Ella demonstrated an average of 11.61 s (range, 1.67 s to 38 s). The results of Ella’s first post-training session was identical to her most recent probe prior to entering training and then steeply dropped to lower levels in the subsequent five sessions. Accuracy of Observations Below, results for all three participants’ accuracy of observations of disengagement are listed and described accordingly. See Tables 2-4 for a visual representation of participant responding. Margaret Across all pre-training sessions, Margaret correctly observed and described client disengagement in six of fifteen opportunities or 40%, the highest of all three participants pre- training. Margaret received a TO score in eight of fifteen pre-training opportunities and an incorrect score in one of fifteen pre-training opportunities. Across eight post-training sessions, Margaret’s accuracy of observations increased to 100% accuracy in 24 opportunities. Nathan Nathan demonstrated 44% accuracy of observations across six pre-training sessions, or eight of eighteen opportunities. Incorrect responding accounted for seven of eighteen opportunities, or 39%. Nathan received three timed-out scores across all pre-training sessions. Nathan’s accuracy of observations increased to 100% post-training, correctly responding in all 17 fifteen opportunities. Additionally, Nathan’s accuracy of observations across all post-training sessions improved to 100% from 44% in pre-training. These data suggest that Nathan’s ability to identify disengagement behaviors both quickly and correctly improved following the intervention. Ella Ella demonstrated the lowest levels of accuracy of observations across eight pre-training sessions, correctly responding in seven of twenty-four opportunities, or 29%. In eleven of twenty-four opportunities, Ella received a timed-out score, or 46%. Incorrect responding accounted for six of twenty-four opportunities in pre-training, or 25%. In six post-training sessions, Ella’s accuracy of observations increased to 89%, or sixteen of eighteen opportunities. Ella incorrectly responded in two instances of post-training sessions with both errors occurring in the first two opportunities of post-training sessions. 18 Figure 3 Results of Participant Latency to Identify Disengagement Behaviors 19 DISCUSSION The aim of the present study was to evaluate whether levels of perceptive sensitivity can be defined, measured, and changed via a staff training package focused specifically on identification of client disengagement behaviors. Results in the post-training phase indicated that all participants demonstrated noticeable decreases in latency from pre-training, as well as improvement in their accuracy of observations. It is encouraging that all participants exhibited behavior change, suggesting that a staff-training package focused on behavioral artistry was effective in improving levels of perceptive sensitivity, an important domain of behavioral artistry. While Nathan’s latency to identify disengagement remained consistently low across all post-training sessions, both Ella and Margaret’s data revealed variable responding. Margaret’s variability in responding could be due, in part, to session timing. Because sessions were conducted twice daily, the first session was conducted at the beginning of the participant’s scheduled BT shift, while the second session occurred most often at the end of the treatment day. It is possible that participants were more fatigued at the end of their scheduled shift which may have impacted their attending. Margaret’s data supports this notion, as latency to observe disengagement during her second daily session was consistently higher than her first. This remained true across all sessions, including pre-training and post-training. Interestingly, in her eighth session where Margaret demonstrated her lowest latency to identify disengagement, only one session was conducted to maintain alignment of probes with the introduction of intervention for another participant. Despite some variability in Margaret’s latency to identify disengagement, her accuracy of observations remained at 100% across all post-training sessions. These results point to enhanced 20 precision in Margaret’s skills as an observer so that, while latency to identify disengagement may vary, Margaret is able to correctly identify when the client became disengaged. This growth in precision may have the most benefit in the practical setting where Margaret works as a BT. Greater ability to correctly and efficiently recognize instances in which her own client has become disengaged may encourage an overall more positive treatment experience for both the client and BT. That is, if Margaret is able to identify instances of client disengagement in the moment, she may be able to use that information as feedback in order to adjust her own behavior and regain client engagement. While outside the scope of the present study, that secondary component may be a compelling direction for future research. Outside of his sessions, Nathan shared with the researcher that he felt that this training improved his skills in the practical setting. He reported an improved ability to identify disengagement with clients in their daily sessions. Interestingly, he mentioned this improvement as it related to his skills in the applied setting, stating that he felt he was able to adapt to changes in client behaviors more effectively following this study. Although anecdotal, Nathan’s reporting potentially suggests BTs could experience improved interpersonal skills with clients in a clinical setting following the training described in the study. In her first post-training session, Ella’s latency to identify disengagement was 38 s, which was identical to her final pre-training probe session. However, in the very next session, her latency to identify disengagement dropped to 9 s. Across all post-BST sessions, Ella’s latency to identify disengagement was 11.61 s, the highest of all participants with the greatest range. The range was due to the high score observed during the first post-training session. Prior to beginning the first post-training session, Ella reported some anxiety about performing accurately to the researcher, which could explain the higher latency for that session. Interestingly, both errors that 21 occurred within the first post-training session were incorrect observations made by premature identifications (i.e., identifying a behavior before it occurred). Premature responding may support the idea that nerves impacted performance in the first session, as it suggests an over- eagerness to stop the video in order to avoid an extended delay to identify disengagement behaviors. The results of this study suggest that the staff-training package is an effective approach for teaching staff to identify disengagement, which might suggest improvement in ‘perceptive sensitivity’, one of the main behavioral artistry traits. The current study contributes to the existing literature, as it supports Notarianni’s (2022) suggestion that traits of behavioral artistry can be taught and changed over time. This is a key component of the present study, as it encourages a further line of research questions aimed at improving other domains of behavioral artistry. While results suggest that the staff-training package was an effective approach to teaching behavioral artistry, one limitation of the present study is its use of participant latency to identify client disengagement as a means to evaluate levels of perceptive sensitivity. While researchers feel that latency to identify disengagement served as an effective proxy for perceptive sensitivity, aligning with the traits defined and described by Callahan et al. (2019), it is possible that better evaluations could exist due to the abstract nature of behavioral artistry. Additionally, the present study did not account for generalization or transfer-of-training. As previously stated, all video examples used in pre-training, post-training, and maintenance sessions were of one client who was trained to demonstrate a range of disengagement behaviors. This could, however, present a limitation to the present study, as there is no data to suggest that 22 this training would be as beneficial across a range of clients or translate to in-vivo presentations of behavior as opposed to video examples. An additional limitation of this study is its lack of a formal maintenance component to assess skill development over time. While initially designed as a component of this study, the maintenance phase was excluded due to extenuating circumstances which limited researchers’ ability to access participants routinely. Maintenance data would have provided valuable information regarding the lasting effects of this intervention such that a BTs progress in perceptive sensitivity could be tracked and measured. The present study isolated one domain of behavioral artistry, meaning that the existing literature has therefore only touched on two of the seven domains of behavioral artistry, “likes people” and perceptive sensitivity. There are many opportunities for future behavioral artistry research, including the remaining five traits of behavioral artistry that have yet to be explored. Expansion in behavioral artistry could contribute valuable information to the field of ABA in areas of staff and educator training as well as potential screening possibilities for incoming staff as noted by Notarianni (2022). One potential future study based on the results of the present investigation could be an evaluation of the extent to which a participant changes their behavior when interacting with a client after identifying client disengagement. Although the present study is encouraging as it shows the trait of perceptive sensitivity can be taught and changed, it is important to assess the effects of these improvements in the applied setting. The ability to engage in a sequence of behaviors to regain client engagement would likely improve treatment experiences and is therefore valuable to the field. 23 Despite the nearly four decades that have passed since Foxx (1985) initially introduced the topic of behavioral artistry, the topic has gone largely unexplored. Since its introduction in 1985, the present study is only the second known experimental study to exist on the topic. Given the proposed link between behavioral artistry and quality of care (Foxx, 1985; Callahan et al., 2019) there are many compelling opportunities for future growth in the area of behavioral artistry and ABA. Future research is therefore essential in establishing other ways in which these traits of behavioral artistry can be improved in hopes of training staff equipped to implement the highest possible quality of care. 24 REFERENCES Axelrod, S., McElrath, K. K., & Wine, B. (2012). Applied behavior analysis: Autism and beyond. Behavioral Interventions, 27(1), 1-15. Callahan, K., Foxx, R. M., Swierczynski, A., Aerts, X., Mehta, S., McComb, M.-E., Nichols, S. M., Segal, G., Donald, A., & Sharma, R. (2019). Behavioral artistry: examining the relationship between the interpersonal skills and effective practice repertoires of applied behavior analysis practitioners. Journal of Autism and Developmental Disorders, 49, 3557–3570. https://doi.org/10.1007/s10803-019-04082-1 Cooper, J. O., Heron, T. E., & Heward, W. L. (2019). Applied Behavior Analysis (3rd Edition). Hoboken, NJ: Pearson Education. Eikeseth, S. (2009). Outcome of comprehensive psycho-educational interventions for young children with autism. Research in developmental disabilities, 30(1), 158-178. Eldevik, S., Hastings, R. P., Hughes, J. C., Jahr, E., Eikeseth, S., & Cross, S. (2009). Meta- analysis of early intensive behavioral intervention for children with autism. Journal of Clinical Child & Adolescent Psychology, 38(3), 439-450. Foxx, R. M. (2008). Applied behavior analysis treatment of autism: The state of the art. Child and adolescent psychiatric clinics of North America, 17(4), 821-834. Freedman, D. H. (2016). Improving public perception of behavior analysis. The Behavior Analyst, 39(1), 89-95. Gorycki, K. A., Ruppel, P. R., & Zane, T. (2020). Is long-term ABA therapy abusive: A response to Sandoval-Norton and Shkedy. Cogent Psychology, 7(1), 1823615. Leaf, J.B., Leaf, R., McEachin, J. et al. Applied Behavior Analysis is a Science and, Therefore, Progressive. J Autism Dev Disord 46, 720–731 (2016). https://doi.org/10.1007/s10803- 015-2591-6 LeBlanc, L. A., Taylor, B. A., & Marchese, N. V. (2020). The training experiences of behavior analysts: Compassionate care and therapeutic relationships with caregivers. Behavior Analysis in Practice, 13(2), 387-393. Ledford, J. R. & Gast, D. L. (2018). Single case research methodology: Applications in special education and behavioral sciences. New York: Routledge, Taylor & Francis Group. Notarianni, E. (2022). Assessing 'Behavioral Artistry' Staff Training on Behavior Technicians' Use of Behavioral Artistry Traits and the Behavior of Young Children With Autism Spectrum Disorder [Master's thesis, Michigan State University]. Proquest. Peters-Scheffer, N., Didden, R., Korzilius, H., & Sturmey, P. (2011). A meta-analytic study on 25 the effectiveness of comprehensive ABA-based early intervention programs for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 5(1), 60-69. Rohrer, J. L., Marshall, K. B., Suzio, C., & Weiss, M. J. (2021). Soft Skills: The Case for Compassionate Approaches or How Behavior Analysis Keeps Finding Its Heart. Behavior Analysis in Practice, 14(4), 1135-1143. Shillingsburg, M. A., Hansen, B., & Wright, M. (2019). Rapport building and instructional fading prior to discrete trial instruction: Moving from child -led play to intensive teaching. Behavior Modification, 43(2), 288-306. Slocum, T. A., Detrich, R., Wilczynski, S. M., Spencer, T. D., Lewis, T., & Wolfe, K. (2014). The Evidence-Based Practice of Applied Behavior Analysis. The Behavior analyst, 37(1), 41–56. https://doi.org/10.1007/s40614-014-0005-2 Taylor, B. A., LeBlanc, L. A., & Nosik, M. R. (2019). Compassionate care in behavior analytic treatment: Can outcomes be enhanced by attending to relationships with caregivers?. Behavior Analysis in Practice, 12(3), 654-666. 26 APPENDIX A: PROCEDURAL FIDELITY CHECKLISTS Probe Session Treatment Fidelity Checklist Session: Observer: For each step of intervention, record “+” if trainer completes the step and a “-“ if the step is not completed. If a step is not applicable, record “N/A” Step in intervention Data 1. Begins recording session. 2. Explains objective of session by saying, “I’m going to show you a series of 3 videos. I’d like you to stop me if at any point you notice the client has become disengaged. If you choose to stop me, I’ll pause the video and ask why you chose to identify disengagement.” or something similar. 3. Plays video. 4. Pauses video upon participant signal. 5. Asks participant why they chose to identify disengagement. 6. Provides neutral feedback, like “thank you” or “okay”. 7. Plays next video. 8. Pauses video upon participant signal. 9. Asks participant why they chose to identify disengagement. 10. Provides neutral feedback, like “thank you” or “okay”. 11. Plays last video. 12. Pauses video upon participant signal. 13. Asks participant why they chose to identify disengagement. 14. Provides neutral feedback, like “thank you” or “okay”. 15. Thanks participant for their participation. 16. Ends recording. Percentage of procedural fidelity (total number of correct steps/ total number of steps x100) Figure 4 Procedural Fidelity Checklist for Probe Sessions 27 Perceptive Sensitivity Intervention Treatment Fidelity Checklist Date: Observer: For each step of intervention, record “+” if researcher completes the step and a “-“ if the step is not completed. If a step is not applicable, record “N/A” Step in intervention Data 1. Introduces topic of behavioral artistry. 2. Provides an overview of presentation. 3. Defines behavioral artistry. 4. Lists behavioral artistry traits. 5. Defines “perceptive sensitivity” trait. 6. Introduces disengagement. 7. Explains importance of disengagement. 8. Defines disengagement 9. Explains objective of DT: section 1. 10. Plays video and provides commentary. 11. Repeats step 10 for all videos in DT: section 1. 12. Explains objective of DT: section 2A. 13. Plays video. 14. Provides support and feedback as appropriate. 15. Repeats steps 13-14 for all videos in DT: section 2A. 16. Explains objective of DT: section 2B. 17. Plays video. 18. Provides support and feedback as appropriate. 19. Repeats steps 17-18 for all videos in DT: section 2B. 20. Explains objective of DT: section 3. 21. Allows for independent responding of participant and withholds feedback. 22. Shares results of DT: section 3. 23. Reviews videos. 24. Answers participant questions. 25. Bids participant farewell. Percentage of procedural fidelity % (total number of correct steps/ total number of steps x100) Figure 5 Procedural Fidelity Checklist for Perceptive Sensitivity Training 28 APPENDIX B: RESULTS OF PARTICIPANT ACCURACY OF OBSERVATIONS Phase Trial 1 Trial 2 Trial 3 C C C I C TO Baseline C TO TO C TO TO TO TO TO C C C C C C C C C Post-PST C C C C C C C C C C C C C C C Table 2 Margaret’s Accuracy of Observations of Disengagement Behaviors 29 Phase Trial 1 Trial 2 Trial 3 C I C C I I Baseline I TO TO I C C C I TO I C C C C C C C C Post-PST C C C C C C C C C Table 3 Nathan’s Accuracy of Observations of Disengagement Behaviors 30 Phase Trial 1 Trial 2 Trial 3 C TO C C I TO Baseline C TO TO I TO TO C TO TO I TO I I TO C I C TO I I C C C C Post-PST C C C C C C C C C C C C Table 4 Ella’s Accuracy of Observations of Disengagement Behaviors 31