TRAINING BEHAVIOR TECHNICIANS ON BEHAVIORAL ARTISTRY BY USING GOAL SETTING AND PERFORMANCE FEEDBACK By Pascale Carpentier A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Applied Behavior Analysis– Master of Arts 2024 ABSTRACT Behavioral artistry (BA) is a concept that defines an individual who engages in effective interpersonal skills while delivering applied behavior analysis (ABA) services effectively. Following complaints pertaining to the ‘bedside manner’ of ABA service providers, researchers have been interested in operationally defining and measuring BA in providers. The present investigation used a multiple probe design across three behavior technicians (BT) to assess the effects of a training on BA combined with goal setting and performance feedback as strategies to increase BT’s levels of the BA trait “Likes People.” Results showed that performance feedback and goal setting did increase BTs’ engagement in two behaviors (affectionate touch and positive body language) associated with the “Likes People” trait. The findings indicate that performance feedback and goal setting are effective methods to include in training packages that can increase the interpersonal skills of individuals working with children with ASD. Increasing the use of behavioral components associated with rapport building within service providers in sessions with client could be critical in disputing complaints about ABA from the public. Key words: autism spectrum disorder, behavioral artistry, training, performance feedback, goal setting TABLE OF CONTENTS INTRODUCTION ...........................................................................................................................1 METHOD .......................................................................................................................................6 Participants ..................................................................................................................................6 Setting and Materials ...................................................................................................................7 Dependent Measures ...................................................................................................................8 Experimental Design .................................................................................................................12 Procedures .................................................................................................................................12 Procedural Fidelity ....................................................................................................................16 RESULTS.......................................................................................................................................17 DISCUSSION ................................................................................................................................23 REFERENCES ..............................................................................................................................29 APPENDIX A: PROCEDURAL INTEGRITY CHECKLIST ......................................................31 APPENDIX B: PROCEDURAL FIDELITY CHECKLISTS .......................................................32 APPENDIX C: DATA COLLECTION MATERIALS ..................................................................35 APPENDIX D: DEMOGRAPHIC INFORMATION SURVEY ...................................................37 iii INTRODUCTION A common and effective treatment strategy for many children with autism spectrum disorder (ASD) is early intensive behavioral intervention (EIBI; Reichow, 2012). Discrete trial training (DTT) is a teaching method used in EIBI settings (Smith, 2001) and consists of teaching several smaller components that make up a larger skill (Smith, 2001; Tarbox & Najdowski, 2008). The instruction of these smaller components involves using a discrete trial which is comprised of the teacher providing a short learning opportunity to their student, the student’s response, and a consequence (Smith, 2001; Lerman et al., 2016). DTT is then defined as a behavioral teaching method where repeated presentation of the learning opportunity to the student promotes learning mastery (Smith, 2001; Tarbox & Najdowski, 2008). Children with ASD can benefit greatly from accurate implementation of DTT by their instructor (Downs et al., 2008). Concurrently, research has also shown that children with ASD prefer providers who spend time rapport building at the beginning of their instructional sessions (Lugo et al., 2018). Other studies have found that by implementing a pairing protocol, both decreases in challenging behaviors and increases in in-seat behavior were observed as the session moved towards instruction (Shillingsburg et al., 2019). The preference for a provider that demonstrates interpersonal skills warrants that the field of ABA not only focus on training individuals to proficiently implement intervention but to jointly increase or develop their use of interpersonal skills during therapeutic sessions with their clients. Foxx (1985) notably hypothesized that a natural affinity for delivering ABA services was due to strong interpersonal and effective communication skills. Foxx (1985) branded providers with this innate aptitude to connect with clients and provide high quality ABA services “behavior artists.” Foxx (1985) listed seven core traits of a behavior artist (Table 1). The “behavior 1 technologist,” in contrast, is an individual that systematically delivers ABA intervention but does not truly understand them. Foxx (1985) noted that providers identified as behavior technologists may be why behavior programs fail in some settings as their approach lacks the opportunity to develop an emotional connection between the provider and the client. Behavioral Artistry Traits Table 1. Seven Behavioral Artistry Traits as defined by Foxx (1985). Despite Foxx’s original publication on behavioral artistry in 1985 and follow up in 1996, the first empirical examination of behavioral artistry was not conducted until 2019 (Callahan et al., 2019). Callahan et al. (2019) were able to statistically relate the seven BA characteristics first introduced by Foxx (1985) to specific factors in a personality measure (Cattell et al., 2013), measure BA traits in future practitioners in the field, and examine the differences in treatment delivery between individuals who had high BA and those with low BA. Callahan and colleagues’ 2 research revealed that college students in special education had higher levels of BA than their counterparts in an ABA program (Callahan et al., 2019). Low BA levels among ABA providers is cause for concern as parents surveyed by Callahan et al. (2019) reported their preference for providers who depicted BA descriptors. The data supported Foxx’s (1985) original concerns that ABA practitioners may often be behavior technologists as opposed to behavioral artists. The findings from Callahan and colleagues (2019) have sparked interest in research that improves providers’ behavioral artistry (Bukszpan et al., 2023; Notarianni, 2022; Saur, 2023). We know of two studies that have evaluated the impact of staff training packages on levels of the BA trait “Likes People” among BTs. Bukszpan and colleagues (2023) operationally defined “Likes People” by measuring the extent to which BTs simultaneously engaged in the following three variables; ‘provided attention by addressing client, used a pleasant tone, and engaged in a light-hearted exchange.’ A training package known as the Teaching Interaction Procedure (TIP) was used to train the BTs on these behaviors thought to represent a part of behavioral artistry (Bubzskpan et al., 2023). BTs in the Bukszpan et al. (2023) study demonstrated increases in their use of the behaviors associated with the BA trait “Likes People” when implementing varied teaching programs. Notably, the measurement strategy of Bukszpan et al. (2023) involved capturing all three behaviors occurring simultaneously. One might expect a low chance of all three responses occurring simultaneously under natural conditions, which was reflected in zero-level responding across all participants during baseline. BTs were then taught to specifically engage in those behaviors at the same time to produce what the authors claimed was representative of BA (Bukszpan et al., 2023). Results observed in the Bukszpan and colleagues (2023) study may be artificially inflated in ways that are not observed under more naturally occurring conditions 3 between a BT and client. In addition, BTs were told to use varied teaching programs during their 10 min sessions with clients, the programs’ procedures may have contributed to the increase in BTs engagement in BA behaviors (Bukszpan et al., 2023). As there is limited research on BA and its traits, other studies have also sought to broaden the training strategies used with BTs and identify measurable behaviors that quantify the “Likes People” trait based on both Foxx (1985) and Callahan et al. (2019). Simultaneous to but independent of Bukszpan et al. (2023), Notarianni (2022) also examined the impact of training procedures on the levels of the “Likes People” trait. The researchers administered a behavioral skills training (BST) on BA and then used video self- modeling (VSM) with master’s students in ABA to increase levels of “Likes People” during natural environment training (NET) sessions with clients (Notarianni, 2022). Unlike Bukszpan et al. (2023), Notarianni (2022) measured each behavior separately, which allowed the researcher to capture the existence of certain behaviors associated with BA occurring under baseline conditions. As a result of capturing BA behaviors existing during baseline conditions, only slight increases in the BA behaviors were observed across participants (Notarianni, 2022). The results suggest that practitioners (1) are likely engaging in some levels of BA prior to training, (2) can learn to increase the extent to which they engage in traits associated with BA, and (3) BTs may require specialized and ongoing instructional procedures to demonstrate a more significant and sustained change in BA behaviors (Notarianni, 2022). Performance feedback and goal setting two strategies that could bolster behavioral artistry interventions and address Notarianni’s (2022) call for more potent interventions in support of behavioral artistry. The combined use of performance feedback and goal setting has been shown to be more effective than using feedback alone with individuals (Fallon et al., 2015). 4 Martens and colleagues (1997) used the two strategies together to increase teacher praise in classrooms. Teachers were involved in setting their goal for the number of praise statements they want to emit to students and were provided with written feedback daily on whether they had met their goal (Martens et al., 1997). Increases in socially significant behaviors for both teachers and students were observed when both strategies were applied in classrooms (Martens et al., 1997) and were rated highly favorable by the educators (Martens et al., 1997). A systematic review of performance feedback research indicates that distinct features such as verbal feedback, visual representation of data, and measured progress towards the goal, were observed to be highly impactful when changing teacher behavior (Criss et al., 2022). The two strategies in an intervention package could have a similar effect when implemented with BTs as they have with teachers. Therefore, the current study seeks to train BTs on the BA trait “Likes People” and then to examine the effects of using goal setting and performance feedback as a training package on the levels of “Likes People” behaviors emitted by the technician during DTT sessions with a client. 5 Participants METHOD The participants were three behavior technicians who had a minimum of 3 months of work experience providing ABA services at a university based EIBI clinic in the Midwest. Two supervising Board-Certified Behavior Analysts (BCBA) at the clinic assisted with recruitment by naming BTs who could benefit from an intervention package that incorporated a training event on behavioral artistry, goal setting and performance feedback. Participants were selected based on meeting the following criteria: demonstrating competency in discrete trial training by scoring 90% or better and having no prior experience with trainings on behavioral artistry. To assess the participant’s ability to accurately implement DTT in a session, a procedural integrity (PI) check for DTT was completed by the primary researcher before enrollment in the study (Appendix A). Prior to enrolling in the study, participants were provided an information sheet detailing the study and a consent form to review on their own. The primary researcher then acquired informed consent from all participants. Participants primarily worked with the same client for study sessions, on rare occasion that the client was absent for a session, the participant worked with a different client. Clients at the EIBI clinic receive 30 hours of one-on-one ABA services, demonstrated learner readiness skills (e.g., the ability to sit at their table in their individual treatment area with few challenging behaviors), and their treatment plans all included match to sample programming. Demographic information about the participants was obtained via survey report (Appendix D). Participant 1 was Lucia, a 42-year-old woman who had been a BT at the clinic for 3 months at the start pf the study. Participant 2 was Gregory, a 22-year-old man who had been at the clinic as a BT for 4 months. Lastly, Bea was a 21-year-old woman who had been employed as 6 a BT at the clinic for 14 months prior to the start of the study. One participant identified as Asian, and two identified as White. All participants identified as Not Latino or Hispanic. Setting and Materials Research sessions were conducted in two clinic treatment rooms. Both rooms resemble a preschool classroom with a communal lunch table, a marked play area, and individual treatment areas for each of the 6 to 10 children assigned to the clinic treatment rooms. The client’s individual treatment area contained one rolling stool for the participant, a child-size table (48.3 cm H by 76.2 cm L by 53.3 cm D), and a child-size wooden chair for the client (65.5 cm H by 39.1 cm D by 40.6 cm L). Chairs were arranged so that the participant sat directly across from, or next to, their client. The individual treatment area was approximately 1.25 m2 in area. No changes were made in the clinic’s structure or schedule to accommodate the study. Each participant’s sessions involved different materials according to their client’s individualized clinical programming. For the DTT sessions, all three participants used different sets of auditory-to-visual matching cards as prescribed by the BCBA. The number of cards in a set varied and the sets varied in topics (e.g., clothing items, faces displaying emotions, community helpers). An example of materials would be laminated flashcards (7.6 cm by 12.7 cm) with pictures on them for a match-to-sample program. As the study was conducted over several weeks and programming evolved over time, some of the materials presented during the DTT sessions were familiar to the client, and some were novel. Participants also had access to edible and leisure items to serve as putative reinforcers, as needed, during the DTT session. Participants also used a token economy, or a points card as prescribed by the BCBA for their client’s behavior intervention plan. An electronic timer was provided to the participant to keep track of time. A data sheet was used by the primary researcher 7 to keep track of the number of trials the participant completed in a session with their client (seen in Figure 10 in Appendix C). A second data sheet was used for the data collection of the dependent variables (seen in Figure 9 in Appendix C). A handheld video camera with a tripod was used to record the research sessions and a laptop computer was used in the behavioral artistry trainings. Dependent Measures Participant Dependent Variables The following behaviors were measured to assess for the trait “Likes People”: smile, affectionate touch, positive body language, and vocal affirmation (Callahan et al., 2019; Notarianni, 2022). Smile was defined as ‘the participant moving the corners of their mouth in an upward direction revealing genuineness and pleasantness towards the client’ (Callahan et al., 2019). Affectionate touch was defined as ‘the participant engaging in physical contact with the client and the contact was comforting or playful’ (Notarianni, 2022). Positive body language was defined as ‘a change in the body orientation of the participant that is celebratory or excited in nature as a response to the client’s behavior and there was no physical contact between the participant and their client’ (Notarianni, 2022). Finally, vocal affirmation was defined as ‘the participant emitting a vocal statement directed towards the client that is positive, encouraging, or celebratory in nature’ (Notarianni, 2022). Definitions, examples, and non-examples for each behavior representing an instance of “Likes People” are listed in Table 2. The study used a 10-s partial interval time sampling method to assess each behavior associated with the “Likes People” trait. A partial interval coding scheme consists of scoring the behavior as occurring if the behavior occurred at any point during the 10-s interval (Ledford & Gast, 2018). 8 Dependent Variables Table 2. Participant dependent variables defined. Interobserver Agreement Interobserver agreement (IOA) for each of the four “Likes People” behaviors was calculated by dividing the number of intervals in agreement by total number of intervals in the session and converting this ratio to a percentage. Agreement was defined as the two observers independently scoring a behavior as occurring or not occurring for the same interval. A 9 disagreement was scored when one observer scored an instance of a behavior at a specific interval and the other observer did not score it as an instance of that behavior for that given 10-s interval. Before the baseline phase, the first and second observers developed the coding scheme for the dependent variables. The two observers met and expanded upon the operational definitions for the “Likes People” BA trait based on previous research (Callahan et al., 2019; Notarianni, 2022). Together, the two observers coded five 5-minute sessions from a previous research study for the four dependent variables. Independently, the two observers coded three videos for the four dependent variables, the following is the average agreement for each behavior across the three videos: 97% for smile, 99% (range=97-100%) for affectionate touch, 98% (range=93-100%) for vocal affirmation, and 99% (range=97-100%) for positive body language. Further, a third observer was trained on how to collect IOA data via a presentation. To meet mastery criteria (80%) on the coding scheme, the third observer coded the same three videos and agreed with the original two observers across all four behaviors: 89% (range=87-90%) for smile, 91% (range=83-100%) for affectionate touch, 94% (range=90-100%) for vocal affirmation, and 99% (range=97-100%) for positive body language. All observers reviewed the video recordings of the study sessions for IOA data collection and used the same measurement procedures as the primary researcher. If IOA fell under 80% during the study, the primary researcher met with the independent observer and discussed discrepancies before conducting any further IOA data collection. IOA was collected for 38% of Lucia and Gregory’s sessions across both the baseline and intervention phases. IOA was collected for 40% of Bea’s sessions across both the baseline and 10 intervention phases. Sessions where IOA data was measured were randomly selected using an online random number generator (Haahr, n.d.). IOA data for the dependent variable smile at baseline was as follows: average IOA for Lucia’s instances of smile towards the client was 85% (range=69-100%), for Gregory it was 94% (range=88-100%), and for Bea it was 86% (range=73-100%). Average IOA for Lucia’s instances of smile towards the client was 84% (range=79-92%), for Gregory it was 92% (range=88-93%), and for Bea it was 84% (range=78-88%) during the intervention phase. The following is the IOA data for the dependent variable affectionate touch. In baseline, average IOA for Lucia’s instances of affectionate touch towards the client was 82% (range=81- 82%), for Gregory it was 86% (range=79-93%), and for Bea it was 100%. Average IOA for Lucia’s instances of affectionate touch towards the client was 88% (range=76-100%), for Gregory it was 93% (range=81-100%), and for Bea it was 96% (range=89-100%) during the intervention phase. IOA data for the dependent variable vocal affirmation at baseline was as follows: average IOA for Lucia’s instances of vocal affirmation towards the client was 88% (range=82-94%), for Gregory it was 93% (range=80-100%), and for Bea it was 90% (range=73-100%). In the intervention phase, average IOA for Lucia’s instances of vocal affirmation towards the client was 89% (range=74-100%), for Gregory it was 95% (range=88-100%), and for Bea it was 96% (range=87-100%). The following is the IOA data for the dependent variable positive body language. In baseline, average IOA for Lucia’s instances of positive body language towards the client was 100%, for Gregory it was 100%, and for Bea it was 100%. In the intervention phase, average 11 IOA for Lucia’s instances of positive body language towards the client was 98% (range=92- 100%), for Gregory it was 93% (range=93-94%), and for Bea it was 90% (range=82-100%). Experimental Design The researchers used a concurrent multiple probe across participants design to examine the effect of a training package that included a training event on the BA trait “Likes People” with participative goal setting and ongoing performance feedback on the BTs’ behavior (Ledford & Gast, 2018). There were two phases: the baseline phase and the intervention phase. In the baseline phase, a minimum of five sessions were conducted with each participant. The intervention phase consisted of a training event on behavioral artistry concluding with a goal setting session and ongoing performance feedback post training event. All intervention components are described below. Once the first participant entered the intervention phase, the remaining participants remained in the baseline phase where probe sessions were conducted every 2-5 sessions prior to moving onto the intervention phase. No more than one baseline or probe session was conducted per day. Gregory moved onto the intervention phase once there was a stable trend in Lucia’s data, while Bea continued with probe sessions every 2-5 sessions. The same scheme was followed until all three participants moved onto the intervention phase. Procedures Across the entire study, all DTT study sessions were observed by the primary researcher and were video recorded for the purposes of data collection, interobserver agreement (IOA), and procedural fidelity. Videos were immediately uploaded to a university OneDrive folder after the session and labeled using the identification system. Only the primary researcher, two independent observers, and advising faculty had access to the data on OneDrive. All training 12 package paper documents were stored in a binder with separate tabs for each participant with participant identification numbers and scanned into the OneDrive. Baseline and probe sessions. Participants were told to conduct a DTT session (minimum length of 2 min, 30 s and maximum length of 5 min) at the client’s individual treatment area using the client’s current programming prescribed by the supervising BCBA. Participants were asked to gather the materials to run the DTT session ahead of time. For consistency, the participant was asked to provide between eight to twelve trials during the DTT session. DTT sessions that were less then 2 min in length were excluded from the study. DTT sessions that were interrupted by the client’s challenging behavior were also excluded from the study. DTT sessions with less than eight trials were excluded from the study and DTT sessions with more than twelve trials were included however coding concluded at the twelfth trial. An example trial for this type of programming might be asking the client to match auditory to visual stimuli by arranging three pictures of farm animals in a straight line on the table at arm’s length to the client and saying, “Where’s the cow?” The participant was provided an electronic timer set to 2 min, 30 s by the primary researcher to keep track of the time in their session; the primary researcher maintained a second timer set to 5 min to end the session. Behavioral Artistry Training Event Upon establishing a stable baseline, the intervention package was introduced to Lucia (P1). The training package included one training event on behavioral artistry with goal setting followed by ongoing performance feedback. The training event occurred once and lasted approximately 45 min. This training event was scheduled and conducted in the morning before client treatment began in the clinic treatment room. 13 At the training event, the participant was first introduced to behavioral artistry and the seven BA traits via PowerPoint presentation. All participants were provided with a printout of the PowerPoint presentation to keep. In-depth information on the “Likes People” trait was provided, specifically about the associated behaviors (e.g., smile, vocal affirmation, positive body language, and affectionate touch). A script was used to ensure that all participants received the same information about the topic. The participant was then shown video models of the “Likes People” behaviors and video models of non-examples. A role play component followed between the primary researcher and the participant with relevant stimuli. Each behavior from “Likes People” was demonstrated by the researcher role playing as the BT with the participant acting as the client The primary researcher and the participant then switched roles. During the role-playing session, the participant had to emit five instances of each of the “Likes People” behaviors. If they did not emit five instances during the role-playing for a certain behavior, another role-playing session occurred but the participant was told to only focus on the behavior they did not meet criteria for, and corrective feedback was given if the behavior was not exhibited accurately. The primary researcher provided positive feedback for behaviors emitted accurately. Only after five instances of each behavior was accurately demonstrated by the participant in a role-playing session would the researcher move on to the next component of the intervention package. Goal Setting Directly after the completion of the training event, the primary researcher introduced the next component of the training package: goal setting. The primary researcher showed the participant their baseline data. The baseline data was presented via four visual graphs, each graph representing one of the “Likes People” behaviors. The primary researcher then asked the participant to establish an attainable goal for each behavior based on their baseline data. This 14 meant choosing a percentage of instances to increase to. Suggestions could be provided by the primary researcher based on the participant’s baseline performance (e.g., an increase between 20- 25% from baseline); however, it was the participant who made the final decision. On average, the goal setting portion lasted 5 min. The primary researcher drew a line on the graphs to visually represent the goals the participant set for themselves. Based on their baseline scores, the participants set the following goals for the four “Likes People” behaviors. For smile, Lucia and Gregory both set a goal of increasing to 55% of intervals with an instance of smile during their sessions while Bea set a goal of increasing to 70% of intervals with an instance of smile. For affectionate touch, Lucia and Gregory both set a goal of increasing to 40% of intervals with an instance of affectionate touch while Bea set a goal of 20%. For vocal affirmation, Lucia set a goal of increasing to 60% of intervals with an instance of this behavior while Gregory set a goal of 85% and Bea set a goal of 90%. For the final “Likes People” behavior, all three participants set a goal of increasing to 20% of intervals with an instance of positive body language during a session. Performance Feedback Ahead of every DTT session after the training event, a performance feedback session was conducted. At the performance feedback session, the primary researcher presented the participant with visual feedback by showing them four separate graphs displaying the participants’ data on the four “Likes People” behaviors with their goal marked on the graph with a line. The primary researcher provided verbal feedback where a description of the data for all four behaviors was given (e.g., verbally informed the participant of their last two data points and reiterating their goal). Positive statements such as “nice job!” were provided if increases towards a goal or meeting a goal were observed. No other statements were made. If none of the goals were met, 15 the primary researcher provided feedback on the participant’s implementation of DTT. Goals were not amended after the goal setting session and no training or roleplaying was conducted during meetings where performance feedback was given. If the participant had any questions regarding how to emit any of the behaviors in their DTT session, they were directed to the PowerPoint presentation printout that was provided at the training event. These feedback sessions lasted on average 2 min. Performance feedback was administered outside of the clinic treatment room in the hallway during treatment hours. Procedural Fidelity For all components of the training package (behavioral artistry training event, goal setting, and performance feedback), a second researcher reviewed video recordings of those components and collected procedural fidelity data using two checklists developed by the primary researcher (seen in Figures 6-7 in Appendix B). Mean procedural fidelity for the training on behavioral artistry with goal setting was 100% and performance feedback procedures was 100%. Procedural fidelity data were collected for 33% of behavioral artistry training events and 31% of Lucia’s performance feedback sessions, for 42% of Gregory’s performance feedback sessions, and for 50% of Bea’s sessions. Sessions where procedural fidelity was measured were randomly selected using an online random number generator (Haahr, n.d.). Lastly, procedural fidelity data were collected for the training on the coding scheme with the third observer in which 100% accuracy was met (seen in Figure 8 in Appendix B). 16 RESULTS Figure 1 presents each participant’s percentage of 10-s intervals with an instance of affectionate touch. Lucia’s percentage of 10-s intervals with affectionate touch at baseline was 13% (range 0-35%). During intervention phase, Lucia’s affectionate touch average increased to 62% (range 33 – 86 %). Gregory’s percentage of 10-s intervals with affectionate touch at baseline was 16% (range 0-27%). During intervention phase, Gregory’s affectionate touch average increased to 76% (range 60 – 93 %). Bea’s percentage of 10-s intervals with affectionate touch at baseline was 0%. During intervention phase, Bea’s affectionate touch average increased to 47% (range 33 – 71 %). Figure 2 presents each participant’s percentage of 10-s intervals with an instance of positive body language. Lucia’s percentage of 10-s intervals with positive body language at baseline was 0%. During intervention phase, Lucia’s positive body language average increased to 12% (range 4 – 24 %). Gregory’s percentage of 10-s intervals with positive body language at baseline was 0%. During intervention phase, Gregory’s positive body language average increased to 23% (range 7 – 33 %). Bea’s percentage of 10-s intervals with positive body language at baseline was 0%. During intervention phase, Bea’s smile average increased to 19% (range 0 – 40 %). Figure 3 presents each participant’s percentage of 10-s intervals with an instance of smile. Lucia’s percentage of 10-s intervals with smile at baseline was 24% (range 0-50%). During intervention phase, Lucia’s smile average increased to 67% (range 43 – 90 %). Gregory’s percentage of 10-s intervals with smile at baseline was 14% (range 0-50%). During intervention phase, Gregory’s smile average increased to 83% (range 56 – 94 %). Bea’s percentage of 10-s 17 intervals with smile at baseline was 48% (range 27-79%). During intervention phase, Bea’s smile average increased to 84% (range 77 – 94 %). Figure 4 presents each participant’s percentage of 10-s intervals with an instance of vocal affirmation. Lucia’s percentage of 10-s intervals with vocal affirmation at baseline was 38% (range 21-53%). During intervention phase, Lucia’s vocal affirmation average increased to 57% (range 40 – 82 %). Gregory’s percentage of 10-s intervals with vocal affirmation at baseline was 64% (range 25-92%). During intervention phase, Gregory’s vocal affirmation average increased to 75% (range 63 – 93 %). Bea’s percentage of 10-s intervals with vocal affirmation at baseline was 70% (range 47-86%). During intervention phase, Bea’s vocal affirmation average decreased to 61% (range 43 – 73 %). 18 Figure 1. Changes in Affectionate Touch Towards Client after Intervention. Percentage of 10-s intervals with instances of affectionate touch by participants towards client during Baseline and Intervention (Goal Setting & Performance Feedback). Open circles indicate sessions where the participant worked with a different client. 19 Figure 2. Changes in Positive Body Language Towards Client After Intervention. Percentage of 10-s intervals with instances of positive body language by participants towards client during Baseline and Intervention (Goal Setting & Performance Feedback). Open circles indicate sessions where the participant worked with a different client 20 Figure 3. Changes in Smile Towards Client After Intervention. Percentage of 10-s intervals with instances of smile by participants towards client during Baseline and Intervention (Goal Setting & Performance Feedback). Open circles indicate sessions where the participant worked with a different client. 21 Figure 4. Changes in Vocal Affirmation Towards Client After Intervention. Percentage of 10-s intervals with instances of vocal affirmations by participants towards client during Baseline and Intervention (Goal Setting & Performance Feedback). Open circles indicate sessions where the participant worked with a different client. 22 DISCUSSION The current study sought to train BTs on four behaviors associated with the “Likes People” behavioral artistry trait and then to examine the effects of using goal setting and performance feedback as a training package on the levels of this BA trait during DTT sessions. All participants experienced increases in two behaviors from their baseline levels: affectionate touch and positive body language. Moderate increases were observed for two participants for smile. These results are consistent with the findings from Notarianni (2022) on the conclusion that BTs can be trained on “Likes People” behaviors and that BA is not something that is innate to an individual. Further, the results also support previous research on the use of goal setting and performance feedback with individuals who work with children in school-type settings (Martens et al., 1997). The present study extends the growing body of research that aims to focus on practitioners’ interpersonal skills. The Callahan and colleagues (2019) study was the catalyst for an increase in the research focusing on behavioral artistry, specifically the “Likes People” trait (Bukszpan et al., 2023; Notarianni, 2022). The operational definitions developed by the researchers for the behaviors associated with “Likes People” in the current study were derived from the initial Callahan et al. (2019) study. As intervention packages are developed to increase this BA trait, measuring other BA traits among providers when in sessions with clients or stakeholders should be the next target. In the Bukszpan et al. (2023) study, an instance of the “Likes People” was defined as all behaviors occurring simultaneously during an interval. Previous research, however, defined an instance of this BA trait when the BT was engaged in any of the “Likes People” behaviors (Callahan et al., 2019; Notarianni, 2022). The approach that Bukszpan and colleagues (2023) 23 used to measure levels of “Likes People” resulted in zero-level baseline data of the participants’ engagement of the behaviors. In contrast, Notarianni (2022) and the current study, observed BTs demonstrating “Likes People” behaviors at baseline, which may be a more representative depiction of naturally occurring events between BTs and clients. In the present study particularly, participants emitted both smile and vocal affirmation, in moderate to high levels. This is not surprising as BTs are often trained to deliver reinforcement through praise statements (i.e., vocal affirmation) especially during DTT sessions. One can imagined that by setting this criterion of all behaviors occurring during the interval to be considered as an occurrence of “Likes People,” that sessions may look formulated which further progresses the narrative of ABA being repetitive and does not embody the original intention of Foxx’s (1985) concept of the behavioral artist. Further, the present study used goal setting as an intervention strategy as the participant sets the criterion for themselves to achieve their desired outcome while Bukszpan et al. (2023) set an arbitrary mastery criterion for “Likes People” of 70%. Though there is no research yet on whether a certain level of each behavior results in the optimal amount of “Likes People” for a session with a client it would appear misleading to indicate that by achieving a set mastery criterion during a session the BT has become a behavioral artist. A fluidity in demonstrating these traits in different settings with clients or stakeholders may be a better representation of the behavior change agent that Foxx (1985) described. Notarianni (2022) established that individuals can be trained on BA traits and proposed that research should evaluate different training strategies to increase BTs levels of behavioral artistry. The intervention package in Notarianni (2022) focused on measuring the cumulation of the “Likes People” behaviors whereas, in the hopes of expanding Notarianni’s work (2022), the current study aimed to provide the feedback on the participants’ isolated performance of each 24 behavior during sessions with clients. The strategies implemented in this training package mirror what BTs are familiar with and execute every day: visual analysis of data. The results indicate that the intervention was effective, and the participants responded to the visual feedback on their progress towards a set goal. Had the present study focused only on cumulative scores of “Likes People,” and increasing this score, missed opportunities to capture affectionate touch and positive body language may have occurred. Limitations noted in Notarianni (2022) were amended in the present study as it acts as an expansion. For one, Notarianni (2022) noted that a 5-s partial interval sampling method may have been too short to capture these behaviors as only small increases of instances of positivity were observed. The present study therefore used a 10-s partial interval sampling method and established that a longer interval length may be enough time to capture instances of the “Likes People” behaviors during sessions with clients. Further, as strict COVID-19 protocols were in place during data collection, Notarianni (2022) was unable to measure participants’ smiles during sessions as face masks were always required in the clinic. To expand on Notarianni’s (2022) work, the current study included smile as one of the behaviors associated with “Likes People” which aligned with the definition from Callahan et al. (2019). Participants in the present investigation did not exhibit positive body language at baseline. The criticism of service delivery appearing as cold and repetitive could be due to a lack of these behaviors that require creative body language and orientation. Participants reported that they found positive body language (PBL) to be the most challenging to exhibit during their sessions with clients. This difficulty was observed before intervention with none of the participants emitting the behavior at baseline and continued after intervention as the levels of positive body language were the lowest among all behaviors across participants. During the 25 training event, positive body language was consistently the last behavior that participants mastered in the role-playing component. There may be multiple reasons as to why PBL was the most challenging behavior for participants. Study sessions took place in a clinic that acts as a preschool classroom; therefore, other BTs were aware when study sessions were being conducted as the recording equipment was visible to everyone. Participants may not have felt as comfortable emitting these behaviors that required more physical use of their bodies with an audience. An inability to disregard the audience and exhibit behaviors that are reinforcing to the client or progress the client’s treatment imply that the participants may also have low levels of ‘self-actualization’, another BA trait (Callahan et al., 2019). Further, with DTT trials occurring more quickly by nature and the factor of material management, BTs may have found it more strenuous to incorporate PBL in their sessions when their hands were occupied with the teaching material. Regardless of levels of PBL being the lowest of all four behaviors, participants were still able to meet their goal after the intervention was implemented. The BTs’ low levels of positive body language, even after intervention, may be surprising for individuals familiar with early intensive behavioral intervention. The purpose of EIBI is to use the principles of ABA to teach children between the ages of 2 and 6 skills that target the core deficits of ASD: social communication and interaction and repetitive behavior (Klintwall & Eikseth, 2014). As intervention goals for children of preschool age often emphasize learning how to react to social cues (Klintwall & Eikseth, 2014), the lack of positive body language by BTs, and their disclosure on the difficulty of emitting these behaviors, reveals further avenues of future research. BTs may have emitted these behaviors less than the other “Likes People” behavioral components because of previous experience of them not being reciprocated by their client partners. As research continues to examine which strategies are preferred when training 26 BTs on “Likes People” and BA, it would be worth investigating if BTs demonstrate more instances of PBL when working with neurotypical children when compared to neurodivergent children who may not reciprocate the technician’s body language. Future research could also investigate whether higher levels of PBL would be observed if BTs and their clients worked in isolated treatment rooms where there is no audience control. There are several limitations to this study that need to be acknowledged. As the purpose of the study is related to dismantling the negative public perception of ABA, measuring client’s behavior during sessions after their provider has been trained on specific rapport building strategies and interpersonal skills is necessary. Though Notarianni (2022) saw no significant changes to client behaviors after training participants on behavioral artistry, the sessions in the current study were shorter with the same number of demands place. As behavior change in clients can be gradual, future research should evaluate if significant behavior changes occur over a longer period. A subsequent limitation to this study was that all the participants worked with clients who demonstrate learner readiness skills at their worktable for a minimum of three minutes. This means that clients exhibited very few challenging behaviors during the study sessions. Research focusing on rapport building has demonstrated that as rapport improved between the client and the therapist decreases in problem behavior in sessions were observed (Magito McLaughlin & Carr, 2005). It could be argued that behavioral artistry could serve as more imperative when working with clients’ who emit problem behavior. Therefore, future research should evaluate whether BTs can emit these “Likes People” behaviors during more challenging sessions with clients and if challenging behaviors decrease with the implementation of these “Likes People” behaviors and improve the quality of treatment that clients receive. 27 As the field continues to expand research on practitioners’ use of interpersonal skills during sessions, investigating the social validity of the training packages should be considered. Despite the observed increases in the “Likes People” behaviors in the current study, it is unknown whether the participants found the procedures appropriate or significant to their work. Participants may have found the time requirements, or that they had to step away from their client during treatment hours, more of a burden compared to potential benefits from the intervention. Future research on BA training strategies should examine the social validity of the procedures being set forth, specifically participant preference for the type of feedback received and the timing of the feedback. Immediate performance feedback could serve to be more reinforcing to BTs when learning to implement behavioral artistry in their sessions. Previous research on training BTs on interpersonal skills implemented immediate feedback via clickers in which participants rated the feedback method as acceptable and not distracting during sessions with clients (Canon & Gould, 2022). Considering that BTs are the individuals on the ground working with clients, their perspective on these strategies should be considered as we move towards developing how to teach them to use these interpersonal skills in sessions with clients. Despite the overwhelming evidence supporting ABA and its procedures, it does not exist without criticism from clients, shareholders, and other providers largely on its inferred rote delivery (Schriebman, 2005). It is therefore critical to broaden research regarding BTs’ interpersonal skills and the client-therapist relationship especially as diagnostic tools are further developed and the number of consumers continues to rise (Maenner et al., 2023). The present study adds to the increasing body of work that focuses on BTs learning to engage in these BA behaviors by expanding on Notarianni (2022)’s work on behavioral artistry’s “Likes People” trait as well the value of using performance feedback and goal setting with BTs. 28 REFERENCES Bukszpan, A.R., Anderson, A., Moon, E., Kaplan, A., & Leaf, J.B. (2023) Training behavior technicians to become behavior artists through the teaching interaction procedure. Behavioral Interventions, 38(4),1-17. Callahan, K., Foxx, R.M., Sxierczynski, A., Aerts, X., Mehta, S., McComb, M., … & Sharma, R. (2019). Behavioral artistry: Examining the relationship between the interpersonal skills and effective practice repertoires of applied behavior analysis practitioners. Journal of Developmental Disorders, 49, 3557-3570. https://doi.org/10.1007/s10803-019-04082-1 Canon, L. F., & Gould, E.R. (2022). A preliminary analysis of the effects of clicker training and verbal instructions on the acquisition of relationship-building skills in two applied behavior analysis practitioners. Behavior Analysis in Practice, 15, 383-396. https://doi.org/10.1007/s40617-021-00555-x Cattell, R.B., Cattell, A.K., & Cattell, H.E.P. (2013). 16PF fifth edition questionnaire. Carmel, IN: Performance Assessment Network. Criss, C.J., Konrad, M., Alber-Morgan, S.R., & Brock, M.E. (2022). A systematic review of goal setting and performance feedback to improve teacher practice. Journal of Behavioral Education, 1-22. 10.1007/s10864-022-09494-1. Downs, A., Downs, R.C., & Rau, K. (2008). Effects of training and feedback on discrete trial teaching skills and student performance. Research in Developmental Disabilities, 29(3), 235-246. https://doi.org/10.1016/j.ridd.2007.05.001 Fallon, L.M., Collier-Meek, M.A., Maggin, D.M., Sanetti, L.M., & Johnson, A.H. (2015). Is performance feedback for educators an evidence-based practice? A systematic review and evaluation based on single-case research. Exceptional Children, 81(2), 227-246. Foxx, R. M. (1985). The Jack Tizzard Memorial Lecture Decreasing Behaviours: Clinical, Ethical, and Environmental Issues. Australia and New Zealand Journal of Developmental Disabilities, 11(4), 189-199. Foxx, R.M. (1996). Translating the covenant: The behavior analyst as ambassador and translator. The Behavior Analyst, 19, 147-161. https://doi.org/10.1007/BF03393162 Haahr, M. (n.d.). True random number service. RANDOM.ORG - True Random Number Service. Retrieved March 21, 2023, from https://www.random.org/. Klintwall, L., & Eikseth, S. (2014). Early and intensive behavioral intervention (EIBI) in autism. Comprehensive guide to autism, 1(1), 117-137. Ledford, J.R., & Gast, D.L. (Eds). (2018). Single case research methodology (3rd ed.). Routledge. 29 Lerman, D.C., Valentino, A.L., & LeBlanc, L.A. (2016). Discrete Trial Training. In R. Lang, T.B. Hancock, & N.N. Singh (Eds.), Early intervention for children with autism spectrum disorder (pp. 47-83). Springer. 10.1007/978-3-319-30925-5_3 Lugo, A.M., McArdle, P.E., King, M.L., Lamphere, J.C., Peck, J.A., & Beck, H.J. (2018). Effects of presession pairing on preference for therapeutic conditions and challenging behavior. Behavior Analysis in Practice, 12, 188-193. https://doi.org/10.1007/s40617-018-0268-2 Maenner, M.J., Warren, Z., Williams, A.R., Amoakohene, E., Bakian, A.V., Bilder, D.A., … & Shaw, K.A. (2023). Prevalence an characteristics of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, Unites States, 2020. MMWR Surveillance Summaries, 72(2), 1-14. Martens, B.K., Hiralall, A.S., & Bradley, T.A. (1997). A note to teacher: Improving student behavior through goal setting and feedback. School Psychology Quarterly, 12(1), 33-41. https://doi.org/10.1037/h0088945 Magito McLaughlin, D., & Carr, E.G. (2005). Quality of rapport as a setting event for problem behavior: Assessment and intervention. Journal of Positive Behavior Interactions, 7(2), 68-91. https://doi.org/10.1177/10983007050070020401 Notarianni, E. (2022). Assessing ‘Behavioral Artistry’ staff training on behavior technicians’ use of behavioral artistry traits and the behavior of young children with autism spectrum disorder (Publication No. 29166560) [Master’s thesis, Michigan State University]. ProQuest Dissertations & Theses Global. Reichow, B. (2012). Overview of meta-analyses on early intensive behavioral intervention for young children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42, 512-520. Saur, R. (2023). Behavioral Artistry and Applied Behavior Analysis: Assessing the Effects of a Staff Training Package on ‘Perceptive Sensitivity’ Levels in Behavior Analytic Treatment (Publication No. 30486249) [Master’s thesis, Michigan State University]. ProQuest Dissertations & Theses Global. Shillingsburg, M.A., Hansen, B., & Wright, M. (2019). Rapport building and instructional fading prior to discrete trial instruction: Moving from child-led play to intensive teaching. Behavior Modification, 43(2), 288-306. Smith, T. (2001). Discrete trial training in the treatment of autism. Focus on Autism and Other Developmental Disabilities, 16(2), 86-92. Tarbox, R.S. & Najdowski, A.C. (2008). Discrete Trial Training as a Tefaching Paradigm. In J.K. Luiselli, D.C. Russo, W.P. Christian, & S.M. Wilcyznski (Eds). Effective practices for children with autism: educational and behavior support interventions that work (pp. 181- 194). Oxford University Press. 30 APPENDIX A: PROCEDURAL INTEGRITY CHECKLIST Figure 5 Procedural integrity checklist used to assess participant’s implementation of DTT. 31 APPENDIX B: PROCEDURAL FIDELITY CHECKLISTS BA “Likes People” Training Event & Goal Setting Procedural Fidelity Checklist Date: Trainer: For each step of intervention, record “+” if trainer completes the step and a “-” if the step is not completed. If a step is not applicable, record “N/A” Participant: Step in intervention Ice breaker. 1. State objectives. Introduce topic of behavioral artistry. Explain the importance of behavioral artistry. 2. 3. 4. Data 5. Define each “likes people” trait (smile, vocal affirmation, affectionate touch, and positive body language). Repeat a total of 4 times. 6. Show synthesis video of examples of BA. Show synthesis video of non-examples of BA. 7. 8. Conduct role (5 minutes). 5-min as BT and 5-min as client (switching roles). 9 Provide behavior specific feedback on role play. 10. Repeat steps 8 and 9 until participant meets criteria for all 4 behaviors. 11. Broadly define goal setting. Present baseline data via visual graph to participant on a single sheet of paper (4 graphs; 1 per behavior). 12. 13. Discuss and describe baseline data with participant. Ask participant to verbally set an attainable and demonstrable goal for each behavior (must be a minimum of 20% increase from BL). 14. 15. Record on the graphs each goal the participant sets for themselves. Store paper graph in data collection binder/folder. 16. Percentage of procedural fidelity (total number of correct steps/ total number of steps x100) Procedural Fidelity Collected by: Figure 6 Procedural fidelity checklist used to assess accuracy of implementation of behavioral artistry event and goal setting session. % 32 Performance Feedback Session Procedural Fidelity Checklist Date of when PF was delivered: For each step of intervention, record “+” if trainer completes the step and a “-” if the step is not completed. If a step is not applicable, record “N/A” Data Step in intervention Present data from previous sessions via visual graph to participant on a sheet of paper (4 graphs; 1 per behavior) prior to observation. Discuss and describe previous sessions’ data with participant. Answer any questions an participant may have. Keep meeting to under 5 minutes. 1. 2. 3. 4. Percentage of procedural fidelity (total number of correct steps/ total number of steps x100) Second Observer: Figure 7 Procedural fidelity checklist used to assess accuracy of implementation of performance feedback session. % 33 BA “Likes People” Second Observer Coding Training Procedural Fidelity Checklist PF Date: Date of Training: Trainers: Daniella and Pascale For each step of intervention, record “+” if trainer completes the step and a “-” if the step is not completed. If a step is not applicable, record “N/A” Participant: Step in intervention Data 1. Introduce topic of behavioral artistry (BA). 2. State the objectives. 3. State the expectations before coding. 4. State the expectations for coding. 5. Discuss the measurement system. 6. Discuss the coding sheet. 7. Define BA definition. 8. List 4 characteristics of “Likes People”. 9. 10. 11. Define each “Likes People” characteristic and include an example and non-example (repeat for a total of 4 times). Practice coding together and allow for any questions to be asked and answered. Provide information on where to find coding materials and next steps on training for coding. 12. Thank graduate assistant for their time. Percentage of procedural fidelity (total number of correct steps/ total number of steps x100) Second Observer: Figure 8 Procedural fidelity checklist used to assess accuracy of implementation of the IOA training for the second observer. % 34 APPENDIX C: DATA COLLECTION MATERIALS PARTICIPANT DVs DATA SHEET 1         2         3       4  BT   (circle one)  Client Initials   Session Number  Coder  (circle one)  Conditio n  (circle one)  Video Label  Start Coding   Directions: Record a “+” for any interval where the participant demonstrates any of the following behavioral indicators of an instance of likes people (smile, affectionate touch, vocal affirmations, and/or positive body language) at any point during the 10- s interval.  Collect the total “+” per each trait at the end of the coding.  Record “+” in CMTIVE if any of the behaviors in that internal recorded a “+.”   BL                          Performance Feedback   First                  Reliability  Trials  2    3    4    5    6    7    8    9    10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29    30  Total  Time Stamps  Likes people  behavior  1    S  AT  VA  PBL  CMTIVE  Percenta ge of Intervals with an instance of a behavior  Figure 9 Data collection sheet for dependent variables. Smile (S) : _______ /30 =  ______%   Affectionate Touch (AT): _______ /30 =  ______%   Vocal Affirmation (VA):  _______ /30 =  ______%   Positive Body Language (PBL): _______ /30 =  ______%   Cumulative (CMTIVE): _______ /30 =  ______%    35                                                                                                                                                                                                                                                                                                                                                                                                                                                                               Participant ID: Trial 1 Trial 2 In Session Trials Tracking Session #: Date: Trial 3 Trial 4 Trial 5 Trial 6 Trial 7 Trial 8 Trial 9 Trial 10 Approx. Session Length: Table 3 In session DTT trials tracking. 36 APPENDIX D: DEMOGRAPHIC INFORMATION SURVEY Demographics Survey Participant: Behavioral Artistry “Likes People” Training Demographics Survey. Please complete this survey. This survey will take less than 5 minutes to complete. Your answers will be helpful in enhancing the quality of the behavioral artistry “Likes People” training package. 1. What race do you identify with? Please select all that apply, if applicable. American Indian or Alaska Native Asian Black or African American Native Hawaiian or Pacific Islander White Other 1 Prefer not to answer 1 7 2. What ethnicity do you identify with? 7 0 Latino(x) or Hispanic 0 7 Not Latino(x) or Hispanic 7 1 Prefer not to answer 1 7 7 Man Woman Non-binary Prefer not to answer 3. What gender do you identify with? 4. What is your age? (in years) ___________ Figure 11 Demographics survey for participant 1 7 0 7 1 7 37