ASSESSING ‘BEHAVIORAL ARTISTRY’ STAFF TRAINING ON BEHAVIOR TECHNICIANS’ USE OF BEHAVIORAL ARTISTRY TRAITS AND THE BEHAVIOR OF YOUNG CHILDREN WITH AUTISM SPECTRUM DISORDER By Emilia Notarianni A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Applied Behavior Analysis – Master of Arts 2022 ABSTRACT ASSESSING ‘BEHAVIORAL ARTISTRY’ STAFF TRAINING ON BEHAVIOR TECHNICIANS’ USE OF BEHAVIORAL ARTISTRY TRAITS AND THE BEHAVIOR OF YOUNG CHILDREN WITH AUTISM SPECTRUM DISORDER By Emilia Notarianni Training high quality practitioners continues to be a priority in the field of Applied Behavior Analysis. The concept of behavioral artistry (BA) identifies traits that may contribute to an individual’s success as a behavior analyst, including attributes such as resilience, sense of humor, and compassion. Despite BA’s potential for meaningful impact in the field, research in this area remains limited. The current study aimed to behaviorally define the BA trait “likes people” and investigated the effects of a behavioral skills training and video self-modeling on the development of behaviors representative of that BA trait. Additionally, data were collected on client behaviors, in order to examine the effects of the BA training on client behavior. Four dyads consisting of one behavior technician and one young child diagnosed with Autism Spectrum Disorder participated in the study, which took place in an EIBI clinic affiliated with a midwestern university. Results showed an increase in the use of the BA trait “likes people” in all adult participants. However, this behavior change did not maintain over time. Child participants demonstrated very little behavior change between phases of this study. These findings indicate that BST and VSM may not be the most effective intervention for training staff on BA. Key words: behavioral artistry, behavioral skills training, staff training, video self-modeling TABLE OF CONTENTS LIST OF TABLES ......................................................................................................................... iv LIST OF FIGURES .........................................................................................................................v INTRODUCTION ...........................................................................................................................1 METHOD .......................................................................................................................................7 Participants and Setting ...............................................................................................................7 Materials ......................................................................................................................................9 Dependent Measures ...................................................................................................................9 Adult Participant Dependent Variables .................................................................................9 Child Participant Dependent Variables ...............................................................................11 Interobserver Agreement .....................................................................................................12 Experimental Design .................................................................................................................14 Procedures .................................................................................................................................15 Baseline ...............................................................................................................................15 BST ......................................................................................................................................15 Post-BST .............................................................................................................................16 Post-BST with VSM ............................................................................................................16 Procedural Fidelity ....................................................................................................................17 BST ......................................................................................................................................17 VSM .....................................................................................................................................17 RESULTS ......................................................................................................................................18 Adult Participants ................................................................................................................18 Child Participants ................................................................................................................20 DISCUSSION ................................................................................................................................23 APPENDICES ...............................................................................................................................29 Appendix A: Data Collection Materials ..............................................................................30 Appendix B: Procedural Fidelity Checklists .......................................................................34 REFERENCES ..............................................................................................................................37 iii LIST OF TABLES Table 1 Adult participant dependent variables defined .................................................................11 Table 2 Child participant dependent variables defined..................................................................12 iv LIST OF FIGURES Figure 1 Adult participants’ percentage of 5-s intervals with instances of positivity towards child participant. Asterisks indicate that a substitute child participated in the session due to the usual child participant being absent from treatment ...............................................................................19 Figure 2 Child Participant Dependent Measures ...........................................................................22 Figure 3 In-session data collection sheet used by adult participants and experimenter to maintain consistency across sessions ............................................................................................................31 Figure 4 Data collection sheet for adult participant dependent variables.. ....................................32 Figure 5 Data collection sheet used for child participant dependent variables ..............................33 Figure 6 Procedural fidelity checklist used to assess accuracy of implementation of the BST intervention ....................................................................................................................................35 Figure 7 Procedural fidelity checklist used to assess accuracy of implementation of the VSM intervention ....................................................................................................................................36 v INTRODUCTION Applied behavior analysis (ABA) is known to be an effective treatment for autism spectrum disorder (ASD) (Eikeseth et al., 2002). Since its introduction in the 1960’s, the field of ABA has continued to evolve through careful research (Foxx, 2008). However, despite the field’s evidence based advances, ABA is often criticized for matters such as practitioner’s hostility towards other therapy providers and its inaccessibility due to the use of field specific jargon (Foxx, 1996). According to Foxx (1996), individual behavior analysts are responsible for challenging the negative perception of ABA. He asserted that repairing the image of ABA at the time was integral to the success of the field. To foster this change, he suggested focusing on the emotional reactions towards the implementation of ABA therapy (Foxx, 1996). By calling attention to the value of the work behavior of behavior analysts, Foxx believed that public perception of ABA will improve. Furthermore, he stated that creating positive setting conditions is essential to conveying the importance of the work that behavior analysts do. For example, an individual would be more likely to be accepting of a behavior analyst who is likeable and positive. This perspective speaks to the importance of rapport building not only with those who are outside of the field, but also those who receive ABA treatment. In addition to addressing the negative public perception of ABA, Foxx examined deficits in behavior analytic practitioners themselves that may have broader impacts on the field of ABA (Foxx, 1985). To illustrate his concept of a relationship-centric behavior analytic practitioner, Foxx coined the term “behavioral artist,” described as someone who is predisposed to be successful at the implementation of behavior analytic treatment as a result of possessing strong interpersonal skills such as a sense of humor, optimism, and attention to detail. In a contrast to 1 the behavioral artist, Foxx also described a “behavioral technologist” as someone who is scripted and procedural in their implementation (Foxx, 1985). Many consumers of ABA continue to describe a behavioral technologist when asked what they think of behavior analysts (Callahan et al., 2019). Foxx (1985) explored the concept of behavioral artistry (BA) by outlining seven common characteristics of a behavioral artist, claiming that identifying providers who demonstrate these traits may help us explain failures in behavior analytic programming. Callahan and colleagues (2019) listed and succinctly defined these traits: • Likes people: is able to establish rapport; demonstrates concern; wants to facilitate positive change; • Has “perceptive sensitivity”: pays careful attention to important indicators of client behavior that may be small, subtle, and gradual; • Doesn’t like to fail: sees difficult clients as a personal challenge to overcome, and as an opportunity for the client to succeed; • Has a sense of humor: recognizes and accepts that much in the educational and human services professions is bizarre, illogical, and humorous; • Looks “for the pony”: is optimistic and sees behavior change in a “glass half full” context; always believes programming will be successful; is less likely to burn out; • Is thick skinned: doesn’t take negative client actions towards herself or himself personally; maintains objectivity and positivity; and 2 • Is “self-actualized”: does whatever is necessary and appropriate to facilitate and produce positive behavior change; is not under audience control Callahan et al. (2019) attempted to measure behavioral artistry using the 16 Personality Factor Questionnaire 5th edition (16PF), which scored participants on 16 personality factors on a bipolar continuum. For each factor, one pole was considered congruent with BA and the opposite was considered incongruent. The study was conducted in three parts: parent validation, comparison of ABA students to students from other fields of study, and therapist observation. The parent survey asked parents of children with ASD to select personality traits that they would prefer in a therapist for their child by choosing between two descriptions of a 16PF factor, one corresponding with the BA congruent pole and the other corresponding with the BA non- congruent pole. The survey results found that parents showed a preference for the traits associated with BA over traits that are not associated with BA (Callahan et al., 2019). A second goal of the study was to examine levels of BA in students studying ABA compared to other human service areas of study such as occupational therapy, child development, and psychology, among others. Undergraduate and graduate students studying ABA and other human service areas of study were surveyed using the 16PF, and researchers found that students studying ABA demonstrated lower levels of behavioral artistry than their peers studying other areas of human services (Callahan et al., 2019). A third element of Callahan and colleagues’ (2019) study aimed to observe and measure the BA trait “likes people” during discrete trial training and natural environment teaching (NET) sessions with clients. Data were collected on therapists with high and low levels of BA, as determined by the 16PF. Researchers used a 10 second partial interval recording system to 3 measure the use of pleasant facial expression, positive tone of voice, sustained gaze towards child, and body positioning and orientation towards the child. Although results indicated that therapists from the high BA category demonstrated higher levels of pleasant facial expressions and positive tone of voice than the therapists from the low BA category, the only statistically significant difference between groups was for pleasant facial expressions (Callahan et al., 2019). However, researchers reported that qualitative differences were observed between the high BA and low BA groups, stating that participants in the high BA group appeared to enjoy their sessions more, and presented as more caring, attentive, and pleasant than participants from the low BA group (Callahan et al., 2019). The discrepancy between researchers’ qualitative observations and statistical results of operationally defined behavior might illustrate the complexity of defining and measuring interpersonal skills. While researchers have experienced difficulty capturing the measurable elements of BA, related research has been successful in the definition and measurement of interpersonal skills associated with rapport building, not only suggesting that measurement of “soft skills” is possible, but these skills can have positive impacts on client outcomes (Shillingsburg et al., 2018). These studies indicate that spending time building rapport with a client prior to beginning intensive skills training (also known as presession pairing) can increase socially significant behaviors and decrease interfering behaviors (Shillingsburg et al., 2018). Therefore, it is important for researchers to continue to evaluate and disseminate effective strategies to ensure therapists are equipped with skills to effectively engage in rapport building with clients. Given the importance of presession pairing, Lugo et al. (2017) aimed to operationally define the behaviors associated with presession pairing and create a behavioral skills training (BST) to teach correct implementation of pairing procedures. BST involves the explanation of a 4 skill, modeling, role play, and feedback (Clayton & Headley, 2019). Lugo et al. (2017) were able to effectively use BST to train staff to implement pairing sessions using specific behaviors outlined in a PSP protocol such as delivering behavior specific praise, imitating and narrating the client’s play, and remaining within a close proximity to the client. Results showed improved client and therapist rapport. However, it is unclear whether this rapport was explicitly created due to the skills outlined in the BST, or some underlying personality traits demonstrated through unmeasured behaviors of the therapist. Lugo et al. (2018) extended the study conducted by Lugo et al. (2017) and evaluated the effects of presession pairing and other antecedent interventions on client preference. In this study, a child was instructed to choose between therapists assigned to one of three conditions: a discrete trial instruction condition, a presession pairing (PSP) condition, and a free play condition. In the PSP and FP conditions, the child had access to the same highly preferred items. In the FP condition, the child was allowed to engage with these items independently, and in the PSP condition, the therapist engaged with these items with the child. The child’s responding indicated a preference for the PSP condition, which suggests that child selections were under the control of therapist social interactions (Lugo et al., 2018). The findings of Lugo et al. (2018) established that therapist behavior is influential to a child’s preferences for certain types of social interactions, which aligns with Foxx’s belief that a therapist’s expression of BA traits during therapy sessions can be instrumental to effective ABA treatment (Foxx, 1985). While both Lugo and colleagues (2017) and Lugo and colleagues (2018) incorporate elements of BA such as creativity, positive regard for clients, and close attention to client behavior, they fail to explicitly incorporate operationally defined BA traits and potential impacts on child behaviors. The combination of an effective method such as BST 5 (Lugo et al. 2017) and the concept of BA could support the acquisition of BA traits in behavior analysts. While BST is effective in teaching procedural skills (Clayton & Headley, 2019), BST alone is unlikely to increase BA due to the variety of ways that BA can be demonstrated across practitioners. For this reason, an additional, ongoing intervention may be required in order to facilitate true acquisition of BA. Video self-modeling is a mechanism that may be useful when providing continuing instruction on BA. When utilizing a video self-model, an individual is asked to watch a video of themselves engaging in a target behavior, and then imitate that model (Hawkins & Heflin, 2010). Video self-modeling has been found to be effective in increasing desired behaviors in adults, as demonstrated in Hawkins & Heflin’s 2010 study. This study found a functional relation between a video self-modeling intervention and an increase in behavior specific praise statements among secondary teachers (Hawkins & Heflin, 2010). This strategy could be ideal when training staff on BA due to the diverse ways that BA traits could be exemplified from individual to individual. The purpose of the present study is to examine the effects of behavioral skills training and video self-modeling on the use of the BA trait “likes people’ in sessions with clients, and explore the effects of explicit instruction of BA for staff on client behavior. 6 METHOD Participants and Setting Four dyads consisting of one adult and one child participated in the study. Adult participants were recruited by asking Board Certified Behavior Analysts® (BCBAs ®) to refer behavior technicians (BTs) who could benefit from focused training on BA. This survey broadly defined BA and each BA trait and required BCBAs® list BTs who they believed could benefit from training on each trait. All adult participants were first year students in an ABA Master’s program at a Midwestern university and were referred by BCBAs® for training on the trait “likes people.” In order to participate in the study, competency in natural environment teaching (NET) (Klintwall & Eikeseth, 2014) was required of the adult participants. At the onset of their employment, all adult participants received formal training in and routinely administered NET. To ensure that all adult participants were prepared to participate, role play sessions were conducted prior to beginning the study in which all adult participants were required to implement NET with at least 90% accuracy on a procedural integrity check. All child participants had a diagnosis of autism spectrum disorder and were receiving ABA services from an early intensive behavioral intervention clinic. Inclusion criteria for child participants included the following skills: vocal manding using at least one-word utterances, demonstration of functional play with toys that did not have a clear beginning and end (e.g., action figures, kitchen, etc.), and manding for an adult’s attention. Exclusion criteria included the use of an assistive communication device and consistently leaving a 10-min play session with an adult. Dyad one consisted of Julia, a 23 year old BT who had 36 months of experience providing ABA treatment, and Bobby, who was 56 months old and had been receiving ABA 7 services for 5 months. Dyad two consisted of Emma, a 25 year old BT who had 36 months of experience providing ABA services and Caleb, who was 46 months old and had been receiving ABA services for 14 months. Dyad 3 consisted of Austin, a 24 year old BT who had 6 months of experience providing ABA therapy and Tanner, who was 60 months old and had been receiving ABA services for 32 months. Dyad four consisted of Amy, a 23 year old BT who had 24 months of experience providing ABA therapy and Jane who was 62 months old and had been receiving ABA services for 30 months. Sessions took place in the clinic treatment rooms where therapy was concurrently being provided to between five and eight children on any given day. These treatment rooms contained individual treatment areas with tables and chairs for each child and BT pairing, as well as a large communal play area in the center of the room. This play area was designated by a large 2 m by 2 m rug and contained shelves with a variety of toys and books and was generally being used by one to three children at a time. Sessions were conducted in a separate play area located roughly 1.5 m from the communal play area, which was set up daily specifically for the purpose of this study. This configuration was utilized with the intention of controlling for potential variables such as interruptions by children or BTs who were not participating in the study or distractions such as additional toys or books, while still remaining within the participants’ natural environment. The session play area was delineated by a 1 meter by 1 meter rug where participants were instructed to stay to the best of their ability during sessions. Beyond this rug, there were no boundaries that designated the session play area as separate from the communal play area. 8 Materials The materials necessary for this study were a video camera, 1 meter by 1 meter rug to delineate the session play area, play sets, and data collection materials. Play sets were used for all participants for one week, and then switched out to prevent satiation. Each play set contained approximately 15 to 20 items and centered around a pretend play theme such as cooking, camping, or a birthday party. For in-session data collection, the adult participants were provided with a clip board, pen, data sheet (seen in Figure 3 in Appendix A), and vibrating timer. The timer was used by the adult participants to ensure that they placed demands at the correct pace throughout sessions. The adult participant’s data sheet was used to ensure that an equal number of trials were run in each session. This data sheet listed verbal operants (imitation, echoic, tact, intraverbal, and listener responding) and contained space to make tally marks every time a trial from each category was run. An identical data sheet was used by the researcher to ensure that the correct number of trials were presented. Sessions were recorded using a video camera, and later uploaded and viewed for data collection purposes. During data collection, researchers utilized two separate data sheets: one for the adult participant’s dependent variables (seen in Figure 4 in Appendix A) and another for the child participant’s dependent variables (seen in Figure 5 in Appendix A). Dependent Measures Adult Participant Dependent Variables The BA trait “likes people” was assessed by scoring instances of positivity emitted by the adult participant towards the child participant. Instances of positivity were considered an occurrence of affectionate touch, vocal affirmations, or positive body language. These behaviors 9 differed from those in Callahan and colleague’s (2019) study, which collected data on participants’ demonstrations of positive facial expressions, pleasant tone of voice, sustained gaze at client, and body proximity and orientation towards client. Results from Callahan and colleagues’ study suggested that demonstrations of these behaviors did not differ significantly between high BA and low BA groups, which lead experimenters of the present study to select different behaviors that aligned with the trait “likes people.” The only exception to this decision was for the measure “positive facial expressions.” However, this measure was ultimately excluded from the present study due to a requirement that adult participants wear facial masks as part of the clinic’s COVID-19 mitigation protocol. Affectionate touch was defined as the adult participant touching the child participant in a playful or comforting manner. Vocal affirmations were defined as “the adult participant making a vocal statement that is positive, encouraging, or celebratory in nature. Positive body language was defined as the adult participant making a change in their body orientation that is celebratory or excited in nature in reaction to a response made by the child participant. Instances of positivity were recorded using a five second partial interval. For examples and non-examples of these definitions, see Table 1. Instances of positivity were recorded using a 5-s partial interval time sampling method. Researchers selected this interval length over a 10-s interval, which was utilized in Callahan and colleagues’ study (2019), with the belief that a 5-s interval would provide a more conservative estimate of BA. The adult participant was only required to engage in one of the three components of instances of positivity to be considered having engaged in an instance of positivity for each interval. Data were collected for a 5 minute session. 10 Table 1 Adult participant dependent variables defined Child Participant Dependent Variables The dependent measures for child behavior consisted of the following behaviors: remaining close to the adult participant, negative vocalizations, and responses to social initiations made by the adult participant. Remaining close to the adult participant was defined as the child participant’s body being positioned within 0.6 meters of the adult participant. This behavior was measured using a 5 second whole interval recording system and data were collected for the first thirty intervals of each session. Negative vocalizations were defined as a vocalization at a volume above conversational level accompanied by a negative facial expression such a frown or grimace or vocal refusal when presented with a task demand and were measured using a 10 second partial interval recording system. Data were collected for the entire 5 minute session. 11 Finally, responding to the adult participant’s social initiations was defined as reacting to a statement, question, or demand placed by the adult participant with a vocal response, gesture, or change in body orientation. This can include the participant shifting their gaze towards an item that the researcher is attempting to show them. Responding to the adult participants social initiations was recorded using percentage of opportunities on the first 15 opportunities during a session. For an abridged definition of each child participant dependent measure, as well as examples and non-examples, see Table 2. Table 2 Child participant dependent variables defined Interobserver Agreement Interobserver agreement (IOA) was calculated for at least 33% of sessions across all conditions. IOA data were collected by a research assistant who was trained on data collection specific to this study. This training consisted of a slideshow presentation that defined each 12 dependent variable in detail, provided examples and non-examples of those behaviors, and explained data collection procedures for each behavior. Mean agreement for Julia’s instances of positivity towards the child participant was 84% (range, 82-88%) in baseline, 86% (range, 82-90%) in the post BST phase, and 88% (range, 82- 92%) in the post BST with VSM phase. Mean agreement for Bobby’s remaining close to the adult participant was 99% (range, 97-100%) in baseline, 100% in the post BST phase, and 100% in the post BST with VSM phase. Mean agreement for his negative vocalizations was 98% (range, 93-100%) in baseline, 100% in the post BST phase, and 93% (range, 83-100%) in the post BST with VSM phase. Finally, mean agreement for his responding to social initiations was 83% (range, 71-90%) in baseline, 83% in the post BST phase, and 76% (range, 70-90%) in the post BST with VSM phase. Mean agreement for Emma’s instances of positivity towards the child participant was 86% (range, 85-87%) in baseline and 83% (range, 73-90%) in the post BST with VSM phase. Mean agreement for Caleb’s remaining close to the adult participant was 94% (range, 90-100%) in baseline and 98% (range, 93-100%) in the post BST with VSM phase. Mean agreement for his negative vocalizations was 99% (range, 97-100%) in baseline and 100% in the post BST with VSM phase. Finally, mean agreement for his responding to social initiations was 80% (range, 67-100%) in baseline and 97% (range, 86-100%) in the post BST with VSM phase. Mean agreement for Austin’s instances of positivity towards the child participant was 93% (range, 88-97%) in baseline and 95% (range, 93-97%) in the post BST with VSM phase. Mean agreement for Tanner’s remaining close to the adult participant was 99% (range, 97- 100%) in baseline and 100% in the post BST with VSM phase. Mean agreement for his negative vocalizations was 100% in baseline and 90% in the post BST with VSM phase. Finally, mean 13 agreement for his responding to social initiations was 88% (range, 75-100%) in baseline and 100% in the post BST with VSM phase. Mean agreement for Amy’s instances of positivity towards the child participant was 90% (range, 83-93%) in baseline and 79% (range, 78-80%) in the post BST with VSM phase. Mean agreement for Jane’s remaining close to the adult participant was 100% in baseline and 100% in the post BST with VSM phase. Mean agreement for her negative vocalizations was 96% (range, 87-100%) in baseline and 100% in the post BST with VSM phase. Finally, mean agreement for her responding to social initiations was 83% (range, 72-90%) in baseline and 83% (range, 75- 92%) in the post BST with VSM phase. Experimental Design A multiple probe across participants design was used to evaluate effect of training on BA on adult and child behavior. (Ledford & Gast, 2018). This was chosen due to the belief that BA traits are non-reversable and would most likely not increase substantially in a baseline condition. In this design, overlapping baseline sessions are conducted to establish pre-intervention levels of responding (Ledford & Gast, 2018). In this study, each adult participant participated in a minimum of five baseline sessions. Following the baseline phase, dyad 1 moved into intervention, while all other dyads remained in the baseline phase. Daily sessions continued for dyad 1, and probe sessions were conducted every 3-5 session for the remaining three dyads. Once a stable trend was established in Julia’s data, dyad 2 was moved into the intervention phase, while the other two dyads remained in baseline with regular probe sessions. The same process continued until all dyads were moved into the intervention phase. Data collection concluded when a stable trend was observed in Amy’s data. 14 Procedures Baseline Adult participants were instructed to run a 5-minute long NET teaching session with the child participant using a provided play set. In NET instruction, the adult follows the child’s lead and interweaves teaching opportunities into play. During these sessions, adult participants were required to provide between eight and twelve learning opportunities to maintain consistent difficulty for the child participant across sessions. An example of a learning opportunity may be saying “do what I do” while pretending to take a bite of cake or asking a child “what is it?” when pointing to a baby bottle. To help the adult participant stay on pace during the sessions, a vibrating timer was provided that vibrated every 30-seconds to prompt them to provide a learning opportunity to the child participant. The experimenter observed all sessions and kept a tally of the number of learning opportunities provided in order to ensure accurate implementation of sessions. If an adult participant were to provide more than 12 or less than 8 learning opportunities during a session, that session would not be used in the study. However, this was not necessary, as all adult participants stayed within the correct number of learning opportunities for all sessions. Sessions were video recorded and adult and child behavior data were later collected from the video recordings. BST Intervention consisted of a BST presentation that utilized a VSM for the modeling portion of the presentation that was conducted with each adult participant. The BST began with a general overview of BA, including a description of why it’s important and relevant research pertaining to the concept. The seven BA traits were listed and briefly described, before the experimenter went into greater detail on the trait “likes people.” The trait was defined in detail, 15 and then the adult participant was provided with session behaviors associated with the trait such as vocal encouragement, excited gestures, and physical touch. Finally, detailed examples and non-examples were reviewed. Following the informational overview, the researcher explained to the adult participant that they were about to see two videos of themselves demonstrating the BA trait “likes people.” While watching these videos, the researcher labeled for the adult participant how their behavior was embodying “likes people.” After viewing the VSM, participants practiced using the trait “likes people” in role play sessions with the researcher. The adult participant was required to increase their levels of behavioral artistry from their baseline average by 75% in order to move past the role play phase. Finally, the researcher gave either corrective or positive behavior specific feedback to the adult participant. Post-BST For dyad 1, there was a brief post-BST phase. Following the BST, sessions were resumed with the child participant. These sessions were identical to sessions conducted during the baseline phase. Data for the first adult participant, Julia, did not indicate significant behavior change from the baseline phase to the post-BST phase, which prompted the experimenter to add in an additional VSM element. Behavior changed slightly for Julia once VSM was introduced, so the experimenter made the decision to provide repeated VSM sessions immediately following BST for remaining participants. Post-BST with VSM The post BST with VSM phase differed from the post-BST phase only in that a brief VSM was provided to the participant before the start of each session with the child participant. All other elements of the session remained the same. The VSM was pulled from the adult 16 participant’s session directly preceding the present session. These VSMs were typically between 10 and 15 seconds long and demonstrated the adult participant engaging in the BA trait “likes people.” Prior to beginning sessions in this phase, the experimenter showed the video clip to the adult participant and labeled ways in which the adult participant was engaging in BA. The adult participant was then directed to begin their session with the child participant. Procedural Fidelity BST Procedural fidelity data was collected for 50% of BST interventions on accuracy of implementation. Data were collected by a research assistant using a checklist developed by the experimenter, which can be found in Figure 6 in Appendix B. Mean procedural fidelity was 100% for the BST intervention. VSM Procedural fidelity data was collected for 33% of VSM sessions across participants. Data were collected by a research assistant using a checklist developed by the experimenter, which can be found in Figure 7 in Appendix B. Mean procedural fidelity was 97% (range, 83-100%) for the VSM intervention. 17 RESULTS Adult Participants Adult participants’ use of the BA trait “likes people” is depicted in Figure 1. In the event of a child participant being absent from treatment, another client from the clinic would fill in during the corresponding adult participant’s session. These sessions are denoted on Figure 1 with an asterisk. In the baseline phase, Julia demonstrated the BA trait “likes people” during an average of 24% (range, 18-34%) of 5-s intervals. Following the BST, this percentage increased to an average of 35% (range, 30-43%) of 5-s intervals. With the introduction of the additional VSM element, Julia demonstrated the BA trait “likes people” during an average of 32% (range, 22-40%) of 5-s intervals. Emma’s baseline average of intervals with the BA trait “likes people” was 28% (range, 20-37%). During the BST with VSM phase, her average increased to 41% (range, 30-53%). Austin demonstrated the BA trait “likes people” during an average of 12% (range, 2-20%) of 5-s intervals in baseline. This average increased in the post BST with VSM phase to 30% (range, 17- 45%) of 5-s intervals. Amy’s baseline average of intervals with the BA trait “likes people” was 20% (range, 8-42%). During the BST with VSM phase, her average increased to 40% (range, 30- 45%). 18 Figure 1 Adult participants’ percentage of 5-s intervals with instances of positivity towards child participant. Asterisks indicate that a substitute child participated in the session due to the usual child participant being absent from treatment 19 Child Participants Figure 2 depicts data collected on the child participants remaining close to the adult participant, negative vocalizations, and responding to social initiations from the adult participant. In baseline, Bobby remained close to Julia for an average of 97% (range, 87-100%) of 5-s intervals. This average increased in both the post-BST and post-BST with VSM phases to 100% (range, 93-100%). His average percentage of 10-s intervals with negative vocalizations were 8% (range, 3-17%), 3% (range, 0-3%), and 8% (range, 0-23%) in baseline, post-BST, and post-BST with VSM, respectively. Bobby responded to Julia’s social initiations during an average of 59% (range, 40-73%) of opportunities during the baseline phase. This average increased to 73% (range, 60-80%) in the post-BST phase and 70% (range, 53-80%) in the post BST with VSM phase. Caleb’s average percentage of 5-s intervals where he remained close to Emma in baseline was 91% (range, 70-100%). In the post BST with VSM phase, his average increased to 95% (range, 50-100%). His average percentage of 10-s intervals with negative vocalizations was 1% (range, 0-3%) in baseline and 0% (range, 0-3%) in the post-BST with VSM phase. His responding to social initiations made a slight increase from baseline, where his average was 57% (range, 40-77%) to post-BST with VSM where his average was 65% (range, 33-80%). In baseline, Tanner remained close to Austin for an average of 96% (range, 77-100%) of 5-s intervals. In the post BST with VSM phase, this average increased slightly to 100%. His average percentage of 10-s intervals with negative vocalizations was 6% (range, 0-27%) in baseline and 5% (range, 0-10%) in the post BST with VSM phase. Finally, he responded to an average of 69% (range, 47-87%) of Austin’s social initiations in the baseline phase. This average decreased to 50% (range, 20-67%) in the post BST with VSM phase. 20 Jane’s average percentage of 5-s intervals in which she remained close to Amy was 98% (range, 90-100%) in baseline and 100% in the post BST with VSM phase. Her average percentage of 10-s intervals with negative vocalizations remained stable between baseline at 3% (range, 0-10%) and the post-BST with VSM phase. Jane’s responded to Amy’s social initiations during an average of 69% (range, 40-93%) of opportunities in baseline, which decreased to 63% (range, 60-69%) in the post BST with VSM phase. 21 Figure 2 Child Participant Dependent Measures 22 DISCUSSION The present study sought to examine the effects of BST with video self-modeling on BA of adult staff and subsequent effects on client behavior. During the first session following the BST intervention, all adult participants exhibited an increase in the percentage of intervals in which they demonstrated BA. This increase, however, did not sustain for Julia and gradually lowered back toward baseline levels for Emma. Austin and Amy demonstrated a higher increase in BA, which persisted throughout their exposure to the intervention. Although the magnitude of results is relatively small, it is promising that adult participants demonstrated an increase in behaviors associated with BA traits originally identified by Foxx et al. (1985) and later defined by Callahan et al. (2019). An increase in behaviors associated with BA suggests that those behaviors can be taught to BTs and are not something that an individual either possesses or does not. Future research could continue to examine strategies that increase BA to even higher levels among BTs. Results for the child participants’ behavior did not show significant behavior change from baseline to post BST with VSM. At this time, it’s inconclusive whether or not staff training on BA has an effect on client behavior for multiple reasons. First, it may take more time for the effects of the adult participants’ behavior change to bring about observable behavior change for children. Second, the behaviors measured may not have been conducive to assessing the effects of the staff training on BA. For example, Jane remained close to Amy during 98% of 10-s intervals in baseline and 100% of 10-s intervals in the post BST with VSM phase. Her high score in baseline did not allow for much behavior change after intervention. Perhaps more meaningful results could have been attained if a child participant who was known to have engaged in high levels of elopement from sessions was selected to participate. 23 During the post BST with VSM phase, Tanner exhibited some noteworthy behaviors outside of he and Austin’s recorded sessions. On three occasions during the post BST with VSM phase, Tanner refused to participate in sessions for this study. When prompted to join the experimenter and Austin in the session play area, Tanner refused to leave his individual instructional area, screamed, and yelled statements such as “No!” “I want to stay here!” and “I want to play with my friend!” The statement “I want to play with my friend” is in reference to the BT who was providing treatment to him on days when refusal occurred, which suggests that there may be a quality to her treatment that Tanner prefers that was lacking in Austin’s sessions. Tanner’s refusal further illustrates the importance of qualitative differences between BTs. Based on the results of the present study, BST may not be the most effective method utilize when teaching BA. Despite BST’s effectiveness for teaching other elements of behavior analytic treatment (Clayton & Headley, 2019; Lugo et al., 2017), it may not be potent enough in isolation for teaching the interpersonal skills associated with BA. Previous studies have shown BST to be effective in teaching skills such as presession pairing (Lugo et al. 2017) and discrete trial training (Clayton & Headley, 2019), which are both very procedural and succinct. Conversely, BA is a rather abstract concept that does not have manualized instructions for implementation. Due to the fluent nature of BA, some individualization of instruction is required, which cannot easily be targeted using BST. Although the use of VSM allows for individualization, the addition of VSM was also insufficient to create and maintain a sizeable behavior change among adult participants. This conclusion is in line with previous research such as Hawkins and Heflin’s (2010) study that aimed to increase the use of behavior specific praise statements by secondary teachers. As in Hawkins and Heflin, although VSM did contribute to behavior change, the participants reported 24 not enjoying watching themselves on video. Participants in the present investigation made bereding comments regarding their own appearance or sound of their voice in the VSM videos. This could indicate that VSM may be aversive to participants rather than a useful teaching tool. Given the potential for discomfort when viewing VSMs, a traditional video model may be a better tool for teaching BA. These video models would feature another person demonstrating BA in a session with a client and would allow for a BT to observe examples of BA without the added strain of watching themselves on video. Additional strategies such as progress monitoring or coaching may also be effective in supporting the acquisition of BA traits. These intervention components could be utilized on their own or in conjunction with another intervention such as BST. Providing coaching to BTs on their use of BA traits allows for in the moment support and might lead to faster progress overall. Similarly, progress monitoring would allow BTs to view their own data, which could incentivize progress towards a pre-determined goal. When using either coaching or progress monitoring, feedback would be an integral part of the instructional process. Each of these strategies allow for individualization, which is an important element of BA. Challenges faced in the present study echoed those of Callahan et al.’s (2019) study. More specifically, it was possible to watch two different BTs and subjectively notice the presence or absence of BA traits but was very difficult to operationally define and measure these traits (Callahan et al., 2019). Although participants increased the frequency at which they made positive statements to the child participant, used physical touch playfully, and gestured to celebrate child success during sessions, it’s unclear whether these behaviors truly encapsulate the trait “likes people.” Additional elements such as tone of voice and facial affect might be useful 25 additions to the definition of likes people but would pose their own challenges when it comes to definition and measurement. An additional measurement concern is the subjective nature of the behaviors associated with BA. This subjectivity is reflected in the IOA scores for this study, specifically for the adult participant’s dependent variables. Although observers agreed on many instances of positivity during sessions, it’s difficult to perfectly define and teach a behavior such as “positive vocal statements” without BTs sounding rigid or ingenuine. Doing so would be counter the purpose of the present study, as well as the conceptual framework of BA (Foxx, 1985; Callahan et al., 2019). One consideration when evaluating levels of BA during sessions with clients is whether or not there is an optimal level of BA for practitioners to strive towards. In the present study, the highest percentage of 5-s intervals during a session in which an adult participant demonstrated behaviors associated with the BA trait “likes people” was 53%. During the remaining 47% of intervals, the adult participant was engaging in important session behaviors such as providing learning opportunities and prompts, managing materials, and collecting data. It’s arguable that these elements of a session are just as important as BA, which indicates that demonstrating BA during 100% of a session is not only unrealistic, but probably unnecessary. The importance of allowing for enough time to engage in session behaviors outside of BA leads to a limitation of the present study. Perhaps 5-s intervals were too short given the behaviors that data were being collected on. In this case, it was not realistic to assume that adult participants would be engaging in behaviors associated with the trait “likes people” every 5 seconds. An interval that is longer than 5-s may produce more accurate data that is reflective of 26 how often behaviors related to BA should occur during sessions. In future research, a 10-s, 15-s, or even 30-s interval may be more appropriate. A second limitation of this study is that it did not test for generalization or maintenance of the adult participants’ use of the BA trait “likes people.” Although all adult participants demonstrated positive behavior change from baseline to post BST with VSM, it’s unclear whether or not the current level of performance would maintain without the support of the VSM or to other settings within the treatment day such as discrete trial training or group instruction. If time permitted, this would have been a valuable addition to the study. There is a great opportunity for further research surrounding the topic of BA. Existing research has focused only on the trait “likes people,” which is only a fraction of BA. Future studies might examine traits such as “is thick skinned,” “has a sense of humor,” “has perceptive sensitivity” and “is self-actualized” within the context of sessions with clients, while other studies may consider how the traits “looks for the pony” and “doesn’t like to fail” are exhibited during conversations about client progress with BCBAs®. For each trait, there is potential for a unique behavioral definition, measurement system, and training intervention, all of which would aid development and dissemination of BA. Furthermore, future studies might take a longitudinal approach to studying BA on both the adult and child level. For adults, it would be valuable to determine just how long it takes for individuals to develop high levels of BA. The current study was conducted over the course of three months. Future researchers may assess levels of BA across 6 months, a year, or even multiple years to assess for acquisition and maintenance of BA traits. On the client level, it would be meaningful to assess if the levels of BA demonstrated by a child’s BT has an effect on that child’s clinical outcomes. 27 A third potential study relating to BA might focus on incorporating BA into the interview process when hiring behavior analysts. On the BT and BCBA® level, it could be valuable to incorporate screening questions that would indicate if a potential employee possesses some or all of the 7 BA traits. Using BA data collected during interviews, researchers could assess correlations between initial levels of BA and employee tenure, rate of acquisition of job related skills, and client outcomes. BA was initially introduced in the 1980s as a result of Foxx’s observations of core deficits in the repertoires of behavior analysts’ interpersonal skills (Foxx, 1985). Despite advances in other areas of the field of ABA, these deficits are still being observed and reported as problems for behavior service providers today (Callahan et al., 2019). As stated by previous researchers, BA has the potential to improve the quality of treatment that clients receive from behavior analysts (Callahan et al., 2019; Foxx, 1985). If this claim is true, it’s imperative that measurement strategies are identified for BA, as doing so could not only influence client outcomes, but be essential to the survival of the field in the broader human services landscape. 28 APPENDICES 29 APPENDIX A Data Collection Materials 30 Figure 3 In-session data collection sheet used by adult participants and experimenter to maintain consistency across sessions 31 Figure 4 Data collection sheet for adult participant dependent variables 32 Figure 5 Data collection sheet used for child participant dependent variables 33 APPENDIX B Procedural Fidelity Checklists 34 Figure 6 Procedural fidelity checklist used to assess accuracy of implementation of the BST intervention 35 Figure 7 Procedural fidelity checklist used to assess accuracy of implementation of the VSM intervention 36 REFERENCES 37 REFERENCES Callahan, K., Foxx, R. M., Swierczynski, A., Aerts, X., Mehta, S., McComb, M.-E., Nichols, S. M., Segal, G., Donald, A., & Sharma, R. (2019). Behavioral artistry: examining the relationship between the interpersonal skills and effective practice repertoires of applied behavior analysis practitioners. Journal of Autism and Developmental Disorders, 49, 3557–3570. https://doi.org/10.1007/s10803-019-04082-1 Clayton, M., & Headley, A. (2019). The use of behavioral skills training to improve staff performance of discrete trial training. Behavioral Interventions, 34, 136–143. https://doi.org/10.1002/bin.1656 Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2002). Intensive behavioral treatment at school for 4- to 7-year-old children with autism. A 1-year comparison controlled study. Behavior Modification, 26, 49 – 68. http://dx.doi.org/10.1177/0145445502026001004 Foxx, R. M. (1985). The Jack Tizzard Memorial Lecture Decreasing Behaviours: Clinical, Ethical, and Environmental Issues. Australia and New Zealand Journal of Developmental Disabilities, 11(4), 189-199. Foxx, R. M. (1996). Translating the covenant: The behavior analyst as ambassador and translator. The Behavior Analyst, 19(2), 147-161. doi:10.1007/bf03393162 Foxx, R. M. (2008). Applied behavior analysis treatment of autism: The state of the art. Child and Adolescent Psychiatric Clinics of North America, 17(4), 821-834. doi:10.1016/j.chc.2008.06.007 Hawkins, S. M., & Heflin, L. J. (2010). Increasing secondary teachers’ behavior-specific prai using a video self-modeling and visual performance feedback intervention. Journal of Positive Behavior Interventions, 13(2), 97–108. https://doi.org/10.1177/1098300709358110 Klintwall, L., & Eikeseth, S. (2014). Early and intensive behavioral intervention (EIBI) in autism. Comprehensive guide to autism, 117-137. Ledford, J. R. & Gast, D. L. (2018). Single case research methodology: Applications in special education and behavioral sciences. New York: Routledge, Taylor & Francis Group Lugo, A. M., King, M. L., Lamphere, J. C., & McArdle, P. E. (2017). Developing procedures to improve Therapist–Child rapport in early intervention. Behavior Analysis in Practice, 10(4), 395–401. https://doi.org/10.1007/s40617-016-0165-5 Lugo, A. M., McArdle, P. E., King, M. L., Lamphere, J. C., Peck, J. A., & Beck, H. J. (2018). Effects of presession pairing on preference for therapeutic conditions and challenging 38 behavior. Behavior Analysis in Practice, 12(1), 188–193. https://doi.org/10.1007/s40617- 018-0268-2 Shillingsburg, M. A., Hansen, B., & Wright, M. (2018). Rapport building and instructional fading prior to discrete trial instruction: moving from child-led play to intensive teaching. Behavior Modification, 43(2), 288–306. https://doi.org/10.1177/0145445517751436 39