THE USE OF EQUIVALENCE-BASED INSTRUCTION TO TEACH GRADUATE STUDENTS BEHAVIOR ANALYTIC TERMINOLOGY By Suzanne Hemwall A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Applied Behavior Analysis—Master of Arts 2021 ABSTRACT THE USE OF EQUIVALENCE-BASED INSTRUCTION TO TEACH GRADUATE STUDENTS BEHAVIOR ANALYTIC TERMINOLOGY By Suzanne Hemwall Individuals pursuing their Board Certified Behavior Analyst® (BCBA®) certification are expected to have an understanding of behavior analytic terminology. In the past 10 years, demand for BCBAs® has grown significantly (Deochand & Fuqua, 2016; Behavior Analyst Certification Board, 2021). One way to teach individuals terminology is via equivalence-based instruction (EBI) and Match-to-Sample (MTS). The present study examined the use of EBI to teach six graduate students behavior analytic terminology. Using a multiple probe across behaviors design replicated across participants, participants were taught 30 different behavior analytic terms, definitions, and examples via a MTS teaching procedure. Participants were then assessed on their performance of untaught relations in a MTS, or selection-based, format as well as a written intraverbal format. While emergent intraverbal responding was limited, all participants demonstrated emergent selection-based responding. The results of this study can inform instructors preparing individuals seeking their BCBA® certification. Keywords: Equivalence-based instruction, Match-to-Sample, conditional discrimination, selection-based responding, topography-based responding TABLE OF CONTENTS LIST OF TABLES.......................................................................................................................... v LIST OF FIGURES....................................................................................................................... vi Introduction .....................................................................................................................................1 Method ........................................................................................................................................... 6 Participants and Setting ............................................................................................................. 6 Materials and Experimental Stimuli ........................………..................................................... 6 Response Measurement and Dependent Variables ................................................................... 7 Procedural Fidelity and Interobserver Agreement ............................................................. 8 Experimental Design ................................................................................................................. 9 Procedures ............................................................................................................................... 10 Pretest .......…………………………………………………………………………….... 10 Intraverbal Probes ...............……………………………………………………………. 10 MTS Teaching Sessions ...........……………………………………………........…….... 12 Remedial Training (All Participants) .......................…………………………………… 13 Posttest .......…………………………………………………………………………….. 14 Social Validity ..........……………………………………………………………………….. 14 Results ......…………………………………………....………………………………………… 15 Discussion .................................................................................................................................... 21 Limitations .............................................................................................................................. 25 Conclusion .............................................................................................................................. 26 APPENDIX .................................................................................................................................. 27 REFERENCES ............................................................................................................................ 48 iv LIST OF TABLES Table 1 List of Terms and Definitions for Stimulus Set 1 (Verbal Behavior) ............................. 28 Table 2 List of Terms and Examples for Stimulus Set 1 (Verbal Behavior) ............................... 29 Table 3 List of Terms and Definitions for Stimulus Set 2 (Generalization and Maintenance) ... 30 Table 4 List of Terms and Examples for Stimulus Set 2 (Generalization and Maintenance) ..... 31 Table 5 List of Terms and Definitions for Stimulus Set 3 (Equivalence-Based Instruction) ...... 34 Table 6 List of Terms and Examples for Stimulus Set 3 (Equivalence-Based Instruction) ........ 35 Table 7 Results of Social Validity Survey by Participant ........................................................... 38 Table 8 Pretest and Posttest Data by Relation and by Participant for Stimulus Set 1 ................. 39 Table 9 Pretest and Posttest Data by Relation and by Participant for Stimulus Set 2................. 40 . Table 10 Pretest and Posttest Data by Relation and by Participant for Stimulus Set 3 ............... 41 v LIST OF FIGURES Figure 1 Graphical Display of Amanda’s Probe, MTS, and Remedial Training Data ................ 42 Figure 2 Graphical Display of Talia’s Probe, MTS, and Remedial Training Data ..................... 43 Figure 3 Graphical Display of Gabby’s Probe, MTS, and Remedial Training Data ................... 44 Figure 4 Graphical Display of Elizabeth’s Probe, MTS, and Remedial Training Data .............. 45 Figure 5 Graphical Display of Brittany’s Probe, MTS, and Remedial Training Data ................ 46 Figure 6 Graphical Display of Katherine’s Probe, MTS, and Remedial Training Data .............. 47 vi Introduction Applied Behavior Analysis (ABA) is the science that studies and aims to teach or improve socially significant behavior using principles of behavior (Baer et al., 1968). ABA has historically been used in a variety of settings such as the field of education, clinical psychology, behavioral medicine, organizational behavior management, and with a wide range of consumers (see Slocum et al., 2014). As an example, ABA-based interventions are cited as effective treatment for individuals with autism and developmental disabilities (Hyman, 2019). A common career within the field of ABA is to work as a Board Certified Behavior Analyst (BCBA®). Demand for BCBAs® has grown significantly over the past decade (Deochand & Fuqua, 2016; Behavior Analyst Certification Board, 2021). In order to become a BCBA®, individuals are required to pass a certification exam via the Behavior Analyst Certification Board (BACB®). Critical to passing the exam is the understanding of behavior analytic concepts, principles, and terminology. The exam consists of 160 multiple choice questions assessing understanding of the BCBA®/BCaBA® Task List 5th ed. that is expected from an entry-level behavior analyst (BACB, 2020). The Task List is broken down into two sections. The first section, Foundations, includes fundamental skills, knowledge, and principles. The second section, Applications, includes more applied skills. Some of the questions on the exam involve the presentation of an example or scenario in which the test taker is to select the term or concept that best corresponds to it (BACB, 2020). The question and the answer choices on the exam may include behavior analytic terminology that test takers should understand in order to comprehend the question and narrow down the correct answer. Not only is it important for individuals pursuing their BCBA® certification to understand these terms and concepts to perform well on the exam, but they also need to have a foundational understanding 1 of concepts and principles in order to practice and apply them to the clients they serve (BACB 2020). One approach to teaching the relation between words and their definitions and examples is through equivalence-based instruction (EBI). EBI uses principles of stimulus equivalence to teach that two or more dissimilar stimuli are related, the same, or belong to the same stimulus class (Critchfield et al., 2018). In order to use EBI, stimuli are divided into different classes or categories and certain relations between stimuli are taught while others are likely to emerge (Fienup et al., 2011). For example, when teaching vocabulary words, the categories may include the vocabulary terms, their definitions, and examples of each word. Using conditional discrimination, individuals can form relations between the stimuli. A typical approach within EBI is called Matching-To-Sample (MTS) (Michael et al., 2011). MTS involves the presentation of a sample stimulus (e.g., a definition of a word), or target stimulus, and the selection of a related comparison stimulus in an array of two or more stimuli (e.g., a word that corresponds to that definition, and at least one other word that does not). For example, when presented with a sample stimulus with the written words, “an open, usually cylinder-shaped drinking vessel,” and the comparison stimuli “cup,” “table,” and “car,” the learner would select the comparison stimulus, “cup.” This type of responding is referred to as selection-based responding (Polson and Parsons, 2000). A defining feature of EBI is the emergence of new relations, or derived relations, following the teaching of one or more relations between stimuli. In the “cup” example above, one derived relation would be the selection of the definition of a cup when presented with the word “cup” without any prior training; this derived relation is referred to as symmetry (Lovett et al., 2011). Selecting the written word “cup” when presented with “cup,” “table,” and “car,” 2 describes the concept of reflexivity (Cooper et al., 2020). Transitivity refers to relations that emerge as a result of two prior trained relations (Cooper et al., 2020). For example, one may teach that the written word, “cup” is related to the definition of a cup and a picture of a cup. The relation between the definition of a cup and a picture of a cup illustrates transitivity. Carr and Felce (2009) explain the emergence of novel stimulus-stimulus relations as a link between the individual’s expressive and receptive language. The derived stimulus relations, or emergent learning, that occurs as a result of EBI allows for “free learning.” This “free learning” provides an efficient way for learners to acquire new understanding of information and skills without requiring direct teaching of each individual skill (Critchfield et al.,2018). Teaching only a few relations that results in emergent relational understanding has practical benefits and suggests EBI is an efficient teaching method that can promote educational opportunities across various content areas (Stewart et al., 2013). Previous research in stimulus equivalence and EBI has evaluated outcomes in adults and college students in the instructional subjects of neuroanatomy (Pytte & Fienup, 2012), visual analysis of graphs (Blair et al., 2019), piano skills (Griffith et al.,2018), and brain-behavior relations (Fienup et al., 2010). Albright and colleagues (2016) used computer-based MTS teaching to evaluate the emergence of selection-based and topography-based responding in ABA graduate students using a pretest-train-posttest design. Topography-based responding involves teaching a specific topography (e.g., intraverbal) based on a specific controlling variable (Polson & Parsons, 2000). Participants were taught to interpret operant functions of behavior (A) based on a description (B), graph (C), and clinical vignette (D). The topography-based oral pretest assessed BA, CA, and DA relations. A multiple-choice pretest assessed all relations and a computer pretest assessed only the relations to be trained. Participants then completed training in AB, AC, and AD 3 relations using MTS and completed tests of symmetry and transitivity. Last, participants completed posttests that were identical to the pretests. All participants showed improvements on their posttest scores compared to their pretest scored and demonstrated maintenance 2 weeks following completion of the posttest. This study demonstrated the effectiveness of selection- based training on emergent topography-based responding, but only assessed four topics relevant to ABA (four functions of behavior) all taught within a single day. Similarly, Walker and Rehfeldt (2012) used stimulus equivalence to teach distance learning graduate students single subject designs using a pretest-train-posttest-maintenance design. The study consisted of topography-based tact and intraverbal pretests assessing whether participants could name the correct design when presented with the definition (BA), graph (CA), and clinical vignette (DA). Participants then moved on to selection-based intraverbal instruction in which they were taught AB, AC, and BD relations using multiple choice questions. Once participants reached mastery criterion during instruction, they moved into a topography-based tact and intraverbal posttest phase that was identical to the pretest phase. Last, participants completed a test for maintenance 16-weeks following the beginning of the study to evaluate the stability of the emergent topography-based intraverbal and tact skills over time. Participant performance varied, however all 11 participants demonstrated the emergence of CA tact relations following AC intraverbal instruction and all participants’ scores improved from pretest to posttest. One limitation of this study, however, is that during training, participants were only exposed to one example and they did not demonstrate generalization to novel clinical vignettes. Many studies have evaluated the emergence of topography-based responding following selection-based instruction—some even with graduate students in ABA. However, there is a gap in the literature that yields the importance for more targeted research on the use of EBI to 4 prepare individuals pursuing their BCBA® certification for the BACB® exam and careers in the field of ABA by improving their understanding of key behavior analytic concepts. The narrow topic studied in the Albright et al. (2016) study is a limitation because individuals who take the BACB® exam are required to understand a much larger amount of content. While EBI may be effective when teaching a limited number of concepts, future research could explore the use of EBI to teach ABA graduate students a broader range of information. The lack of stimulus generalization in the Walker and Rehfeldt (2012) also informs future research to use multiple exemplars during training and assessment for generalization—especially when preparing individuals to take the BACB® exam as they will be exposed to many novel examples. Therefore, the present study assessed the effectiveness of EBI on the acquisition of knowledge of 30 behavior analytic terminology and examples in six Applied Behavior Analysis graduate students. The specific questions the researchers asked were: a. How effective is EBI for teaching ABA graduate students behavior analytic terminology? b. To what extent does EBI result in emergent written topography-based responses? c. To what extent does EBI result in emergent MTS performance with behavior analytic terminology, definitions, and examples? 5 Participants and Setting Method Six students in the Master of Arts in Applied Behavior Analysis program at a university in the Midwest participated in the current study. Participants ages ranged from 22-years old to 26-years old and all spoke English as their first language. Each participant’s prior experience with ABA differed. Talia was a 23-year-old White female with one and a half years of experience working as a behavior technician. Elizabeth was a 23-year-old White female who had worked as a behavior technician for three years. Gabby was a 23-year-old White female who had experience working as a Registered Behavior Technician® (RBT®) for three years prior to the study. Amanda was a 22-year-old White female who had three years of experience working as a behavior technician prior to the study. Katherine was a 22-year-old White female who had worked as a behavior technician for two years prior to the study and was in the process of becoming an RBT® during the study. Brittany was a 26-year-old White female with three years of experience working as an RBT® prior to the study. Participants received undergraduate degrees in the fields of psychology, human development and family studies, child and family development, and behavior analysis. All of the intraverbal probe sessions, pretest and posttest took place online via Desire2Learn® (D2L®), and all teaching sessions occurred online via Google SlidesTM. Materials and Experimental Stimuli Participants were required to have access to a computer with internet and a webcam in order to complete the study. Throughout the experiment, participants used D2L® and Google SlidesTM to complete the pretests, teaching sessions, posttests, and intraverbal probes (described in more detail below). The link to appropriate teaching sessions was sent to 6 participants via their university email once they completed the appropriate intraverbal probe sessions. The intraverbal probe tests on D2L® consisted of open-ended questions. The teaching sessions, pretests, and posttests consisted of a MTS teaching format. Participants were required to use the Respondus Lockdown Browser® and Respondus Monitor® features within D2L® when completing the intraverbal probes, pretest, and posttest. The Respondus Lockdown Browser® is a browser that prevents individuals from using outside resources in the internet browser during testing. Respondus Monitor® uses the individuals’ webcams and requires them to show videos of their surroundings (Respondus LockDown Browser & Monitor, 2020). Behavior analytic vocabulary terms were the experimental stimuli. Stimulus sets were chosen based on terms in chapters of the Cooper et al. (2020) textbook, Applied Behavior Analysis, assigned in the participants’ graduate course. Each stimulus set consisted of 10 terms from each of the following chapters: 18 (“Verbal Behavior”), 19 (“Equivalence-Based Instruction”), and 30 (“Generalization and Maintenance of Behavior Change”), for a total of 30 terms. A list of the terms, definitions, and examples used for each stimulus set can be found in Tables 1-6. “A” stimuli were the MTS stimuli, or the terms. “B” stimuli were the definitions of the terms taken verbatim from the Cooper, Heron, and Heward (2020) textbook. “C” stimuli were examples of each term developed by the experimenters. Response Measurement and Dependent Variables Pretest, intraverbal probe, posttest, and teaching data were collected and scored as a percentage of correct answers. Correct responding during the intraverbal probes was defined as typing in the term that corresponded to the definition or example provided. Correct responses had to include all words of the term or concept. For example, if the definition of “response generalization” was provided and the participant wrote “generalization,” this would be scored as 7 incorrect. Responses were also scored as correct if they included a synonym of one of the words in the concept that did not change the meaning of the word. For example, when presented with the definition of naturally existing contingency, the response “naturally occurring contingency” would be scored as correct. Responses were also scored as correct regardless of whether the participant typed the plural or singular word, and regardless of tense or part of speech. For example, when presented with an example of a mand, “mand,” “mands,” or “manding” would all be scored as correct. Minor spelling errors were also scored as correct. For example, if a participant typed, “transivity” instead of “transitivity,” this answer would be scored as correct. Correct responding during the pretests and posttests was defined as selecting the term, definition, or the example (depending on the relation being trained or tested) that corresponded to the sample stimulus presented at the top of the screen. Correct responding during the MTS training was defined as selecting the comparison stimulus that corresponded to the sample stimulus presented at the top of the screen by using a mouse pointer to click on the correct definition or example. For example, during AB relation training, if the word “tact” was presented as the sample stimulus, the correct response would be selecting the stimulus that displayed the definition of the word “tact.” Procedural Fidelity and Interobserver Agreement Data were collected on procedural fidelity (PF) and interobserver agreement (IOA). Experimenters created a checklist of how to create the MTS PowerPoint based on Cummings & Saunders (2019). Each MTS training consisted of 40 trials and each trial consisted of seven necessary fidelity components that described to which slide each stimulus was to be hyperlinked. Each slide was scored as correct if the experimenter incorporated all necessary components or incorrect if the experimenter missed any of the necessary components when creating the slide. 8 Fidelity was calculated as the number of steps scored as correct by the total number of checklist steps multiplied by 100 to yield a percentage (Cooper et al., 2020). All three MTS trainings were created with 100% fidelity. For IOA, a research assistant, who was a graduate student, scored the intraverbal probes on 30% of sessions for each participant based on a scoring key for every question on D2L®. IOA was calculated as the number of agreed upon answers divided by total number of answers multiplied by 100 (Cooper et al., 2020). IOA for participants’ scores across all intraverbal probes was 100%. IOA was not calculated for pretest or posttest because D2L® automatically scored those tests. Research assistants also scored 30% of each participant’s MTS trainings for each stimulus set. Total count IOA was calculated as the number of correct responses calculated by one researcher divided by the number of correct responses calculated by another researcher multiplied by 100 to yield a percentage (Cooper et al., 2020) IOA for all participants’ MTS performance on stimulus set 1 was 100%. Mean IOA for participants’ MTS performance was 99.84% (range, 97.1% to 100%) on stimulus set 2. IOA for MTS performance on stimulus set 3 was 100% across all participants. Experimental Design A multiple probe design with probe conditions (Ledford & Gast, 2018) across stimulus sets was replicated across participants and embedded in a pretest-train-posttest design to evaluate the effectiveness of the EBI on the acquisition of definitions and examples of behavior analytic vocabulary. This design was appropriate for the current study because the skills taught were unlikely to emerge during baseline in the absence of direct instruction. All participants first completed a pretest and then started in baseline for all three stimulus sets until they demonstrated stable responding. Then participants completed MTS training (described below) for stimulus set 9 1 until they reached mastery criterion. Next, participants completed an intraverbal probe. If they demonstrated mastery, they moved on to MTS teaching for the next stimulus set. If they did not demonstrate mastery, they completed remedial training (described below). Participants followed this format until they completed intraverbal probes following each stimulus set, then finally took a posttest. Procedures Pretest Each participant completed a 180-question multiple choice pretest that tested all relations of stimuli (AB, AC, CA, BA, BC, CB) with no time limit. Participants were given instructions to select the option that corresponded to the term, definition, or example presented and they were told to guess if they did not know the answer. The examples used in the pretest were novel, meaning they were different than the ones used in the MTS teaching sessions and intraverbal probes. D2L® randomized the order of questions for each participant. No feedback was provided for correct or incorrect answers during the pretest. The test format consisted of the target word, definition, or example as the question and four answer choices (A, B, C or D) listed below. Experimenters randomized the three incorrect answer choices for each question using random.org. The pretest was administered to assess baseline knowledge of each relation and to be used as a comparison to test emergent symmetrical and transitive relations on the posttest. Intraverbal Probes There were two different intraverbal probes, “Intraverbal probe A” and “Intraverbal probe B” which each had the exact same format and definitions, but included one of the two examples used in the MTS training (see Tables 2, 4, and 6). Each intraverbal probe consisted 10 of 60 open-ended questions—20 questions per each of the three stimulus sets. Participants had to go through the lockdown browser set up for the intraverbal probes. The participants were instructed to type the term that corresponded to the example or definition presented. If they did not know the answer, they were instructed to type “I don’t know” or some other indicator that showed they attempted to answer the question. The questions included the definition or example presented at the top of the screen and one box below for the participant to write in the corresponding term. Participants had unlimited time to complete each intraverbal probe and no feedback was provided for correct or incorrect answers. After completing the last question, the participants clicked “Submit” and results were saved for the experimenters to score. Mastery criteria on the intraverbal probes were set to 80% correct responding or higher on both BA and CA relations for the corresponding stimulus set. If participants did not reach mastery criteria on the intraverbal probes, they completed remedial training (described below). Each participant completed three initial intraverbal probes following the pretest and prior to MTS teaching sessions. Participants also completed intraverbal probes following mastery of each stimulus set in the MTS teaching sessions. This was done to ensure experimental control by showing that change in scores only occurred when the intervention was applied (Ledford & Gast, 2018). The order of intraverbal probes completed (A or B) was randomized by participant. Each intraverbal probe was taken on a different day and on a different day from the pretest. All intraverbal probe conditions were identical to the intraverbal probe procedure described above. Data were only collected on BA and CA relations during intraverbal probes due to the nature of the intraverbal test format. 11 MTS Teaching Sessions During the MTS teaching sessions, participants were taught AB and AC relations via MTS procedures. Teaching sessions consisted of 40 questions total per stimulus set. The first 10 trials taught term-definition (AB) relations. The next 10 trials taught term-example (AC) relations. The next 10 trials taught term-definition (AB) relations. The last 10 trials taught term- example (AC) relations. The training included two different examples across the two AC relation sections. The MTS teaching sessions were delivered via Google SlidesTM. Experimenters created the sessions following directions outlined in Cummings and Saunders (2019). MTS target stimuli were presented simultaneously with three comparison stimuli below. Prior to training, participants were given the instructions to screen-record their participation in presentation mode for researchers to score and determine when they reached mastery criterion. The instructions for the MTS teaching sessions were to select the definition or example that corresponded to the term presented at the top of the screen. The training also included a slide describing the general procedure of the MTS teaching sessions including information about reinforcement slides and the error correction procedure. Participants screen-recorded their participation in the training and uploaded the video to D2L®. Correct selections were hyperlinked to a slide with a visual that read “CORRECT!” in green font. Incorrect responses were hyperlinked to a visual that read “INCORRECT” in red font and resulted in the delivery of the error correction procedure where participants were presented with the same question again with the correct answer prompted by an arrow above the term. Participants would then select the prompted answer that was hyperlinked to a screen that read: “Correct.” in grey font. Next, the same term was presented one more time without the 12 arrow to transfer stimulus control from the prompt to the term. Correct selections were then hyperlinked to a slide that said “Correct,” in green font. Any errors during the error correction procedure started the procedure over from the initial “INCORRECT” in red font feedback slide. In order to continue after feedback, participants selected a button that read “Next,” and was hyperlinked to either the next term or the next slide in the error correction depending on whether the participant selected the correct stimulus. Mastery criterion was set to 90% correct responding or higher across three consecutive teaching sessions for stimulus set 1. Based on low scores across participants on the intraverbal probes for stimulus set 1, mastery criterion was set to at least five initial MTS teaching sessions with the last two scoring at 90% accuracy or higher for stimulus sets 2 and 3. This was done to provide extra exposure to the terms, definitions, and examples. Experimenters emailed participants once they reached mastery criterion and instructed them of which intraverbal probe to take on D2L®. Next, participants moved on to teaching sessions for stimulus set 2. Experimenters followed an identical procedure for all three stimulus sets. If participants scored below an 80% on either BA or CA relations on the intraverbal probe for the current stimulus set, they were instructed to complete remedial training. Remedial Training (All participants) Remedial training for stimulus set 1 consisted of one additional MTS teaching session, followed by the opposite intraverbal probe. Based on participant responding, remedial training was changed for stimulus sets 2 and 3 to include two additional MTS teaching sessions, followed by the opposite intraverbal probe. Due to time constraints, experimenters also allowed participants to complete up to two trainings a day for stimulus sets 2 and 3. Intraverbal probes could not be taken on the same day following trainings, however after completion of an 13 intraverbal probe, participants were allowed to complete the next assigned trainings on the same day. If participants did not reach mastery criteria on the probes following remedial training, no additional remedial training was provided and participants moved on to the MTS teaching sessions for the next stimulus set. Posttest Each participant completed a 180-question multiple choice posttest that tested all relations of stimuli (AB, AC, CA, BA, BC, CB) following completion of teaching sessions and intraverbal probes of all three stimulus sets. The test format was identical to the pretest administered at the beginning of the study and no feedback was provided for correct or incorrect answers during the posttest. Social Validity Upon completion of the posttest, participants were given a survey to assess the social validity of this intervention. Participants were sent a survey via email that included questions regarding comfort and knowledge in terms before and after participation, time commitment, usefulness of the information they learned, and overall satisfaction with teaching method. Participants were instructed to rate each question on a five-point Likert scale with the following options: strongly agree, agree, neutral, disagree, and strongly disagree. The results of the social validity survey can be found in Table 7. All participants reported that the information they learned was relevant to their schoolwork and their fieldwork and that the MTS teaching sessions were easy to navigate. The majority of participants reportedly felt more confident in the understanding of the targeted terms following their participation in the study and that the intraverbal probes were easy to navigate. 14 All participants’ pretest and posttest data across the three stimulus sets can be found in Results Tables 8-10. Amanda scored a 57.78% on the pretest and improved her score to a 95.56% on the posttest. Her scores on each individual relation improved across all three stimulus sets. Data for Amanda’s intraverbal probes and MTS teaching sessions can be found in Figure 1. Scores are separated by BA and CA relations, which were untrained relations assessed during the intraverbal probes. Scores on BA relations ranged from 0% to 20% across stimulus sets on the first three intraverbal probes in the first intraverbal probe condition. The CA relations scores across stimulus sets ranged from 0% to 40% in the first intraverbal probe condition. Amanda required four MTS teaching sessions to reach mastery criterion for stimulus set 1. She then scored an 80% on BA relations and 70% on CA relations on the intraverbal probe following MTS teaching sessions for stimulus set 1. Scores on both relations for stimulus sets 2 and 3 remained low (range, 0% to 20%). Amanda completed one additional remedial training for stimulus set 1 and improved her CA score to an 80% while her BA score remained at 80%. Next, Amanda completed MTS teaching sessions for stimulus set 2 and reached mastery criterion after five sessions. She scored 60% on BA relations and 50% on CA relations on the intraverbal probe following MTS teaching for stimulus set 2. Her scores for stimulus set 1 remained high at 70% for both relations. Her scores for BA and CA relations for stimulus set 3 were both 0%. Amanda then completed two remedial trainings for stimulus set 2 and her score on BA relations dropped to 50% while her CA score increased to 80%. Amanda then completed seven MTS teaching sessions for stimulus set 3 to reach mastery criterion. Her scores on BA and CA relations for stimulus set 3 were 30% and 10%, respectively. 15 She scored 50% on both relations for stimulus set 2 and scored 70% on BA relations and 80% on CA relations for stimulus set 1. Amanda required two remedial trainings for stimulus set 3 and demonstrated improvement in scores for stimulus set 3 relations. Her score on BA relations increased to 80% while her score on CA relations increased to 50%. Talia scored a 71.11% on the pretest and a 96.11% on the posttest. Her scores on each individual relation on the pretest ranged from 50% to 100% and she demonstrated improvements on her scores for all relations except CA relations in stimulus set 2 in which her score dropped from 100% to 90%. Data for Talia’s intraverbal probes and MTS teaching sessions can be found in Figure 2. Talia’s scores on both BA and CA relations remained low (under 50%) across all three stimulus sets on all three intraverbal probes in the first intraverbal probe condition. Talia required three MTS teaching sessions to reach mastery criterion for stimulus set 1. She then scored an 70% on both BA and CA relations following MTS teaching sessions for stimulus set 1. Scores on both relations for stimulus sets 2 and 3 remained low (range, 0%-10%). Talia completed one additional remedial training for stimulus set 1 and improved her CA score to an 80% while her BA score remained at 70%. Next, Talia completed MTS teaching sessions for stimulus set 2 and reached mastery criterion after five sessions. She scored 30% on BA relations and 50% on CA relations on the intraverbal probe following MTS teaching for stimulus set 2. She did not maintain her scores for stimulus set 1 following the MTS teaching for stimulus set 2 as she scored below an 80% on both BA and CA relations. Her scores for BA and CA relations for stimulus set 3 were 10% and 0%, respectively. Talia completed two remedial trainings for stimulus set 2 and her score on BA relations increased to 50% while her CA score increased to 60%. 16 Talia then completed seven MTS teaching sessions for stimulus set 3 to reach mastery criterion. Her scores on BA and CA relations on the intraverbal probe following MTS teaching sessions for stimulus set 3 were 80% and 70%, respectively. She only maintained mastery criterion at 80% on CA relations for stimulus set 1. The remaining scores across relations and stimulus sets ranged from 60% to 70%. Talia required two remedial trainings for stimulus set 3 and demonstrated a 10% increase on BA relations and a 10% decrease on CA relations for stimulus set 3. Gabby scored a 62.22% on the pretest and improved her score to a 99.44% on the posttest. Her scores on each individual relation improved across all three stimulus sets as well. Data for Gabby’s intraverbal probes and MTS teaching sessions can be found in Figure 3. Gabby scored between a 0% and 60% on BA and CA relations across all three stimulus sets on the three intraverbal probes in the first intraverbal probe condition. Gabby required three MTS teaching sessions to reach mastery criterion for stimulus set 1 and then scored 100% on both BA and CA relations. Scores on both relations for stimulus sets 2 and 3 remained low (range, 0%-30%). Next, Gabby completed MTS teaching sessions for stimulus set 2 and reached mastery criterion after five sessions. She reached mastery criterion for CA relations, but scored 60% on BA relations on the intraverbal probe following MTS teaching for stimulus set 2. She scored a 70% on BA relations and 80% on CA relations for stimulus set 1. Her scores for both relations for stimulus set 3 were 0%. Gabby completed two remedial trainings for stimulus set 2 and her score on BA relations increased to 90% while her CA score decreased to 70%. Gabby then completed seven MTS teaching sessions for stimulus set 3 to reach mastery criterion. Her scores on BA and CA relations on the intraverbal probe following MTS teaching sessions for stimulus set 3 were both 50%. She scored 70% on both relations for stimulus set 2 17 and scored 80% both relations for stimulus set 1. Gabby completed two remedial trainings for stimulus set 3 and improved her BA score 90% and CA score to 100%. Elizabeth scored a 75 % on the pretest and improved her score to a 96.67% on the posttest. Her scores on each individual relation improved or remained the same at 90% or 100% across all three stimulus sets as well. Data for Elizabeth’s intraverbal probes and MTS teaching sessions can be found in Figure 4. She demonstrated high scores on CA relations for stimulus set 1 ranging from 60% to 80% on the first three intraverbal probes in the first intraverbal probe condition. The rest of her scores across stimulus sets 2 and 3 in the first intraverbal probe condition ranged from 0% to 40%. Elizabeth completed three MTS teaching sessions for stimulus set 1 and scored an 80% on BA relations and 90% on CA relations. Both of her BA and CA relations scores were 20% for stimulus set 2 and both were 0% for stimulus set 3. Elizabeth completed five MTS teaching sessions to reach mastery criterion for both stimulus set 2 and stimulus set 3. She scored a 90% on both BA and CA relations on the intraverbal probe following teaching for stimulus set 2. She maintained her CA relation score for stimulus set 1 at 90% but not her BA relation score. Elizabeth’s scores for stimulus set 3 were 0% for both relations. On the intraverbal probe following MTS teaching sessions for stimulus set 3, Elizabeth scored a 30% on BA relations and a 50% on CA relations for the corresponding stimulus set. She maintained mastery criterion for all relations across stimulus sets 1 and 2 save BA relations in stimulus set 1. Elizabeth only required remedial training following the fourth intraverbal probe phase when she did not reach mastery criteria for stimulus set 3. With two remedial training sessions, Elizabeth improved her BA relations score for stimulus set 3 to 60% while her CA relations score dropped from 50% to 40%. 18 Brittany scored an 83.33% on the pretest and improved her score to a 98.89% on the posttest. Her scores on each individual relation improved or remained the same at 100% across all three stimulus sets as well. Data for Brittany’s intraverbal probes and MTS teaching sessions can be found in Figure 5. Brittany’s scores on both relations across the three stimulus sets ranged from 0% to 40% on the first three intraverbal probes. Upon reaching mastery criterion following three MTS teaching sessions for stimulus set 1, Brittany reached mastery criterion for BA relation, but scored a 70% on CA relations for stimulus set 1. Her scores for stimulus sets 2 and 3 remained low (range, 0% to 10%). With two remedial trainings, Brittany increased both her BA and CA scores for stimulus set 1 to 90%. Brittany completed five MTS teaching sessions for stimulus set 2 to reach mastery criterion. Brittany scored a 70% on BA relations and 60% on CA relations for stimulus set 2 on the intraverbal probe following MTS teaching sessions for the corresponding stimulus set. Her scores for stimulus set 3 remained at 0%. Following two remedial trainings for stimulus set 2, Brittany improved both her BA and CA scores to 100%. Next, Brittany completed five MTS teaching sessions for stimulus set 3 and scored 100% on BA relations and 60% on CA relations for the same stimulus set. BA and CA relations remained high for stimulus sets 1 and 2 and ranged from 70% to 90%. Following two remedial trainings for stimulus set 3, Brittany scored 100% on both relations. Katherine scored a 76.67% on the pretest and improved her score to a 93.89% on the posttest. Her scores on each individual relation improved or remained the same across all three stimulus sets as well. Katherine’s intraverbal probe and MTS data can be found in Figure 6. On the first three intraverbal probes in the first intraverbal probe condition, Katherine’s scores on BA and CA relations across stimulus sets ranged from 0% to 50%. Katherine completed three 19 MTS teaching sessions for stimulus set 1 and her BA relations scores on this stimulus set increased from 30% to 50% while her CA relations scores increased from 50% to 60%. Following one remedial training, both of her scores on stimulus set 1 increased to 70%. Katherine then completed seven MTS teaching sessions for stimulus set 2 in order to reach mastery criterion. With training on stimulus set 2, Katherine’s scores on BA relations increased from 0% to 10% and her scores on CA relations increased from 0% to 20%. Both BA and CA relations scores for stimulus set 1 were 60% while both scores on stimulus set 3 remained at 0%. With two remedial trainings for stimulus set 2, Katherine increased her BA relations score to 30% and her CA relations score to 40%. Katherine also required seven MTS teaching sessions to reach mastery criterion for stimulus set 3. On the intraverbal probe following MTS teaching for stimulus set 3, Katherine scored a 10% on BA relations and 20% on CA relations. Her scores across stimulus sets 1 and 2 ranged from 20% to 60%. Katherine completed two remedial trainings for stimulus set 3 that resulted in the same score for BA relations and a 10% decrease in CA relations. 20 Discussion The purpose of the present study was to assess the efficacy of EBI on emergent written topography-based responses and emergent MTS performance with behavior analytic terms, definitions, and examples. The MTS training was effective in producing emergent selection- based symmetrical and transitive responding across all participants and all stimulus sets (i.e. CA BA, BC, CB). However, the MTS training did not result in participants reaching mastery criteria on the written topography-based intraverbal probes across stimulus sets. All participants required remedial training at some point during the study. The study adds to the literature that suggests incorporating EBI into courses in higher education settings may be beneficial (Fienup et al., 2010; Walker et al., 2010; Lovett et al., 2011; Albright et al., 2016). Using a combination of both EBI and more traditional teaching methods in graduate courses could lead to well-developed understandings of necessary terms (Brodsky & Fienup, 2018). Symmetrical and transitive responding are both key factors in demonstrating stimulus equivalence. All six participants demonstrated emergent symmetrical and transitive selection- based responding across all three stimulus sets from pretest to posttest. This is in line with previous research as EBI is intended to be an efficient teaching method in which untrained relations emerge following the training of some relations (Brodsky & Fienup, 2018; Greville, Dymond, & Newton, 2016). The current study extends previous research in that participants demonstrated generalization across novel examples on the posttest with both symmetrical and transitive relations. Participants were taught two examples per term during the MTS teaching sessions and were tested on a novel example during the posttest. Previous research found that participants were unable to demonstrate stimulus generalization across novel examples of single- subject designs following MTS training (Walker & Rehfeldt, 2012). 21 Unlike previous studies, the current study found that there was limited efficacy of MTS training in promoting emergent intraverbal responding. Much of the previous literature in this area yielded promising results on the use of MTS training to promote emergent topography- based responding. Although it is unknown why this study did not produce emergent intraverbal responding, it could have been due to methodological differences from previous studies. In the current study, participants were taught 30 different terms, whereas previous research has demonstrated emergent topography-based responding when teaching a limited number of concepts, typically fewer than 15. For example, previous research assessed the effects of MTS training on emergent intraverbal responding when teaching nine research designs (Sella et al., 2014), four operant functions of behavior (Albright et al., 2016), 12 disabilities and their causes and treatments (Walker et al., 2010), and answering seven open-ended interview questions (O’Neill & Rehfeldt, 2014). The results from the current study, which used a large number of teaching targets, could indicate that MTS training is effective in promoting topography-based responding only under certain conditions (e.g. with 15 or fewer concepts). Brodksy and Fienup (2018) noted that much of the current literature on EBI is not representative of the wide range of material students are expected to learn in higher education courses. The large number of terms used in this study may have been too cumbersome to learn and maintain over time. Another methodological difference between the current study and previous studies is the duration of the intervention, which could contribute to why participants did not maintain high levels of responding. This study occurred over the course of three months, and participants were expected to retain information from each stimulus set for many weeks after the teaching sessions ended without any booster sessions. MTS training may be more effective in promoting topography-based responding when fewer concepts are taught within a smaller time frame. For 22 example, Walker, Rehfeldt, and Ninness (2010), found their intervention to be more effective when training and testing all occurred within one 30 to 45 min session. Although shorter sessions may produce better results, the purpose of the present study was to prepare individuals for the BACB® exam which would require them to maintain skills over a longer period of time (i.e. when they take the exam and throughout their careers). Therefore, future research could assess the efficacy of a similar intervention to the current study in a more condensed length of time. Another suggestion that may lead to improved maintenance over time is to consider adding booster sessions throughout the intervention to aid in the retention of information over a longer period of time. There are other possible explanations of the low scores on the intraverbal probes. First, participants may have not demonstrated emergent topography-based responding due to a possible higher response effort associated with topography-based responding compared to selection-based responding. Scores may have been higher for selection-based responding because the response topography was the same in teaching sessions as it was in the posttest and because there were three distinct answers to choose from during the MTS training and four answers to choose from during the pretest and posttest. Based on these findings, future research should consider the response topography across teaching sessions and assessments and how the similarities or differences may impact participant responding. Second, participants may have had difficulties discriminating between similar definitions within the same stimulus set. Future research could address this by teaching a larger number of concepts that could be more easily discriminated between. For example, researchers could strategically select terms that do not contain any overlap in the key words in the definitions. Future research could also investigate the efficacy of teaching fewer concepts at a time (e.g., 23 focus on one of the three stimulus sets used in the current study). Last, the use of complex definitions may have contributed to the low scores on the intraverbal probes. The definitions were taken verbatim from the Cooper et. al. (2020) textbook and many terms included other behavior analytic jargon that participants may have not been exposed to previously as they were completing their first semester of graduate school during the time of the study. For example, the definition for the term, “Echoic,” includes “point-to-point correspondence,” and “formal similarity.” A lack of familiarity to the terms within the definitions would make a robust understanding of the targeted definitions difficult to achieve. Future research may benefit from using more simplified definitions of terms when teaching vocabulary to participants. Teaching definitions using colloquial language to individuals pursuing their BCBA® certification definitions could have practical benefits in field as well. Behavior analysts’ strict use of technical jargon has a history of adverse effects on people outside of the field who are unfamiliar with the terms (Critchfield et al.,2017). There is also an expectation for behavior analysts to be able to use language that clients and stakeholders can understand (BACB, 2020). Learning how to define and explain terms using simpler language could help future behavior analysts build rapport with clients and avoid some of the negative attitudes associated with behavior analytic language. While all participants met mastery criterion during the MTS teaching sessions, it is possible that participant responding was at least partially under the control of certain words within the definitions instead of the whole formal definitions themselves. Put another way, correct responding may have been under stimulus control of irrelevant features, such as the key words and comparison stimuli (Walker & Rehfeldt, 2012). This could potentially explain why participants scored poorly on the written intraverbal probes, because the key words were present 24 without options for them to choose from. Using simplified definitions as described above may help address this potential limitation and lead to participants developing a more thorough understanding of the terminology and demonstrate stronger emergent topography-based skills. Remedial training was required for four participants for stimulus set 1, five participants for stimulus set 2 and all participants for stimulus set 3. Despite the addition of remedial trainings, participants did not always reach mastery criteria on the intraverbal probes. The extensive use of remedial training in this study could indicate that the initial exposure to terms, definitions, and examples may have not been sufficient enough to promote emergent written intraverbal responses. This explanation lends itself for future researchers to consider the number of MTS teaching sessions prior to assessing emergent topography-based responding to determine if that increases emergent written intraverbal performance. For example, Sella, Ribeiro, and White (2014) required participants to emit six consecutive correct responses for each taught relation before moving on to the testing session. Limitations The current study is not without limitations. One limitation is that researchers were unable to control for participants studying material on their own time. Given that participants were enrolled in an ABA master’s program, they could have used course materials or other outside study sources to study the target terms. While researchers instructed participants not to study on their own time, researchers understood there was no way to control this behavior of participants. However, given the generally low scores across intraverbal probes, it is unlikely that extra studying occurred. Another limitation of the study was that participants experienced multiple technical difficulties throughout the study. For example, there was a glitch in the MTS trainings for some 25 participants in which the reinforcement slide following correct responses was hyperlinked to the error correction slides. These glitches likely were due to issues with each individual participant’s computers given that they did not occur across participants or each time participants completed the MTS teaching sessions. Though these difficulties did not affect results as researchers scored the training sessions based on the first response the participant emitted, they highlight the need for future researchers to consider alternatives in case problems with technology arise. Another technical difficulty participants experienced was with the Respondus Monitor® where some participants reported having to use others’ computers because they were unable to complete the short videos of their surroundings. These examples illustrate the potential limitation of relying on the use of technology for interventions. Conclusions The purpose of the present study was to assess the efficacy of EBI on emergent written topography-based responses and emergent MTS performance with behavior analytic terms, definitions, and examples. The results indicated that MTS teaching was effective in promoting emergent MTS performance, but not written intraverbal topography-based performance. Instead of teaching students to memorize behavior analytic definitions verbatim, instructors may benefit from developing definitions that are simpler, easier to digest and more likely to be maintained over time in preparation for the BCBA® exam and use by future behavior interventionists in their careers. 26 APPENDIX 27 Table 1 List of Terms and Definitions for Stimulus Set 1 (Verbal Behavior). Term (A) Echoic Mand Tact Stimulus Set 1 (Verbal Behavior) Definition (B) An elementary verbal operant involving a vocal response that is evoked by a vocal verbal SD that has formal similarity between an auditory verbal stimulus and an auditory verbal response product, and a history of generalized reinforcement. An elementary verbal operant involving a response of any form that is evoked by an MO and followed by specific reinforcement. An elementary verbal operant involving a response that is evoked by a nonverbal discriminative stimulus and followed by generalized conditioned reinforcement. Intraverbal An elementary verbal operant involving a response that is evoked by a verbal discriminative stimulus that does not have point-to-point correspondence with that verbal stimulus. Taking Dictation An elementary verbal operant involving a spoken verbal stimulus that evokes a written, typed, or fingerspelled response that does not have formal similarity between the stimulus and the response, but does have point-to-point correspondence and a history of generalized reinforcement. Textual An elementary verbal operant involving a response that is evoked by a written verbal discriminative stimulus that does not have formal similarity between the stimulus and the response, but does have point-to-point correspondence and a history of generalized reinforcement. Generative Learning Copying Text Private Events Autoclitic A behavioral effect whereby previously acquired speaker and listener skills enable or accelerate the acquisition of other speaker and listener skills, without dependence on direct teaching or a history of reinforcement. An elementary verbal operant involving a written response that is evoked by a written verbal discriminative stimulus that has formal similarity and a history of generalized reinforcement. Covert events typically accessible only to the person experiencing them. Relation involving two interlocking levels of verbal behavior emitted in one utterance. 28 Table 2 List of Terms and Examples for Stimulus Set 1 (Verbal Behavior). Stimulus Set 1 (Verbal Behavior) Term (A) Example 1 (C) Example 2 (C) Intraverbal probe A Intraverbal probe B Example 3 (C) Pretest/Posttest Echoic Child says “cup” after mom says “cup.” Client says “train” after hearing therapist say “train.” Mand Saying “cookie,” and someone gives you a Saying “I want to watch TV” and someone turns cookie. on the TV. Tact and saying, “There’s an Seeing an apple apple.” Telling someone you’re sad. You say “book” and your child says “book.” Your client hands you a picture icon of an iPad and you give her the iPad. Pointing to cows on the side of the road and saying “cow.” Intraverbal Client says “bus” after hearing “The wheels on the...” Friend asks you when your birthday is, and you respond “September You ask your client “what TV show do you like?” and they say, “PJ 2nd.” Masks.” Writing down someone’s name after Writing down the time of an appointment when Typing a direct quote from your professor they tell you it. told over the phone. during class. Taking Dictation Textual Seeing “C-A-T” and saying “cat.” Seeing the number 5 and saying “Five.” Generative Learning Client asks for the ball after teaching her to label a ball. Client labels a picture of a dog after teaching him to select a picture from a dog from an array of 3 stimuli. Copying Text Writing down a Writing down key terms restaurant’s address after finding it on the website. from the textbook you’re reading. Seeing the written word “bowl” and saying “bowl.” Your child selects a picture of juice from an array after teaching her to request juice. Writing down titles of books you want to read while looking at a list of recommendations. Thinking about which Private Events Thinking about what you want for dinner. Autoclitic “It might be snowing.” Having a headache. movie you want to watch. “I think I’m going on “I know tomorrow is vacation.” Tuesday.” 29 Table 3 List of Terms and Definitions for Stimulus Set 2 (Generalization and Maintenance). Term (A) Response Generalization Multiple Exemplar Training Indiscriminable Contingency Teach Loosely Stimulus Set 2 (Generalization and Maintenance) Definition (B) The extent to which a learner emits untrained responses that are functionally equivalent to the trained target behavior. Instruction that provides the learner with practice with a variety of stimulus conditions, response variations, and response topographies to ensure the acquisition of desired stimulus control response forms A contingency that makes it difficult for the learner to discriminate whether the next response will produce reinforcement. Randomly varying functionally irrelevant stimuli within and across teaching sessions Response Maintenance The extent to which a learner continues to perform the target behavior after a portion or all of the intervention responsible for the behavior’s initial appearance in the learner’s repertoire has been terminated. Generalization probe Any measurement of a learner’s performance of a target behavior in a setting and/or stimulus situation in which direct training has not been provided. Naturally Existing Contingency Program Common Stimuli Any contingency of reinforcement (or punishment) that operates independent of the behavior analyst’s or practitioner’s efforts A tactic for promoting setting/situation generalization by making the instructional setting similar to the generalization setting Setting/Situation Generalization The extent to which a learner emits the target behavior in a setting or stimulus situation that is different from the instructional setting. Instructional Setting The environment where instruction occurs; includes all aspects of the environment, planned and unplanned, that may influence the learner’s acquisition, maintenance, and generalization of the target behavior. 30 Table 4 List of Terms and Examples for Stimulus Set 2 (Generalization and Maintenance). Stimulus Set 2 (Generalization and Maintenance) Term (A) Example 1 (C) Example 2 (C) Example 3 (C) Response Generalization Intraverbal probe A Teaching someone to answer “cookies” when asked their favorite food, and they answer “crackers” the next time the question is asked Intraverbal probe B Pretest/Posttest Teaching someone to wave to say bye and they also say “bye” You teach your child to flip a light switch to turn it on. Without any teaching they press a button that also turns the light on Multiple Exemplar Training Showing a picture of an Saying “do this” eagle, a pigeon and a Running one step “copy me” and “do the same” during imitation training hummingbird during tact training of the word “bird” imitation with objects with a ball, a car, and a train Indiscriminable Contingency Providing tokens on a VR-3 schedule of reinforcement Teach Loosely Setting up a child’s workplace in one area of the classroom during one session and moving to a new Allowing a break from the table when client asks for a break on average every 5x they ask. Teaching someone to form patterns with red, blue and green blocks during one session and purple, green and location in the yellow beads during classroom for the another session next session. Giving a child access to an iPad on a VI-5 minute schedule of reinforcement Using different stimuli every time you teach a client to identify shades of green 31 Table 4 (cont’d) Response Maintenance You begin teaching a child to recite his phone number by showing a visual prompt of all of the numbers. Over time you fade the numbers shown until you do not show any numbers. You ask the child their phone number and they recite it without the visual prompt; You teach a child to play Pop the Pig. A month after mastery/discontinuing teaching, you play pop the pig with the child and they independently perform all of the steps You learned how to say hello in French (Bonjour) in high school. 5 years later you are still able to say “Bonjour” when someone asks you how to say hello in correctly. French Generalization probe Having a new therapist ask a client how old she is after Naturally Existing Contingency being taught this skill by another therapist A girl is leaning back in her chair and the chair falls resulting in injury. Asking a client to tact a Having a client mand for information on the novel picture of her playground after sister. Putting on a sweater when it’s cold out makes you feel warm. teaching this skill in the classroom You feel an itch on your arm and scratch it and it goes away 32 Table 4 (cont’d) Program Common Stimuli Setting/Situation Generalization Using a bar of soap instead of a bottle to teach client to wash their hands because you know the family uses bars of soap at home. A learner is taught to complete a photographic activity schedule in their classroom and independently completes one in the gym. Instructional Setting The basement of a client’s home where ABA therapy sessions are conducted. Teaching a student to read their name in print and cursive because you know future teachers will display their name using different fonts Using a picture of the client’s dog when teaching him to label a dog Teaching a client to brush their teeth at home, and then they independently brush their teeth when sleeping over at their grandparents’ house. You taught your child to ask for a fork for their dinner at home. Your child then independently asks for a fork when you are at a restaurant The client’s classroom where ABA therapy sessions are conducted The treatment space at the clinic you work at 33 Table 5 List of Terms and Definitions for Stimulus Set 3 (Equivalence-Based Instruction). Term (A) Symmetry Stimulus Set 3 (Equivalence-Based Instruction) Definition (B) A type of stimulus-to-stimulus relationship in which the learner, without prior training or reinforcement for doing so, demonstrates the reversibility of matched sample and comparison stimuli Reflexivity A type of stimulus-to-stimulus relation in which the learner, without any prior training or reinforcement for doing so, selects a comparison stimulus Transitivity Describes derived stimulus–stimulus relations that emerge as a product of that is the same as the sample stimulus training two other stimulus–stimulus relations Conditional Discrimination Performance in a match-to-sample procedure in which discrimination between the comparison stimuli is conditional on, or depends on, the sample stimulus present on each trial. Simple Discrimination Responding is under stimulus control of a single antecedent stimulus condition; described by the three-term contingency: SD→R→SR+. Higher-Order Operant Class Behavior defined in terms of general relations between antecedents and responses, rather than in terms of specific stimuli and responses Class-Specific Reinforcement A match-to-sample procedure in which not only is the correct comparison choice conditional on the sample stimulus, but the type of consequence delivered is, too A new member is added to a demonstrated stimulus equivalence class as the result of teaching a new conditional discrimination. The situation or context in which a stimulus (or stimulus class) occurs determines its function. Refers to dimensions of procedural arrangements when teaching multiple conditional discriminations 34 Class Expansion Contextual Control Training Structure Table 6 List of Terms and Examples for Stimulus Set 3 (Equivalence-Based Instruction). Term (A) Symmetry Stimulus Set 3 (Equivalence-Based Instruction) Example 1 (C) Example 2 (C) Example 3 (C) Intraverbal probe A Intraverbal probe B Pretest/Posttest A learner is taught that a A learner is taught that picture of a dog is the the written word same as a real dog “phone” is the same as and is able to understand without teaching that a real dog is the same as a picture of a dog a picture of a phone and understands that a picture of a phone is the same as the written word ”phone” in the absence of any reinforcement A learner is taught that picture of a toothbrush is related to a picture of toothpaste and knows that a picture of toothpaste is related to a toothbrush without any direct teaching Reflexivity Matching a picture of a slide to another identical picture of a slide without any previous teaching or reinforcement Matching the written word ”ball” to another Matching a pencil to another identical pencil written word “ball” without any direct teaching without any direct teaching or reinforcement Transitivity Teaching that the Teaching that the spoken word “tree” is the same as a picture of a tree. Teaching that a picture of a tree is the spoken word “pen” is the same as a picture of a pen. Teaching that a picture of a pen is the same as the written same as the written word tree. Client knows word pen. Client Teaching that a picture of a nickel is the same as a real nickel. Teaching that a a real nickel is the same as the written words “5 cents”. Client knows that a picture of a nickel is the same as the written words “5 cents” without direct teaching. that the spoken word "tree” is the same as the written word tree without direct teaching. Someone points to a picture of a cookie in an array of three different pictures when shown a 3-D toy cookie Conditional Discrimination knows that the spoken word "pen” is the same as the written word pen without direct teaching. Someone points to a blue card in an array of A child places a picture three different of a car on top of colored cards when they hear the word “blue” another picture of a car in an array of 3 when told to “put with same” 35 Table 6 (cont’d) Simple Discrimination Higher-Order Operant Class Class-Specific Reinforcement Class Expansion You put a single ball in front of your client and say “hand me the ball.” They hand you the ball. You clear a table and put a train in front of your client and say “point to the train.” They point to the train Generalized imitation Generalized manding When selecting a When selecting a picture when given the picture when given the written word, you spoken word, you receive a veggie straw, and when selecting the written word when receive a skittle, and when selecting the written word when given the spoken word you receive a high five. given the picture you receive social praise. Teaching that the Teaching that the spoken word “candle” is the same as the written word and a picture of a candle, then introducing an actual candle and teaching that it is the written word “lamp” is the same as the spoken word and an actual lamp, then introducing a picture of a lamp and teaching that it is the same as the picture, the same as the actual spoken word and the written word. item, the spoken word and the written word Your friend tells you to ring the doorbell when you arrive at her house. There is only one button next to your friend’s door. You press the button to ring the doorbell. Generalized instruction- following When selecting a picture when given the spoken word, you receive a toy car, and when selecting the picture when given the written word, you receive a chip. Teaching that a picture of a hairbrush is the same as the written word “hairbrush” and an actual hairbrush, then introducing the spoken word “hairbrush” and teaching that it is the same as the actual item, the picture and the written word 36 Table 6 (cont’d) Contextual Control Training Structure Matching an apple to the word “fruit” when presented with “fruit,” “vegetable” and “grain,” but matching an apple to the word “food” when presented with Matching a red car to a picture of a vehicle when presented with vehicles, animals and furniture, but matching a red car to other red items when presented “food,” “toy,” and with red, blue and “appliance.” green items Matching a hat to the word “clothing” when presenting with “clothing,” “silverware,” and “electronics,” but matching a hat to a picture of snow when presented with pictures of snow, sun and rain One-to-many Linear Series Many-to-one 37 Table 7 Results of Social Validity Survey by Participant. Social Validity Question The information I learned is relevant to my schoolwork. The information I learned is relevant to my fieldwork. I feel more confident in my understanding of the terms following participation in this study. Participation in this study was too time consuming. The MTS teaching sessions were easy to navigate. I had a strong understanding of these terms before participating in the study. The intraverbal probes were easy to navigate. Participant Amanda Talia Gabby Katherine Elizabeth Brittany SA SA SA SA SA SA A A A SA A SA A SA SA N A SA N N N D A D SA SA SA A A SA D D D D D SD SA SA SA D A A Note. SA=Strongly Agree; A=Agree; N=Neutral; D=Disagree; SD=Strongly Disagree. 38 Table 8 Pretest and Posttest Data by Relation and by Participant for Stimulus Set 1. Talia Gabby Katherine Elizabeth Brittany Post Pre Post Pre Post Pre Post 90 100 70 100 100 80 60 100 100 100 100 100 100 80 100 100 100 100 90 60 100 70 80 90 100 100 70 80 90 80 60 100 80 100 70 100 90 90 40 90 90 50 Relation AB BA AC CA BC CB Amanda Pre 50 60 60 80 70 60 Post 90 100 100 100 80 90 Post Pre 30 90 100 90 70 90 90 100 90 60 70 90 Pre 60 80 70 90 70 80 39 Table 9 Pretest and Posttest Data by Relation and by Participant for Stimulus Set 2. Amanda Talia Gabby Katherine Elizabeth Brittany Post Pre Post Pre Post Pre Post Pre Post Relation Pre Post AB BA AC CA BC CB 80 60 60 70 70 60 100 100 90 100 100 100 Pre 60 70 60 100 60 80 90 100 80 80 90 100 100 70 80 100 100 100 100 100 100 100 70 100 100 100 100 100 100 100 100 80 100 90 100 100 90 90 90 90 100 90 100 100 90 90 90 100 70 80 70 70 80 70 100 90 100 100 100 100 40 Table 10 Pretest and Posttest Data by Relation and by Participant for Stimulus Set 3. Amanda Talia Gabby Katherine Elizabeth Brittany Relation Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post 40 100 60 100 60 100 90 100 80 100 90 100 90 70 100 90 100 100 80 100 50 90 80 80 AB BA AC CA BC CB 40 20 60 60 50 30 100 100 80 90 100 100 50 80 70 70 70 60 100 100 100 100 100 100 20 30 30 50 80 60 100 100 100 100 100 100 70 70 70 60 80 80 100 100 100 100 80 90 41 Figure 1 Graphical Display of Amanda’s Probe, MTS, and Remedial Training Data. Note. Probe=Intraverbal probe phase; MTS=Match-to-Sample phase; RT=Remedial Training phase; Post RT Probe=Intraverbal probe phase following remedial training. 42 -10010203040506070809010001234567891011121314151617181920212223242526272829SessionsStimulusSet 1Probe 2MTS Set 2 RTPost RT ProbeRTPost RT ProbeProbe 3Probe 1MTS Set 1 MTS Set 3Probe 4RTPost RT Probe-10010203040506070809010001234567891011121314151617181920212223242526272829Percent Correct ResondingSessionsStimulusSet 2-10010203040506070809010001234567891011121314151617181920212223242526272829SessionsStimulusSet 3BACAAB+AC Figure 2 Graphical Display of Talia’s Probe, MTS, and Remedial Training Data. Note. Probe=Intraverbal probe phase; MTS=Match-to-Sample phase; RT=Remedial Training phase; Post RT Probe=Intraverbal probe phase following remedial training. 43 010203040506070809010001234567891011121314151617181920212223242526272829SessionsStimulusSet 1Probe 2MTS Set 2 MTS Set 3 Probe 4RTPost RT ProbeProbe 3RTPost RT ProbeRTPost RT Probe010203040506070809010001234567891011121314151617181920212223242526272829Percent Correct ResondingSessionsStimulusSet 2010203040506070809010001234567891011121314151617181920212223242526272829SessionsStimulusSet 3BACAAB+ACProbe 1MTS Set 1 Figure 3 Graphical Display of Gabby’s Probe, MTS, and Remedial Training Data. Note. Probe=Intraverbal probe phase; MTS=Match-to-Sample phase; RT=Remedial Training phase; Post RT Probe=Intraverbal probe phase following remedial training. 44 -1001020304050607080901000123456789101112131415161718192021222324252627SessionsStimulusSet 1Probe 2MTS Set 2 MTS Set 3 Probe 4RTPost RT ProbeRTPost RT Probe-1001020304050607080901000123456789101112131415161718192021222324252627Percent Correct ResondingSessionsStimulusSet 2-1001020304050607080901000123456789101112131415161718192021222324252627SessionsStimulusSet 3BACAAB+ACProbe 1MTS Set 1 Probe 3 Figure 4 Graphical Display of Elizabeth’s Probe, MTS, and Remedial Training Data. Note. Probe=Intraverbal probe phase; MTS=Match-to-Sample phase; RT=Remedial Training phase; Post RT Probe=Intraverbal probe phase following remedial training. 45 -100102030405060708090100012345678910111213141516171819202122SessionsStimulusSet 1Probe 2MTS Set 2 MTS Set 3 Probe 4RTPost RT Probe-100102030405060708090100012345678910111213141516171819202122Percent Correct ResondingSessionsStimulusSet 2-100102030405060708090100012345678910111213141516171819202122SessionsStimulusSet 3BACAAB+ACProbe 1MTS Set 1 Probe 3 Figure 5 Graphical Display of Brittany’s Probe, MTS, and Remedial Training Data. Note. Probe=Intraverbal probe phase; MTS=Match-to-Sample phase; RT=Remedial Training phase; Post RT Probe=Intraverbal probe phase following remedial training. 46 -1001020304050607080901000123456789101112131415161718192021222324252627SessionsStimulus Set 1Probe 1MTS Set 1MTS Set 2Probe 4RTProbe 2Post RT ProbeProbe 3RTPost RT ProbeMTS Set 3RTPostRT Probe-1001020304050607080901000123456789101112131415161718192021222324252627Percent Correct ResondingSessionsStimulus Set 2-1001020304050607080901000123456789101112131415161718192021222324252627SessionsStimulus Set 3BACAAB+AC Figure 6 Graphical Display of Katherine’s Probe, MTS, and Remedial Training Data. Note. Probe=Intraverbal probe phase; MTS=Match-to-Sample phase; RT=Remedial Training phase; Post RT Probe=Intraverbal probe phase following remedial training. 47 -100102030405060708090100012345678910111213141516171819202122232425262728293031SessionsStimulusSet 1Probe 2MTS Set 2 MTS Set 3 Probe 4Probe 1RTPostRT ProbeMTS Set 1Probe 3RTPost RT ProbeRTPostRTProbe-100102030405060708090100012345678910111213141516171819202122232425262728293031Percent Correct ResondingSessionsStimulusSet 2-100102030405060708090100012345678910111213141516171819202122232425262728293031SessionsStimulusSet 3BACAAB+AC REFERENCES 48 REFERENCES Albright, L., Schnell, L., Reeve, K. F., & Sidener, T. M. (2016). Using stimulus equivalence- based instruction to teach graduate students in applied behavior analysis to interpret operant functions of behavior. Journal of Behavioral Education 25, 290-309. https://doi.org/10.1007/s10864-016-9249-0 Baer, D. M., Wolf, M. M., Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91-97. Behavior Analyst Certification Board. (2020). Board certified behavior analyst handbook. https://www.bacb.com/wp-content/uploads/2020/11/BCBAHandbook_201119.pdf Behavior Analyst Certification Board. (2021). US employment demand for behavior analysts: 2010-2020. Littleton, CO: Author. Blair, B. J., Tarbox, J., Albright, L. A., MacDonald, J.M., Shawler, L. A., Russo, S. R., & Dorsey, M. F. (2019). Using equivalence-based instruction to teach the visual analysis of graphs. Behavior Interventions, 34(3), 405-418. https://doi.org/10.1002/bin.1669 Brodsky, J., & Fienup, D. M. (2018). Sidman goes to college: A meta-analysis of equivalence- based instruction in higher education. Perspectives on Behavior Science, 41(1), 95-119. https://doi.org/10.1007/s40614-018-0150-0 Carr, D., & Felce, D. (2009). Application of stimulus equivalence to language intervention for individuals with severe linguistic disabilities. Journal of Intellectual & Developmental Disability, 25(3), 181-205. https://doi.org/10.1080/13269780050144262 Critchfield, T. S., Barnes-Holmes, D., & Dougher, M. J. (2018). What Sidman did – Historical and contemporary significance of research on derived stimulus relations. Perspectives on Behavior Science, 41, 9-32. Critchfield, T. S., Becirevic, A., Reed, D. D. (2017). On the social validity of behavior-analytic communication: A call for research and description of one method. Analysis of Verbal Behavior, 33(1), 1-23. doi: 10.1007/s40616-017-0077-7 Crites, Steven A. Dr. (2017) "The Board Certified Behavior Analyst (BCBA) credential: What is it, how to get it," Kentucky Teacher Education Journal: The Journal of the Teacher Education Division of the Kentucky Council for Exceptional Children, 3(2), 1-7. Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis. 3rd Edition. Pearson. Cummings, C., & Saunders, K. J. (2019). Using PowerPoint 2016 to create individualized 49 matching to sample sessions. Behavior Analysis in Practice, 12, 483-490. https://doi.org/10.1007/s40617-018-0223-2 Deochand, N. & Fuqua, R.W. (2016). BACB certification trends: State of the states (1999 to 2014). Behavior Analysis in Practice, 9(3), 243-252. doi: 10.1007/s40617-016-0118-z Fienup, D, M., Covey, D. P., & Critchfield, T, S. (2010). Teaching brain-behavior relations economically with stimulus equivalence technology. Journal of Applied Behavior Analysis, 43, 19-33. doi: 10.1901/jaba.2010.43-19 Fienup, D. M., Hamelin, J., Reyes-Giordano, K., & Falcomata, T. S. (2011). College-level instruction: Derived relations and programmed instruction. Journal of Applied Behavior Analysis, 44(2), 413-416. doi: 10.1901/jaba.2011.44-413 Greville, W. J., Dymond, S., & Newton, P. M. (2016). The student experience of applied equivalence-based instruction for neuroanatomy teaching. Journal of educational evaluation for health professions, 13(32). https://doi.org/10.3352/jeehp.2016.13.32 Griffith, K. R., Ramos, A. L., Hill, K. E., & Miguel, C. F. (2018). Using equivalence-based instruction to teach piano skills to college students. Journal of Applied Behavior Analysis, 51(2), 207-219. https://doi.org/10.1002/jaba.438 Hyman, S. L., & Levy, S. E. (2019). Autism: AAP guidance includes updates, searchable topics, executive summary. American Academy of Pediatrics. https://www.aappublications.org/ news/2019/12/16/autism121619 Ledford, J. R., & Gast, D. L. (2018). Single case research methodology: Applications in special education and behavioral sciences. 3rd Edition. Routledge. Lovett, S., Rehfeldt, R.A., Garcia, Y., & Dunning, J. (2011). Comparison of a stimulus equivalence protocol and traditional lecture for teaching single-subject designs. Journal of Applied Behavior Analysis, 44(4), 819-833. Michael, J., Palmer, D., & Sundberg, M. L. (2011). The multiple control of verbal behavior. The Analysis of Verbal Behavior, 27, 3-22 O’Neill, J., & Rehfeldt, R. A. (2014). Selection-based instruction and the emergence of topography-based responses to interview questions. The Analysis of Verbal Behavior, 30, 178-183. https://doi.org/10.1007/s40616-014-0013-z Polson, D.A., & Parsons, J.A. (2000). Selection-based versus topography-based responding: An important distinction for stimulus equivalence? The Analysis of Verbal Behavior, 17, 105-128. Pytte, C. L., & Fienup, D. M. (2012). Using equivalence-based instruction to increase efficience 50 in teaching neuroanatomy. The Journal of Undergraduate Neuroscience Education, 10(2), A125-A131. Respondus LockDown Browser & Monitor. (2020). Retrieved February 22, 2021, from https://help.d2l.msu.edu/node/4686 Sella, A. C., Ribeiro, D. M., & White, G. W. (2014). Effects of an online stimulus equivalence teaching procedure on research design open-ended questions performance of international undergraduate students. The Psychological Record, 64, 89-103. https://doi.org/10.1007/s40732-014-0007-1 Slocum, T. A., Detrich, R., Wilczynski, S. M., Spencer, T. D., Lewis, T., & Wolfe, K. (2014). The evidence-based practice of applied behavior analysis. The Behavior Analyst, 3, 41- 56. Stewart, I., McElwee, J., & Ming, S. (2013). Language generativity, response generalization, and derived relational responding. The Analysis of Verbal Behavior, 29, 137-155. Walker, B. D., & Rehfeldt, R. A. (2012). An evaluation of the stimulus equivalence paradigm to teach single-subject design to distance education students via blackboard. Journal of Applied Behavior Analysis, 25(2), 329-344. doi: 10.1901/jaba.2012.45-329 Walker, B. D., Rehfeldt, R. A., & Ninness, C. (2010). Using the stimulus equivalence paradigm to teach course material in an undergraduate rehabilitation course. Journal of Applied Behavior Analysis, 43(4), 615-633. doi: 10.1901/jaba.2010.43-615 51