AN EVALUATION OF THE EFFECTIVENESS OF SOCIAL STORIES FOR CHILDREN WITH AUTISM By Kimberly I. Meister A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Applied Behavior Analysis—Master of Arts 2020 ABSTRACT AN EVALUATION OF THE EFFECTIVENESS OF SOCIAL STORIES FOR CHILDREN WITH AUTISM By Kimberly I. Meister Social stories are a popular intervention technique utilized to teach appropriate social skills to individuals with autism spectrum disorder (ASD). While there is an abundant amount of research asserting the effectiveness of social stories as a social intervention strategy for children with ASD, there are also limitations of that previous research. The current study identifies three limitations of existing social story research, including: a lack of adherence to the social story guidelines published by Carol Gray (the creator of social stories), limited information reported on participants with predominant use of elementary school aged and seemingly higher functioning participants, and the inability to exhibit a causal relationship between an independent and dependent variable. In order to address these three limitations, a social story intervention aimed at increasing appropriate social behaviors was implemented using three pre-school age children with ASD. All social stories utilized throughout the study fully adhered to Carol Gray’s guidelines and acted as a sole independent variable. A withdrawal design was employed and provided no evidence of an increase in appropriate behavior displayed by any of the three participants. Keywords: social story, autism, behavioral intervention, pre-school age children TABLE OF CONTENTS LIST OF FIGURES………………………………………………………………………………iv INTRODUCTION………………………………………………………………………………...1 Limitations of Previous Research…………………………………………………………………3 Lack of adherence and description…………………………………………………......................3 Participant characteristics…………………………………………………………………………4 Lack of functional relationships…………………………………………………………………...6 Purpose of the Current Study……………………………………………………………………...6 METHOD…………………………………………………………………………………………8 Participants and Setting…………………………………………………………………………...8 Materials…………………………………………………………………………………………..8 Social Stories……………………………………………………………………………………...8 General Procedures………………………………………………………………………………..9 Measurement…………………………………………………………………………………......10 Corresponding target behavior…………………………………………………………………...11 Experimental Design……………………………………………………………………………..11 Baseline…………………………………………………………………………………………..12 Intervention………………………………………………………………………………………12 Interobserver Agreement and Procedural Fidelity……………………………………………….13 Interobserver agreement………………………………………………………………………….13 Procedural fidelity………………………………………………………………………………..14 RESULTS………………………………………………………………………………………..15 Forest………………………………………………………………………………….………….15 Molly…………………………………………………………………………….……………….15 Tommy…………………………………………………………………………….……………..16 DISCUSSION………………………………………………………………………….………...17 APPENDICES…………………………………………………………………………………...20 APPENDIX A: Social Story Content Evaluation Checklist…………………………..………....21 APPENDIX B: Social Story Topic BCBA Interview……….…………………………...……....23 APPENDIX C: Procedural Fidelity Checklist: Baseline Condition ………………………….....24 APPENDIX D: Procedural Fidelity Checklist: Story Condition…..……………………….…....25 APPENDIX E: Figure 1…………………………………………………………………….…....26 APPENDIX F: Figure 2……………………………………………………………………….....27 APPENDIX G: Figure 3..………………………………………………………………....……..28 REFERENCES…………………………………………………………………………………..29 iii LIST OF FIGURES Figure 1. A picture of Molly’s primary social story……………………………………………..26 Figure 2. Process for identifying social story topics. Adapted from “The New Social Story Book,” by C. Gray, 2015 p. xxxiii- xxxiv. Copyright 2015 by Future Horizons………………..27 Figure 3. A graphical display of the percentage of whole intervals in which Forest, Molly, and Tommy each engaged in their assigned target behavior during observation sessions conducted throughout both baseline and intervention phases……………………………………………….28 iv INTRODUCTION Autism spectrum disorder (ASD) is a developmental disability that is characterized by challenges with social interactions, deficits in communication abilities, and repetitive or restrictive behaviors (American Psychiatric Association, 2013). These deficits in social interaction make it difficult for children with ASD to act appropriately in social situations and develop meaningful relationships with others, which can lead to loneliness, depression, and academic difficulties (Leaf et al., 2012). Most typically developing children are able to learn appropriate social skills naturally by observing and interacting with others, whereas children with ASD often require instructional programs that explicitly teach them these skills (Almutlaq & Martella, 2018). Social stories are a widely suggested intervention technique aimed at teaching social skills to children with ASD (Sansosti, Powell-Smith, & Kincaid, 2004). Social stories contain sentences written from a target student’s perspective and describe appropriate ways to behave in certain situations (Gray & Garand, 1993). The ability to share accurate information with individuals with ASD in a descriptive and meaningful way is the trademark goal of social stories, and it is the sharing of this information that is argued to result in subsequent improvements in behavior (Gray, 2015). Since social stories have become such a popular intervention strategy (Sansosti et al.) amidst the increasing amount of behavioral interventions for children with ASD, it is important to analytically evaluate the effectiveness of social stories as an intervention strategy (Kokina & Kern, 2010). The creator of social stories, Carol Gray, published comprehensive guidelines for creating social stories. The set of criterion published by Gray (2015) can be used as a tool to evaluate whether or not something constitutes a true social story. Gray’s book describes the 1 effectiveness of social stories for people with ASD of varying ages and levels of functioning. The idea that social stories are an effective intervention for people with ASD of virtually any age, and across a wide range of functioning levels might be part of their appeal to practitioners and researchers. Since the creation of social stories in 1993, several research studies have been conducted utilizing social story interventions for individuals with ASD (Qi, Barton, Collier, Lin, & Montoya, 2018). Much of the existing research has suggested that social stories may be an effective social intervention technique in some cases, though there are limitations of that previous research. Reviews evaluating social story effectiveness have shown mixed results. For example, a recent meta-analysis of six social story studies led authors to conclude that social stories meet the criteria to be considered an effective evidence-based intervention (Wang & Spillane, 2009). In a 2018 review of social story effectiveness, the majority of studies reviewed showed that social stories are effective based on PND scores, though only 32% of studies displayed moderate to strong results based on visual analysis (Qi et al.). Additionally, researchers have stated that social stories have been demonstrated as more effective when used with the purpose of decreasing problem behaviors, and results are more variable when they are used in order to increase appropriate behaviors or communication skills (Qi et al., 2018). Results showing that social stories were more effective for decreasing problem behaviors than for increasing appropriate behaviors is a noteworthy finding due to the fact that in her book, Gray warns against making a social story centered around decreasing a behavior, as her stories should always be written with the intention of increasing desirable behaviors (2015). Lastly, authors concluded that the inconsistent treatment effects demonstrated between 2 participants indicate that the effectiveness of social stories is quite variable depending on the individual participant (Qi et al.). Limitations of Previous Research Lack of adherence and description. One limitation in existing social story research is a lack of adherence to Carol Gray’s guidelines. As stated above, some researchers have deemed social stories more effective when used to decrease problem behaviors as opposed to increasing appropriate social behaviors (Qi et al., 2018), which does not align with Gray’s criterion. Similarly, in a 2010 meta-analysis regarding social story interventions, authors noted that though social stories had questionable effectiveness as a whole, they were found to be more effective when focusing on inappropriate behaviors than when teaching appropriate social skills (Kokina & Kern, 2010). Stories that are not written in full accordance with Gray’s guidelines cannot be called social stories (Gray, 2015). Therefore, social story research involving stories that are incongruent with Gray’s guidelines limit the reliability of their results and conclusions, because these studies are not evaluating a true social story intervention. It is possible that researchers implementing interventions such as these are actually investigating the effectiveness of social narratives, rather than social stories. Social narratives are an intervention procedure intended to provide the student with information regarding: social cues, expectations for behaving appropriately, and understanding the feelings of others (Sam & AFIRM Team, 2015). Though social narratives and social stories share some similar goals for their audiences, they cannot be considered the same intervention procedure, because social narratives do not require adherence to Gray’s guidelines as social stories do (Gray, 2015). 3 Current social story research has also been inconsistent when it comes to providing evidence that Gray’s guidelines were adhered to (Styles, 2011). Much of the current research evaluating social stories has remained unclear on the extent to which researchers followed Gray’s guidelines, or have been found to deviate from the established criterion (Reynhout & Carter, 2006). Additionally, multiple studies evaluating social stories have claimed to adhere to the guidelines, but upon closer inspection were found to have implemented them incorrectly. For example, studies have broken Gray’s rules by using incorrect directive sentence ratios (Kuoch & Mirenda, 2003), incorporating sentences regarding what the audience shouldn’t do rather than what they should do (Scattone, Wilczynski, Edwards, & Rabian, 2002), and targeted multiple behaviors/situations within the same story (Sansosti et al., 2004). The inconsistent adherence to social story guidelines when creating the stories used for intervention limits the ability to compare social story research studies in a systematic way, and in turn effects the ability to evaluate the intervention’s effectiveness (Styles, 2011). Participant characteristics. A second limitation of previous research regarding the effectiveness of social stories is the amount of information reported on each participant, as well as an overarching use of elementary school aged, higher functioning participants. Existing social story research is limited in terms of the detail and consistency with which participant information is assessed (Qi et al., 2018). In single case design research, participants included in a study should have relatively similar functional characteristics (Ledford & Gast, 2018). By keeping participant characteristics as consistent as possible, researchers are able to refrain from selection bias acting as a threat to the internal validity of their study (Ledford & Gast). In a 2018 review of social story interventions, Qi et al. suggested that future social story research should aim to provide more detailed information on each participant including 4 diagnosis, intellectual functioning, and social communicative functioning in order to assess levels of similarity between participants. When conducting research involving individuals with ASD, one way to ensure similarity between functioning levels is to compare assessment scores and/or IQ scores between participants. A study using participants that all have reported scores on the same IQ test or type of assessment could be a useful expansion to existing social story research. While assessment and IQ scores are not direct indicators of functioning levels (Aylward & Stancin, 2008) they do provide a way of putting a quantitative measure on individual functioning, and can allow for researchers to make evaluations or comparisons across participants in a more consistent fashion. Because previous social story research has focused primarily on elementary school age participants, the effectiveness of social stories with pre-school age children is largely unknown (Crozier & Tincani, 2007). Additionally, the consistent usage of higher functioning students in current social story research limits knowledge regarding the effectiveness of social story interventions for lower functioning individuals (Kokina & Kern, 2010). For example, a replication study comparing social story interventions to interventions using the teaching interaction procedure included three participants, all of which had scores available representing their placement on the Gillian Autism Rating Scale (GARS-II), and revealed autism quotients of 89, 61, and 61 (Kassardjian et al., 2014). The GARS-II rating index considers a quotient of 69 or less to be an indication that the person is unlikely to have ASD, a 70-84 means the person is possibly on the spectrum, and an 85 or above indicates that the person is likely on the spectrum (Robinson, 2013). According to these score index indications, two of the three participants in the study were considered unlikely to have ASD at all, and as a result it is unclear to what extent the results of this study may generalize to other students with an ASD diagnosis. Further social story 5 research utilizing lower functioning participants is required in order to assess effectiveness of the technique with this population of individuals (Kokina & Kern). Lack of functional relationships. A third limitation that is apparent in previous social story research is the inability to show a causal relationship between the independent and dependent variables (Sansosti et al., 2004). The inclusion of interventions, such as implementing conditioned reinforcement systems, in addition to social stories has limited researcher’s ability to declare a causal relationship between the social story intervention and subsequent changes in participant behavior (Styles, 2011). Future research should aim to implement the social story intervention as a single independent variable in order to evaluate social stories on their own (Reynhout & Carter, 2006). Purpose of the Current Study Due to the variability in results between different reviews of social story effectiveness, and the limitations in previous research, further investigation of social story effectiveness was necessary. Since the majority of social story research that has deemed the technique effective has not used stories consistent with Carol Gray’s guidelines, a systematic evaluation of social story interventions was conducted with the assurance that the stories were created in exact accordance with Gray’s criterion (Kuoch & Mirenda, 2003). Along with creating stories congruent to those outlined by Gray, the limitations of past research regarding consistent assessment scores, participant ages, and levels of functioning were addressed in order to obtain more information on the effectiveness of social stories. Lastly, the current study aimed to introduce only one independent variable throughout the intervention in order to increase experimental control (Sansosti et al., 2004). A systematic implementation of social stories designed for preschool age 6 students with ASD was conducted in order to provide further data on the true effectiveness of the intervention strategy. Specifically, the research question being asked was: What are the effects of the social story intervention on increasing appropriate social behaviors for pre-school age children with ASD? 7 Participants and Setting METHOD Study participants included three children diagnosed with ASD, Forest (3 years old), Molly (3 years old), and Tommy (4 years old). All participants receive ABA therapy services in an Early Intensive Behavioral Intervention (EIBI) center for 30 hours per week. Due to the all- inclusive nature of social stories (Gray, 2015), this was the only criterion required for inclusion in the current study. The Mullen Scales of Early Learning (MSEL, 1995) was administered with all three participants prior to the study’s onset. Using the MSEL scores, each participant’s subscale Developmental Quotient (DQ) score and overall DQ score was calculated (Eapen, Crncec, & Walter, 2013). Forest’s overall DQ score was 30.4. Molly had an overall DQ score of 44.6. Tommy’s overall DQ score was 81.5. All research sessions took place inside of an EIBI classroom. For each session, the reading component of the social story intervention was conducted either in the classroom’s play area or at a table that was free of all other learning materials, and differed from the child’s assigned worktable. The observation component of each session took place in various areas of the school based on the nature of each participant’s unique story. These areas included: the classroom play area (Forest), the classroom lunch table (Molly), and the school hallway (Tommy). Materials Social Stories. Each social story was created according to Gray’s criterion for social stories (2015). The social story content evaluation checklist (Appendix A) was created, based on Grey’s recommendations, in order to ensure that each story adhered to her stated guidelines. The 8 social story content evaluation checklist outlines all of the criterion described by Gray regarding the content and creation specifications for social stories. There were six different social stories used in this study (two for each participant). During intervention sessions each participant had two stories read to them, one describing the appropriate behavior to be increased (the primary social story), and one describing an appropriate social behavior that the student had already been reliably displaying (the supplemental social story). One of Gray’s guidelines explains that 50% of a student’s social story library must be written regarding something that they already do well (2015), which is why a supplemental social story was implemented with each participant. Each story was printed on white paper. All stories contained two pages, with the number of sentences varying depending on the participant. Story length was based on the participant’s age, the complexity of the skill being targeted, and the researcher’s judgment of how long each particular participant was likely to attend to a story. The pages of each story were held together using two metal book rings, placed on the left side of the pages. All letters were written in “Cambria” font and printed in black ink. Additionally, letters appeared in size 16 font, with the exception of the title which was size 24 and bolded. All pictures included in the stories were printed in color ink, and taken directly from the social story CD provided with Gray’s manual. See Figure 1 for an example of a social story depicting the all of the specifications listed above. General Procedures Each social story was created unique to the participant that it was made for, in accordance with Gray’s guidelines as well as the Social story content evaluation checklist displayed in Appendix A. The creation of each participant’s social stories were completed before any intervention sessions took place. The process of creating each social story began with the author 9 interviewing the participants’ BCBA. During these interviews the BCBA answered twelve questions regarding the target student’s interests, general strengths/weakness, social strengths/weakness, and some additional information regarding each participant. A list of the specific interview questions utilized can be seen in Appendix B. Once areas in which the child might be displaying social deficits were identified, the author observed the student during these situations both from a third-person objective, as well as by putting themselves into the participant’s perspective as written in Gray’s guidelines (2015). These same observation methods were conducted for the areas in which the participant was said to exhibit appropriate social behaviors based on the BCBA interview. Once the social story topics were determined for each participant’s primary and supplemental social stories, the first author created two social stories for each participant that were both in full adherence with the Social story content evaluation checklist. See Figure 2 for a visual depiction of this process. Measurement. Data were taken only for the target behavior described in the participant’s primary social story. The implementer used a time- sampling data sheet to collect data during all sessions. Data were collected for all sessions, using a 10 second whole-interval technique (Ledford & Gast, 2018). A whole-interval recording technique was used because social stories target behaviors that will ideally be increased (Gray, 2015). A 10 second trial was marked with a “+” if the corresponding target behavior (described below) occurred throughout the entire 10 second duration of the trial, and a “-“ was recorded if the target behavior either did not occur at all during the interval, or occurred for less than the entire 10 seconds. Observation sessions for Forest and Molly were conducted for 5 minutes. Tommy’s observed activity of walking in the school hallway to recess, varied in length from session to session, and did not ever last a full 5 minutes. Due to this variability in activity duration, observation sessions for Tommy lasted for 10 the entire duration of the activity, ending when he no longer had the opportunity to emit a corresponding target behavior. Corresponding target behavior. A corresponding target behavior refers to a behavior exhibited by the participant that matches the appropriate behavior described in that participant’s primary social story. The target behavior for each participant was operationally defined prior to the beginning of intervention. Forest’s target behavior was using his hands when playing with items in the classroom play area, which was defined as: being present in the classroom play area, and during instances of manipulating/engaging with objects in the classroom play area only hand(s) are used and not any other body parts. Molly’s target behavior of sitting appropriately in her chair during lunch time was defined as: during lunchtime student’s bottom is fully resting on their chair at the lunch table, with the upper part of their body upright and all four legs of the chair resting on the ground. Lastly, Tommy’s target behavior of walking appropriately through the hallway to recess was defined as: transitioning through the school hallway to recess with body upright and with body hands empty. Hands do not exceed shoulder height, or initiate any physical touch with peers, instructors, objects, or parts of the building. Voice does not exceed that of a typical speaking volume. Experimental Design A withdrawal design was used to evaluate the effectiveness of the social story intervention on the occurrence of corresponding target behaviors (Ledford & Gast, 2018). This type of design was chosen in order to provide a convincing demonstration of whether or not a functional relationship existed between the independent and dependent variables (Ledford & Gast). Specifically, the use of this design increased confidence that any observed changes in participant’s target behaviors could be attributed to the social story intervention alone. 11 Additionally, the use of a withdrawal design allowed researchers to control for various threats to the internal validity of the study such as history. Since the current study was carried out with the same three participants for multiple months, controlling for events that might take place during the experiment (history) was a priority. Baseline. Social stories were intended to target a behavior that the participant was not already engaging in reliably. In order to measure this, baseline sessions were run prior to any introduction of the social story. Baseline sessions were run until a consistent pattern in responding was observed using visual analysis. This was done to ensure that there were no sustained changes in the behavior prior to intervention, and also to ensure that researchers were not targeting a behavior that was already reliably occurring or improving. Baseline sessions began when the participant began a predetermined activity in which the target behavior was appropriate to engage in (the same activity described in that participant’s primary social story). The implementer then ran an observation session taking data on the occurrences and non- occurrences of the target behavior as described in General Procedures. For example, Forest had the target behavior of using his hands when playing with items in the classroom play area, which was operationally defined above. A baseline session for Forest began once he had been instructed by a teacher to go play, and he entered the classroom’s designated play area. Once Forest had entered the play area, the implementer began their timer and collected data on the target behavior as described in General Procedures. After five minutes had elapsed the implementer stopped the timer and the observation session was completed. Intervention. Intervention sessions took place for each participant once daily Monday- Thursday. Observation sessions each lasted 5 minutes (Forest and Molly) or the entire duration of the targeted activity (Tommy). Primary story reading sessions were completed no more than 12 10 minutes before the observation began. Intervention sessions began with the implementer bringing the participant to a spot in the classroom and sitting down with them on the floor or in chairs. The interventionist sat directly next to the child, or with the child on an adult’s lap ensuring that they had a clear view of the social story. The interventionist then ensured that the child was looking at, or in the direction of the story prior to beginning the reading. Once attending was established, the implementer read the entire story to the child from start to finish, including the title. Once the reading was completed, the implementer dismissed the child to their teacher. Within 10 minutes of the reading session ending, the participant was instructed to engage in the activity described in their primary social story. Intervention sessions also included a once daily supplemental story reading, which was conducted using the same techniques as described for the primary story reading, but without the observation session component. Using the example described above regarding using hands when playing with items in the classroom play area, once the interventionist finished reading Forest his primary social story, they dismissed him to his teacher. No more than 10 minutes later, Forest was instructed to “go play.” Once the child entered the classroom’s designated play area the observation session began. The interventionist then began their stopwatch and collected data as described in General Procedures. After 5 minutes had elapsed the interventionist stopped the timer and the observation ended. Interobserver Agreement and Procedural Fidelity Interobserver agreement. A graduate research student served as a secondary data collector for 30% of the sessions in each condition, for each participant. The second observer’s data was compared to that of the first author to calculate interobserver agreement (IOA). For each 10- second trial, an agreement was scored if both data collectors recorded an occurrence or 13 non-occurrence of the behavior for that interval. A disagreement was scored if data collectors did not both record an occurrence or non-occurrence of the target behavior for corresponding trials. Point by point percentage agreement was calculated by comparing the code of each interval recorded by the first observer, with the corresponding interval code recorded by the second observer. The total percentage agreement was calculated by dividing the number of agreements by the total number of agreements and disagreements and multiplying by 100 in order to yield a percentage (Ledford & Gast, 2018). Procedural fidelity. Procedural fidelity was measured using two different types of checklists. The first checklist evaluated the extent to which each social story was designed as prescribed by Gray (2015). Both the first author and another graduate research student completed the Social story content evaluation checklist for each story utilized throughout the study. Both implementers were required to be in full agreement that each story adhered to all aspects of the checklist before it was approved for use during intervention sessions. A second procedural fidelity checklist was created in order to measure whether or not all steps of the intervention were implemented as described. A graduate research student (the student who collected IOA data) collected procedural fidelity data using this checklist for 30% of the sessions in each condition, for all participants. This checklist was completed by indicating a simple “yes” or “no” next to each item on the list as a way to indicate whether or not the implementer adhered to each step of the protocol. A score was then calculated by dividing the number of occurrences (total “yes”) by the total number of occurrences and non-occurrences (total “yes” and “no”) and multiplying by 100 to yield a percentage (Ledford & Gast, 2018). See Appendix C and Appendix D for copies of the procedural fidelity checklists utilized for baseline and intervention conditions. 14 RESULTS The percentage of whole intervals with the target behavior occurring during each session across all three participants are depicted in Figure 3. Individual results for each participant are described below. Results for IOA and procedural fidelity will be posted at https://osf.io/sykrg/?view_only=71616bb0ec504206b9a1d5d30a9edcb7 when they become available. Forest. During the initial baseline condition, Forest engaged in the target behavior of using his hands when playing with toys in the classroom play area with percentages ranging from 17%-70% across eight sessions. When moving to the first intervention condition, which included seven sessions, Forest’s target behavior occurred between 17% and 93% of the time. When reversing back to baseline conditions Forest exhibited behavior similar to that in the initial baseline phase, with a range of about 23%-90% across 12 sessions. Finally, when reintroducing the social story intervention responding again remained similar with percentages ranging from about 10%-57% across 7 sessions. Molly. During 10 initial baseline sessions Molly engaged in the target behavior of sitting appropriately in her chair during lunchtime between 0%-100% of the time. After moving to the initial intervention condition consisting of eight sessions, the target behavior occurred with percentages ranging from 0%-97%. When returning to baseline conditions for eight more sessions, Molly exhibited behavior similar to that of the previous conditions with occurrences ranging from 0%-100%. Molly’s behavior continued to resemble that of other conditions when running seven sessions in the second intervention condition with percentages ranging from 0%- 87%. 15 Tommy. During the initial baseline condition, Tommy exhibited the target behavior of appropriately walking through the school’s hallway to recess with a range of 50%-100% across 12 sessions. When moving on to the initial intervention condition, he engaged in the target behavior with scores between 20%-100% throughout eight sessions. Similar to Forest and Molly, Tommy’s behavior in the second baseline condition resembled that of the previous phases, with percentages ranging from 17%-71% across 10 sessions. When returning back to intervention conditions Tommy engaged in the target behavior with scores between 20%-60%, continuing to resemble that of previous conditions. 16 DISCUSSION The social story intervention did not have any effect on increasing the appropriate social behaviors for these three preschoolers with ASD. All three participants engaged in variable behavior across each condition, and none of them exhibited any sustained increase in target behaviors from baseline to intervention conditions. Implications and future directions are discussed in more detail below. A major limitation of previous research is that the extent to which social stories aligned with guidelines established by Gray (2015) were unknown. As a result, it is difficult to understand the extent to which social stories formatted according to the published criterion improve the behavior of individuals with ASD (Karal & Wolfe, 2018). The current study addressed this limitation by developing the Social story content evaluation checklist, providing a comprehensive framework for creating and utilizing social stories as Gray describes. By operationalizing each of Gray’s criterion and ensuring its presence within the checklist, researchers were able to systematically implement each of these guidelines into the social story intervention. Therefore, this study is unique in that it is the first study to make a systematic attempt to implement social stories, as prescribed, at least with children with ASD. It is possible that designing and presenting the social stories as prescribed by Gray played a role in the ineffectiveness of the intervention during the current study. It is possible that social stories are more effective when created in a way that is incongruent to the guidelines, such as Kokina and Kern suggested when noting that social stories were more effective when discussing inappropriate behaviors (2010). However, the extent to which strict adherence to Gray’s guidelines affects student outcomes is unknown. 17 In addition, this present study adds to the limited body of research that evaluates social stories with young children (ages 0-5) with autism. As reported by Kokina & Kern (2010), a majority of research on social stories and children with autism occurs in populations 6-11 years, who exhibit high or average cognitive abilities, and strong social and communication skills. However, Gray (2015) recommends that social stories can be used for all ages and a wide range of developmental abilities, therefore, it is important to evaluate the effectiveness of social stories across a wide range of ages. The social story intervention did not increase target behaviors for three preschoolers with ASD. It is possible that social stories might not be an effective intervention for preschoolers with ASD, however, more research is needed. Given this finding, the lack of systematic adherence to Gray’s (2015) guidelines, and the minimal amount of social story research previously conducted with children under five years of age (Crozier & Tincani, 2007), we caution against claims of intervention effectiveness with this age group until further research is conducted with preschoolers (under five years) with ASD. Additionally, the current study extends upon the existing body of social story literature by having implemented the social story intervention as a single independent variable. A limitation of previous research is that additional interventions (e.g. prompting, reinforcement) have been included alongside social stories, many of which have resulted in positive outcomes (Reynhout & Carter, 2006) and may possibly misrepresent the potency of social stories as a singular intervention. The limited amount of research that has been conducted utilizing social stories as a sole intervention technique, along with the ineffectiveness of the technique within the current study, might indicate the possibility that social stories are more useful when used in conjunction with additional interventions. Future research might aim to compare the effects of a social story 18 intervention implemented as a single independent variable alongside a social story intervention package (e.g., Crozier & Tincani, 2006; Leaf et al, 2012; Litras, Moore, & Anderson, 2010). One limitation of this study is that we were unable to control for all variables in the context in which each target behavior occurred. For example, we had no control over the food packed for Molly’s lunch each day, students/staff being in the school hallway at the same time as Tommy, and the proximity of other children in the play area to Forest. It is possible that these changes could have affected the target behaviors engaged in by participants each day. This potential influence of external factors might provide some explanation for the variability in participant behavior throughout the study, though the extent of this is unknown. However, social stories are an intervention technique often used within classroom settings (Styles, 2011), and previously conducted social story research has low ecological validity because that research has been conducted in unnatural settings (Styles). Therefore, this perceived limitation may instead enhance the external validity of our study. Regardless, future research might explore the extent to which external factors within the classroom environment may improve or impede upon student success when social stories are implemented. The methods described in the current study provide a framework for future researchers to systematically implement a social story intervention. Given that much of the social story research has not implemented social stories as prescribed, utilized various populations of participants, or provided evidence for a functional relationship, the effectiveness of social stories is still unknown. 19 APPENDICES 20 Social Story Content Evaluation Checklist APPENDIX A • Story shares accurate information that is descriptive in content, meaningful for the individual’s life, and safe (physically, socially, and emotionally) for the intended audience. • Before determining goal of story, information about the audience and their relation to the topic was gathered and considered. • At least two observations were conducted in order to gather information on the individual; one from a third person observer perspective and one that places the author inside the audience’s environment and perspective. • 50% of a student’s story library acknowledges/praises what a student is already • Story does not acknowledge problem behaviors directly related to the audience. • Story begins with a title. • The title is followed by an introduction sentence or sentences that clearly state the doing well. topic. and/or explanation regarding the topic. • The introduction is immediately followed by a body that adds more description o Gray’s example on looking for lost toys: “Mom or dad knows how to find my toys. We will try to think and look” (p.xxxviii). • The body is followed by a conclusion that restates the original purpose of the story with the benefit of having additional concepts added. • Story contains a minimum of three sentences. • Story is formatted according to the audience’s specific abilities and needs, attention span, learning style, and if possible talents/interests. Examples of what these aspects may include: o Length (3-12 short sentences for young children) o Sentence structure o Vocabulary o Font/font size o Organization of text and illustrations • Each statement is in first-person or third-person perspective; no second-person perspective usage. • Story maintains a positive and patient tone which includes: o Using positive rather than negative language o “Clearly describing desirable responses and the rationale behind them” in order to “patiently share ideas about what to do in a given situation” (Gray, 2015, p. xlvi). • Story uses past, present, and/or future tense language. • Statements are literally accurate (with the exception of analogies and metaphors). • Story contains clear, accurate, and comfortable vocabulary for the audience. 21 • The story answers six “wh” questions (who, what, where, when, why, how). • The story is comprised of descriptive sentences and may have one or more coaching sentences. o Descriptive sentence: accurately describes relevant aspects of context/reinforces the meaning or importance of the sentences that surround it o Coaching sentence: guide behavior via descriptions of effective responses • Using the social story formula, the story is shown to describe more than direct: (# descriptive sentences in story/ # coaching sentences in story) ≥ 2 • Story has been revised to meet all criteria and reviewed by other TEAM (all people working together in order to serve a person with ASD) members. 22 APPENDIX B Social Story Topic BCBA Interview Social Story Topic Identification: BCBA Interview Date: BCBA Name: Participant: ELI Site: Researcher: 1. What are some of the student’s interests? 2. Please identify some general strengths and social strengths of the student? 3. What are some areas in which the student displays deficits or difficulties? 4. Can you think of any topics/programs/situations that would describe as very aversive to the student? 5. Is there a specific social situation in which you have noticed (on more than one occasion) that the student displays deficits or may lack the desirable behavior for that particular social scenario? 6. Do you have any idea/insight as to why the student might not be displaying the desirable behavior during this situation? 7. Do you believe that addressing this situation would be beneficial to the student? 8. Do you believe that addressing this situation would also provide some benefit to the student’s parents or caregivers? 9. Has the idea of improving any specific social behavior ever been brought up to you by the student’s parents/caregivers? 10. Do you believe that addressing this behavior and improving the social skill will in some way help to set the student up for success in the inclusive environment with their typically developing peers? 11. Do you believe that the student would benefit from having pictures or illustrations present in their unique social story? 12. Has the student ever displayed instances of taking spoken phrases too literally/ concrete thinking? 23 APPENDIX C Procedural Fidelity Checklist: Baseline Condition Participant: ______________ Date: ___________________ Time: _________ - ________ Data Collector: ___________ Session#:_______________ Researcher:______________ Social Story Baseline Procedural Fidelity Checklist During Observation Session 1. Child is instructed to engage in/brought to target activity. 2. Begins MotivAider 3. Does not provide any reinforcement to child for engaging in target behavior. 4. Ran session for a full 5 minutes, or for entire duration of target activity (if activity was less than 5 minutes in duration). Yes No Score Notes: 24 Procedural Fidelity Checklist: Story Condition Participant:______________ Date: ___________________ Time: _________ - ________ Data Collector: ___________ Session#:_______________ Researcher:______________ Social Story Intervention Procedural Fidelity Checklist During Reading Session 1. Brings child to play area floor, couch, or table in treatment room that differs from blue “work tables.” 2. Establishes attending (child looking at/toward social story). 3. Reads through entire story including the title. 4. Reads child correct story (unique story created for them). 5. Time recorded for reading of the second social story. During Observation Session 6. Child is instructed to engage in/brought to target activity no more than 10 minutes after reading session has ended. 7. Begins MotivAider 8. Does not provide any reinforcement to child for engaging in target behavior 9. Ran session for a full 5 minutes, or for entire duration of target activity (if activity was less than 5 minutes in duration) Yes No Score APPENDIX D 25 Notes: APPENDIX E Figure 1. A picture of Molly’s primary social story. 26 Determining social story topics: APPENDIX F Interview participant's BCBA Third-person obervations of participant inside Observations placing author participant's environment/p erspective Story topics identified based on all information Figure 2. Process for identifying social story topics. Adapted from “The New Social Story Book,” by C. Gray, 2015 p. xxxiii- xxxiv. Copyright 2015 by Future Horizons. Social story creation begins 27 APPENDIX G Figure 3. A graphical display of the percentage of whole intervals in which Forest, Molly, and Tommy each engaged in their assigned target behavior during observation sessions conducted throughout both baseline and intervention phases. 28 REFERENCES 29 REFERENCES Almutlaq, H., & Martella, C. R. (2018). Teaching elementary- aged students with autism spectrum disorder to give compliments using a social story delivered through an iPad application. International Journal of Special Education, 33(2), 482-492. American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing. Aylward, P. G. & Stancin, T. (2008). Screening and assessment tools: Measurement and psychometric considerations. Developmental-Behavioral Pediatrics Evidence and Practice, 7, 123-201. Crozier, S., & Tincani, M. (2007). Effects of social stories on prosocial behavior of preschool children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 37, 1803-1814. doi:10.1007/s10803-006-0315-7 Eapen, V., Crncec, R., & Walter, A. (2013). Clinical outcomes of an early intervention program for preschool children with autism spectrum disorder in a community group setting. BMC Pediatrics, 13(3), 1-9. Retrieved from http://www.biomedcentral.com/1471-2431/13/3 Gray, C. (2015). The new social story book. Arlington, TX: Future Horizons. Gray, C. A., & Garand, J. D. (1993). Social stories: Improving responses of students with autism with accurate social information. Focus on Autistic Behavior, 8(1), 1-10. Karal, M. ., Wolfe, P. S. (2018). Social story effectiveness on social iteraction for students with autism: A review of the literature. Education and Training in Autism and Developmental Disabilities, 53(1), 44-58. Kassardjian, A., Leaf, B., Ravid, D., Leaf, J., Alcalay, A., Dale, S., . . . Oppenheim-Leaf., L. M. (2014). Comparing the teaching interaction procedure to social stories: A replication study. Journal of Autism and Developmental Disorders, 44, 2329-2340. doi:10.1007/s10803-014-2103-0 Kokina, A., & Kern, L. (2010). Social story interventions for students with autism spectrum disorders: A meta-analysis. Journal of Autism and Developmental Disabilities, 40, 812- 826. Kuoch, H., & Mirenda, P. (2003). Social story interventions for young children with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 18(4), 219- 227. doi: 10.1177/10883576030180040301 30 Leaf, J. B., Oppenheim-Leaf, M. L., Call, N. A., Sheldon, J. B., Sherman, J. A., Taubman, M.,… Leaf, R. (2012). Comparing the teaching interaction procedure to social stories for people with autism. Journal of Applied Behavior Analysis, 45, 281-298. Ledford, R. J., & Gast, L. D. (2018). Single case research methodology: Applications in special education and behavior sciences third edition. New York, NY: Routledge. Litras, S., Moore, D. W., & Anderson, A. (2010). Using video self-modelled social stories to teach social skills to a young child with autism. Autism Research and Treatment, 2010, 1- 9. Mullen, E. M. (1995). Mullen scales of early learning. Circle Pines, MN: American Guidance Service. Qi, H. C., Barton, E. E., Collier, M., Ling, Y., & Montoya, C. (2018). A systematic review of effects of social stories interventions for individuals with autism spectrum disorder. Focus on Autism and Other Developmental Disabilities, 33(1), 25-34. doi:10.1177/1088357615613516 Reynhout G., Carter M. (2006). Social stories for children with disabilities. Journal of Autism and Developmental Disorders, 36(4), 445-469. doi:10.1007/s10803-006-0086-1 Robinson J. (2013). Gilliam Autism Rating Scale (GARS). In: Volkmar F.R.(eds) Encyclopedia of Autism Spectrum Disorders. New York, NY: Springer Sam, A., & AFIRM Team. (2015). Social narratives. Chapel Hill, NC: National Professional Development Center on Autism Spectrum Disorder, FPG Child Development Center, University of North Carolina. Retrieved from http://afirm.fpg.unc.edu/social-narratives Sansosti, F. J., Powell-Smith, K. A., & Kincaid, D. (2004). A research synthesis of social story intervention for children with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 19(4), 194–204. Scattone, D., Wilczynski, M. S., Edwards, P. R., & Rabian, B. (2002). Decreasing disruptive behaviors of children with autism using social stories. Journal of Autism and Developmental Disorders, 32(6), 535-543. doi:10.1023/A:1021250813367 Styles, A. (2011). Social stories: Does the research evidence support the popularity?. Educational Psychology in Practice, 27(4), 415-436. doi:10.1080/02667363.2011.624312 Wang, P., & Spillane, A. (2009). Evidence-based social skills interventions for children with autism: a meta-analysis. Education and Training in Developmental Disabilities, 44(3), 318-342. 31