EXAMINING THE IMPACT OF IRIS-RTI MODULES ON PRESERVICE TEACHERS’ KNOWLEDGE OF RESPONSE TO INTERVENTION IN READING By Nai-Cheng Kuo A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Curriculum, Instruction, and Teacher Education – Doctor of Philosophy Special Education – Doctor of Philosophy 2013   ABSTRACT i       EXAMINING THE IMPACT OF IRIS-RTI MODULES ON PRESERVICE TEACHERS’ KNOWLEDGE OF RESPONSE TO INTERVENTION IN READING By Nai-Cheng Kuo Changes in legislation and the policies of education (e.g., NCLB 2002, IDEA 2004), the potential benefits of response-to-intervention (RTI), and government funds to support early intervention have made RTI a prominent issue in current education. To prepare teachers for implementing RTI, there are several government-sponsored online professional development programs available for public use. For example, the IDEA ’04 and Research for Inclusive Settings (IRIS) Center at Vanderbilt University, sponsored by the Department of Education, Office of Special Education Programs (OSEP), has developed several modules about RTI. However, although over 470,000 teachers and teacher educators have participated in online learning through IRIS, there is little empirical research to support its impact on preservice teachers. To fill the gap in this literature, this study incorporated ANGEL software to examine how effective IRIS-RTI modules are for improving preservice teachers’ knowledge of RTI in the academic domain, and how they influence preservice teachers’ perspectives toward RTI. A total of 55 pre-service teachers enrolled in a special education teacher preparation program at a large Midwest public university, voluntarily participated in this study. Students were rank ordered based on a pre-assessment score and then stratified into two groups (e.g., odd numbers and even numbers). Twenty-six of them were assigned to the experimental group (five juniors, sixteen seniors, and five interns) and twenty-nine of them were assigned to the control group (eight juniors, seventeen seniors, and four interns). The experimental group received eight IRIS-RTI modules, while a treated control group received eight IRIS modules not related to RTI.   ii       A key finding is that through comparisons across three different test resources (i.e., 66 Teacher Knowledge Survey test items, 29 IRIS test items, and 25 Literature test items) the results indicate that the eight IRIS modules significantly improved the knowledge of the experimental group when compared to the control condition, particularly on the 29 IRIS test items. However, because RTI is such a complex and multi-faceted, using the eight modules as a one-time exposure cannot be expected to provide the breadth or depth of knowledge needed to fully understand or implement RTI. Further investigation is needed to understand how preservice teachers retain the knowledge and skills that they learn from IRIS modules and how teacher educators adjust their instruction accordingly, based on preservice teachers’ performance on a more comprehensive measure after taking the IRIS modules.   iii       Copyright by NAI-CHENG KUO 2013   iv       ACKNOWLEDGEMENTS There are no words to convey how grateful I am to the faculty members, staff, consultants, colleagues, teachers, and students in the Department of Teacher Education (TE), the Department of Counseling, Educational Psychology, and Special Education (CEPSE), the Office of Student Writing Assistance (OSWA), the Writing Center, the Center for Statistical Training and Consulting (CSTAT), the Teaching Assistant Program, the University Outreach and Engagement Program, Marble Elementary School, Glencairn Elementary School, Wilcox Elementary School, MacDonald Middle School, and East Lansing High School. Without all their support, mentoring, and/or assistance, I would not have been able to build a solid foundation in teaching, research, and service, and I would not have been able to complete this dissertation, as well as my dual degrees in Teacher Education and Special Education. In particular, I want to thank my advisor, Dr. Troy Mariage, for all his advice, support, encouragement, and mentoring. I will always remember the meetings with Dr. Mariage every Thursday afternoon. He gave me not only knowledge, but also the tools to pursue the happiness of mine and others’. I appreciate all the encouraging e-mails from him and his wife, Mrs. Mary Mariage. They helped me understand how dialogue and actions can uncover a person’s unlimited and immense potential. Dr. Mariage also taught me how to face challenges and conflicts and how to move forward with intelligence, wisdom, patience, and unceasing efforts. Additionally, in every milestone, he never forgot to remind me of inviting my former advisor, Dr. Susan Peters, who retired in 2011. Dr. Peters is the person who led me to the world of inclusion and response-to-intervention (RTI). Having Dr. Peters and Dr. Mariage as my advisors was and is a treasure; they have stimulated and cultivated v       me to be a well-rounded faculty member who respects diversity, trusts in every student’s potential, and strives to become a good mentor. I would also like to express my deepest appreciation to my committee members: Dr. Suzanne Wilson, Dr. Gary Troia, and Dr. Cheryl Rosaen. Dr. Wilson and Dr. Rosaen guided me to collect, analyze, and present qualitative data. They helped me think critically about what is gained and what is lost in making decisions about research designs. I also enjoyed Dr. Troia’s quantitative research methods course very much. Dr. Troia raised my awareness of the importance of research validity, and he taught me the skills that I can use to ensure and review the appropriateness of a study. Without my committee members’ instruction and critical feedback on my dissertation, I would not have been able to complete my study and to provide information for teacher educators about the effectiveness of an on-line professional development program. A very special “thank you” to Dr. Douglas Campbell, my classmates, and the Writing Center consultants. It is largely through their teaching, guidance, and critical feedback that I can organize my ideas better and improve my English gradually. I am deeply indebted to them for their generosity in sharing, reading and rereading my papers, and brainstorming ideas with me. Finally, I would like to thank my father, Li-Wei Kuo, my deceased mother, Shu-Hwa Qiu, and my sister, Nai-Ling Kuo, who always believe in me. Particularly, my father has been my best special education teacher. Although poliovirus has paralyzed half side of his body since he was little, his humor and fighting spirit always cheer me up when I am down. My father taught me that special education services are not just about what teachers can do for students, but also about how they provide these services to the students. He reminds me of being a humanistic educator who has a deep sense of responsibility to ensure that each student becomes happy without exception. vi       TABLE OF CONTENTS LIST OF TABLES………………………………………………………………………….. x LIST OF FIGURES………………………………………………………………………… xii CHAPTER ONE INTRODUCTION………………………………………………………………………….. Rationale for the Study………………………………………………………………….. The Logic Model of Current and Future Studies Related to RTI……………………….. Theoretical Framework………………………………………………………………….. An Overview of the Research Design…………………………………………………… 1 3 5 7 8 CHAPTER TWO LITERATURE REVIEW…………………………………………………………………... Teacher Knowledge Survey (TKS)……………………………………………………… The IRIS (IDEA ’04 and Research for Inclusive Settings) Modules……………………. The Essential Knowledge Needed for Implementing RTI………………………………. The Development of RTI…………………………………………………………….. RTI Components……………………………………………………………………... Content Knowledge in Reading……………………………………………………… The Processes of Implementing RTI…………………………………………………. Challenges in Implementing RTI…………………………………………………….. School-Wide Collaboration………………………………………………………….. Methodological knowledge in Reading……………………………………………… Cultural and Linguistic Diversity…………………………………………………….. Teacher Quality and Quality Teaching………………………………………………. Preservice Teacher Online Learning…………………………………………………….. 11 11 13 19 19 21 22 23 27 27 28 31 33 34 CHAPTER THREE METHODS………………………………………………………………………………….. Participants and the Setting……………………………………………………………… Data Collection Procedures and the Research Design…………………………………... Measures…………………………………………………………………………………. Pre-and Post-Survey Questionnaires…………………………………………………. Pre-and Post-Assessment Instruments……………………………………………….. Initial-and-Final Thoughts Questions………………………………………………... IRIS Module Assessment Questions…………………………………………………. Procedures of Fidelity Checklists and the ANGEL User Matrix…………………….. Intervention Conditions………………………………………………………………….. Comparison Conditions………………………………………………………………….. Data Analysis……………………………………………………………………………. Pre-and Post-Survey Questionnaires…………………………………………………. 36 36 37 42 42 43 47 48 49 52 54 56 56 vii       Pre-and Post-Assessment Instruments……………………………………………….. Initial-and-Final Thoughts Questions………………………………………………... IRIS Module Assessment Questions, Procedures of Fidelity Checklists and the ANGEL User Matrix………………………………………………………………….. CHAPTER FOUR RESULTS…………………………………………………………………………………… Grouping…………………………………………………………………………………. Equivalence Examination before the Intervention………………………………………. Attrition………………………………………………………………………………….. Participants’ Performance on the 120 RTI-Reading Knowledge Assessment Questions… Within-Group Comparison…………………………………………………………… Between-Group Comparison…………………………………………………………. Predictors and Participants’ Post-Assessment Outcomes……………………………….. Year in the Program………………………………………………………………….. Grade Point Average (GPA)…………………………………………………………. Groups (Experimental vs. Control)…………………………………………………... Pre-Assessment Outcomes…………………………………………………………… Participants’ Growth of Knowledge in Each Module…………………………………… Module 1-Classroom Assessment (Part 1): An Introduction to Monitoring Academic Achievement in the Classroom……………………………………………………….. Module 2-Classroom Assessment (Part 2): Evaluating Reading Progress…………... Module 3-RTI (Part 1): An Overview……………………………………………….. Module 4-RTI (Part 2): Assessment…………………………………………………. Module 5-RTI (Part 3): Reading Instruction…………………………………………. Module 6-RTI (Part 4): Putting It All Together……………………………………… Module 7-RTI (Part 5): A Close Look at Tier 3……………………………………... Module 8-RTI: Considerations for School Leaders………………………………….. Correlation of the Instruments…………………………………………………………… Fidelity of Implementation………………………………………………………………. Perspectives toward RTI………………………………………………………………… Advantages and Disadvantages of Using RTI……………………………………….. Purpose of RTI……………………………………………………………………….. Teachers’ Considerations When Working with Diverse Learners in an RTI Model… School Leaders’ considerations in Developing a Successful RTI Model in Their Building………………………………………………………………………………. The Importance of Teacher Quality in Implementing RTI…………………………... CHAPTER FIVE DISCUSSION AND CONCLUSION………………………………………………………. Knowledge Improvement through IRIS…………………………………………………. Predictors of Knowledge Improvement…………………………………………………. Variations in Learning…………………………………………………………………… User Satisfaction………………………………………………………………………… Perspectives toward RTI………………………………………………………………… Item Analysis……………………………………………………………………………. viii   58 58 58 60 61 63 64 67 67 68 70 70 71 72 72 75 75 76 77 78 78 79 80 81 84 87 93 94 96 97 98 98 100 102 111 114 119 120 122     Limitations of the Study…………………………………………………………………. Summary and Future Research Directions………………………………………………. 126 127 APPENDICES……………………………………………………………………………… Appendix A: Demographic Characteristics Survey……………………………………... Appendix B: Social Validity Questions…………………………………………………. Appendix C: Multiple-Choice Questions (IRIS and Literature Questions)……………... Appendix D: Open-Ended Questions Data Coding……………………………………… Appendix E: Initial-and-Final Thoughts Data Coding………………………………….. 129 130 131 132 145 147 BIBLIOGRAPHY…………………………………………………………………………… 155 ix       LIST OF TABLES Table 1: An Overview of IRIS Modules………………………………………………..…… 15 Table 2: The Distribution of the Multiple-Choice Questions……………………………...... 46 Table 3: Research Questions and Data Collection…………………………………………. 51 Table 4: The Eight IRIS-RTI Modules Used in the Experimental Group………………….. 53 Table 5: The Eight IRIS-RTI Modules Used in the Control Group………………………… 55 Table 6: Summary of Equivalency before Attrition………………………………………… 64 Table 7: Summary of Equivalency after Attrition………………………………………….. 66 Table 8: The Paired Samples Statistics of the Pre- and Post-Assessments within the Experimental Group……………………………………………………………….. 68 Table 9: The Independent Samples Statistics of the Pre- and Post-Assessment between the Experimental Group and the Control Group………………………………………. 69 Table 10: The Results of the Pre-Assessment (Year in the Program)………………………. 71 Table 11: The Between Group Variation and the Within Group Variation (Year in the Program)…………………………………………………………………………. 71 Table 12: The Means and Std. Deviations of the GPAs and Post-Assessment Outcomes…... 71 Table 13: The Correlations between GPAs and Post-Assessment Outcomes……………..… 72 Table 14: The Results of the Pre-Assessment and Post-Assessment Means and Std. Deviations ………………………………………………………………………... 73 Table 15: The Correlations between Pre-Assessment and Post-Assessment Outcomes…….. 73 Table 16: Summary of Hierarchical Regression Analysis for Variables Predicting PostAssessment Outcomes ………..…………………………………………………… 74 Table 17: Individual Participant Data from the Initial-and-Final Thoughts…………….….. 83 Table 18: Comparison of the Experimental Group’s Performance on the Multiple-Choice x       Questions (29 IRIS Test Items) and on the Initial-and-Final Thoughts Questions.. 85 Table 19: Report of the Social Validity Questions (Likert Scale)………………………….. 88 Table 20: Report of Users’ Comments on the IRIS Modules……………………………… 91 Table 21: Summary of Perspectives toward the Advantages of Using RTI……………..… 95 Table 22: Summary of Perspectives toward the Disadvantages of Using RTI……………. 96 Table 23: Summary of Perspectives toward the Purpose of RTI………………………….. 97 Table 24: Summary of Perspectives toward Teachers’ Considerations When Working with Diverse Learners in an RTI Model………………………………………………. 97 Table 25: Summary of Perspectives toward School Leaders’ considerations in Developing a Successful RTI Model in Their Building………………………………………. 98 Table 26: Summary of Perspectives toward The Importance of Teacher Quality in Implementing RTI……………………….……………………………………….. 99 Table 27: Participants’ Initial-and-Final Thoughts Scores in Each Module……………….. 108 Table 28: Summary of Factor Loading for the 120 RTI-Reading Knowledge Assessment (N=81)..……………………………………………………………………........... 124 Table 29: Initial-and-Final Thoughts Data Coding (Module 1)…………………………….. 147 Table 30: Initial-and-Final Thoughts Data Coding (Module 2)…………………………….. 148 Table 31: Initial-and-Final Thoughts Data Coding (Module 3)…………………………….. 149 Table 32: Initial-and-Final Thoughts Data Coding (Module 4)…………………………….. 150 Table 33: Initial-and-Final Thoughts Data Coding (Module 5)…………………………….. 151 Table 34: Initial-and-Final Thoughts Data Coding (Module 6)…………………………….. 152 Table 35: Initial-and-Final Thoughts Data Coding (Module 7)…………………………….. 153 Table 36: Initial-and-Final Thoughts Data Coding (Module 8)…………………………….. 154 xi       LIST OF FIGURES Figure 1: The logic model of current and future studies related to RTI……………………….. 6 Figure 2: Example of the three-tier model……………………………………………………….. 24 Figure 3: The ANGEL web pages………………………………………………………………... 38 Figure 4: Data collection procedures…………………………………………………………….. 40 Figure 5: Procedures of fidelity checklist……………………………………………………… 50 Figure 6: The procedures of grouping……………………………………………………………. 62 Figure 7: Subgroups in the control group and in the experimental group……………………….. 63 Figure 8: The concept of the equivalency examinations before and after attrition………………. 65 Figure 9: Participants’ pre-post-total mean scores on the multiple-choice questions…………. 104 Figure 10: The mean scores on the Initial-and-Final Thoughts questions……...………………. 106 Figure 11: Variation analogy…………………………………………………………………….. 115 Figure 12: Variations in individual participants…………………………………………………. 117 Figure 13: Screen plot of the 120 RTI-Reading Knowledge Assessment Questions ..…………... 123 xii       CHAPTER ONE INTRODUCTION Response-to-intervention (RTI) is “a multi-tier approach to the early identification and support of students with learning and behavior needs” (RTI Action Network, 2013). RTI began to be recognized around 2004, when the Individuals with Disabilities Education Act (IDEA) was reauthorized. IDEA 2004 makes it clear that “in determining whether a child has a specific learning disability, a local education agency may use a process that determines if a child responds to scientific, research–based intervention as a part of the evaluation procedures used to determine if the child is a child with a disability” (P. L. 108-446 § 614(b) (6)). The law establishes that schools are not required to use the IQ-discrepancy model to identify whether a student has a learning disability (IDEA 2004). Such changes are partly due to the fact that the problems inherent in traditional identification methods or so-called “wait-to-fail” methods have been explored and well-documented in the literature (Berninger, 2001; Fletcher et al., 1994; Francis, Fletcher, Shaywitz, Shaywitz, & Rourke, 1996; Reschly & Yssldyke, 2002; Scruggs & Mastropieri, 2002; Siegel, 2003; Stage, Abbott, Jenkins, & Berninger, 2003; Vaughn & Fuchs, 2003). The language of the law has permitted the development of RTI. With the support of the federal law and state policies, RTI has become one of the most important and pervasive program interventions in the United States. According to a recent report provided by the National Center on Response to Intervention [NCRTI] (2013), 42 states have official RTI websites; 34 states have RTI guidance documents; all 50 states allow schools to use RTI for full or partial specific learning disability (SLD) determination; 32 states have state professional development grants for RTI; and 47 states have RTI components in their state performance plans. Although RTI implementation has been growing quickly in U.S. public 1       schools since 2004, debates about whether RTI can be used as a means to identify students with special needs and how RTI can be combined with the methods used in current school systems should not be overlooked (Fuchs, Fuchs, & Compton, 2012). To make RTI work effectively, there is an important and urgent need to prepare preservice teachers with RTI knowledge and skills early on, so they can implement RTI with integrity when they enter the field. Many scholars have suggested that colleges of education should prepare their preservice teachers to participate in schools where RTI is implemented (Brownell, Sindelar, Kiely, & Danielson, 2010; Reschly & Wood-Garnett, 2009). In fact, the National Council for Exceptional Children (CEC) has drafted standards for the position of “academic interventionist” and “behavioral interventionist” that correspond to the advances in Response to Intervention and Positive Behavioral Intervention and Supports, respectively. Several organizations, including the National Early Childhood Technical Assistance Center [NECTAC], have combined RTI and PBIS and termed these “Multi-Tiered Systems of Support” or MTSS (NECTAC, 2012). This study thus began to provide insight on whether online professional development learning modules could positively impact preservice teacher learning regarding RTI in the academic domain of reading. The rationale for and the theoretical framework of the study are discussed in the following sections, followed by an over view of the study and the logic model of current and future studies related to RTI. 2       Rationale for the Study As the National Center on Response to Intervention [NCRTI] (2013) notes: Rigorous implementation of RTI includes a combination of high quality, culturally and linguistically responsive instruction; assessment; and evidence-based intervention. Comprehensive RTI implementation will contribute to more meaningful identification of learning and behavioral problems, improve instructional quality, provide all students with the best opportunities to succeed in school. Because RTI is such a complex and multi-faceted approach, having increased exposure to RTI at an early stage in teacher education is beneficial to preservice teachers. With the development of the Internet, online learning provides an opportunity for preservice teachers to learn across time and distances. Thus, online learning has been widely used in teacher preparation programs (Caywood & Duckett, 2003; Harrell & Harris, 2006; Tallent-Runnels et al., 2006). However, though using online programs for professional development is recommended by scholars (Billingsley, Israel, & Smith, 2011), and teacher educators are increasingly disposed to use online training as part of their preparation of preservice teachers, relatively little is known about whether or how preservice teachers might learn in online learning environments. On the other hand, there are several government-sponsored online professional development programs available for preparing teachers around RTI, such as IRIS Modules, LD 1 Online, RTI Action Network, and NCRTI . However, there is limited literature addressing how effective these online programs are in helping preservice teachers better understand RTI.    In this study, the most popular online learning program in the United States for special educators, the                                                                                                                           1 IRIS Modules (iris.peabody.vanderbilt.edu), LD Online (www.ldonline.org), RTI Action Network (www.rtinetwork.org), and National Center for Response to Intervention (www.rti4success.org) 3       IRIS modules developed by the IDEA ’04 and Research for Inclusive Settings (IRIS) Center at Vanderbilt University as part of a grant from the Office of Special Education Program (OSEP) was chosen for study. The IRIS modules have grown from 8,122 users in the fall of 2004 to 473,413 users in the spring of 2011 (IRIS Center, 2012). The modules also serve as a major way to provide college faculty, who are involved in the preservice preparation of general and special education teachers, with professional development about the instruction and special needs of students with disabilities (IRIS Center, 2012). With more and more IRIS modules being embraced by colleges and universities for providing professional development training, it is important and potentially helpful for teacher educators to understand the impact of these modules before incorporating them into their teacher preparation programs. In particular, the content area within IRIS specific to RTI is among the most developed set of modules in IRIS. This study sought to develop a knowledge measure that encompassed the content of the eight IRIS-RTI modules but also went significantly beyond the IRIS content modules to include two additional areas that were only peripherally addressed in the IRIS-RTI modules but were thought to be central to a comprehensive understanding of RTI. Specifically, a set of questions from the Teacher Knowledge Survey (TKS) that addressed knowledge of RTI and basic reading instruction was included. A comprehensive review of the literature also indicated that two additional areas, teacher quality and culturally relevant instruction, were essential areas of understanding and so were added to the content covered by the eight IRIS-RTI modules (e.g., structural elements of implementing RTI, key principles, assessment). The TKS was included as part of the comprehensive assessment of RTI knowledge for several reasons. First, the National Reading Panel recommended knowledge in five areas in 4       reading, including phonological and phonemic awareness, phonics, vocabulary, fluency, and comprehension. The TKS particularly focuses on the five areas of reading. Second, the role of special education teachers in RTI, the population of preservice teachers in this study, emphasizes the provision of intensive remedial instruction for the most at-risk learners using evidence-based interventions. In the early grades, this often means providing intensive teaching on early reading skills devoted to basic reading, including those areas outlined by the National Reading Panel. Because special education teachers play an important role primarily in the delivery of Tier 2 and 3 intensive interventions, usually around basic reading, including the TKS test items that assesses knowledge in each of these areas is important. Similarly, the questions added around teacher quality and culturally relevant instruction reflected advances in our understanding of teacher effectiveness and the increasing diversity of our school population (e.g., language, economic, cultural, ability, orientation). As was said in the NCRTI statement, “Rigorous implementation of RTI includes a combination of high quality, culturally and linguistically responsive instruction; assessment; and evidence-based intervention.” Any advancements in RTI must consider the roles of teacher quality, quality teaching, and culture in its enactment as well. The Logic Model of Current and Future Studies Related to RTI This study examined the impact of eight IRIS-RTI modules on preservice teachers’ knowledge of RTI in the academic domain of reading and on their perspectives toward RTI. This study is the first in what will hopefully be a multi-step progression in exploring the impact of IRIS modules on preservice teacher preparation for RTI. Several studies can be extended in the future based on the present study. Figure 1 shows the logic model of current and future studies related to RTI: 5       The impact of eight IRIS-RTI modules (a) Preservice teachers’ knowledge of RTI-Reading Preservice teachers’ perspectives towards RTI   Preservice teachers’ teaching effectiveness (d)   (e) (b) (c) University-based teacher preparation program(s) Current study Future studies Students’ learning outcomes and behavior (f) Environmental factors Figure 1. The logic model of current and future studies related to RTI Figure 1 displays that the present study accomplished the first two goals: (a) the impact of IRIS on preservice teachers’ knowledge of RTI in the academic domain of reading and on their perspectives toward RTI, and (b) how preservice teachers’ knowledge of and perspectives toward RTI inform university-based teacher preparation programs. In future studies, it will be important to explore (c) how university-based teacher preparation programs respond to preservice teachers’ needs with respect to using RTI (e.g., coursework, workshops, and training related RTI); (d) what preservice teachers’ teaching effectiveness is after using the IRIS-RTI modules; (e) what students’ learning outcomes and behavior are after their teachers implement RTI; and (f) how environmental factors add challenges to the RTI approach concerning students’ learning outcomes and behavior. 6       Theoretical Framework The RTI approach includes two domains: the academic domain and the behavioral domain. The integration of these two domains, now commonly referred to as Multi-Tiered Systems of Support (MTSS), is significant to help all students maximize their potential (Hallahan, Kauffman, & Pullen, 2012; Hawken, Vincent, & Schumann, 2008; Stewart, Benner, Martella, & Marchand-Martella, 2007). When student’s academic performance is improved, the student’s behavior problems often decrease, and vice versa (Lassen, Steele, & Sailor, 2006). However, before integrating the two domains, it is worth exploring if preservice teachers have the essential knowledge in each domain. This study thus focused on the academic domain, particularly in reading. Regarding the essential knowledge for implementing RTI in the academic domain of reading, it is important that preservice teachers equip themselves with three categories of knowledge: content knowledge (know what it is), methodological knowledge (know how to do it), and contextual knowledge (know how to adjust it accordingly). In terms of content knowledge, preservice teachers need to know about the development of RTI, RTI components, and content knowledge in reading (see Fuchs, Mock, Morgan, & Young, 2003, Reschly & Hosp, 2004, Fuchs & Fuchs, 2006, NCRTI, 2012, Vaughn & Fuchs, 2003). In terms of methodological knowledge, preservice teachers need to have knowledge about the processes of implementing RTI, challenges in implementing RTI, school-wide collaboration, and methodological knowledge in reading (see Mellard, Byrd, Johnson, Tollefson, & Boesche, 2004, Murawski & Hughes, 2009, Cook, Shepherd, Cook, & Cook, 2012, National Research Center on Learning Disabilities, 2006, Fuchs, Fuchs, & Compton, 2004). Finally, regarding contextual knowledge, it is important that preservice teachers are aware of cultural and linguistic diversity as well as teacher quality and 7       quality teaching when implementing RTI (see Rinaldi & Samson, 2008, Klingner & Edwards, 2006, Kratochwill, Volpiansky, Clements, & Ball, 2007, Reschly & Wood-Garnett, 2009, Brownell et al., 2010, Kennedy, 2008). One measure used in this study was based on the 120 RTI-Reading Knowledge Assessment Questions, consisting of three test resources (i.e., 66 TKS test items; 29 IRIS test items; 25 Literature test items), which included all the three categories of knowledge: content, methodological, and contextual knowledge. An Overview of the Research Design This study applied a quasi-experimental approach to examine a set of IRIS modules around RTI. Both quantitative and qualitative methods were used to evaluate the impact of IRIS modules on preservice teachers’ knowledge of RTI in the academic domain of reading and on their perspectives toward RTI. A total of 55 preservice teachers enrolled in a special education teacher preparation program at a large Midwest public university voluntarily participated in this study and they all followed the steps to complete the tasks. The first hypothesis in this study was to test whether the participants’ performed differently on the 120 RTI-Reading Knowledge Assessment Questions before and after the intervention, and then a secondary analysis to examine the three test resources within the multiple choice questions, including detailed knowledge of reading interventions as measured by the TKS (questions 1-66), the knowledge directly taught in the IRIS-RTI modules (questions 67-95), and then questions related to teacher quality and culturally relevant instruction in RTI (questions 96-120). The second hypothesis was to test whether the differences in participants’ post-assessment outcomes could be explained by a set of predictors (e.g., GPA or year in the program). The third hypothesis was to examine whether there were variations in the participants’ online learning. The fourth hypothesis was to know if the experimental group’s performance on the 120 RTI-Reading Knowledge Assessment Questions 8       and their growth of knowledge in each module were correlated. The fifth hypothesis was based on a social validity survey to examine whether the treatment was delivered as intended. Finally, this study explored whether or not the participants’ perspectives toward RTI changed after the intervention. The research questions of this study were driven by these hypotheses. To this end, this study addressed the following questions: 1. What was the participants’ performance on the 120 RTI-Reading Knowledge Assessment Questions before and after the intervention, in terms of TKS, IRIS, and Literature? 2. To what extent did a particular set of predictors explain the differences in participants’ post-assessment outcomes (i.e., year in the program, GPAs, groups, and preassessment outcomes, etc.)? 3. How did the experimental group’s knowledge of RTI-Reading grow in each module? 4. How correlated were the experimental group’s performance on the 120 RTI-Reading Knowledge Assessment Questions and their growth of knowledge in each module? 5. Was the treatment delivered as intended? That is, how useful or not useful did participants find the modules concerning the improvement of their RTI knowledge? 6. What were the participants’ perspectives toward RTI? In summary, by examining the effectiveness of a set of IRIS-RTI modules, this study helps teacher educators understand how preservice teacher candidates learn about RTI in self-directed online professional development modules. Additionally, by using a more comprehensive measure, consisting of TKS, IRIS, and Literature test items, this study helps teacher educators understand whether using the eight modules for one-time exposure was sufficient to cultivate 9       preservice teachers’ knowledge of RTI-Reading and to what extent. This quasi-experimental study, combined with mixed-methods, provides teacher educators with a more complete picture about the impact of IRIS modules on preservice teacher learning. While special education teachers continue to play a critical role in the RTI process, such as to “(a) provide direct services to students receiving Tier 3 instruction, and (b) collaborate with general education colleagues to provide Tier 2 instruction” (Brownell et al., 2010, p. 372), this study should represent a significant addition to the existing literature with respect to university-based teacher preparation for RTI in the academic domain of reading and the impact of one government-sponsored online professional development program for RTI. This study should be of particular interest to teacher educators who are preparing special education preservice teachers for RTI and/or are considering incorporating IRIS modules into teacher preparation programs. 10       CHAPTER TWO LITERATURE REVIEW This chapter reviews the existing literature in three areas: (a) the Teacher Knowledge Survey (TKS), (b) IRIS modules, and (c) the essential knowledge needed for implementing RTI. The review of the literature provides a theoretical foundation for conducting this study. In addition, through the review of TKS, IRIS modules, and the essential knowledge needed for implementing RTI, this chapter provides the reasons for why the measure, consisting of TKS, IRIS, and Literature test items, was used as part of the instrument in this study. Furthermore, research on preservice teacher online learning was reviewed to help teacher educators understand how preservice teachers learn through online technology in comparison to face-to-face instructional methods. Teacher Knowledge Survey (TKS) The Teacher Knowledge Survey (TKS), consisting of 66 multiple-choice questions, was developed by Dr. Louise Spear-Swerling, Professor of Special Education and Reading and Area Coordinator of the Graduate Program in Learning Disabilities at Southern Connecticut State University in New Haven. The TKS “was modeled after the multiple-choice section of a teacher licensure exam called the Foundations of Reading Test, published by Evaluation Systems Group of Pearson” (Spear-Swerling & Cheesman, 2011). To evaluate 140 elementary-level teachers’ knowledge for RTI and reading, Spear-Swerling and her colleagues used the TKS for their study. The TKS includes questions in three areas: five components of reading, assessment, and RTI. Spear-Swerling and Cheesman (2011) described the contents of their survey as follows: 11       All questions provided a stem, four specific answer choices, and a fifth option, ‘‘I don’t know.’’ Each question had only one correct answer. Questions involved both content knowledge (about 33% of items) and application (about 67% of items). Content knowledge items primarily assessed whether a participant understood an important construct or important interrelationships in reading (e.g., What is phonemic awareness? Why is fluency important to reading comprehension?). Application items briefly described a child, teaching situation, or school problem, and asked the participant what he or she would do to assess the child’s reading, teach a particular component of reading, or implement RTI practices. (Spear-Swerling & Cheesman, 2011, pp. 11-12) In short, the TKS is mainly focused on evaluating teachers’ content and methodological knowledge of reading, assessment, and RTI. Regarding the internal consistency of these test items, Spear-Swerling and Cheesman (2011) reported Cronbach’s alpha in the following: Due to relatively small numbers of items in individual categories, reliabilities for many individual categories were below .70. Thus, based on conceptual and theoretical considerations as well as patterns of correlations, items from various categories were grouped into the following three subscales: phonemic awareness/ phonics (PA/PH), number of items = 17, Cronbach’s alpha = .71; fluency, vocabulary, and comprehension (FLU/VOC/COMP), number of items = 24, alpha = .71; and assessment/RTI (AS/RTI), number of items = 25, alpha = .77. Cronbach’s alpha for the entire survey, all 66 items, was .88. (Spear-Swerling & Cheesman, 2011, p. 12) A Cronbach’s alpha above .70 indicates that the internal consistency of the test items is acceptable. That is to say, the test items grouped in each category of the TKS are statistically acceptable. The results of Spear-Swerling and Cheesman’s (2011) study, using this survey to collect data from 140 elementary-level teachers, showed that “many participants not only lacked important pedagogical content knowledge for teaching reading, but they also were unfamiliar with research-based instructional programs and interventions that could serve as valuable resources for them in implementing RTI” (p. 28). In this regard, they argued that teacher preparation programs should highlight pedagogical content knowledge in reading and evidence- 12       based instruction. To investigate the participants’ knowledge of RTI in the academic domain of reading before and after the intervention, the TKS was selected for the present study. Recently, Dr. Spear-Swerling (personal correspondence, 2013) updated the reliabilities (Cronbach alphas) of her instrument, which was based on additional data collected with 390 participants in summer 2012. For the latest data set (n= 390), the alphas across each individual category in the TKS (phonemic awareness, phonics, fluency, vocabulary, comprehension, assessment, and RTI items) all exceeded the original sample (n=140), with the lowest alpha being .85 (for the phonics items) and the highest .93 (for the RTI items). The reliabilities of the instrument in the recent study were higher than the original study, strengthening claims about the internal consistency of the items. The IRIS (IDEA ’04 and Research for Inclusive Settings) Modules The IRIS modules are developed by the IDEA ’04 and Research for Inclusive Settings (IRIS) Center at Vanderbilt University. The IRIS Center, funded by the U.S. Department of Education’s Office of Special Education Programs (OSEP), is to provide “high-quality resources for college and university faculty and professional development providers about students with disabilities.” (IRIS, 2012). As of 2013, the IRIS Center has developed a total of 53 modules for public use. These modules are categorized into different topics by the IRIS Center. Some modules are overlapped across topics. Table 1 displays the topics of IRIS modules. It is worth noting that because it was difficult for the participants of the study to complete all IRIS modules within the two to three months, only eight IRIS-RTI modules (out of ten modules) in the domain of reading were used for the present study. It is possible that the participants would have done better on the TKS if they also completed all IRIS modules around “Reading, Literacy, and Language Arts” (nine additional modules), as well as the other modules. However, due to the 13       time constraint and some overlapping modules across topics, it was meaningful to examine if the eight IRIS-RTI modules in the domain of reading were sufficient to help preservice teachers understand RTI in the domain of reading. If not, the other modules may be spread out throughout their teacher preparation programs in different courses, such as literacy methods and cultural diversity. 14       Table 1 An Overview of IRIS Modules Topics Accommodations Assessment Assistive technology Behavior and classroom management Collaboration Content instruction Differentiated instruction Disability Diversity Learning strategies Math RTI Reading, literacy, language arts Related services School improvement/ leadership Transition Grades: Pre K - 3 Grades: 4 - 8 Grades: High school Modules Accessing the General Education Curriculum, Accommodations, etc. (12 modules) Accessing the General Education Curriculum, Accommodations, etc. (9 modules) Assistive Technology, Bookshare (2 modules) Classroom Management, You're in Charge, etc. (6 modules) Accommodations to the Physical Environment, Addressing the Revolving Door (11 modules) CSR, High-Quality Mathematics Instruction, etc. (9 modules) CSR, Differentiated Instruction, etc. (8 modules) Accommodations to the Physical Environment, Instructional Accommodations, etc. (5 modules) Cultural and Linguistic Differences, Teaching and Learning in New Mexico, etc. (4 modules) CSR, Improving Writing Performance, etc. (11 modules) High-Quality Mathematics Instruction, RTI: Mathematics (2 modules) RTI (Part 1): An Overview, RTI (Part 2): Assessment, etc. (10 modules) Note: Two modules about mathematics were not used in the present study. Classroom Assessment (Part 2), CSR, etc. (9 modules) Assistive Technology, Guiding the School Counselor, etc. (6 modules) Accessing the General Education Curriculum, Accountability, etc. (12 modules) School Counselors (1 module) Accessing the General Education Curriculum, Accommodations, etc. (33 modules) Accessing the General Education Curriculum, Accommodations, etc. (30 modules) Accessing the General Education Curriculum, Accommodations, etc. (29 modules) 15       Each module (shown in Table 1) meets different teacher preparation standards for educating diverse learners, such as the Council for Exceptional Children (CEC) standards, the Interstate Teacher Assessment and Support Consortium (InTASC) standards, and the National Council for Accreditation of Teacher Education (NCATE) standards. Each module consists of five components: • • • • • Challenge – a realistic scenario relevant to education professionals Initial Thoughts – questions that allow students to explore and consider what they currently know about the scenario presented in the Challenge Perspectives and Resources – nuggets of information (e.g., text, movies, audio interviews, activities) that allow students to actively engage in learning the module's main content Assessment – an evaluation tool that offers students the opportunity to apply what they know and to evaluate what topics they need to study further Wrap Up – a summary of the information presented in the previous components (IRIS Center, 2013a) These five components are developed based upon an evidence-based cycle of learning activity (IRIS Center, 2013a). During the fall of 2011, IRIS Center staff and Montrosse (2012) conducted a survey regarding who was using IRIS modules, how they rated these modules, and to what extent these modules influenced their practices and programs. Six hundred and ten users completed the survey, and one-third to one-half of faculty rated the IRIS modules as “very useful” (Montrosse, 2012). To evaluate the effectiveness of IRIS modules, the IRIS Center encouraged college and university faculty to conduct field-testing on the modules. The procedures were: “During field-testing, college and university faculty use one [emphasis added] of the IRIS Modules as part of their instruction. They then distribute and collect a survey, created by the IRIS Center, from their students to assess their reactions to the module” (IRIS Center, 2013b). In this document, the IRIS Center made it clear that “A module is considered to be fieldtested when at least fifty students have responded to the survey.” As of June 2011, twenty-two of 16       the IRIS Modules were field-tested (IRIS Center, 2013b). More detailed information about the faculty participants and the procedures are shown in the following: Faculty participants Thirty-four faculty at fifteen colleges and universities around the country participated in the field-testing of IRIS Modules. Field-testers were chosen from among those faculty who expressed interest in field-testing during their participation as an IRIS research site, an IRIS implementation site, or in response to an IRIS listserv announcement. Faculty told the field- testing coordinator which IRIS Module(s) they were planning to use in their upcoming courses, and the coordinator used this information to assign modules to faculty for field-testing. Each field-tester was paid $250 for his or her time. Participating faculty were asked to: 1. 2. 3. 4. Use the IRIS Module as one of their course requirements. Distribute and collect module survey forms from the students in the class. Complete a faculty survey form. Send all materials to the IRIS Center. (IRIS Center, 2013b) The field test data was collected from a total of 1,744 preservice teachers. The majority of the preservice teachers were in general education (71.7%); the others were in special education (9.5%), counseling (2.5%), psychology (0.9%), and other areas of study. The results show that “the majority of students responding to the survey felt they had learned something from the module,” and “most respondents rated the module as being of high quality and relevant” (IRIS Center, 2013b). Another two IRIS module studies were conducted during the 2004-2005 and 2005-2006 academic years. Both of the studies only examined one single module in relation to students’ learning outcomes. In the first study, a total of 620 students were assigned to a module group and a non-module group, respectively. The study was to examine the participants’ performance on the Initial Thoughts questions (as a pretest instrument) and on the Final Thoughts questions (as a 17       posttest instrument). The responses were scored. “To perform well, students would need to apply content that was covered by the text and/or the module” (IRIS Center, 2013b). The results indicated that “the average posttest score for students who viewed the module was significantly higher than for students who did not” (IRIS Center, 2013b). In the second study, a total of 480 students were assigned to an Independently Viewed group and the Instructor-Enhanced group. Both groups received multiple-choice and open-ended questions. The results show that “although students did gain in their factual knowledge about self-regulation [in both conditions], more involvement by the instructor did not result in enhanced performance” (IRIS Center, 2013b). Due the fact that both studies do not report the procedures of their grouping, the numbers in each group, the backgrounds of the participants, and the instrument questions being asked, it is difficult to determine how effective each of the modules was. In addition, to hold IRIS modules accountable, it is important to report whether the experimental group and the control were equivalent before they took the modules. While some of the other modules continue to be embedded in coursework in different universities, and instructors and students consider the modules to be practical and helpful (e.g., Rodriguez, Gentilucci, & Sims, 2006; Smith et al., 2005), there are limited experimental or quasi-experimental studies that used a set of IRIS-RTI modules. Therefore, this study attempted to provide information about what the participants’ actual performance was and how their perspectives toward RTI changed after using a set of eight IRIS-RTI modules. Detailed information about another eight IRIS modules, selected for the control group in this study, is addressed in Chapter 3. 18       The Essential Knowledge Needed for Implementing RTI The essential knowledge needed for implementing RTI-Reading should contain at least three areas: content knowledge, methodological knowledge, and contextual knowledge. Content knowledge includes the development of RTI, RTI components, and content knowledge in reading. Methodological knowledge refers to the processes of implementing RTI, challenges in implementing RTI, and school-wide collaboration. Contextual knowledge includes cultural and linguistic diversity as well as teacher quality and quality teaching. While these three areas of knowledge are equally important, they are often intertwined with each other and cannot be clearly distinguished. For example, when a teacher adjusts his or her ways to implement RTI in order to meet the needs of diverse students, this may cover both methodological and contextual knowledge of RTI. Similarly, any single component of RTI has content, methodological, and contextual implications. Understanding the importance of universal screening and choosing appropriate assessments (content knowledge), being able to implement and score assessments (methodological knowledge), and adjust assessments for English language learners (contextual knowledge) are all germane to the component “universal screening”. In this study, the purpose of categorizing different types of RTI knowledge is not to regard them as separate or independent items. Rather, it is to ensure that all these three categories of knowledge were properly included and loaded in the 120 RTI-Reading Knowledge Assessment Questions utilized in this study. These important terms are reviewed in the following. The Development of RTI Both the No Child Left Behind Act (NCLB) of 2002 and the Individuals with Disabilities Education Act (IDEA) of 2004 have heightened attention to teacher preparation and have increased the need for both general and special education teachers to provide early 19       intervention for students with special needs. For example, IDEA makes it clear that using the IQachievement discrepancy model is no longer required for identifying students’ learning disabilities, and “in determining whether a child has a specific learning disability, a local education agency may use a process that determines if a child responds to scientific, research– based intervention as a part of the evaluation procedures used to determine if the child is a child with a disability” (P. L. 108-446 § 614(b) (6)). NCLB (2002) also establishes that to improve the education of all students, schools need to provide high-quality teachers, develop assessments, and conduct progress monitoring for their students. These provisions apply to all students, including those with disabilities (NCLB, 2002). In short, the language of the federal legislation has permitted the development of RTI. According to IDEA, schools are no longer required to use the IQ-achievement discrepancy model to identify whether a student has a learning disability, given the fact that the IQ-achievement discrepancy model has been criticized on several major grounds. First, students cannot receive special education services until their academic achievement outcomes are sufficiently below their IQ scores (Fuchs et al., 2003). Second, the discrepancy model does not consider contextual factors, such as students’ socio-cultural backgrounds, and thus it often contributes to the result of a disproportionate rate of minority students placed in special education (Donovan & Cross, 2002). Third, there are many problems inherent in this traditional identification method, such as inconsistent results between students’ IQ scores and their academic achievement in school (Fletcher et al., 1994). Because of these limitations, the IQachievement discrepancy model is not recommended as a single means for identifying students’ learning disabilities (Francis et al., 1996; Fuchs et al., 2003; Siegel, 2003). 20       The problems inherent in the IQ-achievement discrepancy model have led researchers and educators to find alternative ways for identifying students with learning disabilities. Among many alternative approaches for identifying students with learning disabilities as well as other special needs, RTI has gained attention from scholars and national research centers, such as the Division for Learning Disabilities, the Council for Exceptional Children, and the Office of Special Education Programs in the U.S. Department of Education (Fuchs et al., 2003). In accordance with federal laws and regulations (e.g., evidence-based practices and high-quality instruction), RTI implementation is growing quickly across U.S. public schools (NCRTI, 2012). Beside federal laws and regulations, state educational policies also play an important role in determining how schools will identify whether a student has a learning disability and how interventions will be provided (Reschly & Hosp, 2004). RTI Components To compensate for the weaknesses found in the IQ-achievement discrepancy model, RTI has been developed to provide early and research-based intervention for students who are at-risk, are exceptional, or have special needs. Research-based intervention is important because it often applies “(a) explicit instruction with modeling, (b) systematic instruction with scaffolding, (c) multiple opportunities for practice, (d) immediate corrective feedback, and (f) ongoing monitoring of progress” (Smartt & Reschly, 2007, p. 6). In addition, RTI is considered as a way to reduce the number of students later placed in special education because of early identification and intervention (Fuchs & Fuchs, 2006). Through universal screening, a multi-level prevention system, progress monitoring, and data-based decision making, RTI offers an instructional model that benefits all students with and without special needs (NCRTI, 2012). 21       Content Knowledge in Reading To provide effective reading interventions, teachers need to have strong content knowledge (e.g., phonemic awareness, phonics, fluency, vocabulary, and reading comprehension) in order to reduce the possibilities of misjudging students with reading difficulties. SpearSwerling (2008) argued that reading teachers must be well-prepared with in-depth pedagogical content knowledge, such as phonemic awareness, phonics, word decoding, word spelling, English patterns, and reading fluency. She stated that if teachers themselves cannot distinguish the differences between phonemic awareness and phonics, or lack knowledge about word spelling, English patterns, and reading fluency, it will be difficult for them to accurately assess students’ reading difficulties and to provide appropriate interventions (Spear-Swerling, 2008). She used Dutch as an example to address the importance of teachers’ knowledge of word spelling. She stated, “A youngster who misspells Dutch as Duch is not failing to hear the phoneme /t/ or lacking phonemic awareness but instead is probably unfamiliar with a specific spelling convention” (p. 281). Teachers who lack knowledge of spelling words may misjudge students’ weaknesses, so they may provide them with inappropriate interventions even if the interventions are evidence-based. Moreover, Spear-Swerling (2008) criticized popular reading instruction that overemphasizes reading strategies, such as using contextual cues, in fluency instruction. SpearSwerling (2008) argued that skilled readers usually do not rely on contextual cues, such as pictures. Instead, they “pay close attention to all the letters in words but generally do so automatically and without conscious effort” (p. 279). In contrast, poor readers typically use this strategy to compensate for their limited comprehension rather than to read words carefully. Spear-Swerling (2008) used these examples to highlight the importance of teachers’ knowledge 22       in reading and suggested that pedagogical content knowledge should be included in teacher licensure examinations (Spear-Swerling, 2008). The promise of RTI includes identifying students “based on risk rather than deficit, early identification and instruction, reduction of identification bias, and linkage of identification assessment with instructional planning” (Vaughn & Fuchs, 2003, p. 137). Klotz and Mellard (2008) argued that RTI has potential advantages if it is implemented with rigorous processes and fidelity: • All students receive high-quality instruction in their general education setting. • All students are screened for academics and behavior and have their progress monitored to pinpoint specific difficulties. • At-risk students do not have to wait before receiving additional instructional assistance, including special education if needed. • Critical information is provided about the instructional needs of the student, which can be used to create effective educational interventions. • Unnecessary testing that has little or no instructional relevance is limited. • All students receive appropriate instruction, particularly in reading, prior to placement in special education. (Klotz & Mellard, 2008) Although RTI has these potential benefits, adequately implementing RTI remains a major concern for teachers as well as researchers. In the following sections, the aspects of RTI methodological knowledge are discussed. The Processes of Implementing RTI Typically, RTI is represented by a three-tiered triangle model with Tier 1 represented as green, Tier 2 as yellow, and Tier 3 as red (See Figure 2). According to leading RTI scholars (e.g., Fuchs and Fuchs, 2006), all students receive differentiated instruction and evidence-based instruction provided by general education teachers in Tier 1. It is expected that Tier 1 can meet 80 to 85 percent of students’ needs in general classes [the percent is slightly different in different 23       RTI models]. Students who do not appropriately respond to Tier 1 instruction will be provided with more intensive strategic and evidence-based interventions within small groups in Tier 2. Depending on school budgets and resources, Tier 2 can be conducted by general education teachers who have been trained in RTI or conducted by intervention specialists (e.g., subject specialists, paraprofessionals, Title I teachers, or special education teachers) within or outside the general classroom. It is expected that approximately 10 to 15 percent of students who do not adequately respond to Tier 1 instruction should make appropriate progress in Tier 2. Those who still fall significantly behind their peers will be provided with the most intensive interventions in Tier 3, which are tailored to meet the specific needs of students (Fuchs & Fuchs, 2006). Tier 3 (Red Area) Tier 2 (Yellow Area) Tier 1 (Green Area) Figure 2. Example of the three-tier model. 24       It is important to note that more intensive interventions (i.e., Tiers 2 and 3) are only supplements to previous tiered interventions and do not replace initial interventions. For example, John, a struggling reader may receive 90 minutes of core reading curriculum in his general class, but also receives an extra 30 minutes of reading interventions in a small group twice a week within or outside his general class. If appropriate progress is not shown in small group instruction, he may receive the most intensive interventions in one-on-one or tailored instruction for 30 minutes five days a week, focusing on specific skills, such as phonics or word recognition. The intervention time is not fixed and decided by John’s IEP team, and John can move across tiers based on his progress. In terms of assisting students who are struggling with reading in the primary grades, Gersten et al. (2009) offered five recommendations concerning RTI and multi-level interventions. First, for early implementation of Tier 1 intervention, schools should screen all students for potential reading difficulties at least two times a year and regularly monitor these students’ progress. Second, based on the result of the reading assessments, general education teachers should provide differentiated reading instruction for students. Third, for those who rank below the benchmark score on universal screening and do not make appropriate progress at Tier 1 intervention, school teams should develop “intensive, systematic instruction on up to three foundational reading skills in small groups” for those students (Gersten et al., 2009, p. 6). Fourth, school teams should monitor the progress of these students who are at Tier 2 intervention. School teams should use data to determine whether these students should still continue the same intervention or whether a more intensive intervention plan (e.g., Tier 3 intervention) should be developed. Finally, Tier 3 intervention should be provided to those who show no or minimal progress at Tier 2 intervention. Such interventions are provided “on a daily basis that promotes 25       the development of the various components of reading proficiency to students” (Gersten et al., 2009, p. 6). In terms of delivering interventions, standard treatment protocol and problem solving are two basic approaches used to deliver interventions within an RTI model (Fuchs et al., 2003). The standard treatment protocol requires schools to use the same standards and empirically validated treatments to help students with special needs succeed, while the problem solving approach is more sensitive to individual differences (Fuchs et al., 2003). In problem solving, “Practitioners determine the magnitude of the problem; analyze its causes; design a goal directed intervention; conduct it as planned; monitor student progress; modify the intervention based on student responsiveness; and evaluate its effectiveness and plot future actions” (Fuchs et al., 2003, p. 163). Even if a problem solving approach is more flexible than the standard treatment protocol, both approaches have to adopt evidence-based practices to ensure that students are making adequate progress. Furthermore, because different assessments have different diagnostic purposes and have different limitations, it is important to find consistency in identifying nonresponders’ performance across different components before selecting an approach to deliver interventions (Fuchs, Fuchs, & Compton, 2004; Mastropieri & Scruggs, 2005). In short, the processes of implementing RTI include how to implement universal screening tests, how to make decisions based on data, and how to implement progress monitoring and multi-tiered interventions (Fuchs & Fuchs, 2006). In addition, scheduling and staffing are critical issues to make RTI move forward. Schools should provide a manual to guide teachers on how to collect and analyze data, how to align the regular curriculum with the intensive intervention curriculum, and how to use data appropriately to help students move from one tier to another. 26       Challenges in Implementing RTI The implementation of RTI creates different challenges for general and special education teachers. On the one hand, because implementing RTI requires general education teachers to conduct evidence-based interventions and systematic progress monitoring in the general classroom, those who have not received such training are challenged (Mellard et al., 2004). The lack of training will result in poor fidelity of implementation. On the other hand, using RTI to identify students with learning disabilities and to provide multi-level interventions is also new to many special education teachers, and thus they also need training. In using RTI, special educators are most concerned about treatment validity (Fuchs, Fuchs, & Speece, 2002; Mellard et al., 2004). Teachers have to see students make adequate progress before they can “buy-in” to RTI and accept the necessary changes to support RTI, which relies a lot on treatment validity. School-Wide Collaboration To ensure that interventions are tied closely under the RTI framework, school-wide collaboration is another important issue. Hoover and Love (2011) suggested several ways to help schools build a strong school-based RTI team: (a) obtain principal and school district support, (b) identify issues that are school-specific and relevant, (c) select solutions based on the school’s needs, and (d) discuss issues and solutions with an outside expert who is knowledgeable. In a school-based collaboration team, each team member plays an equally important role in implementing RTI. Additionally, co-teaching makes RTI more practical and powerful (Murawski & Hughes, 2009). Regarding school-based collaboration, the National Research Center on Learning Disabilities (2006) clearly defined RTI personnel roles and responsibilities for general educators, specialists, support staff, and administrators in their statement. 27       In addition to school personnel, parent involvement is also vital to the implementation of RTI. Because parents know their children better than anyone else, they can provide a great contribution to instruction and intervention decisions (Cook et al., 2012). Byrd (2011) argued that because RTI is still relatively new and complicated to parents, parental involvement can reduce unnecessary time for negotiation and can avoid misunderstandings. For example, some parents may prefer a one-time decision instead of a series of decisions to provide their children with intensive intervention, and they may mistakenly believe that schools are delaying services for their children (Byrd, 2011). In addition, parents may not know that a referral to special education is only one outcome of RTI, and thus RTI is not a replacement for special education (Byrd, 2011). These methodological issues have a great impact on RTI and require preservice teachers to be well prepared before entering the teaching field. In short, school-wide collaboration should include teamwork and effective communication skills with colleagues, parents, and students. Methodological Knowledge in Reading When teachers are equipped with strong content knowledge in reading, their next step is to follow appropriate procedures to implement interventions. According to Gersten et al. (2008), teachers can follow the following procedures to implement reading interventions in the model of RTI: 1. Screen all students for potential reading problems at the beginning of the year and again in the middle of the year. Regularly monitor the progress of students who are at elevated risk for developing reading disabilities. 2. Provide differentiated reading instruction for all students based on assessments of students’ current reading levels (tier 1). 28       3. Provide intensive, systematic instruction on up to three foundational reading skills in small groups to students who score below the benchmark on universal screening. Typically these groups meet between three and five times a week for 20–40 minutes (tier 2). 4. Monitor the progress of tier 2 students at least once a month. Use these data to determine whether students still require intervention. For those still making insufficient progress, school-wide teams should design a tier 3 intervention plan. 5. Provide intensive instruction daily that promotes the development of various components of reading proficiency to students who show minimal progress after a reasonable time in tier 2 small group instruction (tier 3). (Gersten et al., 2008, p. iii) Before using evidence-based practices in reading interventions, teachers need to use multiple assessments to compare the consistency of students’ reading problems across different tools. Furthermore, teachers also need to compare the consistency of students’ reading problems across the five reading components (i.e., Phonemic awareness, phonics, fluency, vocabulary, and reading comprehension). When students do not respond to intensive interventions appropriately, teachers need to think about how to implement the interventions in different ways, instead of just using easier or simpler materials. In addition, three empirical studies about reading intervention are worth noting. The first study was conducting by Vaughn and her colleagues (2011). This was a longitudinal study (over two years) that examined the effects of an intensive reading intervention for 28 eighth-grade students concerning reading comprehension and word identification. Students in the experimental group received a 50-minute, daily, individualized, intensive reading intervention in small groups (2-4 students per teacher). They found that although students in the experimental groups demonstrated significantly higher scores on reading comprehension and word 29       identification at the posttest than comparison students, they did not close the gap with typically performing peers (Vaughn et al., 2011). Vaughn et al.’s (2011) study raises educators’ awareness of to what extent interventions accelerate the progress of struggling students. More specifically, if interventions are to help students catch up with typically performing peers, struggling students should make significant progress at posttest not only comparing to their own pretest outcomes but also comparing to the overall posttest outcomes of the whole class. Their study highlights that an effective intervention should focus not only on individual progress before and after the interventions. It is also necessary to consider if the progress is sufficient enough to close the achievement gap. The second study was conducted by Wanzek and Vaughn (2008). This was a twoconsecutive-year study to explore how varying amounts of time influence the performance of students with low response to reading intervention. The study involved two subset studies. Fifty students (intervention, n=21; non-intervention, n=29) were in the first study for the single-dose intervention research (25 hours in total), and thirty-six students (intervention, n=14; nonintervention, n=22) were in the second study for the double-dose intervention research (50 hours in total). The intervention treatment included phonics and word recognition, fluency, passage reading, and comprehension. The results show that the outcomes of students in the single-dose and double-dose interventions were not significantly different over time, and most of them demonstrated poor performance on reading fluency. Wanzek and Vaughn’s (2008) study indicates that students who demonstrate initial insufficient response may need different interventions rather than simply extending the duration of the same interventions. This finding raises intervention specialists’ awareness that it is important to understand whether or not 30       students’ insufficient responses to initial interventions are due to a lack of insufficient opportunities for practices or due to inappropriate interventions. The third reading intervention study was conducted by Hagan-Burke and her colleagues (2011). Understanding the complexity and effectiveness of reading interventions, Hagan-Burke and her colleagues (2011) conducted an experimental study (206 students from 57 kindergarten classrooms) to examine the effects and interactions of student, teacher, and setting variables on students’ reading outcomes. The interventions were taught 30 minutes per day in small groups with approximately 100 sessions lasting for 21 weeks. They found quality of teachers’ instructional practices and group sizes were significantly associated with students’ reading outcomes, while the mode of delivery (e.g., pull-in or pull-out interventions) did not have significant influence on students’ reading outcomes (Hagan-Burke et al., 2011). Hagan-Burke et al.’s (2011) study shows that teacher quality and group size matter to struggling students’ reading outcomes. Teachers’ explicit instruction and well-organized class activities can improve students reading comprehension. In addition, struggling students benefit more from efficient small group interventions (2-4 students per group), which can be conducted either in general classes or outside general classes (Hagan-Burke et al., 2011). Cultural and Linguistic Diversity Klingner and Edwards (2006) pointed out that many culturally and linguistically diverse students are easily presumed to have learning disabilities when contextual factors are not significantly acknowledged in implementing RTI. For example, some students may simply not have breakfast or good sleep before they come to school, and that is why they do not respond to interventions appropriately. Moreover, some schools have much better resources than others, which should all be taken into consideration regarding students’ non-responsiveness to evidence- 31       based interventions. These socio-cultural contexts are important to the effects of these interventions. Simply using evidence-based interventions without considering socio-cultural contexts will not necessarily make RTI succeed (Klingner & Edwards, 2006). Orosco and Klingner (2010) conducted a qualitative study on an urban elementary school’s RTI implementation (K-2). They analyzed a school district’s approach to RTI and collected artifacts and documents related to RTI, such as literacy curricula, assessments, teacher observation forms, school demographics, and professional development documents. They observed 10 school meetings related to RTI (twice a month, 90 minutes each session), and conducted 3 interviews with each participant (30-45 minutes). They also collected 48 classroom observations with a focus on Latino English learners (3 times per week, 2 hours each session). Orsoco and Klingner (2010) found that in order to avoid misusing RTI, teachers must know how to use and interpret data adequately. Moreover, teachers should equip themselves with the abilities of implementing RTI in culturally and linguistically complex classrooms (Orsoco & Klingner, 2010). In addition, many school personnel cannot distinguish whether a child whose home language is not English has a learning disability or has difficulties in using a second language to learn (Rinaldi & Samson, 2008). When school personnel lack professional capacity, they may misidentify students to be learning disabled. In addition, while using evidence-based interventions, educators should go beyond just knowing what works; they should critically ask, “what works with whom, by whom, and in what contexts (Cunningham & Fitzgerald, 1996)” (Klingner & Edwards, 2006, p. 108). 32       While RTI will continue to serve more students in the United States, more research concerning disproportionate representation of culturally and linguistically diverse students who are placed in special education programs or in intervention groups should be conducted. Teacher Quality and Quality Teaching When it comes to incorporating RTI or other instructional approaches, the quality of the teacher should not be overlooked. Darling-Hammond’s (2008) review of several high-achieving countries in the world has shown that recruiting adequate people who have a passion for teaching to become teachers is a common feature in these countries. Kennedy (2008) has named this as personal resources. According to Kennedy (2008), the quality of the teacher includes personal resources (what teachers bring with them to their jobs), performance (teachers' day-to-day work), and effectiveness (teachers' impact on students). A quality teacher will flexibly develop or select strategies from their teaching repertoire to meet students’ needs in the local context and to make more efficient use of existing resources in schools (Cochran-Smith, 2003). To cultivate better teachers for implementing RTI, the quality of the teacher should be emphasized in teacher preparation programs. Furthermore, Fenstermacher and Richardson (2005) have pointed out that “a social surround supportive of teaching and learning,” is one of the essential elements for reaching the goal of quality teaching. Providing professional development and supportive resources is important to move RTI forward, and to help establish sustainable improvement in schools (Kratochwill et al., 2007). Reviewing several RTI-related training programs, Kratochwill et al. (2007) summarized five important features of these successful programs as follows. First, these programs were built upon and worked in conjunction with schools’ existing programs and school staff’s previous training backgrounds. Second, demonstration of intervention materials, role- 33       plays and group discussions, and case studies were utilized in these programs. Third, the participants had opportunities to solve problems collaboratively. Fourth, on-going support was provided to the participants, including co-teaching, mentoring, and coaching. Fifth, the participants had opportunities to reflect on treatment integrity (Kratochwill et al., 2007). Kratochwill et al.’s (2007) summaries indicate that successful professional development for RTI should include practical examples, step-by-step strategies, and the integration of existing knowledge and resources. If RTI implementation is to be effective, supportive resources need to be taken into consideration. Based on the above literature, additional questions related to content, methodological, and contextual knowledge of RTI-Reading were developed. Preservice Teacher Online Learning Online approaches to teacher preparation have become an important issue in two- and four-year institutions. University professors in general education often integrate or infuse special education issues through online learning modules or web-based distance education (Smith, Smith, & Broone, 2000). Smith and his colleagues’ (2000) quasi-experimental study showed that although preservice teachers performed equally well in traditional and online instructional settings, online learning provided “ongoing access to instruction in a flexible accessible environment,” which offers “potential advantages to student comprehension and ongoing application across teacher preparation curricula” (Smith, Smith, & Broone, 2000, pp. 28-29). Smith and his colleagues (2000) reviewed and summarized the benefits of online learning as follows: 34       1. Continued education opportunities for those in rural and hard-to-reach areas 2. Increased flexibility for the student pursuing education (e.g., attending class is easier) 3. Controlled use of flexibility of course content and materials for the students 4. An enhanced interactive format offering multiple demonstration and practice opportunities for reinforcing instruction and subsequent comprehension 5. Increased numbers of students can be reached by a smaller number of instructors 6. Expanded geographic areas offering varied information distribution 7. Enhanced communication among students offering diversified perspectives and frames of reference 8. Decreased costs of instruction for students (Smith, Smith, & Broone, 2000, p. 5) Another benefit of online learning is that it can help teacher educators understand preservice teachers’ reflective thinking through embedded media, such as videodisc cases (Abell, Bryan, & Anderson, 1998). As Smith and his colleagues (2000) pointed out, because online learning provides more comfortable space for preservice teachers to express their thoughts, teacher educators can observe their students’ reflections through online learning. A similar technique was also found in the IRIS modules’ Initial-and-Final Thoughts questions. Because there is little research addressing preservice teacher learning related to online learning through a set of IRIS-RTI modules, there is a need to continue studies in this area. In addition, a knowledge measure that encompassed the content of the eight IRIS-RTI modules but also went significantly beyond the IRIS content modules is important to examining if using the eight modules as a one-time exposure is sufficient to help preservice teachers understand RTI and to what extent. 35       CHAPTER THREE METHODS   This study utilized mixed research methods to evaluate the impact of IRIS modules on the participants’ knowledge of RTI in the academic domain of reading, and on their perspectives toward RTI. Through mixed methods, this study provides not only generalizable data but also a deeper understanding of how the IRIS modules influenced the participants’ knowledge of RTIReading. To enhance the quality of this study, quality indicators in both quantitative and qualitative research were taken into consideration. In terms of essential and desirable quality indicators for quasi-experimental research, Gersten, Fuchs, Compton, Coyne, Greenwood, and Innocenti (2005) suggested that the following indicators should be included: conceptualization underlying the study, participant sampling, implementation of the intervention and the nature of comparison conditions, outcome measures, and quality indicators for data analysis (Gersten et al., 2005). In terms of quality indicators for qualitative research, Brantlinger, Jimenez, Klingner, Pugach, and Richardson (2005) suggested that researchers should describe participant sampling, the quality of questions, the mechanisms used to collect responses, participant representation, and confidentiality. This study abided by all these quality indicators. Participants and the Setting A total of 55 preservice teachers who were enrolled in a special education teacher preparation program at a large Midwest public university voluntarily participated in this study, and they followed all the steps to complete the tasks. The recruitment information was announced in the potential participants’ classes after receiving their instructors’ permission. All participants were informed of required written consent procedures prior to participating in the 36       study. After participants were recruited, they were classified into two groups based on the results of their pre-assessment instrument. Controlling participants’ variables in each group assured that the participants were comparable across intervention conditions. The process of controlling the equivalence of the two groups before and after attrition is presented in Chapter 4. In addition, because responding to the assessments and survey questionnaires, as well as completing the intervention, required proper language abilities in speaking, listening, writing, and reading, the participants were asked about their own special needs on the demographic characteristics survey in order to provide them with proper accommodations (e.g., time extension, larger print). Furthermore, because all modules were provided online, there was no risk related to the differences of interventions across conditions. All the participants were assigned a number, so they could not be personally identified in any presentation of this study. Data Collection Procedures and the Research Design Prior to the intervention, the participants received a pre-survey questionnaire. This questionnaire was used to determine the participants’ demographic characteristics and their perspective toward RTI before the intervention. The participants also received a pre-assessment. A 120 RTI-Reading Knowledge Assessment Questions was used for determining which group the participants were assigned. During the intervention, the participants were asked to submit their IRIS logs (including their Initial-and-Final Thoughts answers) and their IRIS module assessment response to ANGEL, which is an online management system that assisted the researcher in collecting and analyzing the data of the present study. One sample of the ANGEL web pages used in this study is shown in Figure 3 below. The ANGEL user matrix automatically recorded the time that the participants turned in the tasks, and it monitored if they followed the steps to complete each component of 37       each Module. In addition, a checklist was provided at the end of each module to ensure procedural fidelity. When the participants completed the eight modules, they received a postsurvey questionnaire (social validity and perspective toward RTI) and a post-assessment (multiple-choice questions).   Figure 3. The ANGEL web pages – Module 1 (as an example). The text is meant for visual reference only. This figure helps readers see how the ANGEL web pages look like in the present study. Each web page has seven icons to represent different components of the module. 38       Figure 3 shows that participants’ work on the five components of each module was monitored by the ANGEL user matrix. Although the steps were already explained on the ANGEL main webpage, when the participants clicked Lessons and entered the intervention page, the steps were reiterated under each component to ensure the fidelity of implementation. The structure and the content of the ANGEL web pages were exactly the same for the experimental group and for the control group. The only difference was that the different groups received different modules. Figure 4 shows the data collection procedures of the present study, which provides an overview of the sequence of the study. 39       IRB approval Pilot study Recruiting participants Pre-survey questionnaire Demographic characteristics and perspectives toward RTI + Experimental group Control group IRIS navigation video clip Complete the eight assigned IRIS-RTI modules & tasks Pre-Assessment: 120 question multiple-choice instrument IRIS navigation video clip Complete the assigned eight IRIS (non-RTI) modules & tasks Steps for completing the modules Component 1: Challenge Component 2: Thoughts Component 3: Perspectives and Resources Component 4: Assessment Component 5a: Wrap Up Component 5b: Wrap Up Intervention Fidelity Checklist Post-survey questionnaire (including openPost-Assessment: + ended questions): 120 question multiple-choice Social validity measure and perspectives toward instrument RT Figure 4. Data collection procedures 40       In October 2012, the IRB-required consent forms were approved (No. i041755). Two senior graduate students, who were not in the special education program, participated in the pilot study to ensure ANGEL worked well and the directions were clear. Adjustments to the directions were made according to their feedback. The recruitment information was then announced in the potential participants’ classes after receiving their instructors’ permission. All participants completed the pre-assessment questions, which included a short survey about their demographic information and open-ended questions about their perspectives toward RTI. The participants were classified into two groups (experimental vs. control) based on the results of their preassessment (120 RTI-Reading Knowledge Assessment Questions). The researcher did not let the participants know that there were an experimental group and a control group in this study, in order to avoid any exclusion bias where they may consciously or unconsciously spend less time and effort on completing the tasks and/ or the post-assessment. After grouping, the participants were asked to complete the assigned modules within two to three months at their convenience in terms of time and place. Each module included five components: Challenge, Thought, Perspectives and Resources, Assessment, and Wrap up. The participants spent two to three uninterrupted hours completing each module, including answering Initial-and-Final Thoughts questions, assessment questions, and fidelity questions. The participants spent 16 to 24 hours completing the eight modules over two to three months. The participants were asked not to do any modules that were not assigned to them during the intervention period. All participants were provided a navigation video clip developed by the IRIS Center to learn about how to use an IRIS module. The researcher’s e-mail address and the ANGEL 24/7 Help Desk phone number were provided to participants in case they had any questions about 41       completing the tasks on ANGEL. After completing all the modules, the participants were given an online post-assessment and a post-survey questionnaire. Upon completing all tasks, each participant received 100 dollars and a professional development online training certificate issued by a special education teacher preparation program at a public university. Measures Several measures were used in this study: (a) pre- and post-survey questionnaires (including open-ended questions), (b) pre- and post-assessment instruments, (c) Initial-and-Final Thoughts questions, (d) IRIS module assessment questions, (e) procedural fidelity checklists and the ANGEL user matrix. These measures are discussed in detail below. Pre- and Post-Survey Questionnaires Pre- and post-survey questionnaires were in this study. The pre-survey questionnaire collected information about the participants’ demographic characteristics (see Appendix A). The post-survey questionnaire was a Likert scale with sixteen questions that obtained descriptive data related to social validity for the intervention. In addition, there was an open-ended question about how they found IRIS modules useful or not useful (see Appendix B). The participants’ responses to the post-survey questionnaire provided a measure of social validity for the intervention. Sample questions are shown as follows: 1. I think the eight IRIS modules are effective in helping me understand the purpose of RTI. 2. I think the eight IRIS modules are effective in helping me understand the key components of RTI. 3. I think the eight IRIS modules are effective in helping me understand the key areas of reading intervention in an RTI model. 42       4. I think the eight IRIS modules are effective in helping me understand how assessment data is used in RTI to inform placement decision. 5. I feel more confident now than in the past to implement RTI. Five open-ended questions were given to participants both prior to and after the intervention to explore the participants’ perspective toward RTI. These questions were: (a) What is the purpose of RTI? (b) What do think about RTI (e.g., advantages, disadvantages, etc.? (c) What issues does a teacher need to consider when working with diverse learners in an RTI model? (d) What do school leaders need to consider in developing a successful RTI model in their building? (e) How is teacher quality important in implementing RTI? Because of the medium security setting on ANGEL and the right-click function being disabled, it made all the questions appearing before the intervention difficult to be copied, pasted, and saved by the participants. Pre- and Post-Assessment Instruments The 120 RTI-Reading Knowledge Assessment Questions consisted of 66 TKS test items, 29 IRIS test items, and 25 Literature test items. With the permission of Dr. Spear-Swerling (the author of the TKS), the 66 TKS test items were used in the present study. The author classified the 66 test items of the TKS as follows.   Phonological Awareness: #4, 5, 11, 13, 16, 20, 15 (n = 7, w/ 4 application) Phonics: #1, 6, 7, 14, 18, 21, 24, 25, 29, 30 (n = 10, w/ 9 application) Fluency: #19, 26, 32, 33, 45, 50 (n = 6, w/ 4 application) Vocabulary: #23, 40, 41, 46, 47, 48 (n = 6, w/ 5 application) Assessment: #2, 12, 34, 35, 37, 42, 43, 44, 49, 51, 52, 63, 65 (n = 13, w/ 8 application) RTI: #8, 9, 10, 27, 28, 53, 54, 55, 56, 58, 61, 62 (n = 12, w/ 5 application) Comprehension: #3, 17, 22, 31, 36, 38, 39, 57, 59, 60, 64, 66 (n = 12, w/9 application) 43       Application vs. Content Knowledge: Application items: #1, 2, 6, 7, 10, 11, 13, 14, 15, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 35, 36, 37, 38, 41, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 57, 58, 59, 60, 63, 64, 66 (n = 44) (All others are content knowledge.) In addition to the TKS test items, the IRIS module open-ended questions were turned into multiple-choice questions as part of the pre-assessment instrument to investigate preservice teachers’ knowledge of RTI-Reading prior to the intervention. For example, one initial assessment question was “Compare and contrast the RTI model and the traditional method of identifying and serving struggling students” (IRIS, 2012). This question was turned into a multiple-choice question as: Which of the following statements is correct regarding identifying and serving struggling students? a. In the RTI model, general education and special education operate somewhat independently. b. In the traditional model, the potential for disproportionate representation of diverse students in special education decreases. c. In the RTI model, summative assessments are primarily used. d. In the traditional model, receiving special education services is conditional on being identified as having a disability. e. I do not know. When turning the IRIS module’s open-ended questions into multiple-choice questions, it was more likely that preservice teachers would complete the pre-assessment within two to three hours. These multiple-choice questions may not test exactly what each initial IRIS module openended question intended to test. However, these questions could still provide an initial 44       understanding of the participants’ knowledge of RTI before they received the intervention of the study. In addition to the TKS and IRIS test items, 25 questions were developed based on a review of the literature. Multiple test resources helped the researcher gain a more comprehensive understanding of the participants’ knowledge of RTI-Reading before and after the intervention. To ensure that the quality of all 54 multiple-choice questions (29 IRIS test items and 25 Literature test items; see Appendix C) was proper, these questions were reviewed by three writing consultants at a university writing center. All three consultants were English native speakers and were graduate students. The graduate students were directed to use Wollack’s (2003) criteria to examine each of these multiple-choice questions. The criteria include: • Each item should be concise and uncomplicated. • The answer to each question should be really correct and not just the best answer among all options. • Each item should be independent from other items, so the examinee cannot get the answer from the alternatives of another item or from the clues. • Each item should have only one objective to avoid being misunderstood by the examinee. • Questions should use positive statements and avoid trickery. Based on the three reviewers’ feedback, changes and adjustments were made. In addition, these questions were reviewed by university faculty members who were familiar with RTI-Reading to ensure the accuracy of the answers. Changes and adjustments were made based on discussions. As was described in Chapter 2, Table 2 below shows that the three categories of knowledge were properly included and loaded in the 120 RTI-Reading Knowledge Assessment Questions utilized in this study. 45       Table 2 The Distribution of the Multiple-Choice Questions Content The development of RTI (e.g., NCLB and IDEA; the discrepancy model) RTI components (e.g., universal screening, a multilevel prevention system, progress monitoring, and data-based decision making) Content knowledge in reading Methodological Challenges in implementing RTI (e.g., fidelity) The processes of implementing RTI School-wide collaboration IRIS 73, 86 67, 68, 69, 70, 71, 72, 74, 76, 77, 78, 83, 87, 88, 90, 92 80, 81 IRIS 75, 79, 94 96, 97, 112 Total Contextual IRIS 4, 5, 12, 16, 18, 32, 33, 34, 40, 42, 49, 65 TKS 5 21 14 Literature Total 10, 28, 53, 54, 58 % 1 8 98, 99, 100 1, 2, 3, 6, 7, 11, 13, 14, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 29, 30, 31, 35, 36, 37, 38, 39, 41, 43, 44, 45, 46, 47, 48, 50, 51, 52, 57, 59, 60, 63, 64, 66 TKS 89 Teacher quality 29 % 33% 8, 9, 27, 55, 56, 61 85, 93, 85 82, 84, 91 Total Literature 62 Methodological knowledge in reading Cultural and linguistic diversity TKS 66 6 50% 45 Literature 103, 109, 110, 111, 113, 114, 115, 116, 117 101, 102, 104, 105, 106, 107, 108, 118, 119, 120 25 Total % 10 17% 10 120 Like the pre- and post-survey questionnaires, the security setting on the pre- and postassessment instruments was set to disallow using the right click function and returning to the original questions (to prevent comparing answers). 46       Initial-and-Final Thoughts Questions While working on each module, the participants were asked to respond in writing to the Initial-and-Final Thoughts questions on ANGEL. The participants submitted their Initial Thoughts answers before they started to learn the content of each module. The participants turned in their Final Thoughts answers after they completed each module. The Initial Thoughts and Final Thoughts questions were exactly the same. The ANGEL user matrix automatically recorded the time when the participants submitted their answers and monitored if they followed the steps. Taking module 1 used in the experimental group as an example, the Initial-and-Final Thoughts questions were: Initial Thoughts Questions: (After watching a short video clip) Please jot down your Initial Thoughts on the following questions. • What kind of information would best help Ms. Begay evaluate her students' learning? • Why is it important for Ms. Begay to be aware of her students' progress? • What steps can Ms. Begay take to monitor her students' progress throughout the year? Wrap-Up Questions (Final-Thoughts Questions): Think back to your initial responses to the following questions. After working through the resources in this module, do you agree with your Initial Thoughts? If not, what aspects of your answers would you change? • What kind of information would best help Ms. Begay evaluate her students' learning? • Why is it important for Ms. Begay to be aware of her students' progress? 47       • What steps can Ms. Begay take to monitor her students' progress throughout the year? (IRIS Center, 2013c) As was reviewed earlier in the second chapter (i.e., IRIS, 2013b), the Initial Thoughts questions could be used as a pretest instrument and the Final Thoughts questions could be used as a posttest instrument to examine if the participants applied the content that was covered by each module to address the scenario questions. IRIS Module Assessment Questions Each module had its embedded open-ended questions to make sure the learners (users) were familiar with the content of each module and could apply knowledge to solve problems. Following the order of the five components to complete the module (i.e., Challenge, Initial Thoughts, Perspectives and Resources, Assessment, and Wrap-Up), the participants submitted their responses to the IRIS module assessment questions to ANGEL right after they completed the component, Perspectives and Resources. Taking module 1 in the experimental group as an example, the assessment questions were: Assessment: Please complete the items below. If you have difficulty, go back and review the Resource and Perspectives pages in this module. • List three advantages of progress monitoring over annual achievement tests. • List two key differences between mastery measurement and CBM. • Name three ways CBM can be used to help at-risk students. • What are the six steps in the CBM process? 48       • How would you use CBM when teaching multiplication fact families (e.g., times tables for 2, 3, 4)? Describe what you would do for each of the six steps. • Create a CBM implementation plan for your classroom. Make sure you include the academic subject, frequency of administration, how you will score and graph the data, and how you will use the information for your instructional planning. (IRIS Center, 2013c) These questions were not formally analyzed for content knowledge in this study, but were used as indicator of whether participants successfully completed each module (i.e., the participants spent 2~3 hours and followed directions). If the participants did not spend sufficient time and follow directions, they were disqualified for the study. Procedures of Fidelity Checklists and the ANGEL User Matrix To prevent the potential for intervention contamination of both the experimental and control groups, given the elapsed time allowed from the beginning to the end, a procedural implementation checklist was provided at the conclusion of each module. The participants were asked in advance not to discuss the contents of the modules with their classmates and/or colleagues during the intervention period. ANGEL recorded the time each participant spent on the module and when they turned in their answers to each task. Participants who did not follow the procedure were disqualified and their responses were not used for this study. In addition, because participants were asked to submit both summative and formative data, the triangulation evidence (i.e., pre- and post-survey questionnaires, pre- and post-assessment instruments, Initialand-Final Thoughts questions, IRIS assessment questions, procedure of fidelity checklist) helped 49       examine the reliability of the data. Due to the time the study place (November-January), it was possible that relevant knowledge from coursework might affect the findings. To help account for this, the participants were asked at the end of each module to describe if they received extra information related to RTI through coursework, the Internet, and or other resources (other than the designated IRIS modules). It is worth noting that because both experimental group and control group had almost the same numbers of juniors, seniors, and interns, the impact from the coursework that they received in their teacher preparation program should have been similar during the intervention period. Figure 5 shows an example of the checklist. Right after the completion of Module 1, the participant will be asked three questions: 1. Did you follow the order to complete the module, yes or no? If the answer is no, you are not qualified for doing the other 7 modules. [Note: It is fine that you can go back to the components to check your understanding. Going back to the components of Module 1 to check your understanding will not count against you.] a. Challenge b. Initial Thoughts c. Perspectives and Resources d. Assessment e. Wrap Up 2. How much time did you spend on each of the components? Please take notes about the time when you are working on the module. We expect the participants to spend a total of two to three hours on the module. 3. Did you receive any information or training related to RTI before working on Module 1 (e.g., coursework, colleagues, classmates, Internet, etc.)? Please describe them if your answer is yes. This will not count against you. 4. Did you discuss the content of Module 1 (and the other seven modules) with your classmates or colleagues during the intervention? If the answer is yes, you are not qualified for doing the other 7 modules. Figure 5. Procedures of fidelity checklist 50       All participants were informed in advance that they must abide by the fidelity of implementation in order to be qualified for receiving their stipend and certificate. In addition to the self-report data on the fidelity checklist, ANGEL automatically monitored the participants work. Table 4 summarizes how data was collected to address the six research questions of this study. Through checking multiple sources of data, the internal validity of the study was enhanced for more reliable causal inference (see Rudestam & Newton, 2001). Table 3 Research Questions and Data Collection Research Questions Data Collection 1. What was the participants’ performance on the 120 RTI-Reading Knowledge Assessment Questions before and after the intervention, in terms of TKS, IRIS, and Literature? • Pre- and post-assessment instruments • Procedures of fidelity checklists and the ANGEL user matrix 2. To what extent did a particular set of predictors explain the differences in participants’ post-assessment outcomes (i.e., year in the program, GPAs, groups, and pre-assessment outcomes, etc.)? • Pre-survey questionnaire • Post-assessment instrument • Procedures of fidelity checklists and the ANGEL user matrix 3. How did the experimental group’s knowledge of RTI-Reading grow in each module? • Initial-and-Final Thoughts questions • Procedures of fidelity checklists and the ANGEL user matrix 4. How correlated were the experimental group’s performance on the 120 RTIReading Knowledge Assessment Questions and their growth of knowledge in each module? • Initial-and-Final Thoughts questions • Post-assessment instrument • Procedures of fidelity checklists and the ANGEL user matrix 5. Was the treatment delivered as intended? That is, how useful or not useful did participants find the modules concerning the improvement of their RTI knowledge? • Post-assessment instrument • Procedures of fidelity checklists and the ANGEL user matrix 6. What were the participants’ perspectives toward RTI? • Pre- and post-survey questionnaires • Procedures of fidelity checklists and the ANGEL user matrix 51       Intervention Conditions After taking the online pre-assessment, the participants in the experimental group completed eight IRIS-RTI modules assigned in a designated order. The modules used in the experimental group were under the topic of RTI as grouped by the IRIS Center. Initially, there were a total of ten modules under the category of RTI. Because this study focused on RTIReading intervention, two modules that focused on mathematics interventions were excluded. Table 4 shows a summary of the eight modules used in the experimental group of the present study. 52       Table 4 The Eight IRIS-RTI Modules Used in the Experimental Group Module 1 Classroom Assessment (Part 1): This module discusses how progress monitoring can affect the academic outcomes of students, and it demonstrates how to implement curriculum-based measurement with a classroom of students. Module 2   Classroom Assessment (Part 2): Evaluating Reading Progress: This module explores in detail the assessment procedures integral to RTI. It also outlines how to use progress monitoring data to determine if a student is meeting the established performance criteria or if more intensive intervention is needed. Module 3   RTI (Part 1): An Overview: This module outlines the differences between the IQachievement discrepancy model and the Response-to-Intervention (RTI) model. It also offers a brief overview of each tier in the RTI model and explains its benefits. Module 4   RTI (Part 2): Assessment: This module explores in detail the assessment procedures integral to RTI. It also outlines how to use progress monitoring data to determine if a student is meeting the established performance criteria or if more intensive intervention is needed. Module 5   RTI (Part 3): Reading Instruction: This module illustrates different research-based reading strategies that may be used with the Response-to-Intervention model to improve reading skills. Module 6   RTI (Part 4): Putting It All Together: This module synthesizes the information in RTI (Parts 1, 2, and 3) to provide teachers and other school personnel with a more comprehensive illustration of how to successfully implement RTI. Module 7   RTI (Part 5): A Closer Look at Tier 3: This module describes which students will receive Tier 3 intervention (i.e., special education services), components of Tier 3 reading interventions, and students' response to this individualized intervention. This module also explores parent involvement and issues related to English language learners. Module 8   RTI: Considerations for School Leaders: This module provides information about ways to build support for RTI, factors that should be addressed when implementing RTI, and methods of collecting data and evaluating the effectiveness of the RTI approach. (IRIS Center, 2013c) Among these eight modules used in the experimental group, two of them were about curriculum-based measurement and progress monitoring of students’ academic performance. The rest of them were all about RTI. 53       Comparison Conditions After taking the online pre-assessment, the participants in the control group also completed eight IRIS modules assigned by the researcher in a designated order. However, these modules were not related to RTI in the academic domain of reading. Because the control group also received a treatment just like the experimental group did, they could still improve their knowledge through the modules, but that was not attributable to the actual intervention (it was different, but equally valued knowledge). To avoid letting the participants in the control group feel that they were doing poorly on the post-assessment or had a sense that they were doing things differently from what they were asked on the pre-assessment, twenty questions which were not related to the actual intervention were included in both pre- and post-assessments for both groups. However, these questions were not analyzed for this present study. The modules used in the control group met two selection criteria. First, they were not under the topic of RTI grouped by the IRIS Center. Second, they did not have a focus on RTI in the academic domain of reading. Table 5 summarizes the contents of each module used in the control group of the present study. 54       Table 5 The Eight IRIS Modules Used in the Control Group Module 1 Classroom Management (Part 1): Learning the Components of a Comprehensive Behavior Management Plan: [This module] highlights the importance of establishing a comprehensive classroom behavior management system composed of a statement of purpose, rules, procedures, consequences, and an action plan. It also provides information about how culture, classroom factors, and teacher actions can influence student behavior. Module 2   You're in Charge! Developing Your Own Comprehensive Behavior Management Plan: This module neatly complements the first behavior module, encouraging students to create and print rules and procedures for their own classrooms based on the PAR model. Module 3   SOS: Helping Students Become Independent Learners: This module describes how teachers can help students stay on task by learning to regulate their behavior. The four strategies discussed are self-monitoring, self-instruction, goal-setting, and selfreinforcement. Module 4   Addressing Disruptive and Noncompliant Behaviors (Part 1): Understanding the ActingOut Cycle: The first in a two-part series, this module discusses problem behavior in terms of the stages of the acting-out cycle and suggests ways to respond to students in the cycle's different phases. Module 5   Addressing Disruptive and Noncompliant Behaviors (Part 2): Behavioral Interventions: The second in a two-part series, this module describes interventions that can increase initial compliance to teacher requests as well as interventions that can be implemented to decrease disruptive and noncompliant behaviors. Module 6   Functional Behavioral Assessment: Identifying the Reasons for Problem Behavior and Developing a Behavior Plan: This module explores the basic principles of behavior and the importance of discovering the reasons that students engage in problem behavior. The steps to conducting a functional behavioral assessment and developing a behavior plan are described. Module 7   What Do You See? Perceptions of Disability: This module encourages students to explore their own attitudes and beliefs about people with disabilities. It highlights the abilities of students with disabilities. Module 8   Related Services: Common Supports for Students with Disabilities: This module offers a description of related services and an overview of the benefits they provide to students with disabilities in the general education classroom. It highlights five commonly used related services (Physical Therapy, Occupational Therapy, Speech-Language Pathology Services, Social Work Services, and Psychological Services) and briefly highlights many of the other related services as identified through IDEA '04. (IRIS Center: Resource Locator, 2013) 55       Except for using different modules, the comparison conditions were exactly the same as the intervention conditions. That is, the participants in the control group also needed to follow all steps to complete the tasks, like the experimental group. They were also asked to spend two to three uninterrupted hours completing each module; they received the same training for completing the modules and tasks; and they received the same incentives as those in the experimental group. Data Analysis Data analyses for the measures used in this study are addressed below. Pre- and Post-Survey Questionnaires A hierarchical multiple regression analysis was conducted to examine the relationships between the participants’ demographic characteristics and their assessment scores. Based on the nature of the variables, different analysis measures, such as ANOVA, t test, and correlation, were applied to examine whether the variables were appropriate for the regression model. It is important to note that some questions about the participants’ demographic characteristics in the pre-survey questionnaire were dropped for the analyses. The reason items were dropped were because they were not distinct variables for this group of participants or because the responses to the questions were ambiguous. For example, 99% of the participants were female and all participants were majoring in the area of elementary education and learning disabilities. The variables gender and major were not distinct for this group of participants, and thus they were dropped. In addition, when being asked if the participants had taken any courses related to RTI, some of them mentioned that although issues about RTI were covered in their courses, they were still not familiar what RTI was and how to implement RTI. However, some of them only listed their course numbers, which made it difficult to interpret what extent their coursework had an 56       impact on their RTI knowledge. Thus, such questions were dropped as well. As was described earlier, because both experimental group and control group had almost the same numbers of juniors, seniors, and interns, the impact from the coursework that they received in their teacher preparation program should have been similar during the intervention period. In terms of the social validity questions (i.e., Likert scale) in the post-survey questionnaire, an independent t test was conducted to compare the responses between the experimental group and the control group. For the open-ended questions about the participants’ perspective toward RTI, the coding scheme was also based on the themes emerging from the participants’ responses, and their responses were read and reread to begin the open coding process. A draft code-book was developed. When the participants used the themes to address their perspective toward RTI, their responses were coded. No participant was double-coded on each theme in the same question. A training protocol for training raters was then developed. A doctoral student who had taken at least three quantitative and/or qualitative research methods courses was hired ($15 per hour) for the inter-rater reliability. The rater randomly selected 50% of the written responses for the examination. The rater was trained using the code-book on a pre-identified set of responses. When the inter-rater reliability exceeded 80%, the rater continued to score the entire sample. When the inter-rater reliability fell below 80%, a problem-solving process was undertaken to come to agreement, adjustments to the code-book were made, and a second round of inter-rater reliability assessment on the question was undertaken. The coding book for this open-ended question is shown in Appendix D. 57       Pre- and Post-Assessment Instruments Both paired t test and independent t test were conducted for the within-group comparison and the between-group comparison regarding the pre- and post-assessment outcomes. Based on the purposes of the study and the nature of the variables, different analysis measures were applied to examine relationships between the independent variables (e.g., GPA) and dependent variables (assessment outcomes). Initial-and-Final Thoughts Questions For the Initial-and-Final Thoughts answers, all qualitative data was turned from words into numbers using the content of each module as the coding scheme. The coding scheme was based on the outline of each module provided by the IRIS Center, which indicated the themes of each module. The content of the modules was then used to examine whether the participants used the themes to address the scenario questions properly. When the participants used the themes to address the scenario questions properly, their responses were coded. No participant was doublecoded on each theme. That is, even if the participant might use the same theme to address the questions in a module multiple times, his or her use of the theme was only recoded for one time throughout the module, which indicated that he or she already knew the theme and could use it to address the question(s) properly. The coding procedure was same as the one used in the five open-ended questions about participants’ perspectives toward RTI. The coding book is shown in Appendix E. IRIS Module Assessment Questions, Procedural Fidelity Checklists, and the ANGEL User Matrix The researcher checked the ANGEL reports to ensure that the participants spent sufficient time (2~3 hours) on each module and provided appropriate responses to the module assessment 58       questions. The ANGEL User Matrix also reported if the participants followed the directions and completed the tasks step by step. If the participants violated any of the conditions, they would be disqualified for the study. 59       CHAPTER FOUR RESULTS The purpose of this study was to begin to provide insight on whether online professional development learning modules can positively impact pre-service teacher learning. Teacher educators are increasingly disposed to use online training as part of their preparation of preservice educators. Yet, we know relatively little about whether or how pre-service teachers might learn in online learning environments. In this study, the most popular online learning program in the United States for special educators, the IRIS modules developed at Vanderbilt University as part of a grant from the Office of Special Education Program (OSEP) was chosen. This study sought to develop a knowledge measure that included the content of eight RTI modules but also went significantly beyond the IRIS content modules to include two additional areas that were only peripherally addressed in the IRIS RTI modules, but thought to be central to a comprehensive understanding of RTI, a set of questions from TKS that addressed knowledge of basic reading instruction and then additional knowledge related to teacher quality and culturally relevant instruction that was drawn from the literature on RTI. This chapter is organized by addressing each of the six research questions in sequence. Question 1 was the within group and between group comparisons of the IRIS-RTI (treatment) on the entire 120 RTI-Reading Knowledge Assessment Questions overall, and then by the three components of the knowledge assessment: TKS test items, IRIS test items, and Literature test items. The second question then began to examine possible factors that might impact postassessment performance through a multiple regression, including such things as GPA, preassessment score, and year in the program. Understanding the factors that might account for performance differences can help us to think about how we deliver online learning and issues 60       that we may wish to consider for helping pre-service teachers learn in these environments. The third question asked how knowledge within the experimental group changed and influenced learning over time across the modules. The fourth question was to examine the correlation between the experimental group’s performance on the 120 RTI-Reading Knowledge Assessment Questions and their growth of knowledge in each module. Knowing the correlation between different instruments could help us to collect triangulation data and to understand if pre-service teachers perform consistently on different measures. The next question addressed the fidelity issue to ensure that the treatment was delivered as planned. Finally, the participants’ perspectives toward RTI were examined. Cronbach’s Alpha indicates that the internal consistency of the pre- and post- assessment items within each sub area was adequate. For the pre-assessment (N=81), the internal consistency was .828 for TKS, .762 for IRIS, and .710 for Literature. For the post-assessment (n=55), the internal consistency was .885 for TKS, .820 for IRIS, and .733 for Literature. Before addressing each research question, the procedures for grouping the participants into the experimental and control groups and the equivalence in these two groups before and after attrition are reported. Grouping Before the intervention was conducted, 81 pre-service teachers from a special education program at a Midwest public university were recruited. Based on the 120 RTI-Reading 2 Knowledge Assessment Questions, the 80 participants were grouped into a control group and an                                                                                                                           2 There were a total of 81 participants in this study. One of the students completed the preassessment three days later than the other participants and after the grouping decision was already made. The student’s delay was due to a death in his/her family. The student was later randomly placed in the experimental group by lot. 61       experimental group. The participants were stratified into three subgroups: juniors, seniors, and interns. The reason for the stratification was to ensure that both the control group and the experimental group had an equal (or close to equal) number of juniors, seniors, and interns. The participant scores were then ranked in each of the subgroups. The participants with the oddnumbered ranking were assigned to the control group, and the participants with the evennumbered ranking were assigned to the experimental group. The grouping procedures are illustrated in Figure 6. Juniors #1, score 25…Control #2, score 27…Experimental #3, score 31…Control #4, score 31…Experimental (and so on) Seniors #1, score 33…Control #2, score 35…Experimental #3, score 36…Control #4, score 37…Experimental (and so on) 80 Participants Interns #1, score 45…Control #2, score 53…Experimental #3, score 57…Control #4, score 57…Experimental (and so on) Figure 6. The procedures of grouping Figure 6 displays that through ranking both the experimental group and the control group ended up having similar numbers of juniors, seniors, and interns. After the stratification, one additional student was added to the experimental group randomly. In total, there were 81 62       participants. Forty participants were assigned to the control group, including 13 juniors, 21 seniors, and 6 interns. Forty-one participants were assigned to the experimental group, including 13 juniors, 22 seniors, and 6 interns (see Figure 7). Control Experimental Junior (N=13) Senior (N=21) Intern (N=6) Total: 40 Junior (N=13) Senior (N=22) Intern (N=6) Total: 41 Figure 7. Subgroups in the control group and in the experimental group igure 7 illustrates that the numbers of the subgroups were close to even in each group (experimental vs. control group). Equivalence Examination before the Intervention After dividing the sample into two groups, control and experimental, an independent t test was run to examine whether the two groups were equivalent. A t value of .549 (p = . 584) indicates that there was no significant difference between the control group and the experimental group in their mean scores on the pre-assessment. Thus, the two groups were equivalent for the purpose of this study. An independent t test was run to examine whether the three sub-groups (juniors, seniors, and interns) were also equivalent across the control group and the experimental group. A t value of .294 (p = . 772) indicates that there was no significant difference between the juniors’ mean scores in the control group and in the experimental group; a t value of .272 (p = . 787) indicates that there was no significant difference between the seniors’ mean scores in the control group 63       and in the experimental group; and a t value of .792 (p = . 448) indicates that there was no significant difference between the interns’ mean scores in the control group and in the experimental group. As Table 6 summarizes, the p values indicate that there were no statistically significant differences among the groups (p > .05). Thus, the control group and the experimental group, including these subgroups, were equivalent. Table 6 Summary of Equivalency before Attrition N 81 Junior 26 Senior 43 Intern 12 Mean Std. Control (n=40) 47.13 15.26 Experiment (n =41) 49.00 15.47 Control (n=13) 40.31 11.89 Experiment (n =13) 41.69 12.15 Control (n=21) 47.52 16.17 Experiment (n =22) 48.82 15.03 Control (n=6) 60.05 9.73 Experiment (n =6) All Group 65.50 T Sig. (p) Cohen’s d .549 .584 .12 .294 .772 .11 .272 .787 .08 .792 .448 .50 12.03 Note: No significant differences were found among the groups. Attrition Attrition refers to “the loss of participants while a study is in progress” (Cooper, 2011, p. 56). It is believed that attrition has a potential impact on the internal validity of a study and/or the external validity (Cooper, 2011). For example, participants may drop out of the study because they do not like how they were treated in the study. 64       In this study, there were 55 participants who completed the study (completion rate: 68%). A review of the email messages from the participants who decided to withdraw from the study indicates that dropouts were not due to factors that were directly related to the study. Several participants explained that because of the holiday season and/or other obligations that had come up, they could not complete the study as they had planned. Although the dropouts seemed not to cause any validity issues for the study, it is important to know whether the dropouts had any impact on the initial equivalence status. Therefore, an independent t test was used to examine whether these two groups and these subgroups were still equivalent after the attrition. The conceptual framework is shown in Figure 11. All Samples Control group Examining equivalence Experimental group Yes, the groups are equivalent. Start the intervention Attrition occurred Dropout Remaining Dropout Are there any significant differences between them on the pre-assessment?   Remaining Are there any significant Are there any significant differences between them differences between them on the pre-assessment? on the pre-assessment?     Figure 8. The concept of the equivalency examinations before and after attrition 65       Figure 8 indicates that the equivalency was examined before and after attrition to ensure the findings were interpreted properly. A t value of 1.469 (p = .150) indicates that there was no significant difference between the remaining participants’ and the dropout participants’ means in the control group; and a t value of 1.857 (p = . 071) indicates that there was no significant difference between the remaining participants’ and the dropout participants’ means in the experimental group as well. In addition, a t value of .726 (p = .471) indicates that there was no significant difference between the remaining participants in the control group and in the experimental group. The results in Table 7 show that the control group and experimental group were still equivalent after the attrition. Table 7 Summary of Equivalency after Attrition N Mean Std. Remaining (n=26) 53.31 16.69 Withdrew (n =15) 43.27 11.42 Remaining (n=29) 49.28 14.27 Withdrew (n =11) 41.45 17.00 Remaining participants (Experimental: n=26) 52.31 16.69 Remaining participants (Control: n=29) 49.28 14.29 Experimental Control 41 40 Group t Sig. (p) Cohen’s d 1.857 .071 .70 1.469 .150 .50 .726 .471 .20 Note: The remaining participants in the control group: junior (n=8), senior (n=17), and intern (n=4); the remaining participants in the experimental group: junior (n=5), senior (n=16), and intern (n=5). The remaining participants were coded. The letter C represents the 29 participants in the control group (C1-C29), and the letter E represents the 26 participants in the experimental group (E30-E55). 66       Participants’ Performance on the 120 RTI-Reading Knowledge Assessment Questions The first research question addressed in this study was: What was the participants’ performance on the 120 RTI-Reading Knowledge Assessment Questions before and after the intervention, in terms of TKS, IRIS, and Literature? The information from the within-group comparison and from the between-group comparison demonstrates whether or not the experimental group made significant progress after the intervention (within-group comparison), and whether or not their progress resulted from the intervention (between-group comparison). Within-Group Comparison The paired t test was conducted to examine if there were statistically significant differences between the participants’ performance on the pre- and post-assessment in the experimental group (n=26). The null hypothesis was that there would be no difference between the experimental group’s pre-assessment outcomes and post-assessment outcomes. The effect size (Cohen’s d ) was computed for the paired t test using online software developed at the University of Colorado. The formula is: Cohen's d = M1 - M2 / spooled where spooled = [(s 1�+ s 2�) / 2] Source: www.uccs.edu/~lbecker/ For the pre- and   post-120 RTI-Reading Knowledge Assessment Questions, the t value of 5.155 (p = . 000) reveals that the experimental group’s post-assessment outcomes were significantly higher than their pre-assessment outcomes. Cohen’s d for this test is 0.82, a large effect size. Thus, the null hypothesis of no difference between the experimental group’s preassessment outcomes and their post-assessment outcomes was rejected. Table 8 displays the 67       results. Cohen’s d for this test indicates a close to medium effect size for the different test resources (i.e., TKS, IRIS, and Literature). Table 8 The Paired Samples Statistics of the Pre- and Post-Assessments within the Experimental Group Pair Number Number of Std. of test Mean participants Deviation items Pre-assessment (Overall) 120 26 52.308 16.694 Post-assessment (Overall) 120 26 66 26 31.539 Post-assessment (TKS) 66 26 29 26 10.731 5.008 Post-assessment (IRIS) 29 26 18.308 5.097 Pre-assessment (Literature) 25 26 10.039 3.862 Post-assessment (Literature) 25 26 12.192 3.919 Cohen’s d 36.342 10.763 Pre-assessment (IRIS) Sig. 66.846 18.652 Pre-assessment (TKS) t 9.140 5.155 .000*** 0.82 3.060 .000*** 0.48 7.178 .027* 0.43 4.077 .000*** 0.55 Note: The mean difference is significant at the 0.05 and 0.001 levels, respectively. The confidence interval is 95%. Cohen’s d 0.2 is considered as a small size; 0.5 as a medium size; and .08 as a large size. No missing value was found in the experimental group. Between-Group Comparison An independent t test was conducted to examine if there was any significant difference existing between the two independent groups’ pre- and post-assessment mean scores. The null hypothesis was that there would be no difference in their pre- and post-assessment mean test scores. For the pre- and post-120 RTI-Reading Knowledge Assessment Questions, the t value of 2.032 (p = .047) reveals that the experimental group’ post-assessment outcomes were significantly higher than the control group’ post-assessment outcomes. Cohen’s d for this test is 68       1.19, a large effect size. Thus, the null hypothesis of no difference in the two independent groups’ pre- and post-assessment mean test scores was rejected. The experimental group outperformed the control group, providing evidence that the intervention was beneficial, particularly on the IRIS questions (see Table 9). Table 9 The Independent Samples Statistics of the Pre- and Post-Assessments Group Pre-Assessment (Overall) N Mean Std. Experimental 26 52.308 16.694 Control 29 49.276 14.270 Post-Assessment (Overall) Experimental 26 66.846 18.652 Control Pre-Assessment (TKS) Experimental 26 31.539 9.140 Control Post-Assessment (TKS) 29 33.655 9.993 Experimental 26 10.731 5.008 Control Post-Assessment (IRIS) 29 30.000 7.937 Experimental 26 36.346 10.763 Control Pre-Assessment (IRIS) 29 56.931 17.534 29 10.103 4.639 Experimental 26 18.307 5.097 Control 29 12.345 4.886 Pre-Assessment (Literature) Experimental 26 10.039 3.862 Post-Assessment (Literature) Experimental 26 12.192 3.919 Control Control 29 9.172 3.864 29 10.931 4.636 t Sig. Cohen’s d .726 .471 0.82 2.032 .047* 0.55 .668 .507 0.18 .961 .341 0.26 .482 .632 0.13 4.427 .000*** 1.19 .830 .410 0.22 1.083 .284 0.29 Note: Some missing values were found in the control group. One participant in the control group only completed 62 questions; the other participants in the control group all completed the 120 RTI-Reading Knowledge Assessment Questions. These missing values were coded as “exclude cases analysis by analysis.” No missing value was found in the experimental group. The significant levels were at .05 (*) and .001 (***), respectively. To conclude, in response to the first research question (i.e., what was the participants’ performance on the 120 RTI-Reading Knowledge Assessment Questions before and after the 69       intervention, in terms of TKS, IRIS, and Literature), the results show that the experimental group outperformed the control group, particularly on the IRIS questions, after the intervention. Predictors and Participants’ Post-Assessment Outcomes The second research question addressed in this study was: to what extent does a particular set of predictors explain the differences in participants’ post-assessment outcomes (i.e., year in the program, GPAs, groups, and pre-assessment outcomes, etc.)? A hierarchical multiple regression analysis was conducted to examine the relationships between the independent s and the dependent variable. Based on the nature of the variables, different analysis measures, such as ANOVA and correlation, were applied to examine the relationship between the targeted independent and dependent variables. After identifying appropriate variables, categorical variables would be recorded as “dummy variables,” and numerical variables would be converted into z-scores (M = 0 and SD = 1) for the regression model. Year in the Program A one-way ANOVA was used to test for year in the program differences (i.e., junior, senior, and intern) on the post-assessment outcomes. The results indicate that year in the program did not differ significantly across the three subgroups on the post-assessment, F (2, 52) = 2.756, p = .073. Thus, this variable was not included into the regression analysis. The results are shown in Tables 10 and 11. 70       Table 10 The Results of the Pre-Assessment (Year in the Program) Year in the Program Junior Senior Intern Total N 13 33 9 55 Mean 51.923 63.515 68.667 61.618 Std. 19.788 17.250 18.180 18.586 Table 11 The Between Group Variation and the Within Group Variation (Year in the Program) Sum of Squares 1787.816 16865.166 18652.982 Between Groups Within Groups Total df 2 52 54 Mean Square 893.908 324.330 F 2.756 Sig. .073 Table 10 and Table 11 indicate that the participants’ year in their special program has no significant impact on their post-assessment outcomes. Grade Point Average (GPA) A Pearson correlation coefficient was computed to assess the relationship between the participants’ GPAs and their performance on the pre-assessment. The result indicates that there was a positive correlation between the two variables [r = .324, n = 55, p = .017], and the relationship was statistically significant. Tables 12 and 13 display the results. Table 12 The Means and Std. Deviations of the GPAs and Post-Assessment Outcomes GPAs Post-Assessment Mean 3.443 61.618 Std. Deviation .323 18.586 71   N 54 55     Table 13 The Correlations between GPAs and Post-Assessment Outcomes GPAs Post-Assessment Pearson Correlation Sig. (2-tailed, p-value) N Pearson Correlation Sig. (2-tailed, p-value) N GPAs 1 54 .324* .017 54 Post-Assessment .324* .017 54 1 55 Note: *Correlation is significant at the 0.05 level (2-tailed). One missing value was found in the GPA variable (a participant who recently transferred from another school did not provide his/her accumulated GPA). The results indicate that the participants’ GPAs were significantly correlated to their postassessment outcomes. Thus, this variable was included into the regression model of the study after converting it into z-scores (M = 0 and SD = 1). Groups (Experimental vs. Control) As was done in the between-group comparison, the results indicate that there was a statistically significant difference between the two independent groups on their post-assessment outcomes. Therefore, this variable was included in the regression model of this study after recording it as a dummy variable. Pre-Assessment Outcomes A Pearson correlation coefficient was computed to assess the relationship between the participants’ pre-assessment outcomes and their post-assessment outcomes. The result indicates that there was a positive correlation between the two variables [r = .676, n = 55, p = .000], and the relationship was statistically significant. The results are displayed in Tables 14 and 15. 72       Table 14 The Results of the Pre-Assessment and Post-Assessment Means and Std. Deviations Pre-Assessment Post-Assessment Mean 50.7091 61.6182 Std. Deviation 15.39321 18.58563 N 55 55 Table 15 The Correlations between Pre-Assessment and Post-Assessment Outcomes GPAs Post-Assessment 1 .676** Pearson Correlation Pre-Assessment Sig. (2-tailed, p-value) .000 N 55 Pearson Correlation Post-Assessment Sig. (2-tailed, p-value) 55 ** .676 1 .000 N 55 55 *Correlation is significant at the 0.001 level (2-tailed). The results indicate that the participants’ pre-assessment outcomes were significantly correlated to their post-assessment outcomes. Thus, this variable was included in the regression model of the study after converting it into z-scores (M = 0 and SD = 1). Based on the results, a three-stage hierarchical multiple regression was conducted with the post-assessment outcomes as the dependent variable. The variable of groups was entered at stage one of the regression to control whether or not the participants received the intervention. The variable of pre-assessment outcomes was entered at stage two, and the variable of GPAs was entered at stage three. The variables were entered in this order as it seemed that the group (i.e., experimental or control) might be most relevant to post-outcomes, whereas a person’s prior 73       knowledge (i.e., pre-assessment outcomes) and his/her GPA might moderate the influence of the intervention. The regression model is shown as follows: Post-assessment scorei =β0 + β1 groupi (i.e., experimental or control group) + β2 preassessment scorei + β3 GPAi + ei The results of the regression statistics are reported in Table 16. Table 16 Summary of Hierarchical Regression Analysis for Variables Predicting Post-Assessment Outcomes Variable Step 1 Group (exp. vs. control) Step 2 Group (exp. vs. control) Pre-assessment score Step 3 Group (exp. vs. control) Pre-assessment score GPA Beta t .269 2 2.012* .204 .656 2 R R Change F .269 2.044* 6.581*** .072 .072 4.050* .706 .498 .426 25.324*** .748 .235 .613 .252 R .559 .061 21.128*** 2.472* 6.393*** 2.624* The hierarchical multiple regression revealed that at stage one, the variable “group” contributed significantly to the regression model, F (1, 32) = 4.050, p < .05) and accounted for 7.2% of the variation in the post-assessment outcomes. Introducing the variable “pre-assessment score” explained an additional 42.6% of variation in the post-assessment outcomes, and this change in R2 was significant, F (1, 51) = 23.324, p < . 001. Adding the variable “GPA” to the regression model explained an additional 6.1% of the variation in the post-assessment outcomes, and this change in R2 was significant, F (1, 50) = 21.128, p < . 001. When all three independent 74       variables were included in the final stage, all of them were significant predictors of the postassessment outcomes. The most important predictor of the post-assessment outcomes was the pre-assessment scores that explained 49.8% of the variation in the post-assessment outcomes. Together the three independent variables accounted for 55.9% of the variance in the postassessment outcomes. Participants’ Growth of Knowledge in Each Module The third research question addressed in this study was: How did the experimental group’s (n=26) knowledge of RTI-Reading grow in each module? The participants’ initial thoughts on the scenario questions of each module were used as an additional pre-assessment and their later thoughts on the same questions were used as an additional post-assessment. The purpose of including these additional assessments was to help understand how the experimental group’s knowledge of RTI-Reading grew in each module. All qualitative data was turned from words into numbers using the content of each module as the coding scheme. No participant was double-coded on each theme. That is, even if the participant might use the same theme to address the questions multiple times, his or her use of the theme would only be recorded for one time throughout the module, which indicates that he or she already knew the theme and could use it to address the question(s) properly. The quantified qualitative data for all eight modules is presented as follows. Module 1-Classroom Assessment (Part 1): An Introduction to Monitoring Academic Achievement in the Classroom. This module included three questions based on the scenario and the contents of this module. The three questions were (a) What kind of information would best help Ms. Begay evaluate her students' learning? (b) Why is it important for Ms. Begay to be aware of her 75       students' progress? (c)What steps can Ms. Begay take to monitor her students' progress throughout the year? The six themes identified in this module were (a) summative assessments, (b) formative assessments, (c) mastery measurement (MM), (d) the CBM approach, (e) benefits of CBM, and (f) steps for progress monitoring. The results show that more participants used the contents of Module 1 to address the scenarios questions after they completed this module. The results indicate that many participants were aware of using formative assessments, such as curriculum-based measurement (CBM) to monitor students’ progress (from 0% to 81%), and most of them were able to describe the steps for monitoring students at the end of the module (from 8% to 88%). However, few participants seemed to understand the use of the mastery measurement (MM) and the benefits of CBM. Module 2-Classroom Assessment (Part 2): Evaluating Reading Progress This module also included three questions based on the scenario and the contents of this module. The three questions were (a) How should Ms. Begay assess her students' reading levels and progress? (b) How will Ms. Begay know if her current reading intervention is working or if her students need a different kind of instruction? (c) How should Ms. Begay communicate José's reading progress? The six themes identified in this module were the six steps for progress monitoring: (a) determining reading probes, (b) administering and scoring the probes, (c) graphing, (d) setting goals, (e) making instructional decision based on evidence, and (f) communicating with students, parents, and other professionals. The results are displayed in The results reveal that many participants were more familiar with the six steps for progress monitoring after completing the second module of this study. In particular, all participants knew that setting goals and communicating through graphs of data are important for progress monitoring (from 42% to 100%). However, there were still many participants who did 76       not mention that teachers should first determine and choose an appropriate reading probe (4%27%). The other area for more participants to grow in is making instructional decisions based on evidence (19%-35%). Module 3- RTI (Part 1): An Overview This module included four questions based on the scenario and the contents of this module. The four questions were (a) What procedures do you think Rosa Parks Elementary is using to provide services to struggling students? Why are school personnel dissatisfied with this process? (b) What approaches are available to schools to help struggling readers and to efficiently identify students who need special education services? (c) What other information might a school find helpful when choosing which approach to adopt? (d) What steps might the STeam propose to help its struggling readers? The seven themes identified in this module were (a) IQ-achievement discrepancy model, (b) early intervening, (c) learning disabilities, (d) problem solving, (e) standard protocol, (f) universal screening, and (g) multi-tiered intervention. The results show that before completing Module 3, none of the participants used the IQachievement discrepancy model to address why the school personnel in the scenario were dissatisfied with the way that their school helps struggling students. None of the participants used the theme, IQ-achievement discrepancy model, to address any questions in this module before completion. However, they became more aware of this theme upon the completion of the module (from 0% to 88%). In addition, more participants understood the benefits of RTI for early intervening and support of learning disability identification (from 8% to 38%), and they used these themes to address the questions. Although no participants used the themes of the problem solving approach to RTI and standard treatment protocol to RTI before completing this module, they seemed to be aware of these two approaches of RTI at the end. The themes of 77       universal screening and multi-tiered intervention were also widely mentioned in the participants’ later thoughts about the questions. Module 4- RTI (Part 2): Assessment This module included three questions based on the scenario and the contents of this module. The three questions were (a) What is RTI? (b) How will teachers initially identify struggling readers? (c) How will teachers determine which students need more intensive instruction? The seven themes identified in this module were (a) universal screening, (b) multitiered intervention, (c) progress monitoring, (d) making instructional decisions based on evidence, (e) research-validated instruction, (f), standard treatment protocol, and (g) IQachievement discrepancy model. The results indicates that most participants did not use the themes, such as researchvalidated instruction, standard treatment protocol, and IQ-achievement discrepancy model, in their responses to the scenario questions. Instead, they mainly used the four major components of RTI to address the questions. Module 5- RTI (Part 3): Reading Instruction This module included four questions based on the scenario and the contents of this module. The four questions were (a) What is RTI? (b) How can teachers increase student reading success in early grades? (c) What components comprise high-quality reading instruction? (d) How is high-quality instruction integrated into the RTI approach? The seven themes identified in this module were (a) universal screening, (b) multi-tiered intervention, (c) progress monitoring, (d) making instructional decisions based on evidence, (e) five components of effective reading 78       instruction, (f) a daily instructional plan (90-minute reading instruction), and (g) English language learners. After completing Module 5, 88% of the participants used the five components of effective reading instruction to address the scenario questions. Among the four components of RTI, it seems that many participants used multi-tiered intervention to address the questions more than the other components. Furthermore, many participants did not mention the importance of developing daily instructional plans, and none of the participants were aware of students whose first language is not English. Module 6- RTI (Part 4): Putting It All Together This module included three questions based on the scenario and the contents of this module. The three questions were (a) How can Mr. Brewster and the other school professionals at Rosa Parks Elementary School prepare to implement RTI? (b) How can the Rosa Parks teachers effectively implement the RTI components in each tier? (c) What other considerations should Mr. Brewster and the other school professionals be aware of when implementing RTI? The twelve themes identified in this module were (a) universal screening, (b) multi-tiered intervention, (c) progress monitoring, (d) making instructional decisions based on evidence, (e) five components of effective reading instruction, (f) collecting student information (including how the student was identified and what intervention has been done), (g) classroom arrangement/management, (h) materials preparation and storage, (i) a data management system, (j) a daily instructional plan (90-minute reading instruction), (k) addressing diversity, and (l) communication. The results indicate that after completing Module 6, many participants were more aware of the four components of RTI (i.e., universal screening, multi-tiered intervention, progress 79       monitoring, and making instructional decisions based on evidence). The results continue to show that the participants were still more aware of the multi-tiered intervention than the other components of RTI. In addition, because this module does not specifically emphasize the five components of effective reading instruction, not many participants used these to address the scenario questions of this module (4%-35%). Another important finding is that more participants knew the steps of implementing RTI and used them to address the scenario questions after they completed this module. These steps included: collecting student information to know how the student was indentified before and what intervention has been provided, arranging and managing the classroom, having material storage, developing a data management system, preparing a daily instructional plan (including 90-minute reading instruction), addressing diversity (including cultural and linguistic differences as well as disabilities), and communicating with students, parents, and other professionals. Module 7-RTI (Part 5): A Close Look at Tier 3 This module included three questions based on the scenario and the contents of this module. The three questions were (a) How can Tier 3 intervention be conceptualized in the RTI approach? (b) How can Tier 3 intervention be implemented? (c) What considerations should schools and districts be aware of when they deliver Tier 3 intervention? The five themes identified in this module were (a) five components of effective reading instruction, (b) researchvalidated instruction, (c) the IEP team, (d) communicating with parents, and (e) English language learners. The results show that some participants had a lot of growth at the Tier 3 intervention of RTI. In particular, these participants were aware of using research-validated instruction at the Tier 3 intervention. Moreover, more than half of the participants mentioned the importance of 80       communicating with parents regarding the third tier of RTI. This is probably because those who are receiving Tier 3 intervention may or may not be further referred to special education. In addition, quite a few of the participants became more aware of students whose first language is not English in order to not misplace them in special education (from 8% to 46%). More participants mentioned issues about who should be involved in the IEP team after completing this module. When addressing the scenario questions, very few participants mentioned the five components of effective reading instruction. This may indicate that IRIS-RTI was particularly good at helping the participants understand the structural factors or components of RTI, but less at the very detailed knowledge around reading instruction, evidenced based practices, and culturally relevant instruction. Module 8- RTI: Considerations for School Leaders This module included four questions based on the scenario and the contents of this module. The four questions were (a) What information does Mr. Irwin need to consider before proceeding? (b) How can Mr. Irwin build support for adopting the RTI approach at Mayflower Elementary? (c) What should schools consider when deciding whether or not to adopt the RTI approach? (d) What are some factors that should be addressed when implementing the RTI approach? (e) How can schools assess whether or not the RTI approach is effective? The twelve themes identified in this module were (a) becoming informed/ knowledgeable about RTI, (b) identifying key individuals and forming a small representative group, (c) presenting information about RTI, (d) evaluating school readiness, (e) identifying funding sources, (f) gaining a schoolwide commitment, (g) implementing the action plans (with guidelines), (h) considering scheduling, (i) providing classroom support for teachers, (j) building parent-school partnerships, (k) establishing a data management system, and (l) assessing implementation fidelity. 81       The results reveals the important finding that most of the participants only had a general understanding of how RTI can be implemented in the school before they completed Module 8. Prior to completing this module, most of them already knew that the school leader should be well-informed and knowledgeable about RTI and should be able to introduce this approach to teachers and other personnel in the school. However, very few of the participants knew the steps of implementing a new approach in the school. These steps include identifying key individuals and forming a small representative group, evaluating school readiness by using a checklist, identifying where the funding is, gaining school-wide commitment before implementing the approach, considering scheduling, providing classroom support for teachers, establishing a data management system, and accessing implementation fidelity before concluding the effectiveness of the approach. Many participants seemed to be more aware of these steps, and they provided concrete examples to address the scenario questions after completing this module. Table 17 shows the individual participant data for the Initial-and-Final Thoughts in each module. On average, all participants used more themes to address the scenario questions after they completed the modules. The average of individual participants’ gain scores were between 1.25 and 4.63, and average of gain scores in the modules were between 0.69 and 5.65. 82       Table 17 Individual Participant Data for the Initial-and-Final Thoughts M1 postpre E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 Average gain scores of the modules M2 postpre M3 postpre M4 postpre M5 postpre M6 postpre M7 postpre M8 postpre 3 1 2 4 3 2 1 3 2 1 2 3 2 6 2 3 2 2 3 2 2 3 1 2 2 4 4 2 4 5 5 3 0 1 1 0 4 0 4 5 3 2 3 1 3 2 3 4 2 2 4 3 4 6 3 2 0 7 3 3 4 4 3 4 3 7 6 1 1 0 3 4 1 5 1 4 3 5 0 0 1 2 1 2 0 0 1 2 0 2 0 4 -2 -1 3 2 -1 -1 1 0 3 -2 2 1 0 0 2 2 -1 1 1 1 3 1 2 -2 1 3 2 1 0 0 -1 3 1 0 -2 1 -1 0 2 -1 6 3 6 6 1 0 1 4 5 7 7 2 5 3 7 0 3 9 5 7 6 -1 9 4 1 1 0 1 0 0 2 0 1 2 3 2 2 2 1 2 1 0 1 2 3 2 2 1 4 2 1 3 2 9 10 3 3 7 6 3 9 9 9 8 3 1 8 5 9 8 1 4 0 7 10 9 2.42 2.69 3.35 0.77 0.69 4.08 1.46 Individual participants’ average gain scores 1.88 1.50 2.50 3.50 3.00 3.00 1.38 1.88 2.38 2.13 3.50 3.13 3.50 4.63 2.50 1.50 3.13 1.25 2.75 3.63 2.13 3.13 1.63 1.75 4.13 3.50 5.65 Table 17 indicates that there were some variations in individual participants’ growth across the eight modules. Some participants gained more knowledge from some modules than the other modules. The average gain scores of the modules show that the participants gained more knowledge in Module 6 and Module 8 than in the other modules. 83       In summary, the intensive module online training seemed to be effective regarding the growth of the experimental group’s knowledge of RTI-Reading. Most participants demonstrated their immediate changes of their knowledge after completing the modules individually. Correlation of the Instruments The fourth question addressed in this study was: How correlated were the experimental group’s performance on the 120 RTI-Reading Knowledge Assessment Questions and their growth of knowledge in each module? The multiple-choice questions were conducted before and after the participants completed all eight modules within the two-to-three-month self-paced online learning period, while the Initial-and-Final Thoughts questions were conducted before and after the participants completed each module within two to three uninterrupted hours. The qualitative data from the participants’ responses to the scenario questions was quantified. The mean scores of the twenty-nine IRIS multiple-choice questions were compared with the mean scores of the quantified data from the Initial-and-Final Thoughts questions. Table 18 shows the comparison of the participants’ performance on the two types of assessments. 84       Table 18 Comparison of the Experimental Group’s Performance on the Multiple-Choice Questions (29 IRIS items) and on the Initial-and-Final Thoughts Questions Items N Pre-assessment Multiple-choice questions (29 IRIS test items) Post-assessment Multiple-choice questions (29 IRIS test items) Pre-assessment Initial-and Final Thoughts questions Post-assessment Initial-and Final Thoughts questions Post-assessment Multiple-choice questions (29 IRIS test items) Post-assessment Initial-and Final Thoughts questions Mean 26 10.731 Std. Deviation t Sig. (p) Pearson’s Cohen’s r d 5.008 29 5.155 .000*** .140 26 32.346 8 3.66 2.23 3.702 8 29 .249 5.097 26 11.231 .82 14.747 .000*** 26 18.307 .433* 7.283 26 18.307 5.097 26 32.346 7.283 Note: The mean difference is significant at the 0.001 level. The confidence interval is 95%. Table 18 indicates that participants made significant progress on both types of assessments. The t value of 5.155 (p = . 000) indicates that the participants made significant progress on the 29 IRIS multiple-choice questions after the intervention. The t value of 14.747 (p = . 000) reveals that the participants also made significant progress on their responses to the questions of the eight scenarios after the intervention. The effect sizes were large in each type of assessment. However, the correlation between the two types of post-assessments was not statistically significant [r = .140, n = 26, p = .140]. 85       One possible reason that the two types of assessments were not significantly correlated is that they might assess the participants’ knowledge or skills differently. Take Module 1 as an example. The questions asked in the multiple-choice assessment were (a)  Which of the following is the main advantage of frequent progress monitoring versus assessing just one time at the end of the year? (b) Which statement is correct when curriculum-based measurement (CBM) is applied? (c) What is the correct order of the six steps in the curriculum-based measurement (CBM) process? The Initial-and-Final Thoughts questions asked about the Module 1 scenario were (a) What kind of information would best help Ms. Begay evaluate her students' learning? (b) Why is it important for Ms. Begay to be aware of her students' progress? (c)What steps can Ms. Begay take to monitor her students' progress throughout the year? The multiple-choice questions seemed to be more rigorous in testing the specific knowledge of the themes, while the scenario questions were less rigorous in exploring whether or not the participants could use the themes to address the scenario questions. The participants who could use the themes to address the scenario questions might or might not be able to answer the specific questions of the themes correctly. Another area for consideration is the correlation between the Initial-and-Final Thoughts questions. The results show that there was no significant correlation between the Initial-andFinal Thoughts questions [r = .249, n = 26, p = .220], which indicates that most participants demonstrated their immediate changes of knowledge after the completion of each single module, regardless of their prior knowledge about the contents of the module. In contrast to the Initialand-Final Thoughts questions, the pre- and post-multiple-choice questions were significantly correlated [r = .433, n = 26, p = .482]. To conclude, in response to the fourth research question (i.e., how correlated were the experimental group’s performance on the 120 RTI-Reading Knowledge Assessment Questions 86       and their growth of knowledge in each module), the results reveal that the two types of assessments were not significantly correlated. Fidelity of Implementation The fifth question addressed in this study was: Was the treatment delivered as intended? That is, how useful or not useful did participants find the modules concerning the improvement of their RTI knowledge? Social validity questionnaires provide information about participants’ acceptability and satisfaction with the intervention that they have received. Table 19 shows the participants’ satisfaction with the modules. The participants in the experimental group seemed to have higher agreement on the questions that were related to the RTI-Reading modules. This might be due to the fact that they were assigned to work on the modules related to RTI-Reading intervention. However, the participants seemed to have lower agreement on the questions that were related to the behavioral intervention modules. The lower agreement might have resulted from the fact that they were not assigned to work on any modules that were related to the behavioral intervention. In contrast, the participants in the control group seemed to have higher agreement on the questions that were related to the behavioral intervention modules. It is possible that such responses emerged due to the fact that they were assigned to work on the modules that were related to the behavioral intervention. Consistent with the results found in the experimental group, the participants in the control group seemed to have lower agreement on the questions that were not related to the modules assigned to them. Overall, the results of the social validity survey indicate that the different modules indeed had different impact on the participants’ satisfaction with the modules. The participants were satisfied with the interventions that they received. There were statistically significant differences 87       between the responses of the participants in the two groups related to RTI-Reading and behavioral intervention. However, there were no statistically significant differences in the questions related to teacher quality, reasons for why students do not respond to high-quality reading instruction, and their confidence in using RTI. Last but not least, the results of the fidelity checklist embedded at the end of each module and at the end of the entire study indicate that all participants, both the experimental and control groups, followed the procedures to complete the tasks of the study. Table 19 Report of the Social Validity Questions (Likert Scale) 1 2 3 4 5 6 7 Likert Scale: 1. Strongly Agree, 2. Somewhat Control Experimental Cohen’s Sig. Agree, 3. Neutral, 4. Somewhat Disagree, 5. d Mean Std. Mean Std. Strongly Agree I think the eight IRIS modules are effective in helping me understand the purpose of RTI 3.62 1.27 4.58 .58 .001 .44 in reading intervention. I think the eight IRIS modules are effective 3.85 1.01 .001 .43 in helping me understand the purpose of RTI 4.66 .67 in behavioral intervention. I think the eight IRIS modules are effective in helping me understand the key 3.38 1.27 4.54 .76 .000 .48 components of RTI in reading intervention. I think the eight IRIS modules are effective in helping me understand the key 4.69 .71 3.62 1.06 .000 .51 components of RTI in behavioral intervention. I think the eight IRIS modules are effective in helping me understand the key areas of 3.21 1.35 4.54 .58 .000 .54 reading intervention in an RTI model. I think the eight IRIS modules are effective 4.72 .65 3.58 1.03 .000 .55 in helping me understand the key areas of behavioral intervention in an RTI model. I think the eight IRIS modules are effective in helping me understand the purpose for 3.21 1.24 4.58 .58 .000 .58 each tier of reading intervention in an RTI model. 88       Table 19 (cont’d) 8 9 10 11 12 13 14 15 16 I think the eight IRIS modules are effective in helping me understand the examples of reading intervention used in RTI? I think the eight IRIS modules are effective in helping me understand how RTI differs from traditional special education services for determining eligibility for learning disabilities. I think the eight IRIS modules are effective in helping me understand how reading assessment data is used in RTI to inform placement decisions. I think the eight IRIS modules are effective in helping me understand how progress monitoring data is used to determine changes in the intensity of reading interventions. I think the eight IRIS modules are effective in helping me understand what issues a teacher needs to consider when working with diverse learners (e.g., English language learners, special needs, low income, etc.) in an RTI model. I think the eight IRIS modules are effective in helping me understand what school leaders need to consider in developing a successful RTI model in reading intervention for their building. I think the eight IRIS modules are effective in helping me understand how a teacher’s quality (including personality, performance in class, and teaching effectiveness) can impact the implementation of RTI. I think the eight IRIS modules are effective in helping me understand what may cause students’ non-responsiveness to high-quality reading instruction. I feel more confident now than in the past to implement RTI. 89   3.24 1.21 4.50 .58 .000 .55 3.93 1.10 4.69 .55 .002 .40 3.28 1.13 4.50 .65 .000 .55 3.31 1.20 4.62 .57 .000 .57 4.76 .44 4.19 .74 .001 .42 3.69 1.31 4.42 .58 .009 .34 4.59 .63 4.27 .87 .126 .21 3.66 1.23 3.77 1.03 .713 .05 4.24 .79 4.54 .58 .21 .121     In addition to the Likert Scale, an open-ended question was conducted to understand how useful or not useful the participants found the modules in improving their RTI knowledge. Regarding the general comments on the modules, all participants in the experimental group (100%) and many participants in the control group (59%) found the modules, either on reading or on behavioral interventions, to be helpful. Many participants gave positive feedback (e.g., informative, joyful, or thorough) on the use of these modules. In terms of the design of the modules, some participants in both groups pointed out that the real-life and hands-on examples as well as the well-organized content and presentations were beneficial. When asked about how the modules helped improve their knowledge of RTI, some participants in the control group expressed that they did not find the modules useful concerning the academic aspects or tiered-support of RTI. Such results were expected because the control group was not assigned to do any modules related to the academic aspects of RTI. Overall, there was more positive feedback found in the experimental group than in the control group regarding RTI knowledge and implementation of RTI. Furthermore, although several participants in both groups mentioned that they would refer to the modules in his/her future teaching career and would recommend these modules to other colleagues, they were not sure if they could retain the information from the modules. One participant in the experimental group found that some module assessment questions were repetitive. However, such repetitive questions might be important to help the participants recall what they had learned in previous modules. The participants’ responses to the use of the modules are presented in Table 20. 90       Table 20 Report of Users’ Comments on the IRIS Modules General comments on the modules Experimental Group (n=26) Control Group (n=29) • The user enjoyed the modules. (8%) • The modules provided a lot of information. (12%) • The modules were useful, helpful, or beneficial. (100%) • The user found the modules informative. (3%) • The modules were useful, helpful, or beneficial. (59%) • The user found the modules interesting. (3%) • The user took notes on all modules. (3%) • The user learned new techniques. (3%) • The user was grateful to have the opportunity to learn about these modules. (7%) • The user was not sure how much information he/she retained. (3%) • The user found that none of his/her courses had been as thorough or informative as these modules. (7%) Comments on the design of the modules • The modules were realistic and motivated the • Different modes of presenting information user. (4%) helped reinforce/recall the learning. (7%) • The modules provided many (hands-on) • The modules included various research-based examples. (15%) strategies. (3%) • The modules provided graphs and lesson • The user was impressed by the software used plans. (4%) for the modules. (3%) • The design of the modules encouraged the • The design of the modules made it easy for user to read the information carefully. (4%) the user to learn and understand. (7%) • The modules explained RTI in an organized • The user found in each module there was way and in a way that made sense. (4%) some information that did not pertain to the lesson and was a bit wordy. (3%) • The formatting of the modules was a little complicated; the user felt like sometimes he/she was clicking back and forth a lot. (3%) 91       Table 20 (cont’d) Comments on RTI knowledge • The modules improved the user’s RTI • The user found the modules useful and/or helpful, knowledge. (15%) particularly concerning behavioral aspects of RTI. • The modules helped the user understand (34%) all of the components in RTI. (8%) • The user did not find the modules useful • The modules helped the user understand concerning the academic aspects of RTI. (28%) each tier, possible varieties, the costs, • The modules increase the user’s confidence in and/or the benefits of using RTI. (8%) RTI. (3%) • The modules provide extended • The modules were long and tedious because some knowledge of RTI. (12%) of the information the user already knew. (3%) • The user became more confident in • The modules did not specifically address the his/her knowledge of RTI. (8%) information that can be used for tier 1, 2, or 3. • The user recommended the modules for (3%) anyone who is not familiar with or • The user found it is like that he/she has taken an would like to become more informed entire hybrid course on RTI, which he/she would with the processes and components have paid for. (3%) included with RTI and reading intervention. (4%) Comments on practical implementation of RTI • The modules provided real life experiences • The user did not know much about from real professionals. (12%) behavioral management but now he/she felt • The modules provided specific things more prepared for teaching. (3%) teachers used in their classrooms. (4%) • The modules included many good examples • The modules helped the user understand how that are common in the classrooms and went to implement RTI in the school/classroom. through every step of going about these (19%) different issues in the classroom. (3%) • The modules serve as a “review” of how to synthesize all of the information and how to apply it to RTI. (3%) • The modules helped the user understand and hear different things that work or do not work. (3%) • The user found all of the information is vital to him/her as a special educator. (7%) • The user planned to refer to the modules in his/her future teaching career (and/or current placement). (15%) 92       Table 20 (cont’d) Comments on the module assessments • The user found the module assessment • No responses were found related to the questions seemed to be repetitive. (4%) module assessments in the control group. • While working on the assessments, the user had to go back frequently and check over what he/she had read to find the answer again because he/she did could not remember the information. (4%) As was shown in Table 20, when an open-ended question was used to explore the participants’ comments on the modules, a variety of responses were found. While some responses seem independent from the others, they are all invaluable to the future improvement of the modules. Overall, the participants’ responses to the Likert scale questionnaire and to the openended question provide evidence that supports the fidelity of the implementation. Additionally, several participants provided important comments and suggestions regarding the design and the content of the modules, such as the design of the assessment questions and the assistance of helping users to retain the knowledge and skill that they learn from the modules. Perspectives toward RTI The final question addressed in this study was: What were the participants’ perspectives toward RTI? Five open-ended questions were conducted to understand the experimental group’s and the control group’s perspectives toward RTI before and after the intervention. The five questions were (a) What do you think about RTI (e.g., advantages, disadvantages, etc.)? (b) What is the purpose of RTI? (c) What issues does a teacher need to consider when working with diverse learners in an RTI model? (d) What do school leaders need to consider in developing a 93       successful RTI model in their building? (e) How is teacher quality important in implementing RTI? Advantages and Disadvantages of Using RTI In terms of advantages, although some participants in the control group initially perceived that RTI is to provide early identification and intervention, the number of the participants decreased in the in the later responses (from 28% to 14%). In contrast, the number of the participants in the experimental group, who believed that RTI is to provide early identification and intervention, largely increased in the later responses (from 8% to 85%). In addition, few participants in both experimental and control groups considered that RTI as increasing the use of research-validated instruction in the core curriculum (0%-15%). Furthermore, none of the participants in the experimental group recognized RTI as an approach that emphasizes both academic and behavioral interventions. Overall, the experimental group mentioned the advantages of using RTI more than the control group did after the intervention. Table 21 summarizes how the participants perceive the advantages of using RTI. 94       Table 21 Summary of Perspectives toward the Advantages of using RTI Theme • Apply research-validated instruction • Assist students with disabilities or target students who need specific attention and support • Avoid misplacement of students in special education; take into account the students’ backgrounds • Bring all stakeholders to work together • Early identification and intervention • Help students both academically and behaviorally • Help students stay in the general classroom as much as they can • Hold teachers accountable • Help all students • Provide students with additional support • Provide on-going progress monitoring • Take into account the students’ backgrounds and cultures Experimental (n=26) Pre Post 0% 15% Control (n=29) Pre Post 0% 7% 8% 8% 3% 7% 4% 8% 0% 3% 0% 8% 0% 4% 4% 15% 38% 0% 0% 8% 85% 0% 0% 8% 4% 19% 15% 4% 0% 28% 0% 3% 0% 28% 24% 3% 0% 0% 14% 7% 0% 0% 7% 10% 10% 0% In terms of disadvantages of using RTI, upon the completion of the modules, many participants in both groups were concerned about the investment of time, cost, training, materials, and resources for conducting RTI. Despite these concerns, most participants in the experimental group believed that in the long run such investment would benefit struggling students. Furthermore, several participants in the experimental group were initially concerned that implementing RTI would cause conflicts among teachers and labeling students. Their concerns seemed to decrease after they learned more about RTI. Table 22 summarizes how the participants perspectives toward the disadvantages of using RTI. 95       Table 22 Summary of Perspectives toward the Disadvantages of using RTI Experimental (n=26) Pre Post Theme • Cause conflicts among teachers and may not accommodate all students in a general education classroom • Delay special education services • Fail to provide help if it is improperly implemented (fidelity) • Hard for general education teachers to tailor intervention and to monitor progress • Hard to approve the effectiveness of intervention; hard to hold accountability • Increase workload for teachers and/or schools (overwhelming) • Interfere with academics and put students further behind • Make distinction between students who have learning disabilities and those who don’t; labeling • Need extra time, cost/funds, assessments, training, resources, and/or personnel (including scheduling issues) Control (n=29) Pre Post 8% 0% 0% 0% 0% 0% 7% 0% 4% 0% 14 % 8% 0% 0% 3% 4% 0% 3% 0% 4% 0% 4% 8% 0% 0% 3% 7% 0% 0% 3% 0% 8% 50% 7% 34% 0% Purpose of RTI The results indicate that after the intervention, more participants in the experimental group than in the control group perceived RTI as an approach that provides early identification and intervention and support of learning disability identification. Additionally, few participants in the experimental group mentioned research-validated instruction (23%), RTI for all students (8%), and RTI for both academic and behavioral interventions (4%). It is interesting to find that more participants in the control group than in the experimental group perceived that RTI as an approach that emphasizes both academic and behavioral interventions (17%) after the intervention. The results are displayed in Table 23. 96       Table 23 Summary of Perspectives toward the Purpose of RTI Theme Early identification and intervention Research-based and/or quality instruction RTI for all students An approach emphasizing both academic and behavioral interventions • Support of learning disability identification • • • • Experimental (n=26) Pre Post 19% 62% 4% 23% 8% 8% Control (n=29) Pre Post 28% 17% 10% 3% 14% 10% 0% 4% 10% 17% 4% 46% 3% 3% Teachers’ Considerations When Working with Diverse Learners in an RTI Model The results in Table 24 indicate that after the intervention, most participants in both groups pointed out that teachers should know their students’ backgrounds and their differences. However, fewer participants in their responses mentioned that teachers should provide instructional support responding to the needs of diverse students. It seems that the participants emphasized the recognition of diversity more than responding to the needs of diversity. Table 24 Summary of Perspectives toward Teachers’ Considerations When Working with Diverse Learners in an RTI model Experimental (n=26) Pre Post 62% 27% 77% 92% Theme • Instructional support • Students’ backgrounds and differences 97   Control (n=29) Pre Post 45% 28% 97% 100%     School Leaders’ Considerations in Developing a Successful RTI Model in Their Building The results indicate that after the intervention, the majority of the participants in both groups believed that school leaders should identify and prepare resources, build school-wide support, and provide training for implementing RTI. Fewer participants mentioned that school leaders should consider students’ backgrounds and their differences as well as involve parents in this model. The results are summarized in Table 25. Table 25 Summary of Perspectives toward School Leaders’ Considerations in Developing a Successful RTI Model in Their Building Theme • • • • Involve parents Identify and prepare resources Build school-wide support and provide training Recognize students’ backgrounds and differences Experimental (n=26) Pre Post 4% 8% 19% 65% 46% 88% 35% 4% Control (n=29) Pre Post 0% 3% 14% 3% 48% 66% 34% 24% The Importance of Teacher Quality in Implementing RTI The themes were identified in the participants’ responses: the fidelity of RTI implementation and teacher attitude. Most participants perceived that teacher quality will determine whether or not teachers implement RTI properly, which will impact the effectiveness of RTI later. Although many participants in both groups were initially concerned about teachers’ attitude, particularly their willingness to implement RTI, many of them in their responses were concerned about the fidelity of RTI implementation that results from whether or not teachers are knowledgeable about RTI and prepared for implementing it. Table 26 summarizes the responses. 98       Table 26 Summary of Perspectives toward the Importance of Teacher Quality in Implementing RTI Experimental (n=26) Pre Post 52% 55% 45% 34% Theme • Fidelity of RTI implementation • Teacher attitude Control (n=29) Pre Post 50% 69% 46% 31% To conclude, in response to the final research question (i.e., what were the participants’ perspectives toward RTI), more participants in the experimental group than in the control group perceived the advantages of using RTI. Participants in both groups were concerned about the investment of implementing RTI, such as time and cost. Both of them also considered that teachers should recognize the differences among students, and school leaders should build school-wide support in order to make RTI work. Finally, most participants in both groups believed that teacher quality plays an important role to increase the fidelity of RTI implementation. Overall, the results of the study indicate that the eight IRIS modules had great potential to help the participants improve their knowledge of RTI-Reading. However, to help participants retain the knowledge and skills that they learned from the modules, and to help them apply such knowledge and skills to real-world settings more thoroughly, it will likely be necessary to have repeated exposure to knowledge, skills, and practical experiences related to RTI. 99       CHAPTER 5 DISCUSSION AND CONCLUSION This study was an examination of how eight IRIS-RTI modules impacted participants’ knowledge of RTI-Reading as well their perspectives toward RTI. The research extended previous studies that were conducted or collected by the IRIS Center (see the field test data, IRIS Center, 2013b). Previous studies mainly used self-report data, learning outcomes from one single module, or one single-group with a pretest-and-posttest designed to address the impact of IRIS modules. While such research methods are meaningful and important in the educational field, there seemed to be an urgent and necessary need to have empirical data to compare and contrast with the existing literature. This study thus applied a quasi-experimental approach to examine a set of modules under an IRIS topic, specifically RTI. Unlike self-report data, in which participants tend to report positively on their beliefs, knowledge, and abilities (Cook & Campbell, 1979), this quasi-empirical study provided information about what the participants’ actual performance and changes were after the intervention, and how their performance differed from those who did not receive the intervention. In addition, examining a single module under an IRIS topic with a group of participants is not likely to show which module under the same IRIS topic is easier or more challenging than the others for this group of participants. Therefore, this study was geared toward filling the gap of the existing literature by examining how participants performed similarly or differently on a set of modules under the same IRIS topic. It is important to note that when a person masters a single module under an IRIS topic, it does not necessarily mean that the person masters this topic entirely. Similarly, when a person does not perform well on a single module under an IRIS topic, 100       it does not necessarily mean that the person has no knowledge about the topic. In other words, it is meaningful to examine a person’s knowledge across different modules under the same topic, especially in an area as multi-faceted and complex as Response to Intervention. Finally, there are external factors that can contribute to a person’s progress after an intervention. Without a control group, the experiment cannot determine whether a person’s progress results from the intervention itself or results from other factors. This study included both within-group comparison data and between-group comparison data (experimental vs. control), thereby adding a more robust design to address the effectiveness of IRIS-RTI modules. This study also attempted to explore what the participants learned from the modules, how well they learned from the content of the modules, what they did not learn or did not learn well, and what the possible implications are for future work on computer modules based on the findings. Although there is strong evidence in support of gains in knowledge through participation in the IRIS-RTI modules, the variations found in this study indicate that learning differed from individual to individual, and from module to module. Therefore, researchers must take into account these variations when examining how effective IRIS modules are. In the following sections, I first highlight and discuss how my research has reinforced what is already known about the area. Second, I discuss the new findings of the present study in comparison with what is already known. Finally, I discuss how the results can extend knowledge about the field, both in theory and in practice. These three sections are organized by the key findings of the present study. This chapter concludes with an item analysis, limitations of the study, strengths of the study, and future research directions. 101       Knowledge Improvement through IRIS There were two measurements used in the present study to examine the participants’ prior knowledge and their knowledge after the intervention. The two measurements were (1) the 120 RTI-Reading Knowledge Assessment Questions, consisting of 66 TKS test items, 29 IRIS test items, and 25 Literature test items, and (2) the Initial-and-Final Thoughts questions in each module. The results from the within-group comparison (see Chapter 4) show that the experimental group made significant progress on the overall assessment and on each sub-type measure (i.e., TKS, IRIS, and Literature). The results from the between-group comparison (see Chapter 4) provide important evidence that the experimental group’s progress on the IRIS questions most likely resulted from the intervention. In addition, the results based on the Initialand-Final Thoughts questions provide the other evidence that most participants in the experimental group did learn from the modules, and they made significant progress on the Final Thoughts questions. The existing literature has shown many positive effects in using IRIS modules for teacher and school leader preparation programs (IRIS Center, 2013b; Rodriguez, Gentilucci, & Sims, 2006; Smith et al., 2005). The results of this study were consistent with the earlier studies. The examination of the assessment data before and after each single module found that the participants’ Final Thoughts answers were significantly different from their Initial Thoughts answers to the scenario questions, which indicates they did learn from the modules. Such findings were also confirmed in the 120 RTI-Reading Knowledge Assessment Questions, where the participants made significant progress on their post-assessment, particularly on the IRIS test items. 102       This study examined the impact of the IRIS modules on a population that has not been previously studied: Pre-service teachers. In the midst of a national movement toward increasing uses of RTI, where some 60% of current schools nationwide are using some form of RTI, teacher preparation programs are looking to fill this gap in their teacher preparation programs. Although recent publications in the field of special education recommend IRIS modules as a high-quality online resource for teacher preparation programs (e.g., Billingsley, Israel, & Smith, 2011), it is important to note that a one-time exposure to the modules may not be sufficient to help preservice teachers make adequate progress. While examining progress, it is important to know that making significant progress does not necessarily mean making adequate progress. The results of the present study indicate that because the participants had a relatively low level of background knowledge, shown by their mean scores on the pre-120 RTI-Reading Knowledge Assessment Questions, it is likely that they had more room to grow (see Figures 9). 103       31.54 Figure 9. Participants’ pre-post-total mean scores on the multiple-choice questions 104       Figure 9 shows the experimental group’s post-test performance on the three subareas of the 120 RTI-Reading Knowledge Assessment Questions. As shown in Figure 13, the greatest growth in knowledge about RTI was in those questions developed around the content from the IRIS modules. There were a total of 29 questions, with the mean post-assessment score being 18.31, or 63%. Examining the mean scores on the TKS and the Literature shows that participants received scores of 36.35 and 12.19, or 55% and 49%, respectively. The average mean score for the experimental group on the post-assessment of 66.85 of 120 shows that the experimental students got just 56% of the questions correct on the post-assessment. While it is not surprising that participants did not improve substantially on questions that were indirectly or not taught in the IRIS modules, if we believe the content in the 120 RTI-Reading Knowledge Assessment Questions is a more comprehensive measure of RTI knowledge, there is ample room for improvement. In future replicated studies, adding maintenance probes would further strengthen the research design of this study. Another area for consideration is that the participants’ mean scores on their InitialThoughts questions also indicate that because they had a relatively low level of background knowledge they had more room to grow in each module as well (see Figures 10). 105       Figure 10. The mean scores on the Initial-and-Final-Thoughts questions Figure 10 shows the performance of the experimental group’s initial thoughts prior to a module and then their final thoughts immediately following the 2-3 hour module. Each module has a variable number of critical themes that were the focus of that module. The total average number of themes taught in each module is indicated by the solid triangle (top line), while the average number of themes identified for each module is indicated by the solid squares (middle line) and the average number of themes identified prior to going through the module is indicated by the solid diamond (lowest line). An examination of the graph shows that while there was some variation in background knowledge prior to the modules, the participants improved in the number of themes identified in all eight modules (100%). This is an important finding, suggesting that the IRIS RTI modules were effective in increasing the average knowledge for the 106       participants who went through the modules. Moreover, there was increase in every module, though there was some variation in the amount of growth in each. A closer examination of individual performance within the Initial-and-Final Thoughts measure (see Table 27) shows that the average participant in the experimental group identified 11of 62 possible themes or 18% in their Initial Thoughts, with the range between 6 and 19. The average participant in the experimental group identified 32of 62 possible themes or 52% in their Final Thoughts, with the range between 20 and 44. The results indicate that even immediately following training, participants may not recall a majority of the themes that were taught in the module. This finding is not surprising and should be interpreted with caution. The low levels of background knowledge students brought into the modules suggests that there will need to be many more exposures to the content than can be accomplished in a two to three hour module. When background knowledge levels are low and the majority of the content is new, it is more likely that naive teachers’ knowledge is at an awareness level in early trials and does not reflect the deep connections of someone with extensive training and experience working with RTI in schools. 107       Table 27 Participants’ Initial-and-Final Thoughts Scores in Each Module.   Initial-Thoughts Module 1 E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 0 1 2 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 1 0 0 1 0 2 3 0 3 0 1 0 2 3 2 2 4 0 3 1 0 1 2 1 3 1 1 3 0 2 2 2 2 0 0 0 0 1 0 0 1 1 0 1 0 0 0 0 0 0 4 0 0 1 0 0 0 0 0 4 5 Final-Thoughts 6 7 8 Total 1 2 3 3 2 3 4 2 1 3 2 5 1 1 1 2 4 1 3 1 2 1 1 2 4 0 4 3 1 3 2 2 1 3 1 2 2 4 2 3 2 2 3 1 1 4 2 0 3 1 0 1 3 1 3 2 6 3 2 0 4 0 1 3 2 3 2 2 5 1 2 0 2 1 3 2 3 1 3 3 2 Average: 11 0 0 1 0 0 1 0 2 0 0 0 1 0 0 0 0 2 0 0 0 0 1 0 1 0 1 3 2 4 2 0 2 3 2 1 1 2 3 2 2 1 2 2 1 1 2 4 8 2 1 1 2 11 13 17 6 9 11 10 15 11 11 9 16 10 7 8 8 10 19 8 8 16 19 7 10 10 13 3 2 4 4 4 2 1 3 2 2 2 4 2 6 2 3 2 2 4 2 2 4 1 2 3 4 4 5 4 6 5 5 3 3 3 4 4 3 5 5 4 4 4 4 4 3 6 4 4 4 6 5 4 6 3 2 1 7 3 4 5 4 4 4 3 7 6 1 1 4 3 4 2 5 1 4 3 5 4 5 6 7 8 Total 3 2 5 4 2 0 4 4 11 3 3 4 3 3 7 5 2 8 1 2 3 4 1 4 4 4 4 4 3 5 3 3 7 4 2 9 3 3 9 7 4 3 2 4 5 2 2 3 4 3 8 5 2 6 2 1 3 3 3 10 4 3 8 2 2 12 4 0 6 0 2 2 4 2 10 4 3 6 Average: 32 1 1 1 1 0 1 2 2 1 2 3 3 2 2 1 2 3 0 1 2 3 3 2 2 4 3 4 5 6 11 10 5 6 9 7 4 11 12 11 10 4 3 10 6 10 10 5 12 2 8 11 11 26 25 37 34 33 35 21 30 30 28 37 41 38 44 28 20 35 29 28 37 33 44 20 24 43 41 In short, Figures 18 and Table 27 display how the experimental group improved its knowledge on the IRIS multiple-choice questions and on the Initial-and-Final-Thoughts questions. The findings indicate that even though there were significant gains in their knowledge, 108       these gains were still short of what was assessed in this study. Although the results indicate that the eight IRIS modules significantly improved the knowledge of the experimental group when compared to the control condition (particularly on the 29 IRIS test items), because RTI is such a complex and multi-faceted, using the eight modules as a one-time exposure cannot be expected to provide the breadth or depth of knowledge needed to fully understand or implement In theory, applying an empirical study can help researchers better determine whether using IRIS modules has an intended causal effect on the participants’ progress. Without a control group, this study would not know that the experimental group only outperformed the control group on the IRIS multiple-choice assessment even thought the experimental group made progress on the entire multiple-choice assessment. In addition, although it is believed that “the greater the disparity between these two sets of answers, the greater the learning that has occurred (learner- and knowledge-centered)” (Tyler & Sims, 2010, p. 11), ensuring that learners make adequate progress is needed. In the IRIS Center’s report of the learner outcomes during 20042005 and 2005-2006, although the experimental group made significant progress after using IRIS modules, it was not clear if they made adequate progress. Through comparing the experimental group’s scores with the control group’s scores and providing the total scores of each assessment, this study reveal that the participants of the present study may not have made adequate progress even if they made significant progress. And, more importantly, because we believe this is the first experimental study with preservice teachers in a special education program, one must account for differences in level of background knowledge and how much total knowledge remains to learn on a comprehensive measure of assessment. In practice, when teachers use a set of modules under the same IRIS topic in their coursework, it can help them better understand which module is easier or more difficult for their 109       students, and in which module students seem to have a lower level of prior knowledge. In addition, adopting multiple test resources will help teachers understand if the selected modules are sufficient to help learners get familiar with the topic thoroughly. For example, in this study, using multiple test resources (i.e., TKS, IRIS, and Literature test items) revealed that simply using the eight modules was not sufficient to improve the participants’ knowledge of RTIReading, given the fact that their mean scores on the post-assessment of the TKS test items and Literatures test items were still very low. Suggestions for teacher preparation programs using IRIS modules are addressed in the following. First, teacher educators can give the pre-120 RTI-Reading Knowledge Assessment Questions (including some supplementary assessments) at the beginning of their classes. Second, regarding the learning objectives of the classes, when teacher educators identify pre-service teachers’ strengths and weaknesses based on the results of the pre-assessment(s), they can assign appropriate modules to assist individual students’ learning. Third, teacher educators should provide sub-assessments, including both pre- and post-assessments, for each module. These subassessment questions can be developed based on the assessment questions or Initial-and-Final Thought questions embedded in each module. Next, teacher educators can debrief individual students’ progress before and after taking the modules to inform their instruction. Finally, teacher educators can conduct the post-120 RTI-Reading Knowledge Assessment Questions (including some supplementary assessments) at the end of their classes. These procedures involve ensuring that pre-service teachers not only make significant progress on their post responses, but also make adequate progress through the coursework and through the assistance of the modules. 110       Predictors of Knowledge Improvement Through a hierarchical multiple regression, the results of the present study show that the three independent variables “group (experimental vs. control),” “GPA,” and “pre-assessment score” all significantly predicted the participants’ post-assessment scores. In other words, the intervention was not the only factor that contributed to the participants’ performance on the postassessment scores. Students’ grade-point averages (GPAs) are widely viewed as an important indicator of his or her academic performance (Astin, 1993; Pascarella & Terenzini, 1991; Sansgiry, Bhosle, & Sail, 2006). Lounsbury, Fisher, Levy, and Welsh’s study (2009) pointed out that students’ GPAs are highly related to their persistence and self-regulation. Self-regulation is known as a strong predictor of learning performance (Song, 2010). In particular, as Sharma, Dick, Chin, & Land (2007) noted, “For computer-based learning environments to be effective, learners must be selfregulated” (p. 385). The results of the present study were consistent with the existing literature that GPAs indeed played a significant role in the participants’ online learning performance. In addition, earlier research has shown that prior knowledge is positively associated with students’ learning performance (Svinicki, 2007; Roschelle, 1995; Strangman & Hall, 2009; Bransford, Brown, & Cocking, 2000; Tompson & Zamboanga, 2004). Thompson and Zamboanga (2004) noted that students’ domain-specific prior knowledge facilitates their learning. Strangman and Hall (2009) argued that students who lack sufficient prior knowledge or are unable to activate their prior knowledge may struggle to access, participate, or make progress in new knowledge. Such evidence was also identified in the present study. There was a significant correlation between the participants’ prior knowledge and their performance on the 120 RTIReading Knowledge Assessment Questions. 111       Despite the fact that the variables of “group,” “GPA,” and “pre-assessment score” had significant contributions to the post-assessment scores of the present study, it was found that the three independent variables only predict 55.9% of the variation in the participants’ performance on the post-assessment (i.e., the 120 RTI-Reading Knowledge Assessment Questions). In other words, there were other variations that might also predict the participants’ post-assessment scores, such as the participants’ intelligence, their previous instruction, their reading ability, their language skills, or even their mood. Another important finding is that the participants’ GPAs was only significantly associated with their post-assessment scores, but not with their pre-assessment scores. In contrast, the participants’ year in the program was only significantly associated with their pre-assessment scores, but not with their post-assessment scores. Such findings raise several important questions, such as “How does pre-service teachers’ self-regulation impact their online learning?” and “Can IRIS modules benefit all pre-service teachers, regardless of their year in the program?” In addition, the participants’ GPAs were not significantly associated with their Initialand-Final Thoughts answers. Most participants demonstrated immediate changes after two-tothree uninterrupted hours in each module. This finding raises another important question: “Do IRIS modules benefit pre-service teachers within an uninterrupted intensive intervention period even if they had a low level of prior knowledge?” In theory, even if any of the variations (e.g., the participants’ intelligence, their previous instruction, their reading ability, their language skills, or even their mood) might only account for a small percentage of the variance in participants’ performance on the post-assessment, future studies should take into account these potential variations. By doing so, it will more precisely show how participants’ performance on the post-assessment are predicted by other factors in addition to the intervention, their GPAs, and their prior knowledge. 112       In practice, if teacher educators want to know how well pre-service teachers retain the knowledge that they learn from the content of the modules, teacher educators should develop a pre- and post-assessment that test similar questions, as the Initial-and-Final Thoughts questions. In this study, the purpose of the pre- and post-assessment questions (29 IRIS test items) was not the same as the purpose of the Initial-and-Final Thoughts questions. For example, one of the preand post-assessment questions asked, “Which is the correct order of the six steps in CBM process?” In the same module, one of the Initial-and-Final Thoughts questions asked, “What steps can Ms. Begay take to monitor her students’ progress throughout the year?” Because in this study the pre- and post-assessments tended to test the participants’ deeper knowledge of a concept, while the Initial-and-Final Thoughts questions tended to test the participants’ general knowledge of the concept, the results show little or no correlation between these two types of assessments, and thus the results cannot be used to explain whether or not the participants retained the knowledge they learned from the modules. Suggestions for teacher preparation programs include: carefully reviewing the content of coursework and field experiences, and adjusting assessments to explore whether pre-service teachers retain the knowledge and skills that they learn from the modules. In terms of carefully reviewing the content of coursework, teacher educators should think about what may be gained or lost when only limited modules can be embedded in a course given the time constraint of a class. Through reviewing course syllabi, teacher educators can further work together to see what and how modules can be distributed between different classes. By doing so, teacher educators can assess pre-service teachers’ knowledge and skills of implementing RTI from different aspects. More predictors can be added to the regression model to understand what factors influence pre-service teachers’ performance on their post assessments, such as their field 113       experiences, coursework, etc. How motivated pre-service teachers are to use the modules should also be taken into consideration. In terms of pre-service teachers retention of the knowledge and skills that they learn from the modules, having consistent assessment questions is critical. For example, when the assessment questions and the Initial-and-Final Thoughts embedded in each module seem not to test the same concept, teacher educators may consider adjusting the existent questions or creating their own assessment questions. For example, teacher educators can modify the question, “What steps can Ms. Begay take to monitor her students’ progress through the year?” into “What are the six steps in CBM progress that Ms. Begay can take to monitor her students’ progress through the year?” Such question can be better used to compare pre-service teachers’ responses on the preand post-assessment question, “Which is the correct order of the six steps in CBM process?” Variations in Learning The results of the Initial-and-Final Thoughts questions (see Chapter 4) accounted for students’ prior knowledge and what they learned as a result of the modules. Some participants seemed to have more prior knowledge of some modules than the other participants. The figures shown in Chapter 4 demonstrate that the participants did learn after taking the modules, but they might not make the same amount of progress in all the modules. This evidence supports the argument that there were variations among the modules and among the participants. Figure 11 illustrates the analogy of the variations after an intervention. 114       Has no variation in a module: Themes are used to address the Initial-and-Final Thoughts questions with the same percentage in a module Has variations in a module: Themes are used to address the Initial-and-Final Thoughts questions with different percentage in a module Has no variation in an individual’s performance: The individual gains the same scores across the eight modules Has variations in an individual’s performance: The individual gains different scores in the eight modules Figure 11. Variations analogy. The text is meant for visual reference only. The figure helps readers see the trend of the changes before and after the intervention within and across modules. 115       Variations in IRIS modules have been discovered by previous studies (e.g., Smith et al., 2005). Smith and her colleagues’ (2005) study showed that although the learners who worked on the various modules demonstrated significant changes in their Initial-and-Final Thoughts answers, their perceptions about learning-related outcomes differed from module to module. Such variations were also found in the present study (see Chapter 4). The results indicate that after the intervention, there were still many themes not addressed by the participants in the modules. In addition, as was described in Chapter 4, with the same module, some themes were more frequently used than others, such as multi-tiered intervention and universal screening. While there is little research about personal variations in using IRIS modules, education research has documented personal variations in response to instruction (e.g., Randi & Corno, 2005). Variations across learners and variations over time have been widely discussed in the field of language acquisition (Swierzbin, Morris, Anderson, Klee, & Tarone, 2000). This study revealed that there were variations in individual participants’ learning (see Figure 12). While there was clearly overall learning as a group, there was quite a range in what might be learned, with some participants doing quite well (i.e., they gained higher scores on the Initial-and-Final Thoughts questions), with others not doing very well. It is likely that the participants’ background knowledge about RTI, interests and experience in RTI, learning style and attitude, motivation, and work ethic were all associated with how they responded to the Initial-and-Final Thoughts questions. 116                                                               Figure 12. Variations in individual participants. The text is meant for visual reference only. The figure helps readers see the trend of the changes in each individual from E30 to E55. 117       In theory, while learner variation is unavoidable in research, the variation should be taken into account in the hierarchic regression model in order to interpret the effectiveness of IRIS models more accurately. In addition, the structures of the Initial-and-Final Thoughts questions and the contextual variables such as module topics should also be considered. It is possible that the participants did not use certain themes because of different language structures or due to different module topics. In practice, when teacher educators are aware of which module(s) pre-service teachers have a low level of prior knowledge and of the content of module(s) pre-service teachers improve less, teacher educators should reinforce these themes in coursework and/or give preservice teachers opportunities to revisit these modules. In this study for example, the participants made less progress about CBM benefits (Module 1), daily instructional plans (Module 5), and English learners (Module 5). Therefore, if these modules have assigned as part of a teacher preparation programs, these are the topics that should be reinforced. In addition, this study revealed that many participants modified their Final Thoughts answers based on the content of the module, and did not mention or keep everything from their Initial Thoughts answers. It is important that teacher educators should be aware whether pre-service teachers’ later answers are simply replaced by new information they learn from modules. Prior to taking IRIS modules, preservice teachers need to have critical thinking skills and should answer Initial-and-Final Thoughts questions more critically and inclusively. They should not simply replace their initial answers by new information. 118       User Satisfaction The results of the social validity survey (see Chapter 4) show that most participants in both the experimental group and the control group were satisfied with the modules. They found the modules useful, helpful, or beneficial to their knowledge of RTI-Reading or behavioral intervention. The experimental group was more satisfied with the modules than the control group when they were asked questions about RTI-Reading. In contrast, the control group was more satisfied with the modules than the experimental group when they were asked questions about behavioral intervention. There were statistically significant differences between the responses of the participants in the two groups when the questions were about RTI-Reading and behavioral intervention. The findings are not surprising because it was the purpose of this experimental design. In the report of IRIS field-testing in 2012, the IRIS center stated that on most items, more than half the students who used IRIS modules rated the content and format of the modules to be very good or excellent. The findings of the present study were consistent with the earlier studies. Many participants of the present study liked the design of the modules, including videos, real-life experience, hands-on examples, and different ways of presentations. When asked questions about how the modules helped the participants improve their knowledge of RTI-Reading, the control group seemed less satisfied with the content of the modules. Although such finding was expected as it was the purpose of the experimental design, it implies that asking users more specific questions related to learning goals will draw more accurate conclusions about how users are satisfied with the content of the modules. In addition, while many participants of the present study were satisfied with the modules and would continue to use the modules as a reference in their future career, some of them were not sure if they could 119       retain the knowledge they learned from the modules. The participants’ feedback revealed that they not only wanted to learn new knowledge effectively but also wanted to retain the knowledge they had learned from the modules. In theory, learning goals should be first identified before asking whether or not users are satisfied with the content of the modules. Users feel more satisfied with the content of the modules when the modules meet the learning goals. In addition, embedding questions that are not related to the learning goals of the modules is helpful to learn if users answer the questions carefully. In practice, the participants’ feedback demonstrates that learning would be much more joyful through a variety of presentations, such as videos, real-life experience, hands-on examples, and interviews. Therefore, teacher educators should consider using different teaching methods, such as lectures, media, small groups, and discussions, when they want to reinforce the themes of the modules in teacher preparation programs. Additionally, prior to taking the modules, teacher educators should teach pre-service teachers learning strategies to help them retain the knowledge from the modules. Perspectives toward RTI The results show that the experimental group was more aware of using RTI as a preventive approach to help struggling learners than the control group. In addition, the more the participants in the experimental group learned about RTI, the less misconception or negative feedback they had regarding RTI. While the experimental group viewed RTI more positively than the control group, very few of them recognized that RTI is an approach that offers both academic and behavioral support. Both the experimental group and the control group mentioned that teacher quality plays an important role to make RTI the success it ought to be. 120       Teacher quality is important to make RTI function effectively. As Davis Bianco (2010) noted, “fidelity of [RTI] implementation or treatment integrity requires that teachers provide instruction and progress monitoring according to the research-based method prescribed or to a best-practice protocol” (p. 6). In other words, without teachers to provide adequate instruction, RTI cannot be effective. In this present study, more than half of the participants in both the experimental and control group were aware that teacher quality is critical to ensure fidelity of implementation within an RTI framework. Additionally, the findings show that 92%-100% of the participants in this study mentioned that teachers should recognize students differ in their needs, language, and culture and that RTI implementation should reflect students’ diverse backgrounds. The finding is consistent with the existing literature that cultural considerations within RTI is an important area that needs to be emphasized (e.g., Klingner & Edwards, 2006). Many schools are adopting RTI to identify students who are at-risk for academic failure early on and to prevent reading difficulties. (National Association of State Directors of Special Education: NASDSE, 2006). However, RTI is offered as a way not only to help prevent and remediate academic difficulties but also to deliver behavioral support in an integrated schoolwide system (NCRTI, 2013). The findings of the present study indicate that very few of the participants in the experimental group pointed out that RTI is an approach that emphasizes both academic and behavioral support, before and after the intervention. In contrast, the control group, which received behavioral intervention, seemed to be more aware of the balanced approach to RTI. While it is true that RTI is widely used in academic aspects, the behavioral aspects should not be overlooked. Because the modules assigned to the experimental group did not have a focus on behavioral intervention, it could lead the experimental group to reinforce their misconception that RTI is an approach only for (or mainly for) academic support. 121       Furthermore, the primary goal of RTI is to improve all students’ academic and behavioral outcomes and to ensure that all students are provided with an appropriate level of evidence-based instruction based on their needs (Fletcher & Vaughn, 2009). In the present study, many participants in both the experimental group and the control group viewed RTI as an approach that benefits struggling students. Few of them mentioned that all students can benefit from this approach. In theory, it is worth examining whether pre-service teachers respond to the needs of diverse students, and not just recognize their differences. Although there is evidence that teachers’ practices are driven by their attitudes and beliefs, there are many factors that impact teachers’ practices, such as their workplace (Renzaglia, Hutchins, & Lee, 1997). These factors should be identified and incorporated into teacher preparation programs when introducing RTI. In practice, successful RTI relies on a strong leadership that orchestrates the many stakeholders’ roles, including special educators, general education teachers, school administrators, parents, and students in the RTI model. In addition, this study identified some participants’ misconceptions about using RTI, such as delaying special education services and causing conflicts among teachers. Therefore, more exposure to RTI is needed to reduce pre-service teachers’ misconceptions about using RTI. Item Analysis The Cronbach’s alpha indicates that the three sub-type measures (i.e., TKS, IRIS, and Literature) of the present study all had acceptable or good internal consistency. However, this does not mean that there is only one underlying factor or dimension in the overall measure or in the three sub-type measures. When an oblique rotation method of the exploratory factor analysis (EFA), Promax, and an extraction method of Principal Component Analysis, were used to 122       examine the 120 RTI-Reading Knowledge Assessment Questions, the results indicate that there may be one or two factors in this measure (see Figure 13), and thus creating a single sum or average score for the items may not be appropriate. Figure 13. Screen plot of the 120 RTI-Reading Knowledge Assessment Questions. The text is meant for visual reference only. This figure helps readers see the shape of how the items are tied together. In addition, the rotation sums of squared loadings indicate that the two factors can only explain 19.893% of the variance of the measure, and some test items do not have a clear discrepancy between the two factors, such as test items 28, 56, 92, and 98 (see Table 28). It is important to note that even if some test items can be removed in order to increase the internal 123       consistency, due to the fact of the small sample size of this study, conducting further examinations before removing the items is considered necessary. As Schmitt (1996) noted, For multidimensional measures, use of alpha as the basis for corrections for attenuation causes overestimates of true correlation. Satisfactory levels of alpha depend on test use and interpretation. Even relatively low (e.g., .50) levels of criterion reliability do not seriously attenuate validity coefficients. (p. 350) If the measure used to examine pre-service teachers’ knowledge of RTI-Reading is multidimensional, simply removing the items may not be helpful to increase the accuracy of the measure. In short, more examinations on this measure are needed in order to increase the amount of spread in a distribution and to examine pre-service teachers’ RTI-Reading knowledge more accurately. Table 28 Summary of factor loadings for the 120 RTI-reading knowledge assessment questions (N=81) Factor loading Item Q38 Q89 Q65 Q48 Q17 Q78 Q60 Q90 Q20 Q83 Q67 Q46 Q120 Q117 Q63 Q32 1 .726 .627 .576 .572 .562 .555 .543 .523 .518 .514 .510 .494 .485 .478 .470 .467 124   2 .262 .221 .160 .200 .317 .173 .384 .172 .276 .152     Table 28 (cont’d) Q98 Q10 Q7 Q81 Q21 Q9 Q8 Q33 Q47 Q64 Q103 Q57 Q44 Q26 Q100 Q75 Q92 Q36 Q105 Q13 Q87 Q23 Q95 Q14 Q97 Q72 Q99 Q71 Q79 Q49 Q77 Q50 Q112 Q119 Q62 Q55 Q102 Q34 Q82 Q104 Q56 .451 .449 .441 .439 .434 .428 .423 .415 .402 .401 .398 .389 .389 .372 .370 .366 .357 .352 .351 .339 .324 .323 .316 .311 .300 .446 .192 .347 .237 .449 .377 .234 .173 .339 .339 .234 .313 125   .400 .149 .265 .286 .186 .190 .126 .303 .227 .272 .187 .291 .151 .320 .113 .247 .155 .121 .595 .575 .568 .556 .545 .518 .481 .475 .469 .444 .425 .421 .404 .401 .396 .392     Table 28 (cont’d) Q35 Q74 Q28 Q31 Q43 Q94 Q110 Q61 Q40 Q24 Q86 .257 .348 .173 .241 .154 -.129 .221 .218 .228 .387 .386 .373 .372 .370 .365 .364 .340 .330 .330 .322 Note. Factor loadings ≥ .30 are in bold. Limitations of the Study There were several areas in the research design that could have been strengthened in the current study. First, the study would have benefited from a maintenance and generalization probe. The goal was to assess the knowledge learned about RTI-Reading. Usable knowledge that is internalized could have been assessed through a follow-up assessment of all or a portion of the 120 RTI-Reading Knowledge Assessment Questions from one to two months after the conclusion of the study. The time demands of the intervention (16-24 hours) made this impractical for this group of participants. Second, because the sample size of the present study was small for conducting an adequate factor analysis of the 120 RTI-Reading Knowledge Assessment Questions, it is important to examine further how effective the participants’ knowledge of RTI-reading was measured with the 120 RTI-Reading Knowledge Assessment Questions with a larger sample size. Third, although the statistical results show that there were no significant differences between the experimental group and the control group in terms of their pre-assessment scores, the experimental group’s pre-assessment scores were nevertheless higher than the control group’s. Because the regression model of the present study shows that the 126       participants’ pre-assessment scores had an impact on their post-assessment scores, future studies may randomly assign participants to each group by lots or by other random methods, instead of ranking. Another alternative is to assign the group which has lower pre-assessment scores to the experimental group in future studies. Summary and Future Research Directions The IRIS modules have been widely used in teacher preparation programs in the United States and around the world. While these modules provide important resources in helping preservice teachers understand RTI, carefully examining pre-service teachers’ learning and using their learning performance data to further inform coursework are urgent and needed. The research design of this study can be used or replicated easily to monitor pre-service teachers’ learning on IRIS modules and hold the module accountable. Additionally, examining the impact of the IRIS modules through a comprehensive assessment measure (i.e., TKS, IRIS, and Literature) is highly recommended because it can help teacher educators understand if the modules selected by them are sufficient to help pre-service teachers build solid knowledge of a specific area. Teacher educators should use pre-service teachers’ performance data on these modules to inform their curricula and instruction. For example, the results of the present study indicate that the participants did not as well on the TKS and Literature test items. This implies that teacher educators may need to add more IRIS modules from related topic or to add more specific components related to reading intervention and cultural considerations with the RTI approach. Moreover, by adding maintenance probes in future studies, teacher educators should invite pre-service teachers to revisit the modules that they perform less well and to apply the knowledge and skills in their future classrooms. In short, studies should examine the following: 1. Larger scale study across different teacher preparation institutions 127       2. Refinement of the 120 RTI-Reading Knowledge Assessment Questions to improve validity and reliability 3. Using the 120 RTI-Reading Knowledge Assessment Questions as an evaluation tool for studying teacher preparation around different aspects of RTI 4. Following pre-service teachers into their initial years to determine if the modules impact teaching practice 5. Need for quasi-experimental studies that examine mixed models of IRIS modules, including stand alone, IRIS + lecture, and IRIS tied to field-based practicum. 128       APPENDICES 129       Appendix A Demographic Characteristics Survey 1. 2. Gender Ethnic/racial group(s): Select all that apply Female Caucasian Male Black American Asian Indian/ Alaskan Native Others: _______________ Chicano Hispanic Other Hawaiian / Pacific Islander Asian / Pacific Islander th th 3. Year of the program Interns (5 -year student) Senior (4 -year students) Others: ____________________ 4. Cumulative GPA 1.9 or less 3.4-4.0 2.0-2.2 2.3-2.7 5. Teaching certificate area Learning disaiblities Deaf education Others: ______________ 6. Do you have any teaching experiences in K-12 schools? No Internship teaching Have you taken any course which helps you understand what RTI is? Have you taken any course which helps you understand how RTI works? 3~4 years No 4~5 years 7. 8. No Student teaching Others: _____ Others: : _______________ Yes. Yes. 130   2.8-3.3     Appendix B Social Validity Questions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Likert Scale: 1. Strongly Agree, 2. Somewhat Agree, 3. Neutral, 4. Somewhat Disagree, 5. Strongly Agree I think the eight IRIS modules are effective in helping me understand the purpose of RTI in reading intervention. I think the eight IRIS modules are effective in helping me understand the purpose of RTI in behavioral intervention. I think the eight IRIS modules are effective in helping me understand the key components of RTI in reading intervention. I think the eight IRIS modules are effective in helping me understand the key components of RTI in behavioral intervention. I think the eight IRIS modules are effective in helping me understand the key areas of reading intervention in an RTI model. I think the eight IRIS modules are effective in helping me understand the key areas of behavioral intervention in an RTI model. I think the eight IRIS modules are effective in helping me understand the purpose for each tier of reading intervention in an RTI model. I think the eight IRIS modules are effective in helping me understand the examples of reading intervention used in RTI? I think the eight IRIS modules are effective in helping me understand how RTI differs from traditional special education services for determining eligibility for learning disabilities. I think the eight IRIS modules are effective in helping me understand how reading assessment data is used in RTI to inform placement decisions. I think the eight IRIS modules are effective in helping me understand how progress monitoring data is used to determine changes in the intensity of reading interventions. I think the eight IRIS modules are effective in helping me understand what issues a teacher needs to consider when working with diverse learners (e.g., English language learners, special needs, low income, etc.) in an RTI model. I think the eight IRIS modules are effective in helping me understand what school leaders need to consider in developing a successful RTI model in reading intervention for their building. I think the eight IRIS modules are effective in helping me understand how a teacher’s quality (including personality, performance in class, and teaching effectiveness) can impact the implementation of RTI. I think the eight IRIS modules are effective in helping me understand what may cause students’ non-responsiveness to high-quality reading instruction. I feel more confident now than in the past to implement RTI. 17. How useful or not useful did the participants find the modules, concerning the improvement of their RTI knowledge? 131       Appendix C Multiple-Choice Questions (IRIS and Literature Questions) (1-66 TKS Questions) 67. Which of the following is the main advantage of frequent progress monitoring versus assessing just one time at the end of the year? a. Identify students who need additional or alternative instruction. b. Evaluate students' progress against national norms. c. Estimate a growth in knowledge from one year to the next. d. Improve the school’s grading system. e. I do not know. 68. Which statement is correct when curriculum-based measurement (CBM) is applied? a. Skills are broken into sequenced sub-skills. b. Assessment is based on a pre-planned instructional sequence for the year. c. Students must meet a specified criterion to demonstrate mastery. d. Skills can be taught in any logical order. e. I do not know. 69. What is the correct order of the six steps in the curriculum-based measurement (CBM) process? a. set goalscreate or select appropriate testsadminister and scoregraph the scores make instructional decisions based on CBM datacommunicate progress b. create or select appropriate testsadminister and scoregraph the scoresset goalsmake instructional decisions based on CBM datacommunicate progress c. communicate progress create or select appropriate testsadminister and scoregraph the scoresset goalsmake instructional decisions based on CBM data d. communicate progress set goalscreate or select appropriate testsadminister and scoregraph the scores make instructional decisions based on CBM data e. I do not know. 70. Which statement is correct with respect to implementing curriculum-based measurement (CBM)? a. The implementation of CBM is time-consuming. b. A school-wide screening measurement is part of CBM. c. There are many online materials to support the use of CBM. d. CBM is more feasible to conduct in resource rooms than in general classrooms. e. I do not know. 71. Regarding the benefits of monitoring progress in reading using curriculum-based measurement (CBM), which of the following statements is correct? 132       a. b. c. d. e. Student performance on CBM probes (or measures) does not need to meet national standards, so the curriculum can be more flexible to meet different students’ needs. Teachers can determine whether their teaching methods are effective by tracking their students' growth throughout the year. Teachers can evaluate whether students are learning from the current instruction by comparing students’ scores to their peers’ performance. The CBM data is not used for accountability purposes, such as Individual Education Plans (IEPs). I do not know. 72. Which of the following statements is correct regarding curriculum-based measurement (CBM)? a. If it is hard to visualize a trend in the student’s performance, CBM may not be an appropriate measurement. b. If most of the points are higher than the goal line, the student's performance is exceeding expectations, and a slightly more ambitious goal is needed. c. If most of the points are below the goal line, the teacher should use easier tasks, instead of conducting the tasks in different way. d. If most of the points are around the goal line, the teacher has achieved the expectation and can discontinue assessment. e. I do not know. 73. Which statement is correct regarding the IQ-achievement discrepancy model? a. The accepted criteria for identifying a student as having a learning disability with the IQ-achievement discrepancy model is a difference of at least two standard deviations. b. The discrepancy model is recommended by IDEA 2004 for determining whether a student has a learning disability and needs special education services. c. The discrepancy model is not based on the concept of the normal curve. d. The discrepancy assessment is typically conducted by school social workers. e. I do not know. 74. Which of the following statements is correct regarding the benefit of using response-tointervention (RTI) with struggling students? a. It is relatively easy to employ. b. It can inform instructional decisions. c. Intervention is primarily provided for students who are eligible for special education services. d. The identification procedure for determining a learning disability relies primarily on a universal screening measure. e. I do not know. 75. Compton is a first-grade student at Rosa Parks Elementary School. His teacher in the general classroom, having administered a universal screening measure, knows that Compton may be a struggling reader. What should his teacher consider when trying to 133       help Compton by using RTI? a. Teachers should first obtain Compton’s parent consent form before implementing RTI. b. The teacher should observe if Compton responds to Tier 1 intervention appropriately. c. The teacher should refer Compton to a resource room. d. The teacher should reevaluate Compton using a non-standardized achievement test in reading. e. I do not know. 76. Which of the following statements is correct when using response-to-intervention (RTI)? a. In the standard protocol approach, the teacher is given many choices for choosing an appropriate intervention. b. In the problem solving approach, the teacher is expected to make the majority of instructional decisions related to moving from Tier 1 to Tier II. c. Progress monitoring is used in both approaches to inform decision making. d. A single intervention is identified and used in both approaches to help struggling students. e. I do not know. 77. Which statement includes all the major components of response-to-intervention (RTI)? a. Universal screening, multi-level prevention, referral process, progress monitoring. b. Universal screening, multi-level prevention, data-based decision making, progress monitoring. c. Universal screening, multi-level prevention, IQ-achievement discrepancy model, progress monitoring. d. Universal screening, multi-level prevention, assistive technology, progress monitoring. e. I do not know. 78. Which of the following statements is correct regarding universal screening? a. Universal screening is the administration of an assessment to those whose previous year's standardized achievement test scores were lower than the benchmark. b. Universal screening is the administration of an assessment to students who have Individualized Education Plans (IEPs). c. Universal screening is the administration of an assessment to all students, regardless of disability. d. Universal screening is the administration of an assessment to all students in the resource room. e. I do not know. 79. Ms. Hayes is a first-grade teacher at Rosa Parks Elementary School who is piloting the response-to-intervention (RTI) approach. She is concerned about how she will incorporate progress monitoring into her weekly lesson plan. Which of the following statements is correct regarding using progress monitoring? 134       a. She can use curriculum-based measurement data to refer struggling students to resource rooms. b. She can use curriculum-based measurement to monitor struggling students’ progress, instead of all students. c. She can implement curriculum-based measurement to inform instructional decisions. d. She can use curriculum-based measurement to replace formal assessments. e. I do not know. 80. By using the Dolch sight word list as a screening measure, what percent is often used to determine which students are in need of intervention? a. Bottom 15 percent of the class. b. Bottom 20 percent of the class. c. Bottom 25 percent of the class. d. Bottom 40 percent of the class. e. I do not know. 81. According to the National Reading Panel, what are the five critical components of effective reading instruction? a. Phonemic awareness, phonics, grammar, vocabulary, and reading comprehension. b. Phonemic awareness, phonics, fluency, vocabulary, and reading comprehension. c. Phonemic awareness, phonics, fluency, reading comprehension, and technology. d. Phonemic awareness, fluency, vocabulary, reading comprehension, and motivation. e. I do not know. 82. Mrs. Harrison is conducting fluency instruction in a response-to-intervention (RTI) model. Her students' reading ability levels range from kindergarten level to third-grade level. Which of the following statements is correct regarding fluency instruction? a. Mrs. Harrison should let students know that their reading does not need to sound like natural speech (e.g., quick and expressive). b. Mrs. Harrison should let students choose their favorite books and allow students to read books at their own pace. c. Mrs. Harrison should be aware that fluency and reading comprehension skills are not closely related. d. Mrs. Harrison should provide students immediate corrective feedback during fluency practices. e. I do not know. 83. Which of the following statements best describes Tier 2 instruction? a. Tier 2 instruction is delivered universally to all students in the classroom. b. Tier 2 instruction is primarily delivered in large groups of students and includes cognitive strategy instruction. c. Tier 2 uses technology-based instruction only. d. Tier 2 instruction provides increased intensity and opportunities to learn. e. I do not know. 135       84. Imagine you are a third-grade teacher who has been preparing to implement response-tointervention (RTI). You have one week to get ready for class before school begins. Which of the following statements is correct? a. You should start to collect or develop non-standardized assessments. b. You should be familiar with the core reading program. c. You should focus on students’ academic achievement plan more than behavior management plan. d. You should obtain parental consent to implement RTI. e. I do not know. 85. Which of the following statements is correct regarding effective communication with students, with school colleagues, and with parents in the implementation of response-tointervention (RTI)? a. Teachers should use language that communicates the flexibility and the inclusiveness of the RTI process. b. Effective communication occurs when teachers and their colleagues share similar backgrounds. c. Teachers do not need to collaborate with their colleagues across different tiered interventions. d. Parents need to be informed about core instruction, interventions, and intervention expenditure. e. I do not know. 86. Which of the following statements is correct regarding the RTI model and the traditional model (e.g., the IQ-achievement discrepancy model)? a. In the RTI model, general education and special education operate somewhat independently. b. In the traditional model, the potential for disproportionate representation of diverse students in special education decreases. c. In the RTI model, summative assessments are primarily used. d. In the traditional model, receiving special education services is conditional on being identified as having a disability. e. I do not know. 87. Which of the following statements is correct regarding Tier 3 instruction? a. Tier 3 instruction differs from that provided in Tiers 1 or 2 in increased explicitness and focus. b. Tier 3 instruction is for students who are eligible for special education services. c. Tier 3 instruction is incorporated with non-standardized assessments. d. Tier 3 instruction focuses more on general skills. e. I do not know. 88. Which of the following is a key principle of using research-validated reading interventions in a response-to-intervention (RTI) model? a. Summative assessment. 136       b. c. d. e. Systematic instruction. Remedial instruction. Independent work. I do not know. 89. To ensure that school personnel are appropriately identifying students whose first language is not English for special education services, school personnel must be aware that: a. Students appreciate being taught by teachers who speak same languages as they do. b. Students feel more comfortable when tasks are written in their mother language. c. Students’ may not be able to answer questions due to their lower English proficiency. d. Students’ conversational English proficiency and academic English proficiency are at a similar level. e. I do not know. 90. Fatima is being referred for special education services in a public school. According to the law, Fatima’s parents have the right to: a. Select teachers to work with their child. b. Review other students’ IEP as a resource of comparison. c. Receive a verbal notification from the school’s main office. d. Grant informed consent for evaluations. e. I do not know. 91. Imagine Fatima has been receiving special education services and now it is time for an IEP annual review meeting. Her progress monitoring data indicates that, although at the beginning of the year Fatima was not making adequate progress in reading, in the last three months her scores have increased. As a member in Fatima’s IEP team, what should you recommend? a. Developing summative assessments for Fatima’s progress. b. Comparing the consistency of multiple sources of information. c. Suggesting that Fatima should be placed in a more advanced level as soon as possible. d. Decreasing the intensity of intervention. e. I do not know. 92. What is the correct order of the five stages of implementing school-wide response-tointervention (RTI)? a. becoming informedbuilding supportcreating an action planimplementing the planevaluating implementation b. creating an action planbecoming informedbuilding support implementing the planevaluating implementation c. creating an action plan building support becoming informed implementing the planevaluating implementation d. building supportbecoming informed creating an action planimplementing 137       the planevaluating implementation e. I do not know. 93. Which of the following statements is correct concerning a representative group of individuals when building support for response-to-intervention (RTI)? a. The team members have received RTI training before participating in this group. b. The team members have the ability to think about the big picture. c. The team members have to believe this approach works best for students. d. The team members should not compromise when making intervention decisions. e. I do not know. 94. Which of the following statements is correct regarding implementing response-tointervention (RTI) at a school? a. A school must determine how it will acquire and allocate needed resources. b. Professional development should be held by university-based teacher preparation programs. c. Having professional staff who can implement RTI effectively is more important than having efficient scheduling. d. The federal government has provided guidelines for implementing RTI and all schools have to implement RTI based on the guidelines. e. I do not know. 95. Imagine that you are a specialist who observes a teacher administering progress monitoring probes incorrectly; what action should you take in order to have effective communication? a. Providing critical feedback, so the teacher will not make the same mistake again. b. Videotaping the teacher and pointing out concrete examples about what the teacher did wrong one by one. c. Making a positive comment first about something you observed. d. Reporting this matter to the school office. e. I do not know. 96. Which law establishes the statement: “…in determining whether a child has a specific learning disability, a local education agency may use a process that determines if a child responds to scientific, research–based intervention as a part of the evaluation procedures used to determine if the child is a child with a disability.” a. Americans with Disabilities Act (ADA). b. Individuals with Disabilities Education Act (IDEA). c. No Child Left Behind Act (NCLB). d. Section 504. e. I do not know. 97. Which of the following statements is correct regarding current educational laws and policies for educating students with special needs? a. IDEA does not mandate that students with disabilities have access to the general education curriculum. b. NCLB does not mandate that schools are accountable for the performance of students 138       with disabilities in general education. c. IDEA does not mandate that K-12 public schools implement Response-toIntervention (RTI). d. NCLB does not mandate any students with special needs to take standardized tests. e. I do not know. 98. Which of the following statements is correct regarding a school’s response-tointervention (RTI) team? a. The school RTI team members should respond to school concerns and issues directly based on the school’s unique needs. b. The school RTI team members are not supposed to talk about school-based RTI issues with anyone outside of the school. c. The school RTI team members should avoid offering options in order to make RTI implementation more efficient. d. The school RTI team members should be those who “buy-in” to RTI. e. I do not know. 99. Which of the following statements is correct with respect to the roles of a response-tointervention (RTI) school team? a. Having a bilingual or ESL specialist in the evaluation team is not essential for referring a student whose first language is not English b. The school team should be observing the student in the classroom as well as in other settings before making referral decisions. c. Interventions should be terminated when a referral begins. d. The team member who is most familiar with special needs should finalize the referral decision. e. I do not know. 100. Which of the following statements is correct regarding the roles of general and special education teachers in response-to-intervention (RTI)? a. General education teachers play the major role in RTI. b. Special education teachers play the major role in RTI. c. School administrators play the major role in RTI. d. General and special education teachers play equally important roles in RTI. e. I do not know. 101. To function effectively in response-to-intervention (RTI) and fulfill federal highly qualified teacher requirements, special education teachers must: a. Master technology and support the movement of inclusive education. b. Master increasingly complex knowledge and sophisticated repertoire of instructional practices. c. Be familiar with the core curriculum and develop non-standardized assessments. d. Be familiar with the RTI model and replace traditional assessments by RTI. e. I do not know. 102. What attitude should teachers have when they encounter students who do not respond 139       to “high-quality instruction”? a. Teachers should place blame for low student achievement on student engagement. b. Teachers should maintain the sense of their efficacy as effective teachers. c. Teachers should alter or adjust controllable factors such as teaching style and curricula. d. Teachers should know that they are not responsible for every student’ learning performance. e. I do not know. 103. Which of the statements is correct regarding interpreting the effectiveness of RTI implementation? a. Students’ responsiveness to intervention indicates that they have some kind of individual deficits and thus additional instructional support is needed. b. Students’ cultural and linguistic backgrounds can have an impact on their responsiveness to interventions. c. The failure of RTI implementation is mostly due the fact that teachers did not use evidence-based interventions. d. In the U.S. context, many studies show that students’ ethnicity, socioeconomic status (SES), and/or language proficiency are not significantly related to their academic performance. e. I do not know. 104. Which of the statements is correct regarding the implementation of a new approach? a. Evidence-based practice is one useful instructional approach that can be used without considering student diversity. b. Teachers should abide by school policies to implement instructional approaches, and they do not need to know the purpose of doing it. c. Teacher educators should discover how to communicate research findings in a way that teachers will find the required change both manageable and rewarding. d. Sustained professional development leads to erratic implementation of a new approach. e. I do not know. 105. Which of the statements is correct? a. Research shows that providing teachers with access to innovative instructional strategies through in-services is better for altering existing patterns of teaching than through pre-services. b. When teachers try out new methods of teaching, they often need regular feedback from people who are knowledgeable in the new strategies or innovation. c. To examine the effectiveness of an innovation, teachers should use students’ achievement scores as the only indicator. d. Many studies indicate that initial changes in student performance due to the use of new methods are easily discernible. e. I do not know. 106. Which of the statements is correct regarding teacher preparation in special education? 140       a. The major trends in preparing special education teachers are from categorical, non-categorical, to mainstream. b. Shifting perspectives on providing services to students with special needs does not lead to changes in how special education is conceptualized and organized. c. Special education teacher preparation is in transition, from an emphasis on student behavior to an emphasis on student content area knowledge. d. Special education teacher preparation is in transition, from an emphasis on student content area knowledge to an emphasis on student behavior. e. I do not know. 107. Which of the following statements is correct regarding quality teaching? a. Evidence-based that practices alone can result in quality teaching. b. Quality teaching typically does not consider learners’ willingness. c. Contextual support, such as resources, is needed for quality teaching. d. Quality teaching emphasizes good teaching more than successful teaching. e. I do not know. 108. Which of the following statements is correct? a. Teachers should presume that parents will help children’s school work at home. b. Studies show that children’s academic performance is more strongly related to parents’ occupational status than to parents’ educational history. c. No matter how much target language the parents use with their children, appropriate instruction in school with continued effort can improve children’s target language. d. The expectations of teachers and parents for children of different class backgrounds do not differ. e. I do not know. 109. Which of the statements is correct regarding Vygotsky’s socio-cultural model? a. Vygotsky argued the differences in children’s development are not from biological and inner sources but from the interaction of the child with the sociocultural world. b. Vygotsky believed that the differences between persons with and without disabilities are quantitative. c. Vygotsky’s theory provides teachers with an alternative view for conceptualizing disability. d. Vygotsky’s theory suggests that changing the nature of participation is not essential for less experienced learners. e. I do not know. 110. Which of the following statements is correct regarding reciprocal teaching? a. Reciprocal teaching is an example of how social constructivism does not necessarily inform the design of instructional interventions. b. Reciprocal teaching suggests that teacher modeling followed by skill practice in non-interactive contexts yields the same student learning outcome as those occur in interactive contexts. 141       c. Studies show that students’ participation in social dialogues does not advance their abilities to direct their own independent reading activity. d. Studies show that teachers who taught students responsively produce greater gains on students’ ability to transfer knowledge and skills. e. I do not know. 111. Which of the following statements is correct regarding instructional scaffolding? a. Students come with attitudes and expectations toward learning, and these expectations are shaped in part by the classroom culture created by teachers. b. Scaffolded tools alone can create learning opportunities and help students capitalize on these opportunities. c. One problem regarding the nature of the scaffolding is that it makes learning become more passive. d. Scaffolding occurs through two mechanisms, structuring and standardized protocol. e. I do not know. 112. Which of the statements is correct regarding RTI? a. RTI movement does not bring changes to how high-quality teachers in general and special education are prepared. b. The roles that special and general teachers play are not clarified in the RTI approach. c. Both NCLB and IDEA require that students with disabilities, particularly at middle and high school levels, have access to highly-qualified teachers. d. RTI policy initiative has engendered enthusiasm at federal, state, and local levels; educators have reached agreements about its nature and purpose. e. I do not know. 113. Which of the following terms is preferred by researchers to refer to a more dynamic or synergistic relationship between home/community culture and school culture? a. Culturally appropriate. b. Culturally compatible. c. Culturally congruent. d. Culturally responsive. e. I do not know. 114. Which of the statements is correct regarding the effect of social class on students’ education? a. The structure of schooling, with its high regard for the cultural capital of the upper classes, promotes a belief among working-class students that they are unlikely to achieve academic success. b. The implementation of RTI is independent from the influence of institutional structures and cultural practices. c. Schools serving middle-class neighborhoods are more regimented and emphasize rules and behavioral control. d. Upper-class children are oriented to “restricted” linguistic codes, while working- 142       class children use “elaborated” linguistic codes. e. I do not know. 115. Which of the statements is correct? a. In urban school districts, teachers are generally more highly-qualified than in suburban school districts. b. The cultural divide between teachers and their students is further complicated by the lack of sustained attention to preparing teachers to teach across lines of ethnicity/race. c. In urban school districts, it is more likely that teachers are assigned to teach subject areas inside their fields of certification. d. The White, monolingual, English-speaking teacher education professors and staff who are responsible for educating teachers for diversity often have experiences themselves in teaching in culturally diverse schools. e. I do not know. 116. Which of the statements is correct regarding instruction for English Language Learners? a. Teachers are aware that struggling readers’ home language often confuses their target language learning, and thus they will try not to make connections with what students already know in their home language. b. Effective teachers often have sophisticated knowledge of subject area instruction as well as second-language instruction. c. Studies suggest that teachers should use implicit instruction in word identification, phonological awareness, and vocabulary instruction to English language learners. d. Research has shown that most English language learners benefit a lot from nonstructured opportunities to practice English. e. I do not know. 117. Which of the statements is correct regarding the evaluation of the effectiveness of RTI for struggling learners? a. The effectiveness of RTI is only decided by students’ performance on standardized tests. b. The effectiveness of RTI should take students’ social and cultural contexts into consideration. c. The potential of RTI models does not include the improvement of educational opportunities for culturally and linguistically diverse students. d. The potential of RTI models does not include the reduction rate of disproportionate representation in special education. e. I do not know. 118. Which of the statements is correct regarding the No Child Left Behind Act (NCLB)? a. NCLB focuses on improving teacher quality at the federal level. b. NCLB requires teachers to hold a teaching certificate on educating students with special needs. 143       c. NCLB defines highly qualified teachers as those with full state certification, not including through alternate routes. d. NCLB requires all teachers teaching core subject academic areas to meet specific competency and educational requirements. 119. Which of the statements is a professional definition of quality teaching in opposition to the technical definition of quality teaching? a. The professional teacher is often confused by an array of options for instruction, and thus he/she makes instructional decisions based his/her prior professional training. b. The professional teacher presumes that teaching is to be a certain and linear process within which knowledge is transmitted more or less directly from teacher to student by following a fixed and scientifically predetermined sequence. c. The professional teacher focuses more on his or her professional knowledge and skills than forming productive relationships with parents and community members. d. The professional teacher routinely selects from a repertoire of teaching strategies that are best suited to the needs of learners in the local context at the same time. e. I do not know. 120. Which of the statements is correct regarding teacher quality? a. Teachers with high quality are not influenced by their prior teacher training. b. Teachers with high quality examine their daily performance. c. Teachers with high quality pay less attention to the impact of students’ family backgrounds and cultures. d. Teachers with high quality care less about raising student scores on achievement tests. e. I do not know.     144       Appendix D Open-Ended Questions Data Coding Advantage of using RTI • Apply research-based instruction and/or eliminate inadequate instruction • Assist students with disabilities or target students who need specific attention and support • Avoid misplacement of students in special education; take into account the students’ backgrounds and cultures, and the quality of instruction • Bring all stakeholders to work together • Early identification and/or early intervention • Help students both academically and behaviorally • Help students stay in the general classroom as much as they can • Hold teachers accountable • Monitor/ help all students • Identify and provide students with different/ tailored/ additional support (e.g., tiers) • Provide on-going progress monitoring Disadvantage of using RTI • Cause conflicts among teachers and may not accommodate all students in a general education classroom • Delay special education services • Fail to provide help if it is improperly implemented (fidelity) • Hard for general education teachers to tailor intervention and to monitor progress • Hard to approve the effectiveness of intervention; hard to hold accountability • Increase workload for teachers and/or schools (overwhelming) • Interfere with academics and put students further behind • Make distinction between students who have learning disabilities and those who don’t; labeling • Need extra time, cost/funds, assessments, training, resources, and personnel (including scheduling issues) What is the purpose of RTI? • Early intervention • Research-based and/or quality instruction • RTI for all students • An approach emphasizing both academic and behavioral interventions • Support of learning disability identification What issues does a teacher need to consider when working with 145   Experimental Control Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post     diverse learners? • Instruction and additional support • Students’ background knowledge/ cultural, linguistic, and family differences/ learning styles/abilities/individual goals What do school leaders need to consider in developing a successful RTI model in their building? • Parent involvement • School resources and services, including materials, time, and money • School-wide support, including teacher/staff training and plans • Student/school demographics (e.g., including race, SES, ability, home life, and native language) How is teacher quality important in implementing RTI? • Fidelity of RTI implementation • Teacher attitude     146   Pre Post Pre Post     Appendix E Initial-and-Final Thoughts Data Coding Table 29 Initial-and-Final Thoughts Data Coding (Module 1) Summative assessment/yearend assessment Formative assessment/ progress monitoring Mastery measurement (MM) E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 147   Curriculumbased measurement (CBM) Benefits of CBM Steps for progress monitoring     Table 30 Initial-and-Final Thoughts Data Coding (Module 2) Determining reading probes Administering and scoring the probes Graphing E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 148   Setting goals Making instructional decisions based on evidence Communicating with students, parents, and other professionals through graphs     Table 31 Initial-and-Final Thoughts Data Coding (Module 3) IQ-achievement discrepancy model Benefits of RTI: identification of learning disabilities Benefits of RTI: Early intervening Problem solving approach to RTI E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 149   Standard treatment protocol approach to RTI Universal screening Multi-tiered intervention     Table 32 Initial-and-Final Thoughts Data Coding (Module 4) Universal screening Researchprogress monitoring validated instruction Multi-tiered intervention Making instructional decisions based on evidence E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 150   Standard treatment protocol IQ-achievement discrepancy model     Table 33 Initial-and-Final Thoughts Data Coding (Module 5) Universal Multi-tiered progress Making Screening intervention monitoring instructional decisions based on evidence E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 151   Five components of effective reading instruction Daily instruction plan/90 minutes of reading instruction English language learners/ language ability     Table 34 Initial-and-Final Thoughts Data Coding (Module 6) Universal Screening progress monitoring Classroom arrangement/management Addressing diversity Multi-tiered intervention Making instructional decisions based on evidence Collecting Student information Materials preparation and storage Communicating with students, school personnel, and parents Data management system Daily schedule Five components of effective reading instruction E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 152       Table 35 Initial-and-Final Thoughts Data Coding (Module7) Five components of effective reading instruction Researchvalidated instruction IEP team E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 153   Communicating with English parents Language Learners     Table 36 Initial-and-Final Thoughts Data Coding (Module8) Becoming informed/ knowledgeable about RTI Identifying key individuals and forming a small representative group Presenting/ sharing information about RTI Evaluating school readiness Identifying funding sources Gaining a school-wide commitment E30 E31 E32 E33 E34 E35 E36 E37 E38 E39 E40 E41 E42 E43 E44 E45 E46 E47 E48 E49 E50 E51 E52 E53 E54 E55 154   Implementing the action plans (with guidelines) Considering scheduling Providing classroom support for teachers Building parentschool partnerships Establishing a data management system Assessing implementation fidelity     BIBLIOGRAPHY 155       BIBLIOGRAPHY Abell, S.K., Bryan, L.A., & Anderson, M.A. (1998). Investigating preservice elementary science teacher reflective thinking using integrated media case-based instruction in elementary science teacher preparation. Science Teacher Education, 82(4), 491-510. Astin, A. W. (1993). What matters most in college? Four critical years revisited. San Francisco: Jossey-Bass. Berninger, V. W. (2001). Understanding the 'lexia' in dyslexia: A multidisciplinary team approach to learning disabilities. Annals of Dyslexia, 51, 23-48. Billingsley, B. B., Israel, M., & Smith, S. J. (2011). Supporting new teachers: How web resources & Web 2.0 technologies can help. Teaching Exceptional Children, 43(5), 20-29. Billingsley, B., Israel, M., & Smith, S. (2011). Supporting new special education teachers. Teaching Exceptional Children, 43(5), 20-29. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: brain, mind, experience, and school (expanded edition). Washington, D. C.: National Academy Press. Brantlinger, E., Jiménez, R., Klingner, J., Pugach, M., & Richardson, V. (2005). Qualitative studies in special education. Exceptional Children, 71, 195-207. Brownell, M. T, Sindelar, P. T., Kiely, M. T., & Danielson, L. C. (2010). Special education teacher quality and preparation: Exposing foundations, constructing a new model. Exceptional Children, 76, 357-377. Byrd, E. S. (2011). Educating and involving parents in the response to intervention process. Teaching Exceptional Children, 43(3), 32-39. Caywood, K., & Duckett, J. (2003) Online vs. on-campus learning in teacher education. Teacher Education and Special Education, 26 (2), 98–105. 156       Cochran-Smith, M. (2003). Teaching quality matters. Journal of Teacher Education, 54(2), 9598. Cook, B. G., Shepherd, K., Cook, S. C., & Cook, L. (2012). Facilitating the effective implementation of evidence-based practices through teacher-parent collaboration. Teaching Exceptional Children, 44(3), 22-30. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues. Boston, MA: Houghton Mifflin Company. Cunningham, J. W., & Fitzgerald, J. (1996). Epistemology and reading. Reading Research Quarterly, 31, 36–60. Darling-Hammond, L. (2008). A future worthy of teaching for America. Phi Delta Kappan, 89(10), 730-735. Davis Bianco, S. (2010). Improving Student Outcomes: Data-driven Instruction and Fidelity of Implementation in a Response to Intervention (RTI) Model. Teaching Exceptional Children Plus, 6(5) Article 1. Retrieved from http://escholarship.bc.edu/education/tecplus/vol6/iss5/art1 Donovan, M. S., & Cross, C. T. (2002). Minority students in special and gifted education. Washington, DC: National Academy Press. Fenstermacher, G. D. & Richardson, V. (2005). On making determinations of quality in teaching. Teachers College Record, 107(1), 186-213. Fletcher, J. M. & Vaughn, S. (2009). Response to intervention: Preventing and remediating academic difficulties. Child Development Perspectives, 3(1), 30-37. Fletcher, J. M., Shaywitz, S.E., Shankweiler, D. P., Katz, L., Liberman, I. Y., Stuebing, K. K., Francis, D. J., Fowler, A., & Shaywitz, B. A. (1994). Cognitive profiles of reading disability: Comparisons of discrepancy and low achievement definitions. Journal of Educational Psychology, 85, 1-18. Francis, D. J., Fletcher, J. M., Shaywitz, B. A., Shaywitz, S.E., & Rourke, B. P. (1996). Defining learning and language disabilities: Conceptual and psychometric issues with the use of IQ tests. Language, Speech, and Hearing Services in Schools, 27, 132-143. 157       Fuchs, D., & Fuchs, L.S. (2006). Introduction to response to intervention: What, why, and how valid is it. Reading Research Quarterly, 41, 92-99. Fuchs, D., Fuchs, L. S., & Compton, D. L. (2012). Smart RTI: A next-generation approach to multilevel prevention. Exceptional Children, 78, 263–279. Fuchs, D., Fuchs, L. S., & Compton, D.L. (2004). Identifying reading disabilities by responsiveness to instruction: Specifying measures and criteria. Learning Disability Quarterly, 27(4), 216-228. Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness-to-intervention for the learning disabilities construct. Learning Disabilities Research & Practice, 18(3), 157171. Fuchs, L. S., Fuchs, D., & Speece, D. L. (2002). Treatment validity as a unifying construct for identifying learning disabilities. Learning Disability Quarterly, 25, 33-45. Gersten, R., Beckmann, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools. (NCEE 2009-4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Gersten, R., Fuchs, L.S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M.S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Council for Exceptional Children, 71(2), 149-164. Hallahan, D. P., Kauffman, J. M., & Pullen, P. C. (2012). Exceptional Learners: An Introduction to Special Education, 12th Edition. Columbus, OH: Pearson. Harrell, P., & Harris, M. (2006). Teacher preparation without boundaries: A two-year study of an online teacher certification program. Journal of Technology and Teacher Education, 14(4), 755-774. Chesapeake, VA: AACE. Hawken, L. S., Vincent, C. G., & Schumann, J. (2008). Response to intervention for social behavior. Journal of Emotional and Behavioral Disorders, 16, 213–225. Hagan-Burke, S., Coyne, M., Kwok, O., Simmons, D., Kim, M., Simmons, L., Skidmore, S., Johnson, C., Ruby, M. (2011) The Effects and interactions of student, teacher, and setting 158       variables on reading outcomes for kindergarteners receiving supplemental reading intervention. Journal of Learning Disabilities. doi: 10.1177/0022219411420571 Hoover, J. J. & Love, E. (2011). Supporting school-based response to intervention: A practitioner's model. Teaching Exceptional Children, 49(3), 40-48. IDEA (2004). Individuals with Disabilities Education Improvement Act. Public Law 108-446. IRIS Center (2012): Fast facts. Retrieved from http://iris.peabody.vanderbilt.edu/about_brochures/IRIS_Fast_Facts_Brochure.pdf IRIS Center (2013a). How to use a module. Retrieved from http://iris.peabody.vanderbilt.edu/instructors/instructorPOP_module_howto.htm IRIS Center (2013b). Field test data. Retrieved from http://iris.peabody.vanderbilt.edu/instructors/eval_data/field_test_data/Fieldtesting_Over view.html IRIS Center (2013c): Resource Locator. Retrieved from http://iris.peabody.vanderbilt.edu/resources.html Kennedy, M. M. (2008). Sorting out teacher quality. The Phi Delta Kappan, 90(1), 59-63. Klingner, J. K., & Edwards, P. (2006). Cultural considerations with response to intervention models. Reading Research Quarterly, 41, 108-117. Klotz, M. B. & Mellard, D. (2008).  Response to intervention and IDEA - LD identification in the RTI instruction model. Retrieved from http://www.ncld.org/learning-disabilityresources/ld-talks/transcript/expert/9 Kratochwill, T. R., Volpiansky, P., Clements, M., & Ball, C. (2007). Professional development in implementing and sustaining multitier prevention models: Implications for Response to Intervention. School Psychology Review, 36(4), 618-631. Lassen, S., Steele, M., & Sailor, W. (2006). The relationship of school-wide positive behavior support to academic achievement in an urban middle school. Psychology in Schools 43(6), 701-712. 159       Lounsbury, J. W., Fisher, L. A., Levy, J. J., & Welsh, D. P. (2009). An investigation of character strengths in relation to the academic success of college students. Individual Differences Research, 7(1), 52-69. Mastropieri, M. A., & Scruggs, T. E. (2005). Feasibility and consequences of response to intervention: Examination of the issues and scientific evidence as a model for the identification of individuals with learning disabilities. Journal of Learning Disabilities, 38, 525-531. Mellard, D. F., Byrd, S. E., Johnson, E., Tollefson, J.M., & Boesche, L. (2004). Foundations and research on identifying model responsiveness-to-intervention sites. Learning Disability Quarterly, 27(4), 243-256. Montrosse, B. E. (2012). IRIS Center Resource Use and Perceived Influence: Survey Results. Cullowhee, NC: Western Carolina University. Murawski, W. W. & Hughes, C. E. (2009). Response to Intervention, collaboration, and coteaching: A logical combination for successful systemic change. Preventing School Failure, 53(4), 267-277. National Association of State Directors of Special Education (2006). Response to intervention: policy considerations and implementation. Alexandria, VA: NASDSE, Inc. National Center on Response to Intervention (NCRTI) (2012). RTI state database. Retrieved from http://state.rti4success.org/ National Center on Response to Intervention (NCRTI) (2013). RTI state database. Retrieved from http://www.rti4success.org/ National Early Childhood Technical Assistance Center (NECTAC) (2012). Response to intervention in early childhood resource. Retrieved from http://nectac.org/topics/RTI/RTI.asp National Research Center on Learning Disabilities (2006). A tiered service-delivery model. Retrieved from http://www.nrcld.org/rti_manual/pages/RTIManualSection3.pdf NCLB (2002). No Child Left Behind Act. Public Law 107-115. 160       Orosco, M. J., & Klingner, J. (2010). One school’s implementation of RTI with English language learners: “Referring into RTI.” Journal of Learning Disabilities, 43, 269–288. Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students: Findings and insights from twenty years of research. San Francisco, CA: Jossey-Bass. Randi, J. & Corno, L. (2005). Teacher and learner variation.  Pedagogy – Learning for Teaching, 47–69. Renzaglia, A., Hutchins, M., & Lee, S. (1997). The impact of teacher education on the beliefs, attitudes, and dispositions of preservice special educators. Teacher Education and Special Education, 20, 360-377. Reschly, D. J., & Hosp, J. L. (2004). State SLD identification policies and practices. Learning Disability Quarterly, 27(4), 197-213. Reschly, D. J., & Wood-Garnett, S. (2009). Teacher preparation for response to intervention in middle and high schools. Washington, DC: Learning Point Associates, National Comprehensive Center on Teacher Quality. Reschly, D. J., & Ysseldyke, J. E. (2002). Paradigm shift: The past is not the future. In A. Thomas & J. Grimes (Eds.) Best practices in school psychology IV (4th Ed.) (pp. 3-20). Bethesda, MD: National Association of School Psychologists. Rinaldi, C., & Samson, J. (2008). English language learners and response to intervention: Referral considerations. Teaching Exceptional Children 40(5), 6-14. Rodriguez, M. A., Gentilucci, J. & Sims, P. G. (2006). Preparing school leaders to effectively support special education programs: Using modules in educational leadership. Paper presented at the Annual Meeting of the University Council for Educational Administration: San Antonio, Texas. Roschelle, J. (1995). Learning in interactive environments: Prior knowledge and new experience. In J. H. Falk & L. D. Dierking (Eds.), Public institutions for personal learning: Establishing a research agenda (pp. 37-51). Washington, DC: American Association of Museums. Rudestam, K. E., & Newton, R. R. (2001). Surviving your dissertation: A comprehensive guide to content and process (2nd ed.). Thousand Oaks: Sage. 161       Sansgiry, S. S., Bhosle, M., & Sail, K. (2006). Factors that affect academic performance among pharmacy students. American Journal of Pharmaceutical Education, 70(5), E1–E9. doi: 10.5688/aj7005104 Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 81-84. Scruggs, T. E., & Mastropieri, M. A. (2002). On babies and bathwater: Addressing the problems of identification of learning disabilities. Learning Disability Quarterly, 25, 155–168. Sharma, S., Dick, G., Chin, W., Land, L.P.W. (2007). Self Regulation and e-learning. European Conference on Information Systems, 383-394. Siegel, L. S. (2003). IQ discrepancy definitions and the diagnosis of LD. Journal of Learning Disabilities, 36, 2-3. Smartt, S., & Reschly, D. (2007). Barriers to the preparation of highly qualified teachers in reading. Washington, DC: National Comprehensive Center for Teacher Quality. Retrieved from http://www.ncctq.org/tqbrief.php Smith, D.D., Pion, G., Skow, K., Tyler, N., Yzquierdo, Z., Givner, C., & Brown, J. (2005). The IRIS Center for Faculty Enhancement: On-line course enhancement modules and materials for use in the preparation of education professionals. New Horizons for Learning, 11, 1-17. Smith, S. B., Smith, S. J., & Boone, R. (2000). Increasing access to teacher preparation: The effectiveness of traditional instructional methods in an online learning environment. Journal of Special Education Technology, 15(2), 37–46. Song, H. S. (2010). The effects of learners’ prior knowledge, self-regulation, and motivation on learning performance in complex multimedia learning environments. New York University, New York. Spear-Swerling, L. & Cheesman, E. A. (2011). Teachers’ knowledge base for implementing response-to-intervention models in reading. Reading and Writing: An Interdisciplinary Journal. Retrieved from http://www.uccs.edu/Documents/coe/people/faculty/cheesmane/SpearSwerlingCheesman RW2011.pdf 162       Spear-Swerling, L. (2008). Response to intervention and teacher preparation. In E. L. Grigorenko (Eds.), Educating Individuals with Disabilities: IDEIA 2004 and Beyond (pp.273-293). New York, NY: Springer Publishing Company. Stage, S. A., Abbott, R. D., Jenkins, J. R., & Berninger, V. W. (2003). Predicting response to early reading intervention from verbal IQ, reading-related language abilities, attention ratings, and verbal IQ-word reading discrepancy: Failure to validate discrepancy method. Journal of Learning Disabilities, 36, 24-33. Stewart, R. M., Benner, G. J., Martella, R. C., & Marchand-Martella, N. E. (2007). Three-tier models of reading and behavior: A research review. Journal of Positive Interventions, 9, 239–252. Strangman, N., & Hall, T. (2000). Background knowledge. Wakefield, MA: National Center on Accessing the General Curriculum. Retrieved from http://www.aim.cast.org/learn/historyarchive/backgroundpapers/background_knowledge Svinicki, M. (2007). What they don’t know can hurt them: the role of prior knowledge in learning. Center for Teaching and Learning, University of Minnesota. Retrieved from http://tls.vu.edu.au/learning_and_teaching/guidelines/ Swierzbin, B., Anderson, M., Morris, F., Klee, C. and Tarone, E. (Eds.) (2000). The interaction of social and cognitive factors in SLA. Proceedings of the 1999 SLRF. Somerville, MA: Cascadilla Press. Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., & Liu, X. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76 (1), 93-135. Thompson, R. A., & Zamboanga, B. L. (2004). Academic aptitude and prior knowledge as predictors of student achievement in introduction to psychology. Journal of Educational Psychology, 96, 778–784. Tyler, N. & Sims, P. (2010). Crossing the borders of disabilities: Providing research-based and technology-delivered information and materials for educators of students with disabilities. A paper submitted to the XIV World Congress of Comparative Education Societies Istanbul, Turkey. Retrieved from http://iris.peabody.vanderbilt.edu/about_papers_reports/Crossing_the_Borders_of_Disabi lities.pdf 163       Vaughn, S., & Fuchs, L. S. (2003). Redefining learning disabilities as inadequate response to instruction: The promise and potential problems. Learning Disabilities Research & Practice, 18(3), 137-146. Vaughn, S., Wexler, J., Leroux, A. J., Roberts, G., Denton, C. A., Barth, A. E., & Fletcher, J. M. (2011). Effects of intensive reading intervention for eighth grade students with persistently inadequate response to intervention. Journal of Learning Disabilities, 45(6), 515-525. Wanzek, J. & Vaughn, S. (2008). Response to varying amounts of time in reading intervention for students with low response to intervention. Journal of Learning Disabilities, 41(2), 126-42. Wollack, J. A. (2003). Helpful tips for creating reliable and valid classroom tests: Writing multiple-choice questions. The Learning Link, 4 (1). Retrieved from http://testing.wisc.edu/LL10-03.pdf 164