UPPER ELEMENTARY SOCIAL STUDIES INSTRUCTION: EXPLORING THE RELATIONSHIP BETWEEN TEXT USE, COMPREHENSION INSTRUCTION, AND STUDENT ENGAGEMENT By Heather A. Schmitt Submitted to Michigan State University in partial fulfillment of the requirements for the degree of School Psychology–Doctor of Philosophy A DISSERTATION 2018 ABSTRACT UPPER ELEMENTARY SOCIAL STUDIES INSTRUCTION: EXPLORING THE RELATIONSHIP BETWEEN TEXT USE, COMPREHENSION INSTRUCTION, AND STUDENT ENGAGEMENT By Heather A. Schmitt Despite increasing expectations for student proficiency in content area reading (e.g., Common Core State Standards), social studies text continues to present unique challenges to students who struggle with reading; for example, textbook readability is often at least two grade levels above the grade for which it is intended (Berkeley et al., 2016; Jitendra et al., 2001). Given the shift toward greater use of demanding informational text around fourth grade, students who struggle with reading stand to benefit from direct teacher support in reading and comprehending social studies texts. Research suggests that one such way to support student learning in the content areas is in the form of comprehension instruction, which includes practices like direct instruction, modeling, collaborative strategy use, guided practice, and independent strategy use (e.g., Duke & Pearson, 2009; Vacca, Vacca, & Mraz, 2016). Moreover, it is important to consider the ways in which classroom factors like these might promote or undermine student engagement in social studies learning activities (Skinner & Pitzer, 2012). The purpose of this study was to systematically observe the use of text materials and the extent to which student reading of those texts is supported through comprehension instruction. The relationship between these classroom-level factors and student engagement was also explored, including closer examination of these relationships among struggling readers. Descriptive and multilevel analyses were used to address the research questions and quantitative variables of interest. Across the fifty fifth-grade classrooms observed, reading comprehension strategies were frequently incorporated throughout social studies instruction. However, comprehension instruction was largely student-centered in nature (collaborative practice with a peer, independent practice) and rarely included teacher-centered practices such as direct instruction or modeling of comprehension strategies. A strong relationship was not found between the reading- related variables of the present study and student engagement as hypothesized, though an unanticipated significant interaction effect on student engagement was identified for text readability level and comprehension instruction. Results are discussed in terms of the current literature on content area instruction and student engagement, and the implications for both research and practice are described. Copyright by HEATHER A. SCHMITT 2018 ACKNOWLEDGEMENTS I would like to thank my advisor, Dr. Sara Witmer, for her guidance and support throughout this ambitious dissertation project. Your time spent thinking about and discussing my research critically was always appreciated. Thank you to Dr. Carol Sue Englert, Dr. Emily Bouck, and Dr. Cary Roseth for your willingness to serve on my dissertation committee and your additional perspectives, feedback, and support. Thank you to the countless teachers and administrators throughout Michigan for their willingness to participate in this project, and to the College of Education at Michigan State University for providing substantial financial support for this research. I am eternally thankful to my rock star research assistants, Courtney Chamberlain, Courtney Darr, and Justina Yohannan, for their efforts and quality of work. Thank you to my graduate school colleagues and friends, particularly June Westdal, Marianne Clinton, Allie Siroky, and Kiley Hierl. I owe you a debt of gratitude for the coffee shop writing dates, stress vent sessions, feedback on my work, and laughter boosts. An important thank you to my non- graduate school friends, especially my rugby family and Lansing crew, who have put up with me and lifted me throughout this journey. Thank you to my fellow Boys Town doctoral interns and to my internship supervisors for their invaluable professional and personal support this past year. This dissertation is dedicated to my forever cheerleaders – my family. Mom, Dad, Preston, Kendra, and Matt, you deserve the biggest of thanks yous. Thank you for teaching me the value of non-stop effort, following your passion, and remaining grounded in achieving your goals. Thank you for always having a sarcastic quip or supportive word, just when I needed them the most. v TABLE OF CONTENTS LIST OF TABLES .................................................................................................................... ix LIST OF FIGURES .................................................................................................................... x CHAPTER I: INTRODUCTION ................................................................................................ 1 Legal Context .................................................................................................................. 1 Common Core State Standards ........................................................................................ 2 Struggling Readers .......................................................................................................... 3 Challenging Texts ................................................................................................ 5 Importance of Social Studies Instruction.......................................................................... 6 Student Engagement ........................................................................................................ 9 Visual of Present Study ................................................................................................. 11 CHAPTER II: LITERATURE REVIEW................................................................................... 13 Theoretical Framework and Model ................................................................................ 13 Models of Student Engagement.......................................................................... 13 Comprehension Instruction Model .................................................................... 16 Theoretical Framework and the Present Study ................................................... 18 Content Area Instruction and Reading ........................................................................... 19 Learning Expectations ...................................................................................... 19 Reading Material and Proficiency Mismatch ..................................................... 21 General textbook features. ..................................................................... 21 Textbook readability. ............................................................................. 24 Instructional Practices: Textbook Use ............................................................... 24 Addressing textbook use. ....................................................................... 26 Instructional Support in Reading ................................................................................... 27 Reading Comprehension Strategy Instruction ..................................................... 28 Vocabulary Instruction ....................................................................................... 30 Instructional Support in Reading: State of the Literature ................................... 32 Comprehension Instruction Components vs. Comprehension Strategies ............. 33 Engagement .................................................................................................................. 34 Defining Engagement ........................................................................................ 34 Engagement in the Present Study ....................................................................... 36 Engagement and the Classroom Context ............................................................ 37 Student Engagement: State of the Literature....................................................... 39 The Present Study ......................................................................................................... 40 Research Questions (RQs) ............................................................................................. 42 CHAPTER III: METHOD ........................................................................................................ 45 Sample .......................................................................................................................... 45 Sample Size ...................................................................................................... 45 Inclusion Criteria .............................................................................................. 46 vi Participants ....................................................................................................... 46 Variables ....................................................................................................................... 48 (a) Student Engagement ..................................................................................... 48 (b) Reading Achievement ................................................................................. 51 (c) Text Readability ........................................................................................... 53 (d) Reading Comprehension Instruction ............................................................. 54 Procedures ..................................................................................................................... 56 Recruitment ...................................................................................................... 56 Observer Training and Agreement .................................................................... 58 Conducting Observations ................................................................................... 60 Student Assent ................................................................................................... 61 Student Engagement and Reading Levels .......................................................... 62 Reading Materials and Teacher Items ................................................................ 62 Overview of Data Analyses ........................................................................................... 63 CHAPTER IV: RESULTS ........................................................................................................ 65 Descriptive Information ................................................................................................. 65 Preparing Data for Analyses .......................................................................................... 67 Descriptive Statistic Analyses: RQ 1 and RQ 2 .............................................................. 68 RQ 1 ................................................................................................................. 68 RQ 2 ................................................................................................................. 68 Preliminary Analyses for Multilevel Models (RQ3, RQ4, RQ5) .................................... 70 (a) No Multicollinearity ..................................................................................... 70 (b) Linear and Normal Distribution of Residuals ............................................... 71 (c) Level 1 and Level 2 Predictor Error Terms ................................................... 71 Multilevel Models ........................................................................................................ 71 Model 1: Baseline Model (Null) ........................................................................ 73 Model 1: Baseline Model (Null) for Struggling Readers ..................................... 75 Model 2: Random-Intercepts Model (Level 1 Predictor, RQ3) ........................... 75 Model 3: Means as Outcomes Model (Level 2 Predictors, RQ4) ........................ 77 Model 3: Means as Outcomes Model (Level 2 Predictors, RQ4) for Struggling Readers ........................................................................ 78 Model 4: Random Intercepts and Slopes Model (Moderation, RQ5) ................... 79 Model 4: Random Intercepts and Slopes Model (Moderation, RQ5) for Struggling Readers ........................................................................ 81 CHAPTER V: DISCUSSION ................................................................................................... 83 Research Question 1 ...................................................................................................... 83 Research Question 2 ...................................................................................................... 84 Research Question 3 ...................................................................................................... 89 Research Question 4 ...................................................................................................... 90 Research Question 5 ...................................................................................................... 92 Additional Findings ....................................................................................................... 94 Limitations and Future Research ................................................................................... 97 Potential for Subject Reactivity .......................................................................... 97 Measurement Issues ........................................................................................... 98 vii Student self-report. ................................................................................ 98 Reading measure. ................................................................................... 99 Observational measure. ......................................................................... 99 Reliability measure. .............................................................................. 100 Aggregated information. ...................................................................... 100 Representativeness of Sample .......................................................................... 101 Limited Sample Size ........................................................................................ 102 Methodology and Model of Comprehension Instruction ................................... 102 Implications for Future Research ..................................................................... 103 Implications for Practice .............................................................................................. 105 Conclusions ................................................................................................................. 107 APPENDICES ....................................................................................................................... 108 APPENDIX A: Student Engagement Survey (EvsD) ................................................... 109 APPENDIX B: Group-Administered Reading Assessment (DIBELS Daze) ................ 111 APPENDIX C: Observation Tool ................................................................................ 116 APPENDIX D: Administrator Recruitment Email ....................................................... 118 APPENDIX E: Teacher Recruitment (Email 1) ........................................................... 119 APPENDIX F: Teacher Consent Form ........................................................................ 120 APPENDIX G: Teacher Demographic Survey and Observation Scheduling (Email 2)............................................................................................. 123 APPENDIX H: Teacher Instructions for Parent Consent (Email 3) .............................. 125 APPENDIX I: Parent Cover Letter .............................................................................. 126 APPENDIX J: Parent Information Form ...................................................................... 127 APPENDIX K: Parent Reminder ................................................................................. 129 APPENDIX L: Research Assistant Training Manual ................................................... 130 APPENDIX M: Observation Reminder Email ............................................................. 148 APPENDIX N: Fidelity Checklist ............................................................................... 149 APPENDIX O: Student Assent Form .......................................................................... 150 APPENDIX P: Teacher and Student Likert Items ........................................................ 152 REFERENCES ...................................................................................................................... 153 viii LIST OF TABLES Table 1 Teacher Demographics ................................................................................................. 46 Table 2 EvsD Student Self-Report Survey Items and Subscales ................................................ 48 Table 3 Reading Comprehension Instruction Components ........................................................ 55 Table 4 Interobserver Agreement Levels: RA Training Videos ................................................. 59 Table 5 Interobserver Agreement Levels: Double-Coded Observations ..................................... 60 Table 6 Overview of RQs, Variables, Measures, and Analysis .................................................. 63 Table 7 Descriptives for Predictor and Outcome Variables: All Students................................... 65 Table 8 Specific Comprehension Instruction Strategies Observed: Frequencies ......................... 69 Table 9 Pearson Correlation Among Predictors ......................................................................... 71 Table 10 Results of Multilevel Modeling Analyses: All Students .............................................. 72 Table 11 Results of Multilevel Modeling Analyses: Struggling Readers .................................... 73 Table 12 Master Grid of Definitions: Strategy/Model Component Examples ..................... …..139 Table 13 RA Training: Paper/Pencil Applied Test Items ................................................... …..145 ix LIST OF FIGURES Figure 1 Visual of present study conceptual framework. .......................................................... 12 Figure 2 Multilevel Model of Engagement. Dashed arrows depict interactions. . ....................... 14 Figure 3 Student Engagement Core (SEC) model. Student engagement italicized for emphasis. .................................................................................................................. 15 Figure 4 Relationships examined through RQ3, RQ4, and RQ5 multilevel models. .................. 44 Figure 5 Reading comprehension instruction and text level interaction. ..................................... 82 x CHAPTER I: INTRODUCTION The purpose of this quantitative observational study was to investigate the use of text materials during social studies instruction, as well as the extent to which teachers incorporate reading comprehension instruction. Further, this study explored the relationship between these two classroom factors and student engagement during instruction. Within this introduction section, a legal context will first be provided with respect to addressing the educational needs of all students (including those with reading difficulties). Then, the Common Core State Standards (CCSS) will be described within the framework of this legislation, including the CCSS’s influence on reading expectations for all students in content area instruction and assessment. Third, the difficulty upper elementary students continue to demonstrate in meeting reading proficiency standards will be described, including the unique challenges that informational social studies texts present to students. The importance of social studies content and instruction will then be highlighted, including factors that have contributed to a shift toward consideration for student proficiency within this domain. Lastly, the ways in which researchers have found engagement to be an important factor in student success, including the development of reading proficiency, is described. Legal Context No Child Left Behind (NCLB) was signed into law in 2002, aiming to increase the quality and effectiveness of schools by raising the achievement levels of all students (particularly those who may be struggling). As a result of this law, schools were considered increasingly responsible for providing the quality of instruction necessary for all students to meet achievement standards. Many schools began to shift their focus toward reading and mathematics in the early 2000s, as student performance in these two domains was to be measured annually 1 through standardized tests. Although NCLB was intended to improve instruction, there were some unintended negative consequences of NCLB provisions. For example, many schools began to focus less on untested areas like science, social studies, and the arts (Applebee, 2013; Coleman 2011). The Every Student Succeeds Act (ESSA) was subsequently passed in 2015, with continued emphasis on the need for schools to be held appropriately accountable for the learning of all students. However, the ESSA added mandates with the intent of providing students with access to a “well-rounded” education. For instance, the ESSA provides funding opportunities to support STEM education and expand social studies programs like history, civics, government, and economics. Common Core State Standards Developed in 2010 and adopted by 38 states as of 2017, the Common Core State Standards (CCSS) were designed to prepare all students for success in college, career, and life by the time they graduate from high school (Norton, Ash, & Ballinger, 2017). Similar to NCLB and ESSA efforts, these standards were created to ensure that all students served within public schools are being held to appropriate expectations. One domain containing particularly comprehensive standards is reading. Kindergarten through twelfth grade (K-12) standards in English Language Arts require that students read stories and literature, as well as more complex texts that provide facts and background knowledge in areas such as science and social studies (Common Core State Standards [CCSS], 2010). Starting in second grade, the CCSS include a reading standard stating that students will be able to “read and comprehend informational texts, including history/science, social studies, and technical text, in the grades 2-3 text complexity band proficiently, with scaffolding as needed at the high end of the range” (CCSS, ELA- Literacy.RI.2.10, 2010). By fifth grade, students are expected to be at the high end of the grade 2 4-5 band for reading and comprehending these informational texts independently and proficiently. As students advance to grades 6-12, states determine how to either (a) incorporate literacy standards specific to history/social studies, science, and technical subjects into their existing standards for those subjects or (b) adopt them as content area literacy standards. For example, Michigan has specific reading standards for literacy in history/social studies for students in grade 6-8, 9-10, and 11-12 (Michigan Common Core State Standards, 2010). In sum, the CCSS emphasize reading and comprehending informational texts. This includes content-rich texts that provide facts and background knowledge in areas like social studies and science, beginning in the elementary grades (CCSS, 2010). Given the CCSS expectations described above, “the responsibility of preparing students to read, write, talk, and think critically about complex texts and across such [content area] text is no longer just the English teacher’s job (Schoenbach, Greenleaf, & Hale, 2010, p. 39). As Snow and Moje (2010) noted, students need literacy skills specific to each discipline to understand content area texts. Comprehension skills taught in English class are useful, but not enough for these subjects. However, reviews of research on teacher perceptions and beliefs suggest that many teachers show resistance to building literacy instruction into content area classes (e.g., Hall, 2005). Many teachers also assume students reach middle school having already learned the literacy skills needed to comprehend content area text materials (Zipperer et al., 2002). Yet, it appears many students struggle to meet expected reading proficiency levels and standards (to be detailed below). Struggling Readers Accountability testing results suggest that one area in which all students continue to demonstrate difficulty is reading. The National Assessment of Educational Progress (NAEP) has 3 become the largest nationally representative assessment of what students know at grades 4, 8, and 12. The most recent assessment results available indicated that 37% of fourth-grade students, 36% of eighth-grade students, and 37% of twelfth-grade students perform at or above the proficient levels in reading (National Center for Educational Statistics, 2015 and 2017). Thus, nearly two-thirds of students are still reading below expected proficiency levels in fourth grade, continuing through middle and high school levels. It appears that beginning in upper-elementary school, the majority of students have difficulty accessing instruction that requires reading, likely given their lack of proficiency in these skills. Further, students’ reading proficiency may vary within a classroom across several skills; for instance, students may differ in their ability to decode text, read fluently, understand vocabulary, and comprehend text (Vellutino, 2003). Given the legal context described prior (e.g., accountability, standards), it is important to consider the ways in which instruction is accessible to all students in the classroom setting, including those who may not be proficient readers. Although a variety of terms and descriptions have been used to characterize students not reading at expected proficiency levels (e.g., disabled readers, poor readers, problem readers), commonly used terms by current researchers and educators include at-risk students or struggling readers (e.g., in NCLB, 2001). Allington (2006) simply defines a struggling reader as a student who has difficulties learning to read, while Lapp and colleagues (2003) more specifically define struggling readers as students unable to read grade-level text fluently and with comprehension. Students are often considered struggling readers if they score below grade level on accountability standardized tests or other assessment measures (IDEIA, 2004; NCLB, 2001). According to the NAEP, students in fourth grade should be able to locate relevant information, integrate information, evaluate how information is presented, and understand the purpose of text features when reading informational texts 4 (National Center for Educational Statistics, 2017). Therefore, it seems important that upper elementary students, including struggling readers, receive instruction and have supported practice opportunities to develop skills for comprehending grade-level informational text. Challenging Texts One factor that may influence the ability for students to read proficiently during instruction is the nature of the text being read. Informational text presents unique reading challenges to students, particularly within the content area of social studies. Social studies texts consistently demonstrate high reading levels; researchers have found these texts to be at least two grade levels beyond the grade for which they are intended to be used (e.g., Berkeley et al., 2016; Jitendra et al., 2001); for example, a sixth-grade text would require eighth-grade reading skills. Moreover, these social studies textbooks appear to be used extensively by teachers as a primary instructional resource (e.g., Benavot, 2011; Zhao & Hoge, 2005) – making the ability to read and understand them that much more important for student learning. Even students who demonstrate proficient reading skills begin to struggle with the demands of content area texts in fourth grade (Allington, 2002). Informational text presents new reading challenges, as students tend to have greater experience with narrative text (i.e., stories) in the early elementary grades. For example, specialized and complex vocabulary alongside the unclear text structures commonly found within social studies textbooks can make comprehension difficult for both struggling readers and those who have historically met literacy expectations (Allington 2002; Berkeley et al., 2014, 2016; Ness, 2007; Reutzel et al., 2002). Despite these well-established difficulties that students have with text, social studies teachers have been observed spending just 10% of their instructional time helping students understand assigned texts (Ness, 2007). Furthermore, researchers found that 97% of teachers use 5 social studies textbooks on a regular basis at the elementary and secondary levels (Ingrao & Middleton, 2012). Struggling readers’ difficulties in navigating these commonly-used texts during instruction can result in accumulating deficits in content area knowledge (Chambliss & Calfee, 1998). Students with lower topic knowledge, in turn, contribute less and are less engaged in classroom discussions, are less able to answer questions, and recall less information after reading than students with greater topic knowledge (Reutzel et al., 2002). Cunningham and Allington (1999) note that classroom teachers are an important factor in the success or failure of struggling readers, as teachers “are responsible for minute-by-minute instruction; the decisions teachers make and the kind of instruction and support they provide make the difference between success and failure” (p.3). In sum, students’ success in social studies depends on their ability to read textbooks and develop content knowledge, which highlights the importance of helping students access text materials to ultimately meet learning expectations. Importance of Social Studies Instruction With an increased focus on accountability and high-stakes testing, areas such as reading, writing, and math have historically been the focus for schools, at the expense of other content areas (Applebee, 2013; Coleman 2011). A study involving over 900 principals supported this notion, concluding that post-NCLB schools are spending more time on the tested areas of reading, math, and science, while at the same time de-emphasizing social studies, civics, geography, languages, and the arts (von Zastrow & Janc, 2004). A report from the Center on Education Policy similarly found six years following the enactment of NCLB, 44% of elementary schools increased time for ELA and/or math, while cutting time in subjects like science, social studies, and the arts (McMurrer, 2007). Further analysis revealed instructional 6 time was cut from social studies in particular by an average of 76 minutes per week, reflecting a 32% decrease from pre-NCLB instructional time (McMurrer, 2008). Although policies such as NCLB shifted attention toward reading and math instruction in the early 2000s, schools now appear to be giving greater consideration for student proficiency in social studies. For instance, the Center for Information and Research on Civic Learning and Engagement (CIRCLE) released a report stating that in the 2012-13 school year, 21 states required a state accountability test in social studies, a dramatic reduction from pre-NCLB when 34 states regularly conducted social studies assessments. However, a more recent CIRCLE report found that there was an increase from 21 to 25 states requiring a state-designed social studies test as of the 2014-15 school year. The Education Commission of the States (2018) notes that three additional states will be requiring a state-designed social studies test as of the 2018-19 or 2019- 20 school year. Although states have variation in their social studies assessments, these numbers seem to reflect a changing landscape and renewed interest in advancing social studies knowledge among students. It remains important to understand what might facilitate student proficiency in social studies for several reasons. First, tests measuring knowledge of social studies topics like civics and U.S. history indicate student underachievement. Compiled results from the NAEP show that just 27%, 23%, and 24% of students in the nation in grades 4, 8, and 12 (respectively) are at or above proficiency in civics. The percentages for U.S. history are lower, with 20%, 18% and 12% of students scoring at or above proficiency levels. These scores appear significantly lower than mathematics and reading, which reach up to 40% proficiency levels. In addition, the ESSA (2015) described prior has presented funding opportunities to expand social studies programs, acknowledging its importance in creating a “well-rounded student". As the chairman of the board 7 overseeing the NAEP stated, “Geography, U.S. history, and civics are core academic subjects that must be a priority…the lack of knowledge on the part of America’s students is unacceptable” (National Assessment Governing Board, 2015). Social studies continues to be a core part of the curriculum, and attention appears to be shifting toward whether or not students are learning social studies content and reaching appropriate levels of proficiency. Despite concerns such as low proficiency levels in social studies and the use of challenging text materials, teachers appear to provide limited support for content area literacy skill development (Bolinger & Warren, 2007; Russell 2010). For instance, a national survey by Russell in 2010 found that social studies teachers encouraged passive and more traditional learning methods like textbook homework assignments, extensive teacher talk (e.g., lecturing), recitation, and seatwork. Moreover, social studies teachers are likely to assign reading tasks focused on basic comprehension and summarizing information, with an emphasis on memorizing facts as a means of understanding content (Boilinger & Warren, 2007; Monte-Sano, 2011; Russell, 2010). This type of instruction does not seem to parallel the movement toward CCSS- aligned curriculum and assessments. Further, the CCSS calls for a shared responsibility in literacy instruction among content area teachers: “Literacy standards for grade 6 and above are predicated on teachers of ELA, history/social studies, science, and technical subjects using their content area expertise to help students meet the particular challenges of reading, writing, speaking, listening, and language in their respective fields” (CCSS, 2010). It remains important for teachers in the content areas, including social studies, to use their expertise to help students read and ultimately learn from content area text. 8 Student Engagement The topics described thus far (e.g., legislation, CCSS, text materials, instructional focus) depict the context surrounding student learning, including factors like text difficulty, student reading proficiency, and emphasis on social studies instruction. These contextual factors have been of interest among researchers focused on student learning outcomes, such as concept knowledge and proficiency levels. However, another body of researchers has focused on the influence these contextual factors may have on student engagement. These scholars often describe engagement as “the direct and only pathway to cumulative learning, long-term achievement, and eventual academic success” (Skinner & Pitzer, 2012, pp. 23-24). In other words, there is merit in considering the ways in which classrooms are promoting or undermining student engagement, the “direct and only pathway” to learning. Despite varying conceptualizations of engagement (to be described in the literature review), reviews of research over the past decade have largely converged in finding a wealth of research supporting student engagement as a key contributor to children’s success in school (Corso et al., 2013; Fredricks, Blumenfeld & Parks, 2004). Student engagement has been identified as a strong predictor of broad outcomes like school attendance, retention, and graduation (Connell et al., 1995; Connell, Spencer, & Aber, 1994; Sinclair et al., 2003; Skinner et al., 1998). At a more local level, researchers have found that student engagement is a robust predictor of classroom learning, grades, standardized test scores, and conduct in school (Hill & Werner, 2006; Marks, 2000; Skinner & Belmont, 1993). A lack of engagement can begin in the early school years if students do not participate or experience success (Finn, 1989); this disengagement adversely affects student achievement, and can initiate a downward spiral for students already struggling in school (Finn, 1989; Newmann, 1992; Steinberg, 1996). 9 Researchers have specifically associated classroom engagement with reading achievement (Ivey & Johnston, 2013), emphasizing the significance of engagement with text and proficiency in reading (Dolezal, Welsh, Pressley, & Vincent, 2003; Taylor, Pearson, Clark, & Walpole, 2000; Wharton-McDonald, Pressley, & Hampston, 1998). For example, an extensive analysis of Program for International Student Assessment results by Brozo, Shiel and Topping (2008) identified student engagement with reading as one of the most powerful factors affecting reading achievement. Moreover, the strong connection between student engagement, achievement (including reading scores), and school behavior outcomes is supported empirically across varying economic and social status levels (e.g., Klem & Connell, 2004). While extensive evidence exists linking student engagement with beneficial individual outcomes (e.g., National Research Council and Institute of Medicine, 2004), it is worth highlighting the benefits for teachers as well. When students in a class are engaged (e.g., participating, paying attention, interested), teachers can allocate more time and effort to promoting learning -- and less to managing distractions in the learning environment (Corso et al., 2013). Additionally, a teacher with engaged students is less likely to experience burnout and more likely to feel invested in instructional activities (Covell, McNeil, & Howe, 2009). Research on exemplary reading teachers in particular demonstrates that high levels of student engagement differentiate higher reading performance classrooms from lower performing classrooms (Pressley & Allington, 2015). Considering an increasing emphasis on student performance outcomes as part of teacher evaluations, these findings further highlight the importance of facilitating student engagement for teachers. It is important to note that although there is a clear need to consider students’ engagement with their academic work, most studies to date have addressed how students engage with school 10 in general. Many researchers define and measure engagement at the school-wide level, often through student self-report of what they think about school or observed behaviors such as the number of school-related activities in which a student participates (Corso et al., 2013; Fredericks et al., 2011). Yet, many aspects of student engagement are specific to a particular context, defined by classroom-level experiences. As Corso and colleagues (2013) describe, the classroom setting is where “the rubber of students’ desire and need for being engaged meets the road of what schools have to offer that may be engaging” (p. 53). For example, if students find the content area texts that they are asked to read too difficult, and teaching practices are not in place to facilitate engagement, students may avoid reading and learning about important content area topics (Strommen & Mates, 2004). In sum, theory and beginning empirical work note the importance of student engagement in general, suggesting that how classroom instruction is designed and provided may impact student engagement (Perry, Turner, & Meyer, 2006). However, it remains unclear the extent to which reading instructional practices may promote student engagement in the classroom, thus facilitating learning. Researchers have historically investigated these instructional practices broadly (e.g., “teacher support”) or qualitatively, leaving a need to empirically study the link among specific characteristics of classroom instruction and student engagement. Visual of Present Study The present study will systematically explore the relationships between the factors described above (i.e., social studies text difficulty, student reading achievement, reading comprehension instruction, student engagement). This is illustrated in Figure 1 below. 11 Figure 1 Visual of present study conceptual framework. 12 CHAPTER II: LITERATURE REVIEW This literature review draws on research from several areas relevant to the present study. First, several theoretical models of student learning will be described to provide context for the importance of studying classroom variables including comprehension instruction and student engagement. Next, literature related to content area instruction and reading will be described. More specifically, the shift toward the use of informational text in upper elementary school and the mismatch between student reading proficiency and text materials will be discussed. Research on accessing instruction will then be reviewed, describing various reading comprehension instructional strategies that might particularly benefit struggling readers. Then, research about engagement is presented. This includes student outcomes related to student engagement, as well as the relationship between student engagement and the classroom context. This literature will serve as a foundation for the need to empirically investigate the relationship between content area text and student engagement, including the potential moderating effect of reading comprehension instruction. Theoretical Framework and Model The present study is informed by several theoretical models of student learning: (a) Skinner and Pitzer’s (2012) Multilevel Model of Engagement, (b) Corso and colleagues’ (2013) Student Engagement Core model, and (c) Duke and Pearson’s (2009) model of comprehension instruction. Each of these will be described below, providing an important context for the present study. Models of Student Engagement As Perry and colleagues (2006) acknowledged, how instruction is designed and provided in the classroom has a significant impact on student engagement. Models of student engagement 13 within the literature provide a helpful framework for considering its relationship with the surrounding learning environment. Skinner and Pitzer (2012) conceptualized engagement within four nested levels (see Figure 2 below). The level of engagement that is the focus of the present study is within the classroom, termed student engagement with academic work. Within this level, the interaction between a student, the curriculum, the teacher, and peers during specific academic tasks are captured. These factors collectively impact outcomes including student learning, coping, and resilience. Figure 2 Multilevel Model of Engagement. Dashed arrows depict interactions. As Skinner and Pitzer described, the academic work level of engagement is especially important to study. Engagement at this level is necessary for students to learn, reflecting student participation in the curriculum both behaviorally and emotionally. Researchers have described this engagement as “hands-on” and “heads-on,” representing constructive, enthusiastic, willing, emotionally positive, and cognitively focused participation in classroom activities that ultimately contribute to learning (Connell & Wellborn, 1991; Skinner, Kindermann, Connell, & Wellborn, 2009; Skinner & Pitzer, 2012). Corso and colleagues provided a model focused on understanding factors contributing to the academic work level of engagement called the Student Engagement Core (SEC) model (Corso, Bundick, Quaglia, & Haywood, 2013). As depicted in Figure 3 below, three primary 14 classroom elements – teacher, student and content – intersect and create classroom interactions. The classroom interaction where all three of these elements intersect at the center is student engagement. Figure 3 Student Engagement Core (SEC) model. Student engagement italicized for emphasis. The SEC model proposes student engagement is highly likely in classrooms where students: consider the teacher an expert in the content and effective in helping them learn it, perceive a positive and supportive relationship with the teacher, and view class content/materials to be relevant (Corso, Bundick, Quaglia, & Haywood, 2013). To this end, student engagement depicts the “processes” and interactions between the content, the teacher, and the student. The teacher and content represent the classroom context surrounding the student, features of which scholars hypothesize to be critical in promoting engagement (to be described in subsequent sections). The creators of the SEC model noted that research is needed to investigate some of these relationships that might foster student engagement in the classroom, emphasizing the benefit of mixed-methods approaches that collect data from multiple sources (e.g., observation, student report) (Corso, Bundick, Quaglia, & Haywood, 2013). To this end, the present study aims to 15 explore how several of the factors and relationships in the SEC model may facilitate student engagement in the social studies classroom. It will focus on all three primary classroom elements and their interactional influence on student engagement – the teacher (reading comprehension instruction, text characteristics), student (reading achievement level) and content (social studies). Ultimately, fostering student engagement in the classroom by helping students access and comprehend social studies informational text should promote success and learning. As Skinner and Pitzer (2012) noted, engagement is “the direct and only pathway to cumulative learning, long-term achievement, and eventual academic success” (pp. 23-24). Comprehension Instruction Model An important aspect of the teacher factor within the SEC model that might promote or undermine engagement when text is present is the nature of classroom instruction. Duke and Pearson (2009) described a research-supported model specific to reading comprehension instruction (the focus of the present study). This model presents one way in which barriers to engagement presented by text might be overcome, based on early conceptualization of the comprehension instruction environment (e.g., work by Pearson and Gallagher, 1983). It is comprised of five components with instructional practices that range from highly teacher- centered to highly student-centered, with shared responsibility in between. Comprehension instruction should be balanced to include both the explicit instruction of comprehension strategies (teacher-centered components) and opportunities for actual reading and discussion of text (student-centered components) (Duke & Pearson, 2009). The model integrates different learning opportunities into the five components as follows: (1) explicit description of the strategy and when/how it should be used (direct instruction) (2) teacher and/or student modeling of the strategy in action (3) collaborative use of the strategy in action (4) guided practice using the strategy and 16 (5) independent use of the strategy As Duke and Pearson (2009) explained, it is important that a teacher and his or her students use multiple strategies constantly, as good readers coordinate the use of comprehension strategies (e.g., not just making predictions). The authors continued to describe several of these comprehension strategies that research suggests are beneficial to teach to developing readers through this model. These strategies include prediction/prior knowledge, think-aloud, identifying text structure, visual representations of text (e.g., graphic organizer, flowchart), summarization, and generating or asking questions. They further acknowledged that research supports cooperative learning as an effective comprehension strategy, but conceptualize this as an instructional medium rather than a comprehension strategy. In other words, peers can be a modality through which the instructional practices and comprehension strategies outlined above might be presented. Given the model of the comprehension instruction environment outlined above, the present study aimed to observe the instructional practices, and more specifically comprehension strategies, present during content area instruction. This model served as a foundation for the observation tool used during social studies instruction. The instructional components as defined and described above were recorded (i.e., direct instruction, modeling, collaborative use, guided practice, and independent use), as well as the specific comprehension strategies (e.g., prediction, visual representation). It is important to note that the present study did not intend to study these components longitudinally during instruction. For instance, the goal was not to see how a teacher might implement the prediction strategy in the classroom, perhaps moving between teacher-centered instruction and more student-centered. Rather, the present study observed a particular strategy at 17 a particular moment in time -- which Duke and Pearson acknowledged is what forms this model (2009). Further, Duke and colleagues (2011) noted that despite the model describing movement from teacher- to more student-centered components, teachers must get used to sliding along the continuum as circumstances demand; “students will inevitably end up back in the middle or even sometimes in the upper left-hand corner” of explicit instruction along the continuum (Duke, Pearson, Strachan, & Billman, 2011). Therefore, all comprehension instruction practices were considered in the present study, along with other factors within the learning environments (e.g., text difficulty, student engagement). The present study used this approach to explore the direct relationship between comprehension instruction and student engagement, which has received limited empirical attention within content area reading (to be described at length in the remaining literature review). Theoretical Framework and the Present Study In sum, the present study focused on the classroom environment as described within the Multilevel Model of Engagement and Student Engagement Core. These models depict the various factors and processes within the classroom environment that might promote (or undermine) student engagement. One key influencer of student engagement specific to reading demands is the nature of comprehension instruction. As described by Duke and Pearson’s model (2009), reading comprehension instruction might include practices like explicit instruction, modeling, collaboration, guided practice, and independent strategy use. These theoretical models provide a collective context for the importance of examining classroom factors, especially comprehension instruction, that might be expected to facilitate student engagement during social studies instruction, thereby promoting learning for all students. These models provided a theoretical foundation for the present study to more systematically explore the extent to which 18 these factors might promote student engagement when reading demands are present. Specifically, an empirical investigation of these relationships during social studies instruction can help us better understand how students can be engaged when the expectation is to learn from challenging social studies text. Content Area Instruction and Reading Learning Expectations As students progress through schooling, the ability to read and comprehend informational texts becomes increasingly important. Beginning in fifth grade, students are expected to read to learn independently across the content areas (Chall, 1983). For instance, within the Common Core State Standards (National Governors Association, 2010) students in second grade are expected to be able to read and comprehend informational text (including history/social studies) with scaffolding as needed. By fifth grade, students are expected to be able to read and comprehend informational text (including history/social studies), independently and proficiently. Expectations continue to heighten for students in middle school, as sixth through eighth grade standards become specific to literacy in history/social studies, stating students will be able to once again read and comprehend these texts independently and proficiently. In sum, the expectation for students to demonstrate independence and depth in their processing of content area texts, including social studies, begins in upper elementary school. Empirical research and experts in reading development and instruction highlight several factors that likely contribute to student difficulty in meeting these expectations, including (a) limited exposure to informational text and (b) unique features of informational text (e.g., Allington, 2005; Duke, 2000; Lubliner, 2004; Spor, 2005) As described by Spor, an expert in literacy and textbook reading, most children learn to read through “a diet of narrative literature” 19 until upper elementary grades, when texts become largely expository in nature (2005). Research findings support this notion; for example, Duke (2000) found an average of just 3.6 minutes per day were allocated to informational text in first-grade classrooms. Jeong, Gaffney, and Choi (2010) used the same procedures as Duke (2000) in fourth-grade classrooms, finding that informational text was used an average of 16 min within just four hours of instruction. Allington and Johnston (2002) described fourth grade as the point at which informational texts traditionally become curricular pedestals, and students are expected to acquire information from these texts. Furthermore, experts note that the assumption that once children learn to read they will be able to read to learn content area text is not sound (Allington, 2005; Spor, 2005). This relates to (b), the unique features inherent to informational text. For instance, expository text is often schematically unfamiliar to young readers, containing new topics and uncommon text structures (Harvey, 1998). As literacy expert Lubliner (2004) explained, it is at the fourth-grade level that the nature of informational text tends to change as do reading demands (linguistically, cognitively, conceptually); consequently, even students with prior success may begin to experience sudden reading difficulties. Struggling readers falling below grade level may especially experience challenges with this shift in text from stories to facts and information, in turn potentially falling further behind academically as they progress through schooling (Spor, 2005). Chall (1983) first observed this phenomenon decades ago, describing it as the “fourth-grade slump.” There are several explanations within the literature to account for student reading difficulties beginning at the upper elementary grade levels. Researchers often describe (a) a mismatch between reading materials and reading proficiency (e.g., Allington, 2002; Berkeley et al., 2016; Harmon, Hedrick, & Fox, 2000) and (b) the reliance on textbooks as instructional tools, without providing students support in learning content through textbooks (e.g., Benavot, 20 2011; Hall, 2004; Ness, 2007; Zhao & Hoge, 2005). Both of these will be described at length below. Reading Material and Proficiency Mismatch One explanation for content-area learning difficulties beginning at the upper elementary grade levels is the common mismatch between reading demands and student reading proficiency levels. Reading demands for social studies learning relate to both general characteristics of textbooks (e.g., organization, structure) as well as the actual readability level of the text (i.e., grade level proficiency needed). General textbook features. As students advance through schooling, texts become much more complex. As noted above, students in upper elementary school are no longer predominantly reading narrative, fiction texts; instead, focus turns to reading and comprehending content-area expository texts (Allington, 2002; Bennett, 2012). Experts note that several limitations inherent to social studies textbooks may “thwart comprehension,” and thus make student learning more difficult (Berkeley et al., 2016). These challenging textbook attributes include poor organization and complex vocabulary, the nature of which will be briefly described through a review of related empirical studies below. Researchers have systematically analyzed textbook organization for the past several decades, finding poor organization at the macro- (global coherence) and micro- (local coherence) levels (Berkeley et al., 2016; Jitendra et al., 2001). This includes unclear content organization within and across chapters of a textbook (i.e., macro level), as well as unclear language and organization within paragraphs (i.e., micro level) (Berkeley et al., 2014). Poor organization of textbooks was coined the “inconsiderateness of text” by Armbruster in 1984. One key dimension 21 of organization is text structure, or “how the ideas in a text are interrelated to convey a message to a reader” (Meyer & Rice, 1984, p. 319). Relative to the structure of narrative texts, the text structure within and across informational social studies texts tends to include considerable variation. Narrative text structures consistently contain story grammars, which include the basic parts of the setting, theme, characters, plot, and resolution (Thorndyke, 1977). In contrast, informational text structures such as description, sequence, compare-contrast, and problem-solution are not used consistently or explicitly. This makes it more difficult for students to comprehend textbook information as it is presented in such a wide variety of ways (Allington, 2002; Berkeley et al., 2014; Dymock & Nicholson, 2010; Harniss et al., 2001). Moreover, recent studies by Berkeley and colleagues (2014, 2016) found not only that multiple text structures occur across social studies textbooks, but that these text structures are unclear. Specifically, paragraphs that could not clearly be identified as a single text structure (e.g., descriptive, temporal, compare/contrast) were found throughout 22% to 36% of social studies chapters. The authors explained that these ambiguous text structures are likely to be even more problematic than multiple text structures for students, as they can mislead students working to comprehend social studies information (Berkeley et al., 2014, 2016). Further, researchers highlight how social studies textbooks often list events with little explanation as to how events are connected to one another; as a result, students often see the textbook as just a listing of facts (Schleppegrel, Achugar, & Oteíza, 2004). Vocabulary is especially important, as researchers have found terminology demands within social studies texts increase rapidly as students reach the upper elementary and middle school grades (e.g., Harmon, Hedrick, & Fox, 2000). Vocabulary encountered in fifth grade texts and beyond includes more specialized, technical terminology. Moreover, these terms are content- 22 specific and tend to relate to denser social studies concepts (Allington, 2002; Bennett, 2012). Students must therefore learn not only what these new and challenging vocabulary terms mean, but also how they represent a “network of ideas that are important for conceptual learning” (Dymock and Nicolson, 2010; Harmon, Hedrick, & Fox, 2000). Additionally, specialized vocabulary and concepts in social studies text are often “rooted in American culture” (Jimenez- Silva, Hinde, & Hernandez, 2013, p. 277), presenting unique challenges to some students. For instance, students of immigrant families may not have background knowledge of U.S. history and culture that can facilitate understanding of specialized social studies vocabulary and concepts given their limited experience with U.S. schools (Cruz & Thornton, 2012; Haynes, 2005). Harmon and colleagues (2000) found fourth- to sixth- grade social studies textbooks often provide practice activities with definitions of vocabulary terms to help students learn the content (e.g., matching words to their definitions). However, these activities rarely moved beyond the simple association level of filling in the blank or matching terms to definitions. Although a few textbooks included higher-level word processing tasks with application of vocabulary terms (e.g., writing sentences using key words), these textbooks provided little support for how teachers might facilitate these activities during instruction. Bennett and colleagues (2012) found that half of the elementary school social studies textbooks in their study provided zero suggestions to teachers regarding vocabulary instruction. The other half of the textbooks provided limited, brief descriptions of a few strategies that can help students understand social studies vocabulary, such as teaching the concept before introducing a new word, having a word wall, and using context clues (Bennett et al., 2012). Researchers have additionally found that exposure to vocabulary terms throughout social studies textbooks can be 23 infrequent. Harmon, Hedrick, & Fox (2000) found keywords in grade four through six textbooks were most frequently mentioned just once or twice. The limited vocabulary instruction and practice opportunities within textbooks do not seem to help students understand the concepts represented by new and complex terms (Harmon, Hedrick, & Fox, 2000). Textbook readability. As a result of the text complexities described above, researchers have consistently found the readability level of textbooks to be quite high. Readability of a text is calculated using formulas intended to gauge the difficulty of a text (e.g., Flesch-Kinkaid Grade Level Score); thus, higher readability scores indicate text that is more difficult to read. Textbooks have traditionally been written at levels far above the grade level for which they are intended to be used (Allington, 2002; Kinder, Bursuck, & Epstein 1992). For example, Kinder and colleagues (1992) found the average readability of eight-grade social studies textbooks to be the eleventh-grade level, ranging from the ninth-grade level to beyond high school. Similarly, Jitendra et al. (2001) discovered that social studies textbooks were written two to four years above the grade level for which they were created. Most recently, Berkeley and colleagues (2014, 2016) found middle school social studies textbooks consistently written at the ninth-grade level, higher than the grade level for which the texts were intended. This gap between textbook grade level and readability levels was found to be at least one year across all of the textbooks. As a whole, higher readability levels hinder students’ ability (especially those who struggle with reading) to comprehend and learn from textbooks (Berkeley et al., 2016). Instructional Practices: Textbook Use Despite the challenges presented to students as a result of general text features and high readability levels, teachers extensively use textbooks as primary instructional resources (Benavot, 2011; Hall, 2004). This textbook-centered approach may serve as an additional 24 explanation for difficulty in content area achievement beyond the fourth-grade level. Prior research has found that teachers spend 70% or more of content area instruction time presenting the contents of textbooks (Wade, 1993). On a similar note, Zhao and Hoge (2005) found through interviews with 50 elementary school teachers that they universally use social studies textbooks as the primary instructional resource; further, these teachers did not mention using any other curriculum resources. Although textbook use during instruction is to be expected given informational literacy demands within the CCSS, research suggests that students might often be left to understand this text with little support. Ness (2007) observed that social studies and science teachers spent just 10% of their instructional time helping students understand frequently used textbooks. This rate is similar to that observed in Durkin’s (1978) milestone work a few decades ago, where just 12% of instructional time was spent supporting student comprehension of social studies text. Frequent use of social studies textbooks during instruction can also be problematic for students given these texts seem to be “bland” and removed from student experiences (Dunn, 2000). In addition, elementary textbooks often include content and instructional activities that do not clearly connect information and ideas, which can make learning more difficult for students. Experts explain that these texts frequently present disconnected facts and lack depth in their discussion of key ideas, in turn under developing various social studies topics (Brophy & Alleman, 2009; Dunn, 2000). Manuals accompanying elementary social studies textbooks rarely guide the development of ideas to foster student understanding and appreciation of topics, which are important facilitators of student learning (Brophy & Alleman, 2009). These concerns beg the question of whether textbooks alone are sufficient in facilitating student interest in and learning of social studies content. 25 Additionally, schools tend to rely upon a single resource for social studies textbooks. Experts note that students, regardless of their independent reading proficiency levels, are often assigned the same texts (Allington, 2002). This practice is worrisome, as one classroom is likely to have variation in students’ independent reading proficiency, or the text level at which we would reasonably expect them to be able to “read to learn” and demonstrate high achievement. Moreover, research suggests that teachers often assume that students’ reading skills will transfer from narrative to expository content area reading (e.g., Zipperer et al., 2002). This can result in neglecting the need to teach students how to read textbooks, despite their inherent text challenges (Berkeley et al., 2016). Therefore, it stands to reason that even a student who demonstrates proficient reading at the third-grade level per narrative texts may struggle to read and comprehend social studies textbooks intended for third graders. Addressing textbook use. Given the above, what instructional considerations might be made by social studies teachers at the upper elementary level and beyond? As content area reading expert Dunn (2000) eloquently stated, a textbook “can be the program, or it can support it.” Dunn (2000) continued to provide research-based recommendations related to this notion, such as using textbooks as a framework for key ideas and lesson sequences, as a source for learning activities, and for background readings/reference materials. Experts in content area instruction report that based on their observations, exemplary teachers limit over-reliance on textbooks by viewing them as a component of social studies curriculum materials, but not the “backbone.” These teachers tend to draw instruction from multiple sources, supplementing textbooks with resources like web-based information, primary source materials, and local historians and field trips to supplement textbooks (Allington, 2002; Brophy & Alleman, 2009; Dunn, 2000). 26 Another recommendation for teachers to help compensate for the textbook deficiencies described at length above is increased provision of instructional support in reading. Supplemental instruction can help students access and understand social studies text at the advanced elementary level and beyond, and has been highlighted as key to student learning for several decades (Allington, 2002; Berkeley et al., 2016; Harniss et al., 2001). Instructional strategies specific to reading that might support students during content area classes are conceptualized in various ways, and will be described at length in the next section. Instructional Support in Reading The demands of content area instruction and the mismatch between reading materials and student proficiency outlined above begs the question of how student reading needs might best be met to facilitate learning in content areas, such as social studies. One recommendation content area experts provide teachers is increased provision of instructional support in reading (e.g., Dunn, 2000). For several decades, literacy-focused practices that help students access and understand social studies text materials have been highlighted as key to student learning (Allington, 2002; Berkeley et al., 2016; Harniss et al., 2001). This broad literature describes the provision of reading instruction to students in the general education classroom who struggle with decoding, fluency, vocabulary, and/or reading comprehension of informational text. These students may be identified as having a disability, but are more often described as struggling readers who fall below grade level, and in turn struggle to meet the demands of the general education curriculum. Teachers can help students make sense of texts by showing them how to use reading strategies during instruction; through repeated and supported practice with these strategies, students learn to independently understand text information (Vacca, Vacca, & Mraz, 2016; van 27 de Pol et al., 2010). These instructional strategies will be described below, as derived from: (a) comprehensive reports and guidance documents with research-based recommendations (e.g., Institute of Education Sciences, 2008; National Research Council, 2005; Texas Reading Initiative, 2002); (b) books outlining research-based content area reading instruction (e.g., Mraz, Rickelman, & Vacca, 2009; Vacca, Vacca, & Mraz, 2016); and (c) empirical studies and articles by experts in content area reading instruction (e.g., Moss, 2005). The two main recommendations across these products is to embed reading comprehension strategy instruction in the content areas and provide vocabulary instruction for content area text. Reading Comprehension Strategy Instruction Reading comprehension strategy instruction includes processes that are used to enhance comprehension before, during, and after reading new information (Swanson et al., 2016). Teachers can facilitate student reading comprehension by providing explicit instruction and modeling use of reading comprehension strategies, as well as guiding the use of these reading comprehension strategies. Explicit instruction in comprehension strategies such as close reading, think-alouds, reciprocal teaching, question-answer relationships (QAR), activating prior knowledge/interest (e.g., making predictions), identifying informational text structure (e.g., description, compare/contrast, problem/solution, cause/effect, sequence) and questioning the author have specifically been found to have positive effects on student reading comprehension and retention of information (Biancarosa & Snow, 2004; Fisher & Frey, 2014b; Goldman & Rakstraw, 2000; Ivey & Johnston, 2013; Kamil et al., 2008; National Reading Panel, 2000; Ness, 2009; Pearson & Duke, 2002; RAND Reading Study Group, 2002; Snow, 2001). By subsequently modeling these strategies and guiding their use, teachers can show students how to read, think about, and learn from content area text (Vacca, Vacca, & Mraz, 2016). When teachers 28 provide comprehension strategy instruction, students learn how to interact with texts and derive meaning. Research-based strategies for content area texts include the following: KWL (know, what to know, learned); discussion webs; guided reading procedure (GRP); intra-act; the IEPC model (imagine, elaborate, predict confirm), and directed reading-thinking activity (DR-TA). Throughout the content area literature, considerable attention has been paid to the reading comprehension strategy of reading guides, or graphic organizers (e.g., Dexter & Hughes, 2011; Gajria et al., 2007; Kim et al., 2004; Vacca, Vacca, & Mraz, 2016). These tools consist of questions, outlines, and/or activities related to instructional text that students fill out as they read. To this end, reading guides and graphic organizers can help students think about and derive meaning from text that is being read. Meta-analyses by Dexter and Hughes (2011), Gajria et al. (2007) and Kim et al. (2004) have found that graphic organizers demonstrate consistently large and positive effects on vocabulary knowledge and overall comprehension of informational text for upper elementary students with learning disabilities. A recent study by Swanson and colleagues (2016) thoroughly investigated the extent to which middle and high school teachers provide comprehension strategy instruction during social studies. The authors developed a multidimensional observation tool with items adapted from the English Language Learner Classroom Observation Instrument and the Classroom Observation Checklist to record and code teachers’ practices during a social studies class period (which served as a model for the present study). Specific reading comprehension strategies included previewing text, building or accessing background knowledge, monitoring comprehension (i.e., teacher asks questions after information presented to monitor understanding), modeling or think- alouds of comprehension strategy (e.g., graphic organizers, main idea identification, summarizing, generating questions), and discussion activities (e.g., facilitating extended, 29 meaningful discourse). Overall, comprehension strategy instruction was observed in just over half of the social studies classes (55%) and a variety of strategies were used. The background knowledge strategy was observed quite frequently, in 43% of classes; for nearly 30% of these classes, the background knowledge strategy comprised at least one quarter of the class period. The comprehension monitoring strategy was observed in 37% of classes. However, the previewing text, discussion, and modeling or think-aloud comprehension strategies were observed quite rarely, in just 16%, 18%, and 20% of classes, respectively. Teachers in the Swanson et al. (2016) study most often taught and asked students to summarize previously read sections of text. It is important to note that school leaders were asked to identify expert social studies teachers, as indicated by their command of social studies content knowledge and experience in the classroom. Therefore, it appears that even among esteemed social studies instructors there exists a need for greater understanding of instructional strategies that might promote content area reading comprehension (Swanson et al., 2016). Vocabulary Instruction Vocabulary within the content areas is often technical and unfamiliar to students, which makes the teaching and learning of these words especially important for students to understand text. The terms in content materials are often abstract and must be learned through definition, application, and repeated exposure (Vacca, Vacca, & Mraz, 2016). To teach vocabulary well, researchers recommend that teachers help students by providing opportunities to define, clarify, and extend knowledge of vocabulary words and concepts (Fisher & Frey 2014a; Harmon, Hedrick, & Wood, 2005, Kamil et al., 2008). There are numerous instructional strategies that can accomplish this task within the content areas per experts and empirical research. The first, activating student prior knowledge of vocabulary words, helps student reflect on what they might 30 already know about certain words within the text. For example, a teacher may facilitate brainstorming during which the teacher picks a key concept/term from within the text, and students generate a list of words related to that concept in a particular amount of time. Teachers might also provide word sorts to activate prior knowledge, during which students work to sort key words from the text into different categories/groups by looking for shared features among their meanings. Researchers have found that when teachers present students with the opportunity to activate prior knowledge of vocabulary, students demonstrate enhanced comprehension of the text relative to no prior knowledge activation (e.g., Hansen 1981; Hansen & Pearson, 1983). Teachers can also highlight for students how they might figure out the meaning of unknown words within text by using its context and examining the surrounding concepts and terms. For example, teachers might show struggling readers how to infer the meaning of a new word using typographic clues (e.g., bold, italics with footnoted definitions) and syntactic/semantic clues (e.g., contrast words, direct examples). Another vocabulary instruction strategy is the use of graphic organizers to create visuals of vocabulary and concept relationships. Although graphic organizers vary in format, one commonly used format is a “network tree” diagram that depicts hierarchical relationships among words. Vocabulary-focused graphic organizers in social studies may present terms in the form of a timeline, showing the chronological or linear relationship among these terms (Parker & Jarolimek, 1997). Lastly, teachers can facilitate student understanding of vocabulary by providing explicit instruction in independent vocabulary- building strategies, including how to strategically use a dictionary and how to analyze word structure and morphological units (Fisher & Frey, 2014a, 2014b; Harmon, Hedrick, & Wood, 2005; Ivey & Johnston, 2013: Kamil et al., 2008; Vacca, Vacca, & Mraz, 2016). 31 The Swanson and colleagues study (2016) described in the prior section also investigated the extent to which middle and high school social studies teachers provide vocabulary support during instruction. Vocabulary instructional strategies included definition provision, morphology use, and context clues. Overall, vocabulary instructional strategies were observed to varying degrees in social studies classrooms. Definition provision was observed quite frequently, in 52% of classes; for nearly 40% of these classes, definition provision comprised at least one quarter of the class period. The context clue and morphology use strategies were observed rarely, in just 11%, and 4% of classes, respectively. The authors concluded that although teachers frequently presented definitions of new words to students, it seems like social studies teachers might be missing key opportunities to help students acquire vocabulary and comprehend subsequent content area text (Swanson et al., 2016). Instructional Support in Reading: State of the Literature Over the past few decades, research has mostly observed and described the classroom practices of teachers to better understand the process and appearance of comprehension and vocabulary instruction (e.g., Lutz, Guthrie, & Davis, 2006). The goal of most of these descriptive studies has been to explore and describe how reading strategy instruction is provided, largely as it relates to literacy and supporting struggling readers. Although this body of descriptive research is rich in information on what reading support during content area instruction may look like, the effectiveness of this supplemental literacy instruction remains less known, particularly during social studies. As van de Pol and colleagues (2010) noted in their review, future research should consider both teacher behaviors (i.e., instructional practices) and student behaviors (e.g., learning, engagement); “the coding of only teacher actions or strategies is not sufficient” (van de Pol, Volman, & Beishuizen, 2010). In addition, the limited research attention toward 32 comprehension instruction in content area classrooms has focused primarily on the secondary level and beyond. This stands in contrast to the need to better understand the influence of comprehension instruction at the upper elementary level, given research suggesting increased used of content area textbooks around fourth grade. Lastly, empirical attention that has been given to reading supports often comes in the form of several strategies combined (e.g., reading strategies along with performance feedback strategies), or a packaged intervention program. Although the impacts of full intervention programs certainly merit study, it remains unclear to what extent reading strategies in particular may help students access instruction during a single class period. The goal of the present study is to look at this more specifically by investigating the relationship between comprehension instruction and student engagement, a necessary precursor to learning (to be described in the subsequent section). Comprehension Instruction Components vs. Comprehension Strategies Two concepts that are important to distinguish for the purposes of this study are comprehension instruction components and comprehension strategies. Both of these terms have been discussed above and will be used throughout the remainder of this document; their distinct meanings should be kept in mind. Comprehension instruction components describe those components within Duke and Pearson’s (2009) research-supported model that are specific to reading comprehension instruction. These include the instructional practices of direct instruction, modeling, collaborative use/guided practice, and independent strategy use. This study measured comprehension instruction components during every 30-second interval of the observation period (further described in the Method section below). Comprehension strategies include those specific reading comprehension strategies and vocabulary strategies that are taught through the components previously described. These include strategies like previewing text/prediction, 33 definitions, prior knowledge, main idea, text structure, and graphic organizers. This study measured comprehension strategies using a running checklist format, marking a strategy if it was observed at any point during instruction (further described in the Method section below). Engagement An important consideration in conjunction with the nature of content area text materials and instructional support in reading is student engagement. As noted prior, researchers view student engagement as theoretically and practically critical to learning, achievement and academic success outcomes (e.g., Skinner & Pitzer, 2012). The ways in which classrooms promote or undermine student engagement is an ever-important empirical question given (a) its relationship with a wealth of positive student outcomes (e.g., Dolezal et al., 2003; Hill & Werner, 2006; Sinclair et al., 2003) and (b) the relationship between student engagement and classroom factors (e.g., instructional practices, materials). This section of the literature review will provide an explanation of how student engagement has been defined and summarize empirical findings on the relationship between the educational context and student engagement. Defining Engagement Although researchers have long been interested in engagement within the school context, terms for this construct as well as the associated definitions have often varied. For example, school engagement, student engagement, academic engagement, student engagement in academic work, student engagement in/with school, and engagement in schoolwork have been operationalized differently (Appleton, Christenson, & Furlong, 2008; Christenson, Reschly, & Wylie, 2012; Skinner, Furrer, Marchand, & Kindermann, 2008). Moreover, definitions often vary across theoretical and practical approaches as to whether the opposite of engagement should be included. Even researchers who agree on including the opposite of engagement use differing 34 terms such as disaffection, disengagement, or alienation – further illustrating complexities and discrepancies within the engagement literature (Skinner & Pitzer, 2012). However, scholars are in consensus that student engagement is multidimensional. Recent literature reviews and scholarly pieces describe engagement as comprised of several components: behavioral engagement (e.g., participation in learning activities, effort); cognitive engagement (e.g., investment in learning, metacognition); and emotional engagement (e.g., enthusiasm about learning, interest) (Corso et al., 2013; Fredericks, Blumenfeld, & Paris, 2004; Jimerson, Campos, & Greif, 2003; Parsons et al., 2015; Skinner & Pitzer, 2012). Researchers universally define engagement to include the behavioral component, and many include the emotional component; far fewer definitions include the cognitive component (Appleton, Christenson, & Furlong, 2008). Though conceptualizations of engagement may differ in their number and types of components, one shared theme is that these components reflect student interactions with classroom activities/materials that may promote (or interfere with) learning (Appleton, Christenson, & Furlong, 2008; Skinner, Kindermann, & Furrer, 2009). Many researchers have considered engagement to be an external representation of a motivated student (e.g., Deci & Ryan, 1985, 2000; Skinner, Furrer, Marchand, & Kindermann, 2008). However, even students emotionally attached to school (motivated) are unlikely to experience academic success unless they are engaged with classroom learning opportunities (Skinner, Kindermann, & Furrer, 2009). Thus, high-quality learning is the result of behaviors and emotions that reflect engagement in academic activities in the classroom (with underlying motivation) (Skinner, Furrer, Marchand, & Kindermann, 2008). 35 Engagement in the Present Study For the purposes of the present study, engagement was conceptualized as part of this larger model of motivation that has emerged over the past few decades. Though motivation is viewed as an important underpinning of engagement, “the latter [engagement] is a construct worthy of study in its own right…engagement can change through cyclic interactions with contextual variables and influence later academic, behavioral, and social outcomes, which are the products of these context-influenced changes in engagement” (Appleton, Christenson, & Furlong, 2008, p. 379). Thus, the present study considered engagement within its theoretical framework of motivation, but focused on investigating the relationship between engagement and classroom context variables. Engagement is operationalized within this motivational framework as having two key features: (a) it includes behavioral and emotional participation in the classroom (described above), and (b) it requires a conceptualization of its opposite, termed disaffection (Connell & Wellborn, 1991; Skinner, Kindermann, & Furrer, 2009; Skinner & Pitzer, 2012; Willms, 2003). Skinner and colleagues (2008, 2009) explained that disaffected behaviors include passivity, lack of effort, giving up, and mental withdrawal (e.g., lack of attention, going through the motions). They continued to describe disaffected emotions as being tired, sad, bored, frustrated, angry, and anxious. This conceptualization of engagement and disaffection along both behavioral and emotional dimensions (i.e., four components) has been validated empirically through structural analysis; including all four distinct components appears to be the best proxy for student engagement among elementary, middle, and high school students (e.g., Furrer et al., 2006; Skinner, Furrer, Marchand, & Kindermann, 2008; Skinner, Kindermann, & Furrer, 2009). 36 Engagement and the Classroom Context One characteristic of the classroom context related to student engagement is teacher support. Teacher support emphasizes students’ perceptions of teacher involvement (e.g., dependability, assuring success) and classroom structure (e.g., collaborative tasks) (Skinner, Kindermann, Connell, & Wellborn, 2009). Furrer and Skinner (2003) found that greater teacher support corresponded with increases in student engagement over the course of the school year. Specifically, third through sixth-grade students reported more effort, attention, and participation (i.e., engagement) in classroom learning activities when teacher support was present. A similar study found that fourth through seventh-grade students’ behavioral disaffection decreased across the course of the school year as a result of greater teacher support (Skinner, Furrer, Marchand, & Kindermann, 2008). This decrease in disaffection included a reduction in students’ lack of effort and/or withdrawal from classroom learning activities. Another conceptualization of teacher support, providing differentiated instruction based on student need and ability, appears to promote the amount of time students are engaged in the classroom (Tomlinson, 2003). Although teacher support across these three studies was defined quite broadly, it sets the stage and provides reason to investigate specific classroom factors/practices that may foster engagement. Lutz, Guthrie and Davis (2006) investigated the more specific teacher support of comprehension instruction. These researchers explored the relationship between an integrated reading-science instructional program and student engagement in fourth grade classrooms. An in-depth examination of individual differences among students revealed that engagement for both high and low achieving readers increased and sustained when teachers provided the instructional program, which included direct instruction in comprehension strategies and collaborative support (i.e., encouraging students to assist peers). Moreover, the pattern of 37 comprehension strategies teachers used appeared to play a role in facilitating student engagement and, in turn, achievement gains. The authors reported that teachers of students demonstrating greater gains in reading comprehension had greater student engagement levels, as well as a greater number and variety of comprehension strategies incorporated during the observed lessons. Similarly, researchers Assor, Kaplan, and Roth (2002) found that third through eighth grade teachers who fostered understanding of and interest in text had higher levels of cognitive and behavioral engagement among students (R2 of .19 to .51). As Lutz and colleagues (2006) stated, results like these “support the theoretical links among engagement, teacher practices, and task characteristics.” While these studies provide important preliminary findings regarding the relationship between the classroom context and student engagement, they are limited in several ways. First, these studies focus on reading instruction. Although Lutz and colleagues (2006) narrowed in on science text, there is a need to explore the relationship between comprehension instruction and engagement (a) outside of reading class (i.e., during content area instruction), and (b) with other content areas texts, such as social studies and math texts. Second, Lutz et al. (2006) measured engagement through observer ratings, making the interpretation of affective indicators of engagement more difficult. The extent to which the observational data correspond with student perceptions of their own engagement remains unclear, as noted by the authors. Due to practical limitations of observation, Lutz and colleagues (2006) were only able to include three classrooms and twelve total students in their study, potentially limiting the generalizability of student engagement findings. Lastly, both of these studies examined and measured reading comprehension instruction in a “package” – i.e., reading-related strategies aggregated with a variety of other strategies throughout analyses. These included practices like teacher feedback, 38 hands-on events, knowledge goals, and up to nearly a dozen other strategies. Thus, it remains unclear to what extent comprehension instructional practices in particular might be important in facilitating student engagement. Student Engagement: State of the Literature As Fredericks and colleagues (2004) noted in their review of the engagement literature, “few scholars include measures of context, needs, and engagement in the same study” (p.80). Thus, despite a wealth of research (a) providing theoretical connections among these constructs (e.g., ecological systems theory, student engagement core) and (b) finding strong correlations between engagement and student success outcomes, there remains a need to better understand the dynamic relationship among student engagement and its surrounding classroom context (i.e., teachers and content). As noted prior, extensive reviews of the literature reveal that most studies to date have addressed how students engage with school in general (e.g., attendance, number of activities), as opposed to engagement at the classroom level (Corso et al., 2013; Fredericks et al., 2011). Wylie and Hodgen (2012) noted that engagement depicts the interaction between effortful learning, the teacher, and classroom learning opportunities – thus, direct contextual factors surrounding student engagement are important to investigate more extensively. In addition, the majority of research has focused on observable indicators of behavioral engagement (e.g., being on task, attending class), failing to account for the role of emotional engagement within the classroom setting (Appleton et al., 2006). Although growing literature suggests emotional and cognitive engagement indicators are associated with positive learning outcomes and motivation (e.g., Fredericks, Blumenfeld, & Paris, 2004), few studies consider this 39 relationship in conjunction with the classroom context to see what might facilitate such engagement. The author was unable to identify empirical studies that consider how the accessibility of instructional materials (e.g., text) might relate to student engagement levels in the classroom. Scholars do acknowledge that complex text above students’ reading levels may create frustration or limit comprehension (e.g., Dougherty Stahl., 2012). They further explain that teacher- provided reading supports can make these texts accessible to struggling readers, facilitating student engagement instead of frustration (Dougherty Stahl, 2012; O’Connor et al., 2002). However, the relationship between curriculum text materials and student engagement remains to be empirically investigated directly. Lastly, it is important to note that older students seem to frequently be the focus of studies including the emotional engagement dimension, as well as studies on engagement within the content areas. For instance, although the recent study by Swanson and colleagues (2016) considered many aspects of social studies instruction (including comprehension instruction), it focused on middle/high school classrooms and only reported an average student engagement level across all classrooms on a 4-point Likert scale. There is certainly a need to consider the dynamic relationship between classroom factors and student engagement (behavioral and emotional) among younger students within the content areas more closely. This is especially important given an increasing emphasis on informational texts beginning in the upper elementary grades (e.g., CCSS). The Present Study Although experts and empirical studies have emphasized the challenging nature of textbooks and the frequent use of textbooks during social studies instruction (e.g., Berkeley et 40 al., 2016), efforts like the CCSS have pushed schools toward a greater focus on content area literacy. With the need to incorporate instructional texts at the upper elementary level, questions remain about the extent to which social studies teachers are appropriately supporting student understanding of textbook information during instruction (e.g., Ness, 2009). Literacy-focused support is especially important for struggling readers, who may have difficulty decoding text, reading fluently, understanding vocabulary, and/or comprehending content area text (Vellutino, 2003). Research over the past few decades on comprehension instruction in content area classrooms has been largely descriptive in nature, resulting in little evidence to support or refute their effectiveness in promoting student engagement and learning (e.g., Lutz, Guthrie, & Davis, 2006; Ness, 2007). Although this body of literature is rich in descriptive information, researchers note that there remains a need to consider instructional practices along with student engagement and learning more systematically (e.g., van de Pol, Volman, & Beishuizen, 2010). Further, the impact of comprehension instruction in particular remains unclear. Researchers often investigate reading strategies in content area classrooms within a packaged intervention program, or according to teacher-reported instructional practices. By directly observing instructional practices (as opposed to using teacher report of instructional practices), this study provided more objective data collection of comprehension strategies present. Researchers tend to measure student engagement without concurrent consideration for the environmental context (e.g., Swanson et al., 2016). Despite a wealth of research connecting things like comprehension instruction and student outcomes theoretically or at the more “macro” level (e.g., engagement with school in general and school completion), the literature would benefit from better understanding of student engagement within the dynamic classroom 41 environment (Fredericks et al., 2004). In addition, most studies have focused on observable behaviors reflecting engagement, failing to account for the role emotional engagement plays within the classroom setting (Appleton et al., 2006). The present study aimed to provide a more holistic picture of student engagement at the classroom level by including the subjective emotional engagement dimensions, while focusing on an individual class period (i.e., learning opportunity). These gaps in the literature were filled by the present study. Specifically, quantitative methods were used to investigate the readability level of text materials used during social studies instruction, the extent to which teachers incorporated reading comprehension instruction to facilitate student access of those texts, and the relationship between those two classroom factors and student engagement. These relationships were further examined among struggling readers in particular, as these students might benefit most from reading comprehension instruction. Research Questions (RQs) RQ 1. What is the readability level of text materials used during social studies instruction in fifth grade classrooms? RQ 2. How frequently does fifth grade social studies instruction include at least one reading comprehension instructional component (proportion of intervals)? How often is each type of comprehension instructional component being observed (i.e., direct instruction, modeling, collaborative use/guided practice, independent strategy use)? For RQ3, RQ4, and RQ5 Level 1 predictors refer to student-level individual variables, while Level 2 predictors refer to classroom-level variables. Figure 4 at the end of this section depicts the hypothesized relationships between variables that will be examined through these three research questions. 42 RQ 3. To what extent does student reading achievement level (Level 1) predict student engagement during fifth grade social studies instruction among all students? RQ 4. To what extent do classroom-level reading factors (Level 2) predict student engagement during fifth grade social studies instruction? More specifically: 4a. To what extent does text readability level (Level 2) predict student engagement among all students? Among struggling readers? 4a Hypothesis: Higher text levels will predict lower student engagement, particularly among struggling readers. 4b. To what extent does frequency of reading comprehension instruction (Level 2) predict student engagement among all students? Among struggling readers? 4b Hypothesis: Higher frequency of reading comprehension instruction will predict higher student engagement, particularly among struggling readers. RQ 5. To what extent does frequency of reading comprehension instruction (proportion of intervals) moderate the relationship between student- and classroom-level reading factors and student engagement? More specifically: 5a. To what extent does frequency of reading comprehension instruction moderate the relationship between student reading achievement level (Level 1 predictor) and student engagement among all students? Among struggling readers? 5a Hypothesis: As the value of reading comprehension instruction increases (moderator), the positive relationship between student reading achievement and engagement decreases (i.e., becomes weaker). This moderating relationship is expected to be particularly strong for struggling readers. 43 5b. To what extent does frequency of reading comprehension instruction moderate the relationship between text readability grade level (Level 2 predictor) and student engagement among all students? Among struggling readers? 5b Hypothesis: As the value of reading comprehension instruction increases (moderator), the negative relationship between text readability grade level and engagement decreases (i.e., becomes weaker). This moderating relationship is expected to be particularly strong for struggling readers, who have lower levels of reading achievement. Figure 4 Relationships examined through RQ3, RQ4, and RQ5 multilevel models. 44 CHAPTER III: METHOD The present observational study used descriptive and multilevel analyses to address the research questions and variables of interest. More specifically, a structured observation tool was used to systematically collect quantitative data on reading comprehension instruction and text use during social studies lessons. A group-administered student engagement survey and reading comprehension assessment were administered to students immediately following the social studies lesson. Then, a text readability analysis program was used to investigate the readability of social studies text materials used during instruction. Sample Sample Size The study observed a total of 50 fifth-grade teachers during social studies instruction. This number was informed by power analysis (e.g., Soper, 2016) and expert recommendations throughout the literature for multilevel modeling. A recent review by McNeish and Stapleton (2016) noted that historically, multilevel models have been shown to provide biased estimates when the number of Level 2 units sampled is below 30. Upon reviewing more current journal articles on multilevel modeling sample size requirements, McNeish and Stapleton (2016) found researchers recommend a range of 15 to 50 Level 2 units for accurate estimates of continuous outcomes (varying by the effect of focus). Throughout the literature, researchers note that samples for multilevel models considering interaction effects should ideally be no fewer than about 50 Level 2 units, with about 20 Level 1 individuals per group; however, researchers also recognize that practicality and feasibility can make these larger sample sizes difficult to obtain (Hox, Moerbeek, & van de Schoot, 2010). A review by Mathieu and colleagues (2012) including 79 studies published in the Journal of Applied Psychology found the median number of Level 2 45 units sampled across studies was 51. Thus, the sample size of 50 participating teachers (Level 2 unit classrooms) was selected. Inclusion Criteria Schools were identified for inclusion in the study that had the fifth-grade level and were geographically located within Michigan. Once administrative support for participation was secured from schools meeting these criteria, those social studies teachers who provided their consent were included in the present study. Participants The fifty participating fifth-grade teachers (N = 50 Level 2 units) were from 34 school districts and 39 different schools across Michigan. 30 of the schools were elementary schools (grades K-5, K-6 or 1-6), and 9 of the schools were upper elementary or intermediate schools (grades 3-5, 4-5, 5-6, or 4-8). A total of 1,083 students participated across the 50 classrooms; class size ranged from 14 to 31 participating students, with an average of 21 to 22 students per class (N = 1,083 Level 1 units). Table 1 provides an overview of teacher demographic information, as indicated by teacher self-report through the online survey. Table 1 Teacher Demographics Variable Gender Female Male Race/Ethnicity White Non-Hispanic African-American Non-Hispanic American Indian/Alaskan Native Hispanic Highest Level of Education Master’s Degree Bachelors/College Degree Teaching Endorsements Helda Percentage 80% 20% 92% 4% 2% 2% 72% 28% Frequency (N=50) 40 10 46 2 1 1 36 14 46 Table 1 (cont'd) Language Arts Social Studies Mathematics Science Other (Early Childhood, Gifted, Spanish) Special Education No Endorsements Bilingual Education Health, Physical Ed, Recreation and Dance Technology Education Years of Teaching Experience 11 to 15 years 21 to 25 years 6 to 10 years 16 to 20 years 1 to 5 years 26 to 30 years 31 to 40 years Grade Levels Currently Teaching 5th Grade Only 5th, 6th Grade 3rd, 4th, 5th Grade School Location Percentage of Class in Special Ed Suburban Rural Urban 6-10% 0-5% 11-15% 16-20% 21-25% Percentage of School Free/ Reduced Price Lunch 26 -50% 51-75% 76-100% 0-25% 26 18 15 13 6 4 3 3 2 1 12 12 11 7 4 2 2 48 1 1 22 20 8 19 17 8 4 2 22 13 9 6 52% 36% 30% 26% 12% 8% 6% 6% 4% 2% 24% 24% 22% 14% 8% 4% 4% 96% 2% 2% 44% 40% 16% 38% 34% 16% 8% 4% 44% 26% 18% 12% aTeachers selected all that applied for teaching endorsements (so percent does not sum to 100). 47 Variables Variables in the present study included the following: (a) student engagement, (b) reading achievement, (c) text readability, and (d) reading comprehension instruction. Each of these variables will be described at length below. (a) Student Engagement The Engagement Versus Disaffection with Learning Scale [EvsD] is a student self-report measure of the behavioral and emotional dimensions of student engagement (Skinner, Furrer, Marchand, & Kindermann, 2008; Skinner, Kindermann & Furrer, 2009). It contains 20 items with four subscales: behavioral engagement (5 items), behavioral disaffection (5 items), emotional engagement (5 items), and emotional disaffection (5 items). A brief description of each subscale, along with the specific items, are in Table 2 below. Items are to be read aloud by a trained individual (typically in the classroom), as students follow along on a paper and pencil questionnaire. Students rate each item on a 4-point Likert scale ranging from 1 (not at all true) to 4 (very true) (Skinner, Kindermann, & Furrer, 2009). Table 2 EvsD Student Self-Report Survey Items and Subscales Subscale Behavioral Engagement Description Effort, attention, and persistence in initiating and participating in learning activities Behavioral Disaffection Lack of effort and withdrawal from learning activities 48 Items 1. I try hard to do well in class. 2. In class, I work as hard as I can. 3. When I’m in class, I participate in class discussions. 4. I pay attention in class. 5. When I’m in class, I listen very carefully. 1. When I’m in class, I just act like I’m working. 2. I don’t try very hard in class. 3. In class, I do just enough to get by. 4. When I’m in class, I think about other things. 5. When I’m in class, my mind wanders. Table 2 (cont’d) Emotional Engagement Emotions indicating motivated involvement during learning activities Emotional Disaffection Emotions indicating withdrawal during learning 1. When I’m in class, I feel good. 2. When we work on something in class, I feel interested. 3. Class is fun. 4. I enjoy learning new things in class. 5. When we work on something in class, I get involved 1. When we work on something in class, I feel bored. 2. When I’m in class, I feel worried. 3. When we work on something in class, I feel discouraged. 4. Class is not all that fun for me. 5. When I’m in class, I feel bad. The EvsD was selected based on an Institute of Education Sciences (IES) report reviewing 21 instruments that measure student engagement at the elementary through high school levels (National Center for Education Evaluation and Regional Assistance, 2011). The EvsD was identified as best meeting the needs of the present study because it: (a) defined engagement multidimensionally; (b) was developed for use with upper elementary school populations, (c) had technical information on psychometric properties (i.e., reliability and validity); and (d) was constructed to reflect engagement at the class level (e.g., When I’m in class, I listen very carefully), as opposed to more broad engagement in school (e.g., I work hard in school). Developers of the EvsD report evidence for reliability and validity through several methods. Internal consistency (Cronbach’s alpha) for a sample of students in grades 3-6 ranged from 0.61 to 0.85 for the four subscales. Combining the behavioral and emotional engagement subscales into two composite scores produced higher levels of internal consistency, 0.79 and 0.86. Inter-individual stability was reported as cross-year correlations ranging from 0.53 to 0.68 49 for the four subscales (Skinner, Kindermann, Connell, & Wellborn, 2009). Construct validity for the EvsD was established through confirmatory factor analysis, distinguishing four factors (subscales) as the best fit for student self-report data. In addition, the behavioral and emotional subscales correlated positively, while the engagement and disaffection subscales correlated negatively. Correlations between student and a teacher-report measure were modest (stronger correlation for behavioral engagement than emotional) (Skinner, Kindermann, Connell, & Wellborn, 2009). Expected age-grade patterns were also identified in which middle school students showed lower levels of engagement than did children in upper elementary school (Skinner, Furrer, Marchand, & Kindermann, 2008). For the purposes of the present study, verb tense was modified from present to past tense for each item of the EvsD (see Appendix A). For example, item one stating, “I try hard to do well in class” was reworded to “I tried hard to do well in class.” This modification was made to help remind students that they should be thinking about the social studies lesson that had just occurred, as opposed to the social studies class in general. Moreover, students were reminded prior to administration to focus only on the social studies class session that they just completed. Separate subscale scores can be calculated on the EvsD as the average of the item ratings for each subscale. Furthermore, psychometric analyses indicate that all item ratings can be combined into a composite score with technical adequacy (by reverse-coding the negatively worded disaffection items and including them with the engagement items); this composite is called the total engagement score (Skinner, Kindermann, & Furrer, 2009). The present study therefore calculated each student’s engagement score by summing ratings for all 20 engagement items and disaffection items (reversed). The value for this student engagement variable ranges 50 from a minimum of 20 to a maximum of 80, with higher scores representing higher levels of behavioral and emotional engagement during the observed class period. An internal consistency analysis (Cronbach’s alpha) was conducted to examine reliability for each of the four subscales of the EvsD student engagement measure used for this study (i.e., with modified verb tense). Internal consistency ranged from 0.66 to 0.79 for the four subscales, similar to original reliability scores (range of 0.61 to 0.85; Skinner, Kindermann, Connell, & Wellborn, 2009). Likewise, combining the behavioral and emotional engagement subscales into two composite scores produced higher levels of internal consistency at 0.80 and 0.84, respectively (original scores of 0.79 and 0.86; Skinner, Kindermann, Connell, & Wellborn, 2009. Internal consistency for all items combined, reflecting one engagement score, was 0.89. These findings are consistent with prior psychometric analyses indicating that the subscales can be used in combination with technical adequacy, and emotional and behavioral engagement items can be combined to reflect one engagement score (Skinner, Kindermann, & Furrer, 2009). (b) Reading Achievement The reading achievement level of each student was measured through the DIBELS Daze, a standardized group-administered reading assessment (see Appendix B). The DIBELS Daze assesses reading comprehension and fluency. It is a reading comprehension task in that it assesses the student’s ability to construct meaning from text using word recognition skills, comprehension strategies, background information/prior knowledge, reasoning skills, and familiarity with linguistic properties (e.g., syntax, morphology). It is also an indicator of reading fluency as students are given three minutes to complete the reading task; the more fluent of a reader the student is, the more items they will encounter during the assessment period (Good et al., 2013). 51 Students are given three minutes to silently read a passage that has every seventh word (approximately) replaced with a box containing a maze of word options (i.e., three possible word choices for the blank). One of the three maze options is correct, while the two other options are incorrect. Students circle the word that they believe best fits the omitted word in the sentence (Good et al., 2013). In the present study, students were administered the fifth grade DIBELS Daze passage titled John Hancock (number 14). The selection of this passage occurred as follows: first, the author reviewed all DIBELS Daze progress monitoring passages, excluding those used for benchmarking (in an effort to avoid what might be familiar to students). Then, the author selected out all informational text passages for potential use (i.e., excluded narrative passages). From those remaining informational progress monitoring passages, the author randomly selected one at the fifth-grade level to include in the present study. Developers of the DIBELS Daze reported evidence for reliability and validity through several methods. Coefficients indicated that these psychometric properties were consistently at acceptable to high levels at the fifth-grade level. Alternate-form reliability was a correlation of 0.66, and inter-rater reliability correlations were 0.99. Concurrent and predictive validity for the DIBELS Daze were established with the DIBELS oral reading fluency (ORF) measure. The fifth-grade assessment correlated at 0.77 for concurrent validity and 0.78 for predictive validity. Criterion-related validity was likewise established with the Group Reading Assessment and Diagnostic Evaluate (GRADE) total score, National Assessment of Education Progress (NAEP) reading passage score, and DIBELS composite score. The fifth grade DIBELS Daze scores correlated with the aforementioned criterions at 0.66, 0.96, and 0.74, respectively. The DIBELS Daze adjusted score (which compensates for guessing) is calculated as the number of correct responses, minus half the number of incorrect responses. The resulting score is 52 rounded to the nearest whole number, with half-points rounded up. If the adjusted score is negative, a score of zero is recorded. The maximum score a student can receive is the number of items on the form; in this case, 52 for the fifth-grade passage (Good et al., 2013). The DIBELS Daze adjusted score was also used to identify struggling readers for the present study. Students who scored below the fifth-grade benchmark score and cut point for risk (40th percentile, raw score of 20) were considered a struggling reader (e.g., IDEIA, 2004; Lapp et al., 2003). Of the 1,083 total students included in the present study, 556 were identified as struggling readers (51% of total sample of students). (c) Text Readability Although there are more than 30 readability formulas available to estimate text difficulty (Meyer, 2003), the Advantage/TASA Open Standard Readability Formula (ATOS; School Renaissance Inc., 2000) was selected for the present study. The ATOS is associated with the Accelerated Reader program, and is consequently one of the most widely used measures for determining the difficulty level of text; it has been researched extensively among readers of all ages and skill levels (Benjamin, 2012). The formula takes into account three text characteristics that research has determined to be most important in predicting difficulty: sentence length (number of words), word length (number of characters), and word difficulty (grade level) (Benjamin, 2012; Renaissance Learning, 2006). The ATOS text analysis system is accessible for free online, and there is no minimum or maximum text length required for analysis. A technical report by Student Achievement Partners found the ATOS to be a reliable and valid measure of text complexity (Nelson, Perfetti, Liben, & Liben, 2012). The researchers found that ATOS readability scores reliably correlated with grade level and student performance-based measures of text difficulty across a variety of text sets and reference measures. For the subset of 53 informational texts, ATOS score correlations (Pearson’s r) were 0.63 with Common Core Exemplar Text grade levels, and 0.71 with state and national test passages grade levels. In addition, an appendix of the Common Core State Standards Initiative (2010) noted the ATOS as a quantitative measure of text complexity that “puts students and texts on the same scale” (p.7). Text readability grade level was calculated based on all reading materials used during the observed instructional period. Copies of all text were obtained by the research assistant immediately following the observation period, including social studies worksheets, textbook sections, workbooks pages, text presented on projectors, and any other social studies text materials. The author manually and individually entered all text materials into the ATOS online analyzer (http://www.renaissance.com/Products/Accelerated-Reader/ATOS/ATOS-Analyzer-for- Text) to derive an ATOS grade level readability score. In instances where just one text material was used during the observation period, the ATOS readability score was the value for this variable. In instances where more than one text material was used during the observation period, the value for this variable was calculated by entering all the text materials into the ATOS online analyzer at once, resulting in a single ATOS score representative of all reading materials. (d) Reading Comprehension Instruction The provision of reading comprehension instruction was recorded using the reading comprehension component section of the observation tool. Swanson et al.’s (2016) observation tool was applied to the present study, with modifications, as presented in Appendix C. The observation tool in the present study divided the instructional period into 30-second intervals, an interval increment used in similar observational studies (e.g., Lutz, Guthrie, & Davis, 2006). Partial-interval recording, where a behavior is recorded if it is observed at any part 54 of the interval, was used for several reasons (Stegge & Watson, 2009). First, partial-interval recording is best suited for behaviors that may occur at moderate to high rates, be of an inconsistent duration, and/or be very brief (Brown-Chidsey & Andren, 2013; Stegge & Watson, 2009). In the case of the present study, the observation tool variable of reading comprehension instruction has the potential to occur at a high frequency, be brief, and last for an inconsistent amount of time throughout the observation period. In addition, research supports the use of partial-interval recording for behaviors in which the frequency of occurrence is of primary importance, which is the case for this variable (Meany-Daboul, Roscoe, Bourret, & Ahearn, 2007). Lastly, partial-interval recording allows an observer to monitor several behaviors and score them during the same interval, which aligns with goals of the present study (Brown- Chidsey & Andren, 2013). The reading comprehension instruction variable was calculated as the proportion of intervals during which at least one reading comprehension component was present. This was written in decimal form; for example, a value of 0.50 for this variable indicates that 50% of observed intervals had comprehension instruction. Reading comprehension components included in the present study were informed by research-based instructional recommendations and models throughout the content area reading literature (as described in the literature review). The author created this variable to represent the various ways in which teachers can provide instruction that support student comprehension of texts. These instructional components include the following: direct instruction, modeling, collaborative use/guided practice, and independent use. Definitions for instructional components are derived from Duke and Pearson’s (2009) conceptualization and those used in Ness’ (2007) observation study. Specific definitions for each of the comprehension instructional components used by observers in the present study are provided in Table 3 below. 55 Table 3 Reading Comprehension Instruction Components Component Direct Instruction An explicit description of a strategy and when/how it should be used Definition (without demonstration/use of strategy) Modeling/demonstrating the strategy in action (e.g., think-aloud of ones’ thoughts while performing task); often verbalizing behavior as demonstrating skill while students observe and follow along; “I do it” (teacher and/or student modeling) Collaborative use of the strategy in action; both teacher and student participate in using the strategy and “do it together.” Or, taking turns using the strategy in action; interactive as teacher guides/gives direction on students’ strategy use (e.g., prompt, questions, feedback, “coaching”). For example, choral responding if class wide, verbally respond in unison to teacher prompts…teacher provides feedback and engages in discussion Independent use of the strategy; opportunity for students to use the strategy on their own per teacher directions/instructional expectations. For example, student by him/herself, students collaboratively using strategy without teacher providing direct instruction, modeling, etc. None of the components above are utilized to provide comprehension instruction during the observed 30-second interval Modeling Guided Practice/ Collaborative Independent None Recruitment For eligible schools (as described in the sample section above), the author used emails with follow-up phone calls as needed to solicit administrator support for participation in the present study (principal approval at a minimum, district-level approval as needed; see Appendix D, Administrator Recruitment Email). Once administrators provided permission for their schools to participate, principals of those schools helped in identifying fifth grade general education teachers who provide social studies instruction. All of those teachers were then emailed to Procedures 56 participate in the present study (see Appendix E, Teacher Recruitment Email 1 and Appendix F, Teacher Consent Form). Administrators who provided permission to contact their teachers will be provided a post-study report of aggregated and summarized results. Teachers that returned their signed consent forms were then sent an email containing a link to the online demographic survey, as well as a request to schedule an observation session (see Appendix G, Email 2). Once an observation time was identified, teachers were emailed instructions for providing information to all parent(s) of students in their class (see Appendix H, Email 3). Passive parent consent procedures (also known as “opt out”) involved distributing a letter to each student’s parent(s) describing the present study and instructing them to contact the author only if they did not want their child to participate (see Appendix I and J). Parents were informed that participation in the research study was separate from any instruction, and refusal to allow their child to participate would affect neither their child’s participation in standard education activities nor their child’s grade or class standing. Teachers sent a reminder to parents three days after the initial information was sent home (see Appendix K). If it was customary for the school/classroom teacher to communicate through email with parents (e.g., forms were sent back and forth between home and school electronically), this correspondence occurred through email. If it was not customary for the school/classroom teacher to communicate through email with parents, or there were parents who were not regularly on email, hard copy distribution occurred. Any child for whom parents opted out of participation in the study was not given the EvsD student engagement survey or the DIBELS Daze reading assessment on the day of the observation. A total of three students were opted out of participation in the study by their parents 57 (n=3). These students were prompted to silently read a book of their choice at their seat during the brief administration of the engagement survey and reading assessment. Observer Training and Agreement Observers included three research assistants (RAs), advanced graduate students currently enrolled in a doctoral-level school psychology program. The author provided extensive training prior to the use of the observation tool that included: (a) an overview of the study and its purpose, (b) an in-depth explanation of the observation instrument, and (c) a paper/pencil test applying definitions to examples (see Appendix L, Research Assistant Training Manual). Once the RAs demonstrated clear understanding of the observation tool (including a score of 100% on the paper/pencil test), the RAs and author individually coded two training videos of social studies instruction. Each training observation session was used to determine inter-rater reliability among the two observers, and each session used new observational materials (i.e., novel instructional period videos). The Proportion Agreement Index (PAI) was calculated by dividing the number of agreements on individual observations of a pair of observers by the total number of observations made. The PAI has possible values between 0.0 (no agreement) and 1.0 (total agreement); satisfactory levels of agreement have been suggested as 0.7 to 0.8 (Croll, 2004). The PAI was calculated for each of the variables coded during the 30-second intervals (instructional format, reading expectation, reading materials, comprehension instruction component, and instructional medium). The author’s codes served as the gold standard, establishing the set of observation codes against which the RA’s codes were compared. Immediately following a training observation session, the author documented and summarized any coding discrepancies among these five variables for each individual RA, and discussed with the RA until she demonstrated an understanding of the correct code. Trainings continued with new observational 58 material until two consecutive observations reached an interobserver agreement level of at least 0.7 to 0.8 across all five variables. Table 4 provides a summary of each RA’s agreement (inter- rater reliability) across variables for the two training videos. Once sufficient interobserver agreement was established (using the procedures above), RAs were assigned to observation sessions throughout data collection, based on their availability as indicated in a shared calendar. Table 4 Interobserver Agreement Levels: RA Training Videos Instructional Reading Format Expectation Reading Materials Comprehension Instruction Component Instructional Medium RA 1 RA 2 RA 3 Video 1 PAIa Video 2 PAI Video 1 PAI Video 2 PAI Video 1 PAI Video 2 PAI 0.85 0.90 0.95 0.93 0.95 0.90 0.75 0.90 0.80 0.93 0.75 0.87 0.95 1.0 0.80 0.90 1.0 0.90 0.90 0.87 0.75 0.83 0.75 0.83 0.80 0.87 0.85 0.87 0.85 0.90 a PAI = percentage agreement index. Values of 0.7 and above considered satisfactory. RA 1 completed a total of 22 observations (4 of which were double coded), RA 2 completed a total of 19 observations (4 of which were double coded), and RA 3 completed a total of 14 observations (2 of which double coded). For the double-coded observation sessions, the author calculated the PAI for each of the five variables. Values ranged from 0.79 to 0.99 across all variables and observations. A total of five inter-rater reliability checks were completed throughout the study to serve as guards against observer drift. Interobserver agreement levels across these double-coded observations are summarized in Table 5. Immediately following a double-coded observation session, the author calculated, documented, and summarized PAIs and any coding discrepancies among the five 59 variables for the RAs. The two RAs reviewed this feedback and then discussed all disagreements with one another until a consensus of the correct code was reached. Table 5 Interobserver Agreement Levels: Double-Coded Observations Instructional Format Reading Expectation Reading Materials RA1 and RA2 PAI a RA 1 and RA 3 PAI a RA 1 and RA 2 PAI a RA 2 and RA3 PAI a RA1 and RA 2 PAI a a PAI = percentage agreement index. Values of 0.7 and above considered satisfactory. 0.92 0.87 0.98 0.93 0.98 0.96 0.96 0.95 0.97 0.98 0.91 0.96 0.88 0.92 0.98 Comprehension Instructional Medium Instruction Component 0.79 0.97 0.93 0.96 0.99 0.83 0.97 0.91 0.93 0.99 Conducting Observations Each teacher scheduled an observation time with the author through email (as noted above), identifying all days/times during which “typical” social studies instruction occurred. All efforts were made to exclude non-typical school days (e.g., half days, assemblies) as well as days devoted to activities that were not focused on classroom instruction (e.g., tests, field trips). In addition, teachers were asked to select a day/time where instruction would include a substantial amount of text use (providing ample opportunity to explore relationships of interest in the present study). When scheduling the observation, teachers were notified that 10 to 15 minutes would be needed immediately following the social studies lesson for the RA to administer group- level measures to students. Teachers confirmed with the author the time frame during which the observation would occur, as well as the subsequent data collection from students (e.g., “lesson will last from 10:00am to 10:45am and data collected from students 10:45am to 10:55am”). RAs also sent teachers reminder emails 48 hours prior to the scheduled observation session in an effort to prevent last-minute cancellations (see Appendix M). Teachers who completed the 60 observation session in its entirety were emailed a $15 Amazon.com gift card within one week of participation as a token of appreciation for their time and effort. For each session RAs had a fidelity checklist (see Appendix N), a clipboard with the observation form, and a timer that vibrated at 30-second intervals to cue them to the next interval (iPhone app Loop ReminderÔ).During each interval, observers coded the following: (a) instructional format, (b) reading expectation, (c) reading materials, (d) comprehension instruction components, and (d) instructional medium. Events occurring at any point during the interval were marked for each of these variables. For example, if two instructional formats were observed during the same 30-second interval, such as whole class and partner, both were marked. Observers focused on what was happening instructionally for the majority of students (i.e., not individual instruction). Observers additionally used a running checklist to record comprehension strategies observed at any point during instruction; i.e., regardless of how frequently a teacher incorporated a specific comprehension strategy throughout the lesson, such as prediction or a graphic organizer, it was marked once on the checklist. The observer(s) recorded all variables until the social studies lesson was complete, per the time frame provided by the teacher in advance. Student Assent Immediately following the lesson observation, all students in the class whose parents did not opt them out received a student assent form (see Appendix O). The RA introduced the study and read the assent form aloud to students. Completed assent forms were then collected by the RA. Any students who do not provide their assent to participate were not given the EvsD student engagement survey or the DIBELS Daze reading assessment. Students who did not provide their assent, along with those whose parents opted them out, were prompted by the RA to silently read 61 a book of their choice at their seat during administration of the engagement survey and reading assessment. A total of 108 students either (a) did not provide their assent to participate or (b) elected to discontinue their participation during the engagement survey or reading measure (and thus their data were excluded). Student Engagement and Reading Levels Students in the class who provided their assent (and whose parents did not opt them out of participation) were given the EvsD engagement survey followed by the DIBELS Daze. Measures were administered by the RA at the whole class level, as described and scripted in Appendix A and Appendix B. Students were also given a Likert scale item to provide additional descriptive information, stating “How interested were you in the topic of [insert topic learned that day] before today’s class?” Responses were on a scale from 1 to 5, with 1 indicating not all interested and 5 indicating very interested (see Appendix P). Students did not write any identifying information on their forms; RAs coded these with a random identification number in advance. Students only circled their responses for each measure within the packet. The author reminded students that their responses would not be accessed by parents or teachers, and results would not count toward anything in their class or their grades. Reading Materials and Teacher Items The finals step was for the RAs to (a) collect copies of social studies text materials used during the observed instruction and (b) provide teachers the Likert scale items. If the teacher did not have text materials ready in advance, the RAs requested original copies of any materials utilized and access to a photo copier. This included all social studies text materials used during instruction, including but not limited to social studies worksheets, textbook sections, workbook pages, and text presented on projectors or personal computers. The RAs also 62 recorded the title and year associated with any text utilized during the observation period. The author calculated the text readability level for each item, as well as the value for the text readability variable, as detailed in the measures section. Teachers responded to a few Likert scale items immediately following the observation period to provide additional descriptive information (see Appendix P). These items included, “In general, how similar was the amount of text used today to the amount you typically use during social studies” and “In general, how similar were today’s instructional practices to how you typically teach with text during social studies?” Responses were on a scale from 1 to 4, with 1 indicating not at all similar and 4 indicating very similar. Overview of Data Analyses An overview of data analysis for each research question is presented in Table 6 below. Research questions were addressed through descriptive statistical analyses (RQ1, RQ2) and multilevel modeling (RQ3, RQ4, RQ5). Table 6 Overview of RQs, Variables, Measures, and Analysis Research Question RQ 1. What is the readability grade level of text materials used during social studies instruction in fifth grade classrooms? Variable(s) Text Readability Grade Level Measure(s) Text readability analysis system (ATOS readability formula) Analysis Descriptive statistics 63 Descriptive statistics Multilevel Regression Model (Model 2, Random- Intercepts) Multilevel Regression Model (Model 3, Means as Outcomes) Multilevel Regression Model (Model 4, Random Intercepts and Slopes) Table 6 (cont’d) RQ 2. How frequently does fifth grade social studies instruction include at least one reading comprehension instructional component (percentage of time intervals)? How often is each type of comprehension instructional component being observed? RQ 3. To what extent does student reading achievement level (Level 1) predict student engagement during fifth grade social studies instruction among all students? RQ 4. To what extent do classroom-level reading factors (Level 2) predict student engagement during fifth grade social studies instruction among all students? Among struggling readers? RQ 5. To what extent does the frequency of reading comprehension instruction (percentage of time intervals) moderate the relationship between student- and classroom- level reading factors and student engagement among all students? Among struggling readers? Reading Comprehension Instruction Observation tool (3a) Reading Achievement Level (3b) Student Engagement (3a) DIBELS Daze reading assessment (3b) Student engagement survey (EvsD) (4a) Text Readability Grade Level (4b) Reading Comprehension Instruction (4c) Student Engagement (5a) Reading Achievement Level (5b) Text Readability Grade Level (5c) Reading Comprehension Instruction (5d) Student Engagement (4a) Text readability analysis system (ATOS readability formula) (4b) Observation tool (4c) Student engagement survey (EvsD) (5a) DIBELS Daze reading assessment (5b) Text readability analysis system (ATOS readability formula) (5c) Observation tool (5d) Student engagement survey (EvsD) 64 CHAPTER IV: RESULTS The present observational study used descriptive and multilevel analyses to address the research questions and variables of interest. More specifically, two research questions were examined through descriptive statistical analyses (RQ1, RQ2), and three research questions were examined through multilevel regression analyses (RQ3, RQ4, and RQ5). This chapter presents findings from the study, organized by first presenting descriptive information (e.g., length of observation, time spent reading text; followed by data preparation procedures (e.g., centering variables); then descriptive statistic results for RQ1 and RQ2; followed by preliminary analyses for multilevel modeling; and finally results for the multilevel models (RQ3, RQ4, and RQ5). These results aimed to provide an enhanced understanding of fifth grade classroom reading- related factors, reading comprehension instruction, and student engagement, while facilitating exploration of the relationships among these variables. Descriptive Information A summary of descriptive statistics for all predictor and outcome variables is presented in Table 7 for all students. Level 1 variables also include descriptive statistics for the subgroup of struggling readers. Table 7 Descriptives for Predictor and Outcome Variables: All Students Measure Student Reading Achievementa All Students Struggling Readers Text Readability Levelb Comprehension Instructionc Student Engagementd All Students Struggling Readers M 19.72 13.75 7.24 0.63 65.26 64.40 Minimum Maximum 0 0 4.7 0.14 26 26 52 19 10.5 0.94 80 80 SD 7.91 4.20 1.04 0.20 9.99 9.95 65 Table 7 (cont’d) All students sample size = 1,083 and struggling reader sample size = 556. aUnits for Reading Achievement = student score on DIBELS Daze bUnits for Text Readability Level = ATOS readability score of text materials used cUnits for Comprehension Instruction = proportion of observed intervals with comprehension instruction dUnits for Student Engagement = student score on EvsD survey The average observation session in the present study lasted 39 minutes, ranging from 20 to 60 minutes (M=38.83, SD=11.33). Teachers were asked to select an observation period that would include “substantial text use”; observation tool data indicated all teachers (100% of classrooms) used text materials at some point during the observed lesson. On average, text materials were used during 67% of all observed intervals, ranging from 30% to 96% (M=67.38, SD = 23.99). During the observed lessons, instruction was most often in whole class format (71% of all intervals), followed by partner and independent/individual formats (14% of all intervals each), and finally small group format (10% of all intervals). Note that more than one format could be used and therefore coded throughout an interval, so the total percentage sums to greater than 100. Teachers responses to Likert scale items indicated similar text use and instructional practices to their typical social studies lessons, with average ratings of 3.26 for text use (SD=0.75, Min = 1, Max = 4) and 3.48 for instructional practices (SD=0.58, Min=2, Max = 4). These findings suggest that per teacher self-report, the observed lessons were generally similar to a “typical day” in that classroom. Student responses to their Likert scale item indicated being neutral to somewhat interested in the social studies topic from the observed lesson, with an average rating of 3.67 (SD=1.22, Min = 1, Max = 5). A bivariate correlation analysis indicated that student interest and student engagement were positively and moderately correlated (r=0.59, p<.01 among all student; r=0.61, p<.01 among struggling readers). 66 Preparing Data for Analyses All predictor variables were transformed prior to analysis through centering procedures. Two centering approaches were applied to the independent variable predictors to minimize the likelihood of multicollinearity in the models and yield a more accurate estimate of the intercepts (Hox, Moerbeck, & van de Schoot, 2010; Woltman, Feldstain, MacKay, & Rocchi, 2012). Group mean (classroom mean) centering was applied to the student-level independent variable of student reading achievement by subtracting the group mean from each individual student’s score on the variable. Grand mean centering was applied to all classroom-level independent variables (i.e., text readability grade level, teacher provision of reading comprehension instruction) by subtracting the sample mean from each individual classroom’s score on the variable (Tabachnick & Fidell, 2014; Woltman, Feldstain, MacKay & Rocchi, 2012). A separate file was created for each of the two levels of data is SPSS. Each file contained the participants’ data for the variable(s) at that level, plus an identification code to link the data from each case between levels. This identification code variable was in string format; it contained the same number of digits for all levels, was given the exact same variable name at all levels (i.e., ID), and used to sort the data file from lowest to highest value (Hox, Moerbeck, & van de Schoot, 2010; Woltman, Feldstain, MacKay, & Rocchi, 2012). The Level 1 file contained all individual data scores for the measurement of student reading achievement level (Level 1 predictor), a categorical code indicating whether the student is a struggling reader, (0=no, 1=yes), and student engagement (outcome). Each individual was also assigned an identification code (range: 01 to 50) based on their classroom membership (Level 2 unit). The Level 2 file contained 50 data points for the classroom-level measures of text readability grade level and 67 teacher provision of reading comprehension instruction, as well as identification codes (range: 01 to 50). RQ1 Descriptive Statistic Analyses: RQ1 and RQ2 The average ATOS readability level in the present study was 7.24, ranging from a minimum of 4.7 to a maximum of 10.5 (M=7.24, SD =1.04). This average score falls within the range of appropriate instructional text material at the sixth-grade level (ATOS score of 7.0 to 8.0). The minimum score falls within the range of appropriate instructional text material at the third grade level (ATOS score of 3.9 to 5.1) and the maximum score falls within the range of appropriate instructional text material at the ninth grade level (ATOS score of 9.67 to 10.99). Observation tool data indicated that the type of reading material utilized by teachers during the observed lessons was most often textbooks (63% of intervals with text), followed by “other text” such as news articles, essays, and passages (40% of intervals with text), and finally worksheet pages (17% of intervals with text). Note that more than one type of text material could be utilized and therefore coded throughout an interval, so the total percentage sums to greater than 100. RQ2 Fifth grade social studies instruction included at least one reading comprehension instructional component an average of 63% of intervals, ranging from 14% to 94% of all intervals observed [M=0.63 (proportion of intervals), SD=0.19]. The most frequently observed comprehension components across classrooms were independent student use and guided/collaborative practice (48% and 45% of all comprehension instruction intervals, respectively). Teachers rarely provided direct instruction (6% of all comprehension instruction intervals) or modeling (4% of all comprehension instruction intervals). Note that more than one 68 component could be present and therefore coded throughout an interval, so the total percentage sums to greater than 100. Of the 50 classrooms observed, 48 included guided/collaborative practice of a strategy (96% of classrooms) and 39 included independent strategy use by students (78% of classrooms). 20 classrooms included direct instruction of a strategy (40% of classrooms) and 16 included modeling (32% of classrooms). Running checklist data summarizing the specific comprehension strategies used at any point during instruction are summarized in Table 8 below. On average, teachers used 4 to 5 different types of comprehension strategies throughout the entire instructional period, ranging from a minimum of 2 to a maximum of 9 types of comprehension strategies (M=4.72, SD=1.69). Table 8 Specific Comprehension Instruction Strategies Observed: Frequencies Strategy Discussion Activity Previewing/Prediction Comprehension Monitoring Vocabulary Percent of Classrooms Definitions Context Clues Morphology Background Knowledge Prior Knowledge Building Knowledge Reading Comprehension Questions Text Annotation Visual/Graphic Organizer Summary/Relevant Information Main Idea Text Structure Frequency (Number of Classrooms Observed In) 33 21 1 27 6 5 35 20 38 17 14 12 6 1 69 66 42 2 54 12 10 70 40 76 34 28 24 12 2 Preliminary Analyses for Multilevel Models (RQ3, RQ4, RQ5) Three assumptions of multilevel modeling were investigated prior to analyses for RQ3, RQ4, and RQ5 including: (a) no multicollinearity among predictors, (b) linear and normal distribution of residuals for all predictors, and (c) all predictors are independent of their level- related error (residuals) and their error terms are independent of each other. The Level 1 predictor was reading achievement level, while the Level 2 predictors were text readability grade level and reading comprehension instruction provision (Hox, Moerbeck, & van de Schoot, 2010). Results for each of these assumptions are described below. (a) No Multicollinearity The relationship among independent variables was examined to determine the presence of multicollinearity. Specifically, bivariate correlations were reviewed the ensure the independent variables were not highly correlated (r = 0.9 or above), as multicollinearity will result in a poor regression model (Kline, 2005; Pallant, 2013). These correlations are presented in Table 9. Based on these findings, it appeared that the association between predictors did not violate assumptions of multicollinearity; the strongest correlation was between text readability grade level and reading comprehension instruction provided (r = -0.16, p<.01). These results suggest that classrooms with higher text readability grade level materials are more likely to have lower levels of reading comprehension instruction provided. Other correlations among predictors were low and nonsignificant with r values of -0.05 for reading achievement level and text readability grade level (p=.10), and 0.05 for reading achievement level and reading comprehension instruction provided (p=.11). Given low levels of correlation between predictors (r<0.9), the assumption of no multicollinearity was met. 70 Table 9 Pearson Correlation Among Predictors Reading Achievement Text Readability Level Reading Achievement Text Readability Level Comprehension Instruction ** = Correlation significant at p<.01 level. 1 -0.05 0.05 -0.05 1 -0.16** (b) Linear and Normal Distribution of Residuals Comprehension Instruction 0.05 -0.16** 1 The two residual terms from the baseline null model of the multilevel analysis (described in detail below) were examined to determine whether the assumptions of linearity and normality were met (Tabachnick & Fidell, 2014). To test the linearity assumption, standardized residuals for both Level 1 (rij) and Level 2 (u0j) were plotted against their normal scores. The resulting scatterplots showed a relatively straight diagonal line (as opposed to non-linear or U-shaped); thus, the residuals met the assumption of linearity. To further test the normality assumption, QQ plots (quantile-quantile plots) were used. The residuals data examined through the QQ plots again indicated a roughly normally distributed curve shape (as opposed to highly skewed) (Finch, Bolin, & Kelley, 2014; Hox, Moerbeck & van de Schoot, 2010). (c) Level 1 and Level 2 Predictor Error Terms Correlations were also reviewed to ensure the independent variables are independent of their level-related error, and that their error terms were independent of each other (intraclass correlation coefficient, as described and interpreted in the baseline null model below). Multilevel Models A multilevel regression model (also known as hierarchical linear model or random coefficient model) was used to address RQ3, RQ4, and RQ5. This approach was used to analyze the data given students (Level 1) are nested within classrooms (Level 2). Multilevel modeling 71 allowed for simultaneous examination of the effects of Level 2 variables (classroom level) and Level 1 variables (student level) given the nested data set (Hox, Moerbeck, & van de Schoot, 2010). Of specific interest was the relationship between student engagement (Level 1 outcome variable) and the predictors of student reading achievement level (Level 1), text readability grade level (Level 2), and teacher provision of reading comprehension instruction (Level 2). Model testing proceeded in four phases: baseline (null) model, random intercepts model, means-as- outcome model, and intercepts- and slopes-as-outcomes model. This sequence was first analyzed with data from all students in the sample, and then repeated with just the subset of struggling reader data. Results for each model are summarized in Table 10 and Table 11 are further described below with reference to the present study’s specific research questions. Table 10 Results of Multilevel Modeling Analyses: All Students Model Model 1: Null (Baseline) Model 2: Random Intercepts Model 3: Means-as- Outcome Model 4: Intercepts- and Slopes-as- Outcomes 65.30** (0.62) 65.30** (0.62) 65.30** (0.63) 65.43** (0.62) 0.11** (0.04) 0.12** (0.04) Level and Variable Level 1 Intercept (!00) ReadAch (!10) TextLevel (!01) ComprInstr (!02) Read Ach*TextLevel (!11) ReadAch*ComprInst (!12) TextLevel*ComprInst (!03) Interactions Level 2 Variance Components Within-class (L1) variance (σ2) 86.27 Intercept (L2) variance (τ00) 14.68 Slope (L2) variance (τ11) Additional information 85.53 14.74 -0.64 (0.61) 0.64 (3.25) 86.26 15.02 -0.69 (0.60) 1.20 (3.21) 0.01 (0.04) 0.32 (0.22) 5.35* (2.03) 85.56 14.34 0.001 ICC R2 .01 Note: L1=Level 1 and L2 = Level 2. L1 sample size = 1,083 and L2 sample size = 50. Values in parentheses are standard errors. *p< .05. **p <.01. 0.15 .01 .02 72 Table 11 Results of Multilevel Modeling Analyses: Struggling Readers Model Model 1: Null (Baseline) Model 3: Means-as- Outcome Model 4: Intercepts- and Slopes-as- Outcomes 64.44 (0.68)** 64.46** (0.67) 64.60** (0.66) 0.23* (0.11) -0.96 (0.65) 2.01 (3.51) -1.06 (0.64) 2.63 (3.46) 86.14 13.32 .01 -0.12 (0.09) 0.69 (0.58) 6.33* (3.64) 85.25 12.50 0.01 .01 Level and Variable Level 1 Intercept (!00) ReadAch (!10) TextLevel (!01) ComprInstr (!02) Read Ach*TextLevel (!11) ReadAch*ComprInst (!12) TextLevel*ComprInst (!03) Interactions Level 2 Variance Components Within-class (L1) variance (σ2) 86.22 Intercept (L2) variable (τ00) 13.43 Slope (L2) variance (τ11) Additional information ICC R2 0.14 Note: L1=Level 1 and L2 = Level 2. L1 sample size = 556 and L2 sample size = 50. Values in parentheses are standard errors. *p< .05. **p <.01. Model 1: Baseline Model (Null) Before examining the three specific research questions, an unconditional null model was first estimated. This model was used to determine (a) whether the Level 2 units (classrooms) differ on the outcome of student engagement and (b) the degree of nonindependence in the outcome of student engagement across Level 1 units (students) through the intraclass correlation statistic. The multilevel equation for this model was: Model 1, Level 1: Yij = β0j + rij Model 1, Level 2: β0j = !00 + u0j 73 where Yij is the student engagement score for student i in classroom j during the observed classroom instruction. In Level 1 of Model 1, β0j is the average student engagement score in classroom j (intercept), and rij quantifies the difference between classroom j’s average student engagement score and student i’s student engagement score (i.e., Level 1 residual). In Level 2 of Model 1, !00 is the grand mean (average student engagement score across all classrooms), and u0j is the difference between classroom j’s average student engagement score and the grand mean (i.e., Level 2 residuals). The first purpose of this model including the student engagement measure (outcome) and no predictors was to determine whether there were any differences at the group level on the outcome variable – i.e., confirming the need for multilevel modeling. In other words, this model was used to test whether variability in the outcome variable, by Level 2 group (classrooms), is significantly different from zero. Examination of the chi-square test was of focus; results were statistically significant, indicating that there is variance in student engagement by the Level 2 classroom groupings – i.e., that there is statistical justification for running multilevel model analysis [χ2(49) = 216.01, p<.001]. As an additional step, the intraclass correlation coefficient (ICC) was calculated. The ICC represented how much variance in the outcome could be accounted for by the clustering/Level 2 predictors. Thus, the ICC was used to determine the proportion of variance in student engagement that can be explained at both the student level (Level 1) and classroom level (Level 2). The ICC was calculated using the σ² (variance of Level 1 residual variance) and τ00 (variance of Level 2 residual) terms derived through the null model analysis. This value represented the percentage of total variance in the student engagement scores that can be accounted for by differences at the classroom level; the remaining percentage not accounted for represents the 74 percentage of variance in student engagement attributable to differences at the individual level. The ICC was defined as: τ00 τ00 + σ² ICC = where σ² is the estimated residual variance (the variance of the values of rij; within classrooms) and τ00 is the estimated variance of the random components (intercept variance; between classrooms). The ICC value of the null model was 0.15. This result suggested that 15% of the variance in student engagement scores was accounted for by differences at the group level (classrooms) and 85% the individual level (student). Because variance existed at both levels of the data, predictor variables were individually added at each level for the remaining models. Model 1: Baseline (Null) Model for Struggling Readers Model 1 and the ICC were then calculated only with data from struggling readers to create a baseline model for elements of RQ4 and RQ5. Examination of the chi-square test was of focus; results were statistically significant, indicating that there is variance in student engagement by the Level 2 classroom groupings – i.e., that there is statistical justification for running multilevel model analysis [χ2(49) = 131.42, p<.001]. The ICC value of this null model was 0.14. This result suggested that 14% of the variance in struggling reader student engagement scores was accounted for by differences at the group level (classrooms) and 86% the individual level (student). Because variance existed at both levels of the data, predictor variables were individually added at each level for the remaining models. Model 2: Random-Intercepts Model (Level 1 Predictor, RQ3) The second model included the Level 1 predictor of student reading achievement level (variable name READACH). Specifically, this model was used to determine the relationship between student reading achievement and student engagement (outcome). By selecting both error 75 terms, the analyses include estimates of both the between-and within-group error. Specifically, u0j starts with the assumption that student engagement varies from classroom to classroom and u1j starts with the assumption that strength of the relationship between student reading achievement and student engagement (outcome) varies from classroom to classroom. The Model 2, Level 1: Yij = β0j + β1j*READACHij + rij multilevel equation for this model was: Model 2, Level 2: β0j = !00 + u0j β1j = !10 + u1j where Yij, β0j, rij, and u0j are defined as described prior. β1j quantifies the relationship between student engagement scores in class j as a function of those student achievement scores (slope), !00 is the estimated mean student engagement score when classrooms interact with students who demonstrate an average reading achievement score, !10 is the estimated average effect of student reading achievement across classrooms, and u1j is the random component of the effect of student reading achievement (i.e., quantifies how the effect of student reading achievement for classroom j differs from the average). classrooms (coefficient !10) was looked at specifically to determine whether the effect of this The resulting estimate for the average effect of student reading achievement across variable was significantly different from zero. This regression coefficient relating student reading achievement to student engagement was positive and statistically significant (β= 0.12, p <.01). Student engagement was higher when their reading achievement levels were also higher (relative to those whose reading achievement was lower). Thus, the relationship can be interpreted and the null hypothesis of no relationship between these two variables was rejected; and as a result, the impact of student reading achievement was controlled for in subsequent models. 76 This above relationship was also examined by comparing the deviance of the null model (Model 1) to the deviance of this model (Model 2). By comparing the σ² value between Model 2 and Model 1, a variance accounted for measure was derived and interpreted. To calculate a measure of effect size (r²), the variance explained by the Level 1 predictor student reading achievement in student engagement (outcome) was computed using the following equation: r² = σ²null - σ²random σ²null where σ²null is the sigma value obtained in the previous null model and σ²random is the sigma value obtained through the present model. The resulting value indicated that student reading achievement explains 1% of the variance in student engagement (r² = 0.01). Model 3: Means as Outcomes Model (Level 2 Predictors, RQ4) Next, the relationship between the Level 2 predictor variables and the outcome variable was tested. The Level 2 predictor variables of text readability grade level (variable name TEXTLEVEL) and reading comprehension instruction provision (variable name COMPRINSTR) were added as Level 2 predictor variables, as well as the same outcome variable as prior. Coefficients were estimated and, as before, significance values were used to determine the relationship between the Level 2 predictor variables and student engagement. The multilevel equation for this model was as follows: Model 3, Level 1: Yij = β0j + rij Model 3, Level 2: β0j = !00 + !01*TEXTLEVELj + !02*COMPRINSTRj + u0j where Yij, β0j, rij, and u0j are defined as described prior. !00 is the grand mean (average student engagement score across all classrooms), !01 is the estimated average effect of TEXTLEVEL across classrooms and !02 is the estimated average effect of COMPRINSTR across classrooms. 77 comprehension instruction provision (coefficients !01 and !02 respectively) were examined to The resulting estimate for the effects of text readability grade level and reading determine whether there was a relationship between each predictor and the outcome. The regression coefficient relating text readability grade level to student engagement was negative and not statistically significant (β= -0.64, p =.30). The regression coefficient relating reading comprehension instruction provision to student engagement was positive and not statistically significant (β= 0.64, p=.85). Therefore, the null hypotheses for RQ4 of no relationship between text readability level and student engagement, as well as no relationship between reading comprehension instruction and student engagement, were retained. For a measure of the effect size of these predictors, the deviance of the null model (Model 1) and the deviance of this model (Model 3) were compared. By comparing the τ² value between Model 3 and Model 1, a variance accounted for measure was derived. To calculate a measure of effect size, the variance (r²) explained by the Level 2 predictors in student engagement (outcome) was computed using the following equation: r² = τ ²null - τ ²means τ ²null where τ²null was the τ value obtained in the previous null model and τ ²means was the τ value obtained through the present model. The resulting value indicated that text readability grade level and reading comprehension instruction provision (Level 2 predictors) explain 2% of the variance in student engagement (r² = 0.02). Model 3: Means as Outcomes Model (Level 2 Predictors, RQ4) for Struggling Readers Model 3 and the effect size calculations were then conducted only with data from struggling readers. The regression coefficient relating text readability grade level to student engagement among struggling readers was negative and not statistically significant (β= -0.96, p 78 =.15). The regression coefficient relating reading comprehension instruction provision to student engagement was positive and not statistically significant (β= 2.01, p=.57). Therefore, the null hypotheses for RQ4 were retained (i.e., no relationship between text readability level and student engagement for struggling readers, no relationship between reading comprehension instruction and student engagement for struggling readers). To calculate a measure of effect size, the variance (r²) explained by the Level 2 predictors in student engagement (outcome) was computed. The resulting value indicated that text readability grade level and reading comprehension instruction provision (Level 2 predictors) explain 1% of the variance in student engagement among struggling readers (r² = 0.01). Model 4: Random Intercepts and Slopes Model (Moderation, RQ5) The final model tested for interactions between the predictor variables. Thus, it included the same terms as Model 3, but added the Level 1 predictor to the Level 1 model and added the Level 2 predictor variables to both Level 2 equations. Coefficients were estimated and, as before, significance values were used to determine whether there were any interaction effects on student engagement in the model. Specifically, this model was used to determine the extent to which COMPRINSTR might moderate the relationship between other predictors and student engagement. The multilevel equation for this model was as follows: Model 4, Level 1: Yij = β0j + β1j*READACHij + rij Model 4, Level 2: β0j = !00 + !01*TEXTLEVELj+ !02*COMPRINSTRj + !03*(COMPRINSTR*TEXTLEVEL)j + u0j β1j = !10 + !11*TEXTLEVELj+ !12*COMPRINSTRj + u1j The regression coefficient for the interaction effect of COMPRINSTR and READACH on student engagement was positive and not statistically significant (β= 0.32, p =.15). This indicated that the degree of comprehension instruction provision did not influence the strength of 79 the relationship between student reading achievement level and student engagement. The regression coefficient for the interaction effect of TEXTLEVEL and READACH on student engagement was likewise positive and not statistically significant (β= 0.01, p =.83). This indicated that the degree of text readability grade level did not influence the strength of the relationship between student reading achievement level and student engagement. However, the resulting regression coefficient for the interaction effect of COMPRINSTR and TEXTLEVEL on student engagement was positive and statistically significant (β= 5.35, p < .05). This indicated that the degree of comprehension instruction provision influenced the strength of the relationship between text readability grade level and student engagement. Closer examination of the plotted interaction effect showed that classrooms with lower text levels had higher engagement levels when comprehension instruction levels were low and lower engagement levels as comprehension instruction levels increased. Conversely, classrooms with higher text levels had higher engagement levels when comprehension instruction levels were high and lower engagement levels as comprehension instruction levels decreased. This above relationship was also examined by comparing the deviance of the null model (Model 1) to the deviance of this model (Model 4). By comparing the σ² value between Model 4 and Model 1, a variance accounted for measure was derived and interpreted. To calculate a measure of effect size (r²), the variance explained by all predictors and interactions in student engagement (outcome) was computed using the following equation: σ²null - σ²random r² = σ²null 80 where σ²null is the sigma value obtained in the previous null model and σ²random is the sigma value obtained through the present model. The resulting value indicated that this model explains 1% of the variance in student engagement (r² = 0.01). Model 4: Random Intercepts and Slopes Model (Moderation, RQ5) for Struggling Readers Model 4 and the effect size calculations were then conducted only with data from struggling readers. The regression coefficient for the interaction effect of COMPRINSTR and READACH on student engagement was positive and not statistically significant (β= 0.69, p =.24). This indicated that the degree of comprehension instruction provision did not influence the strength of the relationship between student reading achievement level and student engagement among struggling readers. The regression coefficient for the interaction effect of TEXTLEVEL and READACH on student engagement was negative and not statistically significant (β= -0.12, p =.21). This indicated that the degree of text readability grade level did not influence the strength of the relationship between student reading achievement level and student engagement among struggling readers. However, the resulting regression coefficient for the interaction effect of COMPRINSTR and TEXTLEVEL on student engagement was positive and statistically significant (β= 6.33, p < .05). This indicated that the degree of comprehension instruction provision influenced the strength of the relationship between text readability grade level and student engagement among struggling readers. Closer examination plotted interaction effect for struggling readers showed trends comparable to those observed in the model with all students. Specifically, classrooms with lower text levels had higher engagement levels among struggling readers when comprehension instruction levels were low and lower engagement levels among struggling readers as comprehension instruction levels increased. Conversely, classrooms with higher text levels had higher engagement levels among struggling readers when comprehension 81 instruction levels were high and lower engagement levels among struggling readers as comprehension instruction levels decreased. This interaction effect is depicted in Figure 5 below. Figure 5 Reading comprehension instruction and text level interaction. To calculate a measure of effect size, the variance (r²) explained by all predictors and interaction effects in student engagement (outcome) was computed. The resulting value indicated that this model explains 1% of the variance in student engagement among struggling readers (r² = .01). 82 CHAPTER V: DISCUSSION The purpose of the present study was to systematically observe the use of text materials during fifth-grade social studies instruction and investigate the extent to which teachers incorporate comprehension instruction. The relationship between these classroom-level factors and student engagement was explored among all students as well as struggling readers. This chapter presents the results of the current study with reference to existing literature. It specifically contains the following: interpretation of the results associated with each research question, limitations and future research directions, and implications for educational practice. Research Question 1 For RQ1 the readability level of text materials used during fifth-grade social studies instruction was examined. The average text level observed in the present study (ATOS readability level of 7.24) fell within the sixth-grade instructional range. These findings are congruent with existing literature suggesting social studies text consistently demonstrates readability levels above the grade level for which it is intended to be used (e.g., Allington, 2002; Berkeley et al., 2016; Jitendra et al., 2001; Kinder, Bursuck, & Epstein 1992). Prior research found text was at least two grade levels beyond the grade for which it was designated; in the present study, text levels were on average one grade level beyond. Although a smaller discrepancy was found in the present study than prior research, these findings still support the notion that the readability level of social studies text represents a “mismatch” between reading material demands and student reading proficiency levels (e.g., Berkeley et al., 2016; Harmon, Hedrick, & Fox, 2000). In this case, even fifth-grade students reading at grade level might have had difficulty accessing the text at the sixth-grade level and beyond. 83 Research Question 2 For RQ2 the frequency with which fifth-grade social studies instruction included reading comprehension instruction was examined. Reading comprehension instruction was present during an average of 63% of all intervals observed, ranging from 14% to 94% depending on the classroom observed. These findings were inconsistent with prior observation research, which historically indicated social studies teachers rarely include elements of reading comprehension instruction (e.g., 10% of instructional time observed in Ness, 2007; 12% of instructional time observed in Durkin, 1978). One explanation for this discrepancy might be the frequency with which classrooms were visited in studies such as Ness (2007) and Durkin (1978). In both of these studies, a small sample of teachers were observed during multiple classroom periods. In the present study a larger sample of teachers were visited instead – but on just one occasion, which could have resulted in more biased observations (e.g., increased likelihood of seeing an “atypical” lesson due to subject reactivity; further discussed in Limitations section below). In addition, some previous studies only informed teachers that the intent was to observe instructional strategies in content area classrooms (e.g., Ness, 2007). In the present study, teachers were told that a class period “with substantial text use” was required due to the nature of the research (further discussed in Limitations section below). Thus, the higher rate of observed comprehension instruction may be skewed as teachers were potentially more aware of what was being investigated. Another explanation could be that most existing research observing reading comprehension instruction in the content areas dates back more than a decade. These studies were prior to (a) the Every Student Succeeds Act mandating “well-rounded” education with funding to expand social studies programs; (b) the Common Core State Standards, which include specific benchmarks for comprehension of informational social studies text; and (c) the majority 84 of states requiring a state-designed social studies assessment. Therefore, the high rate of comprehension instruction observed in the present study might better reflect the current educational landscape, with its shift toward greater emphasis on and expectation for content area literacy instruction. The discrepant findings might also be due to how comprehension instruction was defined and measured. Similar to the present study, Ness (2007) coded reading comprehension instruction when instructional components of Duke and Pearson’s model (2009) were present (e.g., direct instruction, guided practice). However, Ness (2007) only included and coded the following seven reading comprehension practices: question answering, question generating, summarization, comprehension monitoring, cooperative learning, graphic organizer use, and identifying text structure. Although these comprehension strategies overlap some with those included in the present study, Ness appears to have included fewer strategies in her definition of comprehension instruction. Coding system differences such as these might account for “reading comprehension instruction” differences between prior research and the present study. To further address RQ2, the nature of reading comprehension instruction was examined. This included describing the observed comprehension instructional components (e.g., direct instruction, collaborative/guided practice) as well as the specific comprehension strategies (e.g., prediction, graphic organizer use, defining vocabulary). With respect to the components, teachers most often provided guided/collaborative practice of comprehension strategies or had students independently use comprehension strategies; in fact, these two comprehension components totaled more than 90% of observed comprehension instruction intervals. Teachers rarely provided direct instruction or modeling of strategies, which were each identified during just 6% and 4% of observed comprehension instruction intervals, respectively. Although 96% of 85 classrooms included guided/collaborative practice and nearly 80% of classrooms included independent student strategy use, just 40% of classrooms included direct instruction and 32% of classrooms included modeling of comprehension strategies. These findings reflect much greater use of the “student-centered” comprehension instructional components, as opposed to the more “teacher-centered” components described in Duke and Pearson’s research-supported model (2009). This finding is consistent with Durkin’s (1978) milestone work, which noted that fourth- grade teachers were never observed providing “teacher-directed” support during social studies such as direct instruction or demonstrating comprehension concepts; when comprehension instruction was present, teachers incorporated collaborative or independent practice (e.g., questions/conversations, peer-group reading, independent worksheets). A potential explanation for the limited use of “teacher-centered” instruction in the present study could be beliefs held by teachers providing social studies instruction. For decades, researchers have explained that teachers’ beliefs about reading in particular tend to influence their instructional practices (e.g., Bliem & Davinroy, 1997; Pressley, 2006). Literature specific to content area teachers indicates that they often believe reading instruction is not their responsibility (e.g., Alger, 2007; Cantrell, Burns and Callaway, 2009; Donahue, 2000; Hall, 2005; O’Brien & Stewart, 1990; Spencer et al., 2008). Moreover, a study by McCoss-Yergian and Krepps (2010) found that nearly 80% of teachers reported neutral or negative attitudes toward teaching reading in their content area classrooms. It is plausible that teachers in the present study held generally negative attitudes toward teaching reading during social studies or did not view it as their responsibility – and in turn, were less likely to provide “teacher-centered” support such as direct instruction or modeling. 86 An additional explanation for this finding could be that teachers do not have sufficient training in reading instruction. Experts emphasize that most elementary teachers receive training focused on narrative text, with little guidance on teaching informational text (Moss, 2005). Content area teachers themselves frequently report limited knowledge in reading strategy instruction and feeling unprepared to provide it (e.g., Bintz, 1997; Cantrell, Burns, & Callaway, 2009; Hall, 2005; Jackson & Cunningham, 1994; McCoss-Yergian & Krepps, 2010; Spencer et al., 2008). For instance, a survey of upper-elementary teachers found less than a third had taken a course specifically focused on content area reading (Gernon & Grisham, 2002). McCoss-Yergian and Krepps (2010) found nearly 75% of teachers reported being unfamiliar with content area literacy strategies. This would also echo more specific findings from teacher interviews in Ness’s (2007) study, in which social studies teachers indicated that they did not feel qualified and were unlikely to provide explicit instruction in reading comprehension. To this end, it is possible that teachers in the present study did not feel they had the training or expertise necessary to provide direct instruction of comprehension strategies (teacher-centered), but perhaps felt comfortable enough to provide collaborative practice or have students use strategies independently (student- centered). Data for the specific reading comprehension strategies indicated that classroom instruction included an average of four to five different strategies throughout the observed lesson; a minimum of two strategies and as many as nine different comprehension strategies were observed. In sum, teachers commonly facilitated student use of multiple comprehension strategies in their classrooms. Duke and Pearson (2009) highlighted the importance of teachers and students using multiple strategies constantly, as good readers coordinate the use of comprehension strategies (Duke & Pearson, 2009). Thus, it appears teachers in the present study 87 were engaging in a beneficial practice by including several different reading comprehension strategies in their instruction. This finding was additionally consistent with Swanson and colleagues (2016), who observed that a wide variety of reading comprehension strategies were included during instruction among “expert” social studies teachers (as identified by school leaders). Closer examination of the reading comprehension strategies revealed that those often facilitated by teachers included comprehension questions, connecting prior knowledge, discussion activities, and vocabulary definitions (54% to 76% of classrooms). Teachers very rarely incorporated identifying text structure, comprehension monitoring, or vocabulary morphology (2% to 10% of classrooms). These findings were consistent with several research studies in which the most frequently observed reading comprehension strategies included comprehension questions (Durkin, 1978; Giles, Wang, Smith, & Johnson, 2013; Ness, 2007; Swanson et al., 2016), vocabulary definitions (Durkin, 1978; Swanson et al., 2016), and connecting prior knowledge (Durkin, 1978; Giles et al., 2013; Swanson et al., 2016). However, some of the comprehension strategies infrequently observed in this study have been the most prevalent in other research, including identifying text structure (Ness, 2007), summarizing (Ness, 2007), and text annotation (Giles et al., 2013). One explanation for this inconsistency could be the grade levels of focus. The studies that contained some incongruent findings from the present study (i.e., Giles et al., 2013 and Ness, 2007) included observations of middle- and high-school social studies teachers. Durkin’s (1978) study focused on upper-elementary classrooms and had generally more consistent findings with the present study. It also appears that overall, the comprehension strategies observed infrequently in the present study, but more commonly in studies with middle-school teachers, are more complex in nature. Specifically, the strategies of 88 identifying text structure, summarizing, and text annotation involve multiple steps and the aggregation of text information; these were infrequent among fifth-grade teachers in this study, but frequent among middle-school teachers in other studies. The strategies frequently observed in this study –such as comprehension questions, discussion activities, and vocabulary definitions – are seemingly simpler and might require use of particularly basic information from the text. This aligns with Boyle-Baise and colleagues’ study (2008), which described upper-elementary social studies instruction as focused on answering basic, “low level” questions about text. Research Question 3 For RQ3 the relationship between student engagement and student reading achievement level was examined. The resulting coefficient was significant and positive, indicating that student engagement was higher when reading achievement levels were also higher (relative to students whose reading achievement was lower). The effect size calculation for the relationship between student engagement and student reading achievement was small (r²=0.01). These findings were consistent with a large body of literature that suggests student reading achievement and engagement are positively related (Brozo, Shiel, & Topping, 2008; Dolezal, Welsh, Pressley, & Vincent, 2003; Ivey & Johnston, 2013; Klem & Connell, 2004; Reyes et al., 2012; Wharton- McDonald, Pressley, & Hampston, 1998). This existing literature is largely qualitative and descriptive in nature; however, quantitative findings from the present study parallel those from Reyes and colleagues (2012), which indicated that student engagement was significantly and positively related to higher reading grades (p<.001), with a similarly small effect size (Cohen’s d = 0.18). Although causation cannot be inferred, the significant relationship between reading achievement and engagement identified in the present study suggests promoting reading achievement could perhaps be a way to improve student engagement. 89 Research Question 4 For RQ4 the relationships between the classroom-level predictor variables (text readability level, reading comprehension instruction) and student engagement were examined. The resulting coefficient for text readability level was negative and not statistically significant among all students and struggling readers, and the resulting coefficient for reading comprehension instruction was positive and not statistically significant among all students and struggling readers. Though these relationships were not significant, they were in the hypothesized direction (i.e., higher text level corresponded to lower student engagement; higher comprehension instruction corresponded to higher student engagement). The direction of the nonsignificant relationships was consistent between all students and struggling readers; however, it is worth noting that p-values were closer to significant levels among struggling readers. In sum, the nonsignificant results in the present study do not add support to the theoretical and researched links among student engagement, teacher practices, and task characteristics (e.g., text level). This is further described below. The nonsignificant relationship between comprehension instruction and student engagement in the present study diverges from Lutz, Guthrie, and Davis’ (2006) investigation of teacher support and student engagement during integrated reading-science instruction. These authors found that student engagement increased significantly when teachers included various instructional supports, some of which included reading comprehension strategies (e.g., vocabulary definitions). The nonsignificant findings of the present study are also different from those in the Assor, Kaplan, and Roth (2002) study, which found a significant correlation between teachers’ use of strategies that foster understanding of/interest in text and higher levels of engagement among students. Thus, the extent to which reading comprehension instruction in 90 particular (i.e., independent from a “package” including both reading comprehension instructional practices and non-comprehension instructional practices) might be important in facilitating student engagement remains unclear given these nonsignificant results. One explanation for the differing results described above might be research methodology differences between the present study and these previous studies. For instance, Assor, Kaplan, and Roth (2002) studied Israeli students, while the sample in the present study included American students. Teaching practices were indirectly self-reported through questionnaires and interviews by Assor and colleagues (2002), while teaching practices were directly observed in the present study. Lutz and colleagues (2006) focused on behavioral indicators of student engagement and measured it through observational methods, while the present study included both affective and behavioral indicators of engagement and measured it through student self- report. Lastly, unlike both of the above-mentioned studies, the present study measured reading comprehension instructional practices in isolation, as opposed to in aggregate with several other instructional practices. These various methodological difference between previous research and this study may account for the discrepant findings. The relationship between text level and student engagement was found to be negative and nonsignificant. The lack of a significant relationship between text level and engagement was not anticipated, given the sentiment among scholars that complex text above students’ reading levels might create frustration and undermine engagement (e.g., Dougherty Stahl, 2012; O’Connor et al., 2002). This nonsignificant finding suggests that text accessibility might not be as closely related to student engagement as theorized throughout the literature. 91 Research Question 5 For RQ5 the presence of interaction effects was explored. Specifically, it was hypothesized that the degree of comprehension instruction would moderate the relationship between other predictors (i.e., student reading achievement, text readability level) and student engagement. It was anticipated that this moderating relationship would be particularly strong among struggling readers. The interaction term of comprehension instruction and student reading achievement did not account for a statistically significant portion of the variance in student engagement among all students and struggling readers. The interaction term between text readability level and reading achievement likewise did not account for a statistically significant portion of the variance in student engagement among all students and struggling readers. These findings did not support the hypothesis that reading comprehension instruction would moderate the relationship between other predictors and student engagement. An unexpected finding was the positive and statistically significant interaction effect of reading comprehension instruction and text readability level on student engagement for all students as well as struggling readers specifically. Results indicated that classrooms with lower text levels had higher engagement when comprehension instruction levels were low and lower engagement levels as comprehension instruction increased. Conversely, classrooms with higher text levels had higher engagement levels when comprehension instruction levels were high and lower engagement levels as comprehension instruction levels decreased. Put simply, in classrooms with easier texts, students were less engaged when there were higher levels of comprehension support. In classrooms with more difficult texts, students were more engaged when there were higher levels of comprehension support. 92 The significant interaction effect described above aligns with a sentiment among scholars that student success, including engagement, results from instruction that both promotes challenge and provides support (e.g., Gay, 2000; Moley, Bandré & George, 2011; Schussler, 2009; Turner & Meyer, 2004). When students feel appropriately challenged academically they often feel respected, demonstrate a more positive attitude toward their academics, and engage more (Schussler, 2009; Turner & Meyer, 2004). When students feel academic work is too difficult or too easy (unchallenging), they often perceive a lack of respect from teachers, demonstrate a more negative attitude toward their academics, and disengage more (Schussler, 2009). Creating opportunity for student success by balancing challenge and support is described by some researchers as the Goldilocks Philosophy: not too much instructing (overteaching), not too little instructing (underteaching), but just the right amount (Moley, Bandré, & George, 2011). To accomplish a “just right” balance, Moley and colleagues (2011) highlighted that teachers must be intentional about what texts they select, and that through scaffolded instruction students will become more challenged, engaged, and knowledgeable. In sum, challenges presented to students must “operate in tandem” with academic instructional support to ensure student engagement (Moley, Bandré, & George, 2011; Schussler, 2009). The significant interaction effect on student engagement in the present study appeared to potentially exemplify this notion. Classrooms where text materials were easier and comprehension instruction was high may not have achieved this “balance” of support and challenge – resulting in lower student engagement. Classrooms where text materials were easier and comprehension instruction was lower were more “balanced”; students perhaps perceived an ability to succeed without feeling like there was too much instruction/support. Classrooms where texts materials were more difficult and comprehension support was lower may not have achieved the “balance”; students perhaps perceived inability to 93 succeed by being over-challenged and under-supported and were therefore less engaged. Classrooms where texts materials were more difficult and comprehension support was high were more “balanced”; students perhaps perceived an ability to succeed due to appropriate challenge and the right amount of support. Additional Findings A few additional aspects of the present research study are worth discussion. First, text materials were used during an average of 67% of intervals, ranging from 30% to 96% of the observed lesson. Of all intervals where text materials were used, approximately 60% included textbooks. A closer look at textbook use by classroom indicated that a total of 28 classrooms (i.e., 56% of classrooms) were observed using textbooks; 11 of those classrooms used textbooks in combination with other text materials (i.e., 22% of classrooms). A total of 33 classrooms (i.e., 66% of classrooms) used other text materials, either independent from or in addition to traditional textbooks. Other text materials included items such as news articles, essays, videos with text, primary documents, and story passages. These findings indicate that the majority of fifth-grade teachers did not appear to solely use textbooks or “over-rely” on them – which has the potential to undermine student interest and engagement (Zhao & Hoge, 2005). This diverges from literature indicating that social studies textbooks are often the “backbone” of instruction, with infrequent use of other text resources (e.g., Benavot, 2011; Ingrao & Middleton, 2012; Zhao & Hoge, 2005). For example, Swanson and colleagues (2016) observed that when social studies text was being used, it included textbooks 80% of the time – a percentage much higher than the present study. As reported above, less than a quarter of social studies teachers in the present study used both textbooks and other text materials. Supplementing textbooks with other instructional text materials has been highlighted in the literature as one feature of exemplary 94 content area teaching (Allington, 2002; Brophy & Alleman, 2009; Dunn, 2000). Experts explain that by complimenting textbooks with other sources, teachers can promote motivation for reading and may improve content area learning (e.g., Moss, 2005). Although most teachers did not appear to “over-rely” on textbooks, only a small portion of teachers used textbooks in combination with other text materials during the observed instruction. Another interesting finding in the present study was the significant negative correlation between text readability grade level and reading comprehension instruction (r=-0.16). This negative association suggested that classrooms with more complex text materials were significantly less likely to have reading comprehension instruction. This finding could indicate that teachers were appropriately adapting instruction to meet the needs of students in their class; perhaps classrooms with more complex text materials had higher student reading achievement levels, and therefore lower levels of teacher comprehension instruction were appropriate. However, post-hoc calculations show minimal differences in average reading achievement between classrooms with the highest readability grade level texts and lowest readability grade level texts. The five classrooms with the easiest text had an average student reading achievement score of 18.21, while the five classrooms with the most challenging text had an average student reading achievement score of 19.06. Similarly, post-hoc calculations showed minimal difference in average reading achievement levels between classrooms with the highest levels of comprehension instruction and lowest levels of comprehension instruction. The five classrooms with the highest levels of comprehension instruction had an average student reading achievement score of 20.10, while the five classrooms with the lowest levels of comprehension instruction had an average student reading achievement score of 20.96. A more indirect explanation for the correlational finding might be that teachers engaged in behaviors that were intended to support 95 students’ understanding of text, but were not focused on comprehension. For instance, a few teachers in the present study were observed reading text aloud to students. In these classrooms the text was several years above grade level. Although teachers perhaps thought they were helping students understand the text by reading it aloud, they were only supporting decoding as they are not able to simultaneously read text aloud and engage in reading comprehension instruction. Therefore, this practice would have corresponded to fewer intervals of comprehension instruction being coded in the present study. Another explanation for this finding could be that some teachers were particularly concerned with students’ abilities to access text materials; these teachers therefore selected lower readability grade level texts and additionally provided more comprehension instruction to their students. Lastly, the amount of variance explained by the predictors across multilevel models was consistently small (i.e., 1% to 2% of variance). Thus, it appears that other factors not measured in this study influenced student engagement. For instance, the literature suggests that classroom teachers’ attitudes toward content area literacy play an important role in student engagement; although instructional techniques themselves are important, scholars note that the attitude with which content area teachers employ those techniques is an arguably more important factor in student engagement and achievement (e.g., Nourie & Lenski, 1998; Schussler, 2009). Similarly, although the present study measured the presence of reading comprehension instruction, it did not account for the quality with which that instruction was provided. Including a measure of strategy instruction quality, such as the 4-point Likert scale used in Swanson et al.’s (2016) observational study, could have been a beneficial predictor. Another factor noted in the literature that affects student motivation and engagement is the level of autonomy support in classrooms (National Research Council, 2004; Ryan & Deci, 2000). Research suggests that teachers support 96 student engagement when they adopt more autonomy-supportive instruction, which includes practices such as considering students’ interests and preferences, presenting activities relevant to students, including choice making, and highlighting meaningful goals (e.g., Jang, Reeve, & Deci, 2010; Reeve, Jang, Carrell, Jeon, & Barch, 2004). Autonomy-supportive instructional practices were not accounted for in the present study, yet it is likely they enhanced student engagement. To this end, it is important for future observational research to carefully consider (a) including predictor variables that capture the quality of reading comprehension instruction (as opposed to just the presence of particular strategies) and (b) including additional research-supported variables that might better capture variance in student engagement, such as autonomy-supportive practices. Limitations and Future Research This section first provides an overview of several limitations of the present study. Then, it describes helpful implications for future research based on findings from this study. Topics covered include the following: potential for subject reactivity, measurement issues, representativeness of sample, limited sample size, and implications for future research. Potential for Subject Reactivity One factor that potentially biased the observed lessons was the phenomenon of subject reactivity. Teachers may have altered their behaviors due to the presence of an observer in the classroom. Moreover, teachers may have been aware of what was being investigated and, in turn, may have taught in atypical ways. One potential way teachers could have been aware of this was through information presented in the consent form, such as the title of the present study and the purpose of the study. Another way this might have happened was asking teachers to select an instructional period with “substantial text use.” This prompt was provided because a goal of the 97 present study was to examine relationships between reading comprehension instruction, text readability grade level, and student engagement. Therefore, it was critical to ensure that teachers would use text materials and have a reason to potentially engage in comprehension instruction. This prompt was also provided because it was only feasible to observe one lesson per classroom due to limited resources available. One way to address subject reactivity could have been to more randomly select class periods. For instance, teachers could have provided three potential dates from which the author would randomly select one (without informing the teacher of the finalized date). Using video- recorded observations of class periods to collect observational data would have also been beneficial in controlling for reactivity among participating teachers. At a minimum, finding ways to conduct multiple observations, despite practical barriers, would have potentially lessened subject reactivity. Future research might include both in-person and video-recorded observations, while examining the consistency of teaching practices between those conditions within the same classroom. A few methodological decisions were made in the present study to mitigate reactivity, such as (a) assuring teachers that information collected during the class period would not be shared with others (e.g., principals), (b) advising teachers to engage in their typical instruction, and (c) collecting ratings from teachers on how similar the observation day text use and instruction was to a typical day in their classroom. Teachers reported that on average, the text use and instructional practices observed were similar to a typical day. Measurement Issues Student self-report. Student engagement was measured using student self-report, leaving room for dishonesty or a lack of understanding relative to more objective observational methods. Thus, students’ self-reported engagement may not reflect actual student behaviors 98 and/or emotions (Appleton et al., 2006). Although attempts were made to minimize dishonesty and misunderstanding among students (e.g., anonymity of surveys, not teacher administered, encouraging clarification questions), these limitations remain inherent to self-report. Reading measure. The student reading achievement measure, the DIBELS Daze, may not have been the best proxy of student reading achievement. Since just one reading passage was used, it is possible that some students had previous exposure to the John Hancock passage, resulting in inflated scores due to practice effects. Practice effects may also have benefited students with previous exposure to DIBELS Daze procedures, relative to those students who had never encountered this assessment format. The likelihood of biased scores due to practice effects could have potentially been decreased by (a) administering more than one passage and (b) asking teachers if students had ever completed the DIBELS Daze, and more specifically the John Hancock reading passage (and using an alternative passage for all students if they had). Observational measure. Every attempt was made to standardize the coding system in the present study, including a comprehensive training procedure and manual, reliability thresholds for research assistants to begin collecting data, and inter-rater reliability checks throughout data collection. These efforts were made to reduce the subjective nature inherent to observational coding. Despite these attempts to maintain consistency and objectivity, it is possible that different observers could have coded the same observed behaviors differently from one another (or differently from what the research assistant manual indicated). Moreover, students were working in small groups or with partners during 24% of the observed instruction. Although observers were asked to focus on what the expectation and instructional focus was for the majority of students, these formats might have made determining the majority difficult and in turn the item coding more subjective. One way to mitigate inconsistency in coding would have 99 been to video record lessons in each classroom; then, ideally, any of the observation sessions could have been randomly checked for inter-rater reliability between the observer and the author. This would have also allowed for more inter-rater reliability checks, as just five classrooms (10% of observations) were double coded in this study due to scheduling limitations and research assistant availability. Reliability measure. Inter-rater reliability was calculated using the Proportion Agreement Index (PAI) measure. While the PAI remained above satisfactory levels throughout this study, this method may have overestimated agreement levels as it does not account for random guessing by coders. Other methods may have more accurately captured consistency among observers by including change agreement in its calculations, such as the intra-class correlation and Cohen’s kappa (Hallgren, 2012). Aggregated information. This study used measures that were aggregated in nature; for instance, comprehension instruction was included in many analyses as a single component, rather than addressing the multiple types of comprehension instruction as separate components. In addition, the E vs D student engagement measure combined emotional and behavioral dimensions, as well as disaffection and engagement across these two dimensions. This aggregate information makes interpretation of results more challenging, as it limits the ability to discern complexities in what was observed. For example, one cannot interpret to what extent the “dosage” of various types of comprehension instruction components (e.g., direct instruction, collaborative use) might relate to student engagement when using an aggregated frequency count. Similarly, one cannot examine how comprehension instruction might differentially relate to behavioral and emotional engagement among students when using an aggregated measure of engagement, or perhaps how the five different types of comprehension instruction components 100 relate to behavioral and emotional engagement differently. Analyses were also aggregated across students in classrooms, further limiting the interpretation of detail in relationships among variables. The author acknowledges the loss of detail and ability to interpret nuanced results through the measures selected in this study. The tension between this and what can be gained through aggregated information, however, was carefully considered throughout the design of this study and its measurement approaches. This study aimed to examine classroom-level dynamics as opposed to individual student-level dynamics, and to study these relationships more systematically (i.e., using regression analyses). This directly addressed a gap in the existing literature, considering the state of research in content area reading instruction. Decisions were also made based on existing observational tools/studies of content area literacy practices, as well as validated measures of elementary student engagement that can account for an entire class during a lesson (i.e., the E vs D). Thus, the aggregated measures used provided benefit in the form of examining larger classroom dynamics and relationships, which have previously been described in a more descriptive manner or on a smaller scale. Representativeness of Sample A total of 111 students either (a) had parents opt out of participation, (b) did not provide assent to participate, or (c) elected to discontinue participation (and thus their data were excluded). This represents 9% of the total potential student sample. This group of students who did not participate could potentially represent an important sector of the student population. For instance, it might be that parents who opted out of participation did so because their children significantly struggle with reading or are particularly disengaged students. Or perhaps those students who discontinued participation did so because they significantly struggle with reading 101 or are particularly disengaged students. This possibility resembles previous research suggesting that selection bias can occur in school-based studies, where there tends to be less consent and in turn less participation among at-risk students (e.g., Anderman et al., 1995; Esbensen et al., 1999). Similarly, the sample of teachers who participated in the present study could potentially be biased. 72% of participating teachers reported having a master’s degree; just 56% of teachers in the United States at large have a master’s degree (National Center for Education Statistics, 2016). In addition, all of the participating classrooms were located in Michigan, which likely presents different from other regions given varying standards and policies among states. Schools were largely located in suburban and rural regions (44% and 40% respectively), with just 16% of classrooms representing urban areas. Sampling limitations may have been reduced by a more random selection of schools and participating teachers. Future research replicating the current study with a more representative sample of teachers and schools would be important for generalizability of findings. Limited Sample Size Fifty classrooms were observed in the present study so there was minimally sufficient power to run the multilevel model analyses. Although this data analysis approach addressed a gap in the current research, there could have been value in observing a larger number of classrooms. Perhaps the minimal sample size in the present study provided insufficient power to detect hypothesized significant relationships among variables. Methodology and Model of Comprehension Instruction The theoretical foundation for this study, the Duke and Pearson (2009) model of reading comprehension instruction, may not have been effectively captured by the methodology used. This model is described as a continuum of components, where teachers move from teacher- to 102 more student- centered components (while sliding up and down as circumstances demand). The present study only took a “snapshot” of instruction, observing a particular component at a particular moment in time. While Duke and Pearson (2009) acknowledge that this is what forms the models and is worth studying, there are limits to fully interpreting the observed teaching practices during one lesson within this more longitudinal model of comprehension instruction. Implications for Future Research There are several additional implications for research based on the findings of this study. First, other variables may be important to include and reveal significant relationships with predictors from the present study; perhaps focusing solely on student engagement as an outcome is not sufficient. For instance, predictors such as comprehension instruction, text level, and reading achievement may be important in facilitating the ultimate goal of increasing social studies knowledge and achievement (despite their nonsignificant relationship with student engagement in the current student). An example of this type of relationship was identified by Wigfield and colleagues (2008), where student engagement mediated the relationship between comprehension instruction with content area text and student achievement outcomes. The nonsignificant relationship between comprehension instruction and student engagement in the present study was unexpected, as it is not well-aligned with other research. Prior studies have identified a significant relationship between engagement and instructional supports (that include comprehension strategies) (e.g., Assor, Kaplan, & Roth, 2002; Lutz, Guthrie, & Davis, 2006). Given these discrepant findings, further research may help enhance understanding of what specific types of instructional supports are particularly important for facilitating student engagement. For example, future studies could more carefully explore and compare the impact of reading comprehension instruction practices in isolation, the impact of 103 reading comprehension instruction in combination with other practices (e.g., incorporating student choice), and the impact of those other instructional practices in isolation. Findings from the present study were an initial attempt to tease apart the impact of reading comprehension instruction and analyze relationships with student engagement more quantitatively; a more experimental approach would continue to advance this area of research by identifying what specific instructional supports might be key to enhancing student engagement. The significant interaction effect of reading comprehension instruction and text readability level on student engagement provides an additional direction for future research. In classrooms where text was easier, students were less engaged when teachers included higher levels of comprehension instruction; in classrooms where text was more difficult, students were more engaged when teachers included high levels of comprehension instruction. This finding seemed to suggest that providing a “just right” balance in the classroom, not overteaching or underteaching so to speak, might facilitate student engagement (e.g., Moley, Bandré, & George, 2011; Schussler, 2009). Exploring what specific comprehension assessment and instruction practices might best strike this balance between challenging and supporting students, thereby promoting engagement, appears to be worth researching further. Lastly, future research should continue to develop and validate tools that measure engagement during a particular class. The author was unable to identify a valid and reliable tool that measured student engagement during a specific class period or learning activity; many tools measure engagement with school in general, such as attendance or participation in activities (Corso et al., 2013; Fredericks et al., 2011). The present study therefore modified verb tense from present to past tense for each item of the EvsD (e.g., “I try hard to do well in class” to “I tried hard to do well in class”). Internal consistency analyses of the modified EvsD revealed 104 acceptable levels of reliability. Future researchers exploring engagement outcomes could develop a new tool or continue to use this modified EvsD as a way to measure student engagement in the classroom, as opposed to school more generally. This might facilitate more empirical investigation of classroom-level dynamics, an area of need for future research as “few scholars include measures of context, needs, and engagement in the same study” (Fredericks et al., 2004; p.80) -- despite a wealth of theoretical and descriptive information connecting student engagement, instructional practices, and other classroom-level factors. Implications for Practice In addition to the aforementioned research implications, the present study has implications for educational practice. As elementary school teachers are increasingly expected to provide content area literacy instruction (e.g., ESSA, CCSS, state-designed social studies assessment), it becomes increasingly important for social studies teachers to consider the extent to which they include comprehension instruction in their classrooms. Moreover, 51% of students were identified as struggling readers in the present study, suggesting that students likely need reading comprehension support to understand grade-level text used during instruction. Although teachers were observed incorporating reading comprehension instruction at high levels, it was rarely direct instruction or modeling of comprehension strategies (i.e., “teacher-centered” practices). These findings, in conjunction with teachers frequently feeling unqualified or underprepared to teach reading strategies (e.g., Cantrell et al., 2009.; Hall, 2005; Ness, 2007), suggest that more training and professional development in comprehension instruction should be considered by school personnel. How teachers are trained in content area instruction might need significant improvement as they enter the field with increasing literacy demands (e.g., CCSS). Current coursework requirements for teachers-in-training, for example, should potentially 105 include more exposure to content area reading instruction and its importance. Teachers could also benefit from more specialized and ongoing support in executing social studies text use in classrooms. For instance, school-based literacy coaches can provide continuous support through school wide initiatives, classroom observations, identification and modeling of appropriate content area instructional strategies, and the like (e.g., Sturtevant, 2003). Findings from the present study suggested reading-related predictor variables (e.g., text level, amount of comprehension instruction, reading achievement) did not have a strong relationship with student engagement. Prior research emphasizes the importance of factors like autonomy-supportive practices in facilitating student engagement; therefore, it may be the most fruitful for teachers to focus on practices like those, even when using a substantial amount of text, if the goal is promoting engagement. An interaction result from the present study did suggest that the level of text and level of reading comprehension support in the classroom may be important to get balanced “just right” to promote engagement. These findings seem to point to the need for comprehension instruction practices that do not overteach or underteach, but rather provide just the right amount of support to ensure student engagement. The literature further indicates just how intentional teachers must be when selecting texts and designing corresponding instruction that both challenges and supports students. Given the current educational landscape, school professionals must think critically about (a) how much exposure students are having to content area literacy and (b) how students are being set up to succeed in the instructional environment during this exposure (e.g., level of text, comprehension support). 106 Conclusions Though students are increasingly expected to be proficient in content area reading, social studies text presents unique challenges to students. Reading comprehension instruction is a beneficial way to support students, particularly struggling readers, when they encounter these texts. Little is known about how these classroom factors might promote or undermine student engagement in social studies learning activities. In this study, fifth-grade teachers frequently supported student reading of social studies texts by providing comprehension instruction. However, this comprehension instruction was largely student-centered practices (collaborative practice, independent practice) and rarely teacher-centered practices such as direct instruction or modeling. The most frequent comprehension strategies were also seemingly simple (e.g., vocabulary definitions, asking comprehension questions, discussions) as opposed to more multi- step or multi-information comprehension strategies (e.g., summarizing, identifying text structure, text annotation). Although findings did not seem to suggest a strong relationship between reading-related predictors and student engagement, these predictors may have an important relationship with other variables such as student achievement outcomes or interest in expository text. A significant interaction effect was identified in this study for text readability level and reading comprehension instruction on student engagement. These findings suggest that a “just right” balance between comprehension instruction support and text level might promote student engagement. It also highlights just how complex the relationship is between classroom factors and student engagement – and this relationship deserves further attention, as understanding its intricacies brings value to scholars and school professionals alike. Given the information provided in the study, school personnel may consider finding ways to continuously train and develop content area literacy skills among teachers. 107 APPENDICES 108 APPENDIX A Student Engagement Survey (EvsD) [RA Script, Introduction to survey: The first thing you’ll be doing today is filling out this short survey. It is asking what you think and how you feel about class. Please be thinking about the social studies class you just had; in other words, I want you to focus only on the XX minutes of social studies class you had today. I will read the items aloud; please follow along and circle the response that indicates how true the statement is for you. The options are: A. Not at all true, B. Not very true, C. Sort of true, and D. Very true. Only circle one option for each statement. Keep in mind there are no right or wrong answers – I just want to know how you think and feel. Your teachers and parents will not see your responses to this survey; only myself and other trained research assistants will see what you circled. Do not put your name on this survey; we want it to remain anonymous. Do any of you have a question about this? Feel free to raise your hand while I read the survey aloud if you have any questions about what something means or what to do.] This survey asks what you think and how you feel about class. Please be thinking about the social studies class you just had; in other words, focus only on the social studies class you had today. A researcher will read the items aloud; please follow along and circle the response that indicates how true the statement is for you. The options are: A. Not at all true, B. Not very true, C. Sort of true, and D. Very true. Only circle one option for each statement. There are no right or wrong answers – we just want to know how you think and feel. Your teachers and parents will not see your responses to this survey; only trained research assistants will see what you circle. Please do not put your name on this survey; we want it to remain anonymous. When you reach the stop sign, please put your pencil down and do not turn the page until you are told to. 1. I tried hard to do well in class. (A) Not at all true 2. I enjoyed learning new things in class. (A) Not at all true 3. When we worked on something in class, I felt discouraged. (A) Not at all true (C) Sort of true 4. In class, I did just enough to get by. (A) Not at all true 5. Class was fun. (A) Not at all true (B) Not very true (B) Not very true (B) Not very true (C) Sort of true (B) Not very true (C) Sort of true (B) Not very true (C) Sort of true (C) Sort of true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true 109 (C) Sort of true (B) Not very true (C) Sort of true (C) Sort of true (C) Sort of true (C) Sort of true (B) Not very true (C) Sort of true (B) Not very true (C) Sort of true (B) Not very true (B) Not very true (B) Not very true 6. In class, I worked as hard as I could. (A) Not at all true 7. When I was in class, I felt bad. (A) Not at all true 8. When I was in class, I listened very carefully. (A) Not at all true 9. When I was in class, I felt worried. (A) Not at all true 10. When we worked on something in class, I got involved. (A) Not at all true 11. When I was in class, I thought about other things. (A) Not at all true 12. When we worked on something in class, I felt interested. (A) Not at all true 13. Class was not all that fun for me. (A) Not at all true 14. When I was in class, I just acted like I was working. (A) Not at all true 15. When I was in class, I felt good. (A) Not at all true 16. When I was in class, my mind wandered. (A) Not at all true 17. When I was in class, I participated in class discussions. (A) Not at all true 18. When we worked on something in class, I felt bored. (A) Not at all true 19. I didn’t try very hard in class. (A) Not at all true 20. I paid attention in class. (A) Not at all true (B) Not very true (B) Not very true (B) Not very true (B) Not very true (B) Not very true (C) Sort of true (C) Sort of true (C) Sort of true (B) Not very true (C) Sort of true (B) Not very true (C) Sort of true (B) Not very true (C) Sort of true (C) Sort of true 110 (B) Not very true (C) Sort of true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true (D) Very true APPENDIX B Group-Administered Reading Assessment (DIBELS Daze) RA Script and Administration Instructions 1. Introduction to DIBELS Daze The second thing you’ll be doing today is taking a 3-minute reading assessment. Your teachers and parents will not see your score, and it will not count toward anything for your class or grade. However, please still try your best on it. Do not put your name on this assessment; we want it to remain anonymous just like the survey you took. 2. Administer DIBELS Daze (5th grade passage 14, John Hancock; Instructions Verbatim): You are going to read a story with some missing words. For each missing word, there will be a box with three words. Circle the word that makes the most sense in the story. Look at Practice 1. Listen. After playing in the dirt, Sam went (pause) home, summer, was (pause) to wash her hands. You should circle the word “home” because “home” makes the most sense in the story. Listen. After playing in the dirt, Sam went home to wash her hands. Now it is your turn. Read Practice 2 silently. When you come to a box, read all the words in the box and circle the word that makes the most sense in the story. When you are done, put your pencil down. Allow up to 30 sec for students to complete the example and put their pencils down. If necessary, after 30 sec say: Put your pencil down. As soon as all students have their pencils down, say: Listen. On her way home, she (pause) chair, sleep, saw (pause) an ice cream truck. You should have circled “saw” because “saw” makes the most sense in the story. Listen. On her way home, she saw an ice cream truck. When I say “begin,” turn the page over and start reading the story silently. When you come to a box, read all the words in the box and circle the word that makes the most sense in the story. Ready? Begin. Start your stopwatch after you say “begin.” Monitor students to ensure they are reading and circling the words. Use the reminders as often as needed. -If a student starts reading the passage aloud, say: Remember to read the story silently. - If a student is not working on the task, say: Remember to circle the word in each box that makes the most sense in the story. -If a student asks you to provide a word for him/her or help with the task, say: Just do your best. At the end of 3 min, stop your stopwatch and say: Stop. Put your pencil down. Collect all of the Daze worksheet packet. 111 112 113 114 115 APPENDIX C Observation Tool Date: Date of observation, MM/DD/YY Observer: Observer records initials here. Classroom #: Randomly assigned classroom number for de-identified data. Time Start: Time observation begins (social studies instruction period begins) Time Stop: Time observation ends (social studies instruction period ends). Topic: Observer records topic(s) covered throughout observation period For each 30-sec interval, record event occurring at any point during interval by circling; focus on what is happening for majority of students for group-level instruction (i.e., not individual) 116 INTERV INSTR. 1 2 0:00- 0:30 0:30- 1:00 FORMAT Whole Class Small Grp Partner Indep READ EXPEC Read Refer None Whole Class Small Grp Partner Indep Read Refer None READ MATER Textbook Worksht Other None Textbook Worksht Other None ...(continued)… 1 2 0 59:30- 60:00 Whole Class Small Grp Partner Indep Read Refer None Textbook Worksht Other None COMP INSTR Yes No Yes No Yes No *if No, don’t code right *if No, don’t code right *if No, don’t code right COMPR. COMPONENT Direct Inst Modeling Guide/Collab Independ Direct Inst Modeling Guide/Collab Independ INSTR. MEDIUM Teacher Peer Tech Ind/Self Teacher Peer Tech Ind/Self Direct Inst Modeling Guide/Collab Independ Teacher Peer Tech Ind/Self ADDITIONAL NOTES/INFORMATION REGARDING OBSERVATION PERIOD: (e.g., things observed, not captured above that may influence findings) READING MATERIALS: (1) Collect copies of all informational text reading materials used during the instructional period -This includes any text on handouts, within textbooks sections, on projectors, etc. (2) Record title and year associated with text used on copies (e.g., textbook, workbook, novel) (3) Record classroom # on front page of any copies Running Checklist: Specific Strategies Circle/check comprehension strategies observed at any point throughout instruction.. **NOTE: Comprehensive definitions and examples of codes above (drawn from prior research, the literature) are included in the research assistant training manual Vocabulary: o Preview Text/Prediction o Comprehension Monitoring o Discussion Activity (Student Discourse/Discussion) o Definitions o Morphology o Context Clues o Prior Knowledge o Building Knowledge o Main Idea o Summary/Relevant Information o Questions o Text Structure o Visual/Graphic Organizer o Text Annotation Comprehension Strategy: Background Knowledge: 117 APPENDIX D Administrator Recruitment Email Subject Line: MSU Research Opportunity: Student Engagement & Literacy Dear NAME OF ADMINISTRATOR, My name is Heather Schmitt, and I am a doctoral candidate in School Psychology at Michigan State University. I am reaching out to see if you might be interested in allowing fifth-grade teachers and students at your school to participate in a study I am conducting on literacy demands in the content areas. Specifically, I am interested in learning more about how to foster student engagement when text is used during instruction. I am looking to observe teachers during one social studies class (~45 minutes), and then administer two brief class wide measures to students (~15 minutes). Each teacher will be compensated for their ~1 hour of time and effort with a $15 Amazon.com gift card. Attached is a permission form with more detailed information about the study and what participation will involve for your teachers and students. If you are willing to let your teachers participate, please sign and return the attached permission form as soon as possible. If you do not wish to participate in the study, please let me know with a brief email response. Note that by providing your permission, you are just allowing me to contact your fifth-grade teachers about the opportunity -- you are in no way committing teachers to participate in the project. Please do not hesitate to reach out with any questions or concerns you may have. Thank you for your consideration and time! Heather Schmitt, M.A. 118 APPENDIX E Teacher Recruitment (Email 1) Dear NAME OF TEACHER, My name is Heather Schmitt, and I am a graduate student in School Psychology at Michigan State University. I am completing my doctoral requirements by examining social studies instruction and associated reading expectations in fifth grade classrooms. You are being invited to participate in this study because NAME OF ADMINISTRATOR has provided permission and nominated you as potential participants. Your participation is completely voluntary and will involve approximately 1 to 1.5 hours of your time (~1 hour classroom observation, brief demographic survey, assistance distributing forms to parents). As a token of my appreciation for your time and effort, you will receive a $15 Amazon.com gift card for participating. Attached is a consent form with more detailed information about the study and what your participation will involve. If you would like to participate in the study, please sign and return the consent form as soon as possible. If you do not wish to participate in the study, just let me know with a brief email response. Please do not hesitate to reach out with any questions or concerns you may have. Thank you for your consideration and time! Heather Schmitt, M.A. 119 APPENDIX F Teacher Consent Form Research Participant Information and Consent Form You are being asked to participate in a research study. Researchers are required to provide a consent form to inform you about the research study, to convey that participation is voluntary, to explain risks and benefits of participation, and to empower you to make an informed decision. You should feel free to ask the researchers any questions you may have. Study Title: Upper Elementary Social Studies Instruction: Exploring the Relationship Between Text Use, Comprehension Instruction, and Student Engagement Researcher and Title: Dr. Sara E. Witmer (formerly Sara E. Bolt), Associate Professor of School Psychology and Heather A. Schmitt, M.A. Department and Institution: Department of Counseling, Educational Psychology, and Special Education, Michigan State University Address and Contact Information: 620 Farm Lane, Rm 434 Michigan State University East Lansing, MI 48824-1034 (517)599-8917, schmi538@msu.edu 1. PURPOSE OF RESEARCH • You are being asked to participate in a research study on social studies instruction in fifth grade classrooms. In the entire study, 50 teachers are being asked to participate. • • From this study, the researchers hope to learn more about reading expectations and instructional practices during social studies. Findings from this student have the potential to inform how students might be supported in accessing instructional materials and therefore be more engaged. • Your participation in this study will take approximately 1.5 total hours. 2. WHAT YOU WILL DO As a participant in the research study, you will complete the following: • You will complete a brief demographic online survey (10 minutes). • You will provide parents of all students in your class information about the study and a consent form through which they permit their child’s participation. You will also provide parents a reminder three days after the initial communication (30 minutes). • You will coordinate a social studies instructional period during which a trained research assistant can observe classroom activities. This observation session will last approximately 45 minutes, and will not focus on any particular student(s) in your class. You will also need to provide copies of text materials used during the observed lesson that the research assistant requests (45 minutes). • You will coordinate a 10-minute period immediately following the observation during which students in your class will take a brief survey and reading assessment. These will be facilitated by the trained research assistant and administered at the class wide level to all students for whom parents did not provide consent (10 minutes). 3. POTENTIAL BENEFITS • You will not directly benefit from your participation in this study. However, your participation in this study may contribute to the understanding of reading expectations and supports in social studies. 120 4. POTENTIAL RISKS those that occur during a typical school day. 5. PRIVACY AND CONFIDENTIALITY • There are no foreseeable risks associated with participation in this study, apart from • The data for this project are being kept strictly confidential. Your school administrators, students and their parents will not be given any personally identifying information. • Only researchers, trained research assistants and the Institutional Review Board (IRB) will have access to your demographic survey responses, school day observation notes, student survey responses, and student assessment responses. • The researcher and trained research assistants will collect the data at your school in a way such that your name and any identifying information will not be associated with the observation and assessment data that are collected. Your name will only appear on this consent form, and student names will only appear on their assent forms. • Although we will make every effort to keep your data confidential there are certain times, such as a court order, where we may have to disclose your data. • All related information (i.e., consent forms, data, etc.) will be kept in separate locked files and on computers that are password protected. These steps will be taken so that no one outside of the research team and those involved in promoting appropriate research practices at the university can have access to information that might link your name with the associated data. • The results of this study may be published or presented at professional meetings, but your identity and those of all research participants will remain anonymous. 6. YOUR RIGHTS TO PARTICIPATE, SAY NO, OR WITHDRAW • Participation is completely voluntary. Refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled. You may discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. • You have the right to say no. • You may change your mind at any time and withdraw. • You may choose not to answer specific questions or to stop participating at any time. 7. COSTS AND COMPENSATION FOR BEING IN THE STUDY • You will receive a $15 Amazon.com gift card after completion of the study (i.e., following the observation session) as a token of our appreciation for your time and effort. • Withdrawal prior to completion of any given part study will result in no compensation. 8. CONTACT INFORMATION If you have concerns or questions about this study, such as scientific issues, how to do any part of it, or to report an injury, please contact the study coordinator Heather Schmitt: 620 Farm Lane, Rm. 434 Michigan State University East Lansing, MI 48824-1034 (517)599-8917, schmi538@msu.edu If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at Olds Hall, 408 West Circle Drive #207, MSU, East Lansing, MI 48824. 121 9. DOCUMENTATION OF INFORMED CONSENT Your signature below means that you voluntarily agree to participate in this research study. ________________________________________ ___________________________ Signature *Please either mail a signed copy of this form to the researcher’s address above (i.e., 620 Farm Lane, Rm 434, Michigan State University, East Lansing, MI 48824) or scan and email this form to schmi538@msu.edu. You will be given a copy of this form to keep. Date 122 Teacher Demographic Survey and Observation Scheduling APPENDIX G (Email 2) Dear NAME OF TEACHER, Thank you once again for your willingness to participate in my research study. I appreciate your time and effort in advance! The next steps are to: (a) complete a brief online demographic survey and (b) schedule the observation session. These two procedures are described below. (a) Please complete a short survey with some background information about yourself by clicking this link. (b) Reply to this email indicating the days/times at least two weeks out that we can come observe social studies instruction in your classroom. We are looking to observe a typical day of instruction; please exclude “non-typical” school days (e.g., half days, assemblies) and exclude days devoted to activities not focused on classroom instruction (e.g., tests/exams, field trips, watching an entire movie). We are also specifically interested in observing a day/time where instruction will include a substantial amount of text use given the study’s focus. Please keep these two things is mind when selecting potential observation days. Feel free to let me know if you have any questions or concerns about the above! My best, Heather Schmitt ------ TEACHER DEMOGRAPHIC SURVEY (online via Qualtrics) Please provide some background information about yourself by answering the following questions. 1. Gender o Male o Female o Other 2. Race/Ethnicity o Asian/Pacific Islander (Asian American) o African American Non-Hispanic o American Indian/Alaskan Native o White Non-Hispanic o Chicano/Mexican-American 123 o Hispanic o Other (please specify) 3. Highest level of education completed o Bachelors/College Degree o Master’s Degree o Doctoral Degree o Other (please specify) 4a. Teaching endorsements held (please indicate any specific codes/endorsements for each area) o Language Arts o Mathematics o Science o Social Studies o Special Education o World Language and Culture o Arts o Business Education o Technology Education o Health, Physical Education, Recreation, and Dance o Bilingual Education o Other (please specify) 4b. Please indicated any specific codes/endorsements for each area selected above by typing into the text box below 5. Total years of teaching experience (type number into text box below) 6. Grade level(s) currently teaching (select all that apply) o 5th grade o Other (please specify) 7. School location o Urban o Suburban o Rural 8. Percentage of students in your CLASS receiving special education services (please put percentage in box below) 9. Percentage of students in your SCHOOL receiving free and reduced-price lunch (please put percentage in box below) 124 APPENDIX H Teacher Instructions for Parent Information (Email 3) Dear NAME OF TEACHER, Now that we have an observation date scheduled, the next step will be to provide parents of students in your class the chance to “opt out” of participation in the study. Please send each parent the cover letter attached to this email (which has been personalized with your name), as well as a copy of the attached form. Note that parents will contact me directly if they do not want their child to participate in the study (i.e., “opt out”). If the majority of students in your class opt out, we will be unable to proceed with your participation in the study. If it is customary for your classroom and the school to communicate through email with parents (e.g., standard forms are sent back and forth between home and school electronically), you are welcome to complete this correspondence through email. If it is not customary for your classroom and the school to communicate through email with parents, or there are parents who are not regularly on email, please send hard copies of the cover letter and forms home with students. Three days after sending this information home, you will need to send the parents a brief reminder about the study and the opt out due date. Please watch your email closely for a message with instructions and a template for this in the coming days. Thank you for your assistance! Please let me know if you have any questions or concerns. Heather Schmitt, M.A. 125 APPENDIX I Parent Cover Letter Dear Parent(s), I am contacting you to find out if you are willing to let your child participate in a study that is gathering information about teacher practices and reading expectations during social studies. With your child’s participation, the researchers hope to contribute to knowledge about what might help students access instruction and be more engaged in the classroom. The study is being conducted by a researcher from Michigan State University (MSU), who will be observing one of my social studies lessons (focusing on what teachers do and what materials are used). At the end of the observed lesson, students in the class will take a brief survey and 3- minute reading assessment. These two activities will be administered at the class wide level (not individually). Students will not record their names or any identifying information the forms. You child’s participation in the study will take approximately ten minutes and information collected will remain completely anonymous. More details are included in the attached information form. I have already given my permission for the MSU researches to observe my instruction on DATE at TIME; however, your permission is needed for your child to do two short activities described above. If you are interested in having your child participate, please keep these forms for your own reference. You do not need to return the form or do anything else. If you do not want your child to participate, email the study coordinator directly at schmi538@msu.edu by DATE. Indicate that you would like to “opt out” of the study, and include your child’s name, my name (teacher), and the name of our school. If you have any questions, please contact the study coordinator Heather Schmitt by email (schmi538@msu.edu) or phone (517-599-8917). Thanks for your consideration! NAME OF TEACHER 126 APPENDIX J Parent Information Form You are being asked to allow your child to participate in a research study. Researchers are required to inform you about the research study, to convey that participation is voluntary, to explain risks and benefits of participation, and to empower you to make an informed decision. You should feel free to ask the researchers any questions you may have. Study Title: Upper Elementary Social Studies Instruction: Exploring the Relationship Between Text Use, Comprehension Instruction, and Student Engagement Researcher and Title: Dr. Sara E. Witmer (formerly Sara E. Bolt), Associate Professor of School Psychology and Heather A. Schmitt, M.A. Department and Institution: Department of Counseling, Educational Psychology, and Special Education, Michigan State University Address and Contact Information: 620 Farm Lane, Rm 434, Michigan State University , East Lansing, MI 4882; schmi538@msu.edu 1. PURPOSE OF RESEARCH • You are being asked to allow your child participate in a research study on social studies instruction in fifth grade classrooms. • Your child has been selected as a possible participant in this study because he or she is a fifth-grade student whose teacher has agreed to be involved in the study. • From this study, the researchers hope to learn more about reading expectations and instructional practices during social studies. Findings from this study have the potential to inform how students might be supported in accessing instruction and therefore be more engaged. • Your child’s participation in this study will take approximately 10 minutes. 2. WHAT YOUR CHILD WILL DO As a participant in the research study, your child will complete the following: -A 5 minute survey on engagement, read aloud to the entire class by a trained research assistant. -A 3 minute reading assessment, completed at the same times as all of his/her classmates. 3. POTENTIAL BENEFITS • You and your child will not directly benefit from participation in this study. However, participation in this study may contribute to the understanding of reading expectations, supports, and how to engage students during social studies. 4. POTENTIAL RISKS 5. PRIVACY AND CONFIDENTIALITY • There are no foreseeable risks for you or your child associated with participation in this study, apart from those that occur during a typical school day. • The data for this project are being kept strictly confidential. School administrators, students, and parents will not be given any personally identifying information. • Only researchers, trained research assistants, and the Institutional Review Board (IRB) will have access to school day observation notes, student survey responses, and student assessment responses. 127 • The researcher and trained research assistants will collect the data at your child’s school in a way such that your child’s name and any identifying information will not be associated with the assessment data collected. Your child’s name will only appear on their assent forms. • Neither you or your child’s teacher will have access to your child’s data. • All related information (i.e., forms, data, etc.) will be kept in separate locked files and on computers that are password protected. These steps will be taken so that no one outside of the research team and those involved in promoting appropriate research practices at the university can have access to information that might link your name to the project. • The results of this study may be published or presented at professional meetings, but your identity and those of all research participants will remain anonymous. 6. YOUR RIGHTS TO PARTICIPATE, SAY NO, OR WITHDRAW • Participation is completely voluntary. Refusal to participate will involve no penalty or loss of benefits to which you and your child are otherwise entitled. You or your child may discontinue participation at any time without penalty or loss of benefits to which you or your child are otherwise entitled. • You have the right to say no to your child’s participation. • You may change your mind at any time and withdraw. • You may choose not to answer specific questions or to stop participating at any time. 7. COSTS AND COMPENSATION FOR BEING IN THE STUDY • There are no costs that will be incurred as a result of participating in this study. 8. CONTACT INFORMATION If you have concerns or questions about this study, such as scientific issues, how to do any part of it, or to report an injury, please contact the study coordinator, Heather Schmitt: 620 Farm Lane, Rm. 434 Michigan State University East Lansing, MI 48824-1034 (517)599-8917, schmi538@msu.edu If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at Olds Hall, 408 West Circle Drive #207, MSU, East Lansing, MI 48824. 9. DOCUMENTATION If you permit your child to participate in the study, you may keep this form for your records (no further action is needed). If you do not want your child to participate in the study, please email the study coordinator Heather Schmitt (schmi538@msu.edu) indicating that you do not want your child to participate. 128 APPENDIX K Parent Reminder 129 Dear Parent(s), I just wanted to remind you about the opportunity for your child to participate in the Michigan State University research study on teacher practices and reading expectations during social studies. I have already given my permission for the MSU researchers to observe my instruction during a social studies lesson; however, I want to make sure you are willing to let your child to do two short class wide activities (lasting approximately 10 minutes). Below is a copy of the information that was passed along prior. If you are interested in having your child participate, just keep this information for your own reference. You do not need to return a form or do anything else If you do not want your child to participate, email the study coordinator directly at schmi538@msu.edu by DATE. Indicate that you would like to “opt out” of the study, and include your child’s name, my name (teacher), and the name of our school. If you have any questions, please contact the study coordinator Heather Schmitt by email (schmi538@msu.edu) or phone (517-599-8917). Thanks again for your consideration! TEACHER NAME --------------- I am contacting you to find out if you are willing to let your child participate in a study that is gathering information about teacher practices and reading expectations during social studies. With your child’s participation, the researchers hope to contribute to knowledge about what might help students access instruction and be more engaged in the classroom. The study is being conducted by researchers from Michigan State University (MSU), who will be observing one of my social studies lessons (focusing on what teachers do and what materials are used). At the end of the observed lesson, students in the class will take a brief survey and 3- minute reading assessment. These two activities will be administered at the class wide level (not individually), and students will not record their names or any identifying information the forms. You child’s participation in the study will take about ten minutes. More details are included in the attached information form. I have already given my permission for the MSU researches to observe my instruction during a social studies lesson; however, your permission is needed for your child to do two short activities described above. 1. Overview of study – p. 2 a. Brief abstract, research questions 2. Explanation of data collection – p. 4 3. Definitions for checklist – p. 6 a. Overview of data collection responsibilities a. Includes operational definitions of each comprehension strategy on running checklist 4. Definitions for observation –p. 8 a. Includes operational definitions of each variable coded on observation form 5. Master grid of definitions – p. 10 a. Examples for every comprehension strategy and model component combination variable/code 6. Pencil/paper applied test, applying definitions to example – p.14 a. Must pass with 100% 7. Application to training videos of social studies instruction – p.16 a. Minimum of 2 videos; agreement must reach at least 0.7 for each 8. Data Collection Procedures – p.17 a. Fidelity Checklist to guide each data collection session APPENDIX L Research Assistant Training Manual Training Manual Table of Contents 130 1. Overview of Study Despite increasing expectations for student proficiency in content area reading (e.g., Common Core State Standards), social studies text continues to present unique challenges to students who struggle with reading; for example, textbook readability is often at least two grade levels above its intended use (Berkeley et al., 2016; Jitendra et al., 2001). Given the shift toward greater use of demanding informational text around fourth grade, students who struggle with reading stand to benefit from direct teacher support in reading and comprehending social studies texts. Research suggests that one such way to support student learning in the content areas is comprehension strategy instruction, which might include practices like direct instruction, modeling, collaborative strategy use, guided practice, and independent strategy use (e.g., Duke & Pearson, 2009; Vacca, Vacca & Mraz, 2016). Moreover, it is important to consider the ways in which classroom factors like these might promote or undermine student engagement in social studies learning activities (Skinner & Pitzer, 2012). The purpose of this study is to systematically observe the use of text materials during fifth grade social studies instruction and investigate the extent to which teachers support student reading of those texts through comprehension instruction. This study will further explore the relationship between these classroom-level factors and student engagement, including a closer examination of these relationships among struggling readers. RQ 1. What is the readability level of text materials used during social studies instruction in fifth grade classrooms? RQ 2. How frequently does fifth grade social studies instruction include at least one reading comprehension instructional component (proportion of intervals)? How often is each type of comprehension instructional component being observed? For RQ3, RQ4, and RQ5 Level 1 predictors refer to student-level individual variables, while Level 2 predictors refer to classroom-level variables (i.e., those with shared values among students in the same classroom). The figure at the end of this section depicts the relationships between variables that will be examined through these three research questions. RQ 3. To what extent does student reading achievement level (Level 1) predict student engagement during fifth grade social studies instruction among all students? RQ 4. To what extent do classroom-level reading factors (Level 2) predict student engagement during fifth grade social studies instruction? More specifically: 4a. To what extent does text readability level (Level 2) predict student engagement among all students? Among struggling readers? 4a Hypothesis: Higher text levels will predict lower student engagement, particularly among struggling readers. 4b. To what extent does the provision of reading comprehension instruction (Level 2) predict student engagement among all students? Among struggling readers? 4b Hypothesis: Higher provision of reading comprehension instruction will predict higher student engagement, particularly among struggling readers. RQ 5. To what extent does frequency of reading comprehension instruction (proportion of intervals) moderate the relationship between student- and classroom-level reading factors and student engagement? More specifically: 131 5a. To what extent does frequency of reading comprehension instruction moderate the relationship between student reading achievement level (Level 1 predictor) and student engagement among all students? Among struggling readers? 5a Hypothesis: As the value of reading comprehension instruction increases (moderator), the positive relationship between student reading achievement and engagement decreases (i.e., becomes weaker). This moderating relationship is expected to be particularly strong for struggling readers. 5b. To what extent does frequency of reading comprehension instruction moderate the relationship between text readability grade level (Level 2 predictor) and student engagement among all students? Among struggling readers? 5b Hypothesis: As the value of reading comprehension instruction increases (moderator), the negative relationship between text readability grade level and engagement decreases (i.e., becomes weaker). This moderating relationship is expected to be particularly strong for struggling readers, who have lower levels of reading achievement. 132 2. Explanation of Data Collection o Non-examples • Will be used to examine instructional practices during social studies lessons o Attending to what is happening for the majority of students; group level o Examples • Teacher is providing whole-class instruction on using a graphic organizer as students follow along with their own copies of the textbook chapter. • Teacher prompts all students to engage in structured partner discussion about a historical novel they read. • Teacher is working individually with a student to explain how to use a graphic organizer while the rest of the class is discussing information from their textbook. • Teacher is providing instruction to a group of four students on using prediction while the rest of the class is watching a movie. • Teacher is assisting an individual student at his or her desk with defining a word while the rest of the class is writing a short story. • There is no text in front of students and the teacher is reading text aloud (this would be listening comprehension; a different focus from the present study’s reading comprehension) o We will use interval coding procedures, marking whether each code is evident at any point during the interval Therefore, you code whatever happens at any point during the given interval…you do NOT pick only the one that happens the majority of the interval and you do NOT only code occurrences at the start or end It is likely that some intervals will have multiple things marked because several different things happen during that 30 second interval i.e., If it happens at any point during the interval, you code it • Prior to data collection, teachers are asked to select a period of “typical” social studies instruction, and a day/time where there will be a substantial amount of text used as part of instruction o “Typical” meaning a true instructional period; for example, not a special day watching a historical movie or a day spent taking an exam o Substantial text use so we can observe the factors of interest in the present study • All data will be collected in a completely de-identified manner o Each school will be randomly assigned a two-digit number E.g., School A will be 12, School B will be 86 o Each classroom will also be assigned a random two-digit number E.g., Classroom 1 in School A will be 44, Classroom 2 in School A will be 04 o Each student will also be assigned a random two-digit number E.g., Student 1 in Classroom 1 will be 22, Student 1 in Classroom 2 will be 75 § § § § § § 133 § § § § § The RA is responsible for the following activities during each data collection session: o The format will be six digits, with the first two representing the school, the second two representing the classroom, and the third two the student Based on the above example, Student 1 in Classroom 1 will be 124401 • School A = 12, Classroom 1 = 44, Student 1 = 01 Based on the above example, Student 15 in Classroom 2 will be 120415 • School A = 12, Classroom 2 = 04, Student 15 = 15 o So, most materials will use 4-digit number; student-level packet/materials will use 6-digit number. Heather will create/assign these. • Will observe n=50 fifth grade teachers during one social studies lesson o Of the 50 observations, one-third will need to have two observers present for double coding (i.e., 16 observations, randomly assigned) RA 1, Justina: 11 independent, 6 with RA 2, 5 with RA 3 (22 total) RA 2, Courtney D: 11 independent, 6 with RA 1, 5 with RA3 (22 total) RA 3, Courtney C: 12 independent, 5 with RA 1, 5 with RA 2 (22 total) o Use clipboard with observation form/checklist, and vibrate or flash interval timer o For each 30 sec interval, code events occurring at any point during interval o Throughout observation, record specific comprehension strategies teacher uses via running checklist (i.e., check if used at any point during entire observation) o Definitions for each variable/code within form on next page(s) o (a) get student assent (read aloud, have students sign and tear off page, turn in) o (b) administer class-wide student engagement survey (Engagement vs Disaffection) by reading aloud instructions/items to students o (c) administer class-wide reading measure (DIBELS Daze) by reading aloud instructions, timing for 3 min, then collecting completed student packets o After collecting student data, RA will gather copies of all text materials utilized during instruction Teacher is prompted to have this ready for you to collect ahead of time, but you should be provided access to a copier and the original materials if needed. Also record title and year of any text material on the hard copy. o Have teacher complete two Likert items and provide his or her email address • 1) OBSERVATION (~ 1 hour) • 2) COLLECTING STUDENT DATA (~10 min) • 3) DEBRIEF WITH TEACHER (~5 min) Immediately following each observation, RA will distribute the Student Packet and: § § § In general, how similar were today’s instructional practices to how you typically teach with text during social studies? In general, how similar was the amount of text used today to the amount you typically use during social studies? 3. Definitions for Running Checklist of Specific Strategies Definitions are in order of the checklist. Remember, this is a running checklist and will be marked if it occurs at any point during instruction. In other words, if you check “yes” to 134 o COMPREHENSION MONITORING § § § § § § § § § § § o DISCUSSION ACTIVITY (STUDENT DISCOURSE/DISCUSSION) comprehension instruction being present, you would mark one of these specific strategies (unless that strategy had already been used, in which case it would already be checked). o PREVIEW TEXT/PREDICTION § § • Reviewing/examining key elements before reading – e.g., title, headings, graphics of text, beginning and ending sentences (with or without teacher guidance) How title, headings, text graphics relate to upcoming content As begin to read, make predictions about outcome of events in a story or passage; what is going to happen next Making inferences: speculating about characters or actions not clearly stated in text Self-assessment of ones understanding of what is read; when one does not understand, use of a strategy or procedures to clarify understanding “thinking about thinking”; awareness of ongoing comprehension and cognitive processes during reading • e.g., “click or clunk”; rereading, “fix-up” strategy use Teacher facilitates extended, meaningful discourse regarding text material Students engage in discourse with one another regarding text material Teacher asks higher order questions, asks students to justify their responses, encourages students to elaborate regarding text material • e.g., discussion in which students listen and link to others’ ideas, provide evidence from the text to support thinking, and regularly participate Definitions provided to introduce or review vocabulary word meaning Examples and nonexamples provided to explain or review vocabulary Instruction in the use of morphology (word structure and parts, e.g., root, prefix) to understand word meanings Prompting or cuing students to use morphology to understand word meanings (rather than give answer) Instruction in the use of context clue strategies to understand word meanings (e.g., surrounding words/sentences) Prompting or cuing students to use context clues to understand word meaning (rather than give answer) Before reading, students participate in activities to measure their level of knowledge Students connect prior knowledge/what is already known to text (e.g., theme, topic) VOCABULARY o DEFINITIONS § § MORPHOLOGY § o o CONTEXT CLUES BACKGROUND KNOWLEDGE o PRIOR KNOWLEDGE 135 o BUILDING KNOWLEDGE § § § § § § § READING COMPREHENSION STRATEGY Explicit instruction, modeling, prompting, cuing or think aloud in: o MAIN IDEA o SUMMARY/RELEVANT INFORMATION § § Before reading, students participate in activities to build their level of knowledge Teacher provides new information through a story, video, or explanation to build background knowledge Identifying/finding the main idea in text (leaving out details) Focusing on relevant information in the text (removing redundancy); summarizing the text with main ideas from beginning, middle, end e.g., condensing episodes that led to major historical event Generating or answering questions about text before, during or after reading • e.g., QARs, Question-Answer-Relationships Identifying elements of text organization or structure • e.g., compare-contrast, cause-effect, problems/solution statements, sequence, description Visual representations of text information that helps readers understand, organize, and remember information e.g., flowchart, graphic organizer, Venn diagram, conceptual map/chart Marking the text while reading with annotation symbols or notes that identify and represent text components such as key concepts, questions, transition points, etc. • E.g., rectangle around vocabulary, question mark with confusing information, underline main idea, writing summary in margins • E.g., use of sticky notes to record annotations, marking text directly § o VISUAL/GRAPHIC ORGANIZER o QUESTIONS o TEXT STRUCTURE o TEXT ANNOTATION 136 4. Definitions for Observation Form Definition for each code is in order of how each 30 sec interval is coded (i.e., left to right on the form). Remember we use interval coding procedures, so anything observed at any point during an interval gets marked just once. Therefore, codes are not mutually exclusive -- you may mark more than one instructional format during any single interval, more than one type of reading expectation, more than one comprehension component, etc. o INSTRUCTIONAL FORMAT § Whole Class: Instructional activity at the classwide level (e.g., teacher lecturing, discussion with entire class facilitated by teacher) § Small Group: Instructional activity at the small group level (i.e., 3 or more students grouped together) § Partner: Instructional activity at the partner level (i.e., 2 students together) Independent: Instructional activity at the independent, individual student level (e.g., independent reading, worksheet at seat independently, etc.) o READING EXPECTATION § Reading Text: Instructional demand for students to read text directly (i.e., word- for-word decoding) § Referencing Text: Instructional demand for students to reference text that has been read prior, but not to read text directly (e.g., textbook is open to page, students are told to discuss that section from yesterday with a partner) § None: No instructional demand for students involving text; no text is being read directly or referenced by students (i.e., no text materials are present for student) o READING MATERIAL § Textbook: Text read or referenced is within a textbook § Worksheet: Text read or referenced is in worksheet format § Other: Text is read or referenced that is not textbook or worksheet (e.g., novel, essay) § None: No text materials are present for students to read or reference o COMPREHENSION INSTRUCTION § Yes: Instructional activities include one of the reading comprehension strategies (see checklist) § No: Instructional activities do not include one of the reading comprehension strategies from the checklist o COMPREHENSION COMPONENT Indicate which comprehension instruction model component is used to teach the strategy: § Direct Instruction: An explicit description of a comprehension strategy and when/how it should be used (without demonstration/use of strategy) § Modeling (I do): Modeling/demonstrating the strategy in action (e.g., think-aloud of ones’ thoughts while performing task); often verbalizing behavior as demonstrating skill while students observe and follow along; “I do it” (teacher and/or student modeling) IF “No” coded for comprehension instruction, final two variables in interval DO NOT get coded. § 137 § Guided Practice/Collaborative (We do): Collaborative use of the strategy in action; both teacher and student participate in using the strategy and “do it together” OR Taking turns using the strategy in action, interactive as teacher guides/gives direction on students’ strategy use (e.g., prompt, questions, feedback, “coaching”) prompts…teacher provides feedback and engages in discussion e.g., choral responding if class wide, verbally respond in unison to teacher Independent (You do): Independent use of the strategy; opportunity for students to use the strategy on their own per teacher directions/instructional expectations e.g., student by him or herself, students collaboratively using strategy without teacher providing direct instruction, modeling, etc. o INSTRUCTIONAL MEDIUM Indicate through which medium the comprehension strategy instruction is provided: § Teacher: Teacher provides comprehension instruction/support when text is present § Peer: Peer(s) provides comprehension instruction/support through a structured opportunity to collaborate e.g. partner reading, reciprocal teaching, think-pair-share, peer discussion § Technology: Technology provides comprehension instruction/support Indep/Self: Student him/herself provides comprehension instruction/support (e.g., independent worksheet) o ADDITIONAL NOTES/INFORMATION REGARDING OBSERVATION PERIOD Place to note anything observed that may not be captured in the observation tool and seems important to capture/may influence findings § Example: Throughout the class period limited text is actually used (despite requesting a period with substantial text use from teachers) § Example: Teacher frequently uses video clips during the lesson which appears to be very engaging for students § Example: Teacher often reads text aloud, but students do not have a copy of the text, only expected to listen (i.e., listening comprehension) § Example: Students often take turns reading text aloud at the whole class level (i.e., decoding support, but not comprehension) REMEMBER: These are only being coded when they relate to a reading comprehension strategy; for instance, use of technology or peers in other ways does NOT get coded § § § 138 5. Master Grid of Definitions: Strategy/Model Component Examples Table 12 Master Grid of Definitions: Strategy/Model Component Examples Direct Instr. Modeling “I do it” Preview Text/ Prediction Compreh. Monitoring “Predicting is making guesses about what will come next in the text when you are reading. You should make predictions a lot when you read.” “There are things we can do to help us monitor what we understand and what we don’t when we are reading. One of these is called using a “fix-up” strategy. "I’m going to make predictions while I read this book. I’ll start with the cover here. Hm ... I see a picture of an owl. It looks like the owl is wearing pajamas and carrying a candle. I predict this is going to be a make-believe story because owls don’t really wear pajamas or carry candles.” “Now that I reached the end of a paragraph, I stop and think about what I just read. Hm, I don’t really understand what this was about. I can use one of my fix-up strategies, so first I’m going to try re-read the paragraph aloud.” Discussion Activity/ Student Discourse N/A “It can be helpful to spend time talking about what we are reading, especially when the text is challenging or when we might have different ideas or thoughts from another person.” 139 Guided/Collaborative “We do it together”/ “You do it, with assistance” (teacher might prompt, question, feedback, etc.) à"I’ve made some good predictions so far. From this part on I want you to make predictions with me. We’ll each stop and think about what might happen next…now let's hear what you think and why." à“We are going to work on making predictions while we read. After every few pages I will ask each of you to stop and make a prediction. We will talk about your predictions and then read on to see if they come true." à“We just finished reading the 2nd paragraph, so let’s stop. Do we understand what this is about? I don’t think I do, because there were new and difficult words. Let’s use a fix-up strategy…what’s one we can use here?....Yes, let’s look up what some of these words mean that we don’t know.” à“I’ve placed some sticky notes throughout this passage. Whenever you come to one, use our fix- up steps and check if you understand what you’re reading. Place a check mark on the post-it note once you’ve done this, and then keep reading until you reach the next sticky note.” à“Let’s take a moment to really think about the two arguments our book is presenting. On one side, we have pro-slavery and on another anti- slavery. What seemed to be one main argument for those who were pro-slavery?....Anti- slavery?...That’s an interesting point. How might a pro-slavery/anti-slavery person respond to that notion?”… Let’s brainstorm and discuss at least 2 arguments from each side that our books touched on.” Independent “You do it on your own” "It is time for silent reading. As you read, remember what we’ve been working on - making predictions while we read. Be sure to make predictions every 2 or 3 pages.” “Now I want you to read the rest of this chapter and use the fix-up strategy. Place your own post-it whenever you stop and ask yourself if you understand what you’re reading.” “With the person next to you, look back in the chapter we read and find at least 2 more arguments that are pro- or anti- slavery. Discuss what they mean, and how someone from the other group might counter. You will each share out one.” Table 12 (cont’d) Definitions Direct Instr. “When we don’t know what a word means, it is helpful to look up its definition so we can better understand what we are reading about.” Morphology “I’ve never heard the word transcontinental before. By breaking it down into its word parts, I might be able to figure out what it means.” Context Clues “When we aren’t sure what a word means, we can use other known words in the sentence(s) nearby to help us figure it out.” Modeling (I Do) “I see this bolded vocabulary word in the textbook. I wonder what it means, so I’m going to look up the definition in the glossary in the back.” “Transcontinental has the prefix trans-, which we learned a while ago means across. The last part continental has the word continent…so this word might have to do with going across a continent.” “Here it says that the Gettysburg address is a perfect example of Lincoln’s eloquence, as it remains one of the greatest speeches in history. I don’t know what eloquence means, but by looking at the words before and after I see the perfect and greatest also used to describe it. This makes me think it’s positive or good.” Guided/Collaborative (We Do) à“Wait, here’s a new word. I don’t know what table means when talking about Congress. Do you?...Well, then let’s look up this word in the dictionary so you can tell me what it means.” à“While reading this section, we are going to pay special attention to any words we don’t know. I’ll ask one of you to help with any new words we come to by finding the definition and sharing it with the class. Then, once we know what that word means, we will continue reading.” à“We just finished reading this paragraph; does anyone know what nonpartisan means? I don’t either, so let’s try to break it down together into its parts…” à“Here’s a new and difficult word, centurion. Take a moment to break it down by writing it on your paper and underlining the parts. What did you put, Rhonda? Good, that’s the first part. What would be another part, though?...” à“And here’s another new word they use to talk about the speech – brevity. None of us know what that means, so let’s look for clues in the surrounding words. Here are two surrounding words that could help – which do you think we should look at first?...” à“It says that Lincoln’s speech did not garner much attention at first. Hm, garner is a new word. Who would like to share a context clue that they think could help us figure out what it means?...Yes, that could be helpful but it’s still a little unclear. Does anyone see a word that might be more of a clue…?” 140 Independent (You Do) “As you read your textbook, don’t forget to look up the definitions for any of the bolded vocabulary words. Write them down and be sure to do this with any other words you might not know while reading.” “While reading this chapter with your table, you’ll find bolded new words. Instead of looking up definitions, write down the word you don’t know and analyze its word parts like we did last time.” “Now, I want you to read this old speech. It contains a lot of sayings and words that we don’t use very much today. I’ve underlined the ones I want you to focus on, Try your best to use the surrounding words to guess what they mean; highlight the surrounding words that were most helpful.” Table 12 (cont’d) Prior Knowledge Direct Instr. “It can be helpful to think about what we might already know about Ellis Island before we read this chapter about it.” Modeling (I Do) “Based on the next heading, it looks like we are going to read about Ellis Island. I’m going to pause and make a list of three things I’ve heard or might know about Ellis Island before I read.” Independent (You Do) “Today you’re going to read Chapter 3 on Ellis Island quietly with your partner. Make sure you stop at each heading and write down something one of you already knows about that topic.” “Today you’re going to read Chapter 3 on Ellis Island quietly with your partner. Make sure you write down at least two things that you come across while reading that might be helpful to learn more about before reading Chapter 4.” “Go ahead and read the rest of this passage on Harriet Tubman. But, make sure you stop at the end of each paragraph and write down the main idea. There are 8 paragraphs left, so you should have 8 main idea statements.” Guided/Collaborative (We Do) à“Based on the title of this chapter, it looks like we are going to read about Ellis Island today. Let’s go around the room and each say one thing we know about Ellis Island before we keep reading. I know Ellis Island is on the east coast, so I’ll write that on the board. Now Jade, what’s something you know?” à"We are going to work on thinking about what we already know about a topic before we read about it. Whenever we get to a new section, I will stop and ask you to write down something you already know about it.” à“Since Ellis Island is only mentioned briefly, let’s learn more about it so we can better understand this story. Everyone take a moment to look up some information online. Then, we can share what we found to help us build some background knowledge together.” à“As we read the second chapter of this book, I’m going to ask every few pages what topic might be helpful to learn more about. What would be good to know more about before you read the third chapter?..” “Okay, while that is very interesting, do you think it would help us as we continue reading? Why?” à“For these next few paragraphs, I want you to figure out the main idea with me. We’ll stop and see if there are any signal words to help us find it, and get rid of any details...I see a lot of cause and effect words, which do you see?” à“Here’s a paragraph that I’m not sure on the main idea. Take a moment to cross of supporting details, and then I’ll have you share what kind of words you’re left with….” 141 Building Knowledge Main Idea “It might be helpful to learn more about Ellis Island, since the main character only mentioned passing through there in the first chapter of this book. By better understanding what Ellis Island is, we might better understand what she talks about later.” “When I look up Ellis Island on the internet, the History website tells me that it was the gateway for millions of Americans into the United States from 1892-1954. It also tells me… …Wow, I now understand what it must have been like for the main character in our novel and other immigrants at that time.” “A good strategy to use when we encounter a lot of information is finding the main idea. This involves leaving behind all of the details we don’t really need to remember.” “This first few sentences on Harriet Tubman include words like ‘but’, ‘however’ and ‘on the other hand’. I remember these are some of the compare and contrast words we’ve talked about. So, it seems like the main idea would talk about who or what Tubman is contrasted with…” Table 12 (cont’d) Summary/ Relevant Information Direct Instr. “Remember, we can use the GIST procedure to summarize information. What we do with GIST is use just 20 words or fewer to condense and summarize the 5Ws and H (who, what, where, when why, how) of an entire section we read.” Questions Text Structure “There are two types of question for QAR that help us understand what we read. Some information might be right there in the book, known as in the text. Other information might take our own knowledge and thinking instead, called in your head.” “It can be helpful to know what kind of organization the author used. One type of structure is cause and effect, which explains how one event leads to another. Clue words like first, then, next, after, and later might mean cause and effect.” Modeling (I Do) “First, the what is the Underground Railroad. When we think about who, it’s focused on the African- American slaves. When is the early 19th century if I pick the main time period and where was the northern US. Why was to help slaves escape to freedom, and how was through secret routes and safe houses.” “Now that we read this section, I’m wondering what types of diseases affected indigenous populations in the 1600s. If I remember and look back, this text only told us the names of tribes and number of people who died. So, the answer wouldn’t be right in the text. This might take my prior knowledge…” “Here, the textbook says, ‘Hurricane Katrina destroyed homes and stores, so groups like the Red Cross brought in food and medicine.’ I recognize my clue word ‘so,’ which tells me this is a problem and solution statement.” Guided/Collaborative (We Do) à“Now that you have your 5 Ws and H filled in, how might you combine them into a short summary? Remember to eliminate any extra words. I’ll start with the first few for a summary, and you can help me with the rest. The Underground Railroad was secret routes and houses….” à“Now that we finished another section, let’s go back to our GIST to make a summary. What’s one of the Ws..Okay good, the Emancipation Proclamation is what. And who?..Okay, it involved Lincoln but who did it really focus on or impact?...” à"I’ve already asked a question, so now I want us to work together to ask questions. We’ll then stop and ask ourselves if it’s in the book or if we need to use our own knowledge to answer it.” à“Norah said she wonders how many tribes were affected by these diseases. Hm, is this something that we could figure out from the text we read?”…..”Yeah that’s right, we can look back at the tribes listed. And what’s another tool we can use from this page? Yes, this nice chart we can use…” à“Let’s work together to find clue words and figure out what kind of text we’re reading. Look at the bottom of p.4; what clue words do you see besides ‘because’? Yes, there is also the word ‘effect’…does this sound like cause/effect or sequence?” à“As we read about Katrina, I’ll ask for a volunteer to share what kind of sentence it is. Remember there’s compare/ contrast, description, problem/solution, sequence, or cause/effect. Then, I’ll ask another person to share clue words that helped …” (provides feedback for student responses). 142 Independent (You Do) “Take the next 10 minutes to complete the GIST worksheet for the last section on the 13th amendment. Follow the same steps we used together last week and keep it to less than 20 words.” “Now, work with the person next to you as you read the rest of the chapter. Come up with at least 4 questions while you’re reading and take time to decide whether they are in the text or in your head. Then, write down your question and answer.” “Today, you’re going to read this article about Hurricane Patricia. You will notice some sentences are underlined. I want you to circle any clue words, and then write which of the five types of structures it is above.” Table 12 (cont’d) Visual/ Graphic Organizer Direct Instr. “By using this event map as you read, you can visually organize info. Filling out each section will help you understand Battle of Bunker Hill.” Modeling (I Do) “At the bottom of p.5, it says that even though Americans lost, the battle provided them an important confidence boost because they caused significant casualties among the British. This answers why it was important – so I am going to write that in the square here.” Text Annotation “Today we are going to add another symbol; I want us to start marking question marks when things are confusing and marking exclamation points when things are surprising or exciting.” (reading aloud): “’To date, there have been seventeen amendments added to the Constitution since the Bill of Rights was written.’ Hm, I wonder when the last amendment was added. I’m going to mark a big question mark here with my colored pen and jot down ‘last one’ next to it.” Guided/Collaborative (We Do) à"Now that I’ve shown you how to fill out the event map for the Battle of Bunker Hill, I want you to fill one out with me on the Battle of Yorktown. At the end of each paragraph, we’ll stop and see what information might fit in our event map and where...” àWe are going to work some more today on completing event maps. After reading a page, I will remind you to look back and add information to your event map...Now, where did you add the part saying it involved General George Washington and his 5,400 troops? Everyone think, prepare to share, and go” (teacher uses hand signal, students provide choral response) à“I’ve done a lot of great text marking so far while I read. Starting with this section, I want you to mark the text with me. When I ask a question or find something exciting I want you to put the symbol with me and add the note to help us remember what we thought.” à"Let’s keep working on our question and exclamation marking while we read. At the end of each paragraph I’ll ask you all to stop and mark a question mark or exclamation point somewhere. Then, we’ll talk about what you wrote and why.” Independent (You Do) “Here is a blank event map. Remember to fill it out as you’re reading this section on Crossing the Delaware – do not wait until the very end to add information and organize it.” "While you read your historical novels, make sure you use your sticky notes to mark up the text with your questions, main ideas, and reactions. Remember our special symbols and other words that you should write with the symbols.” 143 6. Paper/Pencil Applied Test Below are items that require applying each of the coding definitions to examples. Please think carefully and consider the definitions discussed at length during training. Then, complete in the corresponding interval of the observation form. 1. At the beginning of this 30 second interval, the teacher tells students to open their textbooks to Chapter 3. During the remainder of this interval the teacher reads aloud a brief passage to the entire class. Which codes would you mark? (circle on observation form, interval 1) 2. Now that the teacher has read aloud the passage, at the start of this interval the teacher tells the whole class of students to “Look back at the passage while writing historical events into your blank graphic organizer worksheet.” For the rest of the interval students are working on this task independently. What codes would you mark? (circle on observation form, interval 2) 4. Earlier, students were given a copy of the historical fiction novel “Chains.” At the beginning of this interval, the teacher instructs the entire class to look closely at the front and back covers, write down a prediction on their own, and then talk with a partner about what they think the book will be about based on the covers. Students begin to write and discuss with their partner for the rest of this interval (circle on observation form, interval 4) 3. During a prior interval, the teacher instructed students in get into groups of four to discuss one thing they learned about the Revolutionary War during yesterday’s class, as well as one thing they were left wondering about after reading for homework. Throughout the current interval, students are carrying out the dialogue. What codes would you mark? (circle on observation form, interval 3) 5. Students are watching the last few seconds of a brief video clip. Then, at the middle of this interval, a teacher tells students to open to page 76 of their textbook. He reminds students that the box at the top of the page has the special key words that they read and discussed earlier, and that it is helpful to read this special box to remember and/or learn what the meaning of key words is before reading them in text. (circle on observation form, interval 5) 6. Now (continuing from the interval above) the teacher begins with the first word on page 76 by reading it aloud, asking students to repeat chorally with him, and then asking for a volunteer to define the word. The teacher provides feedback/correction as the volunteer defines the word. (circle on observation form, interval 6) 7. During this interval the teacher tells students “Today, we are going to learn about Brown vs. Board of Education. What are some things you already know or maybe have heard about this case? Talk with the person next to you.” Students then begin to discuss. (circle on observation form, interval 7) 8. Then the teacher (from the interval above) has students open their workbook to the section on civil rights. Students are instructed to continue working with their partner – they are told and begin to switch off/take turns reading each paragraph, saying what the big idea is that the paragraph tells us. (circle on observation form, interval 8) 144 9. At the beginning of the interval, the teacher is reading aloud to students from a textbook as they followed along. Then, the teacher tells them “I’m now going to display that section on the overhead and show you something called marking up the text. Marking up the text is when we mark symbols and words while we are reading to help us keep track of what we are thinking while we read” (circle on observation form, interval 9) 10. The teacher (from the interval above) then says, “Watch while I read and mark up the text in this first paragraph…”. She then proceeds to circle a word while reading and says “Hmm revolution; I remember this was a vocabulary word, so I put a box around it. But there’s also this word radical, which I don’t really know – so I’m going to mark it with a question mark for now. “(circle on observation form, interval 10) 11. A teacher has students working at their desk on a lesson. Each student has been following along with a program on laptops as they read excerpts from Civil War speeches. During this interval, students are answering questions independently about what they read (located below the excerpt). Then also during this interval, students are discussing their answers with their seat partner. (circle on observation form, interval 11) Match the checklist items to the left below with the correct example to the right by writing the letter in the blank. Note that there are extra examples, meaning a few options on the right will be left unmatched with items on the left. Table 13 RA Training: Paper/Pencil Applied Test Items ____Comprehension Monitoring ____Vocabulary, Morphology ____Vocabulary, Context Clues ____Background Knowledge, Prior ____Background Knowledge, Building ____Summary/Relevant Information ____Text Structure A. “In this section we see the words first, then, after, and finally – these words tell us it is organized in the order that things happened, or what we call sequence” B. “I finished this sentence and now I ask myself “Did I understand it?” I did, so I say “click” and keep reading…” C. “Today we are reading a book titled Our Troop Marches On: A Soldier’s Civil War Story. What are some things you already know or learned about the life of Civil War soldiers?” D. “After you finish reading this page on George Washington, you should write down the key points in just three sentences. Just like yesterday, ask yourself “What is most important? What information is unnecessary?” E. “Now, I want you to take turns reading this section out loud with your partner. If you come to a word you don’t know, be sure to ask them for help with what it might mean.” F. “We don’t know this word, but it seems important. Militarism has the suffix “ism” which we know means a type of thinking or practice. And “militar” might come from what word?....” G. “When you come to a word you don’t know, remember to use hints around the word that the author gives to help us define it. Here, we see the word “but” which tells us it might have an opposite meaning.” H. “While reading with your partner, identify the main idea that each paragraph is trying to get across.” I. “Today we are reading a book titled Our Troop Marches On: A Soldier’s Civil War Story. Civil War soldiers had to endure many adverse conditions such as….” 145 7. Application to Training Videos Each of you will use the observation form while watching videos of upper elementary social studies instruction. You will complete two videos. Agreement with the master codes must reach at least 0.7 for each variable on the form. All disagreements will be discussed after codes are compared, regardless of agreement level. Should you not reach the 0.7 level of agreement across all variables on these two videos, you will continue to code additional video(s) until the two most recently coded reach this agreement (i.e., two in a row). Check your email for access to the first video via Google drive. Upon completion, the second video will be sent to you via Google drive as well. 146 Note: Staple all pages of packet together (Assent, Engagement Survey and DIBELS Daze) Note: Label the third page of each packet ahead of time with the student ID numbers o FIDELITY CHECKLIST (n=1 copy of this, bring to observation session) o OBSERVATION TOOL Form (n=1 copy) Note: Label ahead of time with classroom ID number (provided by Heather, 4 digits long). o RUNNING CHECKLIST of Specific Strategies (n=1 copy; included with OBSERVATION TOOL) o TEACHER ITEMS form (n=1 copy; included with OBSERVATION TOOL) Note: Label ahead of time with classroom ID number (provided by Heather, 4 digits long). o STUDENT PACKETS (n= enough for each student in the class) (provided by Heather, 6 digits long). o STUDENT PACKET FOR RAs (n=1 copy for you, use for each observation session) o Copy of OBSERVATION CODE DEFINITIONS (n=1 copy, to use as/if needed) r Email reminder to teacher 24 hrs in advance (use template OBSERV REMINDER EMAIL) r Have names of any students whose parents opted-out from Heather (if applicable) r Bring the following to the data collection session o All printed copies from step 1 o Interval timer (set for 30 sec; vibrate and/or flashing) o Pens/Pencils, Clipboard r Conduct observation of entire social studies lesson o Use OBSERVATION TOOL, interval timer, clipboard, and pens/pencils o For each 30-sec interval, code events occurring at any point during interval o Throughout observation, record specific comprehension instructions strategies used via RUNNING CHECKLIST (i.e., check if used at any point during entire observation) r Collect student data immediately following observation o Any students whose parents opted out of participation should be prompted to move to a separate table in the room with a book for silent reading o For remaining students, hand out STUDENT PACKET o Read aloud assent form, have students tear off two pages, turn in o Read aloud instructions and each item of student engagement survey to entire class o Read aloud instructions for DIBELS Daze, time assessment for three minutes o Collect completed STUDENT PACKETS o Give TEACHER ITEMS form to the teacher to complete o Double check that one number is circled for each item and email address is provided o Gather copies of ALL text materials utilized during instruction (workbook pages, textbook sections, handouts, etc.) o Teacher is prompted to have this ready for you ahead of time, but you should be provided access to a copier and the original materials if needed. o Record title and year of any text material on the copies o Record classroom ID # on front page of any text material copies/packets o NOTE: If assigned to conduct the observation with another RA (for reliability purposes), coordinate at the start of the observation so intervals are identical r Debrief with teacher r Have printed copies of the following: 8. Data Collection Procedures: Fidelity Checklist 147 APPENDIX M Observation Reminder Email Hello TEACHERNAME, My name is RANAME, and I am a research assistant for Heather Schmitt from Michigan State University. Just a friendly reminder that I will be observing in your classroom tomorrow during social studies instruction, starting at TIME. As the consent form mentioned, please have copies of all text materials that you will use during the lesson ready for me to take with me after I observe (e.g., copies of textbook pages, worksheets). Also, remember that I will be providing some classwide measures to your students after the social studies instruction for 10-15 minutes. Please plan for this in your schedule accordingly. If for any reason the observation has to be rescheduled last minute, or you have any questions, please contact me directly as soon as possible at EMAIL or PHONE#. Thank you and see you tomorrow! RANAME 148 r Have printed copies of the following: Note: Staple all pages of packet together (Assent, Engagement Survey and DIBELS Daze) Note: Label the third page of each packet ahead of time with the student ID numbers o FIDELITY CHECKLIST (n=1 copy of this, bring to observation session) o OBSERVATION TOOL Form (n=1 copy) Note: Label ahead of time with classroom ID number (provided by Heather, 4 digits long). o RUNNING CHECKLIST of Specific Strategies (n=1 copy; with OBSERVATION TOOL) o TEACHER ITEMS form (n=1 copy; included with OBSERVATION TOOL) Note: Label ahead of time with classroom ID number (provided by Heather, 4 digits long). o STUDENT PACKETS (n= enough for each student in the class) (provided by Heather, 6 digits long). o STUDENT PACKET FOR RAs (n=1 copy for you, use for each observation session) o Copy of OBSERVATION CODE DEFINITIONS (n=1 copy, to use as/if needed) r Email reminder to teacher 24 hrs in advance (use template OBSERVATION REMINDER EMAIL) r Have names of any students whose parents opted-out from Heather (if applicable) r Bring the following to the data collection session o All printed copies from step 1 o Interval timer (set for 30 sec; vibrate and/or flashing) o Pens/Pencils, Clipboard r Conduct observation of entire social studies lesson o Use OBSERVATION TOOL, interval timer, clipboard, and pens/pencils o For each 30-sec interval, code events occurring at any point during interval o Throughout observation, record specific comprehension instructions strategies used via RUNNING CHECKLIST (i.e., check if used at any point during entire observation) r Collect student data immediately following observation o Any students whose parents opted out of participation should be prompted to move to a separate table in the room with a book for silent reading o For remaining students, hand out STUDENT PACKET o Read aloud assent form, have students tear off two pages, turn in o Read aloud instructions and each item of student engagement survey to entire class o Read aloud instructions for DIBELS Daze, time assessment for three minutes o Collect completed STUDENT PACKETS o Give TEACHER ITEMS form to the teacher to complete o Double check that one number is circled for each item and email address is provided o Gather copies of ALL text materials used (workbook pg, textbook section, handout, etc.) o Teacher is prompted to have this ready for you ahead of time, but you should be provided access to a copier and the original materials if needed. o Record title and year of any text material on the copies o Record classroom ID # on front page of any text material copies/packets o NOTE: If assigned to conduct the observation with another RA (for reliability purposes), coordinate at the start of the observation so intervals are identical r Debrief with teacher APPENDIX N Fidelity Checklist 149 APPENDIX O Student Assent Form [RA Script, Introduction: My name is RA NAME, and I am a researcher from Michigan State University. I am in your classroom today observing a social studies lesson and am mostly interested in what your teacher is doing to help you learn. But, I am hoping you will be willing to participate in our study by completing a 3-minute reading task and filling out a short survey. I won’t be testing you on anything that you learned during class today, and this won’t affect your grade or anything in your class. Your parents have already given their permission for you to participate in this today. More information is provided about the study on this sheet that I will read aloud to you now. If you would like to participate, sign and date this sheet at the bottom. If you would not like to participate, check the box and print your name in the blank next to it. Do any of you have any questions right now? Feel free to raise your hand if you have any questions while I am reading the form aloud.] Research Participant Information and Assent Form You are being asked to participate in a research study. Researchers are required to provide an assent form to inform you about the research study, to let you know that participation is voluntary, to explain risks and benefits of participation, and to empower you to make an informed decision. You should feel free to ask the researchers any questions you may have. Study Title: Upper Elementary Social Studies Instruction: Exploring the Relationship Between Text Use, Comprehension Instruction, and Student Engagement Researcher and Title: Dr. Sara E. Witmer (formerly Sara E. Bolt), Associate Professor of School Psychology and Heather A. Schmitt, M.A. Department and Institution: College of Education, Michigan State University Address and Contact Information: 620 Farm Lane, Rm 434, Michigan State University, East Lansing, MI 48824 1. PURPOSE OF RESEARCH • You are being asked to participate in a research study on social studies instruction in fifth grade classrooms. From this study, the researchers hope to learn more about reading expectations and instruction during social studies. • You have been selected as a possible participant in this study because you are a fifth grade student, and your teacher has agreed to help with the study. Your parents have been given information about the study and have agreed to let you participate as well. • Your participation in this study will take about ten minutes. 2. WHAT YOU WILL DO • After a social studies lesson, you will complete a five-minute survey that will be read aloud to your class by a trained research assistant. • You will then be asked to complete a three-minute reading assessment. 3. POTENTIAL BENEFITS • You will not directly benefit from participation in this study. However, your participation may help us better understand reading expectations and supports in social studies classes. 150 4. POTENTIAL RISKS 5. PRIVACY AND CONFIDENTIALITY • There are no foreseeable risks associated with participation in this study. • The data for this project will be kept confidential to the greatest extent possible according to law. • The trained research assistants will collect data at your school in a way such that your name and identifying information will not be associated with the data that are collected. Your name will only appear on this assent form. • All related information (forms, data, etc.) will be kept in separate locked files and on computers that are password protected. These steps will be taken so that no one outside of the research team and those involved in promoting appropriate research at Michigan State University can have access to information that might link your name with the study. • Your teacher and your parents will not have access to the data. • The results of this study may be published or presented at professional meetings, but the identities of all research participants will remain anonymous. 6. YOUR RIGHTS TO PARTICIPATE, SAY NO, OR WITHDRAW • Participation is voluntary. Refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled. You may discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. o You have the right to say no. o You may change your mind at any time and withdraw. o You may choose not to answer specific questions or to stop participating at 7. COSTS AND COMPENSATION FOR BEING IN THE STUDY • There are no costs that will be incurred and no compensation as a result of any time. participating in this study. 8. CONTACT INFORMATION • • If you have concerns or questions about this study, such as scientific issues, how to do any part of it, or to report an injury, please contact the study coordinator (Heather Schmitt, email: schmi538@msu.edu, phone: 517-599-8917, address: 620 Farm Lane, Rm 434, Michigan State University, East Lansing, MI 48824). If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact the Michigan State University’s Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or mail at 207 Olds Hall, MSU, East Lansing, MI 48824. 9. DOCUMENTATION OF INFORMED ASSENT Your signature below means that you voluntarily agree to participate in this research study. If you do not wish to participate in this research study, check the box below and print your name. _______________________________ Signature I, ____________________________, do not wish to participate in the research study. _________________________ Date 151 APPENDIX P Teacher and Student Likert Items Please read each item carefully and circle one number to indicate how similar today’s instructional period was to your typical social studies instruction. A. In general, how similar was the amount of text used today to the amount you typically use during social studies? (1)-----------------------(2) -----------------------(3) -----------------------(4) text amount text amount was text amount was not similar a little similar was similar was very similar B. In general, how similar were today’s instructional practices to how you typically teach with text during social studies? text amount (1)-----------------------------(2) --------------------------(3) ---------------------------(4) instructional practices instructional practices were not similar were a little similar were similar were very similar instructional instructional practices practices Please read and listen carefully to the item as it is read aloud; then, circle only one number. In class today, you learned about the topic __________________________________. How interested were you in this topic before today’s class? Please circle one number below. 1-----------------2-----------------3-----------------4-----------------5 Interested Interested Interested Interested Not At All Not Very Neutral Somewhat Very 152 153 REFERENCES REFERENCES Alger, C. L. (2007). Engaging student teachers' hearts and minds in the struggle to address (il)literacy in content area classrooms. Journal of Adolescent & Adult Literacy, 50(8), 620-630. Allington, R. L. (2002). You can't learn much from books you can't read. Educational Leadership, 60(3), 16-19. Allington, R. L. (2005). What really matters for struggling readers: Designing research-based programs (what really matters series). Boston, MA: Allyn & Bacon. Allington, R. L., & Johnston, P. H. (2002). Reading to learn: Lessons from exemplary fourth- grade classrooms. Guilford Press. related to parental consent in school-based survey research. Evaluation Review, 19(6), 663-674. Anderman, C., Cheadle, A., Curry, S., Diehr, P., Shultz, L., & Wagner, E. (1995). Selection bias Applebee, A. (2013). Common Core State Standards: The promise and the peril in a national palimpsest. English Journal, 103(1), 25-33. Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369-386. Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument. Journal of School Psychology, 44(5), 427-445. Armbruster, B. B. (1984). The problem of" inconsiderate text.".Comprehension instruction: Perspectives and suggestions, 202-217. Assor, A., Kaplan, H., & Roth, G. (2002). Choice is good, but relevance is excellent: Autonomy‐enhancing and suppressing teacher behaviours predicting students' engagement in schoolwork. British Journal of Educational Psychology, 72(2), 261-278. Benavot, A. (2011). Improving the provision of quality education: Perspectives from textbook research. Journal of International Cooperation in Education, 14(2), 1-16. in the analysis of text difficulty. Educational Psychology Review, 24(1), 63-88. Benjamin, R. G. (2012). Reconstructing readability: Recent developments and recommendations Berkeley, S., King-Sears, M. E., Hott, B. L., & Bradley-Black, K. (2014). Are history textbooks more “considerate” after 20 years?. The Journal of Special Education, 47(4), 217-230. 154 Berkeley, S., King-Sears, M. E., Vilbas, J., & Conklin, S. (2016). Textbook characteristics that support or thwart comprehension: The current state of social studies texts. Reading & Writing Quarterly, 32(3), 247-272. Biancarosa, G., & Snow, C. (2004). Reading next: A vision for action and research in middle and high school literacy. Journal of Adolescent & Adult Literacy, 41(1), 12–24. Bintz, W. (1997). Exploring reading nightmares of middle and secondary school teachers. Bliem, C. L., & Davinroy, K. (1997). Teachers' beliefs about assessment and instruction in literacy. National Center for Research on Evaluation, Standards, and Student Testing (CRESST), Graduate School of Education & Information Studies, University of California, Los Angeles. Bolinger, K., & Warren, W. J. (2007). Methods practiced in social studies instruction: A review of public school teachers' strategies. International Journal of Social Education, 22(1), 68. first: Teaching social studies in elementary classrooms. Theory & Research in Social Education, 36(3), 233-255. Boyle-Baise, M., Hsu, M. C., Johnson, S., Serriere, S. C., & Stewart, D. (2008). Putting reading Brophy, J., & Alleman, J. (2009). Meaningful social studies for elementary students. Teachers and Teaching: theory and practice, 15(3), 357-376. Brown-Chidsey, R., & Andren, K. (2013) Assessment for Intervention: A Problem-Solving Approach, 2nd ed. The Guilford Press, New York, NY Brozo, W. G., Shiel, G., & Topping, K. (2007). Engagement in reading: Lessons learned from three PISA countries. Journal of Adolescent & Adult Literacy, 51(4), 304-315. teachers' perceptions about literacy teaching and learning. Literacy Research and Instruction, 48(1), 76-94. Cantrell, S. C., Burns, L. D., & Callaway, P. (2009). Middle- and high-school content area Chall, J. S. (1983). Literacy: Trends and explanations. Educational Researcher, 12(9), 3-8. Chambliss, M., & Calfee, R. (1998). Textbooks for learning: Nurturing children's minds. Christenson, S. L., Reschly, A. L., & Wylie, C. (Eds.). (2012). Handbook of research on student Blackwell Publishing. engagement. Springer Science & Business Media. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2013). Applied multiple regression/correlation analysis for the behavioral sciences. Routledge. 155 Coleman, D. (2011). Bringing the Common Core to life [Lecture transcript]. Retrieved from New York State Education Department website: http://usny.nysed.gov/rttt/docs/bringingthecommoncoretolife/fulltranscript.pdf Connell, J. P., Halpem-Felsher, B. L., Clifford, E., Crichlow, W., & Usinger, P. (1995). Hanging in there: Behavioral, psychological, and contextual factors affecting whether African American adolescents stay in high school. Journal of adolescent research, 10(1), 41-63. Connell, J. P., Spencer, M. B., & Aber, J. L. (1994). Educational risk and resilience in African‐American youth: Context, self, action, and outcomes in school. Child development, 65(2), 493-506. Connell, J. P., & Wellborn, J. G. (1991). Competence, autonomy, and relatedness: A Corso, M. J., Bundick, M. J., Quaglia, R. J., & Haywood, D. E. (2013). Where student, teacher, motivational analysis of self-system processes. In M. R. Gunnar & L. A. Sroufe (Eds.), The Minnesota symposia on child psychology, Vol. 23. Self processes and development (pp. 43-77). Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc. and content meet: Student engagement in the secondary school classroom. American Secondary Education, 41(3), 50. Covell, K., McNeil, J. K., & Howe, R. B. (2009). Reducing Teacher Burnout by Increasing Student Engagement A Children's Rights Approach. School Psychology International, 30(3), 282-290. Cruz, B. C., & Thornton, S. J. (2012). Visualizing social studies literacy: Teaching content and skills to English language learners. Social Studies Research and Practice, 7(2), 98-111. Cunningham, P. M., & Allington, R. L. (1999). Classrooms that work: They can all read and Deci, E. L., & Ryan, R. M. (1985). Cognitive evaluation theory. In Intrinsic motivation and self write. Addison Wesley Longman, Inc., Reading, MA. determination in human behavior (pp. 43-85). Springer US. Deci, E. L., & Ryan, R. M. (2000). The" what" and" why" of goal pursuits: Human needs and the self-determination of behavior. Psychological inquiry,11(4), 227-268. Dexter, D. D., & Hughes, C. A. (2011). Graphic organizers and students with learning disabilities: A meta-analysis. Learning Disability Quarterly, 34(1), 51-72. Dolezal, S. E., Welsh, L. M., Pressley, M., & Vincent, M. M. (2003). How nine third-grade teachers motivate student academic engagement. The Elementary School Journal, 239- 267. 156 Donahue, D. (2000). Experimenting with texts: New science teachers’ experience and practice as readers and teachers of reading. Journal of Adolescent and Adult Literacy, 43(8), 728– 739. Dougherty Stahl, K. A. (2012). Complex text or frustration-level text: Using shared reading to bridge the difference. The Reading Teacher, 66(1), 47-51. Duke, N. K. (2000). 3.6 minutes per day: The scarcity of informational texts in first grade. Reading Research Quarterly, 35(2), 202-224. Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading comprehension. Journal of education, 189(1-2), 107-122. Duke, N. K., Pearson, P. D., Strachan, S. L., & Billman, A. K. (2011). Essential elements of Dunn, M. A. (2000). Closing the book on social studies: Four classroom teachers go beyond the fostering and teaching reading comprehension. What research has to say about reading instruction, 4, 51-93. text. The Social Studies, 91(3), 132-136. Durkin, D. (1978). What classroom observations reveal about reading comprehension instruction. Reading Research Quarterly, 14(4), 481-533. Dymock, S., & Nicholson, T. (2010). “High 5!” Strategies to enhance comprehension of expository text. The Reading Teacher, 64(3), 166-178. assessments grades. Retrieved from http://ecs.force.com/mbdata/mbquestrt?rep=SUM1804 Education Commission of the States. (2018, April). 50-state comparison: Social studies Esbensen, F. A., Miller, M. H., Taylor, T., He, N., & Freng, A. (1999). Differential attrition rates and active parental consent. Evaluation Review, 23(3), 316-335. Finch, W. H., Bolin, J. E., & Kelley, K. (2014). Multilevel modeling using R. CRC Press. Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59(2), 117-142. Fisher, D., & Frey, N. (2014a). Content area vocabulary learning. The Reading Teacher, 67(8), 594-599. Fisher, D., & Frey, N. (2014b). Scaffolded Reading Instruction of Content‐Area Texts. The reading teacher, 67(5), 347-351. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109. 157 Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011). Measuring Student Engagement in Upper Elementary through High School: A Description of 21 Instruments. Issues & Answers. REL 2011-No. 098. Regional Educational Laboratory Southeast. Furrer, C., & Skinner, E. (2003). Sense of relatedness as a factor in children's academic engagement and performance. Journal of educational psychology,95(1), 148. Furrer, C., Skinner, E., Marchand, G., & Kindermann, T. A. (2006, March). Engagement vs. disaffection as central constructs in the dynamics of motivational development. In annual meeting of the Society for Research on Adolescence, San Francisco, CA. Gajria, M., Jitendra, A. K., Sood, S., & Sacks, G. (2007). Improving comprehension of expository text in students with ld a research synthesis.Journal of learning disabilities, 40(3), 210-225. Teachers College Press. teach. California Reader, 35(2), 11-20. Gay, G. (2000). Culturally responsive teaching: Theory, research, and practice. New York: Gernon, A., & Grisham, D. (2002). Expository texts in the intermediate grades: What teachers Gilles, C., Wang, Y., Smith, J., & Johnson, D. (2013). “I’m no longer just teaching history”: Goldman, S. R., & Rakestraw, J. A. (2000). Structural aspects of constructing meaning from Professional development for teaching common core state standards for literacy in social studies. Middle School Journal, 44(3), 34-43. text. Handbook of reading research, 3(1), 311-335. Good, R. J., Kaminski, R. A., Dewey, E. N., Wallin, J., Powell-Smith, K. A., & Latimer, R. J. (2013). Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Technical Manual. Eugene, OR: Dynamic Measurement Group. Hall, L. A. (2004). Comprehending expository text: Promising strategies for struggling readers and students with reading disabilities?. Literacy Research and Instruction, 44(2), 75-95. Hall, L. A. (2005). Teachers and content area reading: Attitudes, beliefs and change. Teaching and Teacher Education: An International Journal of Research and Studies, 21(4), 403-414. Retrieved from http://dx.doi.org/10.1016/j.tate.2005.01.009 tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23. Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: an overview and Hansen, J. (1981). The effects of inference training and practice on young children's reading comprehension. Reading Research Quarterly, 391-417. 158 Hansen, J., & Pearson, P. D. (1983). An instructional study: Improving the inferential comprehension of good and poor fourth-grade readers. Journal of Educational Psychology, 75(6), 821. Harmon, J. M., Hedrick, W. B., & Fox, E. A. (2000). A content analysis of vocabulary instruction in social studies textbooks for grades 4-8. The Elementary School Journal, 253-271. Harmon, J. M. Hedrick, W.B. & Wood, KD (2005). Research on vocabulary instruction in the content areas: Implications for struggling readers. Reading & Writing Quarterly, 21. Harniss, K., Dickson, S. V., Kinder, D., Hollenbeck, K.L. (2001). Textual problems and instructional solutions: Strategies for enhancing learning from published history textbooks. Reading & Writing Quarterly,17(2), 127-150. Harvey, S. (1998). Nonfiction matters: Reading, writing, and research in grades 3-8. Stenhouse Haynes, J. (2005). Challenges for ELLs in content area learning. In TESOL Annual Convention, Publishers. March (pp. 25-29). Hill, L. G., & Werner, N. E. (2006). Affiliative motivation, school attachment, and aggression in school. Psychology in the Schools, 43(2), 231-246. Hox, J.J., Moerbeek, M., & van de Schoot, R. (2010). Multilevel Analysis: Techniques and Applications, Second Edition (Quantitative Methodology Series). Taylor and Francis. Kindle Edition. Ingrao, M., & Middleton, T. L. (2012). California teachers and social studies: A survey report on approaches to curriculum and instruction. Social Studies Review, 51, 120. Ivey, G., & Johnston, P. H. (2013). Engagement with young adult literature: Outcomes and processes. Reading Research Quarterly, 48(3), 255-275. Jackson, F. R., & Cunningham, J. W. (1994). Investigating secondary content teachers' and preservice teachers' conceptions of study strategy instruction. Literacy Research and Instruction, 34(2), 111-135. Jang, H., Reeve, J., & Deci, E. L. (2010). Engaging students in learning activities: It is not Jeong, J., Gaffney, J. S., & Choi, J. O. (2010). Availability and use of informational texts in autonomy support or structure but autonomy support and structure. Journal of Educational Psychology, 102(3), 588. second-, third-, and fourth-grade classrooms. Research in the Teaching of English, 435- 456. 159 Jimenez-Silva, M., Hinde, E. R., & Jimenez Hernandez, N. V. (2013). English language learners in social studies classrooms. Research on the status of social studies: Views from the field, 275-286. Jimerson, S. R., Campos, E., & Greif, J. L. (2003). Toward an understanding of definitions and measures of school engagement and related terms. The California School Psychologist, 8(1), 7-27. Jitendra, A. K., Nolet, V., Xin, Y.P., Gomez, O., Renouf, K., Iskold, L., DaCosta, J. (2001). An analysis of middle school geography textbooks: Implications for students with learning problems. Reading & Writing Quarterly, 17(2), 151-173. Kamil, M. L., Borman, G. D., Dole, J., Kral, C. C., Salinger, T., & Torgesen, J. (2008). Improving Adolescent Literacy: Effective Classroom and Intervention Practices. IES Practice Guide. NCEE 2008-4027. National Center for Education Evaluation and Regional Assistance. Kim, A. H., Vaughn, S., Wanzek, J., & Wei, S. (2004). Graphic organizers and their effects on the reading comprehension of students with LD A synthesis of research. Journal of Learning Disabilities, 37(2), 105-118. Kinder, D., Bursuck, B., & Epstein, M. (1992). An evaluation of history textbooks. The Journal of Special Education, 25(4), 472-491. Klem, A. M., & Connell, J. P. (2004). Relationships matter: Linking teacher support to student engagement and achievement. Journal of school health,74(7), 262-273. Kline R.B. (2005) Principles and Practice of Structural Equation Modeling, 2nd ed. The Guilford Press, New York, NY Lapp, D., Fisher, D., Flood, J., & Frey, N. (2003). Dual role of the urban reading specialist. Journal of Staff Development, 24(2), 33. Lubliner, S. (2004). Help for struggling upper-grade elementary readers. The Reading Teacher, 57(5), 430-438. Lutz, S. L., Guthrie, J. T., & Davis, M. H. (2006). Scaffolding for engagement in elementary school reading instruction. The Journal of Educational Research, 100(1), 3-20. Marks, H. M. (2000). Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American educational research journal, 37(1), 153-184. Mathieu, J. E., Aguinis, H., Culpepper, S. A., & Chen, G. (2012). Understanding and estimating the power to detect cross-level interaction effects in multilevel modeling. Journal of Applied Psychology, 97(5), 951. 160 implementation in content area classrooms?. Journal of Instructional Pedagogies, 4, 1-18. McCoss-Yergian, T., & Krepps, L. (2010). Do teacher attitudes impact literacy strategy McMurrer, J. (2007). Implementing the No Child Left Behind teacher requirements. Washington, DC: Center for Education Policy. Retrieved from http://www.cep dc.org/displayDocument.cfm?DocumentID=312 McMurrer, J. (2008). Instructional time in elementary schools: A closer look at changes for specific subjects. Washington, DC: Center for Education Policy. Retrieved from http://www.cep-dc.org/displayDocument.cfm?DocumentID=309 McNeish, D. M., & Stapleton, L. M. (2016). The effect of small sample size on two-level model Meany‐Daboul, M. G., Roscoe, E. M., Bourret, J. C., & Ahearn, W. H. (2007). A comparison of estimates: A review and illustration. Educational Psychology Review, 28(2), 295-314. momentary time sampling and partial‐interval recording for evaluating functional relations. Journal of Applied Behavior Analysis, 40(3), 501-514. Meyer, B. J. (2003). Text coherence and readability. Topics in Language Disorders, 23(3), 204 224. Meyer, B. J., & Rice, G. E. (1984). The structure of text. Handbook of reading research, 1, 319 351. choice, motivation and learner engagement. Theory Into Practice, 50(3), 247-253. Moley, P. F., Bandré, P. E., & George, J. E. (2011). Moving beyond readability: Considering Monte-Sano, C. (2011). Beyond reading comprehension and summary: Learning to read and write in history by focusing on evidence, perspective, and interpretation. Curriculum Inquiry, 41(2), 212-249. Moss, B. (2005). Making a case and a place for effective content area literacy instruction in the elementary grades. The Reading Teacher, 59(1), 46-55. Mraz, M., Rickelman, R. J., & Vacca, R. T. (2009). Content-area reading. Literacy instruction for adolescents: Research-based practice, 77. National Reading Panel (US), National Institute of Child Health, & Human Development (US). (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. National Institute of Child Health and Human Development, National Institutes of Health. National Research Council. (2004). Engaging schools: Fostering high school students’ motivation to learn. Washington, DC: The National Academies Press. 161 Nelson J., Perfetti C., Liben D., Liben M. (2012). Measures of text difficulty: Testing their predictive value for grade levels and student performance. New York, NY:Student Achievement Partners. Ness, M.K. (2007). Reading comprehension strategies in secondary content-area classrooms. Phi Delta Kappan, 89(3), 229. Ness, M. K. (2009). Reading comprehension strategies in secondary content area classrooms: Teacher use of and attitudes towards reading comprehension instruction. Reading Horizons, 49(2), 143-166. New York, NY:Teachers College Press. for secondary preservice teachers. The Clearing House, 71(6), 372-374. changing? Abt Associates Inc. Retrieved from http://abtassociates.com/Perspectives/January-2017/Common-Core-Revisions-What-Are- States-Really-Chang. Newmann, F. M. (1992). Student engagement and achievement in American secondary schools. Norton, J., Ash, J., & Ballinger, S. (2017). Common core revisions: What are states really Nourie, B. L., & Lenski, S. D. (1998). The (in) effectiveness of content area literacy instruction O'Brien, D. G., & Stewart, R. A. (1990). Preservice teachers' perspectives on why every teacher O'Connor, R. E., Bell, K. M., Harty, K. R., Larkin, L. K., Sackor, S. M., & Zigmond, N. (2002). Pallant, J. (2013). SPSS survival manual. McGraw-Hill Education (UK). Parker, W. C., & Jarolimek, J. (1997). Curriculum standards for social studies: Expectations of is not a teacher of reading: A qualitative analysis. Journal of Reading Behavior, 22(2), 101-129. Teaching reading to poor readers in the intermediate grades: A comparison of text difficulty. Journal of Educational Psychology, 94(3), 474. excellence. National Council for the Social Studies. Parsons, S. A., Malloy, J. A., Parsons, A. W., & Burrowbridge, S. C. (2015). Students’ engagement in literacy tasks. The Reading Teacher, 69(2), 223-231. Pearson, P. D., & Duke, N. K. (2002). Comprehension instruction in the primary grades. Comprehension instruction: Research-based best practices, 247-258. Pearson, P. D., & Gallagher, M. C. (1983). The instruction of reading Perry, N. E., Turner, J. C., & Meyer, D. K. (2006). Classrooms as contexts for motivating comprehension. Contemporary Educational Psychology, 8(3), 317-344. learning. Handbook of educational psychology, 2, 327-348. 162 Pressley, M. (2006). Reading instruction that works: The case for balanced teaching. New York, Pressley, M., & Allington, R.L. (2015). Reading instruction that works: The case for balanced NY: The Guilford Press. teaching (4th ed.). New York, NY: Guilford. increasing teachers' autonomy support. Motivation and Emotion, 28(2), 147-169. Reeve, J., Jang, H., Carrell, D., Jeon, S., & Barch, J. (2004). Enhancing students' engagement by Reutzel, D. R., Camperell, K., & Smith, J. A. (2002). Hitting the wall: Helping struggling readers comprehend. In L. B. Gambrell, C. Collins-Block, & M. Pressley (Eds.), Improving reading comprehension: Rethinking research, theory, and classroom practice. (pp. 321-353). San Francisco, CA: Jossey- Bass Publishers. emotional climate, student engagement, and academic achievement. Journal of Educational Psychology, 104(3), 700-712. Reyes, M. R., Brackett, M. A., Rivers, S. E., White, M., & Salovey, P. (2012). Classroom Russell III, W. B. (2010). Teaching social studies in the 21st century: A research study of secondary social studies teachers' instructional methods and practices. Action in Teacher Education, 32(1), 65-72. Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68-78. Schleppegrell, M. J., Achugar, M., & Oteíza, T. (2004). The Grammar of History: Enhancing Content‐Based Instruction Through a Functional Focus on Language. TESOL quarterly, 38(1), 67-93. Schoenbach, R., Greenleaf, C. L., & Hale, G. (2010). Framework fuels the need to read: Strategies boost literacy of students in content-area classes. Journal of Staff Development, 31(5), 38-42. Retrieved from http://search.ebscohost.com.ezproxy.mnsu.edu/login.aspx?direct=true&db=eue& AN=508185178&site=ehost-live. intellectual engagement for disengaged students. Theory Into Practice, 48(2), 114-121. Schussler, D. L. (2009). Beyond content: How teachers manage classrooms to facilitate Sinclair, M. F., Christenson, S. L., Lehr, C. A., & Anderson, A. R. (2003). Facilitating student engagement: Lessons learned from Check & Connect longitudinal studies. The California School Psychologist, 8(1), 29-41. Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of educational psychology, 85(4), 571. 163 Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008). Engagement and disaffection in the classroom: Part of a larger motivational dynamic?. Journal of Educational Psychology, 100(4), 765. Skinner, E. A., Kindermann, T. A., Connell, J. P., & Wellborn, J. G. (2009). Engagement and disaffection as organizational constructs in the dynamics of motivational development. Handbook of motivation at school, 223-245. Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children's behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69(3), 493-535. Skinner, E. A., & Pitzer, J. R. (2012). Developmental dynamics of student engagement, coping, and everyday resilience. In Handbook of research on student engagement (pp. 21-44). Springer US. Skinner, E. A., Zimmer-Gembeck, M. J., Connell, J. P., Eccles, J. S., & Wellborn, J. G. (1998). Individual differences and the development of perceived control. Monographs of the society for Research in Child Development, i-231. Snow, M. A. (2001). Content-based and immersion models for second and foreign language teaching. Teaching English as a second or foreign language, 3, 303-318. Snow, C. & Moje, E. (2010). Why is everyone talking about adolescent literacy? The Phi Delta Kappan, 91(6), 66-69. doi:10.2307/27755672 Spencer, V. G., Carter, B. B., Boon, R. T., Simpson-Garcia, C. (2008). If you teach—you teach reading. International Journal of Special Education, 23(2), 1-7. Soper, D.S. (2016). A-priori Sample Size Calculator for Multiple Regression [Software]. Available from http://www.danielsoper.com/statcalc. assessments: A practitioner's guide. Guilford Press. Spor, M. W. (2005). Reading to Learn. Principal Leadership, 5(6), 16-21. Steege, M. W., & Watson, T. S. (2009). Conducting school-based functional behavioral Steinberg, L. (1996). Beyond the classroom: Why school reform has failed and what parents Strommen, L. T., & Mates, B. F. (2004). Learning to love reading: Interviews with older children and teens. Journal of Adolescent & Adult Literacy, 48(3), 188-200. need to do. New York: Touchstone. 164 secondary schools. Washington, DC: Alliance for Excellent Education. Sturtevant, E. G. (2003). The literacy coach: A key to improving teaching and learning in Swanson, E., Wanzek, J., McCulley, L., Stillman-Spisak, S., Vaughn, S., Simmons, D., ... & Hairrell, A. (2016). Literacy and text reading in middle and high school social studies and English language arts classrooms. Reading & Writing Quarterly, 32(3), 199-222. Pearson. Tabachnick, B. G., & Fidell, L. S. (2014). Using multivariate statistics, sixth edition. New York: Taylor, B. M., Pearson, P. D., Clark, K., & Walpole, S. (2000). Effective schools and accomplished teachers: Lessons about primary-grade reading instruction in low-income schools. The Elementary School Journal, 121-165. Texas Reading Initiative. (2002). Research-based content area reading instruction. Revised Edition. Texas Education Agency. Thorndyke, P. W. (1977). Cognitive structures in comprehension and memory of narrative discourse. Cognitive psychology, 9(1), 77-110. Tomlinson, C. A. (2003). Fulfilling the promise of the differentiated classroom: Strategies and tools for responsive teaching. Association for Supervision and Curriculum Development. Turner, J. C., & Meyer, D. K. (2004). A classroom perspective on the principle of moderate Vacca, R. T., Vacca, J. A. L., & Mraz, M. E. (2016). Content area reading: Literacy and challenge in mathematics. Journal of Educational Research, 97(6), 311–318. learning across the curriculum, 12th edition. New York: Pearson. Van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher–student interaction: A decade of research. Educational Psychology Review,22(3), 271-296. in elementary school children. Rethinking Reading Comprehension, 51-81. Vellutino, F. R. (2003). Individual differences as sources of variability in reading comprehension von Zastrow, C. E., & Janc, H. (2004). Academic atrophy: The condition of the liberal arts in America's public schools. Council for Basic Education. Retrieved from http://www.sjsu.edu/people/kathie.kratochvil/courses/CA177/s0/AcademicAtrophy.pdf. Wade, R. C. (1993). Content analysis of social studies textbooks: A review of ten years of research. Theory & Research in Social Education, 21(3), 232-256. Wharton-McDonald, R., Pressley, M., & Hampston, J. M. (1998). Literacy instruction in nine first-grade classrooms: Teacher characteristics and student achievement. The Elementary School Journal, 101-128. 165 Wigfield, A., Guthrie, J. T., Perencevich, K. C., Taboada, A., Klauda, S. L., McRae, A., & Barbosa, P. (2008). Role of reading engagement in mediating effects of reading comprehension instruction on reading outcomes. Psychology in the Schools, 45(5), 432- 445. Willms, J. D. (2003). Student engagement at school. A sense of belonging and participation. Paris: Organisation for Economic Co-operation and Development. Woltman, H., Feldstain, A., MacKay, J. C., & Rocchi, M. (2012). An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology, 8(1), 52-69. Wylie, C., & Hodgen, E. (2012). Trajectories and patterns of student engagement: Evidence from a longitudinal study. In Handbook of research on student engagement (pp. 585-599). Springer US. Zhao, Y., & Hoge, J. D. (2005). What elementary students and teachers say about social studies. The social studies, 96(5), 216-221. Zipperer, F. M., Worley, M. T., Sisson, M. W., & Said, R. W. (2002). Literacy education and reading programs in the secondary school: Status, problems, and solutions. NASSP Bulletin, 86(632), 3-17. 166