SOCIAL STUDIES IN THE MIDDLE GRADES: EXAMINING CORRELATES OF STUDENT ACHIEVEMENT, INTEREST, AND INSTRUCTIONAL EXPOSURE By Corey Savage A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Educational Policy – Doctor of Philosophy 2018 SOCIAL STUDIES IN THE MIDDLE GRADES: EXAMINING CORRELATES OF STUDENT ACHIEVEMENT, INTEREST, AND INSTRUCTIONAL EXPOSURE ABSTRACT By Corey Savage Students across the globe are beginning to transition toward adulthood at a time of political, social, economic, and environmental turmoil. Recent years have seen policy, practice, and research increasingly favor reading, mathematics, and to a smaller extent science, at the expense of history, civics, geography, economics, and other social studies disciplines. Given the importance of knowledge and skills in social studies disciplines for desired civic and political outcomes and the issues young people and the world face moving forward, more attention to social studies and its underlying disciplines in discussions of educational effectiveness is crucial. Using eighth-grade 2014 NAEP data (civics, U.S. history, and geography) and employing structural equation modeling and multivariate logistic regression, this study estimated a) the relationship between students’ opportunity to learn (OTL) and teacher background characteristics on the one hand, and student achievement on the other, b) the relationship between student exposure to various instructional practices in social studies and student interest and c) the relationship between teacher background characteristics and student exposure to instructional practices in the classroom. Several significant relationships were estimated. For example, OTL was positively associated with student achievement in NAEP civics and U.S. history, an undergraduate focus in political science for students’ teachers was associated with increased student achievement in NAEP civics, student exposure to geographic information systems in geography was significantly associated with student interest as was exposure to community projects, and significant differences were estimated in student exposure to various instructional practices depending on teacher background characteristics and school sector. Other significant findings are also presented. Implications for policy and practice are discussed and areas of future research in this crucial substantive area are recommended. Copyright by COREY SAVAGE 2018 ACKNOWLEDGEMENTS I would like to thank my advisor, Anne-Lise Halvorsen, who was instrumental in facilitating this study, and her guidance and suggestions throughout were invaluable. Additionally, my co-chair, Spyros Konstantopoulos, provided excellent advice and guidance on the research design and method used in the study. All of my additional committee members, Margaret Crocco, Peter Youngs, and Gary Sykes, were exceptionally generous in lending their expertise and guidance, resulting in both a stronger study and an excellent learning experience. I am grateful to all others that have invested time and energy in my development as a researcher during my graduate studies. v TABLE OF CONTENTS LIST OF TABLES ....................................................................................................................... viii LIST OF FIGURES ....................................................................................................................... ix INTRODUCTION ...........................................................................................................................1 Defining Relevant Terms .....................................................................................................4 Study context: Social Studies in the Middle Grades ............................................................5 Educational Significance .....................................................................................................6 Outline of the Study .............................................................................................................8 LITERATURE REVIEW ..............................................................................................................11 Historical Background and Context of Social Studies Education .....................................12 Social Studies Student Achievement .................................................................................16 Teacher Background Characteristics, OTL, and Student Achievement in Social Studies 18 Teacher background characteristics .......................................................................19 Opportunity to learn ...............................................................................................23 Student Interest ..................................................................................................................25 Instructional Exposure and Student Interest ......................................................................27 Importance of Instruction for Student Outcomes...............................................................29 Student achievement ..............................................................................................29 Civic outcomes.......................................................................................................31 Predictors of Instructional Exposure ..................................................................................32 Summarizing the Relevant Empirical Literature ...............................................................34 METHOD ......................................................................................................................................36 NAEP Background.............................................................................................................36 Description of Sampling Procedures .................................................................................37 Assessments .......................................................................................................................38 Civics assessment...................................................................................................39 U.S. history assessment..........................................................................................40 Geography assessment ...........................................................................................40 Questionnaires....................................................................................................................41 Variables ............................................................................................................................41 Student, teacher, and school background variables ...............................................41 Opportunity to learn ...............................................................................................43 Instructional exposure ............................................................................................46 Student achievement and interest...........................................................................47 Analysis..............................................................................................................................49 RESULTS ......................................................................................................................................54 Research Question 1 ..........................................................................................................54 Teacher background characteristics .......................................................................54 vi Opportunity to learn ...............................................................................................56 Research Question 2 ..........................................................................................................56 Research Question 3 ..........................................................................................................58 Teacher background characteristics .......................................................................58 School sector ..........................................................................................................61 DISCUSSION ................................................................................................................................67 Summary of Results ...........................................................................................................67 RQ1: Student achievement ....................................................................................68 RQ2: Student interest .............................................................................................68 RQ3: Instructional exposure ..................................................................................69 Situating the Findings in The Existing Literature ..............................................................72 Student achievement ..............................................................................................72 Student interest.......................................................................................................74 Instructional exposure ............................................................................................75 Summary ................................................................................................................77 Limitations .........................................................................................................................78 Implications for Policy and Practice ..................................................................................79 Recommendations for Future Research .............................................................................81 Conclusion .........................................................................................................................83 APPENDICES ...............................................................................................................................85 APPENDIX A: NAEP Sample Questions (Grade 8) .........................................................86 APPENDIX B: Supplemental Tables ................................................................................96 REFERENCES ............................................................................................................................101 vii LIST OF TABLES Table 1. Selected characteristics of students, teachers, and schools (proportions) ........................42 Table 2. Summary statistics of OTL NAEP civics indicators .......................................................44 Table 3. Summary statistics of OTL NAEP U.S. history indicators..............................................45 Table 4. Summary statistics of OTL NAEP geography indicators ................................................45 Table 5. CFA estimates for latent OTL variables ..........................................................................46 Table 6. Student exposure to social studies instructional practices (proportions) .........................47 Table 7. Coefficients for achievement models ..............................................................................55 Table 8. Coefficients and odds ratios for interest models ..............................................................63 Table 9. Odds ratios for instructional exposure model (civics sample) .........................................64 Table 10. Odds ratios for instructional exposure model (U.S. history sample) .............................65 Table 11. Odds ratios for instructional exposure model (geography sample) ...............................66 Table B1. Coefficients for student and school covariates in achievement models ........................96 Table B2. Coefficients and odds ratios for student and school covariates in interest models .......97 Table B3. Odds ratios for student and school covariates in instructional exposure model (civics sample) ...........................................................................................................................................98 Table B4. Odds ratios for student and school covariates in instructional exposure model (U.S. history sample) ...............................................................................................................................99 Table B5. Odds ratios for student and school covariates in instructional exposure model (geography sample) ......................................................................................................................100 viii LIST OF FIGURES Figure 1. Simplified path diagram for achievement in research question 1 ..................................50 Figure 2. Simplified path diagram for interest in research question 2 ...........................................51 Figure 3. Simplified path diagram for instructional exposure in research question 3 ...................52 ix INTRODUCTION Adolescents in the United States and globally find themselves entering a unique, and arguably pivotal point in history. These young people are transitioning toward adulthood as society grapples with a host of both new and reemerging social, environmental, and political woes. Examples of these issues include increasing racial and ethnic tensions, global migration, and populist nationalism (Banks, 2017); increasing income disparities (Mohammed, 2015); increasing costs of education and student loan debt in the U.S. (College Board, 2017; Federal Reserve Bank of New York, 2018); (actual) fake news and attacks on the free press (Brown, 2018; Vosoughi, Roy, & Aral, 2018); increasing environmental disasters (NOAA National Centers for Environmental Information, 2018; UNHCR, 2015); and an intensifying global clean water crisis (World Water Assessment Programme, 2012). Although there have been examples throughout history of tremendous perseverance from younger Americans promoting change, recent statistics on voter turnout and voter registration suggest that the majority of today’s youth (and the general electorate, for that matter) are indifferent to participating in the formal democratic process (Pew Research Center, 2017b; The Pew Charitable Trusts, 2017). I recognize that several hopeful and inspiring social/political movements have emerged in recent years led largely by young people (e.g. Black Lives Matter, the 2017 Women’s March, and the March for Our Lives), but history will tell if these movements translated into more young people registering and exercising their right to vote, holding elected officials accountable at the ballot box. Other indicators of civic health have also been worsening. According to the National Conference on Citizenship, indicators of civic health include: “connecting to civic and religious groups; trusting other people; connecting to others through family and friends; giving and 1 volunteering; staying informed; understanding civics and politics; participating in politics; trusting and feeling connected to major institutions; and expressing political views” (Atwell, Bridgeland, & Levine, 2017). Examples of worsening conditions include increasing polarization politically (Pew Research Center, 2014, 2017a) and the decline of social capital, community engagement, participation in voluntary institutions, member-based organizations such as unions and religious groups, reading multiple daily newspapers, and social/political trust (Atwell et al., 2017; Hetherington, 2005; Putnam, 2000, 2016; Putnam & Feldstein, 2004). While some participation has shifted toward online social networking spaces, among others, many of the examples of participation listed above can bring about important benefits that other emerging types of participation do not always offer: they reach large numbers of citizens of varying age and are sustained over many years; allow people to hear from others with diverse viewpoints; are autonomous but connect to government and other formal institutions; and offer paths to leadership for some of their most active participants (Atwell et al., 2017). Schools are a primary venue for preparing young people to participate in both formal and informal democratic spaces and institutions (Levine, 2012). In the absence of an informed and engaged citizenry, we, by default, resort to a society in which the views and interests of the few outweigh those of the many. Citizens of every age need a foundation of knowledge, skill, and values to meet democratic goals, and this starts with our youth and the school subject and disciplines most directly tasked with such issues: social studies. The available empirical evidence suggests that schooling generally (e.g., Dee, 2004), and social studies education more specifically (e.g., Barton & Avery, 2016; Levine, 2012; Neundorf, Niemi, & Smets, 2016), can have positive impacts on political and civic participation and the acquisition of knowledge, skills, and values needed for effective political and civic engagement. 2 However, evidence also suggests that social studies has been neglected in policy and practice in recent history in the U.S., in favor of other core subjects, particularly among disadvantaged populations (Atwell et al., 2017; Dee, Jacob, & Schwartz, 2013; Fitchett & Heafner, 2010; Fitchett, Heafner, & Lambert, 2014; Halvorsen, 2013; Pace, 2011). Further compounding the problem, large-scale research on social studies education in the U.S., while growing, has not kept up with the research in other core subjects such as mathematics and English language arts, in part due to funding, among other causes/contributors (Camburn & Han, 2011; Halvorsen, 2013). Given the fragile state of civic health in the U.S. and democracies globally, focusing on educational effectiveness and other empirical topics concerned with improving social studies educational policy and practice is vital. This study contributes to current gaps in the social studies education scholarship by modeling the following outcomes: a) assessment outcomes in three of the primary social studies disciplines, b) student interest in their social studies coursework, and c) exposure to various instructional practices. Specifically, I address these gaps by answering the following questions: 1) To what extent are teacher background characteristics and students’ opportunity to learn associated with eighth-grade student achievement in civics, U.S. history, and geography? 2) To what extent is students’ exposure to various instructional practices associated with student interest in eighth-grade social studies? 3) To what extent are teacher background characteristics associated with students’ exposure to various instructional practices in eighth-grade social studies? 3 Defining Relevant Terms Prior to moving forward, I first define three important terms used throughout the study. These terms are instructional exposure, interest, and opportunity to learn, all centrally important constructs/groups of variables in this study. Defining instructional exposure is important merely to prevent confusion. In this study, instructional exposure is defined as student exposure to various instructional practices/approaches, individually measured and modelled (e.g., frequent lecture, exposure to community projects, and discussing current events). This is not to be confused with, for example, instructional time. In prior research, interest has been defined as “a unique motivational variable, as well as a psychological state that occurs during interactions between persons and their objects of interest, and is characterized by increased attention, concentration and affect” (Hidi, 2006, p. 70). As such, interest has both cognitive and affective components. The two general categories of interest in the literature include situational interest (i.e., environmentally triggered interest) and individual interest (i.e., interest developed over time and relatively enduring) (Hidi, 2006; Hidi & Renninger, 2006). The importance of the development of interest for both knowledge acquisition and student engagement is well-documented (Hidi, 2006). I focus in this study on two indicators of student interest, which are described further in the method section. Cross-national, cross-sectional studies have used the opportunity to learn (OTL) framework for measuring student curricular exposure for decades (Schmidt & Maier, 2009). These researchers categorized OTL into the intended curriculum (set by local or national standards), the implemented curriculum (what teachers actually report teaching), and the achieved curriculum (how students perform on an assessment aligned to the curriculum). This prior work, however, has focused primarily on other subject areas, most commonly mathematics. 4 I’ve focused this study on a construct closely related to the implemented curriculum, using teacher-reported content/thematic coverage (curricular themes aligned to the relevant assessment) to develop measures of OTL civics, U.S. history, and geography. I’ve also included measures of instructional time in model estimation, where applicable. Given the importance of OTL for achievement outcomes in mathematics (Brophy, 1986; Brophy & Good, 1986; Schmidt & Maier, 2009), the topic deserves more attention in social studies education. Study Context: Social Studies in the Middle Grades The middle grades (i.e., grades 5-8), when students are entering and transitioning through early adolescence, is a particularly important period for cognitive and social development. Specific to cognitive development, adolescents begin to experience increased attention, improvements in memory, improved deductive reasoning and relativistic thinking, and the development of metacognition (Anderman, 2012). Specific to social development, adolescents are forming their identity during these years (e.g., self-concept, self-esteem, and roles and responsibility in society) (Anderman, 2012). Said differently, adolescent brains are able to handle abstract and complex cognitive processes in ways that children often cannot, and their increased social development allows them to more fully consider their emerging roles within society. However, many have brought attention to the lack of intellectually challenging work and minimal effectiveness of middle grades education in the United States. In the journalism space, an article at The New York Times commented on the chaotic nature of middle schools (Gootman, 2007), and a special issue in Time magazine featured an article titled “Is Middle School Bad for Kids?” (Wallis, 2005), among other critiques. Providing an empirical lens, scholarship on adolescent development has found that a) adolescent motivation decreases and boredom 5 increases during the middle grades overall and in particular classes such as mathematics, science, and social studies and b) that rather than becoming more complex, academic work becomes less ambitious in the middle grades, in terms of cognitive demand (Eccles & Roeser, 2011). While nearly all U.S. states now have requirements for middle level teacher certification or licensure, there is substantial variation across institutions preparing teachers regarding access to courses and experiences specific to middle grades education and young adolescents (Howell, Faulkner, Cook, Miller, & Thompson, 2016). Scholars of social studies education have pointed to this lack of opportunity for middle grades teacher candidates as playing a primary role in the misalignment between middle grades social studies education and the needs of young adolescents (Conklin, 2008, 2009, 2012; Conklin, Hawley, Powell, & Ritter, 2010). Taken together, both public opinion and scholarship suggest that further research is needed in social studies in the middle grades regarding teachers and teaching, among other important lines of inquiry. Educational Significance Young people with higher levels of knowledge, skills, and interest in politics and society (among other attributes) tend to be more active in political and civic life (Barton & Avery, 2016; Carlson, 2012; Delli Carpini & Keeter, 1996; Neundorf, Smets, & García-Albacete, 2013; Verba, Schlozman, & Brady, 1995). The young people of Generation Z (one of many terms used to describe the generation following Millennials) are the future, and it’s our responsibility as researchers and educators to ensure they are well prepared to take on the serious societal, environmental, and political challenges outlined in earlier in this introduction. In order to do so, the field needs to ensure that policymakers, teacher educators, and K-12 educators/administrators have relevant/timely and empirically rigorous research to inform decision-making. There are 6 excellent examples of recent research (discussed in the literature review), which have helped to move the field significantly, and this trend will hopefully continue into the coming years. This study makes several unique contributions. For example, many of the previous studies in social studies education have been single-subject studies (primarily civics or history). As Barton and Levstik (2016) recognized in their recent review, integrating the discussion of research across several social studies disciplines brings important benefits, given the closely related content. This study begins to fill this gap in the literature. Furthermore, data used in recent large-scale social studies research has been primarily concerned with elementary and high school grades. This is in contrast to math and ELA, where both elementary and middle grades are researched fairly extensively, in large part due to the availability of assessment data. As will be discussed in the literature review, large-scale social studies research in the U.S. at the eighth-grade level has been primarily isolated to empirical papers using The IEA Civic Education Study (CivED) from 1999 and the National Education Longitudinal Study of 1988 (NELS:88). This grade level is actually quite common for cross- national assessments, but the United States has not participated in large-scale cross-national studies of civic education since 1999. This grade level is also common to be assessed in NAEP, occasionally also including fourth and twelfth grades, but the empirical work using NAEP social studies subjects has primarily focused on fourth and twelfth grades. This is not to say that there hasn’t been excellent qualitative research in the middle grades, some of which is discussed in this study. However, large-scale quantitative research is a necessary complement to the existing research, given the ability to answer different questions and make more broad generalizations to larger populations. 7 Additionally, NAEP data differentiates between different undergraduate and graduate training (e.g., history, political science, geography, social studies education, etc.), which allows for more fine-grained analyses of how teachers’ training is associated with both student achievement and reported instructional practices. As will be discussed in later sections, prior research has typically coded teachers as either having in-subject training (any social studies subject area) or not. Given the broad range of content from a number of disciplines that our social studies educators are expected to teach, and the fairly general requirements for becoming certified in social studies in the U.S., this is a very important area of further empirical study. It is also worth stressing the absence of recent, large-scale research in geography education, in particular (Bednarz, Heffron, & Huynh, 2013; Segall & Helfenbein, 2008). Geography education includes learning about a range of crucial societal topics such as climate change, demography, economic inequality, geopolitics, war/terror, and the geography of religion, among others (de Blij, 2012). The available research on geography education suggests that relatively little time is spent on geography in K-12 schools, the instruction typically lacks rigor, and teachers are ill-prepared to teach the content in K-12 schools (Bednarz et al., 2013; Segall & Helfenbein, 2008). Geography must receive increased focus in social studies classrooms as well as in research. This study adds needed evidence to the field of geography education. Outline of the Study The following sections take a traditional dissertation outline. First, I review the relevant extant literature, drawing primarily from developmental psychology, political science and economics, and social studies education literatures. Then, I discuss the methods and approach used in this study including the data sources, instruments, assessments, and analytic method. 8 After, I present the results of the study. I conclude with an in-depth discussion of the results including implications for policy and practice and recommendations for future research. In the literature review, I first discuss the historical background and context of social studies, with an emphasis on definition, purpose, controversies, and the policy context. Then, I provide some background on student achievement in social studies, the first of three groups of dependent variables in this study. Next, I discuss the relevant literature concerning OTL and teacher background characteristics as predictors of social studies achievement. From there, I turn to student interest, the second dependent variable, including discussion of the literature focused on instruction as a predictor of interest in social studies. Finally, I review literature discussing the importance of instruction as an outcome itself, given the relationship to important student outcomes, followed by an argument for the study of the relationship between teacher background characteristics and the use of particular instructional practices. The method section begins with providing background on NAEP and its uses. Then, I provide information on the specifics of the participants and sampling used for the NAEP samples utilized in this study. Next, I discuss specifics of the three NAEP assessments used in this study as well as the questionnaires and specific items used as variables in my analyses. I end with presentation of the specific modeling techniques used to answer the research questions. The results and discussion sections take a similar outline, presented by each research question and primary dependent variable. After summarizing the results at the beginning of the discussion section, I situate the findings within the extant literature base, identifying areas where my findings confirm, diverge, and expand on prior literature. Then, I present the limitations of the study. I conclude with implications for policy and practice and recommendations for future research. 9 I had various goals when undertaking this study. First, as I reiterate throughout this dissertation, the country and the world are at a pivotal point in history. While I wholeheartedly agree that literacy is a foundational goal of education and schools also need to do a better job at preparing students for an evolving workforce, preparing young people with the knowledge and skills needed to promote democratic discourse and a well-functioning civil society should be taken equally serious and not pushed aside. My hope is that what comes from this study will add to the growing literature base on the importance of teachers and teaching, not only in the more commonly discussed subjects of mathematics, reading, and science, but also in the social studies. Second, a major goal of this study was to model, with the available extant data, an extremely important construct in learning processes: student interest. This study has begun to demonstrate the importance of varying instructional approaches for student interest, and these findings should spur future research on student interest in social studies (ideally with better measures) as it relates to student engagement in the classroom as well as future civic and political engagement, among other outcomes. Finally, as previous qualitative research as well as this study have suggested, varying teacher background characteristics related to their preparation and experience may be related to varying instructional approaches reported by students’ teachers, approaches themselves related to a set of desired outcomes. In addition to being a catalyst for further needed research, this study should send a message to educational stakeholders that the field needs to consider more fully the impacts of the broad range of possible training experiences for social studies teachers in the U.S. 10 LITERATURE REVIEW Across each of the major areas of empirical focus of this study, the literature base in quite limited. There has been significant growth in U.S. social studies research in recent years, especially on the relationship between instruction, student achievement, and civic outcomes. However, the field still lags behind research on other school subjects, particularly with respect to important lines of inquiry such as opportunity to learn (OTL), teacher/teaching quality, geography education generally, the varying disciplinary backgrounds of social studies teachers, and student interest. I take a small step in advancing the field in these areas. Prior to discussing the relevant empirical literature, this section first addresses the historical background and context of social studies education in the United States, which has been an area of much contention. Then, I provide background and trends on social studies achievement, the first of several focal outcomes in this study. Following this overview, I discuss the relevant empirical evidence regarding the relationship between teacher background characteristics and students’ OTL on the one hand, and student achievement on the other. In the sub-section that follows, I provide an overview on student interest generally as a psychological construct, the second focal outcome in this study. I expand on this discussion of student interest by reviewing the limited research on how a crucial aspect of teaching (i.e., instructional approaches) is related to interest. Given the evidence on student boredom and decreased motivation in the transition to adolescence in the middle grades (Eccles & Roeser, 2011), expanding on this literature base in social studies is necessary. After this review, I transition to the importance of instruction for a range of student outcomes including student achievement and civic outcomes, in addition to the previously discussed research on student interest. I do so in order to highlight the significance and 11 consideration of instructional approaches as proximal outcomes (or as mediators with data where this is possible/meaningful), rather than merely predictors/independent variables. I briefly acknowledging the literature base in social studies that has treated instruction as an outcome, specifically with discussion of the civic opportunity gap for students. Finally, I stress the importance of analyzing the relationship between the characteristics of teachers and the instructional practices they employ in their teaching of social studies, which currently is a large void in the literature base and a crucial area of empirical study. Historical Background and Context of Social Studies Education Social studies as a school subject has a particularly interesting history, with disagreement around both its definition and purpose (Barton & Avery, 2016; Thornton, 2017). Before getting into this contentious space, I first describe the general structure of social studies education in the US. In the early grades, the U.S. curriculum most commonly involves a general study of communities and social life, expanding incrementally from the local community to macro-levels as students advance through elementary school (Halvorsen, 2013). In the middle grades, students are increasingly exposed to the disciplines that fall under the umbrella of social studies: history, civics/government, geography, and economics, and other social science disciplines to a lesser extent (Barton & Avery, 2016). In high school, courses are almost always distinct disciplines or sub-disciplines (e.g., U.S. history, world history, government, geography, sociology, and economics) (Barton & Avery, 2016). There is a general consensus among social studies educators that content in the underlying disciplines of social studies is closely related (Barton & Avery, 2016). Over the years, some have argued for doing away with general social studies classes in earlier grades in favor of coursework in individual disciplines, but these movements have been largely 12 unsuccessful (Barton & Avery, 2016; Thornton, 2017). Regardless of course structure, there is significant contention concerning what should be taught, how, and from what perspective. Some of this contention has been political in nature, often arising at times of focusing events. Recent focusing events include the aftermath of September 11th, the restructuring of standards at state or national levels, or the revisions of the Advanced Placement history assessments (Andrews & Warren, 2018; Finn Jr., 2003b). Taking the aftermath of September 11th as one example, former U.S. Assistant Secretary of Education, professor of education, and President of the Thomas B. Fordham Foundation/Institute, Chester E. Finn Jr., lambasted the state of social studies education, which highlighted a common view among conservatives in the U.S. He described the social studies establishment as those who possessed no respect for Western civilization; who were inclined to view America’s evolution as a problem for humanity rather than mankind’s last, best hope; who pooh- poohed history’s chronological and factual skeleton as somehow ‘privileging’ elites and white males over the poor and oppressed; who saw the study of geography in terms of despoiling the rain forest rather than locating London or the Mississippi River on a map; who interpreted ‘civics’ as consisting largely of political activism and ‘service learning’ rather than understanding how laws are made and why it is important to live in a society governed by laws…(Finn Jr., 2003a, p. I) These controversies also occur in less ideological/political spaces, within individual disciplines related to social studies. For example, drawing on disciplinary ways of knowing, many scholars of history education advocate for teaching historical thinking, which includes mastery in the following concepts: establishing historical significance, using primary source evidence, identifying continuity and change, analyzing cause and consequence, taking historical 13 perspectives, and understanding the ethical dimension of historical interpretations (Monte-Sano & Reisman, 2015). This approach contrasts with simply teaching for factual knowledge or with teaching history for patriotism (Barton & Avery, 2016). Similar tensions exist related to the teaching of government/politics/civics, specifically concerning what constitutes education for democracy, conceptions of the good citizen, and the extent that coursework should move beyond basic concepts of government and law (Barton & Avery, 2016; Westheimer & Kahne, 2004). Geography education, given the broad scope of geography as a field, also has varying perspectives among social studies educators and geographers regarding what should be taught in schools and how (Bednarz et al., 2013). As Evans (2004) discusses at length in his book The Social Studies Wars: What Should We Teach the Children?, these disputes have continuously resurfaced over the last century. From his perspective, there have been five contending camps in the social studies wars. These, as he discusses, included a) traditional historians, who support history as the core discipline of social studies and advocate for teaching content and chronology; b) those at the heart of the new social studies movement in the 1960’s who have advocated for the teaching of social sciences, in addition to history; c) those described as social efficiency educators who have advocated for a curriculum aimed at preparing students for specific roles in society; d) social meliorists, described as Deweyan experimentalists who aim to develop students’ reflective thinking abilities, with an emphasis on social problems, in turn contributing to societal improvement, and e) social reconstructionists/critical pedagogues who view social studies education as a catalyst for social justice and transforming society (Evans, 2004). Social studies education also currently sits within a unique policy space. Social studies is the one major school subject that continues to have no federal requirement (in exchange for Title 14 I funding) for yearly testing in public schools. The focus of federal law regarding yearly testing primarily concerns mathematics and English language arts (ELA) as well as science, to a smaller extent. This federal policy climate does not prevent states or districts from requiring yearly summative assessments in social studies. However, nearly half of the U.S. states do not, and in those that do, the disciplines within the social studies that are tested at a large scale are inconsistent across states (Education Commission of the States, 2016; Grant & Salinas, 2008). In light of this policy climate, existing evidence suggests that social studies and its disciplines have largely taken a backseat to other subject areas throughout recent history (Halvorsen, 2013), and increasingly in favor of other tested subjects (Dee et al., 2013; Fitchett & Heafner, 2010; Fitchett et al., 2014; Pace, 2011; VanFossen, 2005; Vogler et al., 2007). This current, unfortunate state of social studies educational policy (i.e., low priority relative to other school subjects) is one that those across the ideological spectrum are likely to find troubling. While I acknowledge the continuous controversies in social studies education, my primary purpose from here forward is an empirical one. A preponderance of the existing evidence suggests that having students engage and learn within and/or across the disciplines most directly related to social studies contributes to students’ ability to engage effectively in civil society and also their interest in participating (Barton & Avery, 2016; Carretero, Haste, & Bermudez, 2015), outcomes that those of any concerned disciplinary background or ideology should be able to agree are positive. I’ve embarked on this study with these goals in mind, considering how varying inputs in social studies education can contribute to educational outcomes and potential variation in student instructional exposure. Later sections review the empirical work in this area. I focus primarily on civics, U.S. history, and geography, given the 15 focus of my study and availability of data. I now turn to the available evidence regarding proficiency levels and trends in student social studies achievement in recent years. Social Studies Student Achievement Assessments of knowledge and skills, among other measures of academic achievement, provide one way of estimating what students know in a given subject area. Following these trends over time for students overall, as well as sub-groups of students including historically disadvantaged populations, is an important component of quality assurance in education. While the focus of this study is not on group differences in student achievement, providing available background/trends is important for context as well as for demonstrating the importance of controlling for student characteristics in empirical models of social studies achievement. I focus briefly in this section on the available trends of assessment outcomes in social studies, particularly at the grade level of focus in this study. Given much of the available evidence is related to politics and civics, I begin with evidence in this area. Citing data from recent decades, scholars have debated how knowledgeable U.S. citizens are with regard to politics and civics (Galston, 2001; Levine, 2012; Lupia, 2016; Niemi, 2012), often coming to different conclusions and interpretations. For example, citing research and NAEP trends, Niemi (2012) argues that students continue to lack the necessary knowledge to contribute to civic life. Levine (2012) is more optimistic and argues that American students compare well with other nations in the CivED study from 1999, at least in some topic areas. Levine does, however, acknowledge that there is a gap in outcomes on the basis of race, ethnicity, and socioeconomic status (Levine, 2012). As Niemi argues, it is difficult to ignore that a large majority of students at every grade level continue to test below proficient in NAEP civics. NAEP civics is not a perfect assessment 16 and may not measure all aspects of civics that scholars deem important (Levine, 2012). Nonetheless, only 23% of students in eighth grade were at or above proficient in civics in 2014 (which, in fact, are not arbitrarily set cut points). As a point of reference, ~33 – 34% of eighth- grade students in 2014 were at or above proficient in math, reading, and science. Additionally, race, ethnicity, and socioeconomic status (as in other subject areas) are significant predicators of student performance on both national and, in the case of socioeconomic status, international assessments of civic knowledge and skills (Barton & Avery, 2016). On a positive note, scores have increased for lower and middle performing students on NAEP civics since 1998. There has also been an increase in average NAEP civics scores in eighth grade since 1998 for Hispanic students, and the gap between non-Hispanic white students and Hispanic students has narrowed.1 Regarding gender, mixed evidence is available. At earlier grades, females in the U.S. tend to perform better on tests of civic knowledge. Assessments of older students in the U.S. and cross-nationally show females may outperform males on tests of civic skills but not knowledge or overall performance, and a more recent cross-national study found females outperformed males overall at the eighth-/ninth-grade level (Barton & Avery, 2016). The conflicting findings could be due to differences in assessment item format or content (Barton & Avery, 2016; Hahn, Bernard-Powers, Crocco, & Woyshner, 2007). Recent evidence in other subject areas also suggests gender differences in performance on different item types (Reardon, Kalogrides, Fahle, Podolsky, & Zárate, 2018). The empirical research evidence on gender in social studies is sparse and deserves more attention (Hahn et al., 2007). 1 Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1994–2014 U.S. History, Geography, and Civics Assessments. 17 Recent evidence on student achievement in U.S. history and geography is less abundant, but NAEP again provides some insight. There have been increases across the performance distribution in U.S. history since 1994, and the non-Hispanic white/Hispanic achievement gap has narrowed. Still, only 18% of students overall tested at or above proficient in 2014, the lowest percentage (descriptively) across the three subjects, and the gaps that exist by race, ethnicity, and socioeconomic status are still substantial. In geography, average scores and percent at or above proficient are unchanged in 2014 relative to 1994; 27% of students test at or above proficient. Achievement gaps by race and ethnicity have narrowed slightly, but substantial gaps are still present as well as between students in poverty and their peers.2 Smaller gaps consistently favor males over females in U.S. history and geography assessments, but again, these differences could be, in part, due to test format and question types (Barton & Avery, 2016). While the discussion of student achievement in social studies above is not exhaustive, large-scale data on student achievement in social studies disciplines in the U.S. is limited relative to other school subjects. The existing evidence suggests that American students, on average, are limited in their knowledge and skills relative to other core subject areas. Additionally, there are significant gaps across student sub-groups, as is true in other subject areas. I now turn to specific inputs in social studies education and the existing evidence regarding their relationship to student achievement. Teacher Background Characteristics, OTL, and Student Achievement in Social Studies Beyond descriptions of student knowledge and skills in the social studies disciplines, a fairly extensive literature base has explored the relationship between instructional approaches 2 Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1994–2014 U.S. History, Geography, Civics, Mathematics, Reading, and Science Assessments. 18 and student achievement (e.g., Barton & Avery, 2016). This literature is discussed in more depth in later sections. However, two additional lines of inquiry that have received extensive attention in other subject areas have been minimally studied in social studies: a) teacher background characteristics, b) opportunity to learn (the implemented curriculum in K-12 classrooms), and the relationships of these variables to student achievement. The minimal existing evidence in social studies is mixed but suggests varying student achievement outcomes with respect to variation in teacher background characteristics and students’ opportunity to learn in the classroom. Teacher background characteristics. Indeed, the relationship between teacher background characteristics and student achievement has been studied extensively in the context of math and reading (Goldhaber, 2015; Tatto et al., 2016; Wayne & Youngs, 2003), but research on this area in social studies is minimal and quite dated. Exceptions include a small number of studies using NELS:88 (National Educational Longitudinal Study of 1988), CivED (Civic Education Study of 1999), and, more recently, North Carolina administrative data. While other characteristics of teachers may be of interest, I focus here primarily on teachers’ subject matter preparation, alternative certification, and National Board certification, a voluntary form of advanced certification. Economists, developmental psychologists, and social studies educators have studied the extent that subject matter preparation of teachers contributes to student achievement in social studies, with mixed findings depending on grade level and methodological approach. Goldhaber and Brewer (1996) used NELS:88 data to estimate the relationship between in-subject certification/in-subject degrees for teachers and student achievement in math, science, history and English in tenth grade. The authors employed a standard OLS regression technique with cross-sectional data (tenth graders in 1990), controlling for characteristics of students, 19 classrooms, and schools. They found that being certified in math or having an undergraduate/graduate degree in math or science was associated with higher student test scores in math and science, respectively, but these findings were not true for English or history. More than a decade later, Dee and Cohodes (2008) took another look at the NELS:88 data with similar line of inquiry. The authors chose to look at eighth grade, the grade level of focus in this study, to utilize a unique feature in the data: assessment outcomes on two subjects for each student and data on their teachers. Using a matched-pairs analysis to eliminate bias due to non-random sorting of students to teachers (and also including student and school fixed effects as well as other controls), the authors found that in-subject certification was associated with higher test scores in math and social studies, but not English or science. The contradictory findings between these two studies could be due to the different grade levels, different assessments (history in tenth grade versus a broader social studies assessment in eighth grade including history, citizenship, and geography) and/or the different methodological approaches. Evidence from the CivED study also suggests some importance of teachers’ training for student achievement outcomes in civics, specifically. Torney-Purta, Richardson, and Barber (2005) estimated a multilevel model using U.S. data to estimate the relationship between teachers having a civics-related degree and/or in-service training in civic education and eighth- grade student achievement in civics, controlling for books in the home at the individual and school levels. Relative to students whose teachers had neither a civics-related degree or in- service training, the authors found the following: no significant difference for teachers having only a civics-related degree; a significant positive difference for having in-service training only; and an additional significant positive difference for teachers with both. Their findings suggest in- service preparation as being more important; however, both variables are quite broadly defined. 20 A recent publication using 2010 NAEP U.S. history data also provides evidence to this topic. Fitchett and Heafner (2017) analyzed the relationship between teachers’ subject background (among other variables) and students’ U.S. history achievement in fourth grade. They found subject matter background in any social studies related discipline to be significantly related to history test scores when not accounting for instruction, but this relationship no longer held when including measures of instruction in their model. The findings from the limited existing evidence are mixed, but some common themes are present across all studies. One is a very broad definition as to what constitutes in-subject preparation. Each study considers any social studies-related discipline to be in-subject, which makes sense for policy reasons as any related subject would qualify as “in-subject” for certification purposes in the U.S. However, analyzing the extent that teachers’ training in individual disciplines is related to assessment outcomes in different social studies disciplines is an important empirical question. This point was eloquently presented in a qualitative piece from the late 1980’s, discussed in more depth in a later sub-section (Wilson & Wineburg, 1988), but this line of inquiry has not received enough empirical attention since then. Furthermore, none of the studies account for what content is actually taught in the classroom. I’ve attempted to begin exploring these issues in this study. Various forms of initial certification (i.e., traditional versus alternative routes) as well as advanced professional certification are also commonly studied in the context of student achievement in other school subject areas (e.g., Cowan & Goldhaber, 2016; Goldhaber, 2015; Tatto et al., 2016), where findings suggest that selective alternative-certification pathways as well as the voluntary National Board (NBPTS) certification for teachers predict increases in 21 student achievement. These topics have been studied minimally in social studies, primarily in one U.S. state and only in high school. For example, researchers in North Carolina have recently published work related to various pathways into teaching. Since North Carolina assesses its high school students in social studies as part of their state assessments (the assessment focuses on U.S. history, civics, and economics), assessment outcomes in social studies were part of their analyses. In the first of two articles from a larger project (Henry, Bastian, et al., 2014), the authors used a school fixed effects approach with panel data to analyze the effects of various routes into teaching on student achievement: alternative versus traditional, Teach for America (TFA) versus traditional, out-of- state versus in-state, graduate versus undergraduate degree, and public versus private university/college. The authors used an extensive set of student and teacher/classroom covariates including curricular variables as well as whether the teacher was teaching in-subject. While significant effects were found when comparing paths in other subject areas, no significant effects were found in high school social studies. Taking a slightly different modeling approach in the second article (including further disaggregated dummy variables, with in-state public as the reference group), the authors did find significant effects in high school social studies (Henry, Purtell, et al., 2014). Teachers trained out-of-state with only an undergraduate degree and alternatively trained teachers (except TFA) were both slightly less effective than teachers trained at an in-state public university (the most common pathway into teaching), although the effects were quite small. TFA teachers, on the other hand, were slightly more effective that those from an in-state public university, with an effect size of 0.09 (small but educationally meaningful). The further disaggregation in this second study likely highlighted differences that were masked in the first study review. 22 The results from these studies suggest that initial certification pathways for teachers can have an impact on the social studies achievement of their high school students. From a policy perspective, this lends important evidence regarding quality assurance in teacher education. However, the underlying cause to estimated differences in student achievement isn’t apparent. Identifying whether teachers from different pathways have varying teacher knowledge, non- cognitive abilities, or use different instructional practices, for example, would be beneficial to research and practice. I begin to address some of these gaps in this study. Several gaps are present in the social studies research on teacher certification, with relation to student outcomes. The studies cited above are in one state and in high school only. Research on similar topics with national data in middle grades social studies and on multiple assessments is necessary. Additionally, little to no evidence exists regarding the voluntary advanced certification from the National Board of Professional Teaching Standards, specific to social studies. As is the case in much of the literature I discuss throughout the literature review, the inclusion of private schools in analyses is also rare. I partially address these gaps in this study. Opportunity to learn. OTL (i.e., student curricular exposure) has been studied in other subject areas with respect to student achievement, most extensively in mathematics. For example, cross-national research on OTL mathematics conducted by IEA has focused primarily on curricular exposure as it relates to student achievement (or what has also been referred to as the achieved curriculum), finding that increased OTL is associated with increases in student math achievement (Schmidt & Maier, 2009). Evidence in this line of inquiry is largely unavailable in social studies. 23 One exception was a study on U.S. history from 2001. Using the 1994 NAEP U.S. history data and the 1994 High School Transcript Study, J. B. Smith and Niemi (2001) modeled the relationship of curriculum and instruction to student achievement. The authors found that students who took more and higher-level coursework and students who reported greater coverage of a range of history topics performed better on the NAEP test. While not referred to as OTL in their study, the authors did, in fact, examine the relationship between increasing content/thematic coverage and student achievement. However, the measures were specific to prior course taking and prior content coverage since ninth grade and not necessarily specific to the current teacher or assessed year. In another piece from the second author using 1988 high school NAEP civics data (Niemi & Junn, 1998) the authors also explore the role of curriculum (in addition to instruction, among other topics) and its relationship to NAEP civics scores. At the time, this book was an enormous step toward understanding the state of civic education in American high schools and shined a necessary light on such an important area of research. The authors reported positive relationships between curricular exposure and NAEP civics achievement. However, several methodological issues were present including comparisons of descriptive means and coefficients across models without appropriate statistical tests and the modeling of 3- and 4-point Likert items as continuous covariates in regression models in many cases, which assumes these variables are on an interval scale, which can be problematic. Overall course-taking, studies of particular units or curricular programs, and high-level historical/chronological periods have also been researched to an extent, and other studies have used opportunity to learn to describe exposure to instructional approaches (Carlson, 2012; Feldman, Pasek, Romer, & Jamieson, 2007; Heafner & Fitchett, 2015; J. B. Smith & Niemi, 24 2001; Zhang, Torney-Purta, & Barber, 2012). However, OTL as it is most commonly defined and measured, and the relationship to student achievement outcomes in social studies has been studied very little. Using this prior research in other subject areas as a guide, I address the gap in the social studies literature regarding this extremely important construct. Given the importance in other subject areas, the study of teacher background characteristics, opportunity to learn, and student achievement in social studies in the middle grades demands further empirical inquiry. The minimal existing evidence highlights varying student achievement outcomes with respect to variation in teacher background characteristics and opportunity to learn in the classroom. This study begins to fill in gaps by including: specific disciplinary background of teachers, private school students, sophisticated OTL measures, and the use of recent data in eighth grade. Student Interest While related, student engagement and interest are distinct constructs. Student engagement is a multidimensional construct including several behavioral, cognitive, and affective components that measure students’ interactions within learning environments, whereas interest is one’s psychological state during engagement and the motivational dispositions of a person to engage with particular content (Hidi, 2006; Renninger & Hidi, 2016; Sinatra, Heddy, & Lombardi, 2015; Sinatra, Mukhopadhyay, Allbright, Marsh, & Polikoff, 2017). Student interest (characterized by increased attention, concentration and affect) in its various stages from situational to individual/sustained interest, is an important cognitive and affective construct for promoting student engagement and the acquisition of competence, in its various dimensions (Hidi & Renninger, 2006). 25 Evidence is somewhat sparse related to trends and subgroup differences for student interest in social studies in the U.S. As mentioned previously, boredom (a similar, negative affective state) increases for students between the elementary and middle grades in school generally as well as in social studies, among other subjects (Eccles & Roeser, 2011). The minimal available evidence on differences by gender suggests that males are more interested in their social studies coursework than females (Hahn et al., 2007), which is particularly thought- provoking given females have been found to be more engaged than male students (based on a composite measure, which included interest among other indicators) in elementary, middle, and high school, combining both mathematics and social studies3 (Marks, 2000). With regard to other student sub-groups, research on various topics/themes (not necessarily specific to school topics or social studies class itself) provides some context. For example, prior large-scale U.S. and international survey research suggests that students of lower socioeconomic status are less interested in politics and less likely to anticipate participating in voting (Barton & Avery, 2016). Marks (2000) also found students of higher socioeconomic status to have higher levels of engagement at every grade level based on her composite measure, which included indicators of interest. No differences were found by race or ethnicity. Qualitative research in the U.S. also provides helpful perspective as to why these differences may exist. Evidence suggests that less affluent students and those from ethnic and racial minority groups are interested in participating in social change but in potentially different ways than their affluent peers. This evidence also suggest that they may perceive a disconnect 3 Unfortunately, the analyses weren’t done separately by subject. One of the models in the study did include a dummy variable for subject area, which found that conditional on a number of covariates, students were more engaged in math than social studies at the elementary and high school levels, but not at the middle grade level. 26 between what is taught about civics in schools and what they experience in their daily lives (Levinson, 2012; Rubin, 2007). Instructional Exposure and Student Interest Survey evidence suggests that among the mostly widely reported instructional approaches in social studies education include lectures, worksheets, use of the social studies textbook, and discussion, broadly defined (Barton & Avery, 2016; Levstik, 2008; Torney-Purta & Richardson, 2003). Where much of the research on instruction as a predictor of student outcomes in social studies has focused on student achievement and civic outcomes (discussed further in the next section), research on the relationship between instruction and interest is understudied. I focus on the limited research base considering student interest in relation to instructional exposure. Although some of the educational literature has considered interest as a component of engagement rather than a distinct construct itself, this literature is important to review given the inclusion of interest in analysis of student engagement. For example, Stevenson (1990) interviewed 45 students from five high schools on three separate occasions in the 1986-87 school year. The students also completed surveys on the second and third visits. Social studies teachers nominated students from the top, middle, and bottom third in academic achievement, and students were asked about engaging and challenging experiences in both social studies and in school generally. To probe engagement, students were asked about situations where they “put forth their best effort” and where they “were so interested that time passed more quickly than usual.” Follow up questions were given in interviews to understand reasons behind their answers. The data from the interviews was coded inductively. When asked about lessons students found interesting, a majority of students reported examples that enabled them to actively participate in analytic thinking about abstract ideas, inductive reasoning, or evaluation of ethical issues. 27 Instructional approaches used in the examples given by students included writing, class discussions, debates, roleplay, and creative tasks. Results also suggested that students were more interested in actively discussing ideas and values, rather than only focusing on specific facts and generalizations. Various forms of inquiry-based methods, such as authentic intellectual work and use of computer-based simulations, have been analyzed with respect to student interest (distinctly or embedded within measures of student engagement). For example, authentic intellectual/academic work, defined as “the construction of knowledge through disciplined inquiry to produce discourse, products, or performances that have value beyond school,” has positive influences on student engagement in social studies (broadly defined, including measures of interest) (King, Newmann, & Carmichael, 2015, p. 54; Marks, 2000). Regarding computer simulations, Gehlbach et al. (2008) used a pre-post design to examine the impact of a web-based GlobalEd simulation in the middle grades, which uses roleplay and has students negotiate treaties related to current world issues. While there was no comparison group in the study, the authors found that interest in social studies increased for students after the simulation. International evidence also sheds some light on the relationship between teaching and student interest in social studies. Del Favero et al. (2007) used an experimental approach with 100 Italian eighth graders with one intervention group (problem-solving through discussion) and a control group (focused on individual problem-solving). The authors estimated the effect of the intervention (i.e., discussion-based problem solving) on students’ learning of two historical topics, interest, and self-perception of competence in history. Related to interest, they found that the intervention resulted in greater situational interest. In turn, structural equation models showed 28 that situational interest impacted both students’ individual interest and self-perception of competence in history. The existing evidence suggests that instructional exposure of various types can have positive associations with student interest. However, this research in somewhat limited to a few forms of instruction, small samples, older data, and interest is typically not treated as a distinct construct but rather a component of student engagement. Estimating relationships between a range of various instructional practices and student interest would be beneficial to the field. Importance of Instruction for Student Outcomes Before turning to the literature on predictors of instructional exposure, I first review the literature on the importance of instruction in social studies education. In addition to the scholarship discussed above regarding the relationship between instruction and student interest, I review literature concerned with the relationship between instruction and two other groups of outcomes with fairly extensive literature bases: student achievement and civic outcomes. This review is not exhaustive, but rather meant to highlight the importance of exposure to various forms of instruction, and in turn, the study of instruction as a proximal outcome itself. Student achievement. As mentioned in the previous section, survey evidence suggests that the mostly widely reported instructional approaches in social studies education include lectures, worksheets, use of the social studies textbook, and discussion (Barton & Avery, 2016; Levstik, 2008; Torney-Purta & Richardson, 2003). Using surveys, observations, and assessment data (pre/post and using short-response and essays) from a diverse range of 16 Midwestern schools (both public and private), Gamoran and Nystrand (1991) analyzed the relationship between several of these instructional practices (except worksheet use) and student achievement in eighth-grade English and social studies. They found that time spent on lecture, question- 29 answer, and discussion were positively related to achievement in both subjects, and that these instructional practices partly explained the gap in achievement between students of varying socioeconomic status. The study of discussion-based instructional approaches is among the most widely studied topics within social studies scholarship. Cross-national civic education studies have found that student reports of classroom discussion are associated with student civic knowledge among other civic outcomes (Schulz, Ainley, Fraillon, Kerr, & Losito, 2010; Torney-Purta, Lehmann, Oswald, & Schulz, 2001). Evidence from these same cross-national studies, as well as national and smaller-scale studies, also have uncovered the importance of both an open classroom climate and discussion of controversial issues for student achievement, among other outcomes (e.g., Campbell, 2008; Hess & McAvoy, 2014). Several studies have also found that inquiry-oriented approaches can have positive benefits on assessment outcomes in social studies. For example, Parker et al. (2013) employed design-based research methods to study the effects of project-based learning in AP U.S. Government courses. These projects involved the use of simulation and roleplay. Studying 12 AP classes across 4 schools, the authors found evidence for positive effects on student AP scores, relative to comparison AP classes. In geography, a small literature base found that using geographic information systems (GIS) as part on an inquiry-based project, can improve students’ content knowledge and geography skills (Bednarz et al., 2013; Shin, 2006, 2007) Several scholars have taken qualitative and mixed-methods approaches to the study of writing in social studies. Potential benefits of historical and argumentative writing in the context of history education, civics, and social studies include the development of historical reasoning 30 and thinking as well as improved civic skills and intended civic participation (Monte-Sano, 2011; Monte-Sano & De La Paz, 2012; Rubin, 2012). Among other foci of analysis, Carlson (2012) estimated the effect of exposure to several instructional practices in social studies on NAEP civics scores. Using a propensity score approach with 2006 NAEP civics data, the author found that exposure to mock trials/role play was positively related to civics test scores in eighth but not fourth grade and that exposure to debate in the classroom was positively related to civics test scores in fourth but not eighth grade. Civic outcomes. A large body of evidence suggests that in addition to academic outcomes, instructional exposure is also an important predictor of civic outcomes such as political interest, efficacy, tolerance, trust, and participation (Barton & Avery, 2016). For example, an evaluation of Student Voices (implemented in 22 Philadelphia high schools) found that class discussions, community projects, and informational use of the internet were linked to political participation in various forms (Feldman et al., 2007). Using two-wave survey data from high school students in Chicago and California, Kahne, Crow, and Lee (2013) also analyzed the effect of various instructional approaches on political participation. They found that open discussion, measured as “the extent to which, during the past year, students learned about, researched, and discussed contemporary social problems and the extent to which teachers encouraged students to make up their own minds on social and political topics” (Kahne et al., 2013, p. 426) specifically promoted engagement with political issues and elections (“Big P” politics”), whereas providing service learning opportunities increased community-based actions (“Little p” politics). Both forms of instructional experiences promoted participatory citizenship, “the belief that being concerned about and actively involved in community, state, and national issues is everybody’s responsibility” (Kahne et al., 2013, p. 425). 31 An earlier study using data on more than 4,000 high school students in Chicago also analyzed the impacts of instructional approaches in civic education on civic participation (Kahne & Sporte, 2008). The authors developed a scale for what they referred to as classroom-based civic learning opportunities, which included the extent that students reported learning about problems in society and current events, studying issues they care about, experiencing an open classroom climate, hearing from civic role models, and learning about ways to improve the community. An additional predictor of interest was related to students’ opportunities to participate in service-learning projects in the community. Controlling for an extensive set of covariates, the authors found that both the civic learning opportunity scale and service learning were positively associated with commitment to civic participation (a multi-item scale). While more causal studies are needed in this substantive area, the existing evidence suggests that instructional practices are important for both student achievement and civic outcomes, in addition to the limited available evidence estimating the relationship between instruction and student interest. Taken together, exploring variation in instructional exposure for students and variables that predict such variation is a worthy endeavor. Predictors of Instructional Exposure As I outline in this section, the literature base on predictors of instructional exposure has focused primarily on the characteristics of students. However, the characteristics of teachers (and schools, for that matter) in relation to instruction has not received much attention. I first discuss a selection of the literature that has focused on student predictors, then transition to the minimal research looking at teacher-level predictors of student instructional exposure in social studies. Indeed, large-scale research on predictors of instructional exposure have focused primarily on student characteristics. Prior research on civic education in the U.S. has found, in 32 general, that white students and those from higher socioeconomic backgrounds have more civic opportunities afforded to them (e.g., discussing current events, service learning, simulations, making speeches, etc.) (Kahne & Middaugh, 2008), defined as a civic opportunity gap. With respect to history education, in particular, J. B. Smith and Niemi (2001) used path analysis to predict both content coverage and exposure to “active instruction” in twelfth-grade U.S. history classes. The authors found similar findings to the work on the civic opportunity gap. The research on the relationship between pre-service teacher training and the instruction used by teachers has tended to be small-scale with convenience samples (Crocco & Livingston, 2017). One excellent qualitative study from the late 1980’s still provides insight today. Wilson and Wineburg (1988) followed several social studies teachers during their training and practice, highlighting important differences in knowledge and approaches to their practice, in the context of teaching history. These differences, they argued, were due to differences in their disciplinary background and training, tending to approach the teaching of history in ways that reflect their background (e.g., geography, history, political science, anthropology, economics, etc.). In other words, social studies teachers may approach any content across social studies disciplines specifically in the ways that they know how or were trained. With respect to novice/expert differences, one recent large-scale empirical study included a secondary objective in determining whether years of experience was related to two factors/scales (“teacher-text” and “collaborative-research”) measured by teacher-reported instruction (Knowles, 2018). The authors found no significant differences across years of experience for these scales. However, the scales included a fairly large assortment of instructional practices, which potentially masked differences in teacher-reported instruction by 33 years of experience. I address this issue in this study by estimating differences for distinct instructional practices. Readily observed measures of teachers’ background (e.g., training, certification, and years of experience) and the relationship to student instructional exposure in social studies at a large scale has, quite surprisingly, been mostly overlooked in recent literature. This is particularly important in social studies given social studies teachers often come from different backgrounds (e.g., history, political science, geography, social studies education), and, as discussed above, prior qualitative work suggests that teachers with different backgrounds may indeed approach teaching social studies differently (Wilson & Wineburg, 1988), which to my knowledge, has never been studied at a large scale. Further analysis of additional characteristics of teachers such as initial certification, advanced voluntary certification, and years of experience, is also an important area of research. While not a primary focus of this study, the examination of variation in instructional exposure for students of different school types/sectors has also been overlooked. Given prior research suggesting varying academic and civic outcomes for students of different school sectors (Campbell, 2012) and the relationship between instruction and both academic and civic outcomes (discussed above), this is an area in need of far more research and is potentially one missing piece in the puzzle that continues to perplex those concerned with the topic (Campbell, 2012). Summarizing the Relevant Empirical Literature The existing evidence suggests several important avenues of research. Given the importance in other subject areas, the study of teacher background characteristics, students’ opportunity to learn, and student achievement in social studies demands further empirical 34 inquiry. Existing evidence is somewhat mixed but suggests varying student achievement outcomes with respect to variation in teacher background characteristics and students’ opportunity to learn in the classroom. This study begins to fill in gaps by including the following: specific disciplinary background of teachers, private school students, sophisticated OTL measures, and use of recent data in eighth grade. The existing evidence also suggests that instructional exposure of various types can have positive associations with student interest. However, more research is needed across a more expansive set of commonly used approaches. Also, interest is typically not treated as a distinct construct in the extant educational literature on social studies, but rather a component of student engagement. Estimating relationships between a various instructional practices and student interest would be beneficial to the field. While more causal studies are needed in this substantive area, the existing evidence suggests that instructional practices are important for both student achievement and civic outcomes, as well as student interest (at least the small number of instructional practices studied with respect to interest). While exploring the role of student characteristics in relation to instructional exposure has received attention, little to no large-scale evidence exists regarding the relationship between the characteristics of teachers and student instructional exposure. Given the qualitative evidence available (Wilson & Wineburg, 1988) and the very broad training requirements for social studies teachers, exploring these relationships in more depth is crucial. In the next section, I discuss the methodological approaches taken in this study to begin addressing the gaps in the literature discussed above. 35 METHOD The primary purposes of this study were to estimate associations between a) opportunity to learn (OTL) and teacher background characteristics, on the one hand, and student achievement on the other, b) instructional exposure for students and student interest and c) the background characteristics of teachers and instructional exposure for students. To answer these questions, I used structural equation modeling (for models including OTL) and multivariate regression using data from the 2014 National Assessment of Educational Progress (NAEP). This section first provides some background on NAEP, then describes a) the sampling procedure from the original NAEP study and the restrictions imposed on the samples in the current study, b) the instruments (assessments and questionnaires), c) the variables used in the study and their summary statistics, and d) the analytic approaches. NAEP Background The National Assessment of Educational Progress (NAEP), also commonly referred to as The Nation's Report Card, is the only ongoing nationally representative assessment of what students in the United States know and can do in a range of subject areas, including social studies subjects/disciplines. While variations of NAEP have also included State NAEP, the NAEP Trial Urban District Assessment, and the Long-Term Trend, I focus in this study on the National NAEP. The National NAEP study has periodically assessed nationally representative samples of students from public and private schools since 1969, and the subjects assessed have been staggered across years and grades, including grades 4, 8, and 12. In 2014, nationally representative samples of eighth graders were assessed in either civics, U.S. history, geography, or technology and engineering literacy. This study focuses on the first three assessment samples relevant to social studies: civics, U.S. history, and geography. 36 Description of Sampling Procedures NAEP uses a complex, multistage sampling design to construct nationally representative samples of students and schools. The 2014 NAEP was a national-only assessment year, in contrast with state assessment years. For national-only assessment years,4 the first stage of sampling involves selecting between 50 and 100 primary sampling units (PSUs), each containing one or more counties, drawn from a frame of PSUs based on Census information (National Center for Education Statistics, 2018). The PSU sampling frame included all U.S. states and Washington D.C. and excluded U.S. territories and Puerto Rico. Stratification of noncertainty PSUs (i.e., probabilities of selection less than 1) was performed based on prior NAEP achievement data (National Center for Education Statistics, 2018). Schools were then selected from PSUs, with probability proportional to a measure of size based on enrollment (National Center for Education Statistics, 2018). The target population included all students in public and private schools enrolled in eighth grade at the time of assessment (National Center for Education Statistics, 2018). From the stratified frames of public and private schools, systematic random samples of eighth-grade students were drawn from each school (National Center for Education Statistics, 2018). No students were assessed in multiple subjects/disciplines. Thus, the students in each sample are entirely different. However, the schools and teachers across the three assessment samples overlap to some degree. The original study samples were drawn from more than 400 public and private schools in 48 U.S. states and Washington D.C., representing the diversity of students and school contexts across the country. It is not documented why the additional two 4 At the time of writing, no documentation for the sampling design for 2014 is available to either the public or restricted-license users. The description is based on the 2010 sampling design, the last year these three assessments occurred. 37 states did not participate in this particular year. The original civics sample included 9,100 students from 410 schools, the U.S. history sample included 11,200 students from 450 schools, and the geography sample included 9,000 students from 450 schools. This study restricts each sample to students who were eligible to be assessed (rptsamp = 1), had a teacher who completed a survey, and a teacher who reported giving instruction in the subject assessed. Teachers were not directly sampled. Rather, each sampled student or the school administrator nominated the appropriate teacher who was then administered a questionnaire about their background and teaching. Important to note: given the grade level and subject area, the civics, U.S. history, and geography samples do not necessarily represent civics, U.S. history, and geography courses/classes. Restricting each sample as I have is the approach I decided to take, but there may be other appropriate options, as well. After sample restrictions, the civics sample included 6,410 students from 340 schools, the U.S. history sample included 8,180 students from 370 schools, and the geography sample included 5,620 students from 350 schools. These sample sizes and all additional reporting of sample sizes are rounded to the nearest ten, per restricted-use license guidelines (National Center for Education Statistics, 2011). Restricting the sample in this way drops observations from 15 U.S. states entirely, limiting, to some degree, the generalizability of the results. However, given the focus of this study on the characteristics of students’ teachers, this sample restriction was necessary. Summary statistics of the samples are included later in this section. Assessments The National Assessment Governing Board oversees the development of assessment frameworks for each NAEP assessment. This process incorporates input from a range of stakeholders including content experts, school administrators, policymakers, teachers and 38 parents. Subcontractors included the Council of Chief State School Officers, the American Historical Association, American Institutes for Research, the National Council for History Education, the Center for Civic Education, and the National Council for the Social Studies. These frameworks guided the development of assessment items (including multiple choice and both short and extended constructed response), which are generated by educators and curriculum experts for each assessment year, a longstanding contract with Educational Testing Service. Example released assessment items from prior years and the variety of question types and content covered can be found for all three assessments in Appendix A. Each student completed only a fraction of the items from the assessment due to time constraints. Plausible values were estimated by NAEP contractors using a combination of measurement and population-structure models (Mislevy, Johnson, & Muraki, 1992). In other words, both students’ responses to the items they complete as well as other available data are used to generate 20 plausible test scores for each student, which can then be used in analyses by way of multiple imputation, similar to applications for dealing with missing data (Schafer, 1999). Civics assessment. The current civics framework was originally developed in 1998 and draws heavily on the National Standards for Civics and Government, published in 1994 by the Center for Civic Education (Center for Civic Education, 1994; The National Assessment Governing Board, 2014a). The civics assessment was intended to measure civic knowledge, intellectual and participatory skills, and civic dispositions. The content areas included politics and government, foundations of the U.S. political system, the U.S. constitution, world affairs, and the roles of citizens. The intended cognitive domains for the civics assessment included identifying/describing, explaining/analyzing, and evaluate/take/defend. Sample items are presented in Appendix A. 39 U.S. history assessment. The U.S. history framework was originally developed in 1991- 92 with minor updates in 2003. While there is no mention of specific standards used in the development of the assessment framework, the original project committees included “a broad range of historians, educators, policymakers, business representatives, and other interested citizens” (The National Assessment Governing Board, 2014c, p. v). The U.S. history assessment was intended to assess two ways of knowing and thinking about history, a) historical knowledge and perspective and b) historical analysis and interpretation. The content of the U.S. history assessment was organized around four historical themes: continuity and change in U.S. democracy, interaction of peoples and cultures, technological and economic changes, and the changing world role of the U.S. A recent paper investigated/questioned the construct validity of the NAEP U.S. history assessment (M. D. Smith, 2017). This is a worthwhile debate. Nonetheless, NAEP is the best available national U.S. data for the analyses in this study. Sample items are presented in Appendix A. Geography assessment. The current geography framework was originally developed in 1994, predating the release of the first edition of the National Geography Standards. However, the NAEP geography framework and these standards share common goals and expectations for students (The National Assessment Governing Board, 2014b). The geography assessment was organized around three content areas: space and place, environment and society, and spatial dynamics and connections. The assessment was designed for students to answer questions by performing the following cognitive processes: knowing, understanding, and applying. Sample items are presented in Appendix A. 40 Questionnaires NAEP also administers questionnaires to the students assessed, their teachers, and their administrators. Relevant to this study, the student questionnaire contained items about their background (e.g., race, ethnicity, gender, and socioeconomic status) and items reflecting their interest in their social studies coursework. The teacher questionnaire primarily asked teachers about their background and training, their instructional practices, and the extent they focus on various curricular themes. While not mentioned as such in the survey, these curricular themes directly align to each NAEP assessment. From the school survey, I utilized only variables corresponding to school type (i.e., private, charter, traditional public), locale (i.e., urbanicity), and U.S. census region. The individual variables used in this study are described in more detail in the next section. All questionnaires were administered at the same time of the assessment, toward the end of the academic year. Variables In this section, I describe each of the variables used in this study including student, teacher, and school background variables; OTL; instructional exposure; student achievement; and interest. Weighted summary statistics (using probability weights to appropriately reflect the national target population) for the three samples can be found in Tables 1 – 4 and Table 6. Each group of variables is described further below. Student, teacher, and school background variables. Table 1 displays summary statistics for several variables corresponding to student, teacher and school background that were used in this study. Binary variables describe students’ race, ethnicity, gender, whether the student has limited English proficiency (LEP) or has an individualized education program (IEP). I also used two indicators of socioeconomic status in this study: whether the student is eligible for free 41 Table 1. Selected characteristics of students, teachers, and schools (proportions) Variables Student Characteristics Female White, not Hispanic African American, not Hispanic Hispanic of any race Asian American/Pacific Islander Another race or ethnicity Free/reduced-price lunch Books in the home (1 bookcase or more) Limited English proficient (LEP) Individualized education program (IEP) Talk about studies at home (minimum = weekly) Student Interest Class is frequently interesting Social studies/assessed subject is a favorite Teacher Background/Experience Undergrad major/minor/specialization History/history ed Geography/geography ed Political science Gen social sci/social studies ed Other social science (e.g., econ or soc) Education/secondary ed Completed graduate coursework Grad major/minor/specialization History/history ed Geography/geography ed Political science Gen social sci/social studies ed Other social science (e.g., econ or soc) Education/secondary ed Alternative certification NBPTS certified >5 years social studies teaching experience School Characteristics Private Charter City Suburb Town Rural Northeast Midwest South West Civics U.S. History Geography 0.49 0.53 0.14 0.23 0.06 0.03 0.45 0.61 0.05 0.11 0.63 0.52 - 0.62 0.16 0.25 0.39 0.27 0.56 0.72 0.24 0.07 0.06 0.16 0.08 0.41 0.17 0.15 0.69 0.08 0.06 0.29 0.34 0.12 0.25 0.18 0.22 0.38 0.22 0.49 0.51 0.15 0.25 0.06 0.03 0.47 0.61 0.05 0.11 0.63 0.55 0.69 0.61 0.15 0.25 0.37 0.28 0.52 0.73 0.23 0.06 0.08 0.16 0.09 0.42 0.19 0.15 0.69 0.07 0.07 0.31 0.35 0.12 0.23 0.18 0.22 0.38 0.23 0.50 0.55 0.15 0.23 0.06 0.03 0.46 0.60 0.05 0.12 0.62 0.52 0.70 0.64 0.17 0.24 0.42 0.27 0.55 0.72 0.25 0.07 0.08 0.16 0.08 0.40 0.18 0.16 0.69 0.08 0.07 0.30 0.33 0.12 0.24 0.16 0.22 0.39 0.23 Note. Estimates are weighted using survey weights (origwt). The unit of observation is students. 42 or reduced-price lunch (a commonly used indicator of poverty) and binary indicator for whether a student reported having a bookcase or more books at home (a commonly used indicator for socioeconomic status). Used as a proxy for parental engagement, a binary indicator measures whether the student reported talking about their coursework with family at home at least once a week. Several binary variables describe whether a teacher had a major, minor, or specialization in undergraduate and/or graduate education in any of six fields/courses of study related to social studies education and its underlying disciplines. Two variables correspond to different forms of certification: whether they were a) certified through an alternative route, and/or b) were certified by the National Board for Professional Teaching Standards (NBPTS), a voluntary, advanced form of certification. Additionally, one binary variable describes whether a teacher has more than five years of teaching experience in social studies, approximately the point where teachers are considered to be experienced (Berliner, 2001). Last, Table 1 additionally describes several school-level variables used in this study. These included binary items that describe the school sector, the locale/urbanicity, and the census region. While not included in the tables, all models described in later sections additionally include cluster (school) means of all student background variables to approximate school context. Opportunity to learn. Teachers reported the extent their class focused on a range of curricular topics, which were aligned to each of the civics, geography, and U.S. history assessments. This alignment of curricular variables to the assessments allows for answering the question of whether increased OTL is associated with increased achievement. Again, teachers were not specifically told each topic was aligned to the relevant assessment. Summary statistics 43 for each of the curricular items are displayed in Tables 2 – 4. Prior work on OTL has typically used a latent variable modeling approach to measure the construct (e.g., factor analysis or IRT) (Schmidt & Maier, 2009). High correlations among each of the curricular indicators for each of the three disciplinary areas also suggested estimating a latent variable would be appropriate to avoid potential multicollinearity in estimation of structural models. Specifically, I employed confirmatory factor analysis (CFA) to initially measure each OTL variable in their respective sample (e.g., OTL civics in the civics sample) and test whether the data fit the hypothesized measurement models. The factor structure for each OTL variable (civics, U.S. history, and geography) is displayed in Table 5. These estimates are from initial CFAs without any structural component to investigate the measurement of each factor. Identical measurement models were used in estimation of structural relationships, described in later sections. As is evidenced by the RMSEA values for each measurement model, each factor fit the data well. Table 2. Summary statistics of OTL NAEP civics indicators Response (extent of coverage) Not at all Small extent Moderate extent Large extent Variable Politics & government Foundations of the U.S. political system Not at all U.S. Constitution World affairs Roles of citizens in U.S. democracy Small extent Moderate extent Large extent Not at all Small extent Moderate extent Large extent Not at all Small extent Moderate extent Large extent Not at all Small extent Moderate extent Large extent Proportion 0.09 0.09 0.42 0.40 0.11 0.11 0.31 0.47 0.10 0.09 0.28 0.53 0.13 0.35 0.40 0.12 0.09 0.19 0.42 0.30 Note. Estimated using civics sample. Estimates are weighted using survey weights (origwt). The unit of observation is students. 44 Table 3. Summary statistics of OTL NAEP U.S. history indicators Proportion Variable Change/continuity in U.S. democracy Changing role of U.S. in the world Economic changes Technological changes Gathering and interactions of people from various cultures Not at all 0.05 0.20 0.45 0.30 0.06 0.25 0.41 0.28 0.03 0.22 0.44 0.31 0.05 0.29 0.44 0.23 0.04 0.20 0.48 0.27 Note. Estimated using U.S. history sample. Estimates are weighted using survey weights (origwt). The unit of observation is students. Table 4. Summary statistics of OTL NAEP geography indicators Response (extent of coverage) Not at all Small extent Moderate extent Large extent Not at all Small extent Moderate extent Large extent Not at all Small extent Moderate extent Large extent Not at all Small extent Moderate extent Large extent Small extent Moderate extent Large extent Variable Space & place Spatial dynamics & connections Environmental issues Natural resources Other countries and cultures Response (frequency taught) Never or hardly ever 1-2 times a month 1-2 times a week Almost every day Never or hardly ever 1-2 times a month 1-2 times a week Almost every day Never or hardly ever 1-2 times a month 1-2 times a week Almost every day Never or hardly ever 1-2 times a month 1-2 times a week Almost every day Never or hardly ever 1-2 times a month 1-2 times a week Almost every day Proportion 0.26 0.42 0.25 0.07 0.18 0.44 0.27 0.10 0.45 0.39 0.14 0.02 0.24 0.50 0.23 0.03 0.24 0.39 0.23 0.13 Note. Estimated using geography sample. Estimates are weighted using survey weights (origwt). The unit of observation is students. It is not documented why geography curricular variables were measured on a differently worded scale than U.S. history and civics. 45 Table 5. CFA estimates for latent OTL variables 0.02 0.01 0.02 RMSEA Factor Indicator Latent Variable OTL NAEP Civics OTL NAEP US-H OTL NAEP Geography Foundations of the U.S. political system U.S. Constitution Politics & government Roles of citizens in U.S. democracy World affairs Economic changes Technological changes Changing role of U.S. in the world Gathering and interactions of people from various cultures Change/continuity in U.S. democracy Space & place Spatial dynamics & connections Environmental issues Natural resources Other countries and cultures Probit 1.00 0.99* 0.91* 0.87* 0.37* 1.00 0.86* 0.84* 0.81* 0.68* 1.00 0.95* 1.16* 1.09* 1.07* Note. ∗p < 0.05. Statistics included are from three individual CFA models for each of the three relevant samples: civics (N = 6,390), U.S. history (N = 8,160), and geography (N = 5,610). Estimated using robust weighted least squares (WLSMV). The loading for the first listed indicator for each factor was fixed to 1, and the variances freely estimated. OTL NAEP Civics includes correlated errors between world affairs and a) roles of citizens and b) politics & government. OTL NAEP Geography includes correlated errors between spatial dynamics and space & place. Both were to improve model fit, suggested by modification indices. Instructional exposure. A recent chapter from the Handbook of Research on Teaching (5th edition) summarized the research on three groups of instructional practices in social studies education: a) inquiry-oriented approaches, b) discussion, and c) source analysis (Barton & Avery, 2016). I use these categories, generally, as a guide in this study and further include didactic/traditional approaches as well as community-based approaches, also commonly used in social studies education. All of the individual practices/approaches and their summary statistics across samples are included in Table 6. Instructional practices are used as controls in models of achievement, as the primary predictors in models of student interest, and also the outcomes in models for research question 3, discussed later in this section. To measure instructional exposure for students, several Likert items (on a scale from “never or hardly ever” to “almost every day”) were included in the original teacher questionnaire. I grouped these different instructional approaches into the following: didactic/traditional approaches, inquiry-oriented, engaging with sources, discussion- 46 based, and community-based. These variables were on 4-point scales, and I recoded the variables to be binary for use in model estimation. Frequently used instructional approaches were cut at the midpoint of the underlying Likert scales to reflect frequent/non-frequent use. Less frequently used instructional approaches, based on the distribution of responses (i.e., mock trial, GIS, and all community-based variables) reflect either any use or non-use. The only instructional practice specific to a particular content area (civics, history or geography) was the GIS variable. This survey item was asked only of students, not teachers. Table 6. Student exposure to social studies instructional practices (proportions) Instructional approaches Didactic/traditional Lectures Worksheets Engaging with sources Social studies textbook Books/newspapers/magazines Primary documents Film/video Quantitative data (e.g., maps, charts, graphs) Inquiry-oriented Computer simulations Mock trial/role play/drama (use/non-use) Use GIS software in geography Discussion-based Debates/panel discussions Discuss current events Community-based (use/non-use) Community visitors Write letters to state opinion or solve problem in community Community projects Civics U.S. History Geography 0.80 0.62 0.69 0.39 0.64 0.53 0.74 0.18 0.64 - 0.31 0.72 0.12 0.55 0.17 0.80 0.63 0.70 0.37 0.66 0.52 0.74 0.17 0.62 - 0.30 0.72 0.11 0.54 0.17 0.81 0.61 0.70 0.39 0.63 0.54 0.76 0.18 0.66 0.52 0.33 0.74 0.12 0.54 0.16 Note. Estimates are weighted using survey weights (origwt). The unit of observation is students. GIS variable is student-reported. All others are reported by students’ teachers. Commonly used instructional methods were cut at the midpoint of the underlying 4-point Likert scales. Thus, values of 1 reflect use of the method once or twice per week or more. Less commonly used instructional methods (i.e., mock trial, GIS, and all community-based variables) reflect either any use or non-use. Student achievement and interest. The other two groups of student outcomes in this study are student NAEP scores in civics, geography, or U.S. history (depending on the sample) as well as items that measure student interest. As was mentioned above, NAEP uses the plausible 47 value approach to assessing students. In other words, students do not take the entire assessment. Rather, each student takes a sample of the assessment, and 20 plausible test scores are generated for each student based on the available data. In all models of achievement, NAEP scores are rescaled as z-scores such that coefficients reflect effect sizes. Two items are available in the NAEP study to measure student interest in social studies, reported by the students themselves. While by no means perfect, these measures resemble situational interest (measured in this study as frequent/non-frequent interest in class) and individual interest (measured in this study as the extent the assessed discipline/social studies is a favorite school subject) from prior literature (Hidi & Renninger, 2006). Ideally, multiple items/indicators and repeated measures over time would be available to measure each form of interest, but I was limited to what was available in the data. As student interest is also used as a control in models of achievement, I’ve listed the summary statistics for these variables in Table 1. The first variable (frequent interest) measures how often a student is interested in their classrooms. This variable is available in all three samples and the original scale ranged from “never or hardly ever” to “always or almost always.” For this variable, I rescaled so that values of 1 are equal to students who reported either “often” or “always or almost always.” The second variable measured the extent with which the student agreed that social studies/the assessed discipline is a favorite subject, ranging from “not at all” to “a lot.” This item is only available in the U.S. history and geography samples for undocumented reasons. For this variable, values of 1 correspond to students who reported “a little” or “a lot” versus “not at all.” Perhaps somewhat surprising to readers, approximately 70% of students assessed in U.S. history and geography 48 responded either “a little” or “a lot” when asked to what extent they agreed U.S. history/social studies or geography/social studies was a favorite subject. Analysis For models with achievement as an outcome (which include the latent measure of OTL as a predictor), this study employed structural equation modeling, and all other models were estimated using multivariate5 logistic regression in Mplus 8 (Muthén & Muthén, 2017). SEM models were estimated using robust weighted least squares (WLSMV), and logistic regression models were estimated using robust maximum likelihood (MLR) (Muthén & Muthén, 2017). This difference in estimators was due to the added benefit of model fit statistics with WLSMV and the use of logistic regressions rather than probit regressions for the categorical outcomes for models in research questions 2 and 3. In all models, standard errors were adjusted for stratification and clustering, and student sampling weights were used appropriately. As is true for all research questions, models were separately estimated for each sample (civics, U.S. history, and geography). I use the path diagram format to present the models for ease of interpretation across varying backgrounds of potential readers. Boxes represent a manifest/observed variable or group of variables. Each listed variable within boxes is modeled individually. The only variable modeled as latent in this study is OTL, which is represented in Figure 1 as a circle. Each model is discussed further below. A simplified path diagram for research question 1 is presented in Figure 1. To reiterate, the outcome of interest in research question 1 was students’ NAEP score (civics, U.S. history, or geography). The main predictors included a set of binary variables corresponding to teachers’ undergraduate and graduate studies in a range of relevant disciplines, whether the teacher was 5 The one exception was the model of interest in the civics sample, which only included one outcome, and thus was not a multivariate regression model. 49 alternatively certified, whether the teacher was NBPTS certified, a binary indicator representing an experienced teacher (listed in Table 1), and a latent variable measuring students’ opportunity to learn the assessed subject, aligned to the assessment as described above (listed in Table 5). I also controlled for all available characteristics of students and schools, the measure of student interest available across all three samples (listed in Table 1), and all measures of instructional exposure listed in Table 6 (as well as an indicator for overall social studies instructional hours). Included in this list of controls was U.S. census region to control for regional variation (e.g., varying curricular focus or political/cultural differences). Finally, school-averaged student characteristics were also included in each model to control for additional aspects of school context. Figure 1. Simplified path diagram for achievement in research question 1. To answer research question 2, I estimated a model for each of the three samples with the available measures of student interest as the dependent variables. The primary predictors were 50 measures of student instructional exposure, listed in Table 6. A simplified path diagram is displayed in Figure 2. Control variables included all student, teacher, and school background variables listed in Table 1 (with the exception, of course, of the student interest measures), as well as school averages of all student characteristics in Table 1. Given the binary outcomes and use of MLR as an estimator, these models were logistic regressions. Figure 2. Simplified path diagram for interest in research question 2. To answer research question 3, I again estimated a model for each of the three samples. A simplified path diagram is presented in Figure 3. The outcomes in this case were the set of binary variables related to instructional practices. I estimated to what extent teacher background/experience was associated with student instructional exposure (reported by teachers), conditional on student and school covariates (identical to the student and school controls used in the models for research question 2). Additionally, preliminary analyses found that both instructional time as well as variation in disciplinary focus (i.e., focusing class more on civics, geography or U.S. history) was related to both instructional exposure and also the primary independent variables. As such, indicators for both overall instructional hours in social studies as 51 well as proportion of time spent on each civics, U.S. history, and geography were also included as controls. This variation in content focus was modeled in research question 1 by including the appropriate OTL variable (aligned to the outcome of interest). Neither instructional time nor variation in classroom disciplinary focus was related to student interest, so these variables were not included as controls for models estimated for research question 2. Regressions for the binary dependent variables are, again, logistic regressions. Figure 3. Simplified path diagram for instructional exposure in research question 3. Many of the contributions of this study are methodological. To start, this is the first study to estimate OTL in social studies subjects, in a similar fashion to how the construct has been measured in other school subjects in prior research. While it is not a groundbreaking statement to suggest that students tend to learn more if given the opportunity in the classroom, students in the U.S. and globally tend to have varying curricular opportunities across subject areas (Atwell et al., 2017; Passe & Fitchett, 2013; Schmidt, Burroughs, Zoido, & Houang, 2015). This study estimates/quantifies the extent that opportunity to learn content in three distinct social studies disciplines is related to student achievement on each aligned assessment (i.e., civics, U.S. history, and geography), within the limitations of a descriptive study. 52 This study also builds on prior work on measuring teachers’ prior subject background and the relationship to both student achievement and instructional exposure. This is the first study to estimate relationships between varying teacher subject background (i.e., history, political science, geography, social studies education, etc.) and social studies assessments in different disciplines. As discussed in the literature review, prior research has either collapsed all relevant subjects into an overall social studies-related subject background variable or has focused on only one of several disciplines in the social studies (i.e., one discipline such as history or one general social studies assessment). While an earlier qualitative piece analyzed varying teaching approaches by social studies teachers of varying backgrounds (Wilson & Wineburg, 1988), this topic has not been looked at from a large-scale quantitative perspective, to my knowledge. While not a primary focus, this study also makes important contributions regarding school sector (i.e., private, public, charter). This study includes large proportions of students at both charter and private schools across the country, in addition to traditional public school students. Exploring the extent that students at schools in various sectors have varying exposure to different instructional practices is an important empirical question, particularly given the gap in the research as to why private and charter school students have been found to be more civically engaged in a number of studies (Campbell, 2012). Very little is known about private and charter school students in history and geography, as well. Finally, there is currently no published empirical work using the 2014 NAEP social studies data, and again, no prior work has modeled assessment outcomes in multiple social studies disciplines, distinctly. To my knowledge, NAEP geography data has never been modeled empirically, at least not in a peer-reviewed publication. The methods used contribute to a growing literature base in large-scale quantitative analysis of social studies education. 53 RESULTS This section breaks down the results by research question. The relevant tables are referred to throughout. All interpretations of coefficients below are conditional on all other covariates in the respective model. As a reminder the research questions were as follows: RQ1: To what extent are teacher background characteristics and students’ opportunity to learn associated with eighth-grade student achievement in civics, U.S. history, and geography? RQ2: To what extent is students’ exposure to various instructional practices associated with student interest in eighth-grade social studies? RQ3: To what extent are teacher background characteristics associated with students’ exposure to various instructional practices in eighth-grade social studies? Research Question 1 Results for research question 1 are presented in Table 7. I’ve integrated the results across the three assessment samples below. Results are grouped by related variables. Coefficients for student and school characteristics, used as covariates in these models, can be found in Table B1 in Appendix B. Teacher background characteristics. Variables related to teacher background included undergraduate and graduate coursework foci, initial certification type, NBPTS certification, and years of experience. Regarding the relationship to student achievement, few indicators were statistically significant predictors. However, students whose teachers reported an undergraduate major/minor/specialization in political science (about 25% of students) scored 11% of a standard deviation higher, on average, on NAEP civics. Similarly, students whose teachers reported a graduate major/minor/specialization in general social sciences or social studies education (about 16% of students) scored 12% of a standard deviation higher, on average, on NAEP U.S. history. 54 There were no significant relationships estimated between teacher background and student performance on NAEP geography. Further analysis is needed in this area; however, these findings suggest that teachers’ specific subject matter preparation may contribute to their students’ achievement in individual social studies disciplines. Further comparisons of individual disciplinary preparation versus general social studies education and other forms of undergraduate and graduate preparation should be studied. Table 7. Coefficients for achievement models Variable Opportunity to Learn OTL NAEP Civics OTL NAEP U.S. History OTL Geography Teacher Background/Experience Undergrad major/minor/specialization History/history ed Geography/geography ed Political science Gen social sci/social studies ed Other social science (e.g., econ or soc) Education/secondary ed Completed graduate coursework Grad major/minor/specialization History/history ed Geography/geography ed Political science Gen social sci/social studies ed Other social science (e.g., econ or soc) Education/secondary ed Alternative certification NBPTS certified >5 years social studies teaching experience Unweighted N RMSEA NAEP Civics NAEP U.S. History NAEP Geography 0.17* - - -0.03 -0.06 0.11* 0.02 0.02 -0.01 -0.03 0.02 -0.11 0.08 0.04 0.00 0.03 -0.02 -0.02 0.05 5,590 0.01 - 0.12* - 0.02 -0.06 0.03 0.02 -0.01 0.05 0.01 -0.02 -0.07 0.07 0.12* -0.01 -0.01 0.02 -0.02 0.01 7,120 0.01 - - -0.03 -0.02 -0.04 0.08 0.03 -0.05 0.01 0.04 -0.04 -0.01 0.02 0.07 -0.02 -0.01 0.01 -0.03 0.07 4,630 0.01 Note. ∗p < 0.05. Additional covariates include all additional variables in Tables 1 and 6 as well as cluster/school- averages of the student characteristics in Table 1. Given high correlations between the two indicators of student interest and the lack of availability of both items in the civics sample, I modelled only the frequent interest variable as a covariate across all samples. I also controlled for an indicator of instructional hours in social studies, generally, reported by the students’ teachers. Estimates for additional covariates can be found in Table B1 in Appendix B. 55 Opportunity to learn. Opportunity to learn the assessed discipline was measured as a latent factor in each of the three assessment samples as described in the method section. Significant associations with NAEP scores were found in two of the three assessment samples, civics and U.S. history. An increase in one standard deviation of OTL civics was associated with an increase of 17% of a standard deviation on NAEP civics. Similarly, in U.S. history, a standard deviation increase in OTL U.S. history was associated with an increase of 12% of standard deviation on NAEP U.S. history. OTL geography had no relationship to students’ scores on NAEP geography. An indicator for instructional time in social studies overall was included in each model. This variable had no relationship to NAEP scores in any of the three samples. These findings suggest that increased OTL (i.e., assessment-aligned curricular coverage) is important for student achievement, at least for civics and U.S. history. Research Question 2 The dependent variables for this research question were student-reported measures of interest in social studies (i.e., frequent interest in class and the extent the assessed subject/social studies is a favorite subject). A number of positive relationships were found between instructional exposure and student interest. The results are presented in Table 8. Odds ratios greater than 1 represent a greater likelihood and those less than 1 represent a lower likelihood. For interpretation, odds ratios less than 1 should be inversed and the reference groups flipped. Odds ratios should not be compared across variables within models given different initial probabilities. Specific interpretations are documented below. Coefficients and odds ratios for student and school characteristics, used as covariates in these models, can be found in Table B2 in Appendix B. 56 Important to note: each sample contains different students; however, these students are not necessarily in civics, U.S. history, or geography classes. As such, discussion of the civics sample, for instance, refers to the civics assessment sample and should not be confused with students who are in civics classes. As discussed in the method section, each sample is restricted to teachers who reported giving at least some instruction in the assessed discipline. Exposure to geographic information systems (GIS) was the only student-reported instructional measure and was asked only of students taking the geography assessment. All other instructional approaches reported by students’ teachers were general social studies instructional approaches and were included in models across all samples. Students who reported being exposed to GIS during geography instruction were 33% more likely to report being frequently interested in class and 45% more likely to report geography/social studies as a favorite subject. Across all three samples, students who were exposed to community projects were more likely to report being frequently interested in the relevant class/subject, with exposed students being between 29% - 42% more likely than other students, depending on the sample. In both the geography and U.S. history samples, the use of community projects also had a strong association with students reporting geography/social studies (OR = 1.69) or history/social studies as a favorite subject (OR = 1.28). Frequent use of film/video or any use of mock trials/roleplay in the geography sample was also positively associated with students reporting geography/social studies as a favorite subject. Several negative relationships between instructional exposure and student interest were also estimated. Students whose teachers reported frequent use of computer simulations in their civics/social studies classes were slightly less likely to report being frequently interested in the class. In the U.S. history sample, frequent use of a social studies textbook as part of instruction 57 or having students frequently write letters were both negatively associated with frequent student interest. Research Question 3 Students of teachers with different background and training significantly varied in exposure to some of the instructional approaches modeled in this study but not to others. In this section, I first present the results regarding the relationship between teacher background characteristics and student instructional exposure, focusing on significant relationships. Then, I highlight several significant, conditional estimates regarding school sector, used primarily as controls in the models but of substantive interest. All interpretations and results reported below are conditional on all other covariates in the respective model. Results are presented as odds ratios in Tables 9 – 11 and interpreted in the same fashion as described previously for research question 2. Odds ratios for student and school characteristics, used as covariates in these models, can be found in Tables B3 – B5 in Appendix B. Important to reiterate: included in the list of controls/covariates in these models were indicators for the extent that the students’ class focused on each civics, U.S. history, and geography, given the significant relationships between the class disciplinary foci, teacher background characteristics, and the cross-cutting social studies instructional approaches modeled. As in the presentation of results for research question 2, each sample discussed is an assessment sample that included at least some focus in the discipline, such as civics. These samples, however, are not necessarily civics/U.S. history/geography classes. Teacher background characteristics. A number of significant findings were estimated regarding the relationship between teacher background characteristics and student exposure to instructional approaches. Regarding a background in history/history education, two specific 58 instructional practices were significantly associated. Across all samples, students of teachers with an undergraduate focus on history/history education had a lower likelihood of exposure to frequent use of a social studies textbook. In both the civics and geography samples, students of teachers with this background at the graduate level were less likely to be exposed to discussing current events. A background in geography/geography education for students’ teachers was also associated with instructional exposure. In the geography sample, students whose teachers had an undergraduate focus in geography/geography education were 3.53 times more likely to have been exposed to frequent lecture. Students of teachers with this background across all samples were also less likely to have had exposure to writing letters. In all samples, students of teachers with a graduate focus on geography/geography education were less likely to have had exposure to participating in debates or panel discussions. In the U.S. history sample only, students of teachers with this subject matter background at the graduate level were about three times more likely than others to have had exposure to frequent worksheet use and 5.67 times more likely than other students to have been exposed to computer simulations. In the geography sample only, students of teachers with this background were about six times more likely than others to have been exposed to community visitors. Students of teachers with an undergraduate focus in political science were not significantly more or less likely to have had exposure to any of the instructional practices measured in this study. A graduate focus in political science for teachers, however, was related to student exposure to several instructional practices. Students of other teachers across all samples (not those with a graduate focus in political science) were more than five times more likely to have had exposure to frequent lecture. Also across all samples, students of teachers who reported 59 a graduate focus in political science were between 4.44 and 7.74 times more likely to have had exposure to discussing current events. In both the U.S. history and geography samples, students of these teachers were more than seven times more likely than other students to have had exposure to community projects. Other subject matter backgrounds for teachers were also associated with student instructional exposure. For example, students of teachers with any graduate coursework were more likely to be exposed to frequent lecture as well as writing letters, across all samples. Somewhat peculiar, students of teachers with an undergraduate background in general social sciences/social studies education were more likely to be exposed to computer simulations (in all samples), but students with this background at the graduate level were less likely to be exposed to this instructional approach (civics and geography samples). Students of teachers reporting an undergraduate background in “other social science” were less likely to be exposed to community projects (civics and U.S. history samples) and less likely to be exposed to writing letters (U.S. history and geography samples). Regarding an education or secondary education background, students of teachers with this training at the undergraduate level in the U.S. history sample were more likely to be exposed to frequent lecture, and students of teachers in the geography sample with this background at the graduate level were less likely to be exposed to frequent textbook use. Additionally, initial certification type was associated with student instructional exposure. Students of alternatively-certified teachers in both the civics and U.S. history samples were more likely than other students to have been exposed to frequent lecture and worksheets. In the civics sample only, students of these teachers were about two times more likely than other students to have been exposed to discussing current events. Across all samples, students of alternatively- 60 certified teachers were less likely than other students to have been exposed to frequent textbook use. NBPTS certification was also associated with student instructional exposure. Students of NBPTS-certified teachers in the civics and U.S. history samples were more likely to have been exposed to writing letters. Students of NBPTS teachers in the U.S. history sample were also about 2 times more likely to have been exposed to mock trials/roleplay. In both the U.S. history and geography samples, these students were more likely to have been exposed to community projects. In both the civics and geography samples, students of these teachers were also less likely to have been exposed to frequent lecture. Conditional on all other covariates, students of experienced social studies teachers also had varying instructional exposure relative to students of novice social studies teachers. Across all samples, students of experienced social studies teachers were less likely to have been exposed to frequent use of primary documents, film/video, and also less likely to have had exposure to writing letters in the classroom. In the geography sample only, students of experienced social studies teachers were less likely to have had any exposure to community projects. School sector. While not a primary focus of this study, several significant relationships between school sector and student instructional exposure were estimated, and these findings are likely of substantive interest to the field. Odds ratios for these variables can be found in Tables B3 – B5 in Appendix B. Across all three samples, private school students were more than four times more likely than traditional public school students to have been exposed to community projects. Private school students were also between about three times (U.S. history sample) and about 4 times (geography sample) more likely to have been exposed to frequent use of a social studies textbook as part of instruction. Finally, traditional public school students in the civics 61 sample were 2.22 times more likely than private school students to have been exposed to frequent worksheets. Charter school students varied in some ways with regard to their instructional exposure relative to their traditional public school counterparts, as well. In the civics sample, charter school students were nearly seven times more likely to have been exposed to frequent lecture and less likely to have been exposed to frequent use of books/newspapers/magazines and primary documents. In the U.S. history sample, the same was true regarding primary documents. In the geography sample, the same finding was estimated regarding books/newspapers/magazines. In both the U.S. history and geography samples, charter school students were less likely to have been frequently exposed to quantitative data. 62 Table 8. Coefficients and odds ratios for interest models Variable Instructional approaches Didactic/traditional Lectures Worksheets Engaging with sources Social studies textbook Books/newspapers/magazines Primary documents Film/video Quantitative data Inquiry-oriented Computer simulations Mock trial/role play/dramatization GIS Discussion-based Debates/panel discussions Discuss current events Community-based Community visitors Write letters Community projects Unweighted N Civics sample Frequent Interest Logit OR U.S. History sample Geography sample Frequent Interest Logit OR Favorite Logit OR Frequent Interest Logit OR Favorite Logit OR 0.13 -0.07 -0.15 0.02 0.02 0.03 -0.03 1.14 0.94 0.86 1.02 1.02 1.03 0.97 -0.20* -0.04 0.82* 0.96 - 0.08 0.13 -0.14 0.02 0.35* - 1.08 1.14 0.87 1.02 1.42* 5,600 0.11 -0.05 -0.32* -0.04 -0.02 0.08 -0.07 -0.06 -0.05 - 0.02 -0.03 -0.02 -0.16* 0.26* 1.12 0.96 0.73* 0.96 0.98 1.08 0.93 0.94 0.95 - 1.02 0.97 0.98 0.85* 1.29* 0.09 -0.13 -0.10 -0.10 0.03 -0.02 -0.11 -0.11 -0.01 - -0.01 -0.05 -0.07 -0.05 0.24* 1.09 0.88 0.90 0.91 1.03 0.98 0.90 0.90 0.99 - 0.99 0.95 0.93 0.95 1.28* 0.17 0.03 -0.19 -0.02 -0.03 0.05 0.12 0.03 0.01 0.28* 0.08 -0.05 0.02 0.02 0.33* 1.19 1.03 0.83 0.98 0.97 1.05 1.12 1.03 1.01 1.33* 1.08 0.95 1.02 1.02 1.39* 0.03 -0.02 -0.08 -0.11 0.02 0.23* 0.12 0.12 0.19* 0.37* 0.00 -0.18 -0.11 -0.10 0.53* 7,160 4,660 1.04 0.98 0.93 0.9 1.02 1.26* 1.13 1.13 1.21* 1.45* 1.00 0.83 0.90 0.91 1.69* Note. ∗p < 0.05. Additional covariates include all variables in Table 1 (except interest) and cluster/school-averages of the student characteristics in Table 1. Estimates for additional covariates can be found in Table B2 in Appendix B. 63 Table 9. Odds ratios for instructional exposure model (civics sample) Undergraduate History/hist. ed. Geog./geog. ed. Poli sci Soc. sci./social studies ed Other soc. sci. Ed/sec. ed. Grad coursework Graduate History/hist. ed. Geog./geog. ed. Poli sci Soc. sci./social studies ed Other soc. sci. Ed/sec. ed. Alt cert NBPTS >5 yrs SS teaching exp s e r u t c e L s t e e h s k r o W l a i c o S s e i d u t s k o o b t x e t 0.76 2.36 1.41 0.88 0.76 1.50 0.84 1.89 0.89 1.49 0.63 0.84 0.48* 0.92 1.08 0.99 1.00 0.90 s w e n / s k o o B s e n i z a g a m / 1.06 1.24 1.19 1.00 0.84 0.85 y r a m i r P s t n e m u c o d 1.63 0.66 1.47 0.81 0.80 0.92 1.18 1.12 0.96 1.33 0.79 0.97 0.92 1.53 1.22 0.61 0.89 1.15 0.46* 1.75 2.36* 1.25 0.98 1.16 1.28 1.39 1.28 0.18* 0.93 1.31 1.20 2.42* 0.50* 0.67 2.01 0.82 0.99 0.83 1.08 0.88 1.30 1.46 0.80 1.08 0.61 2.05* 0.76 0.50* 0.79 0.75 1.11 2.23 1.34 0.71 0.83 0.76 1.24 1.26 1.21 1.04 1.23 1.27 1.10 1.06 1.45 0.79 0.62 1.17 0.89 1.38 1.31 1.04 1.08 0.73 0.79 1.79 0.80 1.12 0.86 1.29 1.00 o e d i v / m l i F e v i t a t i t n a u Q a t a d r e t u p m o C s n o i t a l u m i s k c o M e l o r / l a i r t y a l p & s e t a b e D l e n a p s n o i s s u c s i d s s u c s i D t n e r r u c s t n e v e y t i n u m m o C r o t i s i V 1.05 0.80 1.18 0.89 0.64 0.77 s r e t t e l e t i r W 1.13 0.36* 1.28 1.13 0.61 1.14 y t i n u m m o C s t c e j o r p 0.60 0.88 1.24 1.21 0.44* 1.10 1.34 1.65 0.63 0.65 0.68 1.08 1.00 1.11 0.83 1.36 0.75 0.98 0.86 1.56 1.82 3.11* 1.40 1.24 0.25* 2.73 1.20 0.92 1.23 1.35 0.95 0.45* 2.38 7.44* 1.20 1.69 0.91 2.15* 1.13 0.78 1.79 1.06 0.76 0.89 0.61 0.40 1.59 1.07 2.03 1.06 1.12 0.64 0.73 0.61 1.73* 0.74 0.81 4.23 1.61 0.81 1.14 1.14 2.00 0.74 1.20 0.98 2.56* 1.18 0.78 1.53 1.04 2.58 0.59 0.43* 0.97 0.95 0.95 1.16 1.11 0.87 0.66 1.09 1.07 1.13 0.83 1.15 1.32 1.22 1.09 0.89 1.12 1.09 1.77 1.07 0.95 1.08 1.08 0.62* 0.49* 1.52 0.85 0.61* 0.65 0.86 1.02 0.54* 0.70 Note. ∗p < 0.05. Unweighted N = 6,050. Additional covariates include student and additional school characteristics in Table 1 (except interest) and cluster/school-averages of the student characteristics in Table 1. I also controlled for an indicator of instructional hours in social studies (overall) and indicators of the proportion of time spent on civics, U.S. history, and geography instruction. Estimates for additional covariates can be found in Table B3 in Appendix B. 64 Table 10. Odds ratios for instructional exposure model (U.S. history sample) s e r u t c e L s t e e h s k r o W l a i c o S s e i d u t s k o o b t x e t s w e n / s k o o B s e n i z a g a m / y r a m i r P s t n e m u c o d o e d i v / m l i F e v i t a t i t n a u Q a t a d r e t u p m o C s n o i t a l u m i s k c o M e l o r / l a i r t y a l p & s e t a b e D l e n a p s n o i s s u c s i d s s u c s i D t n e r r u c s t n e v e y t i n u m m o C r o t i s i V s r e t t e l e t i r W y t i n u m m o C s t c e j o r p 0.69 0.78 1.17 1.10 0.35* 0.92 Undergraduate History/hist. ed. Geog./geog. ed. Poli sci Soc. sci./social studies ed Other soc. sci. Ed/sec. ed. Grad coursework Graduate History/hist. ed. Geog./geog. ed. Poli sci Soc. sci./social studies ed Other soc. sci. Ed/sec. ed. Alt cert NBPTS >5 yrs SS teaching exp 0.99 2.57 1.42 0.82 0.89 1.73* 0.51 1.10 1.41 0.14* 1.38 0.80 1.06 2.17* 0.53 0.73 1.88 0.72 1.55 0.69 0.76 1.52 0.65 3.06* 0.62 1.10 0.96 1.16 2.18* 0.75 0.46* 1.50 0.90 1.06 0.91 0.82 1.05 1.52 1.07 1.21 0.86 0.85 1.71* 0.66 1.43 0.86 0.75 0.79 1.17 1.06 0.94 1.51 0.78 0.92 2.63* 1.15 1.01 1.07 0.89 2.40 0.93 0.69 1.14 0.61 0.52* 0.84 0.87 1.19 2.26 1.46 0.59 0.91 0.79 1.37 1.12 1.74 1.39 1.61 0.74 1.19 0.94 1.34 0.66 0.51 1.34 0.99 1.31 1.34 1.06 1.02 1.13 1.11 1.04 0.77 0.98 1.06 0.98 0.83 0.55 2.14 0.89 1.04 0.96 0.95 1.25 0.87 1.11 0.92 3.05* 0.85 0.71 1.35 0.92 0.72 1.02 1.20 1.28 1.18 1.95 0.54 0.67 0.56 1.24 0.97 1.45 0.77 1.23 0.83 0.89 1.27 0.98 0.84 0.93 0.55 0.84 1.42 0.33* 1.38 1.38 0.55* 1.24 1.31 0.73 0.67 1.55 1.22 2.62* 1.08 1.44 5.67* 0.59 0.31* 0.57 0.95 0.94 1.35 1.20 0.95 1.72 1.37 0.73 1.07 1.05 1.96* 1.52 0.18* 2.01 1.27 0.98 1.15 1.34 0.83 0.53 1.67 4.44* 1.07 1.14 1.02 1.97 1.40 0.94 2.35 1.24 0.85 0.55 1.13 0.49 1.84 1.15 1.58 1.07 0.89 0.82 0.94 0.69 1.76* 0.91 1.11 7.43* 1.33 0.75 1.39 1.31 3.06* 1.33 0.98 1.22 1.13 0.53* 0.50* 1.41 0.69 0.65 0.59 0.92 1.03 0.46* 0.61 Note. ∗p < 0.05. Unweighted N = 7,470. Additional covariates include student and additional school characteristics in Table 1 (except interest) and cluster/school-averages of the student characteristics in Table 1. I also controlled for an indicator of instructional hours in social studies (overall) and indicators of the proportion of time spent on civics, U.S. history, and geography instruction. Estimates for additional covariates can be found in Table B4 in Appendix B. 65 Table 11. Odds for instructional exposure model (geography sample) Undergraduate History/hist. ed. Geog./geog. ed. Poli sci Soc. sci./social studies ed Other soc. sci. Ed/sec. ed. Grad coursework Graduate History/hist. ed. Geog./geog. ed. Poli sci Soc. sci./social studies ed Other soc. sci. Ed/sec. ed. Alt cert NBPTS >5 yrs SS teaching exp s e r u t c e L 0.77 3.53* 1.59 0.76 0.84 1.55 s t e e h s k r o W 0.96 1.52 0.62 1.74* 0.64 0.94 l a i c o S s e i d u t s k o o b t x e t 0.44* 1.05 1.16 0.93 0.85 0.76 s w e n / s k o o B s e n i z a g a m / 0.93 1.44 1.20 0.95 0.76 0.89 y r a m i r P s t n e m u c o d 1.43 0.68 1.56 0.87 1.14 0.75 o e d i v / m l i F 1.02 1.19 0.84 1.57 0.76 1.24 e v i t a t i t n a u Q a t a d 0.87 1.27 0.99 0.59 1.04 1.00 r e t u p m o C s n o i t a l u m i s 0.67 1.41 1.06 2.27* 0.92 0.71 k c o M e l o r / l a i r t y a l p 0.91 0.77 0.80 1.18 1.51 1.13 S I G 1.04 0.90 0.94 1.07 1.17 0.88 & s e t a b e D l e n a p s n o i s s u c s i d s s u c s i D t n e r r u c s t n e v e 1.27 1.53 0.67 0.65 0.75 1.08 1.09 1.30 0.90 1.34 1.02 1.07 y t i n u m m o C r o t i s i V 1.27 0.77 0.67 0.83 0.95 0.86 s r e t t e l e t i r W 1.09 0.38* 1.49 1.10 0.53* 0.93 y t i n u m m o C s t c e j o r p 0.74 0.90 0.50 0.89 0.53 0.91 0.70 1.76 3.52* 1.03 1.10 0.94 0.86 1.77 0.80 0.88 0.85 1.82 2.07 3.21* 1.65 1.02 1.86 0.18* 1.08 0.47 0.86 1.90 0.48* 0.54 2.23 0.69 0.93 1.90 1.24 1.76 0.69 0.90 3.05 1.11 0.94 0.94 0.45* 0.38* 0.85 0.85 1.67 2.14 2.15* 0.62 0.93 0.73 1.61 1.04 1.16 1.28 1.61 1.02 0.92 1.10 1.18 0.60 0.80 1.15 1.06 1.55 1.45 1.06 0.87 0.58 0.40 1.78 1.36 1.74 0.91 0.94 0.94 1.25 3.33 0.65 0.45 0.72 0.91 0.87 1.38 1.43 1.16 1.45 0.98 0.70 0.97 0.95 1.74 0.91 0.83 0.83 1.23 1.11 0.94 1.11 0.96 1.46 0.14* 2.06 1.50 1.07 1.23 1.21 0.75 0.45* 1.64 7.74* 1.31 1.10 0.67 1.33 1.33 0.93 5.99* 1.28 1.05 0.10* 0.69 0.61 1.87 1.48 2.71 1.12 0.96 0.41 0.74 0.62 1.50 1.09 0.56 7.31* 1.53 0.89 1.56 1.27 2.73* 1.28 0.85 1.03 1.19 0.54* 0.57* 1.67 0.81 0.82 0.91 0.61 0.79 0.89 0.44* 0.49* Note. ∗p < 0.05. Unweighted N = 5,280. Additional covariates include student and additional school characteristics in Table 1 (except interest) and cluster/school-averages of the student characteristics in Table 1. I also controlled for an indicator of instructional hours in social studies (overall) and indicators of the proportion of time spent on civics, U.S. history, and geography instruction. Estimates for additional covariates can be found in Table B5 in Appendix B. 66 DISCUSSION Social studies in the middle grades has been under-researched in recent years in the United States. As discussed previously, adolescents are undergoing tremendous cognitive and social development during these years (Anderman, 2012), which allows them to more fully consider complex/abstract concepts and their emerging roles in society. Yet scholars have found that adolescent motivation decreases and boredom increases during the middle grades and rather than becoming more complex, academic work becomes less ambitious, in terms of cognitive demand (Eccles & Roeser, 2011). Furthermore, scholars of social studies education have pointed to teachers and teacher education as playing a primary role in the lack of alignment between the needs of young adolescents and their educational experiences during these years (Conklin, 2008, 2009, 2012; Conklin et al., 2010). In this section, I briefly summarize the results of this study that extend the research on middle grades social studies teaching and learning; situate the results within the extant literature by pointing out consistencies and inconsistencies and highlighting ways that I have extended this scholarship; and discuss the limitations of this study, which are primarily methodological. I conclude by discussing implications for policy and practice and suggesting areas for future research in this critical field. Summary of Results This dissertation was guided by three research questions: 1) To what extent are teacher background characteristics and students’ opportunity to learn associated with eighth-grade student achievement in civics, U.S. history, and geography? 67 2) To what extent is students’ exposure to various instructional practices associated with student interest in eighth-grade social studies? 3) To what extent are teacher background characteristics associated with students’ exposure to various instructional practices in eighth-grade social studies? RQ1: Student achievement. To answer the first research question, I estimated the extent that opportunity to learn/OTL (latent measures of curricular exposure in eighth grade) and teacher background (subject matter preparation, alternative certification, National Board certification, and years of experience) were associated with student assessment outcomes in civics, U.S. history, and geography, conditional on important student, teacher, and school covariates. Both OTL and, to a lesser extent, teacher background were associated with student achievement in civics and U.S. history. OTL civics and OTL U.S. history were positively associated with their respective assessments. Only subject matter preparation in political science at the undergraduate level and general social science/social studies education at the graduate level were associated with student achievement on NAEP civics and NAEP U.S. history, respectively. RQ2: Student interest. To answer the second research question, I estimated the extent that measures of student instructional exposure (exposure to various instructional practices available in the NAEP data) were associated with measures of student-reported interest, conditional on important student, teacher, and school covariates. Several of the instructional practices measured in NAEP and modeled in this study were associated with student interest. Students who reported being exposed to GIS during geography instruction were more likely to report being frequently interested in class and more likely to report geography/social studies as a favorite subject. Across all three samples, teachers who reported engaging students in 68 community projects had students who were more likely to report being frequently interested in their social studies class. In both the geography and U.S. history samples, the use of community projects also had a strong association with students reporting geography/social studies or history/social studies as a favorite subject. Frequent use of film/video or any use of mock trials/roleplay in the geography sample was also positively associated with students reporting geography/social studies as a favorite subject. Several of the instructional practices modeled in this study also had negative relationships to student interest. Students whose teachers reported frequent use of computer simulations in their civics/social studies classes were slightly less likely to report being frequently interested in the class. In the U.S. history sample, frequent use of a social studies textbook as part of instruction or having students frequently write letters were both negatively associated with frequent student interest. RQ3: Instructional exposure. To answer the third research question, I estimated multivariate models across each of the three samples to quantify the extent that background characteristics of students’ teachers were associated with student exposure to various instructional practices. While this analysis was largely exploratory, with very little prior research on which to ground this part of the study, several significant relationships were estimated that will help to advance the field. Recommendations for future inquiry are discussed in a later section. For example, across all samples, students of teachers with an undergraduate focus on history/history education had a lower likelihood of exposure to frequent use of a social studies textbook. In both the civics and geography samples, students of teachers with this background at the graduate level were less likely to be exposed to discussing current events. 69 Students of teachers with a background in political science also varied in their instructional exposure. No significant associations were estimated for an undergraduate focus in political science, but a graduate focus in political science was associated with several instructional practices. Across all samples, students of teachers who reported a graduate focus in political science were less likely to be exposed to frequent lecture and more likely to have had exposure to discussing current events. In both the U.S. history and geography samples, students of these teachers were also more likely than other students to have had exposure to community projects. A background in geography/geography education for students’ teachers was also associated with instructional exposure. In the geography sample, students whose teachers had an undergraduate focus in geography/geography education were more likely to have been exposed to frequent lecture. Students of teachers with this background across all samples were also less likely to have had exposure to writing letters. In all samples, students of teachers with a graduate focus on geography/geography education were less likely to have had exposure to participating in debates or panel discussions. In the U.S. history sample only, students of teachers with this background at the graduate level were more likely than others to have had exposure to frequent worksheet use and computer simulations. In the geography sample only, students of teachers with this background were more likely than others to have been exposed to community visitors. Other subject matter backgrounds for teachers were also associated with student instructional exposure. For example, students of teachers with any graduate coursework were more likely to have been exposed to frequent lecture as well as writing letters, across all samples. Students of teachers reporting an undergraduate background in “other social science” were less likely to be exposed to community projects (civics and U.S. history samples) and less likely to be 70 exposed to writing letters (U.S. history and geography samples). Regarding a background in education/secondary education, students of teachers with this training at the undergraduate level in the U.S. history sample were more likely to be exposed to frequent lecture, and students in the geography sample of teachers with this background at the graduate level were less likely to be exposed to frequent textbook use. Initial certification type was also associated with student instructional exposure. Students of alternatively-certified teachers in both the civics and U.S. history samples were more likely than other students to have been exposed to frequent lecture and worksheets. In the civics sample only, students of these teachers were more likely than other students to have been exposed to frequent discussion of current events. Across all samples, students of alternatively-certified teachers were less likely than other students to have been exposed to frequent textbook use. NBPTS certification also had several significant associations with student instructional exposure. Students of NBPTS-certified teachers in the civics and U.S. history samples were more likely to have been exposed to writing letters. Students of these teachers in the U.S. history sample were also more likely to have been exposed to mock trials/roleplay. In both the U.S. history and geography samples, students of NBPTS-certified teachers were more likely to have been exposed to community projects. Finally, in both the civics and geography samples, students of these teachers were less likely to have been exposed to frequent lecture. Conditional on all other covariates, students of experienced social studies teachers also had varying instructional exposure relative to students of novice social studies teachers. Across all samples, students of experienced social studies teachers were less likely to have been exposed to frequent use of primary documents, film/video, and also less likely to have had exposure to 71 writing letters in the classroom. In the geography sample only, students of experienced social studies teachers were less likely to have had any exposure to community projects. Across all three samples, private school students were more likely than traditional public school students to have been exposed to community projects. Private school students in the U.S. history and geography samples were also more likely to have been exposed to frequent use of a social studies textbook as part of instruction. Finally, traditional public school students in the civics sample were more likely than private school students to have been exposed to frequent worksheets. Charter school students also varied with regard to their instructional exposure relative to students in traditional public schools. In the civics sample, charter school students were more likely to have been exposed to frequent lecture and less likely to have been exposed to frequent use of books/newspapers/magazines and primary documents. In the U.S. history sample, the same was true regarding primary documents (i.e., charter school students had less frequent exposure). In the geography sample, the same finding was estimated regarding books/newspapers/magazines (i.e., charter school students had less frequent exposure). In both the U.S. history and geography samples, charter school students were less likely to have been frequently exposed to quantitative data. Situating the Findings in the Existing Literature In this sub-section, I discuss how the findings in this study relate to the existing literature base. Specifically, I focus on how my findings diverge, confirm, and expand on prior research. I discuss each of the research questions separately, by the outcome modeled in each question. Student achievement. Prior social studies research has analyzed the extent that student characteristics and instruction are related to student achievement (Barton & Avery, 2016). 72 However, this study was the first to model sophisticated measures of OTL across multiple social studies disciplines and estimate the relationship to student achievement in multiple social studies disciplines. As was found in other subject areas (Brophy, 1986; Brophy & Good, 1986; Schmidt & Maier, 2009) and with related measures in history and civics using data from the late 1980’s and early 1990’s (Niemi & Junn, 1998; J. B. Smith & Niemi, 2001), increasing OTL is associated with higher student achievement in both NAEP civics and NAEP U.S. history. Those interested in geography education, however, are likely somewhat disappointed to see a null relationship between OTL geography and NAEP geography (among other null relationships between inputs and NAEP geography achievement). One potential explanation for this could have to do with the lack of content knowledge among social studies educators in geography (Bednarz et al., 2013; Segall & Helfenbein, 2008). In addition to potential ineffective/inadequate curricular and instructional implementation specific to geography, social studies teachers may have had minimal knowledge of what constitutes, for example, space and place or spatial dynamics when responding to the survey items underling the latent OTL geography variable. More research is needed within geography education, an exceptionally important but largely overlooked discipline in American education (Bednarz et al., 2013). Prior research was mixed regarding teacher background characteristics and student achievement outcomes in social studies education, depending on the grade level, research design, and sample studied. The one available study with a causal research design and at the grade level in this study did suggest that in-subject preparation for eighth-grade social studies teachers positively impacted student achievement (Dee & Cohodes, 2008). Research in North Carolina suggested that initial certification type was only important when comparing a highly selective 73 program (such as TFA) with other pathways, regarding its impact on student achievement (Henry, Bastian, et al., 2014; Henry, Purtell, et al., 2014) This study was the first to break down different forms of disciplinary training and estimate relationships to student achievement on multiple disciplinary assessments. Students whose teachers had an undergraduate focus on political science did have slightly higher NAEP civics scores, conditional on other covariates. Among other available teacher background characteristics in this study (including alternative certification, NBPTS certification, years of experience, and other subject matter backgrounds), the only other significant finding was an association between teachers’ graduate focus on general social science/social studies education and student achievement on NAEP U.S. history. Selectivity measures of various alternative pre- service pathways was not available in this study. This study was an initial step forward in analyzing teacher background characteristics as they relate to student achievement in social studies disciplines, and more research is needed. Student interest. The analysis of student interest as a distinct construct in relation to educational inputs has been understudied in social studies. The existing literature base has suggested that social studies students are interested in writing, class discussions, debates, roleplay, creative tasks, and well-designed/challenging computer simulations, and authentic intellectual work (Del Favero et al., 2007; Gehlbach et al., 2008; King et al., 2015; Marks, 2000; Stevenson, 1990). This was the first large-scale study in the U.S. to estimate relationships between exposure to various instructional practices and student interest in their social studies classrooms. Two of the most important findings included student interest in GIS as part of geography instruction and student interest in community projects. Prior qualitative and mixed-methods research was 74 somewhat suggestive of elementary student interest in GIS (Shin, 2006, 2007), but this study was the first to estimate this relationship at a large-scale and with eighth-grade students. While student interest was never specifically modeled regarding community projects or service- learning, prior research has shown that this form of instructional exposure was linked to a host of positive/desired civic outcomes (Feldman et al., 2007; Kahne et al., 2013). Although never modeled empirically, a likely precursor to such civic outcomes would be student interest (as well as student engagement) in their social studies classrooms. This should be studied in further depth, which I discuss in a later section. Several unexpected findings were also estimated in this study. Negative relationships were estimated with student interest for frequent use of the social studies textbook, writing letters (both in the U.S. history sample) and frequent use of computer simulations (in the civics sample). These are fairly broad instructional approaches, but these findings should be investigated further in future research. Instructional exposure. Prior research had primarily focused on student characteristics as predictors of instructional exposure (e.g., Kahne & Middaugh, 2008). This study, however, suggests that teacher and school characteristics are also important to consider. As was discussed in the literature review, no recent empirical literature had focused on variation in instructional exposure for students in social studies classes based on either teachers’ background characteristics (beyond years of experience) or school sector. As Wineburg and Wilson (1988) concluded in their qualitative study 30 years ago, this study suggests that the various forms of disciplinary training for social studies teachers may lead to different instructional approaches taken within their social studies classrooms, at least as reported by the teachers themselves. 75 Initial certification type, advanced professional certification, and years of experience each were associated with distinct forms of instructional approaches, as well. While only exploratory and descriptive, some of the significant findings for these characteristics of teachers would be expected by social studies experts while others are somewhat puzzling. Not surprising is that students of teachers with National Board certification tended to be more exposed to what some in the field would refer to as ambitious forms of instruction. Some of the frequent forms of instructional approaches that students of teachers who were alternatively certified for initial certification may be referred to as less ambitious (e.g., lecture and worksheets). On the other hand, the findings for veteran versus novice teachers, conditional on their other characteristics, were puzzling. While some may expect veteran teachers to be more ambitious in their instruction as they develop further professional knowledge over time, this did not appear to be the case. Whether this is due to burnout later in teachers careers or due to better training for new teachers who recently entered the field is an empirical question in need of further study. Given the descriptive evidence from prior research documenting the importance of instructional approaches for student outcomes in social studies, the descriptive evidence in this study (i.e., variation in approaches across varying teacher backgrounds) suggests that students may not only have unequal access to particular instructional practices based on their own characteristics (as found in prior research) but also that their teachers’ background may, in part, be driving this variation. The analyses concerning this research question, however, were merely exploratory, and further analysis of this line of inquiry is crucial. While not an initially intended line of inquiry in this study, the models estimated in this study also uncovered some very interesting differences in student instructional exposure by school sector, which to my knowledge hasn’t yet been researched. Private schools, in particular, 76 have been largely absent from the social studies literature, with the exception of some overlap between the political socialization and school choice literatures (Campbell, 2012). This study presented descriptive evidence of different instructional approaches across private, traditional public, and charter schools. Of particular importance was the finding that private school students were more likely to have been exposed to community projects. Given findings in this study regarding the positive association between exposure to community projects and student interest, prior research finding that participation in community projects predicts civic engagement (Feldman et al., 2007; Kahne et al., 2013), and prior work suggesting positive civic outcomes for private school students relative to traditional public school students (Campbell, 2012), instructional exposure could be one missing link in this causal chain that continues to perplex researchers concerned with sector differences in civic outcomes (Campbell, 2012). Summary. This study has taken a small step forward in advancing scholarship in social studies education. Advances include discussion of multiple social studies disciplines, modeling of recent eighth-grade social studies data, sophisticated measurement of OTL in social studies, consideration of varying backgrounds of social studies teachers at a large scale and the associations with student achievement and implemented instructional practices, inclusion of both public and private school students and analysis of cross-sector differences, and the study of student interest in social studies as it relates to exposure to various instructional practices implemented within social studies classrooms. These analyses have suggested some implications for policy and practice and have also pointed to several future avenues of research, which are discussed in later sections. Prior to these implications and recommendations, I first present this study’s limitations. 77 Limitations There were four primary groups of limitations in this study. Perhaps the most serious of these limitations is the cross-sectional research design and lack of prior measures. In no way should estimates in this study be considered causal, but rather correlational. As with any study, repeated measures (or an experimental or quasi-experimental design) would have been preferred to allow for causal estimates. There were measures of prior course taking on the student surveys; however missing data and responses of “I don’t know” were so frequent that these measures were not very useful. Second, the sampling and sample restrictions also brought about limitations. The three samples (civics, U.S. history, and geography) are not necessarily students in civics, U.S. history, or geography courses. There were no survey items that asked the teacher or student to name the course title. This issue was mitigated by restricting the sample to students whose teachers reported giving at least some instruction in the assessed discipline. Additionally, in models of student achievement, the relevant OTL variable also acts as a control for the extent that the teacher focused on the assessed subject in the classroom. I also included measures for total instructional time in social studies generally (in models for both achievement and instructional exposure as outcomes) as well as the proportion of time spent on each of the three disciplinary areas (for models of instructional exposure), as discussed in the method section. In addition, while the sampling was meant to be nationally representative, 15 states were dropped from the analysis after restricting to students whose teachers completed a survey. It’s not documented why all teachers across these states chose not to complete the survey, but nevertheless, this does impact the generalizability of the results, to an extent. 78 Third, the measurement of NAEP survey items that I used in this study also presented some limitations. As is true in many large-scale survey studies, many of the variables were measured as ordered-categorical/ordinal. Rather than model each individual category, I chose to cut the variables at the point that made the most substantive sense, and in part, based on the distributions in the data. This resulted in many of the measures being binary, where the underlying constructs were often truly continuous measures. Additionally, measures of both student interest and instructional exposure were somewhat crude. Measures of student interest are typically measured using extensive instruments with a set of indicators. More specific measures on instructional practices (in addition to quality of teachers’ implementation of these instructional practices) would also be ideal. Furthermore, while the breakdown of subject matter preparation is a unique contribution to the literature, it would be reasonable to take issue with combining, for example, history and history education or geography and geography education, as was done in the NAEP surveys. Many historians likely perceive traditional disciplinary history coursework very differently from history education coursework. The same could be said for geography. Others (e.g., general social sciences/social studies education) are also quite broad, and also not ideal measures. These limitations were largely due to the nature of the data. I have attempted to investigate the research questions in this study as best I could by using this existing data. The limitations mentioned point to the need for more/better data in social studies education in order to further investigate the lines of inquiry in this study, among others. Implications for Policy and Practice While it would be inappropriate to give specific recommendations based on correlational/descriptive research, several findings in this study are worth considering with 79 regard to recommendations for policy and practice. First, given the evidence of a decrease in focus on social studies in U.S. schools and the findings regarding the relationship between OTL and student achievement in both this study and other subject areas, we need to consider the ramifications. Simply put, students learn more when they are actually taught and allowed to learn the content. Educational stakeholders need to consider the impact of both the decreased focus in social studies education in recent years and variation in students’ opportunity to learn important curricular topics. Second, this study suggests that the field may want to pay more attention to how social studies teachers are certified and prepared. This study provided descriptive evidence that a) specific subject matter preparation is associated with student achievement and b) teachers with varying backgrounds report teaching quite differently, which may also result in varying student interest, achievement, and civic outcomes, as suggested by this study and prior research. While further research is needed to substantiate the descriptive findings in this study, colleges of education and providers of teacher preparation may want to consider how varying educational/preparatory experiences for their teacher candidates may lead to varying experiences and outcomes for their future social studies students. Specifically, providers should place a greater focus on the knowledge their teacher candidates leave with in specific disciplines and how such knowledge aligns with the disciplines they will be expected to teach. Furthermore, given the importance of specific instructional approaches, providers should emphasize how various approaches stimulate student learning and interest and give teacher candidates practical experiences during preparation so that they are confident and able to implement such approaches effectively once they enter the workforce. 80 Third, given the very different student populations they serve, it is unfortunate to see private school students being more likely than traditional public school students to have an opportunity to participate in more innovative instructional approaches, such as community projects. Given the research discussed in the literature review regarding how important these types of opportunities are for students (Feldman et al., 2007; Kahne et al., 2013), the field should take issue to this inequity between private and public school students. Recommendations for Future Research Very little experimental or quasi-experimental research on social studies education exists in the U.S. Cross-sectional studies are suggestive, but more research is needed that estimates causal effects. All findings in this study should be substantiated with causal research designs. Specifically, longitudinal studies and the inclusion of mediation effects is crucial in social studies. I have hinted throughout this study at the potential mediating effects of a construct such as interest on outcomes such as knowledge, skills, student classroom engagement and civic/political participation. This, quite simply, has not been modeled in the U.S. with respect to social studies. The ways in which these constructs, among others, are related to each other is undoubtedly complex, and models (with the needed longitudinal data) should take this complexity into account. Other specific lines of inquiry are crucial moving forward. First, the field needs to move toward psychometric measurement and empirical study of various teacher knowledge constructs in the social studies disciplines as has been done in math, for example (Hill, Rowan, & Ball, 2005). I considered training in various disciplinary areas in this study, a weak proxy for teacher knowledge, but more specific measurement and analysis of teacher knowledge is needed to have a better understanding of what our social studies teachers know/know how to do, to potentially 81 improve teacher education accordingly. Measures of content knowledge in various social studies disciplines are available in subject-specific teacher licensure data, but, to my knowledge, have not been studied empirically. Limited qualitative research has studied pedagogical content knowledge (PCK) social studies, focused on history (e.g., Gudmundsdottir & Shulman, 1987; Monte-Sano & Budano, 2013). However, given social studies education is such a contested field and the lack of consensus as to what constitutes PCK in social studies (Powell, 2018), the field lags behind other subject areas. Second, more recent research considering OTL has expanded on earlier definitions to include measures of instructional/pedagogical quality (e.g.,Polikoff & Porter, 2014). While not always discussed in the framework of OTL, instructional quality is a major focal point in recent educational policy and practice. Most of the large-scale social studies scholarship focusing on instruction has not considered the quality of instruction. While the quality of the learning environment for instruction has been considered as it relates to classroom discussion, for example, research has yet to apply psychometric measures such as existing classroom observational rubrics to social studies instruction. These data would be available in any U.S. state that evaluates all of its teachers; however, no research exists, to my knowledge, that focuses specifically on social studies teachers. In addition to analyzing instructional quality in the classroom, these data could also be used to analyze variation in teacher education experiences as has been done generally across subjects at a large scale (Ronfeldt & Campbell, 2016). Beyond measures of general pedagogical quality, the field should move toward social studies specific measures and empirical study of how such measures of instructional quality relate to both outcomes of interest as well as teacher and school characteristics. 82 Third, the measures of student interest used in this study were fairly crude, as mentioned above. Instruments, such as those used in Del Favero et al. (2007), should be used in the U.S. context to study the relationship between an extensive set of instructional measures, among other important independent variables in social studies education. This is in addition to the potential mediating effects student interest likely has on a number of important proximal and distal outcomes within the social studies education and political socialization space, as mentioned above. Finally, as could be said for all school subjects, the K-12 and postsecondary research communities need to become better aligned. This is particularly true in social studies, where there is an interest in later civic outcomes in adulthood. Political scientists, for example, conduct classroom-based experiments in their programs frequently (Bennion & Laughlin, 2018), very similar to the research designs used in the earlier process-product research in K-12. These scholarly communities could learn a lot from coordinating with one another to advance the field. Conclusion As has been mentioned throughout this study, consideration of knowledge, skills, and interest in social studies and its underlying disciplines and the relationship to educational inputs (teachers/teaching in particular) is of vital importance and needs to be brought to the forefront of educational policy discussions. At such a crucial stage of development, the middle grades are a necessary grade level for further inquiry, in addition to the existing research at the elementary, high school, postsecondary levels. Again, Generation Z is the future, and if social studies education continues to be overshadowed by other important subject areas in research, policy, and practice, the country would be doing a great disservice to those who will be tasked with moving the nation forward in the coming years. The world and its inhabitants face a difficult road ahead 83 as we grapple with significant political, societal, economic, and environmental challenges. The educational research community, policymakers, and educators can play an important role in improving the status and quality of social studies education to ensure that each generation is better prepared than the last to take on these challenges. 84 APPENDICES 85 APPENDIX A NAEP SAMPLE QUESTIONS (Grade 8) Civics Sample Items The following two questions refer to the passage below: Over recent years the National Rifle Association (NRA) has played an important role in politics. It has a large number of members whose voting strength tends to impress legislators. It holds regular meetings with members of Congress and federal bureaucrats. It gives campaign contributions to candidates who oppose gun control and spends money directly to oppose gun control supporters. 1. The meetings with legislators and bureaucrats described in the passage are known as A. lobbying B. vetoing C. filibustering D. gerrymandering 2. Which of the following would the National Rifle Association (NRA) probably cite in support of its position on gun control? A. The Declaration of Independence B. The Preamble to the Constitution C. The First Amendment to the Constitution D. The Second Amendment to the Constitution 86 The following two questions refer to the situation below: Amanda and her friends have noticed these two problems in their neighborhood. Problem 1: Problem 2: The garbage cans in the public park are overflowing. Many younger children have trouble crossing the busy streets on their way home from school. 3. What is one thing Amanda and her friends could do on their own to help solve Problem 1? What is one thing Amanda and her friends could do on their own to help solve Problem 2? 4. Tell one thing local government could do to help solve Problem 1. Tell one thing local government could do to help solve Problem 2. 5. A central purpose of the United Nations is to A. establish democratic governments worldwide B. maintain international peace and security C. decide where national boundaries lie D. bring international criminals to justice 6. In the Gettysburg Address, President Abraham Lincoln described the United States government as "government of the people, by the people, for the people." Think about the three different parts of Lincoln's description, and EXPLAIN in your own words what he meant by "of the people, by the people, for the people." 87 U.S. History Sample Items Question refers to the list below: Imagine that you are studying the restriction of immigration to the United States in the 1920's. You have the following sources of information available to you. 1. A 1924 newspaper editorial supporting the quota system that limited immigration 2. The text of the 1924 Immigration Act (or the National Origins Act) 3. The text of a speech made at a Ku Klux Klan rally in 1923 4. A Russian woman's account of her experiences during the Russian Civil War and her escape to the United States in 1922 5. A 1924 magazine editorial opposing the quota system that limited immigration 6. The story of a Greek family's journey to the United States in 1906 written by a journalist in 1955 7. The 1910, 1920, and 1930 United States census data telling how many immigrants came from which countries 1. Identify one source that would probably help you understand why immigrants wanted to come to the United States. Explain why you chose that source. 2. The Monroe Doctrine, the Good Neighbor Policy, and the Alliance for Progress were all United States foreign policy positions relating directly to A. Latin America B. Africa C. the Middle East D. China 3. The Great Society programs of the 1960's were primarily based on the idea that A. American society was hopelessly flawed B. the federal government should play an active role in promoting social welfare C. the poor needed to work harder in order to succeed D. the federal government lacked the authority to help the poor 88 Question refers to the map below: 4. How did the building of the railroad system shown on the map affect the city of Chicago? A. Chicago became smaller because the railroads allowed many people to leave. B. Chicago became poorer because more trade moved to the southern states. C. Chicago became an important transportation center because of its central position in the railroad system. D. The railroads allowed Chicago to conduct more trade with Europe than any other city in the country conducted. 5. At the Constitutional Convention in 1787, the large states and the small states disagreed with each other about how the new government should be structured. Identify the most important issue that large and small states disagreed about. Explain how this issue was resolved by the Connecticut (Great) Compromise. 89 Question refers to the photograph below: 6. What social policy is reflected in the photograph? Identify one consequence of this policy for African Americans. The New York Times 90 Geography Sample Items Question refers to the map below. 1. The number 2 on the map is on the A. Colorado River B. Columbia River C. Mississippi River D. Delaware River 2. Tropical forests are being destroyed at the rate of at least eleven million hectares each year, an area the size of Pennsylvania. About half of all tropical forests are already gone. Discuss two major reasons for this high rate of tropical deforestation. 91 3. Look at the table above. The region described in this table is A. South America B. Africa C. North America D. Oceania 4. Switzerland is located in the middle of which mountain range? A. Alps B. Andes C. Urals D. Pyrenees 92 Question is based on the map below, which shows public train routes: 5. Directly on the map, draw the most direct public train route you would take to get from Boston College to Braintree. 93 6. Look at the chart above. In what way is the first set of six countries shown on the chart different from the second set of six countries? Identify two factors that account for the difference between the two sets of countries. 94 7. Which city probably suffered the most damage in the earthquake? A. Lappington B. San Miguel C. Sun City D. Biddleburg 95 APPENDIX B SUPPLEMENTAL TABLES Table B1. Coefficients for student and school covariates in achievement models Variable NAEP Civics NAEP U.S. History NAEP Geography Student Characteristics Female African American, not Hispanic Hispanic of any race Asian American/Pacific Islander Another race or ethnicity Free/reduced-price lunch Books in the home (1 bookcase or more) Limited English proficient (LEP) Individualized education program (IEP) Class is frequently interesting Talk about studies at home (minimum = weekly) School Characteristics Private Charter City Town Rural Northeast Midwest South Cluster/School Averaged * 10 Female African American, not Hispanic Hispanic of any race Asian American/Pacific Islander Another race or ethnicity Free/reduced-price lunch Books in the home (1 bookcase or more) Limited English proficient (LEP) Individualized education program (IEP) Talk about studies at home Unweighted N RMSEA -0.05* -0.40* -0.20* 0.12* -0.13 -0.25* 0.32* -0.97* -0.83* 0.19* 0.12* -0.13 0.05 -0.02 -0.15* -0.17* 0.04 0.17* 0.21* 0.02 0.00 0.02* 0.01 -0.01 -0.04* 0.05* 0.01 0.00 -0.01 5,590 0.01 -0.21* -0.37* -0.18* 0.13* -0.04 -0.26* 0.33* -0.82* -0.77* 0.23* 0.15* -0.09 0.07 0.02 -0.17* -0.03 0.20* 0.23* 0.29* -0.02 -0.02 0.01 0.02 0.05 -0.03* 0.06* -0.03 0.05* -0.02 7,120 0.01 -0.18* -0.54* -0.18* 0.14* -0.21* -0.21* 0.34* -0.82* -0.81* 0.24* 0.11* -0.08 0.13 -0.02 -0.03 -0.10 0.12 0.23* 0.17* -0.02 -0.03* -0.01 -0.02 0.01 -0.02* 0.04* -0.06* -0.01 0.02 4,630 0.01 Note. ∗p < 0.05. Relevant reference groups are White, not Hispanic; traditional public schools; suburbs; and the West census region. 96 Table B2. Coefficients and odds ratios for student and school covariates in interest models U.S. History sample Geography sample Frequent Interest Logit OR Favorite Logit OR Frequent Interest Logit OR Favorite Logit -0.63* -0.34* 0.03 0.34 -0.14 0.06 0.38* 0.22 0.03 0.62* -0.03 0.25 -0.24 0.30 -0.02 -0.03 0.06 -0.03 -0.02 0.05 0.03 -0.01 0.09 -0.01 0.00 0.02 0.00 0.00 OR 0.53* 0.71* 1.03 1.40 0.87 1.06 1.46* 1.25 1.03 1.86* 0.97 1.28 0.78 1.35 0.98 0.98 1.06 0.97 0.98 1.05 1.03 0.99 1.10 1.00 1.00 1.02 1.00 1.00 Student Characteristics Female African American, not Hispanic Hispanic of any race Asian American/Pacific Islander Another race or ethnicity Free/reduced-price lunch Books in the home Limited English proficient Individualized education program Talk about studies at home School Characteristics Private Charter City Town Rural Northeast Midwest South Cluster/School Averaged * 10 Female African American, not Hispanic Hispanic of any race Asian American/Pacific Islander Another race or ethnicity Free/reduced-price lunch Books in the home Limited English proficient (LEP) Individualized education program Talk about studies at home Civics sample Frequent Interest Logit OR -0.50* 0.02 0.11 -0.02 -0.01 0.10 0.36* 0.11 0.07 0.80* -0.08 0.34* 0.02 0.06 -0.14 -0.02 -0.06 -0.07 -0.07 0.00 0.00 0.00 0.07 0.02 -0.01 0.01 0.00 0.03 0.61* 1.02 1.11 0.98 0.99 1.11 1.43* 1.11 1.07 2.22* 0.92 1.40* 1.02 1.06 0.87 0.99 0.94 0.93 0.93 1.00 1.00 1.00 1.07 1.02 1.00 1.01 1.00 1.03 -0.56* -0.11 0.17* 0.07 -0.20 0.11 0.30* 0.27* 0.06 0.62* 0.00 0.11 -0.05 0.29* 0.03 0.28* 0.00 -0.14 0.01 0.03 0.04 -0.01 0.00 0.00 -0.01 0.05 0.10* 0.06 0.57* 0.90 1.19* 1.07 0.82 1.11 1.35* 1.31* 1.06 1.87* 1.00 1.12 0.95 1.34* 1.03 1.33* 1.00 0.87 1.01 1.03 1.04 0.99 1.00 1.00 0.99 1.06 1.10* 1.06 -0.65* -0.38* 0.06 0.23 -0.37* 0.05 0.22* 0.43* 0.07 0.54* -0.05 -0.05 -0.03 0.05 0.01 0.27 0.14 -0.03 0.04 0.00 0.02 0.03 0.22* -0.01 -0.04 0.03 0.03 0.07* 0.52* 0.69* 1.07 1.26 0.69* 1.05 1.24* 1.53* 1.07 1.72* 0.95 0.95 0.97 1.06 1.01 1.31 1.15 0.97 1.04 1.00 1.02 1.03 1.25* 0.99 0.96 1.03 1.03 1.07* -0.50* -0.11 0.24 0.14 -0.22 -0.06 0.27* -0.40* 0.16 0.64* 0.44* 0.45* -0.22* 0.14 0.03 -0.04 -0.06 -0.08 -0.02 0.02 -0.01 0.02 0.06 0.04 -0.01 0.01 -0.04 -0.01 0.61* 0.90 1.27 1.15 0.81 0.95 1.31* 0.67* 1.18 1.90* 1.55* 1.57* 0.80* 1.15 1.03 0.96 0.94 0.92 0.99 1.02 0.99 1.02 1.06 1.04 0.99 1.01 0.96 1.00 Note. ∗p < 0.05. Unweighted civics sample N = 5,600. Unweighted U.S. history sample N = 7,660. Unweighted geography sample N = 4,660. Relevant reference groups are White, not Hispanic; traditional public schools; suburbs; and the West census region. 97 Table B3. Odds ratios for student and school covariates in instructional exposure model (civics sample) Student Female African American Hispanic Asian/Pac Islander Another race/eth FRPL Books in the home LEP IEP Talk at home School s e r u t c e L s t e e h s k r o W l a i c o S s e i d u t s k o o b t x e t s w e n / s k o o B s e n i z a g a m / y r a m i r P s t n e m u c o d o e d i v / m l i F e v i t a t i t n a u Q a t a d r e t u p m o C s n o i t a l u m i s k c o M e l o r / l a i r t y a l p & s e t a b e D l e n a p s n o i s s u c s i d s s u c s i D t n e r r u c s t n e v e y t i n u m m o C r o t i s i V s r e t t e l e t i r W 0.99 0.88 0.85* 0.86 0.95 1.07 1.08 0.98 1.03 0.98 1.07 0.95 1.04 1.05 0.93 0.97 0.97 1.08 1.08 1.02 1.09* 0.98 1.05 1.01 0.79* 0.98 0.92* 1.07 0.99 0.99 1.04 0.99 0.98 0.89 0.76* 0.98 0.96 0.99 1.07 0.94 0.94 0.98 1.02 0.96 1.02 1.00 1.02 0.85 0.92 0.98 1.00 0.89 0.98 0.87 1.03 0.95 0.99 0.98 1.04 1.00 1.04 0.98 0.96 0.85 0.90 1.03 1.00 1.13 0.99 0.95 1.01 1.04 0.93 1.02 0.62* 1.00 0.99 1.25 1.00 0.91 1.00 0.96 1.02 0.96 1.08 0.98 1.08* 0.90 1.13 1.03 0.99 0.97 0.98 1.07 1.31* 0.99 1.08 0.93 1.01 0.94 0.98 1.04 0.99 1.12 0.96 0.99 1.00 0.94 1.04 1.01 1.01 0.95 0.95 0.94 1.02 0.95 0.95 0.95 0.95 0.94 1.01 0.98 0.97 1.02 1.46* 0.94 0.99 0.87 1.11 1.04 y t i n u m m o C s t c e j o r p 0.99 0.92 0.91 1.08 0.91 1.02 0.99 1.00 1.10 1.07 Northeast Midwest South Female Private Charter City Town Rural 4.52* 0.98 1.02 0.62 2.07 0.48 0.33* 0.67 1.25 1.05 1.04 1.01 0.72 1.01 1.23 1.00 0.76 0.95 Note. ∗p < 0.05. Unweighted N = 6,050. Student characteristics listed in the school section of the table are cluster/school averaged * 10. Relevant reference groups are White, not Hispanic; traditional public schools; suburbs; and the West census region. 0.45* 1.67 1.40 1.07 2.18* 1.05 1.89 0.78 1.11 0.98 0.99 0.95 1.09 0.88 0.90 0.97 0.86 0.92 2.32 0.36 1.22 3.32* 1.02 1.31 1.38 1.29 1.22 1.02 1.06 0.95 1.35 1.00 1.07 1.25 0.90 0.96 0.98 1.25 0.96 0.61 0.96 0.83 0.91 1.38 1.24* 1.01 1.05 0.93 0.71 1.00 1.05 1.37 0.87 1.12 0.78 0.78 0.59 0.92 0.59 1.02 0.22* 1.07 1.01 1.11 0.96 0.81 0.81 1.08 1.24 1.41 1.20 1.17 2.33 0.62 0.52 1.13 0.99 1.53 0.48 1.26 1.09 1.24* 1.05 0.87 1.88* 1.00 1.22* 1.02 0.80 0.99 1.66 0.16* 2.31* 1.41 2.07 2.02 0.94 1.26 1.50* 0.99 0.84* 0.76* 1.09 0.97 0.97 1.00 1.08 0.94 0.56 1.80 0.93 1.35 1.05 1.81 1.57 2.33* 1.05 1.03 1.07 1.07 0.90 1.01 0.97 1.16 1.12 1.02 0.58 0.22* 0.83 0.70 0.71 1.21 0.68 1.04 1.16 0.96 0.94 0.96 1.36 1.06 1.03 1.14 1.01 1.02 0.49 6.73* 1.16 0.99 1.85 0.90 2.08 1.91 1.18 0.97 1.07 1.00 1.00 0.96 1.08 1.08 0.99 0.90 1.28 1.58 1.24 0.71 1.11 0.85 1.14 1.22 1.10 0.98 1.02 0.92 1.18 1.02 1.09 0.77* 0.99 1.03 0.82 0.24* 1.11 0.59 1.18 0.96 1.05 0.68 1.02 1.00 1.01 0.97 1.20 0.97 1.03 1.06 1.07 0.99 2.46 1.09 1.36 0.94 1.70 1.37 3.19* 1.05 1.19 0.98 0.96 0.96 1.43 1.12 1.01 1.26 1.01 1.02 0.56 3.77 0.71 0.59 0.59 1.10 0.93 1.31 0.92 1.05 0.93 0.99 1.27 0.96 1.04 0.94 0.90 1.08 Talk at home African American Hispanic Asian/Pac Islander Another race/eth Books in the home FRPL LEP IEP 98 Table B4. Odds ratios for student and school covariates in instructional exposure model (U.S. history sample) Student Female African American Hispanic Asian/Pac Islander Another race/eth FRPL Books in the home LEP IEP Talk at home School s e r u t c e L s t e e h s k r o W l a i c o S s e i d u t s k o o b t x e t s w e n / s k o o B s e n i z a g a m / y r a m i r P s t n e m u c o d o e d i v / m l i F e v i t a t i t n a u Q a t a d r e t u p m o C s n o i t a l u m i s k c o M e l o r / l a i r t y a l p & s e t a b e D l e n a p s n o i s s u c s i d s s u c s i D t n e r r u c s t n e v e y t i n u m m o C r o t i s i V s r e t t e l e t i r W 0.94 0.96 0.92 0.94 0.85 1.03 0.99 0.94 1.04 0.99 0.96 0.91 0.93 0.92 1.00 1.00 0.95 1.02 1.21* 0.99 0.98 0.99 1.03 1.00 1.10 1.06 0.92* 1.04 1.05 1.00 0.95 1.09 1.08 0.96 1.13 0.97 0.99 0.98 1.08 0.95 0.98 1.06 1.11 1.14 0.96 1.00 1.06 0.79* 0.99 1.03 0.98 1.09 1.01 0.93 0.82* 1.01 1.03 1.05 1.08 0.96 0.95 1.00 0.97 1.10 0.99 0.97 1.03 1.01 0.93 1.04 1.00 1.09 1.10 1.14 1.12 0.93 1.08 1.03 1.03 0.96 0.94 1.03 1.01 1.04 0.89 0.98 1.06 0.87 1.00 1.04 1.01 1.02 1.00 0.95 1.04 0.96 1.01 0.97 0.95 0.97 0.97 1.11 0.97 1.03 0.98 0.95 1.02 0.84 0.99 0.94 1.08 1.14 1.20* 1.12 1.24 1.07 1.08 1.09 0.96 0.94 1.06 0.92 0.97 0.95 1.03 1.03 0.95 0.85 0.97 1.04 y t i n u m m o C s t c e j o r p 0.97 1.18 1.04 1.10 1.21 1.06 0.94 0.97 1.11 1.00 Northeast Midwest South Female Private Charter City Town Rural 4.62* 1.83 1.22 0.31 1.30 0.77 0.86 0.71 1.46* 0.76* 0.94 1.04 0.74 1.10 1.02 1.80* 0.71* 0.89 Note. ∗p < 0.05. Unweighted N = 7,470. Student characteristics listed in the school section of the table are cluster/school averaged * 10. Relevant reference groups are White, not Hispanic; traditional public schools; suburbs; and the West census region. 2.38 0.44 1.44 2.30 1.07 1.48 1.59 1.27 1.18 0.98 1.05 0.96 1.21 1.01 0.96 1.32 0.92 1.28* 1.03 1.96 1.26 0.99 1.53 0.61 1.16 1.04 1.14 1.00 0.98 0.82* 0.60 0.92 0.88 1.02 1.15 1.15 0.53 0.28* 0.95 0.91 0.91 1.31 0.74 1.17 1.01 0.94 0.95 0.91 0.91 1.09 1.10 1.13 1.11 1.11 0.73 2.54 0.85 0.68 0.50* 0.96 0.79 1.04 1.14 1.01 0.94 0.95 0.72 0.93 1.02 0.98 1.00 1.03 2.04 0.64 0.49* 1.19 0.75 1.78 0.70 1.81 1.12 1.11 1.05 1.08 1.24 1.00 1.07 1.28 0.96 0.90 0.93 1.21 0.79 0.71 0.63 0.57 0.22* 0.67 0.71* 1.12 0.96 0.77 0.73 1.10 1.23 1.95* 0.98 1.08 0.52 2.22 1.58 1.25 1.63 1.67 2.43* 0.83 0.89 0.97 0.91 1.00 1.01 0.88* 0.82 0.90 1.37* 0.92 1.24 0.23* 1.76 1.44 1.59 2.36 1.17 1.60 1.12 1.04 0.90 0.85 1.17 1.01 1.02 0.94 1.25 0.97 1.02 1.59 1.33 0.63 1.20 0.79 1.16 0.98 1.02 0.97 1.05 0.87 1.63* 0.94 0.94 0.84 1.13 1.05 3.02* 1.16 1.44 0.71 1.66 2.21 3.41* 1.21 0.91 0.98 0.87 0.94 1.44 1.07 0.93 0.89 1.35 1.06 0.80 1.84 0.65 1.28 0.94 1.47 0.97 1.79 1.12 1.02 1.07 1.14 0.88 1.16 1.14 1.17 0.86 1.07 0.61 0.88 0.73 0.66 0.90 1.01 0.91 1.40 1.20 1.09 1.14 0.86 0.73 0.94 1.13 1.18 0.92 1.16 0.79 0.34 1.04 0.46 1.07 1.43 1.52 0.92 0.91 1.03 1.06 0.95 1.02 0.96 1.06 1.24 1.06 1.09 Talk at home African American Hispanic Asian/Pac Islander Another race/eth Books in the home FRPL LEP IEP 99 Table B5. Odds ratios for student and school covariates in instructional exposure model (geography sample) Student Female African American Hispanic Asian/Pac Islander Another race/eth FRPL Books in the home LEP IEP Talk at home School s e r u t c e L 0.88* 0.93 1.02 1.00 0.94 1.03 1.01 1.06 0.97 1.05 s t e e h s k r o W 1.05 1.10 1.04 1.01 0.91 0.96 1.02 0.96 1.12 1.00 l a i c o S s e i d u t s k o o b t x e t 0.98 0.99 0.93 0.81* 0.84 1.04 0.95 0.96 1.01 1.04 s w e n / s k o o B s e n i z a g a m / 0.93 0.93 1.01 0.81* 0.74 1.02 0.98 0.98 1.04 1.09* y r a m i r P s t n e m u c o d 1.02 0.91 1.11 1.05 1.10 1.02 0.95 0.91 0.88 1.04 o e d i v / m l i F 1.04 1.07 1.02 1.03 1.07 0.96 0.98 0.96 1.02 1.04 a t a d e v i t a t i t n a u Q 1.01 0.96 0.94 1.08 1.12 0.85* 0.96 1.13 0.87 1.07 r e t u p m o C s n o i t a l u m i s 0.97 1.07 1.00 1.07 0.92 0.92 1.07 1.07 1.14 1.00 k c o M e l o r / l a i r t y a l p 0.93* 0.88 0.99 1.01 0.85 1.01 1.00 1.03 0.98 1.04 S I G 1.10 1.27* 1.15 1.07 1.41 1.26* 0.78* 1.48* 1.42* 1.46* & s e t a b e D l e n a p s n o i s s u c s i d s s u c s i D t n e r r u c s t n e v e y t i n u m m o C r o t i s i V 0.98 0.92 1.08 0.98 1.04 0.97 0.93 1.01 0.96 0.97 1.00 0.91 1.05 1.05 0.77 1.00 0.92* 0.68* 1.06 1.01 0.98 0.72* 0.89 0.84 0.95 1.12 0.93 0.81 1.01 1.07 s r e t t e l e t i r W 0.98 0.98 1.07 1.11 0.93 1.01 1.00 0.89 1.13 0.98 y t i n u m m o C s t c e j o r p 0.99 0.90 0.82 0.86 0.86 1.10 1.05 1.09 0.90 1.02 Private Charter City Town Rural Northeast Midwest South Female 4.40* 1.10 1.16 0.38 0.93 0.79 1.06 0.78 1.03 0.87 0.96 1.02 1.53 0.98 0.95 1.44 0.71* 0.73* Note. ∗p < 0.05. Unweighted N = 5,280. Student characteristics listed in the school section of the table are cluster/school averaged * 10. Relevant reference groups are White, not Hispanic; traditional public schools; suburbs; and the West census region. 0.84 1.20 1.86* 0.79 1.32 1.28 1.63 1.48 1.00 0.90 1.06 0.87 0.85 1.02 1.02 0.79 1.17 0.99 0.67 0.33* 1.22 0.56 0.85 0.91 1.65 1.14 1.13 1.01 1.09 0.94 0.54* 0.96 1.13 1.29* 1.08 0.88 0.73 1.87 1.48 1.04 1.77 1.70 1.54 0.75 0.85 1.01 0.92 1.02 0.74 0.92 0.96 0.75 1.10 0.83* 0.84 0.97 1.05 1.24 1.06 1.12 1.14 0.85 1.04 1.01 1.03 0.91* 0.97 0.98 1.02 0.96 0.99 0.97 1.25 0.44 0.54 1.52 0.82 2.25 0.65 2.87* 1.19* 1.17* 1.12 0.96 1.49 0.96 1.13 1.21 0.99 0.91 2.35 0.26* 2.45* 2.78* 2.21* 4.04* 1.40 1.85 0.96 1.05 0.95 0.89 0.85 1.13 0.99 1.13 1.22* 1.01 4.20* 1.59 1.37 0.76 1.64 1.29 2.69* 1.07 0.92 1.02 0.86 1.08 0.57 1.10 0.98 0.85 1.22 0.83 0.46 2.12 0.66 0.65 0.54 1.33 1.40 1.33 1.04 1.03 1.04 1.34* 1.62 0.89 0.96 1.09 1.19 0.99 0.76 0.49 0.98 1.01 0.82 1.96 1.09 1.43 1.07 0.91 1.02 1.13 1.52 1.08 0.93 1.42* 1.10 0.96 0.55 1.59 0.68 1.18 0.92 1.48 1.20 1.79 1.13 1.06 1.08 0.97 1.55 0.97 1.06 1.22* 0.96 0.84* 0.59 1.57 0.74 0.89 0.92 0.77 0.81 1.41 1.12 1.04 1.09 0.86 0.86 0.89 0.95 1.34 0.98 1.06 0.68 1.18 0.80 0.84 0.55 1.00 0.43 1.23 1.02 1.15 0.92 0.83 0.74 1.01 1.12 1.12 1.15 1.22 0.89 2.36 1.24 1.51 2.54 0.94 1.00 1.57 1.06 1.09 1.01 0.98 0.67 1.01 1.13 0.95 1.24 0.88 1.40 0.29 1.45 2.29 1.02 1.10 1.29 1.10 0.99 1.12 1.07 0.91 1.09 1.00 1.05 1.31 1.10 0.97 African American Hispanic Asian/Pac Islander Another race/eth Books in the home FRPL LEP IEP Talk at home 100 REFERENCES 101 REFERENCES Anderman, E. M. (2012). Adolescence. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook, vol 3: Application to learning and teaching. (pp. 43- 61). Washington D.C.: American Psychological Association. Andrews, G. P., & Warren, W. J. (2018). How the politicization of history education led to Michigan’s fall. Phi Delta Kappan, 99(8), 19-24. doi:10.1177/0031721718775673 Atwell, M. N., Bridgeland, J., & Levine, P. (2017). Civic deserts: America’s civic health challenge. Retrieved from https://www.ncoc.org/wp- content/uploads/2017/10/2017CHIUpdate-FINAL-small.pdf Banks, J. A. (Ed.) (2017). Citizenship education and global migration: Implications for theory, research, and teaching. Washington D.C.: American Educational Research Association. Barton, K. C., & Avery, P. G. (2016). Research on social studies education: Diverse students, settings, and methods. In D. H. Gitomer & C. A. Bell (Eds.), Handbook of research on teaching (5th ed., pp. 985-1038). Washington, D.C.: American Educational Research Association. Bednarz, S. W., Heffron, S., & Huynh, N. T. (2013). A road map for 21st century geography education: Geography education research. Retrieved from http://media.nationalgeographic.org/assets/file/NGS_RoadMap_GERC_6-21.pdf Bennion, E. A., & Laughlin, X. E. (2018). Best practices in civic education: Lessons from the Journal of Political Science Education. Journal of Political Science Education, 14(3), 287-330. doi:10.1080/15512169.2017.1399798 Berliner, D. C. (2001). Learning about and learning from expert teachers. International Journal of Educational Research, 35(5), 463-482. doi:10.1016/S0883-0355(02)00004-6 Brophy, J. E. (1986). Teacher influences on student achievement. American Psychologist, 41(10), 1069-1077. doi:10.1037/0003-066X.41.10.1069 Brophy, J. E., & Good, T. L. (1986). Teacher behavior and student achievement. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 328-375). New York, NY: Macmillan. Brown, B. D. (2018). America is no longer the standard bearer for a free press. Retrieved from http://time.com/5133507/press-freedom-united-states/ Camburn, E. M., & Han, S. W. (2011). Two decades of generalizable evidence on U.S. Instruction from national surveys. Teachers College Record, 113(3), 561-610. 102 Campbell, D. E. (2008). Voice in the classroom: How an open classroom climate fosters political engagement among adolescents. Political Behavior, 30(4), 437-454. doi:10.1007/s11109- 008-9063-z Campbell, D. E. (2012). Civic education in traditional public, charter, and private schools: Moving from comparison to explanation. In D. E. Campbell, M. Levinson, & F. M. Hess (Eds.), Making civics count: Citizenship education for a new generation. Cambridge, MA: Harvard Education Press. Carlson, D. (2012). Out of the classroom and into the voting booth? Analyzing the effects of education on political participation. ProQuest Digital Dissertations, Retrieved from https://search.proquest.com/openview/e1c219e028b0afe0b5e44a03933a3e1e/1.pdf?pq- origsite=gscholar&cbl=18750&diss=y Carretero, M., Haste, H., & Bermudez, A. (2015). Civic education. In L. Corno & E. M. Anderman (Eds.), Handbook of educational psychology (3rd ed., pp. 295-308). New York, NY: Routledge. Center for Civic Education. (1994). National standards for civics and government. Calabasas, CA: Center for Civic Education. College Board. (2017). Trends in college pricing 2017. Retrieved from https://trends.collegeboard.org/sites/default/files/2017-trends-in-college-pricing_0.pdf Conklin, H. G. (2008). Promise and problems in two divergent pathways: Preparing social studies teachers for the middle school level. Theory & Research in Social Education, 36(1), 36-65. doi:10.1080/00933104.2008.10473359 Conklin, H. G. (2009). Purposes, practices, and sites: A comparative case of two pathways into middle school teaching. American Educational Research Journal, 46(2), 463-500. doi:103102/0002831208326558 Conklin, H. G. (2012). Company men: Tracing learning from divergent teacher education pathways into practice in middle grades classrooms. Journal of Teacher Education, 63(3), 171-184. doi:10.1177/0022487111426294 Conklin, H. G., Hawley, T. S., Powell, D., & Ritter, J. K. (2010). Learning from young adolescents: The use of structured teacher education coursework to help beginning teachers investigate middle school students’ intellectual capabilities. Journal of Teacher Education, 61(4), 313-327. doi:10.1177/0022487110371378 Cowan, J., & Goldhaber, D. (2016). National board certification and teacher effectiveness: Evidence from Washington state. Journal of Research on Educational Effectiveness, 9(3), 233-258. doi:10.1080/19345747.2015.1099768 Crocco, M. S., & Livingston, E. (2017). Becoming an “expert” social studies teacher: What we know about teacher education and professional development. In M. M. Manfra & C. M. 103 Bolick (Eds.), The Wiley handbook of social studies research (pp. 360-384). Malden, MA: Wiley-Blackwell. de Blij, H. (2012). Why geography matters: More than ever (2nd ed.). New York, NY: Oxford University Press. Dee, T. S. (2004). Are there civic returns to education? Journal of Public Economics, 88(9-10), 1697-1720. doi:10.1016/j.jpubeco.2003.11.002 Dee, T. S., & Cohodes, S. R. (2008). Out-of-field teachers and student achievement. Public Finance Review, 36(1), 7-32. doi:10.1177/1091142106289330 Dee, T. S., Jacob, B. A., & Schwartz, N. L. (2013). The effects of NCLB on school resources and practices. Educational Evaluation and Policy Analysis, 35(2), 252-279. doi:10.3102/0162373712467080 Del Favero, L., Boscolo, P., Vidotto, G., & Vicentini, M. (2007). Classroom discussion and individual problem-solving in the teaching of history: Do different instructional approaches affect interest in different ways? Learning and Instruction, 17(6), 635-657. doi:10.1016/j.learninstruc.2007.09.012 Delli Carpini, M. X., & Keeter, S. (1996). What Americans know about politics and why it matters. New Haven, CT: Yale University Press. Eccles, J. S., & Roeser, R. W. (2011). Schools as developmental contexts during adolescence. Journal of Research on Adolescence, 21(1), 225 – 241. doi:10.1111/j.1532- 7795.2010.00725.x Education Commission of the States. (2016). Civic education policies: State assessments include civics, citizenship education or social studies. Retrieved from http://ecs.force.com/mbdata/MBQuest2RTANW?Rep=CIP1604S Evans, R. W. (2004). The social studies wars: What should we teach the children? New York, NY: Teachers College Press. Federal Reserve Bank of New York. (2018). Quarterly report on household debt and credit (Q4 2017). Retrieved from https://www.newyorkfed.org/medialibrary/interactives/householdcredit/data/pdf/HHDC_ 2017Q4.pdf Feldman, L., Pasek, J., Romer, D., & Jamieson, K. H. (2007). Identifying best practices in civic education: Lessons from the Student Voices program. American Journal of Education, 114(1), 75-100. doi:10.1086/520692 Finn Jr., C. E. (2003a). Foreward. In J. Leming, L. Ellington, & K. Porter (Eds.), Where did social studies go wrong? (pp. I-VII). Washington D.C.: Thomas B. Fordham Foundation. 104 Finn Jr., C. E. (2003b). The social studies wars. Retrieved from https://www.hoover.org/research/social-studies-wars Fitchett, P. G., & Heafner, T. L. (2010). A national perspective on the effects of high-stakes testing and standardization on elementary social studies marginalization. Theory & Research in Social Education, 38(1), 114-130. doi:10.1080/00933104.2010.10473418 Fitchett, P. G., & Heafner, T. L. (2017). Student demographics and teacher characteristics as predictors of elementary-age students' history knowledge: Implications for teacher education and practice. Teaching and Teacher Education, 67, 79-92. doi:10.1016/j.tate.2017.05.012 Fitchett, P. G., Heafner, T. L., & Lambert, R. (2014). Examining elementary social studies marginalization: A multilevel model. Educational Policy, 28(1), 40-68. doi:10.1177/0895904812453998 Galston, W. A. (2001). Political knowledge, political engagement, and civic education. Annual Review of Political Science, 4(1), 217-234. doi:10.1146/annurev.polisci.4.1.217 Gamoran, A., & Nystrand, M. (1991). Background and instructional effects on achievement in eighth-grade English and social studies. Journal of Research on Adolescence, 1(3), 277- 300. doi:10.1207/s15327795jra0103_5 Gehlbach, H., Brown, S. W., Ioannou, A., Boyer, M. A., Hudson, N., Niv-Solomon, A., . . . Janik, L. (2008). Increasing interest in social studies: Social perspective taking and self- efficacy in stimulating simulations. Contemporary Educational Psychology, 33(4), 894- 914. doi:10.1016/j.cedpsych.2007.11.002 Goldhaber, D. (2015). Teachers clearly matter, but finding effective teacher policies has proven challenging. In H. F. Ladd & M. E. Goertz (Eds.), Handbook of research in education finance and policy (2nd ed., pp. 157-173). New York, NY: Routledge. Goldhaber, D., & Brewer, D. J. (1996). Evaluating the effect of teacher degree level on educational performance. Retrieved from https://nces.ed.gov/pubs97/97535l.pdf Gootman, E. (2007). Trying to find solutions in chaotic middle schools. The New York Times. Retrieved from https://www.nytimes.com/2007/01/03/education/03middle.html Grant, S. G., & Salinas, C. (2008). Assessment and accountability in the social studies. In L. S. Levstik & C. A. Tyson (Eds.), Handbook of research in social studies education (pp. 219-236). New York, NY: Routledge. Gudmundsdottir, S., & Shulman, L. (1987). Pedagogical content knowledge in social studies. Scandinavian Journal of Educational Research, 31(2), 59-70. doi:10.1080/0031383870310201 Hahn, C. L., Bernard-Powers, J., Crocco, M. S., & Woyshner, C. (2007). Gender equity in social studies. In S. S. Klein, B. Richardson, D. A. Grayson, L. H. Fox, C. Kramarae, D. S. 105 Pollard, & C. A. Dwyer (Eds.), Handbook for achieving gender equity through education (pp. 335-357). Mahwah, NJ: Lawrence Erlbaum. Halvorsen, A. (2013). A history of elementary social studies: Romance and reality. New York, NY: Peter Lang Publishing Inc. Heafner, T. L., & Fitchett, P. G. (2015). An opportunity to learn U.S. history: What NAEP data suggest regarding the opportunity gap. The High School Journal, 98(3), 226-249. doi:10.1353/hsj.2015.0006 Henry, G. T., Bastian, K. C., Fortner, K. C., Kershaw, D. C., Purtell, K. M., Thompson, C. L., & Zulli, R. A. (2014). Teacher preparation policies and their effects on student achievement. Education Finance and Policy, 9(3), 1-40. doi:10.1162/EDFP_a_00134 Henry, G. T., Purtell, K. M., Bastian, K. C., Fortner, K. C., Thompson, C. L., Campbell, S. L., & Patterson, K. M. (2014). The effects of teacher entry portals on student achievement. Journal of Teacher Education, 65(1), 7-23. doi:10.1177/0022487113503871 Hess, D. E., & McAvoy, P. (2014). The political classroom: Evidence and ethics in democratic education. New York, NY: Routledge. Hetherington, M. J. (2005). Why trust matters: Declining political trust and the demise of American liberalism. Princeton, NJ: Princeton University Press. Hidi, S. (2006). Interest: A unique motivational variable. Educational Research Review, 1(2), 69- 82. doi:10.1016/j.edurev.2006.09.001 Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111-127. doi:10.1207/s15326985ep4102_4 Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers' mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42(2), 371- 406. doi:10.3102/00028312042002371 Howell, P. B., Faulkner, S. A., Cook, C. M., Miller, N. C., & Thompson, N. L. (2016). Specialized preparation for middle level teachers: A national review of teacher preparation programs. RMLE Online, 39(1), 1-12. doi:10.1080/19404476.2015.1115322 Kahne, J. E., Crow, D., & Lee, N. J. (2013). Different pedagogy, different politics: High school learning opportunities and youth political engagement. Political psychology, 34(3), 419- 441. doi:10.1111/j.1467-9221.2012.00936.x Kahne, J. E., & Middaugh, E. (2008). Democracy for some: The civic opportunity gap in high school (CIRCLE Working Paper 59). Retrieved from http://eric.ed.gov/?id=ED503646 Kahne, J. E., & Sporte, S. E. (2008). Developing citizens: The impact of civic learning opportunities on students’ commitment to civic participation. American Educational Research Journal, 45(3), 738-766. doi:10.3102/0002831208316951 106 King, M. B., Newmann, F. M., & Carmichael, D. L. (2015). Authentic intellectual work: Common standards for teaching social studies. In W. C. Parker (Ed.), Social studies today: Research and practice (pp. 53-64). New York, NY: Routledge. Knowles, R. T. (2018). Teaching who you are: Connecting teachers’ civic education ideology to instructional strategies. Theory & Research in Social Education, 46(1), 68-109. doi:10.1080/00933104.2017.1356776 Levine, P. (2012). Education for a civil society. In D. E. Campbell, M. Levinson, & F. M. Hess (Eds.), Making civics count: Citizenship education for a new generation. Cambridge, MA: Harvard Education Press. Levinson, M. (2012). No citizen left behind. Cambridge, MA: Harvard University Press. Levstik, L. S. (2008). What happens in social studies classrooms? Research on K-12 social studies practice. In L. S. Levstik & C. A. Tyson (Eds.), Handbook of research in social studies education (pp. 50-64). New York, NY: Routledge. Lupia, A. (2016). Uninformed: Why people seem to know so little about politics and what we can do about it. New York, NY: Oxford University Press. Marks, H. M. (2000). Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American Educational Research Journal, 37(1), 153-184. doi:10.3102/00028312037001153 Mislevy, R. J., Johnson, E. G., & Muraki, E. (1992). Scaling procedures in NAEP. Journal of Educational Statistics, 17(2), 131-154. doi:10.2307/1165166 Mohammed, A. (2015). Deepening income inequality. Retrieved from http://reports.weforum.org/outlook-global-agenda-2015/top-10-trends-of-2015/1- deepening-income-inequality/?doing_wp_cron=1524844373.3990080356597900390625 Monte-Sano, C. (2011). Beyond reading comprehension and summary: Learning to read and write in history by focusing on evidence, perspective, and interpretation. Curriculum Inquiry, 41(2), 212-249. doi:10.1111/j.1467-873X.2011.00547.x Monte-Sano, C., & Budano, C. (2013). Developing and enacting pedagogical content knowledge for teaching history: An exploration of two novice teachers' growth over three years. Journal of the Learning Sciences, 22(2), 171-211. doi:10.1080/10508406.2012.742016 Monte-Sano, C., & De La Paz, S. (2012). Using writing tasks to elicit adolescents’ historical reasoning. Journal of Literacy Research, 44(3), 273-299. doi:10.1177/1086296x12450445 Monte-Sano, C., & Reisman, A. (2015). Studying historical understanding. In L. Corno & E. M. Anderman (Eds.), Handbook of educational psychology (3rd ed., pp. 281-294). New York, NY: Routledge. 107 Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (8th ed.). Los Angeles, CA: Muthén & Muthén. National Center for Education Statistics. (2011). Restricted-use data procedures manual. Retrieved from https://nces.ed.gov/pubs96/96860rev.pdf National Center for Education Statistics. (2018). NAEP technical documentation. Retrieved from https://nces.ed.gov/nationsreportcard/tdw/sitemap.aspx Neundorf, A., Niemi, R. G., & Smets, K. (2016). The compensation effect of civic education on political engagement: How civics classes make up for missing parental socialization. Political Behavior, 38(4), 921-949. doi:10.1007/s11109-016-9341-0 Neundorf, A., Smets, K., & García-Albacete, G. M. (2013). Homemade citizens: The development of political interest during adolescence and young adulthood. Acta Politica, 48(1), 92-116. doi:10.1057/ap.2012.23 Niemi, R. G. (2012). What students know about civics and government. In D. E. Campbell, M. Levinson, & F. M. Hess (Eds.), Making civics count: Citizenship education for a new generation. Cambridge, MA. Niemi, R. G., & Junn, J. (1998). Civic education: What makes students learn. New Haven, CT: Yale University Press. NOAA National Centers for Environmental Information. (2018). U.S. billion-dollar weather and climate disasters. Retrieved from https://www.ncdc.noaa.gov/billions/ Pace, J. L. (2011). The complex and unequal impact of high stakes accountability on untested social studies. Theory & Research in Social Education, 39(1), 32-60. doi:10.1080/00933104.2011.10473446 Parker, W. C., Lo, J., Yeo, A. J., Valencia, S. W., Nguyen, D., Abbott, R. D., . . . Vye, N. J. (2013). Beyond breadth-speed-test: Toward deeper knowing and engagement in an advanced placement course. American Educational Research Journal, 50(6), 1424-1459. doi:10.3102/0002831213504237 Passe, J., & Fitchett, P. G. (Eds.) (2013). The status of social studies: Views from the field. Charlotte, NC: Information Age Publishing. Pew Research Center. (2014). Political polarization in the American public. Retrieved from http://www.people-press.org/2014/06/12/political-polarization-in-the-american-public/ Pew Research Center. (2017a). The partisan divide on political values grows even wider. Retrieved from http://www.people-press.org/2017/10/05/the-partisan-divide-on-political- values-grows-even-wider/ 108 Pew Research Center. (2017b). U.S. trails most developed countries in voter turnout. Retrieved from http://www.pewresearch.org/fact-tank/2017/05/15/u-s-voter-turnout-trails-most- developed-countries/ Polikoff, M. S., & Porter, A. C. (2014). Instructional alignment as a measure of teaching quality. Educational Evaluation and Policy Analysis, 36(4), 399-416. doi:10.3102/0162373714531851 Powell, D. (2018). Brother, can you paradigm? Toward a theory of pedagogical content knowledge in social studies. Journal of Teacher Education, 69(3), 252-262. doi:10.1177/0022487117708553 Putnam, R. D. (2000). Bowling alone: The collapse and revival of American community. New York, NY: Simon and Schuster. Putnam, R. D. (2016). Our kids: The American dream in crisis. New York, NY: Simon & Schuster. Putnam, R. D., & Feldstein, L. M. (2004). Better together: Restoring the American community. New York, NY: Simon & Schuster. Reardon, S. F., Kalogrides, D., Fahle, E. M., Podolsky, A., & Zárate, R. C. (2018). The relationship between test item format and gender achievement gaps on math and ELA tests in fourth and eighth grades. Educational Researcher. Advance online publication. doi:10.3102/0013189x18762105 Renninger, K. A., & Hidi, S. (2016). The power of interest for motivation and engagement. New York, NY: Routledge. Ronfeldt, M., & Campbell, S. L. (2016). Evaluating teacher preparation using graduates observational ratings. Educational Evaluation and Policy Analysis, 38(4), 603-625. doi:10.3102/0162373716649690 Rubin, B. C. (2007). “There’s still not justice”: Youth civic identity development amid distinct school and community contexts. Teachers College Record, 109(2), 449-481. Rubin, B. C. (2012). Making citizens: Transforming civic learning for diverse social studies classrooms. New York, NY: Routledge. Schafer, J. L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research, 8(1), 3-15. doi:10.1177/096228029900800102 Schmidt, W. H., Burroughs, N. A., Zoido, P., & Houang, R. T. (2015). The role of schooling in perpetuating educational inequality: An international perspective. Educational Researcher, 44(7), 371-386. doi:10.3102/0013189x15603982 109 Schmidt, W. H., & Maier, A. (2009). Opportunity to learn. In G. Sykes, B. Schneider, & D. N. Plank (Eds.), Handbook of education policy research (pp. 541-559). New York, NY: Routledge. Schulz, W., Ainley, J., Fraillon, J., Kerr, D., & Losito, B. (2010). ICCS 2009 international report: Civic knowledge, attitudes, and engagement among lower-secondary school students in 38 countries. Amsterdam, Netherlands: International Association for the Evaluation of Educational Achievement. Segall, A., & Helfenbein, R. J. (2008). Research on K–12 geography education. In L. S. Levstik & C. A. Tyson (Eds.), Handbook of research in social studies education (pp. 259-283). New York, NY: Routledge. Shin, E. (2006). Using geographic information system (GIS) to improve fourth graders' geographic content knowledge and map skills. Journal of Geography, 105(3), 109-120. doi:10.1080/00221340608978672 Shin, E. (2007). Using geographic information system (GIS) technology to enhance elementary students' geographic understanding. Theory & Research in Social Education, 35(2), 231- 255. doi:10.1080/00933104.2007.10473334 Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1-13. doi:10.1080/00461520.2014.1002924 Sinatra, G. M., Mukhopadhyay, A., Allbright, T. N., Marsh, J. A., & Polikoff, M. S. (2017). Speedometry: A vehicle for promoting interest and engagement through integrated STEM instruction. The Journal of Educational Research, 110(3), 308-316. doi:10.1080/00220671.2016.1273178 Smith, J. B., & Niemi, R. G. (2001). Learning history in school: The impact of course work and instructional practices on achievement. Theory & Research in Social Education, 29(1), 18-42. doi:10.1080/00933104.2001.10505928 Smith, M. D. (2017). Cognitive validity: Can multiple-choice items tap historical thinking processes? American Educational Research Journal, 54(6), 1256-1287. doi:10.3102/0002831217717949 Stevenson, R. B. (1990). Engagement and cognitive challenge in thoughtful social studies classes: A study of student perspectives. Journal of Curriculum Studies, 22(4), 329-341. doi:10.1080/0022027900220403 Tatto, M. T., Savage, C., Liao, W., Marshall, S. L., Goldblatt, P., & Contreras, L. M. (2016). The emergence of high-stakes accountability policies in teacher preparation: An examination of the U.S. Department of Education’s proposed regulations. Education Policy Analysis Archives, 24(21), 1-57. doi:10.14507/epaa.24.2322 110 The National Assessment Governing Board. (2014a). Civics framework for the 2014 National Assessment of Educational Progress. Retrieved from https://www.nagb.gov/content/nagb/assets/documents/publications/frameworks/civics/20 14-civics-framework.pdf The National Assessment Governing Board. (2014b). Geography framework for the 2014 National Assessment of Educational Progress. Retrieved from https://www.nagb.gov/content/nagb/assets/documents/publications/frameworks/geograph y/2014-geography-framework.pdf The National Assessment Governing Board. (2014c). U.S. History framework for the 2014 National Assessment of Educational Progress. Retrieved from https://www.nagb.gov/content/nagb/assets/documents/publications/frameworks/history/2 014-history-framework.pdf The Pew Charitable Trusts. (2017). Why are millions of citizens not registered to vote? Retrieved from http://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2017/06/why-are- millions-of-citizens-not-registered-to-vote Thornton, S. J. (2017). A concise historiography of the social studies. In M. M. Manfra & C. M. Bolick (Eds.), The Wiley handbook of social studies research (pp. 9-41). Malden, MA: Wiley-Blackwell. Torney-Purta, J., Lehmann, R., Oswald, H., & Schulz, W. (2001). Citizenship and education in twenty-eight countries: Civic knowledge and engagement at age fourteen. Amsterdam, Netherlands: International Association for the Evaluation of Educational Achievement. Torney-Purta, J., & Richardson, W. K. (2003). Teaching for the meaningful practice of democratic citizenship: Learning from the IEA civic education study in 28 countries. In J. L. Patrick, G. E. Hamot, & R. S. Leming (Eds.), Civic learning in teacher education: International perspectives on education for democracy in the preparation of teachers (pp. 25-44). Bloomington, IN: ERIC/ChESS. Torney-Purta, J., Richardson, W. K., & Barber, C. H. (2005). Teachers’ educational experience and confidence in relation to students’ civic knowledge across countries. International Journal of Citizenship and Teacher Education, 1(1), 32-57. UNHCR. (2015). UNHCR, the environment & climate change. Retrieved from http://www.unhcr.org/en-us/540854f49 VanFossen, P. J. (2005). “Reading and math take so much of the time…”: An overview of social studies instruction in elementary classrooms in Indiana. Theory & Research in Social Education, 33(3), 376-403. doi:10.1080/00933104.2005.10473287 Verba, S., Schlozman, K. L., & Brady, H. E. (1995). Voice and equality: Civic voluntarism in American politics. Cambridge, MA: Harvard University Press. 111 Vogler, K. E., Lintner, T., Lipscomb, G. B., Knopf, H., Heafner, T. L., & Rock, T. C. (2007). Getting off the back burner: Impact of testing elementary social studies as part of a state- mandated accountability program. The Journal of Social Studies Research, 31(2), 20–34. Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146-1151. doi:10.1126/science.aap9559 Wallis, C. (2005). Is middle school bad for kids? Time. Retrieved from http://content.time.com/time/magazine/article/0,9171,1088694,00.html Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73(1), 89-122. doi:10.3102/00346543073001089 Westheimer, J., & Kahne, J. E. (2004). What kind of citizen? The politics of educating for democracy. American Educational Research Journal, 41(2), 237-269. doi:10.3102/00028312041002237 Wilson, S. M., & Wineburg, S. S. (1988). Peering at history through different lenses: The role of disciplinary perspectives in teaching history. Teachers College Record, 89(4), 525-539. World Water Assessment Programme. (2012). The United Nations world water development report 4: Managing water under uncertainty and risk. Retrieved from http://www.unesco.org/new/en/natural-sciences/environment/water/wwap/wwdr/wwdr4- 2012/ Zhang, T., Torney-Purta, J., & Barber, C. H. (2012). Students' conceptual knowledge and process skills in civic education: Identifying cognitive profiles and classroom correlates. Theory & Research in Social Education, 40(1), 1-34. doi:10.1080/00933104.2012.649467 112