THE IMPACT OF HIGH SCHOOL SCIENCE COURSEWORK ON POSTSECONDARY STEM OUTCOMES By Christopher Klager Michigan State University in partial fulfillment of the requirements for the degree of Educational Policy—Doctor of Philosophy A DISSERTATION Submitted to 2020 THE IMPACT OF HIGH SCHOOL SCIENCE COURSEWORK ON POSTSECONDARY ABSTRACT STEM OUTCOMES By Christopher Klager Policymakers have attempted to increase academic preparation and equitable opportunities to learn science by increasing science course graduation requirements in nearly every state. In Michigan, the Michigan Merit Curriculum required the graduating high school class of 2011 to, among other requirements, obtain three science credits including biology and either chemistry or physics. Many studies have shown a relationship between science coursework in high school and postsecondary STEM outcomes like choosing a STEM major or graduating with a STEM degree. However, these studies suffer from self-selection issues meaning that we cannot interpret the estimates as causal. Meanwhile, quasi-experimental studies of course requirement policies have found disappointing effects of science coursework requirements on achievement and graduation outcomes. Using detailed high school transcript data from the years before and after the implementation of the Michigan Merit Curriculum, I estimate the effect of the Michigan Merit Curriculum on science course-taking and characterize the changes in science course pathways. I also investigate how the policy differentially affected schools based on a variety of school characteristics. My second aim is to provide evidence about the effect of taking science coursework on postsecondary STEM outcomes related to field of study and college course-taking. I find that before the Michigan Merit Curriculum was implemented, a relatively small proportion of students, less than 30% of students met all of the science course requirements that the Michigan Merit Curriculum called for. After the policy was put in place, there was no change in the number of total science credits students took, but students took about .08 fewer biology credits, .14 more chemistry credits, and .07 more physics credits in high school. The number of students meeting all Michigan Merit Curriculum science requirements increased by 6 percentage points. Difference-in-differences analyses indicate that the policy affected the schools where the fewest students met the Michigan Merit Curriculum science requirements before the policy. I also show that after the Michigan Merit Curriculum students were 1 percentage point more likely to major in STEM, including .5 percentage points more likely to major in biology and .7 percentage points more likely to major in engineering. Students were also slightly more likely to graduate with degrees in the physical sciences. However, these differences in postsecondary outcomes were largely driven by the most advantaged students. In many cases, students who have traditionally been well-served in STEM subjects increased the rate of majoring and graduating with STEM degrees, but for groups that have not—females, racial/ethnic minority students, and students with the lowest achievement—the rates stayed largely the same. Overall, results suggest that although there were changes in course-taking patterns for students that were not meeting the requirements prior to the Michigan Merit Curriculum, the long-term gains were most experienced by students who were already well- served in science. Instrumental variable analyses seeking to understand the causal relationship between chemistry and physics course-taking in high school and postsecondary outcomes showed inconsistent results, although there was evidence that being induced to take chemistry or physics in high school was beneficial for encouraging students to take physical science courses early in college and improved grades in some college STEM disciplines. This dissertation would not have been possible without the support of the Michigan ACKNOWLEDGMENTS Education Research Institute, who provided access to data, and the American Educational Research Association, who provided funding for my studies through the AERA Grants Program. Thank you to both organizations for their support. This research result used data collected and maintained by the Michigan Education Research Institute (MERI). Results, information and opinions solely represent the analysis, information and opinions of the author(s) and are not endorsed by – or reflect the views or positions of – grantors, MERI partners or any employee thereof. This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation under NSF award NSF-DRL #1749275. Opinions reflect those of the author and do not necessarily reflect those AERA or NSF. Thank you to my dissertation committee members who all contributed to my development as a scholar. Christina Schwarz always pushed me to think about issues of equity in science education and was incredibly supportive as I searched for research topics. Josh Cowen taught me about how to do quantitative education policy research, from formulating research questions to performing robustness checks, all while thinking about the implications of my work. Joe Krajcik made a special effort to include me in all parts of the CESE research project, and I learned a great deal about science curriculum development from him. Bob Floden, my academic advisor and dissertation co-chair, helped me navigate graduate school and provided guidance about coursework, research, and hunting for a job. Barbara Schneider, my dissertation co-chair, always treated me as her intellectual equal, even when I didn’t think of myself as one. Barbara iv provided me with invaluable mentorship and extraordinary opportunities for professional development, far beyond any obligation, and for that I am eternally grateful. Thank you to my cohort in the education policy program and the other students in the college of education who I met over the years. You were all great classmates, colleagues, and friends. Soobin Kim deserves special thanks for helping me understand my dissertation data and for helping me think through Michigan’s science coursework policy. Finally, I could not have done any of this without the love and support of Amy, who started as a classmate and turned into my companion for life. v TABLE OF CONTENTS LIST OF TABLES ....................................................................................................................... viii LIST OF FIGURES ....................................................................................................................... xi CHAPTER 1: INTRODUCTION ....................................................................................................1 High School Course Graduation Requirement Policy .........................................................1 The Michigan Merit Curriculum ..........................................................................................4 Benefits of High School Science Course-taking ..................................................................6 Pursuit of a Postsecondary Degree ..........................................................................7 Persistence in College ..............................................................................................7 Labor Market Outcomes ..........................................................................................8 Interest in STEM ......................................................................................................8 Pursuing a STEM Major ..........................................................................................9 College STEM Grades ...........................................................................................10 Graduating with a STEM Degree ..........................................................................11 Quasi-experimental Evidence on Course-taking ...............................................................11 Conceptual Framework ......................................................................................................13 Research Questions ............................................................................................................15 CHAPTER 2: METHODOLOGY .................................................................................................17 Data ....................................................................................................................................17 Coding of Course Catalogs and SCED ..................................................................18 Variables ................................................................................................................19 Research Question 1 – Science Course Sequences ............................................................22 Descriptive Analysis of Science Course-taking Patterns .......................................22 Research Question 2 - Effects of MMC Policy .................................................................23 Intent-to-treat Estimates .........................................................................................24 Treatment-on-treated Estimates .............................................................................25 Research Question 3 - Heterogeneity by School Characteristics .......................................27 Interaction of Course-taking Policy and School Characteristics ...........................27 Research Question 4 – Relationship Between Course-taking and STEM Outcomes ........28 Instrumental Variable Estimation ..........................................................................28 Description of Analytic Sample .........................................................................................32 CHAPTER 3: DESCRIPTION OF HIGH SCHOOL SCIENCE COURSE-TAKING PRE- AND POST-MMC ...................................................................................................................................34 Description of Science Course-taking Pre- and Post-MMC ..............................................34 Discussion ..........................................................................................................................46 CHAPTER 4: EFFECTS OF THE MMC POLICY ON HIGH SCHOOL COURSE-TAKING AND POSTSECONDARY STEM OUTCOMES .........................................................................49 Number of Science Credits Taken and Passed – Pre-post Regression Analysis ...............49 Number of Science Credits Taken and Passed – Difference-in-differences Analysis .......57 vi CHAPTER 5: HETEROGENEITY OF EFFECTS BY INDIVIDUAL AND SCHOOL CHARACTERISTICS ...................................................................................................................84 Student-level Heterogeneity in High School Course-taking ..............................................84 Student-level Heterogeneity in College Major and Graduation ........................................92 Heterogeneity in Science Credits by School Characteristics ...........................................103 Heterogeneity in STEM Major Choices and STEM Degrees by School Characteristics ..................................................................................................................109 CHAPTER 6: THE RELATIONSHIP BETWEEN HIGH SCHOOL SCIENCE COURSE- TAKING AND EARLY POSTSECONDARY STEM OUTCOMES ........................................118 Instrumental Variable Estimation of the Effect of High School Science Course-taking on STEM Major Selection and STEM Degree Attainment ..................................................118 Instrumental Variable Estimation of the Effect of High School Science Course-taking on Taking and Performance in College Science Courses .....................................................120 Discussion ........................................................................................................................125 Postsecondary STEM Outcomes – Pre-post Regression Analysis ....................................71 Postsecondary STEM Outcomes – Difference-in-differences Analysis ............................76 Discussion ..........................................................................................................................81 CHAPTER 7: CONCLUSION AND IMPLICATIONS .............................................................126 Summary of Findings .......................................................................................................126 Limitations .......................................................................................................................128 Understanding the Effects of Taking Science Courses ....................................................131 Understanding How School Characteristics Moderate Effects ........................................133 REFERENCES ............................................................................................................................135 vii LIST OF TABLES Table 1 Classification of courses by SCED code ..........................................................................20 Table 2 Summary statistics of sample characteristics ....................................................................33 Table 3 Science credits taken by grade ..........................................................................................35 Table 4 Science course sequences .................................................................................................37 Table 5 Student level heterogeneity in science course-taking by grade ........................................39 Table 6 School level heterogeneity in science course-taking by grade .........................................44 Table 7 Estimated impact of the MMC on science credits taken and passed ................................50 Table 8 Estimated impact of the MMC on science credits taken and passed, no transition cohorts ............................................................................................................................51 Table 9 Estimated impact of the MMC on biology, chemistry, and physics credits taken and passed .............................................................................................................................................53 Table 10 Estimated impact of the MMC on non-MMC science courses taken and passed ...........55 Table 11 Estimated impact of the MMC on MMC science requirements taken and met ..............57 Table 12 Descriptive statistics by "treatment" status, pre- and post-MMC ...................................59 Table 13 Difference-in-differences estimates of the MMC on science credits taken and passed .............................................................................................................................................61 Table 14 Difference-in-differences estimates of the MMC on biology, chemistry, and physics credits taken and passed .................................................................................................................63 Table 15 Difference-in-differences estimates of the MMC on biology, chemistry, and physics credits taken and passed - Quintile 1 versus Quintile 5 .................................................................65 Table 16 Difference-in-differences estimates of the MMC on MMC requirements taken and met ...........................................................................................................................................68 Table 17 Difference-in-differences estimates of the MMC on science outcomes, no transition cohorts ............................................................................................................................................70 Table 18 Estimated impact of the MMC on postsecondary STEM major selection .....................73 viii Table 19 Estimated impact of the MMC on college graduation and STEM degrees ....................75 Table 20 Difference-in-differences estimates of the MMC on postsecondary STEM major selection .........................................................................................................................................78 Table 21 Difference-in-differences estimates of the MMC on college graduation with STEM degrees ...........................................................................................................................................80 Table 22 Heterogeneous impact of the MMC on high School science coursework by student characteristics .................................................................................................................................86 Table 23 Estimated impact of the MMC on science credits by 8th grade MEAP quintiles ..........91 Table 24 Heterogeneous impact of the MMC on STEM college major selection by student characteristics .................................................................................................................................93 Table 25 Estimated impact of the MMC on postsecondary outcomes by 8th grade MEAP quintiles ..........................................................................................................................................97 Table 26 Heterogeneous impact of the MMC on STEM degree completion by student characteristics .................................................................................................................................99 Table 27 Estimated impact of the MMC on postsecondary outcomes by 8th grade MEAP quintiles ........................................................................................................................................103 Table 28 Heterogeneous impact of the MMC on high school science credits by school characteristics ...............................................................................................................................105 Table 29 Heterogeneous impact of the MMC on science course-taking by high school level course-taking ................................................................................................................................108 Table 30 Heterogeneous impact of the MMC on STEM college major selection by school characteristics ...............................................................................................................................110 Table 31 Heterogeneous Impact of the MMC on STEM College Major Selection .....................112 Table 32 Heterogeneous impact of the MMC on STEM college graduation by high school characteristics ...............................................................................................................................114 Table 33 Heterogeneous impact of the MMC on STEM college graduation by high school level course-taking characteristics ........................................................................................................116 Table 34 Instrumental variable estimates of taking chemistry and physics in high school on college STEM majors and degrees ..............................................................................................119 ix Table 35 Instrumental variable estimates of taking chemistry and physics in high school on college STEM course-taking ........................................................................................................122 Table 36 Instrumental variables estimates of high school chemistry and physics on college STEM grades ...............................................................................................................................124 x LIST OF FIGURES Figure 1 High school science course graduation requirements as of February 2019 ......................2 Figure 2 Conceptual Framework ...................................................................................................14 xi CHAPTER 1: INTRODUCTION Science, technology, engineering, and mathematics (STEM) play an increasingly important role in the world as new discoveries are made, jobs change to align with developments, and issues of extreme importance—like climate change, antibacterial resistance, and global pandemics—change our lives. School science plays a key role in preparing young people to engage in a world where everyone is affected by issues related to STEM. One way that policymakers have attempted to improve science learning and preparation is through the use of increased high school course graduation requirements. This dissertation has two aims related to high school science coursework and subsequent STEM-related outcomes. First, I estimate the effect of Michigan’s policy on science course- taking and characterize the changes in science course pathways. Also, I seek to understand how the policy differentially affected schools based on a variety of school characteristics. My second aim is to provide a causal estimate of the effect of taking science coursework on postsecondary STEM outcomes related to field of study and college course-taking. Together, the answers to these questions contribute to our understanding of what goals we can hope to achieve through course-taking policies, and in what ways we must look toward other policy levers to improve STEM outcomes. High School Course Graduation Requirement Policy Over the last forty years, states have dramatically increased the course requirements for graduation from high school, especially in science and math. In 1982, only one state required three science credits to graduate (Federman, 2007). Now, most states require three science credits; there are only seven states that require two credits, and three states and the District of Columbia require four credits (Education Commission of the States, 2019). In addition, many 1 states require particular courses to graduate such as chemistry or physics. States have also sought to standardize the number of credits required to graduate. In 1982, thirteen states left decisions about science course requirements up to local school districts (Federman, 2007). In 2019, that number is only four (Education Commission of the States, 2019). See Figure 1 for a summary of state science course requirements as of February 2019. A more recent and related trend is state laws requiring schools or school districts to offer advanced courses such as AP, IB, or dual enrollment options. Today, nearly half of the states have some form of policy about access to these advanced courses (Education Commission of the States, 2016). Figure 1 High school science course graduation requirements as of February 2019 The aim of increasing course graduation requirements is to improve college and career readiness through improved academic skills (e.g., National Commission on Excellence in Education, 1983; White House, 2016). Students need to build academic skills so that they can 2 graduate from high school and enter work or postsecondary education with the skills they need to succeed. The focus of these policies on math and science stems from beliefs that students are underprepared in those subjects and are often tied to arguments about global economic competitiveness and the United States’ standing in world rankings on assessments such as PISA and TIMSS (e.g., Desilver, 2017). These policies also seek to provide equitable opportunities to students. There are well- documented gaps in the types of courses students are able to take, both between schools and within schools. Between schools, we know that schools that have high proportions of black and Latino students are less likely to offer higher level science courses like chemistry and physics (U.S. Department of Education Office of Civil Rights, 2018), and schools in less-resourced communities have difficulties implementing advanced coursework (Long, Conger, & McGhee, 2019). Within schools, student socioeconomic status explains differences in curricular intensity (Attewell & Domina, 2008) and English learner status explains differences in advanced course- taking (Johnson, 2019). Racial gaps within schools in advanced course-taking are also prevalent (Kelly, 2009; Riegle-Crumb & Grodsky, 2010). High school science course-taking may expose students to additional scientific content and practice. There are potential academic benefits to taking additional or more rigorous science courses which include increased science learning, preparation for postsecondary science study, and generation of scientific literacy skills that will be applicable in everyday life such as when making decisions about health or environmental concerns. High school science coursework may also be a potential policy lever for generating interest in STEM careers or giving students who already have an interest the skills they need to persist and be successful in college or career. This is especially important for the impact such 3 policies might have on groups who have traditionally been underrepresented in STEM majors and STEM fields. Physical science fields in particular show large gender disparities, with only about twenty percent of bachelor’s degrees in engineering and physical science being awarded to women in 2016 (National Science Foundation, 2019). There are also concerns about who has access to STEM coursework that may be important for future STEM success as students who take more advanced STEM coursework are more likely to declare a STEM major (National Science Board, 2019; also see literature review). Policies that require students to take more and more rigorous courses do not come without costs, though. Studies have documented the unintended consequences that can result from changes in course requirements. For example, several studies have found that students were more likely to drop out of high school in response to increased requirements (Jacob Dynarksi, Frank, & Schneider, 2017; Montgomery & Allensworth, 2010; Plunk, Tate, Bierut, & Gruzca, 2014). Rodriguez and McGuire (2019) found that when additional AP courses were offered in high schools, black-white gaps in course-taking actually increased when the policies were meant to give additional access to advanced coursework to groups that were underrepresented. Arce- Trigatti (2018) found that when schools were required to add AP courses, they did so at the expense of CTE courses, showing that resources for implementing new coursework are limited and may come at the expense of other programs. The Michigan Merit Curriculum The Michigan Merit Curriculum (MMC) is a set of high school course graduation standards that was first implemented for the high school freshman class of 2007-2008 (those who were expected to graduate during the 2011-2012 school year). The new requirements include four credits of language arts, four credits of math, three credits of social studies, three credits of 4 science, one credit of physical education, one credit of art, and one credit of online learning. Additionally, in order to satisfy the math and science requirements students are required to take specific math and science courses; in math they must take algebra I, algebra II, and geometry and in science they must take biology and either chemistry or physics.1 The MMC not only set standards for the number of credits required to graduate, but also set requirements in math and science that were more rigorous than the coursework many students were completing prior to the implementation of the policy. Prior to the MMC, school districts were left to set course graduation requirements themselves, leading to a lack of uniformity in coursework across the state. Dynarski, Frank, Jacob, and Schneider (2012) cite a state-wide survey that found only roughly one-third of Michigan school districts required four credits of math and three credits of science for graduation prior to the policy. Less than 12% of schools required chemistry or physics (Jacob et al., 2017). The introduction of standardized course requirements provides an opportunity to study the effects of the policy on students’ science course-taking behaviors. Prior to the introduction of the MMC policy, only about half of high school students enrolled in chemistry and one-fourth enrolled in physics (Wallsworth, Kim, & Troutman, 2015). National statistics were similar, with about two-thirds of high school graduates taking chemistry and one-third taking physics in 2005 (U.S. Department of Education, NCES, 2018). This policy differentially affected schools, with some seeing little change in students’ science course sequences and others substantially increasing the rates of advanced course completion in subjects such as chemistry and physics. 1 The requirements laid out in the MMC have subsequently been changed. Currently, students must still obtain three credits in science including biology and either chemistry, physics, anatomy, or agricultural science. The third science credit can come from a computer science or CTE courses. The law also provides the option for “successfully completing a program or curriculum that provides the same content as the chemistry or physics benchmarks, as determined by the department” (MI Comp L § 380.1278b (2015)). Currently these benchmarks are the Michigan Science Standards which are nearly identical to the Next Generation Science Standards (NGSS). 5 Descriptive analyses of the number and types of science courses students took before and after the policy can shed light on how such course-taking policies affect science course-taking. Given the lack of science coursework requirements before the policy, one might assume that the MMC would substantially change the number and/or type of science courses students took. Kim and colleagues (2019) showed that math course-taking after the introduction of the MMC increased (on average students took .16 more credits and passed .14 more credits). The results were driven by students in the schools with the highest proportion of economically disadvantaged students and by the least academically prepared students. Benefits of High School Science Course-taking High school science coursework has been linked to a variety of positive postsecondary and STEM-related outcomes. The literature specific to science coursework, though, has issues of self-selection that have not adequately been accounted for, leaving doubt about the magnitude of the relationship between high school science and outcomes policy might hope to affect. The list of outcomes reviewed here is not exhaustive of the outcomes science educators might hope to affect through high school coursework. For example, nurturing scientific literacy should be a goal for all students, especially in a world where scientific facts are called into question without basis, but data limitations do not allow me to include science literacy as an outcome. It is also worth noting that there is a strand of literature that investigates pedagogical features of high school science classes and their effects on postsecondary outcomes (e.g., Sadler & Tai, 2001; Schwartz, Hazari, & Sadler, 2008; Schwartz, Sadler, Sonnert, & Tai, 2009; Tai & Sadler, 2007; Tai, Sadler, & Loehr, 2005; Tai, Ward, & Sadler, 2006). These studies document relationships but suffer from many of the same issues of self-selection and recall bias that other studies on coursework suffer from. Additionally, teacher support has been shown to be an 6 important feature of high school science courses that can impact students’ decisions to pursue STEM later (Russell & Atwater, 2005). This literature review focuses on course-taking in general, because that is the focus of this dissertation. I recognize that potentially important determinants of postsecondary STEM outcomes cannot be tested using the transcript data used in this study. Pursuit of a Postsecondary Degree. There is some evidence that taking more rigorous and advanced science coursework improves rates of pursuing postsecondary education. For example, Long, Conger, and Iatorola (2012) used propensity score matching to compare Florida high school students who took advanced science courses to students who did not but were otherwise similar (including in their coursework in other subjects). They found that students who took advanced science courses were about nine percentage points more likely to attend a four-year college and slightly less likely to attend two-year college than students who took no advanced science courses. The drop in two- year attendance may be explained by shifts from students who might have attended a two-year college attending a four-year college instead. Although the authors control for many observables through their matching technique, the results of this study cannot be interpreted as causal since propensity score matching is unable to control for the unobservables of concern. For example, the thing that might make one student take an advanced science course while another very similar student does not is exactly the sort of unobservable that might also make the student who took the advanced science course more likely to attend a four-year college. Persistence in College Rigorous high school coursework has been found to be associated with persistence toward a college degree (Long et al., 2012; Burkam & Lee, 2003; Horn & Kojaku, 2001; 7 Schneider, Swanson & Riegle- Crumb, 1998). This is true in the general sense, but also for students pursuing STEM degrees (Adelman, 2006). For example, using data from the 1995-96 Beginning Postsecondary Students Survey, Horn and Kojaku (2001) showed that students who took more rigorous science courses in high school persisted in college at higher rates than those who did not. This is especially important in STEM fields where there are concerns about high rates of attrition from STEM majors (e.g., Ost, 2010), especially among female physical science majors. Labor Market Outcomes A number of studies, employing quasi-experimental designs, have linked coursework to increased wages (Altonji, 1995; Goodman, 2019; Levin & Zimmerman, 1995; Rose & Betts, 2004). In general, these studies have focused on math course-taking, although some have linked science course-taking to labor market outcomes as well. Rose and Betts (2004) found effects for math which were larger than the effect of science coursework. Another issue that several of these studies face is that they do not have data that allows them to investigate the effect of science courses independent of other coursework (Altonji, Blom, & Meghir, 2012). These studies indicate there may be an effect for science courses, but that it is likely less important than math and may be small in magnitude. Interest in STEM Perhaps the best evidence linking high school course-taking to interest in STEM found that taking calculus, a second year of chemistry, or taking physics was related to increased career interest for college freshman, controlling for their recollections of STEM career interest in middle school (Sadler, Sonnert, Hazari, & Tai, 2014). This study attempts to control for self- selection by including the retrospective measure of middle school STEM career interest. 8 However, the data were collected from college students and is subject to significant hindsight bias, so it is unclear to what extent controlling for recollections of career interest solves self- selection issues. Other studies (Burkam & Lee, 2003; Muller, Riegle-Crumb, Schiller, Wilkinson, & Frank, 2010) have found positive relationships between courses and interest but were unable to account for selection and thus it is unclear how reliable the positive relationship findings are. Pursuing a STEM Major Many studies have investigated the relationship between high school science course- taking and pursuit of STEM majors. In general, they find strong positive relationships (Engberg & Wolniak, 2013; Federman, 2007; Legewie & DiPrete, 2014; Maple & Stage, 1991; Tyson, Lee, Borman, & Hanson, 2007; Wang, 2013). Studies linking science courses to STEM majors have found differential impacts based on the type of high school course and particular major. Trusty (2002) found that for women, taking more advanced math (calculus) increased odds of majoring in STEM. For men, taking physics increased odds of majoring in STEM. In another study, taking physics in high school was a positive predictor of declaring a STEM major and was especially important for females (Bottia, Stearns, Mickelson, Moller, & Parker, 2015). Gender differences are also of interest when it comes to STEM majors because of discrepancies in the STEM fields that women take courses in in high school (i.e., more biology and chemistry, less physics [National Science Board, 2019]) and the majors they choose (National Science Foundation, 2019). Majoring in engineering is also positively related to number of science and math courses taken in high school (Main, Darolia, Koedel, Yan, & Ndashimye, 2017). 9 College STEM Grades Students’ performance in STEM college courses has been shown to be important for remaining in STEM majors and persisting until graduation. Chen (2014) found that poorer performance in STEM courses, relative to non-STEM courses, during the first year of college was associated with changing to a non-STEM major or dropping out of college. Likewise, Ost (2010) found that females were especially susceptible to leaving physical science majors when their grades in STEM courses were low compared to their grades in other fields. If students are to be able to pursue the college courses of study that they want without feeling like they are floundering, they need to be well-prepared. This preparation could potentially be achieved by taking STEM courses in high school, as the literature reviewed below indicates a relationship between high school courses and performance in college. Ost’s (2010) findings related to gender are especially pertinent given the underrepresentation of women in physical science careers (National Science Foundation, 2019). In a survey of introductory science college courses, Sadler and Tai (2007) found that taking biology in high school helped with biology grades in college, taking chemistry helped with college chemistry grades, and the same was true with high school physics coursework and physics grades. In another study, this link in physics was shown again. Students who took physics in high school got better grades in introductory college physics, and students who took two years of physics and calculus did even better (Sadler & Tai, 2001). Other studies have also documented a positive relationship between high school physics preparation and performance on concept inventories and in college physics classes (Harlow, Harrison, & Meyertholen. 2014; Salehi, Burkholder, Lepage, Pollock, & Wieman, 2019). However, Sadler and Tai (1997) 10 showed that the relationship between taking physics in high school and introductory physics course grade in college was small. Graduating with a STEM Degree In addition to literature that links increased science enrollment in high school to general degree completion (e.g., Adelman, 2006; Schneider et al, 1998), there are also studies that link science coursework to STEM degree completion. Maltese and Tai (2001) found that increased science enrollment was a positive predictor of earning a degree in STEM. Engineering degree attainment is positively related to number of science and math courses taken in high school and number of math and science courses offered by a student’s high school (Main et al., 2017). Tyson (2011) also found a relationship between science courses taken in high school and STEM degree attainment. A related strand of literature has identified a relationship between taking advanced science course-work and similar outcomes (Long et al., 2012; Mattern, Shaw, & Ewing, 2011). Quasi-experimental Evidence on Course-taking There have been several high-quality quasi-experimental investigations of the effects of mandatory coursework policies on student outcomes that have employed methods that account for selection bias. They have found little evidence that more science coursework corresponds to improved outcomes such as course grades (Montgomery & Allensworth, 2010) and test scores (Buddin & Croft, 2014; Saw, 2016), although Jacob and colleagues (2017) did find that the MMC policy increased science ACT scores by .2 points. On the contrary, there is evidence that increasing course requirements may lead to unanticipated and undesirable outcomes such as increased dropout rates, especially for the least prepared students (Jacob et al., 2017; Montgomery & Allensworth, 2010; Plunk et al., 2014). These studies find heterogenous effects, 11 with students with lower academic performance or who are from more economically disadvantaged schools experiencing larger effects of mandatory math and science coursework (Jacob et al., 2017; Plunk et al., 2014l; Saw, 2016). An Arkansas policy that required schools to offer AP programs led to a reduction in CTE courses offered as schools shifted resources to the new AP courses, and little evidence of positive impact on academically weaker schools (Arce-Trigatti, 2018). Darolia and colleagues (2018) found no impact of offering more science courses in high schools on students’ STEM postsecondary outcomes. Together, it appears that providing access to science courses to students may not be enough on its own to improve STEM outcomes for the students who tend to lack access to advanced coursework. There are many studies that document a positive relationship between students’ high school science course-taking and postsecondary STEM outcomes. However, these studies have not adequately accounted for selection bias. Students who take more or higher-level science courses in high school likely have chosen those routes due to unobserved characteristics such as pre-existing interest in STEM or ambition that also lead to later achievement and attainment (Domina, Penner, Penner, & Conley, 2014; Warne, 2017). Ignoring this self-selection effect leads to over-estimating the impact of advanced STEM course-taking in high school (Dougherty, Mellor & Jian, 2006; Federman, 2007). Quai-experimental studies of mandatory course-taking requirements, that attempt to take self-selection into account by exploiting exogenous variation in course-taking, have found much more subdued effects. Thus, it is unclear to what extent, if any, a policy like the MMC had on students’ postsecondary STEM outcomes as many of these outcomes have not been tested quasi-experimentally. 12 As policies continue to stress the importance of high school science coursework for increasing interest, preparation, and persistence in STEM fields (e.g. 38 states and the District of Columbia require at least three science credits for high school graduation as of 2019 [Education Commission of the States, 2019] and nearly half of states have policies in place requiring school districts to offer Advanced Placement, IB, or dual college enrollment option [Education Commission of the States, 2016]), the mixed findings from previous research raise questions about the efficacy of such a strategy for meeting postsecondary STEM goals. This is especially concerning for the recruitment of groups that traditionally have not been well-represented in STEM fields, since “more of the same” might not be the prescription they need to pursue STEM at higher rates. Establishing credible causal estimates of the effects of additional advanced science coursework is critical for targeting coursework policies and interventions to improve STEM outcomes. Conceptual Framework Using opportunity to learn (Hallinan, 1988; McDonnell, 1995; Schmidt & Maier, 2009) as a conceptual lens, this study seeks to estimate how school characteristics, beyond measures of socio-economic disadvantage and achievement, interact with science course-taking policy to explain gaps in postsecondary STEM outcomes. The attainment of a STEM occupation is a multistep process that extends from early experiences to high school coursework, through postsecondary education, and into a career (Maltese & Tai, 2011). Advanced high school coursework may be a key influencer of students’ decisions to pursue a STEM degree (Long et al., 2012; Mattern et al., 2011) and provides equitable opportunities for students to explore and prepare for later STEM learning (Tate, 2001). Performance in college STEM coursework can serve as an important factor for persisting in a STEM degree field (Maltese & Tai, 2011; Ost, 13 2010), and access to advanced coursework can support students’ performance in college (Sadler & Tai, 2007). This view takes into account individual characteristics, such as gender and race/ethnicity, where there are gaps in participation in STEM fields (Chen & Weko, 2009; Mann & DiPrete, 2013; National Science Board, 2019; Riegle-Crumb & King, 2010; Riegle-Crumb et al., 2012). In high school, female students are less likely to take physics than male students (Riegle-Crumb, Kyte, & Morton, 2018; Riegle-Crumb & Moore, 2014; Xie & Shauman, 2003), and Black and Hispanic students are less likely to take advanced coursework as their White counterparts within schools (Kelly, 2009). Figure 2 Conceptual Framework Student Characteristics Gender Race/ethnicity Socioeconomic status Prior achievement Special education classification Limited-English proficiency classification Advanced science course-taking via mandatory graduation requirements Treatment (MMC) Outcomes Enrollment in 2-year and 4-year college Declaration of a STEM major Grades in key STEM coursesa Graduation with STEM degree School Characteristics Key Characteristics of Interest Advanced science coursework pass rate College attendance rate Rate of STEM major Rate of STEM graduation Instrumental Variable Proportion enrolled in advanced science Other Measures Socioeconomic status Achievement Less is known about how institutional factors relate to STEM outcomes and how they interact with policies mandating science coursework. Between schools, those with high proportions of minority students tend to offer fewer advanced and rigorous courses (Riegle- 14 Crumb & Grodsky, 2010; U.S. Department of Education, 2018), leading to substantial racial gaps in advanced science course-taking (Ho & Kao, 2018). In order to understand the impact of a policy such as the MMC on students’ opportunity to learn, it is necessary to understand how it interacts with institutional characteristics such as the quality of the school where the policy is being implemented. Traditionally this has been measured using test scores, but recent evidence suggests that other institutional factors may be key indicators of school quality (Jennings, Deming, Jencks, Lopuch, & Schueler, 2015). In this analysis, I view the school-level aggregates of advanced science course pass rate, college attendance, rate of student selection of a STEM major, and rate of graduation with a STEM degree as key moderators of the mandatory science course-taking policy (see Figure 2). Research Questions This dissertation explores the science course pathways that students take through high school, how those pathways differ within and between schools, and the effects of science coursework policy on STEM outcomes. Of particular interest, are institutional factors, beyond school-level achievement, that may have interacted with the policy to produce differential outcomes for students. 1) How did course-taking patterns change in Michigan after the MMC policy was enacted, and how did those changes differ based on student and school characteristics? 2) Did the MMC policy affect the number of science courses students attempted, the number they completed, the type of science courses students attempted and completed, proportion of students declaring a STEM major, and proportion of students graduating with STEM degrees? 15 3) How did the effects of the MMC policy differ based on student characteristics and school characteristics including advanced science coursework taking rate, college attendance rate, rate of student selection of a STEM major, and rate of graduation with a STEM degree? 4) Does taking additional science coursework increase the rate of declaring a STEM major, rate of taking an introductory science class during the first year of college, improve grades in key college STEM classes (introductory biology, chemistry, and physics), and increase rate of graduation with STEM degrees? 16 CHAPTER 2: METHODOLOGY Evaluating whether high school course-taking is successful at improving STEM-related outcomes is difficult due to endogeneity in students’ course-taking decisions and a lack of experimental data. This study employs a quasi-experimental design using detailed transcript data from multiple cohorts of a representative sample of Michigan high schools to provide improved estimates of the effect of mandatory advanced high school science coursework on STEM-related outcomes. These transcript data offer a richer description of science course-taking than administrative records alone, providing not only course titles and credits earned but also detailed course-catalog descriptions of course contents. Data I use data from the Michigan Transcript Study (MTS), which includes detailed high school transcript data from a representative sample of Michigan high schools. The resulting dataset contains the course-taking records of over 300,000 students from 129 high schools across seven cohorts (high school freshman classes of 2002-2003 through 2008-2009). These data cover a time period during which Michigan increased high school graduation requirements via the MMC policy. The data include course title, content, credits, and grades. Courses are identified using NCES School Courses for the Exchange of Data (SCED) codes, taking into account the reported course title as well as school-specific course catalog information about the content of particular courses. SCED codes allow for the identification of biology, chemistry, and physics courses, which are key predictors and outcomes in this study. The course-taking data from transcripts is linked with MCER (Michigan Consortium for Education Research), a state longitudinal dataset that includes demographic information about Michigan high school students such as gender, race/ethnicity, economically disadvantaged status, 17 special education status, limited-English proficiency status, and assessment scores. School-level characteristics such as enrollment, number of teachers, per-pupil expenditures, and aggregated school-level demographic characteristics are available as well. Approximately ninety-eight percent of the students in the transcript data match to student- and school-level administrative records. These transcript and demographic data are linked to STARR (Student Transcript and Academic Record Repository) which contains student course enrollment and completion information from all public colleges and universities in the state of Michigan. The NSC (National Student Clearinghouse) contains college attendance and completion information for a wider range of institutions (i.e., those outside of Michigan, private not-for-profit, and to a lesser extent for-profit institutions [Dynarski, Hemelt, & Hyman, 2015]), allowing for the measurement of these outcomes for a larger proportion of college-going students (over 94%) in the transcript data and across more cohorts than STARR. Coding of Course Catalogs and SCED NCES publishes the SCED (U.S Department of Education, 2007) which is a set of codes that identify high school courses. Its purpose is to classify courses so that different entities, for example school districts, can unambiguously compare courses that originated from different sources. When transcripts were collected from the sample of Michigan schools, the same course was named differently depending on which school it originated from. For example, the basic chemistry course (SCED code 03101) at one school might be called “Chemistry I”, while at another school it might be called “Foundations of Chemistry.” The former course could be quite easily assigned the SCED code 03101, while the latter course title is more ambiguous, although it would also likely be assigned the SCED code 03101. Without additional information about the 18 contents of the course, some assumptions must be made about how to classify the course, as course titles do not always reflect the contents of the course (Cogan, Schmidt, & Wiley, 2001). In the Michigan Transcript Study, the research team had access not only to the course titles that appeared on transcripts, but also to the course catalogs that contained descriptions of the courses. In addition to classifying course titles, which was governed by a set of rules to avoid ambiguous coding, the team also classified each course based on the description in the course catalog and how that description matched up with the description of the content of the course provided in the SCED. This additional information, which is missing from administrative datasets, provides a more accurate description of what the course was. The coding rules for course titles and descriptions are laid out in detail in the coding manual for the Michigan Transcript Study (Wallsworth et al., 2015). Variables In this section I describe how various variables are defined for the forthcoming analyses. First, high school science courses are divided into eight bins based on the NCES SCED code they are assigned. Table 1 summarizes the bins and which SCED codes fit into each. The purpose of this binning is twofold. First, any analysis that considers individual SCED codes to be separate courses would be unwieldy due to the large number of codes. The MMC policy is not specific enough about what counts as a biology course to consider only the basic biology SCED code as counting. A course like molecular biology would fulfill a biology requirement, so it is lumped together with basic biology for the purpose of calculating biology credits. Second, although the course catalogs provide options for classifying course titles from transcripts within each school, the same course can appear many different ways on transcripts and sometimes there are multiple courses with similar enough titles that it is ambiguous which course catalog option 19 they match with. Also, there are a substantial number of schools that were unable to provide course catalogs (at all or for some years that the transcripts covered). As such, trying to classify courses into exact SCED codes could introduce uncertainty when a broader SCED bin is sufficient for my analysis. Table 1 Classification of courses by SCED code Biology 3051 Biology 3052 Biology - Advanced Studies 3053 Anatomy and Physiology 3054 Anatomy 3055 Physiology 3056 AP Biology 3057 IB Biology 3058 Botany 3059 Genetics 3060 Microbiology 3061 Zoology 3062 Conceptual Biology 3063 Particular Topics in Biology 3097 Biology - Independent Study 3098 Biology - Workplace Experience 3099 Biology - Other Physics 3151 Physics 3152 Physics - Advanced Studies 3153 Principles of Technology 3155 AP Physics B 3156 AP Physics C 3157 IB Physics 3161 Conceptual Physics 3162 Particular Topics in Physics 3197 Physics - Independent Study 3198 Physics - Workplace Experience 3199 Physics - Other Chemistry 3101 Chemistry 3102 Chemistry - Advanced Studies 3103 Organic Chemistry 3104 Physical Chemistry 3105 Conceptual Chemistry 3106 AP Chemistry 3107 IB Chemistry 3108 Particular Topics in Chemistry 3147 Chemistry - Independent Study 3148 Chemistry - Workplace Experience 3149 Chemistry - Other Earth Science 3001 Earth Science 3002 Geology 3004 Astronomy 3005 Marine Science 3006 Meteorology 3007 Physical Geography 3008 Earth and Space Science 3047 Earth Science - Independent Study 3048 Earth Science - Workplace Experience 3049 Earth Science - Other 20 Table 1 (cont’d) Physical Science 3159 Physical Science 3160 IB Physical Science Environmental Science Integrated Science 3201 Integrated Science 3202 Unified Science 3203 Applied Biology/Chemistry 3003 Environmental Science 3207 AP Environmental Science 3208 IB Environmental Systems Note. All science SCED codes not listed here were categorized as "Other" and represent less than one percent of all course records that were classified as science courses. STEM majors and degrees were classified using Classification of Instructional Programs (CIP) codes which were developed by NCES to classify fields of study. STEM CIP codes were defined as those appearing on the Department of Homeland Security’s STEM Designated Degree Program List. Majors and programs were defined as engineering if the CIP code was 14.XXX (i.e. any CIP code with the two-digit 14 prefix), biology if 26.XXX, and physical science if 40.XXXX. Distinctions within physical science between chemistry (40.05XX) and physics (40.08XX) were not made due to the extremely low numbers of students majoring and earning degrees in the specific subfields of physical science. College courses came from the Michigan STARR database and included fields for course title, department code, and number. Using course catalogs from Michigan universities, I classified biology, chemistry, and physics departments, creating flags for if courses were offered by the particular department. Some schools had general science or physical science departments, so courses within those departments were flagged as biology, chemistry, or physics if the course titles contained the word biology, chemistry, or physics. Sensitivity analyses (not presented here) of models that use college courses as an outcome use a broader definition that includes courses 21 from other departments that contain the word biology, chemistry, or physics in the title. For example, a course titled “Physics for Engineers” might not be flagged as a physics course under the former definition but would in the latter, broader definition. Courses were also classified as being 100-level and under or 200-level and under and used college specific information about course numbering systems to determine these classifications. Research Question 1 – Science Course Sequences How did course-taking patterns change in Michigan after the MMC policy was enacted, and how did those changes differ based on student and school characteristics? Descriptive Analysis of Science Course-taking Patterns A descriptive analysis of how the MMC changed science course-taking behaviors is useful for understanding how a state-wide policy like the MMC might be implemented. Pre- policy, schools varied in the number of science credits that they required students to take and in the types of courses that were taken by students to fulfill those requirements. First, I describe what science course pathways students took through high school (see Stevenson, Schiller, & Schneider, 1994) pre-policy and how those pathways differed within- and between- schools based on individual and school characteristics such as gender, race/ethnicity, achievement, and socio-economic status. I produce similar descriptions for post-policy cohorts and document changes between the pre- and post-policy cohorts’ science course pathways and how the changes also relate to student and school characteristics. Analysis of course pathways includes using the pathways elucidated by Schneider and colleagues (1997), but also to see what proportion of student took and met all MMC requirements during the pre- and post-policy timeframes. I also break each course down by credits taken at each grade level, to see if patterns in timing of course-taking differed in addition to popularity of courses. 22 For example, consider a hypothetical school where pre-policy all students take earth and space science as 9th graders, biology as 10th graders, and then an integrated physical science course as 11th graders. As 12th graders one-quarter of students, the highest achievers, take chemistry. Post-policy, students at the same school still take earth and space science as 9th graders, biology as 10th graders, but then take chemistry as 11th graders, with 25% of the lowest achievers taking integrated physical science. The top quarter of achievers then go on to take physics as 12th graders, while a small percentage of the students who took integrated physical science as 11th graders take chemistry as 12th graders. In this example, both the average number of credits taken increased as did the average level of courses taken. All students in the school did not meet the graduation requirement of taking chemistry or physics, but evidence from Kim’s (2018) study of MMC math requirements showed that many students did not meet the requirements and were still allowed to graduate as normal. High-school science course-taking does not have a hierarchical structure like math or English course sequences. Students have more choices about the courses they take, especially with regard to higher-level courses. Course sequences can help summarize the level or rigor of courses a student takes without the complexity that can arise from the many options students have for science course-taking (Riegle-Crumb, 2006; Riegle-Crumb, Muller, Frank, & Schiller, 2005). Research Question 2 - Effects of MMC Policy Did the MMC policy affect the number of science courses students attempted, the number they completed, the type of science courses students attempted and completed, proportion of students declaring a STEM major, and proportion of students graduating with STEM degrees? 23 Intent-to-treat Estimates In order to understand the overall effect of the policy on outcomes such as number of science credits attempted, number of science credits completed, proportion of students choosing a STEM major, and proportion of students graduating with a STEM degree, I use a pre-post regression analysis.2 Since the policy was enacted state-wide, there is no comparison group in this analysis, but rather I show what happened state-wide to the outcomes of interest after the policy was enacted, controlling for the trend in those outcomes before the policy. This will give an estimate similar to an “intent-to-treat” estimate, which tells us what happened to schools in the state regardless of whether the policy affected the science course-taking practices in their school. I estimate the following: !!"#=β$+ β%&&'+ β&'(ℎ(*+#+ ,'()β+ -()β+ µ"+ε!"# !!"# is the outcome for student i in school s in cohort c, &&' is equal to 1 for post-policy cohorts, and '(ℎ(*+# indicates the year the cohort started high school and controls for the time trend. ,'() is a vector of student characteristics,3 -() is a vector of time-varying school characteristics,4 and µ" is a school fixed effect. ε!"# is the error term. The coefficient of interest is β% which represents the difference in the outcome ! between pre- and post-policy students. 2 Jacob et al. (2017) showed that the policy increased ACT science scores by .2 points but reduced graduation rates for the least prepared students (although these findings were sensitive to specification choices). Kim (2018) showed that students who took algebra II enrolled in college more often, were more likely to graduate with a bachelor’s degree on time, and economically advantaged students who took algebra II were more likely to major in STEM. Kim et al. (2019) showed that post-MMC cohorts took .2 additional years of math, higher-level math courses, and had higher college enrollment rates. 3 Student characteristics included in all models are gender, race/ethnicity, economically disadvantaged status (longitudinal measure as proposed by Michelmore & Dynarski, 2017), special education status, limited-English proficiency status, and lagged assessment scores. 4 School characteristics included in all models are the log of student expenditures, enrollment, enrollment squared, percent economically disadvantaged, and the log of the number of teachers. Models estimating effects on postsecondary outcomes also include unemployment figures to control for economic conditions that may have affected students’ college decisions. 24 The coefficient from the pre-post analysis that represents the difference between pre- and post-policy outcomes cannot be interpreted as causal because, although I control for time trend, there are potential confounding factors that may have affected science course-taking beyond just the introduction of the policy. The estimate for β%does represent what happened in the state, but results cannot be attributed to the policy alone. Treatment-on-treated Estimates Not all schools in the state were affected by the MMC policy. Some districts already had graduation requirements in place that required three credits of science or norms in place where students took chemistry or physics at high rates. In order to find out how the MMC policy affected the number of science credits attempted, number of science credits completed, proportion of students choosing a STEM major, and proportion of students graduating with a STEM degree, I divide the schools in the transcript sample into an untreated and treated group. The untreated group is those schools that were (relatively) unaffected by the policy based on their pre-policy course-taking patterns and the treated group is those that should have been affected by the policy based on their pre-policy course-taking patterns. The untreated or policy- unaffected group is the top two quintiles of 2003 school proportion of students taking all MMC science requirements. The treated or policy-affected group is the bottom two quintiles of 2003 school proportion of students taking all MMC science requirements. I also run models where the groups are determined based on quintiles of proportion of students meeting the MMC science requirement in 2003. In order to estimate the effect of the policy on treated schools compared to untreated schools, I will use a difference-in-differences strategy. Many of the conditions that might have affected all schools in the state, making the coefficient of interest from the previous analysis 25 difficult to interpret as causal, should affect the treated and untreated groups similarly. Also, the impact of the policy on schools that we would expect the policy to actually affect is of more relevance for future policymaking as they were the target of the policy in the first place. Difference-in-differences shows changes in the level of the outcome after the introduction of the policy (Shadish, Cook, & Campbell, 2002). The treated and untreated groups are formed based on 2003 (pre-policy) school proportions of students who took and met the MMC science requirements. The top two and bottom two quintiles are compared, as well as sensitivity analyses with a different threshold for categorization as treated and untreated. The following model will be estimated: representing the time in which the policy was active. The interaction between treatment status school, s, was classified as being policy-affected by the MMC (i.e. they were in the bottom two !!"#=β$+β%0*12+13"+β&&&'#+β*0*12+13"∗&&'#+,'(β+-() β+5!"# where !!"# is the outcome for student i in school s and cohort c. 0*12+13# represents whether a quintiles of science course-taking in 2003). &&'# is equal to 1 in all years post-policy, and the MMC is 0*12+13"∗&&'# and is equal to 1 if a school-cohort was policy-affected and subject to MMC science requirements. Otherwise the interaction is 0. ,'( represents a vector of student characteristics and -() represents a vector of school characteristics. 5!"# is the error term. The coefficients of interest are β% which represents the pre-policy difference between the policy-affected and unaffected groups, β& which represents the difference between pre- and post- policy for the policy-unaffected group, and β* which represents the differential effect the policy- affected group experienced compared to the policy-unaffected group. Added together, β& and β* show the total change that the policy-affected group experienced after the policy took effect. 26 Sensitivity analyses use only the top and bottom quintiles to see to what extent the results are driven by schools at the extreme ends of the distribution of 2003 MMC science course- taking. Additionally, the transition cohorts (i.e. those cohorts that were already in high school when the policy was announced but who were not subject to its requirements) were dropped for both the pre-post models and difference-in-differences models to check that schools that opted to change course offerings earlier than required by the policy did not mask the effects by attributing increased science course-taking to the pre-policy group. Research Question 3 - Heterogeneity by School Characteristics How did the effects of the MMC policy differ based on student characteristics and school characteristics including advanced science coursework taking rate, college attendance rate, rate of student selection of a STEM major, and rate of graduation with a STEM degree? Interaction of Course-taking Policy and School Characteristics There is likely heterogeneity in the effect of a course-taking policy like the MMC, both at the student and school levels. To investigate these differences at the student level, interactions of student characteristics such as gender, race/ethnicity, and economically disadvantaged status and the policy are included in the regression models as well as comparisons of students by quintile of the continuous achievement variable. Similarly, I test school characteristics by quintile. Some school characteristics are tested frequently such as achievement level, level of economic disadvantage at the school, and proportions of minority students at the school. I test other school characteristics that may specifically impact the implementation of or reaction to the MMC science coursework requirements including pre-policy (i.e. 2003) proportion of students taking chemistry, physics, and AP coursework, college attendance rate, rate of student selection of a STEM major, and rate of graduation with a STEM degree. 27 Research Question 4 – Relationship Between Course-taking and STEM Outcomes Does taking additional science coursework increase the rate of declaring a STEM major, rate of taking an introductory science class during the first year of college, improve grades in key college STEM classes (introductory biology, chemistry, and physics), and increase rate of graduation with STEM degrees? Instrumental Variable Estimation The well-documented positive relationship between high school science course-taking and postsecondary STEM outcomes cannot be interpreted as causal because the studies documenting the relationship suffer from self-selection. Key unobservable variables such as preexisting interest or motivation in STEM are (very) likely correlated with both the predictor of interest, high school science course-taking, and the outcome. Notably, in their study, Sadler and colleagues (2014) attempted to control for pre-high school interest in STEM by asking college students to recall what their interest in STEM was during middle school. This method likely has not accounted for the problem of unobservables because it focused on only one potential confounder and because the method they used is susceptible to hindsight bias. In order to make causal claims about the relationship between high school science courses and postsecondary STEM outcomes, exogenous variation in course-taking is needed. This could be accomplished through a randomized experiment where students’ course-taking is determined by something unrelated to the unobservable of concern (i.e. their random assignment to take a course or not). Of course, randomly assigning students to take high school physics or chemistry is not viable, so other sources of exogenous variation are necessary. To understand the causal relationship between taking chemistry or physics in high school and STEM-related postsecondary outcomes, I employ instrumental variable (IV) estimation. An 28 IV is a variable that is correlated with the causal variable, in this case taking high school chemistry or physics, and that is uncorrelated with other determinants of the outcome, in this case a postsecondary STEM outcome such as graduating with a STEM degree (Angrist & Pischke, 2009). Since the IV is assumed to be uncorrelated with unobserved determinants of the outcome, it lets us identify a portion of the variation in the relationship between the causal variable and the outcome that is exogenous (Murnane & Willett, 2011). In this case, the IV is the proportion of students in a school-cohort that take chemistry (or physics). Similar IVs are commonly used in the literature on course requirements more broadly, especially in math, and often in relation to labor market outcomes (e.g. Altonji, 1995; Altonji, Blom, & Meghir, 2012; Federman, 2007; Kim, 2018; Levine & Zimmerman, 1995; Rose & Betts, 2004) but have not been applied to science coursework for the sake of estimating STEM- related outcomes. This instrumental variable must meet the two requirements laid out previously. First, it must be correlated with the causal variable. In this case, we would expect that the higher the participation in taking physics at a school, the more likely an individual student at that school is to take physics as well. A school norm might dictate that most students take physics, so a student who otherwise might not have taken physics is essentially induced to do so because their peers are taking it or the school norm is to do so. Second, the IV must be uncorrelated with unobserved determinants of the outcome. This is known as the exclusion restriction and is not testable (i.e. there is no way to know for certain that the IV is uncorrelated with the error term since we do not observe the determinants of the error term). However, we can assume that the only mechanism through which proportion of students in a school taking physics affects an individual students’ propensity to graduate from 29 college in STEM (as one example of an outcome), conditional on other observable characteristics, is through its influence on making that student take physics. One situation in which these assumptions may break down is if a student attends a STEM school where a high proportion of students take physics and the student attended that school precisely because of their interest in STEM, one of the chief unobservables we care about. Whether or not a student attends a STEM school is observable and can be controlled for in regression models (and the sample for this project does not contain any STEM schools) so this scenario is not of concern for the IV meeting the exclusion criteria. Although whether the exclusion restriction is violated is not testable, methods exist for bounding estimates obtained from IV estimation based on assumptions about the relationship between the IV and unobservables in cases where the instrument is only “plausibly exogenous” (e.g. Conley, Hansen, & Rossi, 2012; Nevo & Rosen, 2012). The IV estimate can provide an upper bound for the causal relationship, and sensitivity analyses can show how much the IV estimate overestimates the relationship if it does not take care of the self-selection issues entirely. IV methods provide an estimate of the local average treatment effect (LATE) which in this case represents the variation in taking high school physics that is affected by the proportion of students at the high school that take physics. It does not represent an estimate of the overall effect of taking high school physics on postsecondary STEM outcomes. However, those students who were induced to take physics in high school based on the proportion of students in their school-cohort who also did are exactly the kind of students who might be induced to take high school physics by other policies that we could enact (and not just course graduation requirement policies). In this sense, it is like a “treatment-on-treated” estimate of high school science course- taking. 30 The data for this research question come from pre-MMC cohorts of students, so that students’ decisions about course-taking were not affected by the MMC science requirements. First, I estimate the relationship between science course-taking and subsequent STEM outcomes. !!"# = β$+6!"#β%+-"#β&+β*'!"#+7"+8#+ε!"# physics course in the first year of college, grades in introductory STEM courses, and graduation where !!" is a postsecondary STEM outcome (choice of a STEM major, taking a chemistry or with a STEM degree) for student i in school s and cohort c. 6!"# is a vector of student characteristics and -"# is a vector of time-varying school characteristics. '!"# is the predictor of cohort c. 7" is a school fixed effect and 8# is a cohort fixed effect. 5!"# is the error term. Standard interest, the type of STEM course (chemistry, physics, both) taken by student i in school s and errors are clustered by high school. using two-stage least squares, I estimate the first stage: This above model is similar to those from the studies cited earlier that suffer from self- selection concerns. The error term contains unobservables, such as interest in science, that make '!"# endogenous and the interpretation of β* non-causal. In order to estimate the causal LATE, '!"#=ρ$+ρ%+:;ℎ((<:ℎ2*1'"#+,!"#ρ&+-"#ρ*+7"+ µ#+ω!"# where '!"# is whether student i in school s and cohort c took a science course of interest, :;ℎ((<:ℎ2*1'"# is the instrumental variable representing the share of students in school s and cohort c that took science course C. ,!"# represents a vector of student characteristics and -"# represents a vector of time-varying school characterizes. 7" is a school fixed effect and 8c is a cohort fixed effect. The second stage is: !!"#=β$+β%',"#>+,!"#β&+-!"#β*+7"+ µ#+5!"# 31 where !!"# is the postsecondary outcome of interest, ',"#> is the predicted outcome from the first stage, and all other terms are the same as in the first stage equation. The coefficient ?1 is the LATE of taking course C on outcome Y. Description of Analytic Sample Table 2 provides descriptive statistics for the analytic sample and for the sample broken into pre- and post-policy cohorts. This includes 1023019 student-year observations from 293749 students. These students come from 126 unique schools. The data cover seven cohorts of students (2002-2003 freshmen to 2008-2009 freshmen; cohorts are referred to by the last year of the school year, so 2002-2003 is 2003 etc.). The MMC policy was announced so that the 2006 and 2007 cohorts are “transition cohorts” which were not affected by the MMC requirements formally, but the policy’s existence was known so course-taking may have begun changing in this time period in preparation for the 2008 start. The 2008 and 2009 cohorts are the post-policy cohorts. In Table 2 we see that there were few changes in student characteristics between the pre- and post-MMC periods, although the proportion of students who were ever economically disadvantaged increased (likely due to the economic downturn in 2008 and 2009). The descriptive statistics show no change in the overall number of science credits students earned or received. The numbers of credits in chemistry and physics, though appear to have gone up substantially compared to the pre-policy levels. 32 Table 2 Summary statistics of sample characteristics Full Sample 41.6% 39.3% 47.4% 49.7% 66.8% 23.9% 5.0% 3.5% 0.8% 5.0% 49.5% 66.7% 24.8% 4.5% 3.2% 0.8% 4.9% 49.4% 66.6% 25.2% 4.3% 3.1% 0.8% 4.9% Pre-MMC Post-MMC Student Characteristics Female White Black Hispanic Asian Other Race Ever Limited English Proficient Ever Economically Disadvantaged Student Course-taking Total Science Credits Taken Total Biology Credits Taken Total Chemistry Credits Taken Total Physics Credits Taken Total Science Credits Earned Total Biology Credits Earned Total Chemistry Credits Earned Total Physics Credits Earned School Characteristics Enrollment Number of Teachers Per-Pupil Expenditures 83209 Student Observations 288001 Student-Year Observations School Observations 126 Note. Student characteristics and student course-taking are calculated as the mean with one observation per student. School characteristics are the means of school-year observations formed from student characteristics. Dollar amounts are expressed as real 2010 dollars. 3.09 1.05 0.76 0.31 2.62 0.89 0.64 0.28 3.09 1.08 0.53 0.22 2.65 0.93 0.47 0.20 3.09 1.07 0.60 0.24 2.64 0.92 0.52 0.22 293749 1023019 126 210540 735018 126 1259 61.6 5798 1276 62.6 5788 1147 54.4 5880 33 CHAPTER 3: DESCRIPTION OF HIGH SCHOOL SCIENCE COURSE-TAKING PRE- AND POST-MMC How did course-taking patterns change in Michigan after the MMC policy was enacted, and how did those changes differ based on student and school characteristics? Description of Science Course-taking Pre- and Post-MMC Prior to the introduction of the MMC policy in Michigan, school districts were free to set their own graduation course requirements. This meant that, although there are many common courses that students take in high school such as biology and chemistry, there may have been variation in what courses were offered and what students took between schools. Likewise, within schools students may have been on different course tracks based on achievement or simply because they were given choices about which science courses to take. The MMC should, in theory, have standardized to a certain extent the science courses students took. Everyone under the MMC should take biology, one of chemistry or physics, and another science credit. In this analysis, I describe the course-taking patterns pre- and post-policy and look for changes that occurred based on student and school characteristics. Table 3 shows the number of credits students took in each of seven course bins by grade for the pre- and post-policy time periods for all schools in the sample. Looking at the 9th grade, Biology was the most popular course followed by earth science and integrated science. There were also substantial credits earned in physical science, but very few in chemistry and physics. Post-MMC more students took Biology during the 9th grade with fewer taking earth and integrated science. In the 10th grade, there are fewer students taking biology after the MMC was introduced, although these may have shifted to 9th grade to replace earth science and integrated science courses that were no longer offered. Both pre- and post-policy, though, biology was by 34 far the most popular 10th grade science class. 10th grade was also the most common time that students took biology. Table 3 Science credits taken by grade Biology Chemistry Physics Earth Science Physical Science Integ. Science Envir. Science 0.02 0.18 0.34 0.10 0.33 0.55 0.17 0.15 0.25 0.08 0.05 0.05 Pre-MMC 9th Grade 10th Grade 11th Grade 12th Grade Post-MMC 0.00 9th Grade 0.00 10th Grade 0.01 11th Grade 12th Grade 0.03 Note. Table entries reflect the average number of credits taken in each subject by a student in each grade. Pre- and post-MMC is based on a student's cohort, defined as whether they were subject to MMC course-taking requirements based on the year they started 9th grade. 0.39 0.46 0.14 0.15 0.05 0.29 0.43 0.11 0.13 0.05 0.03 0.05 0.12 0.01 0.01 0.01 0.01 0.02 0.10 0.16 0.01 0.04 0.18 0.16 0.13 0.06 0.02 0.01 0.17 0.05 0.01 0.01 0.00 0.00 0.01 0.02 0.20 0.07 0.04 0.02 Chemistry becomes a much more popular option in the 10th grade, with .175 credits earned, on average, in chemistry during the pre-MMC period. Besides biology, there is no other course that reaches above .075 credits during 10th grade. Most students appear to be taking either biology or chemistry, with others scattered among many other offerings. Post-MMC more credits are earned in Chemistry during the 10th grade, up to .292 on average, and the share of credits in physics, earth science, physical science, and integrated science drop. In both the pre- and post- policy periods, chemistry is the most common course for 11th graders, although by the 11th grade physics is becoming more common. Biology credits are much less common starting in 11th grade compared to their 9th and 10th grade levels. However, compared to other courses biology 35 continues to be a common area of study. The number of credits being earned in chemistry also increased during the 11th grade from the pre- to the post-policy time period. Physics course credits are quite uncommon compared to biology and chemistry credits, although they become more common in the 11th and 12th grades. This is especially true for 11th grade physics in the post-MMC time period. Overall, we see a shift toward fewer and earlier credits in biology and an increased number of credits in chemistry and physics, especially during the 10th and 11th grades. There is almost no difference in the total number of credits students earn in science; earth science and integrated science courses seem to have been sacrificed in favor of chemistry and physics courses. The analyses in Chapter 4 will bring understanding about whether the changes described here were possibly due to the MMC. Individual courses and when students took them are one way to understand how course- taking evolved with the introduction of the MMC. However, course sequences may help us better understand how combinations of courses changed for students throughout high school. Using categories developed by Schneider et al. (1997) I have classified students into one of six course sequences. They appear in Table 4 from highest, or most rigorous, to lowest. The highest course sequences became slightly more popular after the MMC was enacted, and fewer students were in the lowest category, earning very few or basic science credits. Compared to the results from Schneider et al. (1997) there were far fewer students taking both chemistry and physics as well as taking biology along with another core science course (chemistry or physics). That many students in Michigan were not taking both chemistry and physics is not too surprising, given that even after the MMC students were only required to take one or the other. What is surprising is that so many students in Michigan were taking such low levels of science. Going back to the 36 breakdown of courses by grade, this may be due to the high levels of students taking earth science, physical science, and integrated science courses which Schneider and colleagues did not consider to be core science courses. Table 4 Science course sequences Pre-MMC Post-MMC Schneider et al. (1998) Chemistry and Physics and Second Year Core Science Chemistry and Physics Biology and Other Core Science Chemistry Biology and Biology II Biology or Lower Science Took MMC Passed MMC Note. Course sequences are drawn from Schneider et al. (1998). MMC requirements include three science credits, one of which must be biology and one of which must be either chemistry or physics. 3.7% 6.2% 7.0% 19.2% 6.7% 57.2% 7% 17% 13% 24% 8% 31% 4.3% 7.3% 8.6% 26.1% 3.0% 50.8% 38.6% 32.7% 29.8% 26.1% - - The course sequences proposed by Schneider et al. (1997) do not include a category that is the same as what the MMC required. The MMC called for three credits, one of which must be biology and one of which must be either chemistry or physics. Table 4 also includes the proportions of students meeting the MMC course sequence before and after the policy. We see that although there was a sizable increase in the proportion of students taking and passing the MMC courses, there were still over 60% of students who did not take and almost 70% of students who did not pass the MMC requirements after the policy went into effect. This is similar to the findings of Kim (2018) who reported that many students were not meeting the MMC math requirements as well. 37 Table 5 presents results similar to Table 3, except Table 5 divides the sample based on individual characteristics during the pre-policy period rather than by pre- and post-policy. Panel A shows male and female students’ credits taken in each science course by grade. There are not major differences between the numbers. Panel B shows non-white versus white students, and there are some differences. In the 9th grade non-white students take more credits in earth and integrated science classes and then more biology credits in the 10th grade. The greater number of credits taken by non-white students in chemistry and physics in the 11th and 12th grades are largely driven by Asian students who had high rates of taking those courses. A similar pattern is apparent in the results for economically disadvantaged students. Students who have been economically disadvantaged at any time during high school take biology and chemistry later, in 10th and 11th grade most commonly rather than 9th and 10th grade like the students who were never economically disadvantaged. The never economically disadvantaged group also took physics courses in the 11th and 12th grade at a higher rate. Panel D shows the results for the first (lowest) quintile and fifth (highest) quintile of achievement based on 8th grade MEAP scores. The highest achieving students were more likely to take biology in the 9th grade and take more credits in chemistry and especially physics. 38 Table 5 Student level heterogeneity in science course-taking by grade Physics Chemistry Biology 0.01 0.01 0.21 Earth Science 0.22 Physical Science Integ. Science Envir. Science 0.14 0.14 0.18 0.18 0.00 0.00 0.03 0.02 0.13 0.11 0.22 0.19 0.03 0.02 0.10 0.10 0.35 0.38 0.20 0.22 0.03 0.03 Panel A: Male vs. Female 9th Grade 0.34 0.35 10th Grade 0.50 0.54 11th Grade 0.15 0.18 12th Grade 0.12 0.18 Panel B: White vs. Non-white 9th Grade 0.36 0.32 10th 0.46 0.66 Grade 11th Grade 0.16 0.17 12th Grade 0.14 0.16 Panel C: Never vs. Ever economically disadvantaged 9th Grade 10th Grade 11th Grade 12th Grade 0.38 0.31 0.50 0.55 0.17 0.15 0.16 0.13 0.03 0.03 0.10 0.11 0.09 0.13 0.24 0.17 0.38 0.35 0.34 0.45 0.18 0.14 0.01 0.01 0.02 0.02 0.13 0.10 0.15 0.20 0.01 0.01 0.02 0.02 0.15 0.08 0.18 0.12 0.07 0.06 0.06 0.05 0.06 0.05 0.00 0.00 0.05 0.04 0.02 0.01 0.04 0.03 0.01 0.01 0.05 0.19 0.04 0.26 0.01 0.00 0.16 0.11 0.02 0.02 0.14 0.26 0.02 0.03 0.00 0.00 0.06 0.09 0.06 0.04 0.05 0.07 0.00 0.00 0.03 0.09 0.02 0.01 0.02 0.06 0.01 0.01 0.04 0.22 0.07 0.21 0.01 0.01 0.15 0.13 0.01 0.04 0.15 0.23 0.02 0.03 0.00 0.00 0.06 0.07 0.05 0.06 0.05 0.07 0.00 0.00 0.03 0.07 0.01 0.02 0.03 0.05 0.01 0.01 0.00 0.01 0.04 0.06 39 0.01 0.03 0.03 0.02 0.29 0.46 Table 5 (cont’d) Panel D: 8th grade MEAP math score (Q1 vs. Q5) 9th Grade 10th Grade 11th Grade 12th Grade Note. Table entries reflect the average number of credits taken in each subject by a student in each grade. 0.14 0.14 0.26 0.32 0.41 0.10 0.11 0.02 0.04 0.05 0.06 0.03 0.01 0.03 0.00 0.13 0.38 0.02 0.04 0.09 0.08 0.28 0.07 0.06 0.25 0.08 0.63 0.41 0.17 0.18 0.12 0.19 0.01 0.01 0.24 0.10 0.09 0.03 0.07 0.01 0.03 0.01 0.00 0.03 0.01 0.16 0.00 0.00 0.01 0.00 0.01 0.01 0.02 0.03 40 Overall, though, the differences in when students took courses based on student characteristics are relatively minor. In some cases students took biology or chemistry a year earlier than their comparison group, and these relationships generally were no surprise with high achieving and never economically disadvantaged students taking these courses earlier. The lack of apparent differences in Table 5 may be explained by the distribution of individual characteristics amongst schools. For example, there are not large differences in the proportion of female students between schools. If course access is being limited or course-taking norms are set at the school level then variation may be apparent across schools but not based on individual characteristics within schools. Likewise, although higher achieving students took some courses more often and took others earlier in high school, within schools there may have been limited options for which courses to take and when to take them, leading to less than dramatic differences between the highest and lowest achieving students. Finally, schools may have offered different levels of the same course (i.e. basic biology versus advanced biology), which the bins used in Table 5 cannot distinguish between. In additional analyses not presented here, it is not uncommon to see students in the same school taking courses that were coded with different SCEDs, even though they fall under the same larger course umbrella. Matching transcript entries to exact SCED codes was difficult in many schools because of the plethora of ways the (apparently) same course was represented on transcripts. Thus, breaking the SCED bins down further would lead to less reliable results as well as still show a largely school-level phenomenon in many cases. Table 6 is similar to Table 5 except Table 6 breaks course-taking down by school characteristics. In each panel, student course-taking is based on the 2003 level of each school characteristic. The panels compare students from schools in the first (lowest) quintile to the fifth 41 (highest) quintile. Panel A shows results based on the proportion of students from a school enrolling in postsecondary programs within six years after their expected high school graduation date. In the schools that enrolled the most students in postsecondary education, students took more credits in biology, chemistry, earth science, and especially in physics. The differences are generally small for any given grade, but in sum the represent many more credits earned in the aggregate over the course of four years of high school. A similar pattern is present in Panel B which shows the first and fifth quintiles of schools where students ever declared a STEM major in college. There are not many apparent differences by grade but student in the fifth quintile of schools earn more credits in total. Panel C shows results by proportion of students earning STEM degrees. In the lowest quintile, students take substantially more credits in biology, and are more likely to take integrated science in their first year as opposed to earth science. The lack of major differences in this Panel may be due to the small proportion of students graduating with STEM degrees (i.e. the differences between schools in the first and fifth quintiles on this factor are not large to begin with, so they may not translate into seeing differences in course-taking in high school). The lack of differences across the characteristics in Panels A-C also may reflect that other factors unrelated to course-taking determine which quintile a school falls in. Panel D shows course-taking by grade broken into the lowest and highest quintiles of percent of students in the school who were economically disadvantaged. The schools with the least economically disadvantaged students (quintile 1) took far fewer biology courses than in the schools with the highest proportion of economically disadvantaged students. However, it is more common for them to take a freshman year biology course, a sophomore year chemistry course, and a junior year physics course. The same pattern is apparent in Panel E when schools are divided by average achievement. The totals for the biology columns in Panels D and E indicates 42 that students in the poorest and lowest achieving schools took more than one biology class, whereas students in the least poor and highest achieving schools move on to take chemistry and physics more commonly in the 10th and 11th grades. 43 Table 6 School level heterogeneity in science course-taking by grade Biology Chemistry Physics Earth Science Physical Science Integrated Science Environmental Science 0.32 0.40 0.49 0.47 0.13 0.19 0.15 0.16 0.30 0.35 0.47 0.50 0.13 0.19 0.14 0.14 0.01 0.01 0.18 0.25 0.37 0.39 0.11 0.13 0.01 0.02 0.19 0.25 0.34 0.41 0.12 0.11 Panel A: Proportion Enrolling in College (Q1 vs. Q5) 0.00 0.01 9th Grade 0.01 0.02 10th Grade 0.05 0.16 11th Grade 12th Grade 0.10 0.21 Panel B: Proportion Ever Declaring STEM Major (Q1 vs. Q5) 9th Grade 0.00 0.01 0.01 0.02 10th Grade 0.05 0.18 11th Grade 12th Grade 0.12 0.23 Panel C: Proportion Earning STEM Degree (Q1 vs. Q5) 9th Grade 10th Grade 11th Grade 12th Grade Panel D: Proportion ever economically disadvantaged (Q1 vs. Q5) 9th Grade 10th Grade 11th Grade 12th Grade 0.00 0.00 0.01 0.03 0.09 0.17 0.22 0.20 0.01 0.01 0.01 0.02 0.17 0.09 0.17 0.20 0.01 0.01 0.25 0.22 0.51 0.42 0.14 0.12 0.02 0.02 0.27 0.17 0.34 0.52 0.11 0.19 0.41 0.31 0.74 0.54 0.20 0.18 0.17 0.14 0.40 0.28 0.48 0.85 0.21 0.18 0.16 0.18 44 0.07 0.32 0.08 0.08 0.15 0.03 0.10 0.05 0.04 0.19 0.09 0.10 0.15 0.03 0.10 0.03 0.20 0.31 0.11 0.08 0.12 0.03 0.10 0.04 0.24 0.29 0.08 0.09 0.03 0.13 0.05 0.09 0.04 0.08 0.03 0.04 0.01 0.01 0.01 0.01 0.27 0.10 0.04 0.04 0.02 0.04 0.02 0.02 0.06 0.11 0.05 0.04 0.01 0.02 0.00 0.01 0.27 0.17 0.03 0.04 0.02 0.01 0.02 0.01 0.08 0.15 0.04 0.02 0.01 0.01 0.00 0.01 0.35 0.14 0.08 0.06 0.07 0.02 0.06 0.01 0.13 0.09 0.05 0.02 0.01 0.01 0.00 0.01 0.10 0.31 0.05 0.08 0.04 0.09 0.01 0.07 0.00 0.00 0.01 0.01 0.00 0.00 0.01 0.01 0.00 0.00 0.01 0.02 0.00 0.00 0.02 0.05 0.00 0.00 0.01 0.05 0.00 0.00 0.01 0.04 0.00 0.00 0.01 0.03 0.00 0.00 0.01 0.02 Table 6 (cont’d) Panel E: 8th grade MEAP math score (Q1 vs. Q5) 9th Grade 10th Grade 11th Grade 12th Grade Note. Table entries reflect the average number of credits taken in each subject by a student in each grade from the first and fifth quintiles of school characteristics. 0.35 0.18 0.14 0.08 0.14 0.03 0.10 0.05 0.00 0.00 0.02 0.03 0.09 0.19 0.25 0.18 0.01 0.01 0.20 0.25 0.57 0.34 0.17 0.12 0.35 0.44 0.89 0.43 0.22 0.18 0.18 0.15 0.08 0.15 0.03 0.07 0.01 0.02 0.00 0.01 0.34 0.12 0.11 0.05 0.10 0.02 0.07 0.01 0.00 0.00 0.01 0.01 0.00 0.00 0.02 0.04 45 Discussion When looking for changes in course-taking between the pre- and post-MMC time periods, there was a trend of reduced biology course-taking, earlier and more often chemistry course-taking, and earlier and more often physics course-taking. These trends are similar to the differences observed in the school heterogeneity analysis. Table 6 showed that the schools with the smallest proportion of economically disadvantaged students and the schools with the highest average 8th grade achievement were different from the poorest and lowest achieving schools in the same way. These descriptive analyses indicate that as the MMC course guidelines were implemented, the poorest and lowest achieving schools changed to look more like the least poor and highest achieving schools. Put another way, the MMC policy may have led to students in the schools with the highest levels of economic disadvantage and lowest levels of achievement to actually take less science, in favor of a more focused list of classes that aligned with the MMC science requirements. Not only did schools change which courses students were taking, but they changed when they took them as well. This likely reflects constraints on the number of science courses students have time to take (and that schools have the resources to offer), especially given the other course-taking changes that occurred at the same time that the MMC changed science requirements. For example, Kim et al. (2019) showed that students in Michigan took, on average .2 more credits of math after the MMC. Such changes, across multiple subjects, may have meant that schools had to be more strategic about when they encouraged students to take certain science courses. Kim et al. (2019) also observed that the change in math credits taken was driven by low- SES schools, meaning that those schools may have been the most constrained in how they offered science courses. 46 Comparing course-taking sequences in Michigan to the national sample from Schneider et al. (1997), we see that it was not as common for students to reach the highest level of course sequences in either the pre- or post-MMC period. This is reflected too in the proportion of students taking and meeting the MMC science requirements, even after the policy was introduced. This is reflected in the magnitude of the changes seen in Table 3 between the pre- and post-policy time periods. There are some shifts in biology, chemistry, and physics course- taking but they are not large enough to greatly increase the proportion of students taking the MMC curriculum. The total number of science credits students took between the pre-MMC and post-MMC periods did not change much. In general, it appears that schools allocated three one-year timeslots for science and that that did not differ much between the pre- and post-MMC periods. Again, this may reflect the constraints that high schools face as students must meet requirements in many course disciplines. Given that most students only had three opportunities to take biology and either chemistry or physics, in order to meet the MMC science course requirements they would need to use their time wisely. There were still substantial portions of students taking non- MMC required science classes (which would not prevent them from meeting the MMC requirements since they need a third science credit to do so), but it also means that if they were taking a non-MMC class their other two science classes must be biology and either chemistry or physics. In many cases, either because of student or school factors, this appears to have not happened. In summary, a descriptive analysis of course-taking trends does not reveal massive changes in the course-taking of students before and after the MMC or by individual and student characteristics. Although there is some evidence of shifts in which courses students took and 47 when students took them, these changes may have mostly affected the lowest SES and lowest achieving schools as the pre-MMC highest SES and highest achieving schools looked quite similar in course-taking patterns to the post-MMC sample as a whole. The regression analyses that follow will help in understanding if any changes can be attributed to the MMC and which differences observed in the analyses associated with Research Question 1 true changes. 48 CHAPTER 4: EFFECTS OF THE MMC POLICY ON HIGH SCHOOL COURSE- TAKING AND POSTSECONDARY STEM OUTCOMES Did the MMC policy affect the number of science courses students attempted, the number they completed, the type of science courses students attempted and completed, proportion of students declaring a STEM major, and proportion of students graduating with STEM degrees? Number of Science Credits Taken and Passed – Pre-post Regression Analysis The results reported in this section represent “intent to treat” estimates of the MMC policy. They indicate what actually happened to students in the state, regardless of whether the policy actually affected the course-taking patterns in their school. As such, they represent the actual outcome of the policy and are relevant to policymakers as they show what might be expected to happen across an entire state if a policy that affects some schools and not others, like the MMC, were introduced. Columns 1, 2, and 3 of Table 7 show the impact of the MMC policy on the number of science credits taken by students during their high school careers. Column 1, which includes student covariates but no school covariates shows no statistically significant change in the total number of science credits students took under the MMC policy. Results which include school characteristics (column 2) and school fixed effects to isolate within-school changes (column 3) also indicate that there was no statistically significant change in the number of science credits students took after the MMC went into effect. Columns 4, 5, and 6 of Table 7 show the impact of the MMC policy on the number of science credits passed by students during their high school careers. Although the policy does not appear to have had an effect on the number of science credits students took, shifts in the types of science courses students enrolled in may have changed the difficulty of the credits students did 49 take, translating into differences in the number of credits students passed. Besides changing the actual courses students took, the policy may have affected the content of courses that students were already taking which also could affect the number of credits passed as teachers and schools reorganized curricula to better align with the changes in the MMC. All three specifications show no statistically significant effect on the number of science credits passed by students. X X X X X X (4) X (1) -0.008 (0.072) 3.248 -0.012 (0.071) 2.806 (3) 0.002 (0.066) 3.254 Passed (5) 0.039 (0.083) 2.812 (6) 0.003 (0.063) 2.812 Taken (2) 0.042 (0.083) 3.254 Table 7 Estimated impact of the MMC on science credits taken and passed Change in credits Pre-MMC Mean Student Covariates School Covariates School Fixed Effects N 251241 Note. Coefficients represent the change in number of science credits taken (columns 1-3) and passed (columns 4-6) during high school. Columns 1 and 4 include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), columns 2 and 5 also include school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and columns 3 and 6 also include school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 251825 251241 251241 251825 251241 X X X X X One possibility is that school reacted to the announcement of the MMC, rather than the actual enactment of the policy, and students started taking additional science credits in cohorts prior to the 2008 cohort. If this were the case, the increase in science credits taken and passed during the two transition cohorts may be masking the effect of the MMC policy since in Table 7 these cohorts are included in the pre-MMC group. To test this possibility, I have performed the 50 same analyses as in Table 7 with the transition cohorts removed so that the comparison between the pre- and post-MMC students does not include those who might have received some MMC- like instruction if schools transitioned their coursework earlier than required. The results of these models are found in Table 8 and indicate that the lack of significant results in Table 7 on total science credits taken were not due to the transition cohorts. There is still no statistically significant effect of the MMC policy on total science credits taken or total number of science credits passed. (3) X X X X X X -0.036 (0.131) 2.827 -0.023 (0.132) 3.260 (1) 0.009 (0.169) 3.251 Passed (5) 0.079 (0.168) 2.827 (4) 0.014 (0.170) 2.819 Taken (2) 0.054 (0.168) 3.260 Table 8 Estimated impact of the MMC on science credits taken and passed, no transition cohorts Change in credits Pre-MMC Mean Student Covariates School Covariates School Fixed Effects N 175771 Note. Coefficients represent the change in number of science credits taken (columns 1-3) and passed (columns 4-6) during high school. Columns 1 and 4 include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), columns 2 and 5 also include school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and columns 3 and 6 also include school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 176276 175771 175771 175771 175771 X (6) X X X X X Although there were no changes in the total number of science credits that students took or passed, it is possible that the MMC policy led to shifts in which science courses students took. For example, schools may have shifted students away from a general physical science or 51 integrated science course toward chemistry and/or physics courses to meet MMC guidelines. There is some evidence that this may be the case, as shown in Chapter 3. Table 9 shows the results of analyses similar to those presented in Table 7, with the outcome changed to particular courses mandated in the MMC policy: biology, chemistry, and physics. Panel A of Table 9 shows a decrease in the number of credits students took in biology, with column 1 showing an estimate of .084 fewer credits taken in the model with student characteristics only (p < .05), .076 (p < .05) fewer biology credits in the model with school characteristics, and .082 fewer credits taken in the model with school fixed effects (p < .05). Prior to the MMC policy, the average student in the sample took about 1.15 biology credits, so it appears that in some cases students who were taking multiple biology credits may have shifted some of those extra credits to other types of science courses. Panel B of Table 9 shows changes in chemistry course-taking and passage. Column 1, which includes student covariates only, shows an increase of .152 (p < .001) credits taken in chemistry and column 4 shows a change of .112 (p < .001) credits passed in chemistry. Columns 2 and 5, which include school covariates, tell a similar story with an increase of .157 (p < .001) credits taken and .118 (p < .001) credits passed. Columns 3 and 6, which include school fixed effects and represent within-school changes in chemistry credits taken and passed, show similar results as well. Students took .142 (p < .001) more chemistry credits and passed .103 (p < .001) more chemistry credits after the MMC policy went into effect. These changes are larger in magnitude than the observed decreases in biology credits taken and, when compared to the pre- MMC average of about .58 chemistry credits taken by the average student represent a substantial change in the amount of chemistry students took during high school after the MMC policy was enacted. 52 (1) (3) (4) Taken (2) Passed (5) (0.021) 0.515 (0.020) 0.541 -0.052 (0.036) 0.997 -0.084* (0.035) 1.147 -0.076* (0.037) 1.149 -0.082* (0.034) 1.149 0.152*** 0.157*** 0.142*** (0.023) (0.023) 0.580 0.581 (0.024) 0.581 -0.060+ (0.033) 0.997 -0.061+ (0.034) 0.995 0.112*** 0.118*** 0.103*** (0.020) 0.515 0.066*** 0.068*** 0.066*** (0.014) 0.220 Table 9 Estimated impact of the MMC on biology, chemistry, and physics credits taken and passed Panel A: Biology Change in credits Pre-MMC Mean Panel B: Chemistry Change in credits Pre-MMC Mean Panel C: Physics Change in credits Pre-MMC Mean Student Covariates School Covariates School Fixed Effects N 251241 Note. Coefficients represent the change in number of science credits taken (columns 1-3) and passed (columns 4-6) during high school. Columns 1 and 4 include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), columns 2 and 5 also include school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and columns 3 and 6 also include school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.072*** 0.074*** 0.071*** (0.014) (0.014) 0.238 0.238 251825 251241 251241 251825 251241 (0.014) 0.220 (0.014) 0.220 (0.015) 0.238 X X X X X X X X X X X X (6) In physics, I find smaller but still statistically significant increases in the number of credits students took and passed. Columns 1 and 4 show results from models including student covariates only and indicate increases of .072 (p < .001) physics credits taken and .066 (p < .001) physics credits passed. Columns 2 and 5, which include school covariates show similar results, with students taking .074 (p < .001) more credits and passing .068 (p < .001) more credits. 53 Columns 3 and 6 show that within-school changes in the number of credits students took and passed in physics were .071 (p < .001) and .066 (p < .001) respectively. The observed increases in physics credits taken and passed are smaller in magnitude than the observed increases in chemistry. Since the MMC policy only required one year of either course, but not necessarily both, it appears than many schools and students opted to take chemistry to satisfy the requirement rather than physics, although more students were taking physics post-MMC. Overall, Table 9 indicates changes in the number of credits taken and passed in chemistry and physics which, when taken together, are similar in magnitude to changes reported by Kim and colleagues (2019) on math credits. They found an increase of .063 to .081 math credits taken and .030 to .072 math credits passed per year (multiplied by four years, these changes on the low end are about the same as the shifts I observe in physics), depending on model specification. The difference in science is that rather than an increase in the number of science credits taken, there was a shift in the types of science courses that students enrolled in. To further investigate these shifts, I performed analyses similar to those found in Table 9 for other types of science courses that were not mentioned in the MMC policy. The results of these analyses are found in Table 10 and provide little evidence that any one science discipline was the source of the increased number of chemistry and physics credits, beyond the decrease observed in Table 9 for biology. The coefficients do not consistently reach statistical significance, meaning we cannot feel confident that the negative point estimates are different from zero. The lack of uniformity in school course sequences in science (as shown in Table 6) provides an explanation for why there was a lack of statistically significant results for where shift in courses occurred; within a particular school there may have been a clear shift away from, for example, Earth Science into 54 Chemistry, but across schools there would be much less consistency in what kinds of shifts occurred given the differences in what students took before the MMC. (1) (6) (3) (4) Passed (5) -0.025 (0.031) 0.311 -0.024 (0.034) 0.312 -0.027 (0.035) 0.312 -0.048 (0.043) 0.366 -0.006 (0.006) 0.029 -0.051 (0.045) 0.366 -0.009 (0.007) 0.029 Taken (2) -0.065 (0.044) 0.417 -0.007 (0.007) 0.035 -0.068 (0.046) 0.417 -0.009 (0.007) 0.035 -0.086* (0.042) 0.416 -0.008 (0.007) 0.035 -0.069+ (0.041) 0.365 -0.008 (0.007) 0.029 251825 251241 251241 Table 10 Estimated impact of the MMC on non-MMC science courses taken and passed Panel A: Earth Science Change in credits Pre-MMC Mean Panel B: Environmental Science Change in credits Pre-MMC Mean Panel C: Integrated Science Change in credits Pre-MMC Mean Panel C: Physical Science Change in credits Pre-MMC Mean Student Covariates School Covariates School Fixed Effects N Note. Coefficients represent the change in number of science credits taken (columns 1-3) and passed (columns 4-6) during high school. Columns 1 and 4 include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), columns 2 and 5 also include school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and columns 3 and 6 also include school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 -0.009 (0.031) 0.217 -0.017 (0.033) 0.216 -0.012 (0.031) 0.189 -0.010 (0.033) 0.217 -0.005 (0.030) 0.189 -0.026 (0.029) 0.248 -0.024 (0.032) 0.248 -0.024 (0.034) 0.248 -0.005 (0.031) 0.189 251825 251241 251241 X X X X X X X X X X X X 55 Table 11 shows the results of models that test whether there was a difference in the number of students meeting MMC science requirements before and after the policy was enacted. The models in Columns 1 through 3 show that there was a six to seven percentage point increase in the proportion of students who took at least three science credits with at least one being biology and at least one being either chemistry or physics. All specifications were statistically significant with p < .001. This is a substantial increase given that prior to the MMC, only 33% of students took all the science courses needed to meet the MMC requirements. However, less than 40% of high school students in the state took the MMC requirements after the policy went into effect, which means that the policy did not have the effect that was intended when it mandated science course requirements meant to apply to all students. Table 11 also shows that there was a five to six percentage point increase in the proportion of students passing the MMC science course requirements. Prior to the MMC, 29% of students passed the MMC science requirements, so as with the proportion of students taking the MMC science courses, I observe a relatively large increase compared to what was happening in the state before the policy but also a very large proportion of students who were not passing the mandated course requirements. Overall, the results of the MMC on science course-taking show shifts in the types of courses students took, with more students taking chemistry and physics courses and fewer biology courses. These changes resulted in a six to seven percentage point increase in the proportion of students taking the MMC-required science courses. However, the total number of science credits students took did not change. This is not surprising given that the MMC required courses in math, English, and other academic areas, and there are only so many hours in the day that can be devoted to science. 56 X X X X X X Met (5) X X (1) X (6) X X X (3) (4) Taken (2) (0.014) 0.290 0.058*** 0.066*** 0.060*** (0.014) (0.015) 0.331 0.332 (0.017) 0.332 (0.016) 0.291 0.049*** 0.056*** 0.052*** (0.013) 0.291 Table 11 Estimated impact of the MMC on MMC science requirements taken and met Post-MMC Pre-MMC Mean Student Covariates School Covariates School Fixed Effects N 251241 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Coefficients represent the change in proportion of students taking (columns 1-3) and meeting (columns 4-6) all MMC requirements during high school (at least three total science credits, one in biology, and one in chemistry or physics). Columns 1 and 4 include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), columns 2 and 5 also include school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and columns 3 and 6 also include school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 251825 251241 251825 251241 251241 Number of Science Credits Taken and Passed – Difference-in-differences Analysis The previous results show the estimated effect of the MMC policy on all schools, regardless of whether the policy affected the behavior of the schools. Another policy-relevant estimate is the effect the policy has on schools that are actually affected by the MMC science coursework requirements compared to the schools where the MMC policy had little effect (i.e. schools where students did not take three years of science, one year of biology, and one year of chemistry or physics compared to schools where these requirements were already being met by many students). These findings represent “treatment on treated” estimates, showing the effect of the coursework policy if schools were not already meeting the requirements. 57 The following analyses compare the differences in pre- and post-policy values for two groups: a “policy-affected” group which includes schools where a relatively low proportion of students were taking or passing the MMC science requirements prior to the policy and a “policy- unaffected” group where a relatively high number of students were already taking the MMC required science courses. Although both groups were subject to the MMC requirements starting with the 2008 cohort of freshmen, we would expect that the policy would have a differential impact on the groups since their pre-policy course-taking behaviors were different. The “policy- affected” group would need to change its behavior more than the “policy-unaffected” group in order to meet the MMC requirements once they were implemented. Table 12 provides descriptive statistics for the low (policy-affected) quintile and high (policy-unaffected) quintile groups. These statistics are also broken down by pre- and post- policy. Comparing the two groups, the policy-affected schools had higher proportions of racial minority students (52.2% white versus 83.4% white) and a higher proportion of students who were ever economically disadvantaged (52.0% versus 21.3%). They also took more than one science credit fewer and only half as much physics as the policy-unaffected group. They also took almost half a credit less in biology and more than a quarter of a credit less in chemistry. The credit disparities make sense given that the groups were formed based on proportion of students taking MMC science requirements and credits are the basis for that measure. In the post-MMC period, we see that the policy-affected group took more science credits in total as well as in biology, chemistry, and physics than they did in the pre- period. The policy- unaffected group took fewer biology credits but more chemistry and physics credits. In all cases, the policy-unaffected group still took more credits than the policy-affected group in the post- MMC time period. 58 Q4 & Q5 Pre-MMC Post-MMC Q1 & Q2 Q4 & Q5 Table 12 Descriptive statistics by "treatment" status, pre- and post-MMC Student Characteristics Female White Black Hispanic Asian Other Race Ever Limited English Proficient Ever Economically Disadvantaged Student Course-taking Total Science Credits Taken Total Biology Credits Taken Total Chemistry Credits Taken Total Physics Credits Taken Total Science Credits Earned Total Biology Credits Earned Total Chemistry Credits Earned Total Physics Credits Earned School Characteristics Enrollment Number of Teachers Per-Pupil Expenditures Student Observations Student-Year Observations School Observations Note. Student characteristics and student course-taking are calculated as the mean with one observation per student. School characteristics are the means of school-year observations formed from student characteristics. Dollar amounts are expressed as real 2010 dollars. Q1 & Q2 49.1% 83.4% 8.8% 2.2% 4.9% 0.7% 3.5% 21.3% 3.55 1.24 0.67 0.30 3.11 1.10 0.61 0.28 1420 70.1 5859 86635 350729 38 50.0% 52.2% 38.8% 5.9% 2.4% 0.7% 4.6% 52.0% 2.24 0.77 0.39 0.15 1.89 0.64 0.34 0.14 1158 57.5 5907 69135 209365 45 50.4% 53.5% 35.1% 7.5% 3.1% 0.8% 5.4% 59.1% 2.73 0.91 0.70 0.30 2.33 0.77 0.59 0.28 989 47.5 5895 25405 77974 38 49.7% 79.7% 11.6% 2.6% 5.4% 0.6% 3.1% 29.9% 3.55 1.19 0.82 0.37 3.11 1.05 0.72 0.35 1365 65.7 6099 33404 131393 35 Table 13 provides difference-in-differences estimates of the MMC policy’s effect on schools that had low pre-policy levels of students taking the MMC science course requirements compared to students in schools that had relatively high levels of pre-policy MMC science 59 course adherence. The group that was low on initial MMC science course-taking is the “treated” group because the MMC policy should change their behavior. The high MMC science course- taking group is the comparison group and includes schools that the policy would have affected less because students there were already taking the MMC-required courses. Schools were divided into quintiles based on the 2003 proportion of students taking (Columns 1 and 2 of Table 13) and passing (Columns 3 and 4 of Table 13) MMC science requirements. The bottom quintiles represent the policy-affected group and the top quintiles represent the (relatively) policy-unaffected comparison group. Panel A of Table 13 shows comparisons of quintiles one and two compared to quintiles four and five. The middle quintile, quintile three, is not included in this analysis. The coefficient of interest if the interaction between the MMC and Low group indicator, which represents the difference in change in science credits earned (Column 1) and passed (Column 2) between the two groups. We see in Column 1 that the policy-affected group increased the number of science credits taken by .466 (p < .001) more than the policy-unaffected group where many students were already taking the MMC-required three science credits. This indicates that after the implementation of the MMC policy, the schools it meant to target did increase the number of science credits students took relative to the policy-unaffected schools. A similar pattern is seen in other columns of Panel A which show that the policy-affected group took and passed more science credits that the comparison group in the other specifications. 60 Table 13 Difference-in-differences estimates of the MMC on science credits taken and passed Quintiles based on Taking MMC (1) (2) Taken Passed Quintiles based on Passing MMC (3) (4) Taken Passed 1.769 193942 2.382 184640 2.035 184640 2.565 193942 -0.267* (0.108) -1.536*** (0.277) 0.504*** (.0133) -0.171 (0.111) -0.899*** (0.206) 0.441*** (0.128) -0.205+ (0.119) -1.116*** (0.200) 0.466*** (0.145) -0.254* (0.109) -0.821*** (0.222) 0.554*** (0.144) Panel A: Quintiles 1 and 2 vs 4 and 5 MMC Low MMC * Low Pre-MMC Low Group Mean N Panel B: Quintile 1 vs 5 MMC Low MMC * Low 1.701 Pre-MMC Low Group Mean N 97962 Note. Coefficients represent the number of science credits taken (columns 1 and 3) and passed (columns 2 and 4) during high school. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score) and school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 -0.189 (0.120) -1.444** (0.456) 0.780** (0.281) -0.166 (0.142) -1.733** (0.500) 0.650* (0.269) -0.136 (0.115) -1.536** (0.487) 0.695** (0.242) -0.097 (0.122) -1.107* (0.513) 0.570* (0.235) 1.932 96838 2.299 96838 2.274 97962 61 The coefficient labeled MMC in Panel A of Table 13 represents the difference in science course credits taken and earned for the policy-unaffected group. The coefficients are negative and in some cases statistically significant, providing evidence that the increase in science credits in the policy-affected group was offset by a decrease in the number of credits taken and earned by the policy-unaffected group. This may have been in response to the other requirements of the MMC which left less time in the school day for students to take extra science classes beyond the ones required by the policy. Panel B of Table 13 provides similar estimates to Panel A, except Panel B compares only the first and fifth quintiles, leaving out the three middle quintiles. The interaction between MMC and Low shows a similar finding to the one observed in Panel A, although the magnitude of the coefficients is larger. This shows that the lowest MMC-compliant schools before the policy changed the number of credits students took and earned: .650 (p <.001) and .780 (p < .001) for credits taken and .570 (p < .001) and .695 (p < .001) for credits earned more than the most pre- policy compliant schools. Table 14 shows the results of the difference-in-differences estimates on the number of biology, chemistry, and physics credits students took and earned. Panel A, which shows the biology results, shows that after the MMC policy went into effect, the schools where students took and passed MMC requirements at high rates before the implementation of the policy decreased the number of biology credits students took by .185 credits (p <.001) as well as the number of biology credits students subsequently passed. This decrease was offset in the policy- affected group, who relative to the policy-unaffected group saw an increase of .200 (p < .01) to .229 (p < .001) credits taken, depending on the construction of the groups for comparison. This 62 finding makes sense as biology was the requirement the students in policy-affected schools were most likely to have already been meeting. Panel B shows results for Chemistry. In this case, the policy-unaffected group showed small increases in the number of chemistry credits taken (.066 [p < .10] to .087 [p < .05] credits) and the policy-affected group showed additional increases of .166 (p < .01) credits. Thus, both groups increased the number of chemistry credits taken and earned, although the comparison group increased more than the policy-unaffected comparison group. Table 14 Difference-in-differences estimates of the MMC on biology, chemistry, and physics credits taken and passed Panel A: Biology MMC Low MMC * Low Pre-MMC Low Group Mean Panel B: Chemistry MMC Low MMC * Low Pre-MMC Low Group Mean 0.066+ (0.038) -0.183** (0.059) 0.166** (0.061) 0.431 0.043 (0.032) -0.152** (0.058) 0.142** (0.049) 0.378 63 Quintiles based on Taking MMC (1) Taken (2) Passed Quintiles based on Passing MMC (3) (4) Taken Passed -0.185*** (0.051) -0.472*** (0.089) 0.200** (0.059) -0.148** (0.049) -0.399*** (0.089) 0.191*** (0.055) -0.186*** (0.050) -0.417*** (0.086) 0.229*** (0.059) -0.161** (0.047) -0.636*** (0.102) 0.213*** (0.055) 0.822 0.696 0.858 0.593 0.087* (0.040) -0.101+ (0.054) 0.150* (0.063) 0.482 0.062+ (0.035) -0.263*** (0.068) 0.095+ (0.053) 0.339 0.057* (0.022) -0.037 (0.038) 0.057 (0.038) 0.058* (0.024) -0.042 (0.042) 0.077+ (0.040) 0.054* (0.023) -0.026 (0.042) 0.073+ (0.039) 0.048* (0.022) -0.092* (0.043) 0.054 (0.038) Table 14 (cont’d) Panel C: Physics MMC Low MMC * Low Pre-MMC Low Group Mean N Note. Coefficients represent the number of biology, chemistry, and physics credits taken (columns 1 and 3) and passed (columns 2 and 4) during high school. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score) and school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 193942 184640 184640 0.166 0.157 0.173 0.132 193942 Panel C shows the same analysis for physics. The policy-unaffected group shows a post- MMC increase of .058 (p < .05) physics credits taken and .054 (p < .05) physics credits passed. Meanwhile, the policy-affected group shows less robust results. The coefficients of the interaction between MMC and the Low group indicator do not reach statistical significance and are smaller in magnitude than the coefficients for chemistry. It appears that in schools where students were meeting MMC science requirements before the MMC policy was enacted, there was an increase in physics course-taking but that there was not an additional increase beyond that for the policy-affected group. This may signify that in the policy-affected schools chemistry was a more popular option than physics for meeting the MMC chemistry/physics requirement, whereas in policy-unaffected school similar increases were seen in both chemistry and physics. 64 Table 15 repeats the analyses from Table 14 but uses quintile one as the policy-affected group and quintile five as the policy-unaffected group, leaving out quintile three as in Table 14 but also quintiles two and four. Thus, the comparison is between the schools that had the lowest level of students meeting the MMC requirements pre-MMC and the schools that had the highest level of pre-policy compliance. Compared to Table 14, we see that the decrease the policy- unaffected group sees in biology courses taken and passed, represented by the coefficient MMC, is smaller but still statistically significant. Students in the policy-unaffected group took .145 (p < .05) fewer biology credits after the MMC policy. Relative to the policy-unaffected group, the policy-affected group in this case saw a large increase of .284 (p < .01) biology credits taken. A similar pattern is observed for credits passed and for the comparison using proportion of students passing the MMC requirements to create the policy-affected and unaffected groups. Table 15 Difference-in-differences estimates of the MMC on biology, chemistry, and physics credits taken and passed - Quintile 1 versus Quintile 5 Panel A: Biology MMC Low MMC * Low Pre-MMC Low Group Mean Quintiles based on Taking MMC Quintiles based on Passing MMC (1) Taken (2) Passed -0.145* (0.055) -0.628* (0.247) 0.284** (0.098) 0.784 -0.102* (0.050) -0.414 (0.248) 0.267** (0.093) 0.660 (3) Taken (4) Passed -0.136** (0.047) -0.539* (0.225) 0.294** (0.108) 0.807 -0.103* (0.044) -0.567* (0.228) 0.298** (0.101) 0.600 65 0.426 0.436 0.369 0.311 0.073 (0.050) -0.327* (0.138) 0.214+ (0.109) 0.057 (0.041) -0.225+ (0.132) 0.178+ (0.092) 0.051 (0.038) -0.402** (0.124) 0.199* (0.094) 0.058 (0.043) -0.360** (0.125) 0.257* (0.110) Table 15 (cont’d) Panel B: Chemistry MMC Low MMC * Low Pre-MMC Low Group Mean Panel C: Physics MMC Low MMC * Low Pre-MMC Low Group Mean N Note. Coefficients represent the number of biology, chemistry, and physics credits taken (columns 1 and 3) and passed (columns 2 and 4) during high school. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score) and school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.069* (0.032) -0.141+ (0.078) 0.024 (0.052) 0.066* (0.032) -0.140+ (0.078) 0.019 (0.050) 0.074* (0.034) -0.068 (0.088) 0.004 (0.050) 0.072* (0.035) -0.119 (0.090) 0.012 (0.050) 97962 97962 0.172 96838 0.131 0.183 96838 0.151 Taken together, the results for biology in Tables 14 and 15 show that in the schools where the fewest students were taking or passing MMC science requirements in 2003, biology was an area where students increased course-taking after the policy. This may reflect the 66 relatively low number of credits that the pre-policy students in the lowest quintile took (.784). Students in the highest quintiles reduced the amount of biology they took and, as a result, passed. Panel B of Table 15 shows large increases in the amount of chemistry students took and passed, especially in relation to the pre-policy mean for the low group. Relative to the most policy-unaffected students, the most policy-affected students increased .214 (p < .10) to .257 (p < .05) on chemistry credits taken. Panel C shows no increase in physics credits taken or passed for the policy-affected group in relation to the policy-unaffected group, however the policy- unaffected group increased the number of physics credits taken and passed by .072 (p < .05) and .074 (p < .05) respectively. Similar to what was shown in Table 14, students in the policy- unaffected groups were more likely to increase their physics course-taking while students in the policy-affected group saw large relative increases in chemistry course-taking. Table 16 presents difference-in-differences estimates for taking and passing all MMC science requirements. In both Panel A, which shows comparisons between the two lowest and two highest quintiles, and Panel B, which compares the lowest to the highest quintile, the low quintile students have a substantially lower pre-policy level of both taking and meeting the MMC science requirements. This is to be expected given that the quintiles are constructed by proportion of students taking or passing the MMC science requirements in 2003. Students in the policy-unaffected, high quintile groups show modest increases in the proportion of students taking and meeting the MMC science requirements. 67 Quintiles based on Passing MMC (3) (4) Taken Passed Table 16 Difference-in-differences estimates of the MMC on MMC requirements taken and met Quintiles based on Taking MMC (1) (2) Taken Passed 0.157 184640 0.194 193942 0.134 184640 0.050+ (0.026) -0.178*** (0.044) 0.064 (0.043) 0.041+ (0.024) -0.258*** (0.045) 0.049 (0.039) 0.045+ (0.026) -0.219*** (0.045) 0.049 (0.040) 0.054+ (0.027) -0.256*** (0.047) 0.050 (0.044) Panel A: Quintiles 1 and 2 vs. 4 and 5 MMC Low MMC * Low Pre-MMC Low Group Mean N Panel B: Quintile 1 vs 5 MMC Low MMC * Low 0.142 Pre-MMC Low Group Mean N 97962 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score) and school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. 0.075* (0.025) -0.271** (0.085) 0.048 (0.065) 0.068* (0.028) -0.220** (0.069) 0.062 (0.060) 0.076** (0.025) -0.264** (0.083) 0.046 (0.058) 0.064* (0.027) -0.154* (0.064) 0.050 (0.055) 0.120 193942 0.162 96838 0.178 97962 0.180 96838 + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 68 Coefficients on the MMC indicator show a five to seven percentage point increase after the MMC policy took effect. There is no evidence of a differential effect for the lower quintile, policy-affected group. This may be due to the sizable organizational task of increasing from a pre-policy average of 15.7% of students taking the MMC science coursework. Even though steps like the observed increase in biology, chemistry, and physics coursework may push some students over the MMC minimum requirements, the barriers to getting large proportions of students in some schools may have been too much, especially in a short timeframe. Table 17 shows sensitivity analyses for all outcomes tested in the difference-in- differences analyses. In this table, the transition cohorts (2006 and 2007) have been removed to compare only cohorts from before the policy was announced to those that were fully affected by the MMC science coursework policy. In general, the MMC by Low interaction shows similar or slightly larger point estimates to the similar analyses from Tables 13, 14, and 16. This may indicate that some schools in the policy-affected group did react to the announcement of the MMC science coursework policy by changing coursework, but that overall Table 17 tells a similar story to the analyses that included the transition cohorts. In sum, the difference-in-differences analyses show that the policy had a differential effect on the science coursework that students in the policy-affected and unaffected schools took and passed. Students in the policy-affected group took more science credits, especially in chemistry, compared to the policy-unaffected group where students took less biology and saw smaller gains in the chemistry and physics coursework compared to the policy-affected group. It is clear that the policy-unaffected group was affected by the coursework changes that the MMC instituted, although to a different degree and in different disciplines than the policy-affected group. In terms of the relevance of these changes for policymakers, despite the observed changes 69 in coursework, the proportion of students who took and met MMC science requirements after the policy went into effect was still quite low considering the policy was meant to have all students in the state take the required courses. Table 17 Difference-in-differences estimates of the MMC on science outcomes, no transition cohorts Quintiles based on Quintiles based on Panel A: Science Credits MMC Low MMC * Low Pre-MMC Low Group Mean Panel B: Biology Credits MMC Low MMC * Low Pre-MMC Low Group Mean Panel C: Chemistry Credits MMC Low MMC * Low Pre-MMC Low Group Mean 70 Taking MMC (1) (2) Taken Passed Passing MMC (3) (4) Taken Passed -0.3 (0.200) -1.316*** (0.198) 0.644*** (0.178) -0.237 (0.199) -1.076*** (0.207) 0.603*** (0.159) -0.327 (0.202) -0.966*** (0.235) 0.705*** (0.176) -0.347+ (0.207) -1.705*** (0.276) 0.676*** (0.165) 2.224 1.903 2.413 1.630 -0.273*** (0.075) -0.579*** (0.096) 0.289*** (0.073) -0.217** (0.073) -0.496*** (0.095) 0.273*** (0.067) -0.282*** (0.079) -0.521*** (0.094) 0.325*** (0.071) -0.238** (0.075) -0.734*** (0.102) 0.305*** (0.066) 0.743 0.631 0.771 0.521 0.039 (0.063) -0.248*** (0.060) 0.216** (0.069) 0.385 0.029 (0.061) -0.204*** (0.058) 0.184** (0.058) 0.340 0.090 (0.068) -0.147* (0.060) 0.182* (0.071) 0.440 0.061 (0.066) -0.310*** (0.068) 0.129* (0.062) 0.303 0.154 0.114 0.142 0.136 0.089* (0.034) -0.111* (0.044) 0.074+ (0.043) 0.105** (0.034) -0.052 (0.039) 0.076+ (0.043) 0.103** (0.037) -0.068 (0.044) 0.105* (0.046) 0.100** (0.036) -0.050 (0.044) 0.097* (0.045) Table 17 (cont’d) Panel D: Physics Credits MMC Low MMC * Low Pre-MMC Low Group Mean Panel E: MMC Requirements MMC Low MMC * Low Pre-MMC Low Group Mean N 135804 Note. Coefficients in Panels A-D represent credits taken (columns 1 and 3) or passed (columns 2 and 4). Coefficients in Panel E are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score) and school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.034 (0.046) -0.234*** (0.044) 0.112* (0.053) 0.026 (0.047) -0.322*** (0.041) 0.101+ (0.054) 0.026 (0.044) -0.279*** (0.040) 0.098+ (0.050) 0.029 (0.044) -0.310*** (0.042) 0.095+ (0.049) 135804 129521 129521 0.087 0.120 0.101 0.154 Postsecondary STEM Outcomes – Pre-post Regression Analysis Part of the motivation for science coursework policies like the MMC is to better prepare students for postsecondary schooling, especially with regard to participation in STEM fields. The 71 previous analyses I have presented showed that there were shifts in the types of science courses students took, and in some schools students took substantially more science credits than they did pre-policy. The question is do these changes in high school course-taking translate into changes in postsecondary behavior. In this section I test whether the MMC had an effect on a variety of STEM-related postsecondary outcomes that policymakers may hope the policy could impact. These include enrollment in college, declaring a STEM major, and graduating with a STEM degree. In Table 18 I show the results of analyses investigating the proportion of students in post- MMC cohorts who declared a STEM major. Panel A shows changes in the proportion of students who ever declared a STEM major (within 6 years of their expected high school graduation date). The coefficient of .010 (p < .01) indicates that there was a one percentage point increase in students declaring a STEM major after the MMC policy. This is a fairly large increase given that the pre-MMC sample mean for ever declaring a STEM major was 16.2% of students. There were statistically significant results for ever declaring a biology (.5 percentage points [p < .05]) and an engineering major (.7 percentage points [p < .001]). These changes, although small in absolute magnitude are large in relation to the pre-policy means of 4.9% and 3.8% for ever declaring a biology and engineering major respectively. 72 Table 18 Estimated impact of the MMC on postsecondary STEM major selection (2) (1) (4) (3) Physical Science 0.001 (0.001) 0.018 251241 Biology 0.005* (0.002) 0.049 251241 STEM 0.010** (0.004) 0.162 251241 Engineering 0.007*** (0.002) 0.038 251241 0.001 (0.002) 0.027 171049 0.006* (0.003) 0.074 171049 0.015** (0.005) 0.244 171049 0.011*** (0.002) 0.057 171049 Ever Major Pre-MMC Mean N Ever Major, Conditional on Enrollment Pre-MMC Mean N First Year Major Pre-MMC Mean N First Year Major, Conditional on 0.010*** Enrollment (0.002) 0.040 Pre-MMC Mean N 171049 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable and local unemployment by year. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.007*** (0.002) 0.026 251241 0.013** (0.005) 0.145 171049 0.001 (0.002) 0.026 251241 0.002 (0.002) 0.039 171049 0.002+ (0.001) 0.011 171049 0.008* (0.004) 0.097 251241 0.002+ (0.001) 0.008 251241 The ever majoring in STEM estimates presented in Table 18 are for the entire sample of students, regardless of whether they ever enrolled in college and had a chance to choose a major 73 at all. Given the observed changes in college enrollment observed by Kim et al. (2019), it may be that the findings for STEM majors in Panel A of Table 18 are driven by the increased number of students enrolling in college. Thus, I also test the effect of the MMC on ever declaring a STEM major conditional on enrolling in college. In Panel B, I show that conditional on enrolling in college, there was still a 1.5 percentage point increase in STEM majors (p < .01), a .6 percentage point increase in biology majors (p < .05), and a 1.1 percentage point increase in engineering majors (p < .001). So, it was not just the case that the absolute number of student majoring in STEM increased due to more people entering college. Instead, as a proportion of people enrolling and choosing a major, there were still increases in the prevalence of STEM majors. Panels C and D repeat the analyses from panels A and B, except the outcome is declaring a STEM major in the first year of enrollment and declaring a STEM major in the first year conditional on enrolling in college. For STEM and for engineering the pattern is similar to the ever declaring analyses. There was a .8 percentage point increase in the proportion of people majoring in STEM during their first year (p < .05) and a .7 percentage point increase in people declaring an engineering major in their first year (p < .001). Likewise, in the first year, conditional on enrolling at all, there was a 1.3 percentage point increase in STEM majors and a 1.0 percentage point increase in engineering majors. In terms of graduation outcomes, Table 19 shows that after the MMC students in the state completed .2 percentage points (p < .05) more physical science degrees (Bachelor’s, Associate, and Certificate Programs combined) within six years of expected high school completion. When Bachelor’s degrees are the outcome, there is a statistically significant increase of .6 percentage points (p < .05) in the number of STEM degrees earned within six years, .2 percentage points (p < .05) for physical science degrees, and .4 percentage points (p < .05) for engineering degrees. 74 These estimates, while small, are meaningful given the low number of bachelor’s degrees earned in each discipline. Table 19 also shows that students were more likely to earn a bachelor’s degree within six years in a STEM field. There was a .6 percentage point increase (p < .05) in the proportion of people in the sample who earned a STEM degree. This is meaningful given the pre-MMC sample mean of 5.3%. In physical science, there was a .2 percentage point increase (p < .05), and in engineering there was a .4 percentage point increase (p < .05). Both of these values are also meaningful given the small numbers of physical science and engineering bachelor’s degrees awarded. (4) (2) 0.003 (0.002) 0.030 0.002* (0.001) 0.009 (1) STEM 0.005+ (0.003) 0.056 Biology Physical Science Engineering 0.001 (0.001) 0.018 Table 19 Estimated impact of the MMC on college graduation and STEM degrees (3) Graduated - Any Degree or Certificate Pre-MMC Mean 0.004* Graduated - BA within 6 Years (0.002) 0.028 Pre-MMC Mean N 187020 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable and local unemployment by year. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.006* (0.003) 0.053 187020 0.002* (0.001) 0.008 187020 0.001 (0.001) 0.018 187020 75 These “intent to treat” estimates for the MMC show that more students in the state majored in STEM fields and graduated with STEM degrees than expected given trends in the state before the MMC policy was enacted. It is difficult to attribute those changes to science coursework, especially given all of the other requirements, including math, that the MMC instituted. There are also other factors, beyond the MMC, that may have occurred at the same time that might affect these results. However, the within-school comparisons provided by the fixed effects models in Tables 18 and 19 paint a picture of more students majoring and graduating in STEM compared to before the MMC. Postsecondary STEM Outcomes – Difference-in-differences Analysis The results in Table 20 show the difference-in-differences analysis for postsecondary STEM major outcomes. When comparing the bottom quintiles in 2003 proportion of students taking MMC science requirements to the top quintiles, differential effects on the policy-affected group show that the MMC may not have had the intended effect. For the ever majoring in STEM outcome, the MMC coefficient indicates that the policy-unaffected group saw consistent increases in the STEM (1.9 percentage points [p < .001]) and engineering majors (1.4 percentage points [p < .001]). Both coefficients, especially the engineering coefficient, are sizable in comparison the pre-MMC means. There was no detectable effect for the policy-unaffected group on the biology or physical science majors. The increase in STEM and engineering majors in the policy-unaffected group was offset in the policy-affected group. For ever declaring a STEM major, the policy-affected group saw a decrease of 2.4 percentage points (p < .001), a decrease of 1.2 percentage points in biology (p < .001), and a decrease of 1.1 percentage points in engineering (p < .001) compared to what the policy-unaffected group experienced. Despite the changes in high school course-taking for the 76 policy-affected group compared to the policy-unaffected group, they did not translate into positive differential effects for the policy-affected group. To the contrary, the policy-unaffected group saw gains in the proportion of students majoring in STEM and engineering. Results for ever majoring in STEM fields conditional on enrollment in college, presented in Panel B of Table 20, are similar in magnitude and pattern to the results in Panel A of Table 20. Likewise, Panels C and D of Table 20, which show first year college majors, paint a similar picture. The coefficients for STEM and engineering are positive and significant for the policy-unaffected students, and those gains are offset in the policy-affected group, as evidenced by the negative significant coefficients for STEM, biology, and engineering. 77 Table 20 Difference-in-differences estimates of the MMC on postsecondary STEM major selection (2) (3) (1) (4) STEM Biology 0.019*** (0.005) 0.008 (0.007) -0.024*** (0.007) 0.005 (0.003) -0.007 (0.004) -0.012*** (0.003) 0.172 184640 0.052 184640 0.019 184640 0.042 184640 Panel A: Ever Major MMC Low MMC * Low Pre-MMC Mean N Panel B: Ever Major, Conditional on Enrollment MMC Low MMC * Low Pre-MMC Mean N Panel C: First Year Major MMC Low MMC * Low Pre-MMC Mean N 0.024*** (0.006) 0.012 (0.007) -0.023*** (0.007) 0.250 129719 0.016** (0.005) 0.000 (0.004) -0.016* (0.006) 0.103 184640 78 Physical Science 0.000 (0.002) 0.005* (0.002) 0.002 (0.003) 0.000 (0.002) 0.007* (0.004) 0.004 (0.004) 0.028 129719 0.000 (0.001) 0.002+ (0.001) 0.002 (0.003) 0.008 184640 Engineer 0.014*** (0.002) 0.002 (0.003) -0.011*** (0.003) 0.018*** (0.003) 0.002 (0.004) -0.011** (0.004) 0.061 129719 0.012*** (0.002) 0.002 (0.002) -0.009*** (0.002) 0.029 184640 0.005 (0.003) -0.010+ (0.006) -0.013*** (0.005) 0.076 129719 0.002 (0.002) -0.004+ (0.002) -0.008** (0.003) 0.028 184640 0.003 (0.003) -0.006+ (0.004) -0.009* (0.004) 0.021** (0.006) 0.000 (0.006) -0.017+ (0.009) 0.016*** (0.002) 0.002 (0.003) -0.010** (0.003) Table 20 (cont’d) Panel D: First Year Major, Conditional on Enrollment MMC Low MMC * Low 0.042 Pre-MMC Mean N 129719 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable and local unemployment by year. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.001 (0.002) 0.003+ (0.002) 0.004 (0.004) 0.012 129719 0.041 129719 0.150 129719 Table 21 shows similar results for graduation within six years of expected high school completion as the results for declaring majors. Students in the policy-unaffected group showed gains of 1.1 percentage points (p < .01) in STEM degrees (Bachelor’s, Associate, and Certificate programs combined), .3 percentage points in physical science (p < .05), and .8 percentage points in engineering (p < .001). Like majors, however, the interaction term which signifies the differential effect between the policy-affected and policy-unaffected group shows those gains were not sustained in the policy-affected group. These students were 1 percentage point (p < .01) less likely to get a STEM degree and .6 percentage points less likely to get an engineering degree (p < .01) relative to the policy-unaffected group. The same pattern is present for Bachelor’s degrees earned within six years of expected high school graduation as well. 79 Physical Science Table 21 Difference-in-differences estimates of the MMC on college graduation with STEM degrees (4) (1) (2) (3) STEM Biology Engineer 0.001 (0.002) -0.002 (0.002) -0.003 (0.002) 0.021 0.003* (0.001) 0.001 (0.002) -0.001 (0.001) 0.010 0.011** (0.004) 0.002 (0.006) -0.010** (0.003) 0.063 0.008*** (0.002) 0.003 (0.003) -0.006** (0.002) 0.034 Panel A: Graduated - Any Degree or Certificate MMC Low MMC * Low Pre-MMC Low Group Mean Panel B: Graduated - BA within 6 Years MMC Low MMC * Low Pre-MMC Low Group Mean N 135903 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable and local unemployment by year. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.009*** (0.002) 0.003 (0.003) -0.006** (0.002) 0.031 0.012** (0.004) 0.002 (0.006) -0.010** (0.003) 0.060 0.003* (0.001) 0.002 (0.002) -0.001 (0.001) 0.009 0.001 (0.002) -0.002 (0.002) -0.003 (0.002) 0.020 135903 135903 135903 These results show that despite the effects of the MMC policy on the low pre-MMC compliant group in terms of course-taking, the students in schools that were already doing well at getting students to take MMC science requirements were the ones who benefitted from the policy 80 in terms of changes in proportion of students declaring STEM majors. A similar pattern emerges in the graduation with STEM degree results. Students from the policy-unaffected schools increased in the proportion earning STEM, physical science, and engineering degrees. The negative and significant coefficients on the MMC * Low interaction term for STEM majors and engineering majors represent the differential impact of the policy on the policy-affected group which wipe out those gains. Discussion Although, across the state, the intent-to-treat analyses found no evidence of a net gain in science credits taken and passed by students, there were shifts in the types of science courses students did take. Students took fewer biology credits, and substantially more chemistry and physics credits. Despite these changes, however, over sixty percent of students did not take the MMC required science credits after the policy went into effect, indicating that despite some changes in the courses students took there were still many schools that struggled to get students into the required courses or who failed to comply with the mandate despite having the means to do so. The treatment on treated estimates, which used difference-in-differences to compare schools that had low levels of students meeting the eventual MMC science requirements before the policy went into effect to those schools that had high levels of pre-policy compliance, showed that schools in the low group increased the number of science credits students earned while the high group reduced the number they earned. This may be the case of the high schools tightening course sequences to meet the published guidelines by doing away with extraneous science credits that did not meet the MMC standards. There is some evidence that this is the case in the findings for Research Question 1 where there were shifts away from earth science, 81 integrated science, and biology. These estimates also showed that these reductions in course came in biology for the high group while the low group increased biology credits taken and substantially increased the chemistry credits taken. Meanwhile, the high schools increased the number of physics credits students took, with little evidence of a differential effect of in physics credits taken for the low group. In terms of actually meeting the MMC science requirements, the high group increased the proportion of students doing so, and there was little evidence of a differential effect for the low group. So, despite greater changes in the number and types of courses students in the low group took after the MMC went into effect, they did not raise a substantially larger proportion of students to meeting the MMC requirements compared to the high group. This may reflect the enormity of the task in the low group compared to the high group. In schools where many students were already meeting the MMC requirements, getting more students to do so may have been relatively easy given that the necessary courses were already offered and being taken by many students. On the contrary, in the schools where few students were meeting MMC requirements before the policy, the courses required may not have been offered at all, there may have been less qualified staff in place to teach many more students in such courses, students may not have been prepared in middle school for a course sequence that would lead to chemistry or physics, or mandated changes in other academic subjects may have taken priority over meeting MMC science requirements. The intent-to-treat estimates for STEM majors in Table 20 show that after the MMC more students chose to major in STEM and especially engineering. This translated into more students earning degrees in STEM and engineering as well. The treatment-on-treated estimates show that for students in schools where high proportions of students were meeting the MMC 82 requirements before the policy was enacted, the proportion of students going on to major in STEM and engineering increased. The differential effect on declaring a STEM or engineering major was negative for the low pre-policy group, meaning that while the “policy-unaffected” group was increasing the number of STEM majors and graduates, the “policy-affected” group stayed roughly the same in terms of the proportion of STEM majors and graduates. In the case of biology, the low pre-policy compliance group actually dropped in the proportion of students studying biology compared to the high compliance group. Together the results on high school science credits and postsecondary STEM outcomes indicate that although the effect of the policy may have been greater in schools where it was meant to target (i.e. those where students were not taking biology, chemistry, or physics and three total science credits), it did more to improve the postsecondary STEM outcomes for students in the schools where it should have had less of an impact. The mechanism behind this finding is unclear, although it may have to do with the capacity of schools to provide quality instruction for students in courses like chemistry and physics. Teachers in those subjects are often in short supply, so schools where they already worked may have had an advantage over schools that needed to hire science teachers qualified to teach high level science courses (especially during the economic downturn that coincided with the beginning of the MMC). The course sequence changes, reaching back to middle school, may have advantaged the schools that were already sending students to chemistry and physics as well. The MMC science course policy may have exacerbated inequalities in students’ opportunities to learn as the schools best positioned to adapt were able to do so while the schools who should have been most affected saw fewer long-term gains in students’ STEM-related outcomes. 83 CHAPTER 5: HETEROGENEITY OF EFFECTS BY INDIVIDUAL AND SCHOOL CHARACTERISTICS How did the effects of the MMC policy differ based on student characteristics and school characteristics including advanced science coursework pass rate, college attendance rate, rate of student selection of a STEM major, and rate of graduation with a STEM degree? Student-level Heterogeneity in High School Course-taking The findings from Research Question 1, which described the course credits students earned in various science courses showed that there were differences in which courses students had access to or chose to take. Because of these differences, the MMC policy may have affected students differently. For groups that were taking relatively high levels of MMC science courses the policy may not have had as large an impact on course-taking or making small changes in course-taking to meet the MMC standards may have been possible. For students from groups with relatively low levels of MMC science course-taking, the policy may have had a greater impact since the changes in course-taking may have been greater. In this section I test the effect of the MMC policy by group membership to see if there were differential effects based on individual student characteristics. First, I test the interaction of various individual student characteristics and the MMC policy on students’ high school science course-taking behaviors. In Panel A of Table 22 results for female students are presented. The Female row indicates the difference between female and male students pre-policy. Female students took .056 more total science credits (p < .001), .107 (p < .001) more biology credits, and .051 more chemistry credits (p < .001). They also passed more credits in these courses than male students. In physics, however, we see that female students took .053 fewer credits (p < .001) and subsequently passed fewer credits as a result. The mmc row 84 indicates the change in credits taken and passed for male students after the MMC policy went into effect. Male students took and passed more chemistry and physics credits (.151 [p < .001] and .073 [p < .001] respectively) after the MMC policy went into effect. The MMC * Female row shows the differential effect for female students after the policy was enacted. Here we see that the only differential changes were that female students took .036 fewer biology courses (p < .001) compared to post-policy males and .018 (p < .05) fewer chemistry courses. This still comes out to a net increase in the number of chemistry credits females took after the policy, although males gained ground in chemistry course-taking. Notably, there was no differential effect for physics, meaning that although male and female students took more physics classes after the MMC, females did not gain any ground in the pre-existing disparity in physics course-taking. In terms of taking and meeting the MMC science requirements, we look to columns 9 and 10. The coefficients in columns 9 and 10 are percentage point changes in the proportion of students taking and meeting the MMC science requirements. Female students took and met the MMC science requirements 2-3 percentage points more often pre-policy. After the policy, the number of students taking the MMC increased by 6.6 percentage points (p < .001) and meeting the MMC requirements by 5.4 percentage points (p < .001). The only evidence of a differential effect for female students was that they gained .1 fewer percentage points (p < .05) compared to males in terms of taking the MMC and there was no differential change in proportion of students meeting the MMC. 85 Table 22 Heterogeneous impact of the MMC on high School science coursework by student characteristics (2) Science Passed 0.009 (0.065) (3) Biology Taken -0.064+ (0.034) (4) Biology Passed (5) Chem. Taken (6) Chem. Passed (7) Physics Taken (8) Physics Passed (9) MMC Taken (10) MMC Passed -0.045 0.151*** 0.103*** 0.073*** 0.066*** 0.066*** 0.054*** (0.031) (0.013) (0.020) (0.024) (0.013) (0.013) (0.014) 0.056*** 0.130*** 0.107*** 0.129*** 0.051*** 0.057*** (0.013) (0.005) (0.005) (0.009) (0.013) (0.010) 0.053*** (0.008) 0.046*** 0.023*** 0.029*** (0.008) (0.003) (0.003) - - (1) Science Taken Panel A: Female MMC 0.019 (0.068) Female MMC * Female Panel B: Black MMC -0.035+ (0.018) 0.076 (0.085) - -0.012 (0.018) 0.074 (0.083) 0.036*** (0.010) -0.036 (0.036) -0.029** (0.009) -0.018* (0.008) 0.001 (0.007) -0.005 (0.008) -0.000 (0.008) -0.010* (0.005) -0.004 (0.005) -0.016 0.144*** 0.107*** 0.074*** 0.068*** (0.034) (0.014) (0.026) (0.023) (0.015) 0.053** (0.016) 0.046** (0.015) - - Black MMC * Black 0.160 (0.142) 0.076 (0.144) 0.077 (0.058) 0.048 (0.058) -0.002 (0.029) -0.010 (0.029) -0.008 (0.015) -0.011 (0.015) 0.058*** (0.013) 0.063*** (0.013) -0.350+ (0.203) -0.335 (0.203) -0.216* (0.086) -0.209* (0.086) -0.006 (0.038) -0.019 (0.035) -0.014 (0.032) -0.012 (0.032) 0.035 (0.024) 0.029 (0.022) 86 -0.062+ 0.138*** 0.101*** 0.070*** 0.065*** 0.059*** 0.053*** (0.033) (0.013) (0.023) (0.020) (0.014) (0.014) (0.014) -0.070* (0.027) 0.081*** (0.021) -0.052* (0.022) -0.052** (0.016) -0.023* (0.010) -0.023* (0.010) -0.023 (0.018) -0.017 (0.016) 0.054 (0.035) 0.084+ (0.047) 0.039 (0.033) 0.025 (0.023) 0.011 (0.021) 0.026 (0.027) -0.019 (0.021) 0.006 (0.020) (0.066) 0.598*** 0.593*** 0.124*** 0.130*** 0.131*** 0.108*** (0.119) (0.022) -0.058+ 0.148*** 0.107*** 0.073*** 0.068*** 0.064*** 0.055*** (0.033) (0.013) (0.014) 0.083** 0.063*** 0.057*** (0.026) (0.016) (0.014) 0.084** (0.027) (0.029) (0.024) (0.023) (0.112) (0.033) (0.015) (0.017) -0.056 (0.046) -0.050 (0.045) 0.172*** (0.033) 0.139*** (0.032) -0.071** (0.026) -0.063* (0.025) 0.104*** (0.027) -0.082** (0.026) - - - -0.088* (0.035) 0.126* (0.062) -0.080* (0.035) -0.090 (0.055) -0.006 (0.067) 0.169+ (0.091) -0.163** (0.050) 0.095 (0.063) 0.007 (0.063) -0.001 (0.064) Table 22 (cont’d) Panel C: Hispanic MMC Hispanic MMC * Hispanic Panel D: Asian MMC Asian MMC * Asian Panel E: Ever limited English proficient MMC LEP MMC * LEP 0.006 (0.065) 0.037 (0.074) -0.001 (0.068) 0.025 (0.085) 0.058 (0.137) -0.052 (0.120) -0.119 (0.108) -0.154 (0.111) -0.087* (0.035) -0.032 (0.036) 0.131* (0.065) - -0.062+ 0.138*** 0.102*** 0.069*** 0.065*** 0.059*** 0.053*** (0.013) (0.033) -0.008 -0.020 (0.029) (0.015) (0.023) -0.010 (0.020) (0.020) 0.000 (0.019) (0.013) 0.006 (0.015) (0.014) 0.007 (0.017) (0.015) -0.009 (0.017) 0.055 (0.051) 0.100* (0.044) 0.039 (0.032) 0.040 (0.026) 0.022 (0.026) 0.028 (0.023) -0.022 (0.021) 87 - -0.025 (0.020) 0.035 (0.075) -0.054 (0.047) 0.021 (0.077) -0.070* (0.033) -0.143** (0.053) -0.061** (0.021) (0.024) (0.021) (0.016) -0.042 0.101*** 0.081*** 0.066*** 0.064*** (0.032) (0.016) Table 22 (cont’d) Panel F: Ever economically disadvantaged MMC Econ Dis. MMC * Econ Dis.. N Note. Coefficients represent the change in number of science credits taken and passed during high school. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and school fixed effects. All models include a time- trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.101*** (0.013) -0.040 (0.028) 0.093*** (0.019) 0.092*** (0.013) 0.050** (0.017) 0.003 (0.016) 0.045** (0.015) 0.010 (0.016) 251241 251241 251241 251241 -0.044 (0.071) -0.070 (0.071) -0.026 (0.029) 251241 251241 - - - - - 0.039*** (0.007) 0.037*** (0.007) 0.065*** (0.009) 0.064*** (0.009) 0.040* (0.016) 0.040** (0.015) 0.026+ (0.015) 251241 251241 251241 251241 88 Given the widespread concerns about female participation in STEM fields, especially in the physical sciences, it is concerning to see that the MMC did not have a differential effect on female participation in physics. However, it did increase total participation in physics for all students, including females, which is encouraging. Although there were differential negative effects on biology and chemistry course-taking, these may just reflect the fact that females took more biology and chemistry than males pre-policy, and as such had less room for increased participation after the policy. The changes in course credits taken and passed may also reflect streamlining of course sequences to align with MMC requirements, meaning that students (especially females) may have been pushed toward other courses instead of science courses that interested them. For Black students, the only differential effects compared to white students were decreases in biology course-taking and passage compared to white students. Black student took .216 fewer biology courses (p < .05) and as a result passed .209 fewer biology credits (p < .05). These findings may reflect shifts from extra biology courses to other academic courses. For Hispanic students there was a differential increase in biology courses taken of .126 credits (p < .05), but otherwise compared to white students there was no change. Asian students took .139 fewer credits in chemistry (p < .001) and .071 fewer credits in physics (p < .01). These differential effects negated the increases that white students experienced after the MMC, which may reflect high levels of chemistry and physics course-taking pre-policy for Asian students, as evidenced by the positive and significant coefficients in the Asian row of Table 22. Black and Hispanic students did not experience any differential changes in taking or passing on all MMC science requirements compared to white students, but Asian students decreased compared to white students by 10.4 percentage points (p < .001) and 8.2 percentage points (p < .001) 89 respectively. These differential effects negate all gains that students saw after the MMC, meaning that fewer Asian students met MMC science requirements after the policy was enacted. Students who had ever been labeled as Limited English Proficient showed differential gains in biology and chemistry course-taking (.131 [p < .05] and .100 [p < .05] credits respectively). For students who had ever been economically disadvantaged, the only differential change, compared to students who had never been economically disadvantaged, was an increase of .093 chemistry credits taken (p < .001) and .050 chemistry credits passed (p < .001). Students who had ever been economically disadvantaged also took all MMC science requirements at a differentially higher rate than those who had never been economically disadvantaged. The differential effect was 4.5 percentage points (p < .01) more for economically disadvantaged students, on top of the 4.0 percentage point (p < .05) increase amongst those who had never been classified as economically disadvantaged. Overall, we see some positive differential gains in credits taken for groups whose pre- policy levels of course-taking were lower than the comparison group. This was especially prevalent in chemistry and mostly absent in physics. In some cases, there were shifts away from biology credits, and in the case of Asian students, away from chemistry and physics credits compared to white students. The policy was intended to increase the number of biology, chemistry, and physics credits students took. In the cases where the pre-policy levels of course- taking were low it appears that the policy worked as intended, although whether the magnitude of the effects is as large as the policy’s creators intended is unclear. If the purpose of the policy was to provide more than a nudge in the right direction, then the effects are likely disappointing despite being positive. 90 (4) Q4 (5) Q5 -0.035 (0.054) -0.003 (0.051) -0.056+ (0.031) -0.029 (0.030) 0.086*** (0.023) 0.072*** (0.021) 0.074*** (0.020) 0.070*** (0.020) 0.041* (0.018) 0.044** (0.016) (2) Q2 0.039 (0.074) 0.029 (0.070) -0.070+ (0.037) -0.053 (0.034) 0.194*** (0.028) 0.142*** (0.023) 0.083*** (0.017) 0.078*** (0.017) 0.083*** (0.018) 0.070*** (0.017) (1) Q1 0.122 (0.125) 0.088 (0.125) -0.100 (0.062) -0.080 (0.059) 0.234*** (0.030) 0.156*** (0.024) 0.073*** (0.016) 0.063*** (0.014) 0.107*** (0.016) 0.079*** (0.013) (3) Q3 0.013 (0.064) 0.026 (0.060) -0.059 (0.038) -0.035 (0.037) 0.153*** (0.029) 0.116*** (0.026) 0.089*** (0.020) 0.084*** (0.019) 0.070*** (0.019) 0.061*** (0.018) Table 23 Estimated impact of the MMC on science credits by 8th grade MEAP quintiles All Science Taken All Science Passed Biology Taken Biology Passed Chemistry Taken Chemistry Passed Physics Taken Physics Passed Took MMC Credits Passed MMC Credits N Note. Coefficients for the first eight outcomes represent the change in number of science credits taken and passed during high school. Coefficients for the last two outcomes represent percentage point change in the number of students taking or meeting all MMC science requirements. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 -0.078 (0.078) -0.078 (0.072) -0.081* (0.035) -0.069* (0.033) 0.030 (0.026) 0.023 (0.024) 0.040+ (0.022) 0.042* (0.021) -0.014 (0.019) -0.004 (0.017) 43383 60657 52204 48762 46235 Table 23 shows heterogeneity in course-taking outcomes by quintile of student achievement as measured by the 8th grade MEAP standardized exam. In terms of total science 91 credits taken and passed as well as biology credits taken and passed, there is no evidence of changes for any of the achievement groups with the exception of .081 (p < .05) fewer biology credits taken and .069 (p < .05) fewer passed for the highest achieving quintile. In chemistry and physics there were increases in the number of credits taken and passed across the achievement distribution. The students in the top quintile, however, only saw increases in physics. In terms of taking and passing the MMC, the lowest four quintiles saw increases in the proportion of students doing so, and there was no discernible change for the highest quintile group. These findings likely reflect that the highest achieving students were already taking MMC requirements, with the possible exception of physics, so they saw little change. Meanwhile, the students in the bottom four quintiles gained in chemistry and physics course-taking, which were places they were not meeting MMC requirements before the policy went into effect. Student-level Heterogeneity in College Major and Graduation Table 24 presents results for ever declaring a STEM major and declaring a STEM major in the first year of postsecondary education by student characteristics. Female students were differentially more likely to declare a biology major after the MMC (1.3 percentage points [p < .001]) and less likely to declare an engineering major (-1.1 percentage points [p < .001]) than male students. The result in engineering negates the gains males saw in engineering major declaration, meaning that females stayed roughly the same in the proportion declaring engineering majors while males increased in the proportion declaring engineering majors. 92 Table 24 Heterogeneous impact of the MMC on STEM college major selection by student characteristics Ever Major (5) (2) (4) (1) (6) First Year Major (7) Physical Science STEM Biology 0.007+ (0.004) -0.062*** (0.002) 0.003 (0.002) 0.010* (0.004) 0.016*** (0.003) -0.007 (0.004) 0.009* (0.004) 0.004 (0.004) -0.003+ (0.002) 0.009*** (0.001) 0.008*** (0.002) 0.000 (0.001) -0.002*** (0.000) 0.002* (0.001) 0.001 (0.002) 0.002 (0.001) 0.002 (0.002) 0.001 (0.002) 0.000 (0.002) 0.002+ (0.001) 0.001 (0.001) -0.001 (0.001) 0.002+ (0.001) -0.000 (0.001) (8) Engineering 0.009*** (0.002) -0.032*** (0.002) -0.005** (0.002) 0.008*** (0.002) 0.008*** (0.001) -0.009*** (0.002) 0.007*** (0.002) 0.007** (0.002) -0.021* (0.009) -0.007+ (0.004) -0.001 (0.003) -0.009* (0.004) 93 -0.012+ (0.006) -0.000 (0.003) -0.000 (0.001) -0.008** (0.003) (3) Physical Science STEM Biology 0.007 (0.004) -0.083*** (0.003) 0.006+ (0.003) -0.002 (0.002) 0.017*** (0.002) 0.013*** (0.002) -0.000 (0.001) -0.005*** (0.001) 0.002 (0.001) 0.014*** (0.004) 0.020*** (0.004) -0.023*** (0.005) 0.011** (0.004) 0.006 (0.006) 0.005* (0.002) 0.002 (0.003) -0.002 (0.003) 0.005* (0.002) 0.003 (0.003) 0.001 (0.001) -0.001 (0.001) -0.003* (0.001) 0.001 (0.001) -0.001 (0.002) 0.013*** (0.002) -0.046*** (0.003) -0.011*** (0.002) Engineering 0.010*** (0.002) 0.009*** (0.002) -0.014*** (0.002) 0.008*** (0.002) 0.007* (0.003) Panel A: Female MMC Female MMC * Female Panel B: Black MMC Black MMC * Black Panel C: Hispanic MMC Hispanic MMC * Hispanic 0.004* (0.002) 0.074*** (0.009) 0.003 (0.011) 0.008* (0.004) 0.104*** (0.008) 0.062*** (0.012) Table 24 (cont’d) Panel D: Asian MMC Asian MMC * Asian Panel E: Ever limited English proficient MMC LEP MMC * LEP Panel F: Ever economically disadvantaged MMC Econ Disadvantaged MMC * Econ Disadvantaged N 0.010** (0.004) 0.014 (0.010) -0.004 (0.012) 0.025*** (0.004) 251241 0.005* (0.002) -0.004 (0.007) -0.004 (0.005) -0.018*** (0.002) -0.008*** (0.001) 0.009*** (0.002) 0.001 (0.001) 0.003 (0.005) 0.002 (0.005) 0.006*** (0.002) 0.019*** (0.005) 0.030** (0.011) 0.001 (0.001) 0.003* (0.001) -0.004+ (0.002) 0.007*** (0.002) 0.011*** (0.003) 0.005 (0.006) 0.003+ (0.001) 0.014*** (0.002) -0.003*** (0.001) -0.001 (0.001) 0.008* (0.003) 0.072*** (0.009) -0.006 (0.013) 0.002 (0.001) 0.045*** (0.008) -0.014 (0.013) 0.008* (0.004) 0.011* (0.005) -0.005 (0.006) 0.001 (0.002) -0.003 (0.003) -0.001 (0.003) 0.001+ (0.001) 0.001 (0.002) 0.001 (0.003) 0.002+ (0.001) 0.002+ (0.001) -0.003* (0.001) 0.006*** (0.002) 0.010** (0.004) 0.022** (0.008) 0.006*** (0.002) 0.006* (0.002) 0.002 (0.005) 0.016*** (0.004) 0.002 (0.002) 0.002* (0.001) 0.012*** (0.002) -0.008*** (0.002) -0.005*** (0.001) -0.001** (0.000) -0.001 (0.001) -0.035*** (0.004) -0.009*** (0.002) -0.005*** (0.001) -0.016*** (0.002) -0.018*** (0.004) -0.002 (0.002) -0.002** (0.001) -0.012*** (0.002) 251241 251241 251241 251241 251241 251241 251241 94 Table 24 (cont’d) Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable, local unemployment by year, and school fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 95 Black and Hispanic students also saw differential decreases in the number of STEM and engineering majors compared to white students, meaning that while white students saw gains in the proportion of students choosing those areas of study, there was less interest or no change in STEM and engineering majors among Black and Hispanic students. Asian students, on the other hand, saw relatively large differential gains in the proportion of students declaring STEM majors, biology majors, and engineering majors compared to white students. Students who had ever been labeled Limited English Proficient saw little in the way of differential effects compared to their peers. Meanwhile there were effects across the board for students who had ever been economically disadvantaged. They were less likely to declare a STEM major, a biology major, a physical science major, or an engineering major. In all cases, any gains that were seen among non- economically disadvantaged students were negated by the negative differential effects. 96 (1) Q1 (2) Q2 (5) Q5 (4) Q4 -0.004 (0.004) -0.001 (0.002) -0.001 (0.001) 0.001 (0.001) -0.000 (0.003) -0.000 (0.001) -0.000 (0.001) -0.000 (0.001) -0.006 (0.005) 0.002 (0.003) -0.000 (0.002) 0.001 (0.002) 0.003 (0.004) 0.004+ (0.002) 0.001 (0.001) 0.000 (0.002) -0.000 (0.010) 0.009 (0.006) -0.004 (0.004) 0.008+ (0.005) 0.015 (0.009) 0.004 (0.005) 0.003 (0.002) 0.011** (0.004) (3) Q3 0.010 (0.007) 0.002 (0.004) 0.000 (0.002) 0.008* (0.003) 0.011+ (0.006) 0.001 (0.003) 0.001 (0.002) 0.007** (0.002) Table 25 Estimated impact of the MMC on postsecondary outcomes by 8th grade MEAP quintiles Ever study STEM Ever study biology Ever study physical science Ever study engineering First year study STEM First year study biology First year study physical science First year study engineering N 43383 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable, local unemployment by year, and school fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.030* (0.013) 0.006 (0.006) 0.002 (0.005) 0.029*** (0.008) 0.008 (0.012) -0.004 (0.005) 0.001 (0.004) 0.024*** (0.006) 46235 60657 52204 48762 In terms of differential effects on STEM majoring based on achievement, Table 25 presents results for quintiles of achievement based on 8th grade MEAP scores. Here we seen that the gains in engineering majors were driven by students in the top three quintiles of achievement. 97 Otherwise there is little evidence of differences in who drove changes in STEM majors overall and in other science disciplines. Table 26 presents results for graduation with STEM degrees based on student characteristics. Columns 1 through 4 show graduation with any degree or certificate within six years while columns 5 through 8 show graduation with a bachelor’s degree within six years. For females, there was a differential increase of .5 percentage points (p < .05) in STEM degrees and .2 percentage points (p < .05) in physical science degrees. It is possible that the shift in science course-taking toward more chemistry and physics credits translated into higher rates of completing STEM and physical science degrees for females more than males. Black students experienced a .3 percentage point (p < .05) increase in biology degrees compared to white students and .4 percentage points (p < .01) fewer in engineering. Hispanic students earned .8 percentage points (p < .05) fewer STEM degrees overall compared to white students, and Asian students earned .2 percentage points (p < .05) more bachelor’s degrees in engineering relative to white students. The findings for Black and Asian students are noteworthy because they run counter to the differential course-taking findings from Table 22; Black students took fewer biology courses relative to white students but graduated with more degrees relative to white students and Asian students took less chemistry and physics compared to white students but earned more degrees in engineering. Graduation is a distal outcome, however, and there are many opportunities for students to change their plans along the way even if their high school course-taking had an effect on postsecondary plans. 98 Table 26 Heterogeneous impact of the MMC on STEM degree completion by student characteristics Any Degree within Six Years (4) (1) (2) Bachelor's Degree within Six Years (5) (6) STEM Biology Engineer STEM Biology 0.003 (0.003) -0.025*** (0.002) 0.005* (0.002) -0.001 (0.002) 0.007*** (0.001) 0.002+ (0.001) 0.005+ (0.003) 0.002 (0.002) -0.001 (0.002) 0.006* (0.003) 0.000 (0.003) -0.008* (0.003) -0.000 (0.001) -0.002 (0.001) 0.003* (0.001) 0.001 (0.001) 0.000 (0.002) -0.002 (0.002) (3) Physical Science 0.001 (0.001) -0.001 (0.001) 0.002* (0.001) 0.002* (0.001) -0.001 (0.001) -0.000 (0.001) 0.002* (0.001) -0.002* (0.001) -0.001 (0.001) Panel A: Female MMC Female MMC * Female Panel B: Black MMC Black MMC * Black Panel C: Hispanic MMC Hispanic MMC * Hispanic 99 0.002 (0.002) -0.032*** (0.002) 0.002 (0.001) 0.004 (0.003) -0.021*** (0.002) 0.005* (0.002) -0.001 (0.002) 0.007*** (0.001) 0.002 (0.001) 0.004+ (0.002) 0.004** (0.002) -0.004** (0.001) 0.003+ (0.002) 0.002 (0.003) -0.005+ (0.003) 0.006* (0.003) 0.004 (0.002) -0.001 (0.002) 0.006* (0.003) 0.002 (0.003) -0.008* (0.003) -0.000 (0.001) -0.001 (0.001) 0.003* (0.001) 0.001 (0.001) 0.001 (0.002) -0.002 (0.002) (7) Physical Science 0.001 (0.001) -0.001 (0.001) 0.002* (0.001) 0.002* (0.001) -0.001 (0.001) -0.000 (0.001) 0.002** (0.001) -0.002* (0.001) -0.001 (0.001) (8) Engineer 0.003 (0.002) -0.028*** (0.002) 0.001 (0.001) 0.004* (0.002) 0.005*** (0.002) -0.003** (0.001) 0.004* (0.002) 0.003 (0.003) -0.005+ (0.003) 0.002* (0.001) 0.005+ (0.003) -0.003 (0.004) 0.002* (0.001) 0.002+ (0.001) -0.002+ (0.001) 0.002+ (0.001) -0.002*** (0.000) 0.001 (0.001) 0.043*** (0.006) -0.013+ (0.008) 0.000 (0.001) -0.005+ (0.003) 0.000 (0.002) -0.000 (0.002) -0.007*** (0.001) 0.005+ (0.003) 0.072*** (0.009) 0.004 (0.012) Table 26 (cont’d) Panel D: Asian MMC Asian MMC * Asian Panel E: Ever limited English proficient MMC LEP MMC * LEP Panel F: Ever economically disadvantaged MMC Econ Disadvantaged MMC * Econ Disadvantaged N 0.005+ (0.003) 0.007+ (0.004) -0.005 (0.004) 0.007* (0.003) -0.015*** (0.002) -0.004* (0.002) 187020 0.002 (0.001) 0.001 (0.001) -0.007*** (0.001) -0.004* (0.002) 0.002 (0.001) 0.000 (0.001) -0.006*** (0.001) 187020 187020 187020 187020 187020 187020 187020 100 0.002 (0.002) 0.027*** (0.006) 0.019+ (0.010) 0.003 (0.002) 0.009** (0.003) -0.002 (0.003) 0.006* (0.003) 0.070*** (0.009) 0.006 (0.012) 0.001 (0.001) 0.042*** (0.006) -0.012 (0.007) 0.006* (0.003) 0.006 (0.004) -0.004 (0.003) 0.000 (0.001) -0.005+ (0.003) 0.001 (0.002) 0.002** (0.001) 0.006+ (0.003) -0.004 (0.004) 0.002** (0.001) 0.003+ (0.001) -0.002+ (0.001) 0.003+ (0.002) 0.026*** (0.006) 0.020* (0.009) 0.004* (0.002) 0.008** (0.003) -0.002 (0.003) 0.006** (0.002) -0.006*** (0.001) 0.008** (0.003) -0.015*** (0.002) -0.000 (0.002) -0.007*** (0.001) 0.002* (0.001) -0.002*** (0.000) 0.006** (0.002) -0.006*** (0.001) Table 26 (cont’d) Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable, local unemployment by year, and school fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 101 There were no observed differential changes for LEP students. The gains in STEM and engineering degrees after the MMC were largely negated by the differential effects of the policy for students who had ever been classified as economically disadvantaged. The differential effect on STEM degrees was -.4 percentage points (p < .05) in STEM and -.7 percentage points (p < .001) in engineering. Table 27 presents results for STEM postsecondary graduation outcomes broken down by quintiles of 8th grade MEAP achievement. These tables show that the gains in STEM and engineering degrees were driven by students in the top three quintiles of achievement. In physical science, students in the top quintile also saw a gain of 1.1 percentage points in physical science degrees. These findings are consistent when limited to bachelor’s degrees only. This may reflect that, regardless of the magnitude of changes the MMC science requirements had on student high school course-taking, students who were best positioned before the policy in terms of achievement were best able to take advantage of the new requirements for future outcomes. 102 (5) Q5 (1) Q1 (4) Q4 (3) Q3 -0.001 (0.001) -0.000 (0.000) -0.000 (0.000) -0.000 (0.001) 0.000 (0.001) -0.000 (0.000) 0.000 (0.000) 0.000 (0.001) 0.007* (0.003) 0.001 (0.002) 0.001 (0.001) 0.005* (0.002) 0.006* (0.003) 0.001 (0.001) 0.001 (0.001) 0.005** (0.002) (2) Q2 0.001 (0.002) 0.001 (0.001) -0.001 (0.001) 0.001 (0.001) 0.001 (0.002) 0.001 (0.001) -0.001 (0.001) 0.001 (0.001) 0.019*** (0.005) 0.007* (0.003) 0.002 (0.002) 0.010* (0.004) 0.021*** (0.005) 0.008** (0.003) 0.002 (0.002) 0.011** (0.004) Table 27 Estimated impact of the MMC on postsecondary outcomes by 8th grade MEAP quintiles Any STEM degree Any biology degree Any physical science degree Any engineering degree STEM bachelor's degree Biology bachelor's degree Physical science bachelor's degree Engineering bachelor's degree N 43383 Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable and local unemployment by year. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 60657 52204 48762 46235 0.034*** (0.009) -0.005 (0.005) 0.011*** (0.003) 0.029*** (0.008) 0.034*** (0.009) -0.005 (0.005) 0.011*** (0.003) 0.029*** (0.008) Heterogeneity in Science Credits by School Characteristics Table 28 shows results of models that estimate the change in number of science courses students took in high school after the MMC, broken down by quintiles of school characteristics in 2003. Panel A provides results by school average 8th grade MEAP performance. Students from 103 schools with high average achievement scores reduced the number of science credits taken in biology. Across the quintiles of student achievement chemistry credits taken increase, with the largest coefficients in the lowest quintiles. For example, schools in the lowest quintile on student achievement increased the average number of chemistry credits taken by .211 credits (p < .01). In physics, students in the lowest quintile and highest two quintiles increased the number of physics credits students took. Overall, it appears that the results for low quintile schools and high quintile schools on achievement were not much different. Chemistry and physics credits increased, while biology credits in the highest achieving schools decreased. When schools were separated into quintiles by proportion of students classified as economically disadvantaged, the results were largely the same as they were for quintiles of achievement. Panel B shows that students in the lowest quintile (i.e. those who had the lowest proportion of students classified as economically disadvantaged) decreased biology credits taken. Similar to the results for achievement as well, chemistry credits increased for all quintiles and physics credits increased for the four quintiles with the lowest proportion of eligible students. These findings indicate that, regardless of the average makeup of the student body in the schools, they responded by getting more students to take MMC required science courses. Compared to the lack of changes in the student-level heterogeneity analyses on achievement and economic disadvantage, these results also indicate that it was a school-level response rather than an individual one. Put another way, it appears that schools were encouraging their students to take more chemistry and physics rather than students within schools making their own decisions about taking chemistry and physics based on their achievement or economic status. 104 (1) Q1 (2) Q2 (3) Q3 (4) Q4 0.327 (0.281) 0.037 (0.138) 0.211** (0.065) 0.093* (0.034) -0.146 (0.107) -0.073 (0.067) 0.272*** (0.066) 0.031 (0.034) -0.180* (0.071) -0.125*** (0.025) 0.072* (0.029) 0.075* (0.034) -0.002 (0.160) -0.080 (0.087) 0.137* (0.055) 0.061** (0.020) 0.020 (0.139) -0.060 (0.077) 0.229** (0.075) 0.083* (0.032) Table 28 Heterogeneous impact of the MMC on high school science credits by school characteristics (5) Q5 Panel A: 8th grade MEAP math scores All science credits Biology credits Chemistry credits Physics credits Panel B: Percent economically disadvantaged 0.081 All science credits (0.152) -0.079 Biology credits (0.098) 0.145* Chemistry credits (0.054) 0.061** Physics credits (0.019) Panel C: Percent enrolling in postsecondary education -0.016 All science credits (0.182) Biology credits -0.126 (0.101) 0.201** Chemistry credits (0.069) 0.068* Physics credits (0.030) Panel D: Percent studying STEM All science credits Biology credits Chemistry credits Physics credits -0.115 (0.082) -0.125*** (0.028) 0.089** (0.027) 0.095** (0.032) -0.005 (0.227) -0.105 (0.151) 0.187** (0.063) 0.044 (0.044) -0.168* (0.060) -0.135*** (0.029) 0.066* (0.027) 0.078+ (0.044) -0.142* (0.058) -0.161*** (0.037) 0.056+ (0.030) 0.090+ (0.044) -0.080 (0.214) -0.159 (0.096) 0.145* (0.056) 0.078** (0.023) -0.008 (0.129) -0.033 (0.067) 0.137* (0.051) 0.042+ (0.023) -0.046 (0.116) -0.142* (0.059) 0.041 (0.047) 0.061 (0.038) -0.044 (0.093) -0.028 (0.073) 0.103* (0.047) 0.080* (0.036) -0.199+ (0.114) -0.039 (0.063) 0.051 (0.038) 0.047 (0.031) 0.331* (0.131) 0.158* (0.070) 0.262*** (0.063) 0.022 (0.023) 0.334* (0.151) 0.133 (0.087) 0.315*** (0.065) 0.017 (0.025) -0.066 (0.144) -0.125 (0.102) 0.176* (0.073) 0.104** (0.028) -0.199 (0.198) -0.081 (0.104) 0.122* (0.057) 0.071+ (0.035) 105 0.270+ (0.145) 0.038 (0.079) 0.262*** (0.065) 0.049* (0.022) 0.038 (0.177) -0.076 (0.106) 0.150+ (0.086) 0.119** (0.034) 0.032 (0.209) -0.079 (0.124) 0.165* (0.067) 0.042 (0.029) -0.211* (0.089) -0.121* (0.043) 0.069+ (0.036) 0.058+ (0.033) Table 28 (cont’d) Panel E: Percent graduating with a STEM degree -0.061 All science credits (0.039) -0.119** Biology credits (0.034) 0.119*** Chemistry credits (0.024) 0.073* Physics credits (0.034) Note. Coefficients represent the change in number of science credits taken during high school. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 When it comes to other metrics for assessing school heterogeneity, the results are similar. Panel C shows that schools in the top quintile of students eventually enrolling in college reduced the number of science credits taken by .168 (p < .05) and the number of credits taken in biology by .135 (p < .001). In the lowest quintile, we find the opposite; students in these schools increased total science credits taken by .331 (p < .05) and biology credits by .158 (p < .05). Schools in the top two and bottom two quintiles of college enrollment increased chemistry credits and schools in quintiles two and four showed increases in physics credits taken. The biology and total science credit findings are unsurprising given that part of the reason schools may have had low college enrollment is because students were not taking sufficient science courses to be well prepared for postsecondary schooling, or the schools were aware that not 106 many students attended postsecondary education and tailored their course offerings to reflect this. Panel D shows results where schools are broken into quintiles based on proportion of students ever choosing a STEM major, and Panel E shows results where quintiles are based on students graduating from any program with a STEM degree. The patterns in these panels are similar to the ones from Panel C: students in the top quintiles decreased total science credits and credits in biology while students in the lowest quintiles increased these credits. Across the quintiles more students took chemistry and physics. Table 29 shows comparisons between the lowest quintile and highest quintile on 2003 school proportion of students taking chemistry, physics, and AP science courses. Due to a lack of variation at the low end of the distribution on these factors, the middle quintiles are not included as they were largely uninformative due to their size. These results are consistent with the other school-level heterogeneity analyses. Biology courses were less common, especially in the highest quintile. Chemistry courses were more common across the groups, as were physics courses. Overall, the results from the school-level heterogeneity analyses show that the gains in chemistry and physics course-taking were not confined to schools with particular characteristics. Combined with results from the student-level heterogeneity analyses it appears that schools increased the number of chemistry and physics courses they taken, and that students within the schools were given differential access or made different choices about what courses to take courses based on student-level characteristics. 107 (2) Q5 (1) Q1 -0.194 (0.177) -0.232* (0.096) 0.101* (0.046) 0.111* (0.042) -0.080 (0.099) -0.057 (0.053) 0.164** (0.051) 0.015 (0.031) -0.055 (0.100) -0.115* (0.054) 0.113*** (0.032) 0.056** (0.017) Table 29 Heterogeneous impact of the MMC on science course-taking by high school level course- taking Panel A: 2003 proportion of students taking chemistry All science credits Biology credits Chemistry credits Physics credits Panel B: 2003 proportion of students taking physics All science credits Biology credits Chemistry credits Physics credits Panel C: 2003 proportion of students taking an AP science course -0.675 All science credits (0.110) -0.215** Biology credits (0.001) 0.116** Chemistry credits (0.001) 0.088 Physics credits (0.043) Note. Coefficients represent the change in number of science credits taken during high school. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared), and school fixed effects. All models include a time-trend variable. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 -0.021 (0.069) -0.084* (0.037) 0.133*** (0.025) 0.066*** (0.015) -0.039 (0.087) -0.108* (0.046) 0.118*** (0.027) 0.081*** (0.017) 108 Heterogeneity in STEM Major Choices and STEM Degrees by School Characteristics Table 30 presents results for heterogeneity analyses based on school characteristics predicting whether students ever declare a STEM major. Panel A shows that students from the school with the highest average achievement were more likely to major in STEM and engineering after the MMC, while students from the four lowest quintiles of student achievement saw no change. Likewise, Panel B shows that students from schools in the lowest quintile (i.e. those schools where the fewest students were economically disadvantaged) increased STEM majors by 2.3 percentage point (p < .05) and engineering majors by 1.4 percentage points (p < .001). Students in the second lowest quintile also increased engineering majors by 1.3 percentage points (p < .01). This same pattern is true in Panels C, D, and E. Students in the top quintile of college enrollment, STEM major, and STEM degree attainment increased the rate of declaring a STEM or engineering major and students in the second highest quintile also increased the rate of ever declaring an engineering major. 109 (4) Q4 (5) Q5 (1) Q1 (2) Q2 (3) Q3 0.001 (0.008) 0.004 (0.008) 0.000 (0.002) 0.002 (0.004) 0.003 (0.009) 0.003 (0.005) -0.005 (0.003) 0.013** (0.004) -0.004 (0.009) -0.001 (0.005) -0.000 (0.003) 0.002 (0.004) -0.007 (0.009) -0.001 (0.007) -0.002 (0.003) 0.001 (0.004) 0.027** 0.006 (0.008) (0.012) 0.008+ 0.005 (0.004) (0.005) 0.002 -0.003 (0.003) (0.003) 0.009+ 0.018*** (0.004) (0.004) Table 30 Heterogeneous impact of the MMC on STEM college major selection by school characteristics Panel A: 8th grade MEAP math scores 0.010 Ever study STEM (0.007) 0.004 Ever study biology (0.003) -0.000 Ever study physical science (0.001) 0.004 Ever study engineering (0.004) Panel B: Percent economically disadvantaged 0.023* Ever study STEM (0.009) 0.004 Ever study biology (0.004) Ever study physical science 0.003 (0.002) 0.014*** Ever study engineering (0.004) Panel C: Percent enrolling in postsecondary education Ever study STEM Ever study biology Ever study physical science Ever study engineering Panel D: Percent studying STEM Ever study STEM Ever study biology Ever study physical science Ever study engineering 0.029*** (0.008) 0.010* (0.004) 0.004 (0.002) 0.017** (0.004) 0.031*** (0.008) 0.010* (0.004) 0.003 (0.002) 0.015** (0.004) 0.007 (0.007) 0.005 (0.005) -0.002 (0.003) 0.010** (0.003) -0.007 (0.006) -0.002 (0.006) -0.003 (0.002) 0.002 (0.003) -0.006 (0.008) -0.004 (0.005) -0.001 (0.002) -0.002 (0.003) -0.005 (0.010) -0.000 (0.005) -0.002 (0.004) 0.001 (0.004) -0.006 (0.010) 0.003 (0.006) -0.004 (0.003) 0.003 (0.003) 0.001 (0.008) 0.004 (0.004) 0.003 (0.003) -0.001 (0.004) 0.002 (0.006) 0.004 (0.002) 0.003 (0.003) -0.001 (0.004) 0.005 (0.006) 0.006* (0.003) 0.000 (0.003) 0.010** (0.003) 0.003 (0.007) 0.007 (0.004) 0.002 (0.002) 0.000 (0.004) 0.010 (0.007) 0.003 (0.002) 0.003 (0.003) 0.004 (0.004) 110 0.015* (0.006) 0.002 (0.003) 0.003 (0.003) 0.004 (0.003) -0.001 (0.007) 0.004 (0.002) 0.000 (0.002) 0.000 (0.003) -0.010 (0.008) -0.002 (0.007) -0.003 (0.003) -0.002 (0.003) Table 30 (cont’d) Panel E: Percent graduating with a STEM degree Ever study STEM Ever study biology Ever study physical science Ever study engineering Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable, local unemployment by year, and school fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.013+ (0.007) 0.007 (0.005) 0.000 (0.003) 0.012** (0.004) 0.029** (0.009) 0.007 (0.004) 0.002 (0.003) 0.018*** (0.004) In Table 31, similar analyses are performed for percent of students taking chemistry, physics, and AP science courses in schools in 2003. Students in schools that had the most students taking chemistry and physics increased the rate of studying STEM by 1.6 percentage points (p < .05) and 2.0 percentage points (p < .01) respectively. Schools with the lowest levels of chemistry, physics, and AP course-taking saw increases in the proportion of students studying engineering. 111 (2) Q5 (1) Q1 0.016* (0.006) 0.005 (0.005) -0.003 (0.003) 0.005+ (0.003) 0.006 (0.007) 0.004 (0.003) 0.002 (0.002) 0.007* (0.003) Table 31 Heterogeneous Impact of the MMC on STEM College Major Selection Panel A: 2003 proportion of students taking chemistry Ever study STEM Ever study biology Ever study physical science Ever study engineering Panel B: 2003 proportion of students taking physics Ever study STEM Ever study biology Ever study physical science Ever study engineering Panel C: 2003 proportion of students taking an AP science course Ever study STEM Ever study biology Ever study physical science Ever study engineering Note. Coefficients are the result of linear probability models. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable, local unemployment by year, and school fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.010** (0.004) 0.005* (0.002) 0.001 (0.001) 0.008*** (0.002) 0.008+ (0.005) 0.002 (0.002) 0.000 (0.002) 0.008*** (0.002) 0.020** (0.005) 0.011* (0.004) 0.003 (0.002) 0.007+ (0.003) 0.043 (0.036) 0.020 (0.026) -0.005 (0.002) 0.014 (0.005) 112 In Table 32, results show that students from the highest achieving schools, schools with the lowest proportion of economically disadvantaged students, and schools with the highest proportion of students enrolling in college, declaring a STEM major, and graduating with a STEM degree saw increases in total STEM degrees, physical science degrees, and engineering degrees. With respect to rates of chemistry, physics, and AP science course-taking, Table 33 shows that physical science degrees and engineering degrees differed by pre-policy rate of physics course-taking. Students in schools with the highest rates of physics course-taking were .4 percentage points (p < .05) more likely to get a degree in physical science after the MMC. Students in the schools with the lowest rates of physics course-taking were .5 percentage points (p < .05) more likely to get a degree in engineering. These heterogeneity analyses tell a similar story to the difference-in-difference analyses of students’ postsecondary outcomes. Students in schools that were already well-positioned to send students into college STEM fields benefitted the most from the MMC. However, we also see that schools that offered the fewest higher-level science courses before the policy increased the number of students they sent to engineering programs, indicating that the increased chemistry and physics offerings these schools experienced may have encouraged some students to pursue postsecondary engineering studies. 113 (1) Q1 (2) Q2 (3) Q3 (4) Q4 -0.001 (0.004) 0.001 (0.002) 0.000 (0.001) -0.003 (0.004) 0.000 (0.006) -0.003 (0.003) -0.001 (0.003) 0.004 (0.005) Table 32 Heterogeneous impact of the MMC on STEM college graduation by high school characteristics Panel A: 8th grade MEAP math scores -0.003 Any STEM degree (0.002) 0.000 Any biology degree (0.002) 0.001 Any physical science degree (0.001) -0.003+ Any engineering degree (0.002) Panel B: Percent economically disadvantaged 0.017* Any STEM degree (0.006) 0.003 Any biology degree (0.004) Any physical science degree 0.006** (0.002) 0.010* Any engineering degree (0.005) Panel C: Percent enrolling in postsecondary education Any STEM degree Any biology degree Any physical science degree Any engineering degree 0.012 (0.007) -0.001 (0.003) 0.001 (0.003) 0.012** (0.004) 0.002 (0.005) 0.001 (0.002) 0.004* (0.002) -0.002 (0.005) 0.001 (0.004) 0.002 (0.003) -0.000 (0.002) -0.000 (0.002) 0.004 (0.004) 0.002 (0.003) 0.003 (0.002) -0.000 (0.004) 0.011+ (0.006) 0.006+ (0.003) 0.004 (0.002) 0.001 (0.004) 0.002 (0.005) 0.001 (0.003) 0.002 (0.002) -0.000 (0.004) 0.003 (0.006) -0.002 (0.002) -0.000 (0.003) 0.006 (0.005) -0.003 (0.003) 0.001 (0.001) -0.000 (0.001) -0.003 (0.003) (5) Q5 0.022** (0.007) -0.000 (0.004) 0.007** (0.002) 0.019*** (0.004) -0.004 (0.003) -0.001 (0.002) -0.001 (0.001) -0.002 (0.002) 0.023** (0.006) 0.004 (0.004) 0.007*** (0.002) 0.015*** (0.003) 114 0.001 (0.006) 0.004 (0.004) -0.000 (0.003) -0.004 (0.004) 0.004 (0.005) 0.004 (0.003) -0.000 (0.003) -0.001 (0.003) 0.002 (0.003) -0.002 (0.002) 0.001 (0.002) 0.003+ (0.002) -0.003 (0.003) 0.004 (0.003) -0.001 (0.001) -0.005* (0.002) 0.025*** (0.006) 0.003 (0.003) 0.007** (0.002) 0.017*** (0.003) Table 32 (cont’d) Panel D: Percent studying STEM Any STEM degree Any biology degree Any physical science degree Any engineering degree Panel E: Percent graduating with a STEM degree 0.020* Any STEM degree (0.007) 0.001 Any biology degree (0.003) 0.006** Any physical science degree (0.002) 0.015*** Any engineering degree (0.004) Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable, local unemployment by year, and school fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.005 (0.006) -0.001 (0.004) 0.003+ (0.002) 0.004 (0.005) 0.009 (0.006) 0.000 (0.004) 0.004+ (0.002) 0.005 (0.006) -0.001 (0.003) 0.001 (0.002) 0.002 (0.001) -0.002 (0.003) -0.001 (0.003) -0.001 (0.002) -0.001 (0.001) -0.000 (0.002) 115 (1) Q1 (2) Q5 0.001 (0.006) 0.002 (0.004) 0.002 (0.002) -0.002 (0.004) 0.010+ (0.006) 0.006 (0.004) 0.004* (0.002) 0.002 (0.005) 0.005 (0.004) -0.001 (0.002) 0.003* (0.001) 0.003 (0.003) Table 33 Heterogeneous impact of the MMC on STEM college graduation by high school level course- taking characteristics Panel A: 2003 proportion of students taking chemistry Any STEM degree Any biology degree Any physical science degree Any engineering degree Panel B: 2003 proportion of students taking physics Any STEM degree Any biology degree Any physical science degree Any engineering degree Panel C: 2003 proportion of students taking an AP science course Any STEM degree Any biology degree Any physical science degree Any engineering degree 0.005+ (0.003) 0.000 (0.001) 0.002+ (0.001) 0.004+ (0.002) 0.005 (0.003) -0.001 (0.002) 0.001 (0.001) 0.005* (0.002) 0.045 (0.017) 0.029 (0.006) 0.020 (0.008) 0.005 (0.023) 116 Table 33 (cont’d) Note. Coefficients are the result of linear probability models and are interpreted as percentage point change in the dependent variable after the MMC went into effect. Biology, physical science, and engineering are sub-fields within STEM. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time- trend variable, local unemployment by year, and school fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 117 CHAPTER 6: THE RELATIONSHIP BETWEEN HIGH SCHOOL SCIENCE COURSE- TAKING AND EARLY POSTSECONDARY STEM OUTCOMES Does taking additional science coursework increase the rate of declaring a STEM major, rate of taking an introductory science class during the first year of college, improve grades in key college STEM classes (introductory biology, chemistry, and physics), and increase rate of graduation with STEM degrees? Instrumental Variable Estimation of the Effect of High School Science Course-taking on STEM Major Selection and STEM Degree Attainment Table 34 presents results of instrumental variable estimation of the effect of being induced to take chemistry or physics in high school on STEM major selection and graduation using proportion of students in a school-cohort who took such courses as the instrument. The estimates provided in Table 34 represent the Local Average Treatment Effect (LATE) for students to whom this situation applies. The estimates rely on only pre-MMC cohorts and do not estimate the effect of the MMC policy. Rather, they provide estimates of what we might expect would happen for students that the MMC policy was meant to affect. 118 (2) (4) (3) (1) STEM 0.013+ (0.007) 0.038 0.006 (0.006) 0.026 0.009 (0.006) 0.027 Biology Physical Science Engineering 0.003 (0.005) 0.018 0.002 (0.003) 0.008 0.004 (0.003) 0.008 -0.004 (0.008) 0.049 0.001 (0.006) 0.026 -0.003 (0.005) 0.017 0.010 (0.013) 0.162 0.014 (0.011) 0.097 0.007 (0.008) 0.051 Table 34 Instrumental variable estimates of taking chemistry and physics in high school on college STEM majors and degrees Panel A: Ever Took Chemistry in HS Ever Declared STEM Major Mean Year 1 Declared STEM Major Mean Earned STEM Bachelor's Degree Mean Panel B: Ever Took Physics in HS Ever Declared STEM Major Mean Year 1 Declared STEM Major Mean Earned STEM Bachelor's Degree Mean N Note. Coefficients represent the local average treatment effect of taking chemistry (Panel A) or physics (Panel B) in high school using the likelihood of taking chemistry or physics as an endogenous variable in two-stage least squares estimation. The instrumental variable is proportion of students taking chemistry or physics in the 2003 cohort. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable, local unemployment by year, and school and cohort fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. 0.020** (0.007) 0.038 0.012* (0.006) 0.026 0.013* (0.006) 0.027 0.016 (0.013) 0.162 0.007 (0.010) 0.097 0.015+ (0.008) 0.051 -0.001 (0.008) 0.049 -0.006 (0.006) 0.026 -0.002 (0.005) 0.017 0.009+ (0.005) 0.018 0.005 (0.003) 0.008 0.004 (0.003) 0.008 177320 177320 177320 177320 + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 119 Panel A of Table 34 shows the results for taking chemistry, instrumented by the proportion of students in a school-cohort taking chemistry. It shows no effects of being induced to take chemistry on majoring in STEM, biology, physical science, or engineering. It also shows no effect for graduating with a degree in any STEM field or STEM as a whole. Panel B shows the results for being induced to take physics. Here we see some significant findings for engineering. Taking physics in high school increased the proportion of students ever declaring a major in engineering by 2.0 percentage points (p < .01), declaring an engineering major in the first year by 1.2 percentage points (p < .05), and completing a bachelor’s degree in engineering by 1.3 percentage points (p < .05). Given that results for Research Question 2 indicated that the MMC policy increased chemistry and physics course-taking, these findings may help explain that the mechanism through which the MMC policy affected engineering majors and degrees was, in part, through increased physics participation in high school. However, the lack of differential gains in physics course-taking for groups that have traditionally been underrepresented in engineering, as seen in the heterogeneity analyses, mean that the MMC would need to do more to broaden participation in engineering. It could, though, as high school physics course-taking does seem to be a pathway through which engineering participation can be broadened. Instrumental Variable Estimation of the Effect of High School Science Course-taking on Taking and Performance in College Science Courses Table 35 shows the results for the effect of taking high school chemistry and physics on taking biology, chemistry, and physics courses in college. The results for high school chemistry, in Panel A, show only that students who are induced to take chemistry in high school are 5.6 percentage points (p < .01) less likely to take a biology course in their first year. This may be 120 because they are already meeting a college science requirement in high school by taking chemistry or, more likely, that they have opted to meet a general science credit requirement that would have normally been met with a biology course by taking something else instead. It does not appear that the difference is made up for by taking chemistry or physics courses, however, as the estimates for these courses are not statistically significant. In Panel B, the same analyses are performed except taking high school physics is the predictor. Here we see that taking high school physics predicts increases in ever taking chemistry (2.9 percentage points [p < .05]) and ever taking physics (3.3 percentage points [p < .01]). However, these results do not extend to the first year of college course-taking indicating that students may be moving toward majors that require chemistry or physics rather than trying to meet a general first-year science requirement by opting to take chemistry or physics. 121 Chemistry (2) (3) Physics 0.014 (0.011) 0.119 -0.002 (0.010) 0.046 (1) Biology 0.007 (0.014) 0.182 -0.010 (0.016) 0.129 -0.003 (0.016) 0.305 -0.056** (0.021) 0.224 Table 35 Instrumental variable estimates of taking chemistry and physics in high school on college STEM course-taking Panel A: Ever Took Chemistry in HS Ever Take Course Mean Take Course in Year 1 Mean Panel B: Ever Took Physics in HS Ever Take Course Mean Take Course in Year 1 Mean N Note. Coefficients represent the local average treatment effect of taking chemistry (Panel A) or physics (Panel B) in high school using the likelihood of taking chemistry or physics as an endogenous variable in two-stage least squares estimation. The instrumental variable is proportion of students taking chemistry or physics in the 2003 cohort. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable, local unemployment by year, and school and cohort fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.033** (0.011) 0.119 0.004 (0.009) 0.046 0.006 (0.016) 0.305 -0.018 (0.018) 0.224 0.029* (0.013) 0.182 0.015 (0.014) 0.129 177320 177320 177320 Table 36 shows similar estimates except this time the outcomes is GPA in biology, chemistry, and physics courses. These analyses are also repeated for courses equivalent to 100 level or lower and 200 level or lower in each discipline to reflect the introductory courses student take in each subject and which we might expect taking high school chemistry or physics to 122 affect. In Panel A, there are no apparent effects of being induced to take chemistry on biology or chemistry course performance. However, there are substantial increases in physics course performance of .663 grade points in all physics courses (p < .001), .495 grade points in 100 level physics courses or lower (p < .05), and .643 grade points in 200 level courses or lower (p < .01). In Panel B, there are no effect of being induced to take physics on course grade performance. It is unclear why chemistry in high school would affect physics in college (but not chemistry in college). It may be the case that taking chemistry in high school represents a big step in terms of opportunities to develop quantitative science skills, which translates into college STEM success, whereas the type of student who has been induced to take physics has already passed a threshold of basic quantitative science skills and does not benefit as much from the additional high school science course. It is unclear, however, if this is actually the case or if some other mechanism is at work. 123 (8) Physics 100 or Lower (9) Physics 200 or Lower Table 36 Instrumental variables estimates of high school chemistry and physics on college STEM grades (1) All Biology Courses (2) Biology 100 or Lower (3) Biology 200 or Lower (4) All (5) Chemistry Chemistry Courses 100 or Lower (6) Chemistry 200 or Lower (7) All Physics Courses -0.093 (0.075) 2.378 -0.044 (0.083) 2.296 -0.000 (0.103) 2.380 0.026 (0.102) 2.378 0.245 (0.156) 2.426 0.228 (0.166) 2.400 -0.004 (0.112) 2.296 0.495* (0.241) 2.690 0.259+ (0.155) 2.418 0.663** (0.203) 2.600 Panel A: Ever Took Chemistry in HS Average grade Mean Panel B: Ever Took Physics in HS Average grade Mean N Note. Coefficients represent the local average treatment effect of taking chemistry (Panel A) or physics (Panel B) in high school using the likelihood of taking chemistry or physics as an endogenous variable in two-stage least squares estimation. The instrumental variable is proportion of students taking chemistry or physics in the 2003 cohort. All models include student controls (gender, race/ethnicity, age, age squared, migrant, whether ever classified as limited-English proficient, ever classified as economically disadvantaged, ever classified for special education, and standardized 8th grade MEAP math score), school controls (average log number of teachers for the cohort, average proportion of students ever economically disadvantaged, average total student expenditures, and enrollment and enrollment squared) as well as a time-trend variable, local unemployment by year, and school and cohort fixed effects. Standard errors are clustered at the school level. Standard errors are in parentheses. + p < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 0.172 (0.107) 2.426 0.090 (0.175) 2.690 0.125 (0.111) 2.400 0.168 (0.105) 2.418 -0.067 (0.076) 2.380 0.129 (0.142) 2.600 26674 17554 26522 24216 44955 45906 12525 37806 0.141 (0.142) 2.601 17411 0.643** (0.204) 2.601 124 Discussion Literature on the relationship between high school science course-taking and college science course-taking and performance indicates that there is a strong, robust relationship between the two. However, the literature is fraught with selection issues. The IV estimates have attempted to account for those issues by isolating the local average treatment effect for students who were induced to take chemistry or physics due to their peers doing so. I find some effects of high school course-taking, which makes theoretical sense and aligns with the current literature on the topic. The instrumental variable, proportion of students taking each science course, may suffer from being correlated with unobserved determinants of the outcomes tested. If this is the case, though, we can assume in which direction they are correlated and take the estimates presented here as upper bounds for the effect. A policy like the MMC, which changes students’ course-taking to include more chemistry and physics, should then affect a number of students who were on the margins about studying science in college and needed a nudge to take chemistry or physics in high school to encourage them to continue studying. It should be noted, however, that the estimates provided here are narrow in who they apply to and cannot be taken as a measure of what might occur in a whole state as a result of a policy like the MMC that dramatically increased course-taking. It does show, though, that there are some students for whom taking chemistry or physics is helpful for later chemistry and physics outcomes. 125 CHAPTER 7: CONCLUSION AND IMPLICATIONS In these analyses I have aimed to describe the effect of the Michigan Merit Curriculum policy on high school science course-taking and what the policy meant for students’ postsecondary STEM outcomes. I used seven cohorts of transcript data from 128 Michigan high schools which spanned the time before and after the MMC policy was implemented. These transcript data were linked to state administrative records and the National Student Clearinghouse postsecondary database to examine how high school course-taking may have affected postsecondary behavior. In this chapter, I discuss my results and consider the implications for future science course policy as well as for the learning of science in high schools. Summary of Findings The Michigan Merit Curriculum policy on science coursework changed the types of science courses students in the state took. Chemistry and physics credits increased while biology credits decreased. The magnitude of these shift in chemistry and physics were substantial given the number of credits students were earning in those courses before the policy was introduced. However, less than 40% of students met all of the MMC science requirements after the policy went into effect. Although this represented an increase in the proportion of students reaching that standard, the policy was not effective at inciting mass change in science course-taking throughout the state. One thing the MMC policy did not do was to increase the total number of science credits students took. Rather than being exposed to more science, some students were exposed to different science courses. The shifts in science courses affected the schools that the policy was most meant to target. In schools where fewer students took the MMC required courses before the policy, we see 126 the largest shifts after the policy. Yet those shifts in course-taking were not large enough to push a substantial portion of students above the MMC minimum thresholds. The MMC requirements are arbitrary; they were chosen because policymakers believed that they represented a high level of high school science learning. So in one sense, the fact that many students did not meet the threshold says nothing about the quality of their high school science experiences. On the other hand, we see that the policy was relatively ineffective at achieving its goal, and if policymakers hoped to use a similar policy to make improvements in high school science experiences, they should temper their expectations. This is relevant for other science education reforms, such as the NGSS. If schools are not well-resourced, do not have the will to make changes, or must balance high reform expectations with many other competing demands then the hope for systemic change may be misplaced. The MMC also increased the number of students majoring in STEM fields, especially engineering. The IV estimates indicate that the driving factor behind these changes may have been that more students were taking physics. The benefits also extended to graduation with engineering degrees. Given the number of students in the state who were still not taking chemistry or physics after the MMC was adopted, there may be room for even more growth in engineering majors if more students are given the opportunity and encouraged to take physics. However, although the heterogeneity analyses indicated that many of the effects on science course-taking – increases in chemistry and physics course-taking in particular – were primarily among the lowest-achieving students, the benefits of the policy in terms of college majors and degrees were seen amongst the groups of students that already benefit the most from STEM environments in high schools. White, Asian, male, and high-achieving students increased their 127 postsecondary STEM outcomes while other groups who are traditionally underrepresented in STEM failed to make up any ground despite the shifts in their high school outcomes. In terms of school characteristics, we see similar results. Schools where students did not traditionally take chemistry and physics courses saw gains in the share of students who did after the MMC took effect. However, the gains in postsecondary STEM majors and degrees were seen mostly by those schools where many students were already taking such classes. Their relative advantage in science course-taking may have allowed students to focus on other areas as well, for example increasing the level of math courses they took, which could have helped them in later STEM endeavors. The IV estimates show that taking high school physics can positively affect whether students take physics courses in college and whether they choose engineering majors. Taking chemistry courses in high schools can affect college physics grades. Thus, although many of the postsecondary benefits that were seen after the MMC were from students in schools with high achievement, low poverty, and pre-existing positive STEM outcomes, widespread adoption of chemistry and physics courses could lead to more positive postsecondary outcomes in engineering and physics. Limitations Despite gaining a clearer picture about the content of courses from transcripts and course catalogs than we would be able to do with transcripts alone, there is still much we do not observe about what actually happens in the classrooms where science is taking place. It is one thing to call a course chemistry and to write a short paragraph describing the topic the course will cover. It is another thing to have a qualified teacher in the classroom teaching chemistry, providing meaningful activities that cover course content, and encouraging students to use scientific 128 practices to understand phenomena. This study measures the effects of taking chemistry and physics courses without knowing anything about their quality. A policy like the MMC might have a much greater impact on distal outcomes like college majors and degrees if it is accompanied by resources to help schools ensure that their course offerings are helping student learn science. In this study, we cannot know the quality of courses and thus cannot speak to the effects such a policy might have. In this study I have taken a fairly narrow view of what constitutes a science course, choosing for simplicity to adopt the definitions produced by NCES in the SCED. However, during the time period that this study covers it was becoming increasingly common for schools to offer courses such as Forensic Science and Computer Science as science courses. I have chosen to classify these as CTE courses, as the SCED does, so as not to have to make my own distinctions about what counts as science and what does not. The consequence of this decision is that in some schools it may appear that students are taking less science than they are getting credit for. This has implications for how students meet the MMC standards if schools are counting a course like Forensic Science for the third science credit needed for graduation. Likewise, many schools offered agricultural science courses that best fit under a CTE category in the SCED. In some cases the distinctions were quite minor (e.g. Plant Science versus Plant Biology – the former would be a CTE course while the latter would be a science course even though the only difference might be how a school chose to name the course). Course descriptions from the course catalogs helped in this decision-making process, although classification of all CTE courses was beyond the scope of this project. This study also only looks at two cohorts of post-policy data, limiting the measurement to relatively short-term impacts. Although schools had time between when the policy was 129 announced and introduced to do things like reshape course sequences, hire qualified staff, and collect lab materials, these things are easier said than done especially when considering all of the other demands placed on schools. It is very possible that it took several years for some schools to comply with the MMC, so longer term outcomes might show different results. Conversely, schools may have abandoned or changed their efforts to meet MMC science requirements after a few years as they experienced the difficulties of implementing the policy or as school conditions changed. The MMC change occurred around the time of the economic downturn of 2009. It is impossible to attribute the changes we see in these results to the MMC without considering the effect other factors may have had to confound those results. Although I have controlled for economic factors in the models that look at postsecondary outcomes, it is possible that changes in major choices and degree earned were due to students choosing those fields due to economic conditions (or other unknown factors). Basically, although I have taken care to account for such circumstances, the results in this study should be interpreted with caution if being used to make causal claims about course-taking policies. Likewise, the MMC was a policy that affected not only science course-taking, but math course-taking and course-taking in other academic areas. To attribute changes that were due to the MMC as a whole to just the science course-taking portion of the policy is not warranted. Thus, the findings related to the postsecondary STEM outcomes may or may not be due to changes in high school science course-taking. It is also likely that math course-taking, which Kim (2018) and Kim et al. (2019) have shown were affected by the MMC policy, played a role in changing students’ postsecondary outcomes. 130 Understanding the Effects of Taking Science Courses School science is the main way many young people are exposed to science ideas, even though there are many out-of-school opportunities for students to experience science as well. Without norms or policies in place that mean students actually take science courses in high school, we cannot expect that students are gaining a full array of science skills and knowledge. Students should take science throughout their high school careers—a rationale for increasing the number of credits required—and they should take a variety of classes—including “higher level” physical science classes that some students miss out on—so that they are exposed to the variety of content we find important enough to include in standards like the NGSS. High school science coursework is also a relatively easy policy lever to manipulate, but we need to have a clear understanding of how effective using such a lever can be for improving outcomes we care about. If increasing the number or rigor of the credits students need to graduate from high school is not achieving what we hope for, then our focus should be on other policies, like those that target quality of instruction or teacher quality and diversity. If it is not enough to simply require students to take certain courses, then we need to think more about what happens in classrooms that can affect outcomes we deem important. My results show that there were some shifts toward chemistry and physics classes, but that the longer-term consequences of these shifts were felt mostly by the groups that have traditionally benefitted from science instruction. Again, it matters what happens in science classrooms, and getting students into the “right” high school science classes is not enough to affect meaningful change. Without emphasizing equitable, high-quality instructional practices in science, a policy like the MMC will continue to perpetuate the inequality that already exists in the physical sciences. 131 Exposure to courses like chemistry and physics have important implications for students’ opportunities to learn and develop interest in science. They also offer important contexts for students to develop scientific literacy that is vital for navigating our world. Because I could not measure these important outcomes of science course-taking, we do not know how course-taking policies might affect them. Is it better for students to learn about antibiotic resistance in a Biology II course than it is for them to learn about climate change in an environmental science course? It is unclear. Is a student’s ability to formulate scientific explanations better developed in a chemistry class than in an earth science class? Not necessarily. Although learning science in the context of biology, chemistry, and physics is important for students’ understanding of how the world works, more important than any particular content is the way science is taught. The MMC does nothing to ensure that when students take a course like chemistry or physics that they are receiving high-quality science instruction. There is just the assumption that because chemistry and physics are “more rigorous” then they must be better. The sorts of policies that seek to change students’ experiences in science beyond just which courses they take, though, can’t work if students are not in science classes in the first place, so there may be a place for policies that aim to increase the courses all students are able to access. Unfortunately, the MMC did not increase the number of credits students took in science, but if that were the goal it could have been written so as to require more science. Another complication is that there are a limited number of hours in the day for students to take science given all the other academic requirements they face. Again, this points toward polices that improve the quality of science instruction that students do receive rather than trying to increase the amount or vague notions of rigor in science. 132 The IV estimates indicated that there may be some benefit to having students take high school chemistry and physics on postsecondary STEM outcomes. The estimates were not as robust as the previous literature would lead one to believe. Taken with the observed effects of the coursework policy change, though, it would seem that there are many benefits to taking chemistry and physics in high school, especially for students who may have interest in pursuing engineering. Understanding How School Characteristics Moderate Effects The school environment plays a role in what courses students have access to and how they experience those courses. Coursework policies might change access, but how that access (or lack of access depending on the implementation of the policy in schools) translates into effects on postsecondary outcomes may depend on the interaction of school characteristics and the policy. Two schools that both have a large proportion of students taking a course like physics may have very different features that moderate how effective taking a physics course is for postsecondary outcomes. In this dissertation I showed that the highest-SES and highest average achievement schools drove the gains in STEM majors. However, schools at both ends of the spectrum of pre- policy chemistry and physics course-taking saw benefits when it came to increasing the proportion of students studying STEM in college. There may be two things at play here. First, there may be benefits to taking chemistry and physics, as shown in the IV estimates, which those schools that had low levels of course-taking before the policy benefitted from. Second, schools that were already well-resourced in chemistry and physics, or well-resourced generally, may have been able to better implement new courses or extend the courses they did offer to more 133 students. In sum, it appears that a variety of schools, based on the characteristics studied here, can benefit from increasing student course-taking in science disciplines. In the current science policy environment, issues of implementation of policy are critical. In the case of NGSS the implementation issues are different than in a course-taking policy like the MMC, but some lessons are applicable to the NGSS example. The lackluster uptake of the MMC policy across school characteristics means that we must consider how widespread uptake of a policy like NGSS will be. To a certain extent, schools must have felt that what they were doing was all they could do given their resources, that what they were doing was adequate and that the policy did not apply to them, or they implemented it in a way that did not align with the policy’s intentions. There is no reason to believe that another policy that affects science courses, especially one as complex as NGSS, would be any different. Policymakers must decide how they will encourage and support schools in adopting the NGSS, especially in schools where science teaching positions are hard to fill and where student course-taking in science may not be a top academic priority. This dissertation shows that changes in course-taking can have implications for students’ science-related outcomes. It is important with other policies to consider how positive effects can be maximized to reach the most students. 134 REFERENCES 135 REFERENCES Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college. Washington, DC: U.S. Department of Education. Altonji, J. G. (1995). The effects of high school curriculum on education and labor market outcomes. The Journal of Human Resources, 30(3), 409-438. Altonji, J. G., Blom, E., & Meghir, C. (2012). Heterogeneity in human capital investments: High school curriculum, college major, and careers. Annual Review of Economics, 4, 185-223. doi: 10.1146/annurev-economics-080511-110908 Angrist, J. D., & Pischke, J. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton, NJ: Princeton University Press. Arce-Trigatti, P. (2018). The impact of state-mandated Advanced Placement programs on student outcomes. Economics of Education Review, 63, 180-193. doi:10.1016/j.econedurev.2018.02.001 Attewell, P., & Domina, T. (2008). Raising the bar: Curricular intensity and academic performance. Educational Evaluation and Policy Analysis, 30(1), 51-71. doi:10.3102/0162373707313409 Bottia, M. C., Stearns, E., Mickelson, R. A., Moller, S., & Parker, A. D. (2015). The relationship among high school STEM learning experiences and students’ intent to declare and declaration of a STEM major in college. Teachers College Record, 117, 1-46. Buddin, R., & Croft, M. (2014). Do stricter high school graduation requirements improve college readiness? (ACT Working Paper No. 2014-1). Iowa City, IA: ACT Inc. Burkam, D. T., & Lee, V. E. (2003). Mathematics, foreign language, and science coursetaking and the NELS:88 transcript data (Working Paper No. 2003-01). Washington, DC: National Center for Education Statistics. Chen, X. (2014). STEM attrition: College students’ paths into and out of STEM fields (NCES 2014-001). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Chen, X., & Weko, T. (2009). Students who study science, technology, engineering, and mathematics (STEM) in postsecondary education (NCES 2009-161). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. 136 Cogan, L. S., Schmidt, W. H., & Wiley, D. E. (2001). Who takes what math and in which track? Using TIMSS to characterize U.S. students’ eighth-grade mathematics learning opportunities. Educational Evaluation and Policy Analysis, 23(4), 323-341. doi:10.3102/01623737023004323 Conley, T. G., Hansen, C. B., & Rossi, P. E. (2012). Plausibly exogenous. The Review of Economics and Statistics, 94(1), 260-272. Darolia, R., Koedel, C., Main, J. B., Ndashimye, F., & Yan, J. (2018). High school course access and postsecondary STEM enrollment and attainment. (CALDER Working Paper 186). Washington, DC: CALDER. Desilver, D. (2017). U.S. students’ academic achievement still lags the of their peers in many other countries. Pew Research Center. Retrieved from https://www.pewresearch.org/fact- tank/2017/02/15/u-s-students-internationally-math-science/ Domina, T., Penner, A., Penner, E., & Conley, A. (2014). Algebra for all: California’s eight- grade algebra initiative as constrained curricula. Teachers College Record, 116(8), 1-32. Dougherty, C., Mellor, L., & Jian, S. (2006). The relationship between advanced placement and college graduation. Austin, TX: The National Center for Educational Accountability. Dynarski, S., Frank, K., Jacob, B., & Schneider, B. (2012). Impacts of the Michigan Merit Curriculum on student outcomes: Preliminary findings from the first cohort. Michigan Consortium for Educational Research. Retrieved from http://michiganconsortium.org/downloads/brief-2-impacts-of-michigan-merit-curriculum- student-outcomes-preliminary-findings-from-first-cohort.pdf Dynarski, S., M., Hemelt, S. W., & Hyman, J. M. (2015). The missing manual: using National Student Clearinghouse data to track postsecondary outcomes. Educational Evaluation and Policy Analysis, 37(1_suppl), 53S–79S. doi:10.3102/0162373715576078 Education Commission of the States. (2016). Advanced Placement: All high schools/districts required to offer AP. Denver, CO: Education Commission of the States. Retrieved from http://ecs.force.com/mbdata/MBQuestRT?Rep=AP0116 Education Commission of the States. (2019). High school graduation requirements: What are the state’s course requirements for high school graduation? Denver, CO: Education Commission of the States. Retrieved from https://c0arw235.caspio.com/dp/b7f930000e16e10a822c47b3baa2 Engberg, M. E., & Wolniak, G. C. (2013). College student pathways to the STEM disciplines. Teachers College Record, 115, 1-27. Federman, M. (2007). State graduation requirements, high school course taking, and choosing a technical college major. The B. E. Journal of Economic Analysis & Policy, 7(1), 1-32. 137 Hallinan, M. T. (1988). Equality of educational opportunity. Annual Review of Sociology, 14(1), 249–268. doi:10.1146/annurev.so.14.080188.001341 Harlow, J. J. B., Harrison, D. M., & Meyertholen, A. (2014). Correlating student interest and high school preparation with learning and performance in an introductory university physics course. Physical Review – Physics Education Research, 10, 1-9. doi:10.1103/PhysRevSTPER.10.010112 Ho, P., & Kao, G. (2018). Educational achievement and attainment differences among minorities and immigrants. In B. Schneider (Ed.), Handbook of sociology of education in the 21st century (pp. 109-129). Cham, Switzerland: Springer. Horn, L., & Kojaku, L. K. (2001). High school academic curriculum and the persistence path through college: Persistence and transfer behavior of undergraduates 3 years after entering 4-year institutions. (NCES 2001-163). Washington, DC: National Center for Education Statistics. Ingersoll, R. M., & May, H. (2012). The magnitude, destinations, and determinants of mathematics and science teacher turnover. Educational Evaluation and Policy Analysis, 34(4), 435-464. Ingersoll, R. M., & Perda, D. (2010). Is the supply of mathematics and science teachers sufficient? American Educational Research Journal, 47(3), 563-594. Jacob, B., Dynarski, S., Frank, K., & Schneider, B. (2017). Are expectations alone enough? Estimating the effect of a mandatory college-prep curriculum in Michigan. Educational Evaluation and Policy Analysis, 39(2), 333-360. doi:10.3102/0162373716685823 Jennings, J., Deming, D., Jencks, C., Lopuch, M., & Schueler, B. (2015). Do differences in school quality matter more than we thought? New evidence on educational opportunity in the 21st century. Sociology of Education, 88(1), 56–82. doi:10.1177/0038040714562006 Johnson, A. (2019). A matter of time: Variations in high school course-taking by years-as-EL subgroup. Educational Evaluation and Policy Analysis. Advance online publication. doi:10.3102/0162373719867087 Kelly, S. (2009). The Black–White gap in mathematics course taking. Sociology of Education, 82(1), 47–69. doi:10.1177/003804070908200103 Kim, S. (2018). Return to Algebra II: The effect of mandatory math coursework on postsecondary attainment. Working paper. East Lansing, MI: Michigan State University. Kim, S., Wallsworth, G., Xu, R., Schneider, B., Frank, K., Jacob, B., & Dynarski, S. (2019). The impact of the Michigan Merit Curriculum on high school math course-taking. Educational Evaluation and Policy Analysis, 41(2), 164-188. doi:10.3102/0162373719834067 138 Legewie, J., & DiPrete, T. A. (2014). The high school environment and the gender gap in science and engineering. Sociology of Education, 87(4), 259-280. doi:10.1177/0038040714547770 Levine, P. B., & Zimmerman, D. J. (1995). The benefit of additional high-school math and science classes for young men and women. Journal of Business & Economic Statistics, 13(2), 137-149. Lillard, D. R., & CeCicca, P. P. (2001). Higher standards, more dropouts? Evidence within and across time. Economics of Education Review, 20(5), 459-473. doi:10.1016/S0272- 7757(00)00022-4 Long, M. C., Conger, D., & Iatarola, P. (2012). Effects of high school course-taking on secondary and postsecondary success. American Educational Research Journal, 49(2), 285-322. doi:10.3102/0002831211431952 Long, M. C., Conger, D., & McGhee, R. (2019). Life on the frontier of AP expansion: Can schools in less-resourced communities successfully implement Advanced Placement science courses? Educational Researcher, 48(6), 356-368. doi:10.3102/0013189X19859593 Main, Darolia, Koedel, Yan, & Ndashimye (2017). The role of high school math and science course access in student college engineering major choice and degree attainment. Paper presented at American Society for Engineering Education. Maltese, A. V., & Tai, R. H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among U.S. students. Science Education, 95(5), 877–907. doi:10.1002/sce.20441 Mann, A., & DiPrete, T. (2013). Trends in gender segregation in the choice of science and engineering majors. Social Science Research, 42(6), 1519–1541. doi:10.1016/j.ssresearch.2013.07.002 Maple, S. A., & Stage, F. K. (1991). Influences on the choice of math/science major by gender and ethnicity. American Educational Research Journal, 28(1), 37-60. Mattern, K. D., Shaw, E. J., & Ewing, M. (2011). Advanced Placement exam participation: Is AP exam participation and performance related to choice of college major? (Research Report 2011-6). New York, NY: College Board. McDonnell, L. (1995). Opportunity to learn as a research concept and a policy instrument. Educational Evaluation and Policy Analysis, 17(3), 305-322. doi:10.3102/01623737017003305 MI Comp L § 380.1278b (2015). Award of high school diploma; credit requirements; personal curriculum; annual report. Lansing, MI: Michigan Legislature. 139 Michelmore, K., & Dynarski, S. (2017). The gap within the gap: Using longitudinal data to understand income differences in student achievement. AERA Open, 3(1), 1-18. Doi: 10.1177/2332858417692958 Montgomery, N., & Allensworth, E.M. (2010). Passing through science: The effects of raising graduation requirements in science on course-taking and academic achievement in Chicago. Chicago: Consortium on Chicago School Research. Muller, C., Riegle-Crumb, C., Schiller, K. S., Wilkinson, L., & Frank, K. A. (2010). Race and academic achievement in racially diverse high schools: Opportunity and stratification. Teachers College Record, 112(4), 1038-1063. Murnane, J. B., & Willett, R. (2011). Methods matter. New York, NY: Oxford University Press. National Commission on Excellence in Education. (1983). A Nation at Risk. Washington, DC: U.S. Department of Education. National Science Board. (2019). Science & engineering indicators 2020. Alexandria, VA: National Science Foundation. National Science Foundation. (2019). Women, minorities, and persons with disabilities in science and engineering. Washington, DC: National Science Foundation. Nevo, A., & Rosen, A. M. (2012). Identification with imperfect instruments. The Review of Economics and Statistics, 94(3), 659-671. Ost, B. (2010). The role of peers and grades in determining major persistence in the sciences. Economics of Education Review, 29(6), 923-934. doi:10.1016/j.econedurev.2010.06.011 Plunk, A. D., Tate, W. F., Bierut, L. J., & Grucza, R. A. (2014). Intended and unintended effects of state-mandated high school science and mathematics course graduation requirements on educational attainment. Educational Researcher, 43(5), 230-241. doi:10.3102/0013189X14540207 Riegle-Crumb, C. (2006). The path through math: Course sequences and academic performance at the intersection of race-ethnicity and gender. American Journal of Education, 113(1), 101-122. Riegle-Crumb, C. & Grodsky, E. (2010). Racial-ethnic differences at the intersection of math course-taking and achievement. Sociology of Education, 83(3), 248-270. doi:10.1177/0038040710375689 Riegle-Crumb, C., & King, B. (2010). Questioning a White male advantage in STEM: Examining disparities in college major by gender and race/ethnicity. Educational Researcher, 39(9), 656–664. doi:10.3102/0013189X10391657 140 Riegle-Crumb, C., King, B., Grodsky, E., & Muller, C. (2012). The more things change, the more they stay the same? Prior achievement fails to explain gender inequality in entry to STEM college majors over time. American Educational Research Journal, 49(6), 1048– 1073. doi:10.3102/0002831211435229 Riegle-Crumb, C., Kyte, S. B., & Morton, K. (2018). Gender and racial/ethnic differences in educational outcomes: Examining patterns, explanations, and new directions for research. In B. Schneider (Ed.), Handbook of sociology of education in the 21st century (pp. 131- 152). Cham, Switzerland: Springer. Riegle-Crumb, C., & Moore, C. (2014). The gender gap in high school physics: Considering the context of local communities. Social Science Quarterly, 95(1), 253–268. doi:10.1111/ssqu.12022 Riegle-Crumb, C., Muller, C., Frank, K., & Schiller, K. (2005). National Longitudinal Study of Adolescent Health: Wave III education data. Chapel Hill, NC: Carolina Population Center, University of North Carolina at Chapel Hill. Rodriguez, A., & McGuire, K. M. (2019). More classes, more access? Understanding the effects of course offerings on Black-White gaps in Advanced Placement course-taking. The Review of Higher Education, 42(2), 641-679. Rose, H. & Betts, J. R. (2004). The effect of high school courses on earnings. The Review of Economics and Statistics, 86(2), 497-513. Russell, M. L., & Atwater, M. M. (2005). Traveling the road to success: A discourse on persistence throughout the science pipeline with African American students at a predominantly white institution. Journal of Research in Science Teaching, 42(6), 691- 715. doi:10.1002/tea.20068 Sadler, P. M., Sonnert, G., Hazari, Z., & Tai, R. (2014). The role of advanced high school coursework in increasing STEM career interest. Science Educator, 23(1), 1–13. Sadler, P. M., & Tai, R. H. (1997). The role of high-school physics in preparing students for college physics. The Physics Teacher, 35, 282-285. Sadler, P. M., & Tai, R. H. (2001). Success in introductory college physics: The role of high school preparation. Science Education, 85, 111-136. Sadler, P. M., & Tai, R. H. (2007). The two high school pillars supporting college science. Science, 317, 457-458. doi:10.1126/science.1144214 Salehi, S., Burkholder, E., Lepage, G. P., Pollock, S., & Wieman, C. (2019). Demographic gaps or preparation gaps?: The large impact of incoming preparation on performance of students in introductory physics. Physical Review - Physics Education Research, 15(2), 1-17. doi:10.1103/PhysRevPhysEducRes.15.020114 141 Saw, G. (2016). Three essays on estimating the effects of school and student improvement interventions. (Unpublished doctoral dissertation). Michigan State University: East Lansing, MI. Schmidt, W., & Maier, A. (2009). Opportunity to learn. In G. Sykes, B. Schneider, & D. Plank (Eds.), Handbook of education policy research (pp. 541-559). New York: Routledge. Schneider, B., Swanson, C. B., & Riegle-Crumb, C. (1998). Opportunities for learning: Course sequences and positional advantages. Social Psychology of Education, 2, 25-53. Schwartz, M. S., Hazari, Z., & Sadler, P. M. (2008). Divergent views: Teacher and professor perceptions about pre-college factors that influence college science success. Science Educator, 71(1), 18-35. Schwartz, M. S., Sadler, P. M., Sonnert, G., & Tai, R. H. (2009). Depth versus breadth: How content coverage in high school science courses relates to later success in college science coursework. Science Education, 93, 798-826. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. Somers, M., Zhu, P., Jacob, R., & Bloom, H. (2013). The validity and precision of the comparative interrupted time series design and the difference-in-difference design in educational evaluation. New York, NY: MDRC. Stevenson, D. L., Schiller, K. S., & Schneider, B. (1994). Sequences of opportunities for learning. Sociology of Education, 67(3), 184-198. Tai, R. H., & Sadler, P. M. (2007). High school chemistry instructional practices and their association with college chemistry grades. Journal of Chemical Education, 84(6), 1040- 1046. Tai, R. H., Sadler, P. M., & Loehr, J. F. (2005). Factors influencing success in introductory college chemistry. Journal of Research in Science Teaching, 42(9), 987-1012. doi: 10.1002/tea.20082 Tai, R. H., Ward, R. B., & Sadler, P. M. (2006). High school chemistry content background of introductory college chemistry students and its association with college chemistry grades. Journal of Chemical Education, 83(11), 1703-1711. Tate, W. (2001). Science education as a civil right: Urban schools and opportunity-to-learn considerations. Journal of Research in Science Teaching, 38(9), 1015-1028. doi:10.1002/tea.1045 142 Trusty, J. (2002). Effects of high school course-taking and other variables on choice of science and mathematics college majors. Journal of Counseling & Development, 80(4), 464-474. doi:10.1002/j.1556-6678.2002.tb00213.x Tyson, W., Lee, R., Borman, K. M., & Hanson, M. A. (2007). Science, technology, engineering, and mathematics (STEM) pathways: High school science and math coursework and postsecondary degree attainment. Journal of Education for Students Placed at Risk, 12(3), 243-270. doi:10.1080/10824660701601266 U.S. Department of Education (2007). Secondary school course classification system: School codes for the exchange of data (SCED) (NCES 2007-341). Washington, DC: U.S. Department of Education, National Center for Education Statistics. U.S. Department of Education, National Center for Education Statistics. (2018). Table 225.30: Percentage of public and private high school graduates taking selected mathematics and science courses in high school, by sex and race/ethnicity: Selected years, 1982 through 2009. In Digest of Education Statistics (2018 ed.). U.S. Department of Education, National Center for Education Statistics. Retrieved from https://nces.ed.gov/programs/digest/d18/tables/dt18_225.30.asp U.S. Department of Education, Office of Civil Rights. (2018). 2015-2016 civil rights data collection: STEM course taking. Washington, DC: U.S. Department of Education, Office of Civil Rights. Wallsworth, G., Kim, S., & Troutman, R. (2015). Transcript data coding manual. East Lansing, MI: Michigan Consortium for Educational Research. Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of support. American Educational Research Journal, 50(5), 1081– 1121. doi:10.3102/0002831213488622 Warne, R. T. (2017). Research on the academic benefits of the Advanced Placement program: Taking stock and looking forward. SAGE Open, 7(1), 1-16. doi:10.1177/2158244016682996 White House. (2016). STEM for all. Retrieved from https://obamawhitehouse.archives.gov/blog/2016/02/11/stem-all Xie, Y., & Shauman, K. (2003). Women in science: Career processes and outcomes. Cambridge, MA: Harvard University Press. 143