SUPPORTING MULTIPLE PATHS TO SUCCESS: A FIELD EXPERIMENT EXAMINING A MULTIFACETED, MULTILEVEL MOTIVATION INTERVENTION By Kristy Amber Robinson A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Educational Psychology and Educational Technology – Doctor of Philosophy 2019 ABSTRACT SUPPORTING MULTIPLE PATHS TO SUCCESS: A FIELD EXPERIMENT EXAMINING A MULTIFACETED, MULTILEVEL MOTIVATION INTERVENTION By Kristy Amber Robinson This randomized field experiment examined the effectiveness of a multifaceted, multilevel motivation intervention in a large, introductory engineering course for undergraduates. At the level of the individual, students (n = 682) were randomly assigned to complete a variety of intervention or control activities, including interventions designed to promote feelings of belonging, incremental theories of intelligence (the belief that abilities can grow with effort), and utility value (relevance or usefulness) for engineering coursework. Course instructors (n = 8) were randomly assigned to learn about strategies for supporting students’ motivation in the treatment condition or to learn about knowledge development in an active control condition. The study employed a 2 (TA training vs. control) x 2 (student-level utility value vs. control) x 3 (student-level incremental, belonging, or control) design to examine the main and interactive effects of the single and combined interventions. Random assignment resulted in individual students participating in up to three intervention conditions, or in up to three control conditions. Outcome measures included proximal outcomes assessed at the end of the semester (motivation and course grades) and distal measures of engineering identity, engineering major retention, and GPA assessed at the end of the following semester. Interactions with prior achievement were also examined to determine whether the intervention effects were stronger for low-achieving students. Overall, there were no statistically significant effects of interventions on the outcome measures, compared to control conditions, and no significant moderating effects based on prior achievement. Furthermore, fidelity and manipulation checks suggested that while the utility value intervention was successfully implemented, non-significant effects of the interventions may have been attributable to implementation limitations and existing motivational supports within the course. However, results point to the feasibility of multifaceted motivation interventions, which can be co-designed with teachers to leverage the complex, dynamic nature of motivation as it occurs with individuals and contexts. To Jared and the children, my wings and anchors iv ACKNOWLEDGMENTS There are many people who were instrumental in my educational journey, too many to mention or adequately thank here. My village consists of my family, steadfast friends and colleagues, and incredible mentors who have all carried me through moments of self-doubt and triumph alike. Firstly and always, I am indebted to my husband Jared. I can only strive daily to deserve you and to reflect your vision of me as the best version of myself. Thank you for so enthusiastically believing in me. Secondly, people often ask me how I could pursue graduate work with four children at home. While of course it can be a challenge to balance academic and family life, my answer to this question turns it on its head: Thomas, Kimball, Amelia, and Calvin, you are my greatest source of inspiration and motivation to pursue ambitious, worthwhile goals. You keep me focused on the important things and you make me so excited to come home every day. To my parents and parents-in-law, my siblings and siblings-in-law, and my extended family (looking at you, Marcell): every person should have such a cheering section. Mom and Dad, thank you for working so hard to provide me with the privileges I needed to set and achieve my goals. My successes in work and life are built on yours. Many friends and neighbors have given me more support than I’ve deserved and much more than I can describe here. I must thank here my friend Mandy for sharing so much of my journey and for showing me how it’s done with grace and resilience. In addition, my lab colleagues sustain my belief that I have found my people in academic work. In particular: John, thank you for your unfailing kindness and care, for making me feel I belong, and for making everyday research so fun. Stephanie, thank you for simultaneously setting the standard for excellence while being an incredibly humane, attentive mentor and friend. Patrick, thank you for making me laugh often and for driving me to try and keep up with you. I have been privileged to receive support from stellar academic mentors. To Stefinee, Gary, and Pat, who have smoothed my path for the past sixteen years: thank you for using your kind-hearted brilliance to provide encouragement, opportunities, and guidance at exactly the right times. In addition, v my dissertation committee is a dream team, each of whom has uniquely shaped my scholarly development. Thank you for instilling in me a commitment to precision in theory and methods, to innovation in service of students and teachers, and to generosity in mentorship. Lastly, I will be forever grateful to my advisor Lisa. I have enjoyed our time together immensely. Thank you for providing me with so many opportunities to stretch my skills, for teaching me how to think and write, and for moving mountains on my behalf. Your training gives me the confidence and the tools to aim high, and to me, aiming high will always involve striving to live up to your example. vi TABLE OF CONTENTS LIST OF TABLES ........................................................................................................................................ x LIST OF FIGURES ..................................................................................................................................... xi CHAPTER 1: Introduction ........................................................................................................................... 1 CHAPTER 2: Literature Review .................................................................................................................. 5 How to Support Motivation: Theoretical Framework ................................................................................... 5 Expectancy-Value Theory ............................................................................................................... 6 Achievement Goal Theory ............................................................................................................... 8 Self-Determination Theory .............................................................................................................. 9 Integrative Principles for Supporting Motivation .......................................................................... 10 Motivation Interventions ............................................................................................................................. 13 Student-Level Interventions ........................................................................................................... 15 Single construct interventions. ............................................................................................... 15 Multi-construct interventions. ................................................................................................ 19 Teacher-Level Interventions .......................................................................................................... 21 Single-construct interventions................................................................................................ 21 Multi-construct interventions. ................................................................................................ 23 Implications for Complex, Multilevel Motivation Interventions ................................................... 25 Present Study .............................................................................................................................................. 26 RQ 1: What are the main effects of various motivation interventions? ......................................... 29 RQ 1a: How do brief social psychological interventions affect value, perceived competence, achievement goals, engineering identity, achievement, and persistence in engineering? ................... 29 RQ 1b: Do students whose instructors are trained to support motivation have higher value, perceived competence, mastery goals, engineering identity, achievement, and persistence in engineering, and lower performance goals?........................................................................................ 30 RQ 1c: Does prior achievement moderate the main effects of each intervention? ............... 31 RQ 2: What are the combined effects of a complex motivation intervention? .............................. 31 RQ 2a: Do student-level interventions targeting theories of intelligence and value combine to show additive or interactive effects on value, perceived competence, achievement goals, achievement, engineering identity, and persistence in engineering? .................................................. 31 RQ 2b: Do student-level and teacher-level motivation interventions combine to show additive, specialized, or synergistic effects on value, perceived competence, mastery goals, achievement, engineering identity, and persistence in engineering? .................................................. 32 CHAPTER 3: Method ................................................................................................................................. 35 Setting and Participants ............................................................................................................................... 35 Recruitment and Eligibility ............................................................................................................ 35 Procedure .................................................................................................................................................... 36 Assignment to Conditions .............................................................................................................. 36 Survey Administration ................................................................................................................... 37 Student surveys. ..................................................................................................................... 37 TA surveys. ............................................................................................................................ 38 Intervention Procedures ................................................................................................................. 38 Student-level interventions. ................................................................................................... 38 TA intervention. ..................................................................................................................... 39 vii Measures ..................................................................................................................................................... 41 TA Assessments ............................................................................................................................. 41 Personal teaching efficacy. .................................................................................................... 42 Teacher motivation self-efficacy. .......................................................................................... 42 Design principles self-efficacy............................................................................................... 42 Value for teaching. ................................................................................................................. 42 Student Survey Assessments.......................................................................................................... 42 Task value. ............................................................................................................................. 43 Engineering self-efficacy and academic perceived competence. ........................................... 43 Achievement goals. ................................................................................................................ 43 Theories of intelligence. ......................................................................................................... 43 Engineering identity. .............................................................................................................. 43 Belonging in engineering. ...................................................................................................... 43 Motivational climate. ............................................................................................................. 44 Observational Assessments ............................................................................................................ 44 Institutional Data ............................................................................................................................ 45 Achievement and persistence. ................................................................................................ 45 Prior achievement. ................................................................................................................. 45 Analytic Plan ............................................................................................................................................... 45 Power analyses. ...................................................................................................................... 46 CHAPTER 4: Results ................................................................................................................................. 47 Preliminary Analyses .................................................................................................................................. 47 Confirmatory Factor Analyses ....................................................................................................... 47 Theories of intelligence. ......................................................................................................... 47 Belonging. .............................................................................................................................. 47 Motivational climate. ............................................................................................................. 47 Motivation and identity. ......................................................................................................... 48 Missing Data .................................................................................................................................. 49 Participant flow. ..................................................................................................................... 49 Missing data analysis. ............................................................................................................ 50 Multiple imputation. .............................................................................................................. 52 Descriptive Statistics and Correlations .......................................................................................... 52 Fidelity and Manipulation Checks ................................................................................................. 53 Randomization check. ............................................................................................................ 53 Teacher training. .................................................................................................................... 54 Utility value. .......................................................................................................................... 55 Mindsets. ................................................................................................................................ 56 Belonging. .............................................................................................................................. 56 Summary of fidelity and manipulation checks. ..................................................................... 57 Main and Interactive Effects of Motivation Interventions .......................................................................... 57 Moderation Analyses ..................................................................................................................... 59 CHAPTER 5: Discussion ............................................................................................................................ 60 Main Effects of Interventions ........................................................................................................ 61 Utility value intervention. ...................................................................................................... 61 Incremental and belonging interventions. .............................................................................. 63 TA intervention. ..................................................................................................................... 64 Interactive Effects of Motivation Interventions ............................................................................. 65 Implications ................................................................................................................................... 66 Limitations ..................................................................................................................................... 68 viii Conclusion .................................................................................................................................................. 71 APPENDICES ............................................................................................................................................ 73 APPENDIX A: Tables ................................................................................................................................ 74 APPENDIX B: Figure ................................................................................................................................. 90 APPENDIX C: Study Measures ................................................................................................................. 92 APPENDIX D: Observation Instrument ..................................................................................................... 98 APPENDIX E: Results After Removing Multivariate Outliers ................................................................ 101 REFERENCES ......................................................................................................................................... 105 ix LIST OF TABLES Table 1: Study Timeline .............................................................................................................................. 75 Table 2: Field Experiment Conditions ....................................................................................................... 76 Table 3: Sample Prompts for Utility Value Treatment and Control Group Writing Assignments ............. 77 Table 4: Hypothetical TA Scenarios for Workshop Self-Assessments ....................................................... 78 Table 5: Construct List and Measurement Occasions ............................................................................... 79 Table 6: Research Questions and Planned Analyses ................................................................................. 80 Table 7: Descriptive Statistics for Study Variables ................................................................................... 81 Table 8: Correlations for Study Variables ................................................................................................. 82 Table 9: Means of Teaching Assistants’ Motivation for Teaching ............................................................ 83 Table 10: Levene’s Test for MANOVA Examining Main and Interactive Effects of Interventions ............ 84 Table 11: Results of Multiple Analysis of Variance ................................................................................... 85 Table 12: Univariate Tests for the Interaction of Utility Value and Belonging/Incremental Conditions .. 86 Table 13: Results of Logistic Regression Analyses Examining Effects of Interventions on Major Retention .................................................................................................................................................... 87 Table 14: Results of MANCOVA Analyses Examining Interactions of Interventions with Prior Achievement ............................................................................................................................................... 88 Table 15: Univariate Tests for the Interaction of Utility Value and TA Conditions with Prior Achievement ............................................................................................................................................... 89 Table E1: Results of Multiple Analysis of Variance Removing Multivariate Outliers ............................. 102 Table E2: Results of Logistic Regression Analyses Examining Effects of Interventions on Major Retention, Multivariate Outliers Removed ................................................................................................................ 103 Table E3: Results of MANCOVA Analyses Examining Interactions of Interventions with Prior Achievement, Multivariate Outliers Removed .......................................................................................... 104 x LIST OF FIGURES Figure 1: Estimated marginal means for belonging component of engineering identity as a function of the interaction between mindset/belonging and utility value interventions.......................................................91 xi CHAPTER 1: Introduction Improving educational outcomes is a priority for the United States, where attrition from college is a national concern, and attrition from science, technology, engineering, and mathematics (STEM) fields is particularly high (Eagan, Hurtado, Figueroa, & Hughes, 2014; Kena et al., 2016). National data indicate that approximately 40% of students who begin pursuing higher education will not graduate within six years (Kena et al., 2016), and low rates of STEM degree attainment (as low as 26-46% among those who initially intended to pursue a STEM field) are a particular concern given the STEM workforce shortage in the United States (Chen & Soldner, 2013; Eagan et al., 2014; Huang, Taddese, & Walter, 2000). Persistence and achievement in science, technology, engineering, and mathematics (STEM) is a focus of national policy and research aimed at building a diverse and vibrant STEM workforce (National Science Board, 2016; National Science Foundation, 2015). One way to address this issue is through understanding and supporting students’ motivation for pursuing STEM. Indeed, observed declines in motivation throughout college (Musu-Gillette, Wigfield, Harring, & Eccles, 2015; Robinson, Lee, et al., 2018), even within a single semester (Kosovich, Flake, & Hulleman, 2017), and low retention rates in college STEM fields suggest that there is a need for greater understanding of how to support motivation in college. Theory and research suggest that supporting student motivation can boost achievement and retention in STEM fields. Indeed, a variety of motivation theories, including expectancy-value theory (Eccles, 1983), achievement goal theory (Ames, 1992; Elliott & Dweck, 1988), and self-determination theory (Ryan & Deci, 2000a), describe the processes by which motivation leads to achievement-related behaviors in STEM, and by which educational contexts can support students’ STEM pursuit via motivation. As described in detail in the literature review, each of these theories contribute overlapping, complementary, and unique insights into how students’ motivation can be supported via specific strategies embedded within instructional contexts, teacher speech, feedback, and assignments. The three theories illuminate which specific motivation constructs are important to focus on and how each of those 1 constructs might be supported by a few common, crosscutting instructional design principles (Linnenbrink-Garcia, Patall, & Pekrun, 2016). Building from theoretical recommendations for supporting motivation, interventions aimed at supporting motivation for students in STEM fields have shown promising results for achievement and retention outcomes (Rosenzweig & Wigfield, 2016). However, many questions are left unanswered. Motivation interventions are designed from various theoretical perspectives and take a variety of approaches, including brief exercises administered directly to students (Cohen, Garcia, Purdie-Vaughns, Apfel, & Brzustoski, 2009; Harackiewicz et al., 2014; Yeager et al., 2014), multifaceted motivation supports integrated throughout curricula (Martin, 2008), and teacher training programs (Cheon & Reeve, 2015; Rubie-Davies, Peterson, Sibley, & Rosenthal, 2015; Turner, Christensen, Kackar-Cam, Trucano, & Fulmer, 2014). In particular, motivation interventions can be grouped according to direct versus indirect (or contextual) approaches: while some interventions intervene directly with students with the express purpose of changing individual psychological beliefs, others take a more indirect approach of attempting to shape motivational beliefs on the individual or group level through modifying the learning context. Direct comparisons of the comparative efficacy of these approaches have not been possible to date, and the promise of combining student-level and contextual approaches is as yet unexplored. Thus, while motivation interventions have a large and growing body of evidence suggesting promising effects for students in STEM, it is difficult to compare effects and theoretical mechanisms across studies, so the mechanisms of motivation interventions are not well understood and effects are not always replicable when administered in a new context. For example, utility value interventions (Harackiewicz et al., 2014) have shown effects on grades and STEM persistence (Canning et al., 2018), and there is some evidence that these effects are mediated via increased engagement with school assignments (Harackiewicz, Canning, Tibbetts, Priniski, & Hyde, 2016) and increased perceptions of value (Hulleman, Godes, Hendricks, & Harackiewicz, 2010); however, effects on value itself are not often measured and are not always found when examined (Hulleman, Kosovich, Barron, & Daniel, 2017). Similarly, interventions designed to boost perceptions of belonging among racial/ethnic minority students 2 (Walton & Cohen, 2007, 2011) show effects on distal outcomes such as achievement and there is some evidence to suggest that perception of belonging account for these differences (Walton & Cohen, 2011); however, the specific mechanisms by which social psychological interventions improve distal outcomes is in need of further investigation. Understanding these mechanisms and the conditions necessary for their success is vital for effectively and efficiently boosting students’ opportunities for success in STEM. Further, current approaches to motivation interventions often do not reflect the theorized complexity of motivation as it occurs within individuals and contexts. The effects of multifaceted interventions are in need of examination, particularly when considering that theory conceptualizes motivation as multifaceted rather than unidimensional (Pintrich, 2003). For example, a central claim of classic expectancy-value theory is that high levels of both expectancy for success and the value of the task are needed for optimal outcomes (Atkinson, 1957; Trautwein et al., 2012); seemingly in contrast to this idea, brief social psychological interventions are often only effective for students with lower prior achievement or lower expectancy for success (Harackiewicz & Priniski, 2017). These effects may be due to brief social psychological interventions actually supporting motivation constructs beyond the one construct each is designed to support; however, this has not been directly tested. Localized effects of brief interventions may also be attributable to the complex motivational processes unfolding outside the intervention itself. This disparity between theory and evidence can be resolved only through direct, repeated testing of these theorized mechanisms. Field studies are needed to examine how principles from theory and lab research can be implemented in real-world classrooms and to understand how multiple forms of motivation can be supported. Rigorous field experiments in authentic settings, such as the present study, will enable important tests of theorized contextual supports for motivation and subsequent outcomes, including assessing whether there is an additive or multiplicative effect of multidimensional interventions as compared to single-construct interventions, or of combining teacher-level and student-level interventions. This dissertation study aimed to test the effectiveness of intervening to support multiple constructs theorized to be distinct but related, and to examine the potential additional effects of training 3 instructors to support motivation in an introductory college STEM course. Specifically, I examined the main and interactive effects of three student-level interventions (belonging, incremental theories of intelligence, and utility value) alongside a teacher-level intervention designed to help course instructors incorporate motivationally supportive practices in their daily instruction. To increase understanding of the outcomes and mechanisms of these interventions, I examined an array of motivational, identity, and achievement-related outcomes. This research adds to theory through a strong test of theorized mechanisms for supporting motivation, with the aim of providing the empirical evidence needed for making concrete recommendations informing STEM teaching practices. 4 CHAPTER 2: Literature Review In this chapter, I outline the theoretical framework for the study that consists of a multitheoretical perspective drawing on expectancy-value theory, achievement goal theory, and self-determination theory. I describe theoretical recommendations for motivation interventions, including which variables might be most effective to intervene on and strategies for supporting those beliefs. Next, I summarize the current state of empirical literature on motivation interventions, including single-construct interventions, multi- construct interventions, student-focused interventions, and teacher-focused interventions. I aim to show how and why a multifaceted, multilevel approach to motivation interventions may be particularly effective in supporting students’ complex motivational needs. How to Support Motivation: Theoretical Framework The present intervention study draws on multiple theories of motivation, taking an integrative approach to supporting students’ motivation. Motivation theories come from diverse traditions and differ in several important ways, including the focal constructs, conceptualizations and definitions of constructs, and the processes deemed to be most important for energizing achievement-related behaviors (Schunk, Meece, & Pintrich, 2014). However, when considering implications for practice, theoretical perspectives largely converge in their recommendations for teachers and classrooms (Linnenbrink-Garcia et al., 2016). Furthermore, an integrative theoretical perspective (Pintrich, 2003) allows researchers and practitioners to leverage the combined evidence of extensive literatures from each theoretical tradition, with particularly strong recommendations for practice emerging from the points of confluence among theoretical perspectives. The interventions in this study were selected as representing an integration of theoretical perspectives. Specifically, three motivation theories guided the choice of which motivational processes might be most salient for students’ achievement and major choice, while also balancing the need for parsimony and cost effectiveness in intervention approaches. Expectancy-value theory (Eccles, 1983; Wigfield & Cambria, 2010), achievement goal theory (Ames, 1992; Elliott & Dweck, 1988), and self-determination theory (Ryan & Deci, 2000b) inform the 5 design of the multifaceted intervention and explain the mechanisms by which the intervention is hypothesized to shift students’ motivation and associated outcomes. These three theories were chosen because they represent the most parsimonious yet comprehensive account of the multiple ways students’ motivation can be activated, supported, and enhanced in the context of interest in this study (introductory college engineering). Other theories of motivation provide insight into which motivation constructs might be most important to target and specific strategies for supporting them (e.g., interest theory: Renninger & Hidi, 2011; attribution theory: Weiner, 1985), but were not chosen because of the considerable overlap in constructs and processes of focus. Broadly, expectancy-value, achievement goal, and self-determination theories provide answers to two fundamental motivational questions: “Can I do this?” and “Why do I want to do this?” As explained in the following sections, the selected interventions target students’ perceptions of value for course content, growth mindset (or the belief that abilities can grow with effort), and feelings of belonging/relatedness at the student level. The focus on value arises primarily from expectancy-value theory, the focus on growth mindset from achievement goal theory, and belonging/relatedness primarily from self-determination theory. At the classroom level, the intervention takes a more integrative perspective and focuses on developing instructors’ strategies for supporting students’ value/interest in course material, perceptions of competence, learning goals, opportunities for autonomy, and belonging/relatedness in the classroom. Expectancy-Value Theory Expectancy-value theory posits that students’ expectancies for success and their beliefs about the value of engaging in achievement tasks are theorized to be the most important proximal predictors of achievement-related behaviors (Wigfield & Cambria, 2010). Students may value a domain/field or a particular task for multiple reasons: because it is intrinsically enjoyable or interesting (interest value), because it is useful to their current or future goals (utility value), or because it is personally important to their identities (attainment value). Conversely, value for tasks may be tempered by students’ perceptions of the costs involved in pursuing those tasks. Specifically, students may perceive that they have to give up 6 other valuable opportunities (opportunity cost), that there is too much work or effort involved in pursuing a task (effort cost), or that it is too stressful (psychological cost). Expectancies for success involve a student’s beliefs about his or her ability to successfully complete a task, such as passing an exam, graduating in a particular major, or pursuing a career in a particular field. Correlational research (Andersen & Ward, 2014; Estrada, Woodcock, Hernandez, & Schultz, 2011; Larson et al., 2015; Perez, Cromley, & Kaplan, 2014) and experimental research (Harackiewicz & Priniski, 2017; Lazowski & Hulleman, 2016; Rosenzweig & Wigfield, 2016) provide strong support for the roles of value and expectancies in supporting STEM achievement and persistence. Furthermore, though the majority of research has focused on parents rather than teachers or schools as key socializers of expectancies and values (e.g., Harackiewicz, Rozek, Hulleman, & Hyde, 2012; Wigfield, Eccles, Schiefele, Roeser, & Davis-Kean, 2006), research suggests that task value can be supported through active involvement in classroom learning, activities that align with student interests, support for autonomy, connections between course material and real life, and increasing feelings of belonging (Guthrie, Wigfield, & VonSecker, 2000; Hulleman & Harackiewicz, 2009; Pugh, Linnenbrink-Garcia, Phillips, & Perez, 2015; Yeager et al, 2014). Most classroom-based research from an expectancy-value perspective has focused on utility value, a relatively extrinsic form of value, perhaps because it is theorized to be the most malleable in response to environmental stimuli (Harackiewicz & Priniski, 2017). Less is known about strategies for supporting attainment value or interest value, although expectancy- value theory and research from other theoretical perspectives, as described later, provide some ideas about how these values may be supported. Similarly, while there is relatively little research thus far about classroom predictors of expectancies, theory and a few studies suggest that the classroom environment can support students’ perceptions of competence through providing appropriate challenge, vicarious success experiences, and providing encouraging, informational feedback (Harter, 1978; Usher & Pajares, 2008; Vallerand & Reid, 1988). In addition, students’ beliefs about the causes of events (attribution theory) and about the malleability of their intelligence (achievement goal theory) influence their future expectancies, such that 7 students who believe their failures and successes are attributable to controllable factors (e.g., effort) and who believe their abilities will grow with effort are more likely to expect success in the future (Graham & Williams, 2009; Perry, Chipperfield, Hladkyj, Pekrun, & Hamm, 2014; Weiner, 1985; Yeager et al., 2016). Achievement Goal Theory Achievement goal theory proposes that students’ goals in achievement situations are important predictors of engagement, interest, and achievement (Ames, 1992; Elliott & Dweck, 1988; Kaplan & Maehr, 2007; Maehr & Zusho, 2009; Midgley, Kaplan, & Middleton, 2001). Mastery (or learning) goals involve an aim to learn and improve, while performance (or ego) goals involve a desire to demonstrate competence and compare favorably with others. An added dimension of approach vs. avoidance further distinguishes students’ performance goals into those of approaching success (e.g., comparing favorably with others) versus avoiding failure (e.g., unfavorable comparisons with others). In general, mastery goals are associated with more favorable outcomes as compared to performance-avoidance goals, however findings have been mixed regarding relations between performance-approach goals and achievement (Barron & Harackiewicz, 2001; Gonida, Voulala, & Kiosseoglou, 2009; Hulleman, Schrager, Bodmann, & Harackiewicz, 2010; Linnenbrink, 2005; Skaalvik, 1997; Walker & Greene, 2009). Despite the lack of consensus regarding the potential benefits of performance-approach goals, researchers and practitioners generally agree that mastery goals are beneficial (Harackiewicz, Barron, Tauer, & Elliot, 2002; Hulleman & Senko, 2010). One way to promote mastery goals may be through influencing students’ beliefs about intelligence. In addition to providing further explanation regarding “why” students do academic tasks, achievement goal theory proposes that an incremental view of intelligence and ability, or the belief that these qualities are malleable, is a correlate of adaptive motivation including mastery goals (Cury, Elliott, Da Fonseca, & Moller, 2006; Corrion et al., 2010; Elliott & Dweck, 1988). More broadly, achievement goal theorists (Ames, 1992; Maehr & Midgley, 1991) developed a six-facet framework for supporting mastery goals in the classroom: task, authority, recognition, grouping, evaluation, and time (TARGET). 8 These six facets propose that: (1) tasks that are optimally challenging and interesting will promote mastery goals, (2) teachers’ provision of autonomy or sharing of authority influences mastery goals, (3) rewards or recognition of students’ efforts and growth, rather than their comparative performance, will promote mastery goals, (4) student grouping strategies can make social comparisons more or less salient, (5) evaluations can help or hinder students’ focus on improvement vs. performance, and (6) teachers’ allotment of time conveys messages about whether mastery or performance is more important (Maehr & Midgley, 1991). Self-Determination Theory Self-determination theory posits that humans have innate needs for autonomy, competence, and relatedness that drive behavior and facilitate well-being (Ryan & Deci, 2000a). Autonomy is the need to feel that behavior is self-directed and is not compelled by outside forces (deCharms, 1968; Deci, 1975). The need for competence, in contrast to expectancy value theory’s expectancies for success, involves multiple components of curiosity, desires for challenge, and feeling efficacious to meet challenges (White, 1959). Relatedness is the desire to feel that one is valued by and belongs with important others (Baumeister & Leary, 1995; Ryan & Deci, 2000b). Satisfaction of these three needs promotes intrinsic motivation, or engaging in activity for the inherent enjoyment or interest associated with the task rather than external reasons (extrinsic motivation; Ryan & Deci, 2000a). In the absence of purely intrinsic motivation, satisfaction of the three basic needs can result in more internalized forms of extrinsic motivation. Within self-determination theory, motivation is characterized on a spectrum describing varying degrees of quantity and quality: amotivation is a complete lack of motivation, while extrinsic motivation is regulated via rewards, punishments, values, or congruence with one’s identity. Multiple types of extrinsic motivation are further distinguished by increasingly internalized regulation styles: external, introjected, identified, and integrated. External regulation is the form of extrinsic motivation that depends entirely on external rewards and punishments, whereas integrated regulation is the most internalized regulation style, with behavior being regulated by a sense of congruence with the self. Between external and integrated on the continuum are two other forms 9 of regulation: introjected, which is characterized by regulation via internal rewards and punishments and self-control, and identified regulation, where behavior is directed by an individual’s valuing or personal importance assigned to the behavior. These forms of motivation parallel expectancy-value’s conceptualizations of task value, with utility value being the most extrinsic reason for engaging in a task (Wigfield & Eccles, 1992), perhaps in parallel to identified regulation, and attainment value reflecting a more internalized form of extrinsic motivation, such as integrated regulation, and interest value perhaps reflecting intrinsic motivation due to their shared focus on enjoyment of the task itself (Wigfield et al., 2006). Indeed, interest value is often termed “intrinsic” value in the literature (Eccles, 1983). Self- determination theory adds a uniquely detailed and specific account of how the quality of students’ motivation may be enhanced through increasing autonomy, competence, and relatedness. In essence, because of the considerable overlap in expectancy-value and self-determination theory constructs, principles from self-determination theory are very relevant for informing the design of supports for more internalized forms of value and subsequent outcomes. For example, students in autonomy-supportive classrooms report higher need satisfaction, lower amotivation, and higher autonomous (e.g., identified or integrated) motivation (Cheon, Reeve, & Moon, 2012; Cheon & Reeve, 2015). In addition, interventions aimed at boosting racial/ethnic minority students’ sense of belonging (e.g., Walton & Cohen, 2011) have increased their achievement relative to control groups, suggesting that relatedness can also be a fruitful avenue for motivation interventions. Integrative Principles for Supporting Motivation Taken together, the three theories outlined above provide a rich set of recommendations that can be applied to support motivation in educational settings. In the context of STEM fields, integrating propositions from expectancy-value, achievement goal, and self-determination theories would suggest that students who feel capable of success in learning or doing STEM tasks, who value those tasks, who believe their efforts and strategies will result in learning and growth, who feel connected to important others in the field, and who are oriented toward mastering skills and knowledge will be most likely to engage, persist, and succeed in those tasks (Linnenbrink-Garcia, Wormington, et al., 2018). These three 10 theories provide an integrative account of how students’ overall quantity and quality of motivation may be improved, including the most important constructs to be examined, their relations to one another, and their relations to key outcomes. They also explain how existing motivation interventions that are commonly used in research and practice (e.g., mindsets, belonging, and utility value) may function to improve student outcomes. Synthesizing decades of theory and empirical research, Linnenbrink-Garcia and colleagues (2016) developed five design principles cutting across six major motivation theories. While the three theories summarized above provide a comprehensive account of the mechanisms by which motivation can be supported for the current study, recommendations for practice draw on research from all six theoretical perspectives. For example, principles for supporting interest value as conceptualized by expectancy-value theory draw on research from an interest theory perspective (e.g., Pugh, Linnenbrink-Garcia, Phillips, & Perez, 2015). Similarly, recommended practices for supporting expectancies draw on social cognitive theory’s body of research on sources of self-efficacy (Bandura, 1977; Usher, 2009). The five design principles summarizing recommendations for supporting motivation gleaned from all six theoretical perspectives (Linnenbrink-Garcia et al., 2016, pp. 233-234) are: 1) “Support competence through well-designed instruction, challenging work, and informational and encouraging feedback.” 2) “Support students’ autonomy through opportunities for student decision making and direction.” 3) “Select personally relevant, interesting activities that provide opportunities for identification and active involvement.” 4) “Emphasize learning and understanding and de-emphasize performance, competition, and social comparison.” 5) “Support feelings of relatedness and belonging among students and with teachers.” The first design principle, supporting competence, draws on expectancy-value theory’s emphasis on expectancies and the benefits of mastery goals for competence and expectancies as outlined in achievement goal theory. Specifically, the recommendations of appropriate challenge and encouraging, 11 informational feedback are derived from the Task and Evaluation components of TARGET. Principle #1 highlights the common socio-cognitive origins of these two theories, as well as self-determination theory’s proposition that challenge and capabilities must be in balance for optimal motivation. Principle #2, supporting autonomy, arises from the central role of autonomy in self-determination theory’s explanation of optimal motivation, and from achievement goal theory’s proposition that autonomy is a key ingredient for promoting mastery goals (Ames, 1992). Personal relevance and active involvement (Principle #3) arise from all three theories as well, with their shared emphasis on interesting tasks and personal connections as predictors of mastery goals (achievement goal theory), active involvement and relevance to promote high value for tasks (expectancy-value theory), and perceptions of autonomy that are supported via giving rationales and making connections between students’ lives and learning material (self-determination theory). The fourth principle can be traced directly to achievement goal theory, and highlights strategies for promoting mastery goals and minimizing the negative effects of performance goals (Ames, 1992). Lastly, the fifth principle is derived largely from self-determination theory’s proposition that connections to important others facilitate intrinsic motivation (Ryan & Grolnick, 1986), but can also be framed through the lens of other theoretical perspectives as a way to promote interest and engagement. For example, students who feel connected to their teachers and peers, and whose teachers exhibit enthusiasm for the subject matter, also report greater interest (Lazarides, Gaspard, & Dicke, 2018; Linnenbrink- Garcia, Patall, & Messersmith, 2013), suggesting that relatedness is another avenue for supporting value as conceptualized by expectancy-value theory, and for supporting the “task” component of achievement goal theory’s TARGET framework. These design principles highlight the variety of ways students can be motivated to engage and persist in academic tasks. It is important to consider that these principles are not discrete, but rather interrelated, and that constructs such as value and competence perceptions can be highly interrelated and exert bidirectional influences on one another. These principles are useful tools for thinking about the various ways researchers and educators can intervene to buffer declines in students’ motivation. For 12 example, while supporting perceptions of autonomy may be helpful for many students, it may be that other students benefit most from a boost to their perceptions of competence before they are able to realize the benefits of autonomy. Taking an integrative approach to motivation allows for flexibility, personalization, and precision in diagnosing and addressing common motivation problems. However, little research has investigated multifaceted approaches to supporting motivation. Motivation Interventions Motivation interventions are designed from a variety of theoretical perspectives and have taken many forms, including short-term randomized trials, design-based collaborations with instructors, and teacher training programs, among others (Lazowski & Hulleman, 2016; Hulleman & Barron, 2016). Thus far, the majority of experimental research on motivation interventions and their mechanisms has examined brief social psychological interventions, wherein researchers target one construct with precision under carefully controlled conditions (Wilson & Buttrick, 2016). These interventions are typically administered outside of class time, not referring to a particular learning task or course (e.g., Blackwell, Trzesniewski, & Dweck, 2007; Walton & Cohen, 2011), or as a small feature added on to an existing course (e.g., Harackiewicz et al., 2016; Hulleman et al., 2017). There are many benefits to these interventions; in particular, they are advantageous because there is evidence that they reduce achievement gaps over fairly long periods of time with only a brief exercise targeting a particular motivation construct. However, extant motivation theories conceptualize motivation as multifaceted rather than unidimensional (Ames, 1992; Pintrich, 2003; Schunk et al., 2014; Wigfield & Cambria, 2010). Further, theory suggests that motivation constructs have unique developmental origins (Maehr & Meyer, 1997) and influence one another within an individual to support or undermine the overall quality of motivation (Ames, 1992; Vansteenkiste, Simons, Lens, Sheldon, & Deci, 2004; Wigfield & Cambria, 2010). Students also differ in their patterns of motivation such that the type of motivation driving one student may not “work” as an inroad for intervention for another student (Pintrich, 2003). Indeed, while brief social psychological interventions have proven to be beneficial for students at risk for low achievement outcomes, they do not show effects for all students (Schwartz, Cheng, Salehi, & Wieman, 2016). The 13 complex nature of motivation and the varying effects of single-construct interventions suggest that students may realize more benefits, perhaps resulting in greater magnitude, duration, or quality of motivation, when multiple constructs are targeted (Rosenzweig & Wigfield, 2016). Another concern facing motivation researchers is the level at which interventions should be administered (Martin, 2008; Plewis & Hurry, 1998). While some motivation interventions have been administered directly to students, attempting to shape individual beliefs, other interventions have taken the form of teacher training programs aimed at helping teachers incorporate motivationally supportive practices into their instruction (e.g., Reeve, Jang, Carrell, Jeon, & Barch, 2004; Turner et al., 2014). Teacher-level interventions are especially important to examine as brief, student-focused interventions do not take the place of educational reform (Brophy, 2008; Yeager & Walton, 2011). Indeed, effective design of educational contexts may eliminate the need for student-level motivation “boosters,” which could be construed in some cases as inoculation against unmotivating classrooms. While current trends of decreasing motivation continue, however, there could be benefits to intervening with both students and teachers, with students receiving direct benefits through student-level interventions and indirect benefits of teachers’ increasing motivationally supportive practices. Furthermore, motivation interventions at multiple levels may be mutually reinforcing, reciprocally related, and synergistic in their effects (Cook, Purdie-Vaughns, Meyer, & Busch, 2014). For example, students who learn about the malleability of intelligence outside of class may be more likely to enact this belief in courses that reinforce the importance of effort and strategy use. Students may also be more prepared to respond to instructor messages about the importance of effort and strategy use if they have previously endorsed the belief that intelligence is malleable. Indeed, in a national, large-scale study of mindset interventions, effects of mindset interventions appeared to be enhanced in schools with behavioral norms that supported a growth mindset (Yeager et al., 2018). The effects of a student-level intervention may be dampened, however, if students’ courses emphasize performance and social comparisons over learning and growth. 14 Thus far, there is little evidence that interventions targeting multiple constructs have measurably larger effects compared to single-construct interventions (Lazowski & Hulleman, 2016), perhaps because there have been relatively few experimental tests of multicomponent motivation interventions (Hulleman & Barron, 2016). There is even less experimental or quasi-experimental evidence examining the effectiveness of integrating teacher-focused and student-focused approaches to supporting motivation. This may be due to the difficulties of using random assignment when involving teachers and students, or perhaps there is only a finite amount of variability in students’ motivation that can be effectively boosted via interventions. In their systematic review of STEM interventions, Rosenzweig and Wigfield (2016) posit that the effects of complex interventions may be more than the sum of their parts, but researchers should aim for precision in evaluating the effects of interventions by examining components of complex interventions (Rosenzweig & Wigfield, 2016). This will allow research and practice to balance effectiveness and cost in determining the “best” approaches to boost students’ motivation. Student-Level Interventions Single construct interventions. Brief interventions leveraging social psychological mechanisms to reduce achievement gaps have become increasingly popular, with a growing body of literature supporting their effectiveness across various settings and populations (Harackiewicz & Priniski, 2017; Paunesku et al., 2015; Yeager & Walton, 2011). These interventions focus on a variety of student beliefs, including incremental theories of intelligence (Blackwell et al., 2007), social belonging (Walton & Cohen 2007; 2011), personal values (Cohen, Garcia, Apfel, & Master, 2006), attributions for success and failure (Okolo, 1992), and value for learning tasks (Hulleman et al., 2017). In this review, I focus on theories of intelligence (mindset) interventions, belonging interventions, and utility value interventions. These three interventions were selected for the present study because of the relatively large bodies of research examining these interventions and because of their relevance to the needs of students in the setting of the study. First, most interventions from an expectancy-value perspective have been aimed at supporting utility value, perhaps because utility value is viewed as being the most malleable construct (Harackiewicz 15 et al., 2016). Utility value or “relevance” interventions prompt students to think about the relevance or importance of course material to their own lives, either through directly communicating the relevance of the content (Brown, Smith, Thoman, Allen, & Muragishi, 2015; Durik, Shechter, Noh, Rozek, & Harackiewicz, 2015) or through asking students to generate connections between the content and their own lives. This process is theorized to boost perceptions of value and thus lead students to engage more fully in their coursework (Harackiewicz & Priniski, 2017). Impacts of utility value interventions are typically measured in terms of achievement, like GPA in courses, and retention outcomes (e.g., Harackiewicz et al., 2012; Hulleman, Godes, et al., 2010), and are moderated by individual differences. For example, in past studies, utility value interventions have been most beneficial to first-generation college students from underrepresented racial/ethnic minority groups (Harackiewicz et al., 2016) and students with low prior achievement (Hulleman et al., 2016). Second, interventions to promote incremental theories of intelligence (or growth mindset) similarly originate from a social psychological perspective and involve a brief exercise that is designed to initiate recursive motivational processes that unfold over time. In the intervention, students learn that their abilities and intelligence can grow with effort, and this learning (often involving a brief writing exercise) is posited to help students maintain high effort by interpreting their successes as evidence that their efforts and strategies paid off, while inoculating them against the belief that failures reflect low, fixed ability on the task. The intervention has also been cast as a remedy for stereotype threat (Aronson, Fried, & Good, 2002; Good, Aronson, & Inzlicht, 2003), based on the idea that the negative effects of stereotypes about intelligence could be counteracted if threatened students came to endorse the idea that intelligence were malleable rather than fixed. Growth mindset intervention studies have provided evidence for the intervention’s effects on incremental beliefs, enjoyment of school, and achievement (Aronson et al., 2002; Blackwell et al., 2007; Good et al., 2003; Paunesku et al., 2015). Similar to the utility value intervention, growth mindset interventions often show specific effects on traditionally disadvantaged groups but not the sample as a whole, such as African American students (e.g., Aronson et al., 2002) females in math (Good et al., 2003), or students at risk of dropping out of high school (Paunesku et al., 2015). 16 Third, belonging interventions (Walton & Cohen, 2007; 2011; Walton, Logel, Peach, Spencer, & Zanna, 2015) aim to help students interpret experiences of adversity as transient and manageable rather than indicators of their lack of belonging in the setting. In prior research, belonging interventions have had effects on achievement, students’ interpretation of daily adversity, academic attitudes, and social integration (Walton & Cohen, 2007; 2011; Walton et al., 2015). In general, effects of belonging interventions are localized to students at risk of low belonging, such as women in male-dominated engineering fields (Walton et al., 2015) or Black students in college (Walton & Cohen, 2007; 2011), and may in some cases be detrimental to majority students (Walton & Cohen, 2007). Brief, precise interventions have many benefits. First, researchers are able to closely control the conditions, minimizing the risk of flawed implementation and maximizing the possibility that observed differences between the treatment and control groups can be attributed to the intervention. Second, the psychological mechanisms explaining change can be precisely targeted and examined based on close adherence to theoretical expectations and prior research. Third, these brief interventions take only a small amount of students’ time, very little (if any) involvement from teachers beyond class time or assignment points, and they require minimal financial investment. Lastly, these interventions have been fairly consistently effective at boosting academic outcomes for those students who are most at risk for negative outcomes such as low achievement or attrition from college programs. These are clear benefits for those hoping to scale and implement motivation interventions in a variety of fields and settings. However, some potential drawbacks should be considered regarding brief interventions targeting a single construct. Most notably, these interventions most often positively impact only a subset of students and have null (or even negative) effects on other groups of students (Schwartz et al., 2016). In addition, though they are brief and ostensibly scalable (Paunesku et al., 2015), they must be adapted to each new context, which requires extended involvement and testing by experts in theory and evaluation (e.g., Yeager et al., 2016). They are not always replicable without the exact conditions identified by psychologists, and so continued involvement and evaluation by experts is necessary for continual fidelity in implementation. Like any intervention, brief interventions require continual refinement and replication 17 across multiple samples in order to fully understand and maximize the conditions for success, and this work is ongoing. In addition, it may be that the seeming simplicity of the concepts involved in these interventions may contribute to misconceptions and overly simplistic understanding leading to flawed implementation by practitioners (Dweck, 2016; Gross-Loh, 2016). Thus, brief interventions may not actually have the advantage of simplicity, ease, or cost over multifaceted or design-based motivation interventions. In addition, though brief interventions are ostensibly aimed with precision at boosting a particular motivation construct (e.g., theories of intelligence, utility value) which then prompt recursive processes unfolding over long periods of time, outcomes are typically measured in terms of distal indicators such as achievement and retention, with fewer studies testing the specific mechanisms theorized to be enacted by each intervention. For example, interventions aimed to support students’ beliefs that their intelligence can grow with effort have not supported theorized effects of these interventions on stereotype threat or its direct outcomes (e.g., Aronson et al., 2002; Good et al., 2003). Importantly, while theories of intelligence are theorized as precursors to mastery goals (Dweck & Leggett, 1988; Dweck, 2003) and correlational evidence shows strong relations between these constructs at the same time point (Blackwell et al., 2007), effects of the intervention on mastery goals or competence beliefs and subsequent behaviors have not been examined in real-world classrooms. Similarly, there is evidence that utility value interventions result in increased grades and interest for some groups (Hulleman & Harackiewicz, 2009; Hulleman, Godes, et al., 2010), and these increases can at least partially be explained by increased value or expectancy (Hulleman et al., 2016), however the specific mechanisms leading to these effects are not well understood. For example, Hulleman and colleagues (2016) found that a utility value intervention boosted expectancy for success rather than utility value in a sample of undergraduate students. Harackiewicz and colleagues (2012, 2016) typically examine effects on utility value using essay content, but it is unclear whether students would endorse higher levels of utility value on traditional survey measures, independent of their required essays, which may be inflated in their reference to personal relevance due to social desirability or the need to fulfill the 18 requirements of the assignment. Thus, it is not clear whether utility value interventions are affecting distal outcomes via utility value, expectancies, both expectancies and values, or via other mechanisms altogether. Lastly, belonging interventions have assessed direct effects on belonging perceptions, interpretations of daily adversity, and achievement (Walton & Cohen, 2007, 2011), however these studies have not tested the proposed role of motivational constructs such as value (e.g., intrinsic motivation). There is a need for longitudinal research examining the theorized mechanisms by which brief interventions lead to changes in achievement outcomes. Understanding these mechanisms will allow researchers and practitioners to more precisely and effectively design motivation interventions and to understand their effects, including null effects. In addition, if these brief interventions initiate motivational processes with lasting effects, it is possible that interventions targeting more than one construct or interventions supporting adaptive motivational beliefs on the classroom level may lead to larger, longer-lasting, or qualitatively different effects on motivational beliefs. A few interventions taking a multi-faceted approach to student-level interventions are described below. Multi-construct interventions. A few promising lines of research have examined the effects of targeting multiple motivational processes simultaneously. A few researchers have attempted to layer brief social psychological interventions, leveraging the affordances of experimental designs to test main and interactive effects of administering multiple student-level interventions. An alternative approach involves a more wholistic, long-term approach by embedding a variety of supports for multiple motivation constructs in student-facing training modules (Martin, 2008). First, the approach of combining brief social psychological interventions is promising in its ability to address multiple forms of motivation, thereby improving the overall quantity and quality of students’ motivation. However, studies assessing the effect of “layering” brief social psychological interventions have not found measurably greater effects for the combined interventions as compared to each intervention alone (e.g., Paunesku et al, 2015). For example, Good and colleagues (2003) did not find evidence for the expected additive effects of combining a growth mindset intervention with an attribution retraining (or competence-supportive) intervention. This could be due to the overlap in the 19 constructs each intervention aimed to shift, a problem in motivation intervention research highlighted by Pintrich (2003). Specifically, though mindsets and attributions are distinct constructs, mindsets are predictors of attributions, with malleable views of intelligence predicting more adaptive attributional patterns (Dweck & Leggett, 1988; Dweck & Molden, 2017). Harackiewicz and colleagues (2016) similarly found that a values affirmation intervention had no additional effects when combined with a utility value intervention, though this could be because there were no main effects of the values affirmation intervention in this study. On the other hand, the lack of combined effects may suggest that greater dosage or targeting a wider variety of motivational constructs may be needed to maximize the benefits of complex interventions. Taking a more time-intensive and wholistic approach to a student-level intervention, Martin (2008) integrated principles from multiple theoretical perspectives to design a series of modules focused on adaptive and maladaptive cognitions and behaviors, including self-efficacy, mastery orientation, value, anxiety, and self-handicapping. Teachers led students through discussions and exercises focused on the motivational and behavioral principles. Students in the treatment group appeared to show adaptive patterns of increase and decrease in some of the constructs of interest based on self-report pre- and post- intervention surveys, however students were not randomly assigned to conditions and those in the treatment group appeared to have more maladaptive patterns of motivation and behavior before treatment. Thus, the different patterns of change in the treatment group as compared to the control group cannot be attributed to the intervention alone. Overall, the scant and disjointed evidence on multifaceted, student-level interventions signals a need for careful selection of the constructs under examination, combined with rigorous tests examining the specific mechanisms theorized to be enacted. Indeed, one study showed that the addition of choice (or support for autonomy) within a utility-value intervention resulted in enhanced effects of the utility-value intervention (Rosenzweig et al., 2018), suggesting that careful selection of motivational processes and study design in alignment with theoretical expectations for how motivation should be expected to change 20 is a promising direction for future motivation intervention work. Examining single-construct interventions as compared to multi-component interventions is important to ensure that multiple approaches are needed. Teacher-Level Interventions Some interventions, rather than being administered directly to students, are directed at changing teacher behavior and instructional strategies in efforts to promote student motivation. This work relies on the assumption that motivation can be shaped by the classroom environment. Similar to research on complex student-level interventions, the literature testing the effects of teacher- or classroom-level motivation interventions is somewhat scant and lacks cohesion (Hulleman & Barron, 2016). A variety of variables have been assessed as outcomes of these interventions so it is difficult to compare their effectiveness across studies. These interventions often use a quasi-experimental or design-based approach rather than experimental designs (Hulleman & Barron, 2016). While some of the research on teacher-level interventions has compared treatment and control groups (e.g., Feng & Tuan, 2005; Patrick, Mantzicopoulus, & Samarapungavan, 2009; Stipek, Givvin, Salmon, & Macguyvers, 1998; Ziegler & Heller, 2000), this is relatively rare and random assignment to conditions is even less common. Thus, inferences about the effectiveness of these interventions must be qualified by the likelihood of selection effects. Additionally, teacher-level motivation interventions are varied, often involve new designs, and are rarely tested more than once, so there is little evidence of replicability or generalizability even among the most effective of teacher-level interventions. In this section, I review teacher-focused interventions with a particular focus on those with multiple studies supporting their effectiveness, whether in the same setting or across several settings, those with treatment and control groups, and those examining mechanisms of intervention effects in alignment with major motivation theories. Single-construct interventions. Teacher-focused motivation interventions targeting only one construct are less common that multi-construct teacher-focused interventions, but provide evidence that teachers can be trained to support motivation and that changing teacher behaviors can result in subsequent changes in students’ motivation and related outcomes. One prominent example of a teacher-focused 21 motivation intervention is framed from a self-determination theory perspective. Cheon and Reeve (2015) designed a program to help physical education (PE) teachers develop more autonomy-supportive teaching strategies, built on their prior intervention work in this area (Cheon & Moon, 2010; Cheon et al., 2012; Hardré & Reeve, 2009; Reeve et al., 2004). After participating in the three-part intervention, which consisted of self-assessments, discussions, presentation, and working through scenarios, followed by refinement and consultation with the researchers, teachers exhibited significantly more autonomy- supportive practices than the control group teachers. Furthermore, experimental group students reported greater increases in need satisfaction and engagement compared to control group students. Indeed, training teachers to incorporate autonomy supportive practices is perhaps the most well-developed line of research on teacher-focused motivation interventions, with meta-analytic results providing a variety of productive recommendations for teacher training programs arising from the varying rates of success among programs (Su & Reeve, 2011). Ziegler and Heller (2000) implemented an intervention designed to support students’ explanations of success and failure (attributions) and subsequent effort and achievement from an attribution theory perspective. High school physics teachers were trained to give feedback that shaped students’ explanations of success or failure (attributional retraining), and this feedback was provided across 10 weeks of instruction. Students in the treatment condition were more likely to explain their successes and failures in terms of effort, less likely to explain their failures in terms of task difficulty, and had more adaptive trajectories of achievement compared to students in the control group. These findings show that teacher behavior as shaped by researchers in alignment with motivation theory can have impacts on students’ motivational beliefs. Further, comparing Ziegler and Heller’s findings to a similar study implementing attributional retraining via a computerized tutor (Okolo, 1992), which did not result in significant effects on students’ attributions, suggests that the teacher might play a critical role in shaping these beliefs. However, other differences in the sample and study design cannot be ruled out as possible explanations for this difference in effectiveness. 22 This prior empirical research aimed to support one form of motivation (autonomy support and attributions for success and failure). Further research is needed, however, to determine with precision whether teacher-level motivation interventions, even those designed with only one motivation construct in mind, may effectively target more than one motivation construct, and whether the effects of multifaceted interventions can persist for longer periods of time, and beyond the immediate context of the intervention classroom. This is particularly important considering that these interventions took place in physical education settings (e.g., Cheon & Reeve, 2015) and K-12 settings (Ziegler & Heller, 2000) and involved several teacher actions that may be aligned with supports for other motivation constructs. Multi-construct interventions. Interventions targeting more than one motivational construct appear more commonly in the literature than single-construct teacher-focused interventions. In one study using random assignment and targeting multiple motivation constructs, Rubie-Davies and colleagues (2015) conducted a series of 4 workshops cutting across theoretical perspectives for a randomly assigned group of elementary school teachers. The workshops aimed to promote practices associated with high teacher expectations, including practices associated with belonging, competence, autonomy, and mastery goals. Students of teachers in the intervention group had a significantly higher rate of growth in math achievement as compared to students whose teachers were in the control group. However, effects on students’ motivation were not assessed so the mechanism of support is unknown. In another teacher-focused intervention, Guthrie and colleagues (1998, 2004, 2006) worked with 3rd grade teachers to incorporate stimulating tasks related to reading assignments, combined with cognitive strategies for reading comprehension, in efforts to boost self-efficacy, interest/intrinsic motivation for reading, and reading comprehension. All teachers participated in the 10-day professional development workshop. Based on classroom observations, researchers identified two instructional conditions: teachers who used a high number of stimulating tasks and those who used a low number of instructional tasks. Students in the high group had higher teacher-rated intrinsic motivation and higher reading comprehension scores (Guthrie et al., 2006). Further, the combined motivational and cognitive supports appeared to increase the benefits to students, resulting in greater comprehension, motivation, and 23 strategy use compared to either condition alone (Guthrie et al., 2004). This study points to the efficacy of combining multiple student supports, and while motivational gains were only seen in terms of interest, other motivational constructs besides self-efficacy may have been supported but were not assessed. Maehr and Midgley (1996; Anderman, 1996) took a design approach using principles from achievement goal theory, working with teachers and administrators to develop motivationally supportive school policies and classroom practices at two schools. These recommendations and practices were based on the TARGET principles for supporting mastery goals. Results indicated that teachers who were highly involved in the intervention efforts reported more motivationally supportive practices over time, and students at the two intervention schools reported higher mastery goals, lower performance goals, and higher self-efficacy, compared to students in a comparison school. Taking a similarly design-based approach but using multiple theoretical perspectives, Turner and colleagues (2014) implemented a 3-year intervention wherein middle school teachers discussed principles of motivation (belonging, competence, autonomy, and value) and specific strategies for supporting the motivational principles four times a year for 3 years. Of the six instructors who were randomly selected for observation, half showed a trajectory of increasing use of motivationally supportive practices and increasing student engagement over the 3 years, while the other half exhibited low and stable trajectories of both motivationally supportive practices and student engagement. Those teachers who were most responsive to training required long periods to practice and refine these new practices, up to two years, in order to effectively support students’ motivation and engagement (Turner et al., 2014). Emblematic of the similar intervention work by these authors (e.g., Turner, Warzon, & Christensen, 2011), these two studies indicate that long-term training, continued support, and perhaps most importantly, willingness on the part of teachers may be needed to effect changes in teacher behavior and subsequent gains in student outcomes. This may be particularly true for interventions focused on complex, multi-theoretical processes. Other studies, however, hint that even a brief training of integrative motivational design principles can be effective. Notably, Godin and colleagues (2015) trained instructors in a summer science program to support motivation using motivation design principles. Though there was not a control group 24 of instructors, results provided evidence that students perceived the program to be motivationally supportive across measures that align with each of the design principles. The curriculum itself was also designed to be motivationally supportive, which may have aided instructors in their motivationally supportive delivery of instruction. Furthermore, Godin et al. (2015) and a study of longer-term effects (Linnenbrink-Garcia, Perez, et al., 2018) indicated that students in the program increased their task value and expectancies for success in science, completed more science courses, and via increased task value were more likely to choose a concentration that was relevant to the summer program, compared to a student control group. Taken together, studies of teacher-focused motivation interventions provide evidence that instructors can be trained to support students’ motivation, though not all instructors may effectively implement motivational support strategies. Additionally, these studies illustrate that even interventions theorized to support only one type of motivation may in fact alter multiple types of motivation. For example, the TARGET framework is designed around promoting mastery goals; however, TARGET practices were shown to support competence beliefs (Anderman, 1996; Maehr & Midgley, 1996), and also involve theorized supports for competence beliefs. Implications for Complex, Multilevel Motivation Interventions Motivation interventions broadly considered to be multifaceted include those that target more than one construct, those that arise from a variety of theoretical perspectives (or no single theoretical perspective), and those that intervene on multiple levels (e.g., student, classroom, teacher, and/or school; Lazowski & Hulleman, 2016; Rosenzweig & Wigfield, 2016). Synthesizing evidence from student- focused, teacher-focused, single-construct, and multi-construct interventions, it is evident that there are clear barriers to designing and implementing teacher-level and multifaceted motivation interventions, perhaps most notably the time and costs involved in the planning and initial implementation stages. As we have seen with brief social psychological interventions, even seemingly simple interventions require investments of time and resources on the part of researchers, evaluators, instructors, and students, time and resources that could otherwise be devoted to improving study skills or content knowledge. It is often 25 difficult enough to secure an instructor’s approval for researchers to embed a handful of brief experiences in the context of a course, as instructors are loath to give up valuable instructional or homework time. Asking instructors to participate in broader-based interventions requiring more time and resources, at least up front, is a difficult task that may not always result in success. Indeed, this can be seen in the results of Turner and colleagues (2014), wherein teachers required up to two years of training and refinement to effectively support students’ motivation and engagement, and not all teachers were responsive to the training. Despite these potential drawbacks, the benefits of complex, multilevel interventions could be many, particularly when teachers are able to support multiple forms of motivation. Interventions aimed to support more than one motivational construct have the potential to reach a broader range of students, and because motivation constructs influence one another, boosting multiple forms of motivation could qualitatively shift students’ beliefs about themselves in relation to STEM fields. In addition, working with instructors to meet students’ motivational needs could broaden the impact of interventions beyond the students who are able and willing to participate in targeted student-level interventions. Teacher-level interventions may optimally work in concert with student-level interventions to promote a greater likelihood of matching students’ motivational needs, their readiness to learn, and their instructors’ abilities to meet motivational needs. Present Study Declines in motivation are a major concern during college, particularly in STEM fields, as these declines hinder achievement and promote attrition from STEM fields (Maltese & Tai, 2011; Musu- Gillette et al., 2015; Robinson, Lee, et al., 2018; Robinson, Perez, et al., 2018). While prior research provides promising evidence that motivation interventions can be used to mitigate declines, current intervention approaches are in need of further investigation to explore their efficacy for new contexts, a wider array of students, and more sustained gains in both quantity and quality of motivation. In particular, combining popular intervention approaches with an integrative, teacher focused approach may provide enhanced opportunities for broadening students’ pathways to motivation and success in STEM fields. A 26 multifaceted, multilevel approach to motivation interventions also allows for a closer test of complex motivation processes as they occur within individuals and contexts. Thus, the present intervention study examined the effectiveness of a multifaceted approach to motivation interventions, including the effectiveness of interventions designed to support value, belonging, and incremental theories of intelligence at the student level as well as the potential additional impact of an instructor workshop based on the five motivation design principles (Linnenbrink-Garcia et al., 2016). I used a multifaceted, integrative approach to support motivation by intervening with instructors and directly with students and to test the effects of these interventions in terms of proximal and distal effects on motivation, identity, achievement, and retention. Rather than attempting to compare and rank the efficacy of various approaches, the study aimed to better elucidate “how different personal and contextual factors interact to generate different patterns of motivated behavior” (Pintrich, 2003, p. 671). While many intervention studies have exhibited effects on achievement and interest in the short term and others have evinced more long-term effects on persistence and behavior, it is also important to consider the quality and duration of outcomes, including outcomes beyond achievement and retention. In this study, I examine achievement goals, task values, competence beliefs, engineering identity, achievement, and major persistence as outcomes of the interventions. Though achievement and retention gains are of broad interest to researchers and practitioners alike, there is also value to enhancing the quality and quantity of other outcomes, regardless of their impact on achievement and retention (Brophy, 2008; Pekrun, 2006). In addition, while psychosocial outcomes (motivation and identity) are worthwhile aims in and of themselves, they often also relate to other outcomes that may or may not be assessed in a single study, such as achievement, retention, educational attainment, career outcomes, and well-being. Achievement and retention gains are also not sufficient to infer which specific processes are activated by interventions. By examining fine-grained distinctions between types of value, for example, we may infer whether an intervention is activating more extrinsic types of value (such as utility value), or more sustained and intrinsic value, such as attainment or interest value. Lastly, examining a broad range of outcomes allows researchers to more effectively adapt interventions to multiple contexts. For example, an 27 intervention that effectively boosts utility value but not theories of intelligence may not be appropriate for a setting wherein students already perceive high value for the task but do not feel that their efforts will result in improved learning or skill development. While motivational constructs are clearly relevant outcomes of the focal interventions in this study, identity development is an outcome that has received increasing attention from practitioners and educational psychologists alike in recent years (Kaplan & Flum 2009; Kaplan & Flum, 2012; Nasir, Rowley, & Perez, 2016; National Academies of Sciences, Engineering, and Medicine, 2016; Roeser, Peck, & Nasir, 2006). Identity is an important predictor of persistence and career attainment in STEM (Estrada et al., 2011; Robinson, Perez, et al., 2018), and may be supported by various forms of motivation, including competence beliefs (Chemers, Zurbriggen, Syed, Goza, & Bearman, 2011; Robinson, Perez, et al., 2018; Robnett, Chemers, & Zurbriggen, 2015) and mastery goals (Hernandez, Schultz, Estrada, Woodcock, & Chance, 2013). From an expectancy-value perspective, intervening to support several forms of motivation is an optimal way to support identity development, as identities are comprised of several forms of motivation that are highly salient and important to the individual (Eccles, 2009; Perez et al., 2014). As a complement to this perspective, Kaplan and Flum (2012) highlight parallels between achievement goal orientations and identity formation styles, noting that a motivational orientation towards intrapersonal growth (e.g., mastery) as opposed to social comparison and performance is similarly adaptive for motivational and identity development outcomes. This suggests that teachers who create mastery goal structures while prompting students to make deep personal connections to course material may not only facilitate learning and motivation, but may also promote adaptive patterns of identity development. This is especially important for college students, who face important decisions about their academic major and future career identities, and who will draw on information gleaned during class about their capabilities, values, and social belonging in their field of study. To parallel prior research, I examined achievement and retention outcomes of the motivation interventions. To extend prior research and in alignment with my integrative theoretical approach, I also examined a variety of motivational constructs as outlined above. Because I hypothesized that multifaceted 28 and multilevel motivation interventions would facilitate deeper, higher-quality motivation that facilitates identity commitments to the subject of study, I also examined identification with engineering as an outcome variable. RQ 1: What are the main effects of various motivation interventions? RQ 1a: How do brief social psychological interventions affect value, perceived competence, achievement goals, engineering identity, achievement, and persistence in engineering? The first aim of the study was to replicate prior studies (Harackiewicz et al., 2016; Hulleman et al., 2016; Paunesku et al., 2015; Walton & Cohen, 2007, 2011; Yeager et al., 2016) by examining main effects of three brief social psychological interventions (utility value, growth mindsets, and belonging) on students’ task value, expectancy for success (perceived competence), and achievement in an engineering course. I extend prior research by additionally examining effects on achievement goals and long-term effects on engineering identity, engineering achievement, and retention. Consistent with prior research (Canning et al., 2018; Hulleman et al., 2010, 2017), I hypothesized that the utility value intervention would have significant, positive main effects on utility value, interest value, perceived competence, and achievement; but because utility value is not theorized to influence attainment value or engineering identity, I expected that the utility value intervention would have no main effects on these outcomes (Hecht, Canning, Tibbetts, Priniski, & Harackiewicz, 2016). The TARGET framework includes recommendations that teachers connect course activities to students’ interests and provide rationale, suggesting that boosting utility value may also boost mastery goals (Maehr & Midgley, 1991). Based on theoretical expectations and effects on achievement in prior research (Wigfield & Cambria, 2010; Harackiewicz et al., 2016), I expected that students in the utility value intervention would be more likely to stay in the engineering major. Based on achievement goal theory and correlational findings in prior research (Blackwell et al., 2007; Cury et al., 2006; Corrion et al., 2010; Dweck & Leggett, 1988), I expected that the mindsets intervention would also result in main effects on mastery goals and long-term achievement. Prior intervention research has not examined theories of intelligence in relation to performance goals, value, 29 identity, or retention; however, achievement goal theory and findings from prior observational research would predict decreased performance goals and increased likelihood of retention as a result of incremental theories of intelligence (Dweck & Leggett, 1988). Though theory and research have not yet linked value or identity to theories of intelligence, incremental views of intelligence may buffer a student from declines in value and identity by allowing them to interpret difficulties and high effort as chances to improve their knowledge rather than signs of incompetence or lack of belonging. Lastly, prior research suggests that belonging interventions result in gains on achievement and sense of fit (i.e., belonging) only for racial/ethnic minority students (Walton & Cohen, 2007, 2011). Therefore, I predicted that there would be no main effects on achievement or retention. Belonging interventions have not examined motivational outcomes, therefore I did not have specific hypotheses about these effects, however self-determination theory suggests that increased belonging may result in higher-quality motivation due to need satisfaction. This may be observed in higher interest or attainment value for students in the belonging treatment group. Indeed, Roeser, Midgley, and Urdan (1996) found that perceptions of belonging mediated the effect of positive teacher-student relationships on students’ enjoyment of school, further supporting the potential effects of belonging on value, particularly interest value. In addition, reading about how others overcome struggles in college may increase students’ academic perceived competence via vicarious experience, a source of self-efficacy as proposed by social cognitive theory (Bandura, 1977). RQ 1b: Do students whose instructors are trained to support motivation have higher value, perceived competence, mastery goals, engineering identity, achievement, and persistence in engineering, and lower performance goals? The second aim of the study was to assess whether instructor training will support student outcomes via teachers’ greater use of motivationally supportive practices. Because the teacher training emphasized multiple motivation constructs, I hypothesized that the training sessions would result in main positive effects on students’ interest value, utility value, attainment value, perceived competence, mastery goals, achievement, engineering identity, and persistence in engineering, with negative effects on performance goals. 30 RQ 1c: Does prior achievement moderate the main effects of each intervention? Consistent with prior research (Hulleman et al., 2010, 2017), I assessed whether prior achievement moderated the effects of the interventions on perceived competence, value, achievement goals, achievement, engineering identity, and persistence in engineering. I hypothesized that the brief social psychological interventions would be most beneficial for students with low prior achievement. I expected that prior achievement would also moderate the effects of the instructor training condition alone and of the combined student- and classroom-level interventions, but to a lesser extent, such that a broader group of students would benefit from the classroom-level intervention. RQ 2: What are the combined effects of a complex motivation intervention? RQ 2a: Do student-level interventions targeting theories of intelligence and value combine to show additive or interactive effects on value, perceived competence, achievement goals, achievement, engineering identity, and persistence in engineering? A key aim of the study was to assess the unique and combined effects of intervening to support multiple, distinct motivation constructs. Because each student-level intervention targeted distinct constructs theorized to support iterative motivation processes, I hypothesized that students experiencing multiple interventions would have higher interest value, utility value, attainment value, perceived competence, mastery goals, achievement, engineering identity, and persistence in engineering relative to students receiving neither or either of the two interventions on their own. In particular, I expected students in both the mindsets and utility value intervention groups to have higher perceived competence than students in either condition alone (additive effects) because past utility value interventions have sometimes increased expectancy (Hulleman et al., 2017) and incremental theories of intelligence should also support students’ perceptions of competence (Elliott & Dweck, 1988). Regarding interactive effects, expectancy value models theorize that expectancy and value interact to influence achievement outcomes (Trautwein et al., 2012); students with high expectancies and high value in combination appear to realize the greatest benefits to their achievement, and high value may actually be detrimental to students with low expectancies. Additionally, the mindsets intervention may 31 amplify the effects of the utility value intervention on interest or attainment value if students are more mastery-oriented as a result of the mindsets intervention and so use their utility value writing assignments to connect their learning to more intrinsic forms of value. Similarly, the belonging intervention may buffer students from declines in competence and thus in combination with the utility value intervention, result in interactive effects on achievement. Thus, I expected the value and incremental theories or belonging interventions to have significant positive interaction effects on achievement, either by buffering students from the negative effects of value for students with low perceived competence or by prompting students to focus on more intrinsic forms of value in their essays. However, prior research indicates that the utility value, belonging, and mindsets interventions on their own are most beneficial for students at risk for negative outcomes (e.g., Yeager et al., 2016), so it is possible for this interaction to actually be negative, with students who have increased perceived competence due to the mindsets intervention benefitting less from the utility value intervention with regard to their achievement. This may be due to ceiling effects of one form of motivation impacting achievement, so the two conditions combined may have resulted in positive interactive effects on perceived competence and/or value due to the bidirectional relations among expectancies and value unfolding over time throughout the semester. RQ 2b: Do student-level and teacher-level motivation interventions combine to show additive, specialized, or synergistic effects on value, perceived competence, mastery goals, achievement, engineering identity, and persistence in engineering? Another central aim of the study was to examine the effects of combining student- and teacher-level motivation interventions. I hypothesized that students in the incremental/belonging and instructor training or the utility value and instructor training conditions would have higher motivation, achievement, engineering identity, and persistence in engineering than students in the control groups or students in the incremental only, utility value only, or instructor training only conditions. This hypothesis aligns with the matching hypothesis, which posits that classroom supports are most effective when they are aligned with students’ existing motivational orientations (Barron & Harackiewicz, 2001). In effect, student-level interventions would prime students to respond more favorably to teacher-level interventions, or vice versa. These effects may 32 be additive or interactive. For example, a student-level intervention to support belonging may show additive effects with teacher training if TAs are also able to regularly, effectively, and uniquely buffer students from declines in belonging. The resulting effect would appear as the combined effect of both, unique supports for belonging. Specialized effects may also appear if each intervention affects unique outcomes. Interactive effects would appear if shifts in motivation or attention as a result of student-level interventions strengthen the effects of teacher-level supports, or vice versa. For example, there is some evidence that the effects of students’ personal goals (as shifted by the mindsets intervention) on achievement-related outcomes may be strengthened by the goal context in the classroom (Lau & Nie, 2008; Murayama & Elliott, 2009). Specifically, I expected that students who learned about the malleability of intelligence in a direct intervention and through regular interactions with their instructor would have higher mastery goals, value, and perceived competence, lower performance goals, and higher identity, achievement, and persistence compared to either condition alone. Students who participated in the belonging intervention and whose instructors supported their sense of belonging in class should have higher intrinsic value and attainment value, higher mastery goals, lower performance goals, and higher achievement, identity, and persistence compared to students in either condition alone. Lastly, students in the utility value intervention who also had weekly opportunities to connect their learning to their lives were expected to have higher value, higher perceived competence, and higher achievement, identity, and persistence compared to either condition alone. I further expected that students in the utility value condition and teacher training condition would have higher value compared to students in either condition alone (additive effects) due to TAs’ use of strategies to boost value. However, the utility value intervention may be enough to show ceiling effects, so teacher training may simply impact other forms of value. Interaction effects would be due to the effectiveness of one intervention depending on receiving another intervention, suggesting that student-level or teacher-level interventions “prime” students to be receptive to other motivational supports. 33 Comparing the three conditions in which students received both a student-level intervention and a teacher-level intervention, students in the incremental + teacher training condition were expected to have the highest mastery goals, students in the utility value + teacher training condition were expected to have the highest utility value, and students in the belonging + teacher training condition were expected to have the highest interest or attainment value. These hypotheses were based on theoretical expectations for the motivation constructs of interest that are most closely related to the constructs targeted by each intervention. However, motivation constructs were not measured directly after each intervention, so it was possible that “dispersion” effects would be seen, with student-level interventions aimed at particular constructs resulting in higher levels of other motivation constructs due to bidirectional influences over time. Achievement, identity, and persistence were tentatively expected to have similar levels across these three conditions, however this is not a strong hypothesis due to lack of prior research. Lastly, I expected that students in receiving the benefits of three different interventions (including the incremental/belonging, utility value, and instructor training conditions combined) would have the highest motivation, achievement, engineering identity, and persistence in engineering compared to all other groups. However, it is important to consider that outcomes could be susceptible to ceiling effects, with one intervention alone being sufficient to increase a particular form of motivation. However, combined effects may rather be seen in terms of increases in a broader array of motivation variables. 34 CHAPTER 3: Method Setting and Participants This randomized field experiment took place in an introductory engineering course at a large, Midwestern university. This course, Introduction to Engineering Design (EGR 100), was required for all students intending to major in engineering fields and was typically taken in the first year of college, with ~75% of students taking the course in their first semester. The course objectives of engineering design and project management were accomplished through weekly lectures, readings, quizzes, writing and design assignments, and individual and group projects cutting across multiple engineering fields. Students were expected to learn the basic skills and knowledge needed to successfully manage and complete an authentic engineering design project. At this university, admission to specific engineering majors after the second year of college depended on performance in introductory courses during students’ first two years, including EGR 100. The course was comprised of 27 sections with 20-44 students in each section for a total enrollment of 1,107 students. All sections met together for a 1-hour weekly lecture, and each lab section met separately for two hours per week under the direction of a graduate student teaching assistant. Teaching assistants (TAs, N = 9) were assigned to 3 sections each. Graduate TAs were responsible for administering and grading all course assignments, exams, and group projects. The study was deemed exempt by the university’s Institutional Review Board (Protocol # x17-1070e). Recruitment and Eligibility All students and TAs in the course participated in study activities, including survey data gathering, as part of instructional activities. Students were given the opportunity to consent to release their survey responses and academic record data for use in the study or opt out of the study while still completing the activities for course credit. Students who opted out of the study participated in the study activities along with their classmates, however their data was not used or reported in the study. 35 All teaching assistants for the course were recruited to participate in a training workshop during their regularly scheduled TA meetings. They were also given the opportunity to provide informed consent for participation in the study, with no penalties for not participating. The student participants who agreed to be in the study (N = 1,021) were 78.4% white, 13.4% Asian/Asian American, 1.9% African American, 3.5% Hispanic/Latino, 2.8% Multiracial; 24.9% female; and 11.3% first-generation college students. All nine TAs agreed to be in the study, however one TA left the university and the study during week 6 of the semester. This TAs’ sections were redistributed to other TAs, although due to the risk of contamination, these students were not included in the analytic sample. Response rates and missing data analyses are reported in the Results section. Procedure The sequence of events for the study, including intervention procedures and measurement timing, is presented in Table 1. Assignment to Conditions Teaching assistants were randomly assigned to a motivation workshop (treatment) condition or an active control condition. In the treatment condition, TAs learned to support their students’ motivation; the active control condition consisted of workshops on how students develop novice and expert knowledge in engineering and how students demonstrate their developing knowledge. Each condition is described in detail in the Intervention Procedures section below. Treatment and control TA workshops took place during regularly scheduled weekly TA meetings as part of their teaching assistantship responsibilities. The first workshop took place during the first week of classes, and the second workshop took place during the 6th week of classes. Teaching assistants and the overall course instructor were blind to TA conditions. At the conclusion of the semester, control group teaching assistants were invited to participate in a motivation workshop the following semester. Within each section, students were randomly assigned to a utility value intervention group after enrollment in the course had stabilized (i.e., after the add/drop deadline had passed). Assignment to utility value conditions was blocked by gender, first generation college student status, and membership in an 36 underrepresented racial/ethnic group to ensure adequate representation of these groups across study conditions. The mindsets and belonging interventions were administered by the university, with 1/3 of incoming students assigned to the incremental condition, 1/3 to the belonging condition, and 1/3 to a control condition. Therefore, assignment to utility value conditions was also blocked by mindsets/belonging treatment groups to ensure an even distribution of mindsets/belonging intervention and control students across utility value conditions. This resulted in a 2 teacher conditions x 3 student incremental/belonging conditions x 2 student value conditions design (see Table 2). Survey Administration Student surveys. Self-report surveys assessing motivation and identity beliefs in relation to engineering were administered at the beginning of the semester (T1), near the end of the semester (T2), and during the following semester (T3). The first survey was administered as part of a larger study aimed at assessing the impact of first-year programs, so all first-year students listed as prospective engineering majors were invited to take the baseline survey at a required orientation meeting a few days before courses started. All students who were enrolled in EGR 100 received course credit for taking the baseline survey. Any students who were enrolled in the course but not in the target group for the larger study were invited to take T1 for the current study via email and an in-class announcement on the first day of class. Students were encouraged to complete the survey before classes began, although students were also allowed to complete the survey during the first week of classes, as the larger course lecture did not meet until 1.5 weeks into the semester. The T2 survey was administered as part of homework in the course during week 13 of the semester, and a final (T3) survey was administered as part of the larger engineering study near the end of the following semester, with incentives (course credit or gift cards) depending on enrollment in follow-up target courses. Students enrolled in target courses (e.g., introductory courses typically taken in the second semester) were invited via announcements in class and received course credit for taking the survey. Students receiving course credit for completing surveys were able to complete the surveys for credit even 37 if they chose not to participate in the study. Students not enrolled in these courses were invited via email and were offered a $10 gift certificate for taking the survey. TA surveys. TA surveys were administered during the first week of classes (TAT1) and at the end of the semester (TAT2). The first survey was administered during the first TA workshop, and the second was administered online via an email invitation. Intervention Procedures Student-level interventions. The implicit theories of intelligence (mindsets) and belonging interventions were conducted during the summer before first-year students arrived at the university, and were modeled after interventions designed by Dweck, Walton, Cohen, Yeager, and colleagues (e.g., Walton & Cohen, 2011; Yeager et al., 2016). These interventions were part of a university-wide study, with all entering freshmen randomly assigned to participate in one of the two interventions or a control condition. Students in treatment and control conditions completed online sessions consisting of instructional videos followed by a series of questions. All students were also asked to complete a brief writing exercise following the final online session. In the treatment conditions, students viewed and responded to content about the malleability of the brain (mindsets) or about overcoming setbacks in college (belonging). In the mindsets intervention, students viewed content explaining how the brain builds connections over time and in response to experiences. In the belonging intervention, participants viewed clips of older students who described experiences of adversity in college and how those experiences had been common among first-year students rather than reflections of low belonging (e.g., due to marginalized group membership). In both interventions, students answered comprehension questions following each video module, and after the second module students were asked to use the concepts they had learned to write a letter to a future student who might be struggling. The letters enacted a “saying is believing” process (Aronson et al., 2002), a tactic of “stealth” interventions (Robinson, 2010) by which participants are persuaded to internalize a message by explaining and thus endorsing beliefs about belonging and growth mindsets without being directly instructed to change their beliefs. For the current study, I obtained students’ intervention group status directly from the study administrators. 38 The utility value (UV) intervention was modeled after work by Harackiewicz, Hulleman, and colleagues (Harackiewicz et al., 2016; Hulleman et al., 2017). The intervention consisted of three writing exercises assigned and graded as homework in the course. Students in both treatment and control conditions were asked to complete the writing assignments during weeks 2, 5, and 8 of the semester. Students in the treatment condition were prompted them to a select specific concept that was covered in class that week and write an essay of at least 200 words explaining the relevance of the concept to their own lives. Students in the control group were asked to write an essay of at least 200 words summarizing a specific concept that was covered in class, but were not prompted to write about the relevance of the topic to their lives. Full text of the writing prompts is displayed in Table 3. Essays were graded by 13 undergraduate graders employed by the overall course instructor. Graders provided personalized feedback to enable students to improve on subsequent assignments. Grading and feedback were based on rubric criteria such as organization, content (including personal relevance for the treatment condition), spelling, and grammar. Names were removed from essays and replaced with number IDs before grading, and graders were randomly assigned to grade groups of essays within the same condition throughout the semester. After grading was complete and students received their grades and feedback, section TAs were provided with a list of students’ grades on the assignment. This procedure ensured that both graders and TAs remained blind to students’ writing assignment conditions. TA intervention. TAs in both treatment and control conditions participated in the study workshops during their regularly scheduled weekly TA meetings. The treatment and control TA workshops were facilitated by the author and consisted of two sessions each. Prior to the first session, TAs in the treatment condition received an email about the workshop and were invited to read an article about instructional design principles for supporting motivation (Linnenbrink-Garcia et al., 2016). TAs in the control condition also received an email about the upcoming workshop and were invited to read an article about developing expert knowledge in a domain (National Research Council, 2000). Taking an indirect approach (Yeager et al., 2016), communication about the workshops was framed as being part of regular training, with participation in the study as soliciting TAs’ help in translating the ideas from our 39 research to their peers. To minimize contamination effects, TAs in both groups were advised of the research design, informed that they were randomly assigned to a condition, and asked not to discuss the contents of the articles or workshops with TAs in the other condition. The first one-hour session took place during a business-as-usual TA meeting in the first week of classes. In the treatment condition, TAs completed a short survey assessing their beliefs about teaching in general and their beliefs about their ability to support students’ motivation. Following the survey, I delivered an interactive presentation highlighting five instructional design principles for supporting motivation (Linnenbrink-Garcia et al., 2016), key research findings supporting the design principles, examples of how design principles can be enacted, common barriers to implementation, and practical tips including the importance of practice and persistence for instructors learning new strategies. Group discussions and writing prompts were embedded throughout to allow TAs to reflectively apply the concepts to their own course and discuss real-world examples. Following the presentation (~30 minutes), TAs read two short descriptions of hypothetical TAs: one described an instructor who uses motivationally-supportive practices and the other described an instructor who uses practices that undermine motivation (adapted from Reeve et al., 2014; see Table 4). This exercise, along with the self-assessment survey TAs took at the beginning of the workshop, were used as prompts for self-reflection and discussion about specific ways to implement motivationally supportive practices in their classrooms. Finally, instructors collaborated to develop specific strategies for implementing the design principles in the course and brainstorm solutions to barriers that may arise when trying to implement the strategies. First, they wrote individually about one specific goal or practice they could adopt in alignment with each of the design principles. After writing, they shared their ideas in small groups and discussed specific situations in which each practice could be used. Instructors in the control group participated in a similarly structured workshop focused on how students develop knowledge in engineering. TAs completed a survey assessing their general beliefs about teaching, then participated in an interactive presentation highlighting six principles of developing expert knowledge in a domain, including differences between novice and expert knowledge, how to recognize 40 novices and experts, the importance of developing pedagogical expertise in addition to content knowledge, and advice from past TAs in the course. TAs were prompted to discuss how the principles applied to their course, and were asked to write about one specific goal or practice they could adopt in alignment with each expert knowledge principle. Results of the writing portion were shared in small groups, and TAs were asked to try implementing their goals in the course. The second TA workshop took place at mid-semester (week 7) during a weekly, hour-long TA meeting. Both workshops involved a brief review of the principles learned in the first workshop, then expanded on these ideas through discussion of TAs’ experiences in the classroom to that point, and generation of advice for future TAs on how to support motivation (treatment group) or markers of novice and expert knowledge in engineering (control group). In the treatment group workshop, instructors reported their experiences supporting motivation in the semester thus far and collaborated to evaluate and refine their practices. TAs in the control group reported their observations of students’ progress developing content area knowledge, and developed a list of “markers” of students’ novice and expert knowledge. Measures Constructs of interest were assessed via self-report, observation, and directly from institutional data. A construct list and measurement occasions for each construct are listed in Table 5, and all survey items are listed in Appendix C. Reliabilities and confirmatory factor analyses are reported in the Results section. TA Assessments Short surveys assessing TAs’ beliefs about teaching and demographic information were administered at the beginning (TAT1) and end of (TAT2) the semester. TAs responded to items about personal teaching efficacy, motivation self-efficacy, and design principles self-efficacy items using a Likert-type scale with 1 = Not certain at all to 5 = Absolutely certain. TAs’ beliefs about the value of teaching were assessed on a Likert-type scale with 1 = Strongly disagree to 5 = Strongly agree. 41 Personal teaching efficacy. Two items from PALS (Midgley et al., 2000) assessed TAs’ confidence in their ability to teach generally (e.g., How certain are you that you can help all students make significant improvement?”). Teacher motivation self-efficacy. Four items from Skaalvik and Skaalvik (2007) were used to assess TAs’ self-efficacy for supporting students’ motivation (e.g., “How certain are you that you can get students to do their best even when working with difficult problems?”). Because these items could lead to contamination by prompting TAs in the control group to focus on their students’ motivation, these items were administered to the control group TAs only in the TAT2 survey at the end of the semester. TAs in the treatment condition responded to these items at both TAT1 and TAT2. Design principles self-efficacy. Survey items were also used to assess TAs’ confidence in their ability to execute specific practices based on the five design principles, including competence (5 items, “Provide students with an appropriate level of challenge”), autonomy (3 items; “Provide students with opportunities to make choices”), value/interest (5 items; “Make activities personally relevant to students”), emphasizing mastery and de-emphasizing performance (4 items; “Emphasize learning and understanding”), and belonging (4 items; “Build strong relationships with my students”). These items were administered to TAs in the treatment condition at both time points, but to TAs in the control group only at TAT2. Value for teaching. TAs also completed 8 items indicating their beliefs about the value of teaching, adapted from Conley (2012). Items assessed utility value (2 items, “Teaching engineering will be useful for me later in life”), attainment value (2 items, “Being someone who is good at teaching is important to me”), interest value (2 items, “I enjoy teaching engineering”), and cost (2 items, “For me, teaching engineering may not be worth the effort”). Student Survey Assessments Multi-item self-report measures were used for the baseline (T1), end-of-semester (T2), and post- semester (T3) surveys to assess students’ motivation and identity beliefs. Students also responded to items about the motivational climate of their EGR 100 lab section in the T2 survey. 42 Task value. Students were asked at all three time points about their value for engineering. Utility value (5 items, “Engineering is practical for me to know”), attainment value (4 items, “Being someone who is good at engineering is important to me”), and interest value (5 items, “I enjoy doing engineering”) were assessed using scales adapted from Conley (2012). Engineering self-efficacy and academic perceived competence. Students reported their perceived ability to successfully complete authentic engineering tasks (12 items, “I can identify a design need”) as well as their confidence in their ability to complete academic tasks in engineering courses (5 items, “I can learn the content taught in my engineering-related courses”). These items were adapted from engineering self-efficacy scales developed by Mamaril, Usher, Li, Economy, and Kennedy (2016). Achievement goals. Mastery goals (“One of my goals in engineering is to learn as much as I can”), performance-approach goals (“It’s important to me that other students think I am good at engineering”), and performance-avoidance goals (“It’s important to me that I don’t look stupid in engineering”) were measured at all three time points using three 5-item scales adapted from PALS (Midgley et al., 2000). Theories of intelligence. Students’ beliefs about the malleability of intelligence, including separate items for incremental (or growth, “You can always substantially change how intelligent you are”) and entity ( or fixed, “To be honest, you can’t really change how intelligent you are”) theories, were assessed at T1 and T2 using an 8-item measure from Dweck (1999). Engineering identity. Students’ identification with engineering (9 items, “I have come to think of myself as an ‘engineer’”) was assessed using a measure adapted from Pugh, Linnenbrink-Garcia, Koskey, Stewart, and Manzey (2009) and Estrada et al. (2011). While these items have not been used together in prior research, and it’s possible they form one identity factor, based on prior data from MSU engineering students, I anticipated that the items would arise from three identity factors: belonging, centrality, and future self. Belonging in engineering. Students’ sense of belonging in the college of engineering (3 items), with their peers in engineering (2 items), and with engineering professors (2 items) was assessed using a 43 scale adapted from Mendoza-Denton, Downey, Purdie, Davis and Pietrzak (2002). Items began with the stem, “Indicate the number that best describes your current feelings about [the College of Engineering/your classmates/peers in your engineering classes/your engineering professors,” and students rated items on a scale from 1 to 10, with labels referring to belonging in each category (e.g., 1 = “miserable”, 10 = “thrilled to be here”; 1 = do NOT feel comfortable with them, 10 = feel comfortable with them). Belonging with peers and the TA in EGR 100 was assessed at T2 using the same measure adapted to reflect the course-level constructs. Motivational climate. Perceived autonomy support (6 items; “My EGR 100 TA provides me with choices and options”), perceived teacher control (4 items; “My EGR 100 TA puts a lot of pressure on me”), and perceived competence support (4 items; “My EGR 100 TA praises my efforts and strategies”) were measured at the end of the semester (T2) using scales adapted from Jang, Kim, and Reeve (2016). Students’ perceptions of the TAs’ achievement goals have been shown to be more cognitively valid than students’ perceptions of the classroom goal structure (Koskey, Karabenick, Woolley, Bonney, & Dever, 2010). Therefore, students’ perceptions of TAs’ mastery goals (6 items; “My EGR 100 lab TA thinks trying hard is very important”), performance-approach goals (3 items; “My EGR 100 lab TA tells us how we compare to other students”), and performance-avoidance goals (4 items; “My EGR 100 lab TA tells us that it is important that we don’t look stupid in class”) were assessed near the end of the semester using measures adapted from PALS (Midgley et al., 2000) and Koskey et al. (2010). Lastly, instructors’ use of strategies that connect course concepts to real life (4 items; “My EGR 100 lab TA relates course material to real life”) and students’ perceptions that their instructor is personable (11 items; “My EGR 100 lab TA is friendly”) were assessed at T2 using measures adapted from Godin et al. (2015) and Linnenbrink-Garcia et al. (2013). Observational Assessments Observational measures were used at mid-semester (weeks 8), one week following the second TA training, to assess instructors’ use of motivationally supportive practices. Each TA was observed for one hour by two trained graduate and undergraduate observers who were blind to the TA conditions. 44 Observation protocols were adapted from Reeve et al. (2004), Turner et al. (2014), and an unpublished codebook by Linnenbrink-Garcia, and were aligned with design principles for supporting motivation (Linnenbrink-Garcia et al., 2016; see Appendix D). The protocol prompted observers to count the number of time TAs employed a set of specific behaviors in alignment with each of the design principles in increments. Observers tallied each time a TA exhibited a specific behavior (e.g., “Instructor praises student(s) for effort or strategy use”). A full list of the observation protocol items is presented in Appendix D. Lab instruction was also audio- and video-recorded to facilitate additional coding and for the TAs to view as part of their professional development. Institutional Data Lastly, achievement and persistence in engineering majors was obtained directly from course instructors and from institutional records. Achievement and persistence. Exam, assignment, and overall course grades were obtained directly from the instructor, and students’ GPA in the subsequent semester as well as their major status at the end of the following semester was obtained from university records. Prior achievement. Math placement test scores, an indicator of prior achievement, were obtained directly from university records. While prior studies have used the first exam in the course as an indicator of prior achievement (e.g., Harackiewicz et al., 2016), the incremental theories of intelligence intervention took place before the beginning of the semester, therefore earlier achievement indicators were necessary. Analytic Plan The data analytic plans for fidelity checks, manipulation checks, and research questions are presented in Table 6. Preliminary analyses included an examination of all continuous data. I also assessed measurement properties of the multi-item student survey scales using Cronbach’s alpha and confirmatory factor analyses (CFA). Missing data analyses were conducted to explore potential bias introduced by systematic missing data, and fidelity checks examined whether randomization procedures and experimental procedures were successful in enacting the intervention model. Manipulation checks 45 examined the effectiveness of the interventions in shifting the proximal psychological and behavioral processes proposed by the intervention’s theory of change as mechanisms for more distal outcomes. Analyses of main effects for each of the three interventions (RQ#1) and combined effects of the three interventions (RQ#2) consisted of multiple analyses of variance (MANOVA) with categorical indicators of treatment/control conditions predicting all continuous outcome variables (values, achievement goals, perceived competence, course grade, identity, and GPA). Logistic regression analyses were used to assess main and interactive effects on retention in engineering. To determine whether prior achievement moderated any of these main effects (RQ#1c), I conducted multiple analyses of covariance (MANCOVA) and logistic regression analyses, with prior achievement as a covariate in each model and interacting with treatment conditions. Power analyses. A priori power analyses with a full factorial 2 x 2 x 3 design and one covariate, α error probability = .05, suggested that a total sample size of 639 was needed to detect an effect size of .20. Prior research suggests that effect sizes range from 0.08-0.38 for utility value interventions, 0.36-0.76 for mindsets interventions, -0.04-.1.57 for belonging interventions, and 0.49-.1.94 for teacher-level interventions (Lazowski & Hulleman, 2016; Harackiewicz et al., 2016; Hulleman et al., 2010; Reeve et al., 2004; Rubie-Davies et al., 2015; Walton & Cohen, 2007). 46 CHAPTER 4: Results Preliminary Analyses Confirmatory Factor Analyses Confirmatory factor analyses using FIML estimation were conducted on the full student sample prior to computing composite scores for the latent constructs measured at the student level only. Confirmatory factor analyses were not possible at the TA level due to insufficient sample size. Reliabilities were acceptable for all scales, and are reported in Table 7. Theories of intelligence. Growth and fixed mindset variables were measured at T1 as manipulation checks for the growth mindsets intervention. Following Dai and Cromley (2014), I tested a two-factor structure for theories of intelligence (incremental and entity), which showed acceptable fit at T1, χ2 (19) = 166.72, RMSEA = .09, CFI = .97, TLI = .96. Belonging. Perceptions of belonging were also measured at T1 as manipulation checks for the belonging intervention. A three-factor model of belonging (belonging in the college of engineering, with engineering classmates, and with engineering professors) fit the data well at T1, χ2(17) = 126.23, RMSEA = .08, CFI = .98, TLI = .96. Motivational climate. Motivational climate variables were assessed at T2, and consisted of students’ reports of their TAs’ motivationally supportive practices (perceived need support, perceived TA achievement goals, etc.). CFAs for the motivational climate variables were somewhat exploratory, as factor structures of many of these variables have not been examined in prior research. In addition, no prior research has established the separability of these motivational climate variables. An initial eight- factor model including perceived autonomy support, perceived teacher control, perceived competence support, TA mastery goals, TA performance-approach goals, TA performance-avoidance goals, connection to real life, and TA is personable factors did not converge to an interpretable solution. Specifically, this model resulted in a non-positive definite covariance matrix with estimated correlations among some variables as close to or higher than one. Follow-up exploratory factor analyses in Mplus 47 further indicated that students did not view some constructs as separate, such as performance-approach and -avoidance goals. Thus, perceived TA performance goals were combined into a single factor. TA mastery goals and need support (autonomy support and competence support) were also combined into a single factor, labeled “TA mastery/need Support.” The modified model included 5 factors: TA mastery/need support, perceived teacher control, TA performance goals, connection to real life, and instructor is personable. Two items (one perceived competence support item and one perceived teacher control item) were dropped due to low loadings. The 5-factor model showed acceptable fit, χ2 (655) = 3261.86, RMSEA = .07, CFI = .90, TLI = .89, with the exception of TLI being slightly below the cutoff. These five variables were used to assess whether the students perceived the classroom environment to vary as a function of the instructional intervention. Motivation and identity. For the main analyses examining the effects of the motivation interventions on motivation, identity, achievement, and persistence, a variety of indicators were assessed at T2 and T3. Initial CFAs established the factor structure and separability of scales from the same theory (e.g., three values and expectancy beliefs) and are described below. Following these initial CFAs, a larger, 10-factor model was used to establish the separability and factor structure of all motivation and identity variables used in the main analyses. Value and perceived competence. A four-factor model of engineering academic perceived competence and three task values (interest, attainment, and utility) at T2 fit the data well, χ2 (129) = 615.59, RMSEA = .07, CFI = .95, TLI = .94. The one-factor design self-efficacy model also fit the data well, χ2 (5) = 12.20, RMSEA = .04, CFI = .996, TLI = .99, however a two-factor model of academic perceived competence and design self-efficacy did not fit the data well, χ2 (35) = 2523.06, RMSEA = .31, CFI = .39, TLI = .22, and indicated that students did not view these as distinct constructs. Thus, design self-efficacy was not used in subsequent analyses. Perceived competence was retained rather than design self-efficacy to parallel prior intervention research examining effects on academic competence beliefs, and because I did not have specific hypotheses about intervention effects on design self-efficacy. 48 Achievement goals. A three-factor model of students’ personal achievement goals (mastery- approach, performance-approach, and performance-avoidance) at T2 fit the data well, χ2 (74) = 223.93, RMSEA = .05, CFI = .98, TLI = .97. Engineering identity. As hypothesized, the one-factor model of T3 engineering identity including all nine items did not show acceptable fit, with high RMSEA, χ2 (27) = 338.871, RMSEA = .13, CFI = .93, TLI = .91. A three-factor model with belonging, centrality, and future-self components of engineering identity showed improved fit as indicated by a change in CFI greater than .01, χ2 (24) = 196.660, RMSEA = .10, CFI = .96, TLI = .94, however RMSEA was still high and one item had a low loading (< .70). After dropping this item, the three-factor model showed acceptable fit, χ2 (17) = 123.99, RMSEA = .095, CFI = .97, TLI = .96, with RMSEA < .10. All motivation and identity variables. The 10-factor model combining all student motivation (values, perceived competence, achievement goals) and identity (centrality, belonging, future self) variables as modeled above fit the data well, χ2 (695) = 2020.60, RMSEA = .05, CFI = .94, TLI = .94. This provided evidence that participants viewed these variables as distinct and separable. Missing Data Participant flow. Inclusion criteria were based on completion of the experimental and control conditions. Students were included in the analytic sample if they (a) participated in one of the three conditions of the university-level intervention (mindsets, belonging, or control), (b) completed at least one of the three essays in their assigned treatment or control condition for the utility value intervention and (c) did not switch experimental conditions during the semester. Based on these inclusion and exclusion criteria, the analytic sample for the study was 682. Of the 1,107 students enrolled in the course, 927 (84%) completed at least part of the first survey, 761 (69%) completed at least part of the second survey, and 712 (64%) completed at least part of the third survey. Students who did not take one survey were still invited to take subsequent surveys; 51 students took the T2 survey after not taking T1, and 43 students took the T3 survey after not taking T1 or T2. In all, 85 students in the course did not take any of the three 49 surveys; the total number of students who took any of the three surveys and thus consented to participate in the study was 1,021 (92% of students enrolled in the course). Of the 1,021 students in the survey sample, 270 did not participate in the belonging or mindsets intervention (and did not participate the control condition) and so were excluded from the study. After these 270 were removed, an additional 61 students were excluded due to contamination of the TA condition (their assigned TA left the university mid-semester, and students were assigned a new TA). Finally, 6 students were removed (all from the treatment condition) because they completed no essays for the utility value intervention, and an additional 2 students were removed because they switched UV conditions mid-semester (due to students forwarding assignment emails among themselves). Thus a total of 682 students (67% of survey sample) remained in the analytic sample. Missing data analysis. In the first step of missing data analysis, students who met the inclusion criteria (n = 682) were compared to those who did not meet the inclusion criteria (completion of experimental and control conditions) but who consented to participate in the study (n = 339). First, inclusion vs. exclusion due to completion or non-completion of study procedures did not appear to be associated with experimental conditions, including the three levels of the university-wide brief social psychological interventions, χ2 (2) = .02, p = .99, the UV treatment vs. control, χ2 (1) = 1.33, p = .25, and TA treatment vs. control, χ2 (1) = 43, p = .51. In other words, non-completion of intervention procedures appeared to be evenly spread across experimental conditions, so subsequent comparisons for missing data analyses (e.g., students with complete vs. missing data) were made for the full sample, not broken down by experimental conditions. A MANOVA comparing Math ACT scores and spring GPA for students included vs. excluded from the study was statistically significant, Wilks’ Λ (693, 2) = .935, p < .001, η2 = .065, providing evidence that those who participated in the study conditions were systematically different from those who did not complete one of the study conditions. Follow-up comparisons indicated that the two groups differed in terms of both Math ACT and spring GPA, with students included in the study having higher GPAs (M = 3.17, S.E. = 0.04) and ACT scores (M = 29.29, S.E. = 0.16) than students excluded from the study (Grades: M = 2.86, S.E. = 0.06; Math ACT: M = 27.51, S.E. = 0.23). Students 50 included vs. excluded also differed in terms of first-generation college student status, χ2 (1) = 11.75, p < .001, and membership in an underrepresented racial/ethnic group, χ2 (1) = 9.50, p = .002; first-generation college students and students from underrepresented racial/ethnic groups were significantly more likely to be excluded from the study. Gender, however, was not significantly related to inclusion in the study, χ2 (1) = .07, p = .797. Of students in the analytic sample (N = 682), missing data for survey variables ranged from 7.5%-29.2%, with an average missing rate of 21.47%. Little’s missing completely at random (MCAR) test was significant, χ2 (723) = 963.81, p < .001, providing evidence for non-random patterns of missing data. Rates of missing vs. complete data did not appear to differ across experimental conditions, χ2 (11) = 12.43, p = .33, therefore subsequent missing analyses were conducted on the full sample. Potential correlates of missing data were explored using a MANOVA for prior achievement and spring semester grades and chi square analyses for categorical variables (gender, race/ethnicity, first-generation college student status, and condition), comparing participants with complete data (n = 256) to participants with any missing data (n = 426) at the construct level. The overall MANOVA comparing Math ACT and spring semester grades for those with missing and complete data was significant, Wilks’ Λ (463, 2) = 9.96, p < .001, η2 = .041. Follow-up comparisons indicated that students with complete data had higher spring semester GPA (M = 3.33, S.E. = .05) than students with missing survey data (M = 2.98, S.E. = .06), but the two groups did not significantly differ on Math ACT (Mcomplete = 29.50, Mmissing = 29.03). Students with missing vs. complete data also appeared to be evenly distributed across gender groups, χ2 (1) = .001, p = .973, and experimental conditions, χ2 (11) = 12.43, p = .332. First-generation college student status, χ2 (1) = 3.90, p = .048, and race/ethnicity, χ2 (1) = 4.00, p = .046 were associated with missing data, however, with continuing-generation students and racial/ethnic majority students being more likely to have complete data compared to first-generation and underrepresented racial/ethnic minority (URM) college students. The results of the missing data analyses indicated that inclusion in the study and missing data on particular variables was associated with prior achievement, end-of-year achievement, first-generation 51 college student status, and URM group membership. Thus, these variables were used as covariates of missing data in multiple imputation to minimize bias in parameter estimates due to missing data. Multiple imputation. Because patterns of missing did not appear to differ across experimental conditions, multiple imputation was conducted to impute missing dependent variables for the full sample together, including the correlates of missing data identified above. Missing data was not imputed for Spring 2018 engineering major status, as only 8 cases were missing and this variable was examined in a separate analysis. Five imputed data sets (Schafer, 1999) were created using the automatic imputation setting in SPSS version 24. In addition to the correlates of missing data identified above, composite scores for survey measures on other survey waves were also included as correlates of missing data. Multiple imputation was completed after identifying and adjusting outliers as described below. Descriptive Statistics and Correlations Scales were computed by taking the mean of the individual items. For all continuous study variables, including dependent variables used in manipulation checks and main analyses, univariate outliers were identified using Grubbs’ (1969) test and adjusted to the highest retained value (< 2.5 standard deviations from the mean) that reflected the original high or low direction indicated by responses. No univariate outliers were identified for the majority of study variables. However, T2 engineering perceived competence, interest value, attainment value, utility value, and mastery-approach goals, as well as the future self sub-component of T3 Engineering Identity, each had 2-7 univariate outliers, all indicating particularly low levels of each construct. These outliers were adjusted to reflect the lowest value retained in the data. After adjusting outliers, all variables displayed skewness (< 2) and kurtosis (< 3) within acceptable ranges of normality, with the exception of the course grade, which was negatively skewed and leptokurtic (see Table 7). Multivariate outliers for all continuous variables used in the main analyses were identified by computing Mahalanobis distance (Tabachnik & Fidell, 2007). Overall, 8 cases were identified as multivariate outliers. Rather than adjusting or excluding these cases, the main analysis was conducted 52 with and without these cases in order to examine whether the robustness of the results was influenced by the presence of multivariate outliers. Descriptive statistics and reliabilities for all student-level variables are presented in Table 7. Correlations are presented in Table 8. Intraclass correlations indicated that less than 2% of the variance in all outcome variables was accounted for by the section TA, with five of the twelve outcome variables showing less than .01% of the variance at the TA level. Thus, it was not necessary to account for between-section variance in the analyses. Fidelity and Manipulation Checks Fidelity and manipulation checks were conducted on the analytic sample only and prior to conducting multiple imputation, and thus students missing one or more variables in the analysis were not included in these analyses. Treatment fidelity is often used as an umbrella term encompassing both procedural and psychological aspects of interventions implementation (Nelson, Cordray, Hulleman, Darrow, & Sommer, 2012). For clarity and in alignment with the broader psychological literature, I use fidelity to refer to how well the components of the intervention were delivered (i.e., the successful completion of study procedures; O’Donnell, 2008). Fidelity checks enhance understanding of whether unfavorable results were due to an “ineffective intervention model or the failure to implement the model fully” (Nelson et al., 2013). In contrast, manipulation checks examined whether the interventions successfully enacted the proximal psychological and behavioral processes proposed by the intervention logic model (Hulleman & Cordray, 2009). First, I examined whether random assignment to conditions resulted in treatment and control groups that were approximately equivalent before intervention procedures. Next, I conducted fidelity checks for the utility value intervention and manipulation checks for the other three interventions. Randomization check. A MANOVA comparing prior achievement (Math ACT) and baseline motivation variables (incremental and entity theories, three types of belonging, and three types of value) across utility value treatment (n = 211) and control (n = 225) groups was not significant, Wilks’ λ (426, 9) = 0.985, p = .673, partial η2 = .015. A MANOVA comparing these same student variables (Math ACT 53 and baseline motivation variables) across TA treatment (n = 251) and control (n = 185) conditions was also non-significant, Wilks’ λ (426, 9) = 0.982, p = .557, partial η2 = .018. These results indicated that random assignment with blocking for the utility value and TA training interventions at the student level was effective in balancing key baseline student characteristics. An ANOVA comparing students’ prior achievement across mindsets intervention, belonging intervention, and control groups indicated that students’ prior achievement did not significantly differ across the three conditions, F (468, 2) = .129, p = .897, partial η2 = .001. Lastly, descriptive examinations of baseline TA data indicated that random assignment to conditions resulted in approximately equivalent teaching experience in terms of the number of prior semesters TAs had taught introductory engineering across the two conditions (Mcontrol = 1.50, Mtreatment = 1.40). Means for TAs’ reported motivation for teaching are displayed in Table 9. In general, TAs in the treatment group appeared to have higher value for teaching and lower costs associated with teaching compared to the control group TAs at baseline (pre-intervention). Statistical tests of mean differences were not possible due to the small sample size of TAs (N = 8). Teacher training. As a manipulation check for the implementation of the teacher training component, two indicators were used as evidence for whether or not TAs’ confidence and behavior regarding motivationally supportive practices increased after participation in the treatment workshops. First, instructor self-report data were examined for evidence of within-person change and end-of-semester between-group differences in instructor confidence for supporting motivation. Due to low sample sizes, statistical tests were not possible and therefore these data were used descriptively. Overall, TAs appeared to show positive shifts in their motivation for teaching and in their self-efficacy for supporting students’ motivation, although there did not appear to be notable, systematic patterns of differences in mean levels or in changes from TAT1 to TAT2 across treatment and control groups. Second, students’ end-of-semester motivational climate survey responses were compared across treatment and control groups using a MANOVA assessing differences in students’ perceptions of teachers’ motivationally supportive practices. The effects of teacher training on teacher practices was 54 assessed by comparing perceived motivational climate variables reported by students (perceived autonomy support, perceived control, competence support, TA achievement goals, connection to real life, and instructor is personable) across treatment (n = 299) and control (n = 211) TA groups. The overall MANOVA was significant, Wilks’ λ (677, 5) = .96, p < .001, partial η2 = .04. Follow-up univariate tests indicated that TA mastery goals/need support (Mtreatment = 3.68, Mcontrol = 3.88), connection to real life (Mtreatment = 3.38, Mcontrol = 3.70), and instructor is personable (Mtreatment = 3.68, Mcontrol = 3.93) variables differed significantly across conditions. However, the differences were all in the opposite direction of what was expected, indicating that students perceived the control group TAs to be significantly more motivationally supportive than the treatment group TAs along all dimensions except for TA control, TA performance-approach goals, and TA performance-avoidance goals, which did not show significant differences across conditions. Utility value. To evaluate whether the utility value intervention prompted students to make more personal connections to the course material, a random sample of essays (~10% from each condition) was coded for the degree of personal connection in the essay, mirroring the fidelity check by Harackiewicz and colleagues (2016). Trained research assistants coded essays on a scale from 0 to 4, with 0 = no personal/value connections, 1 = “general utility applied to humans generically,” 2 = “utility that is general enough to apply to anyone, but is applied to the individual,” 3 = “utility that is specific to the individual,” and 4 = “strong, specific connection to the individual that includes a deeper appreciation or future application of the material” (Harackiewicz et al., 2016, p. 750). This metric was used to compare essays across the utility value and control conditions to ensure that students in the treatment condition made more personal connections than students in the control group. In total, 2,611 essays (1,298 treatment condition, 1,313 control condition) were collected from students who consented to participate in the study. A sample of the utility value and control group essays (n = 249, 118 treatment condition) was randomly selected for coding, with each essay coded by two independent coders. After the first round of coding, interrater reliability was high, with absolute agreement of 80%. Disagreements (n = 49) were resolved by consulting a third trained coder and through 55 discussion. A two-tailed independent samples T-test with equal variances not assumed (Levene’s test F = 86.41, p < .001) indicated that students in the utility value condition (M = 2.15, SD = .93) made more personal connections than control group students (M = 1.15, SD = .65) in their essays, t (205.77) = -9.82, p < .001. Mindsets. To test whether the mindsets intervention effectively manipulated students’ beliefs about intelligence, I compared students who completed the mindsets intervention (n = 195) to first-year students who completed the control condition (n = 194) on incremental and entity mindsets at T1. First- year students and upperclassmen were examined in separate analyses, as upperclassmen completed the interventions over a year prior to the beginning of the study, and so specific effects on theories of intelligence may not have been expected to persist. The overall test was not significant, Wilks’ λ (386, 2) = .99, p = .12, partial η2 = .011. These results indicated that the mindsets treatment did not significantly impact students’ beliefs about the malleability of their intelligence at the beginning of the semester compared to the control group. For upperclassmen who completed the T1 survey and participated in the intervention (n = 21), and whose intervention participation took place at least one year prior to the T1 survey, the overall MANOVA was also not significant, Wilks’ λ (18, 2) = 0.97, p = .74, partial η2 = .03, indicating that there were no differential effects of the intervention across first-year and upperclassmen groups within the course. Belonging. Similar to the manipulation check for the mindsets intervention, the test for the belonging intervention compared first-year students who completed the belonging intervention (n = 219) to first-year students who completed the control condition (n = 192) on three types of belonging reported at the baseline survey. The overall test was non-significant, Wilks’ λ (407, 3) = 0.99, p = .27, partial η2 = .01, indicating that compared to students in the control group, the belonging intervention did not impact students’ Time 1 sense of belonging in the college of engineering, belonging with their peers in engineering courses, or belonging with engineering professors. Similarly, a MANOVA examining whether there were long-term effects on the three types of belonging at T1 for upperclassmen was not significant, Wilks’ λ (17, 3) = 0.96, p = .85, partial η2 = .05, indicating that effects did not differ at T1 for 56 first-year students vs. upperclassmen, although only 21 students were included in the upperclassmen analysis. Summary of fidelity and manipulation checks. Overall, treatment fidelity and manipulation checks for each intervention revealed that only the utility value intervention’s prompt for students to reflect on the relevance of course material appeared to be successful. Despite the non-significant results for many of the interventions, I proceeded with the planned analyses examining the main research questions to explore whether (1) the interventions succeeded in shifting other forms of motivation beyond those explored in manipulation checks, (2) combined interventions exhibited effects on outcomes, and (3) interventions had effects only for students with low prior achievement. In particular, the planned analyses would allow me to understand whether a particular intervention might only work for students with low prior achievement, and whether a particular intervention might only show effects when combined with other intervention approaches. Main and Interactive Effects of Motivation Interventions The imputed data set was used for the analyses addressing the main research questions, including main and interactive effects of the motivation interventions on students’ T2 motivation, T3 engineering identity, spring semester GPA, and spring retention in an engineering major. Pooled results, calculated using the averaged estimates provided across the five imputed datasets, are presented for the MANOVA and MANCOVA analyses below. Although only 8 cases were missing spring major status and imputation was not used for the spring major variable, imputed data was also used for logistic regression analyses due to the missing data for the covariate, Math ACT scores. All analyses were run with and without the cases identified as being multivariate outliers, and results did not differ substantially with the exclusion of these cases (see Appendix E). Research questions 1a and 1b asked whether there were main effects of each intervention on achievement goals, values, perceived competence, identity, achievement, and engineering major retention. For the overall MANOVA examining main and interactive effects of the interventions on motivation, identity, and grades, Levene’s tests for homogeneity of variance across conditions were largely not 57 significant (see Table 10), providing support for the assumption of homogeneity of variance. However, Levene’s test was significant for utility value, F (11, 670) = 2.03, p = .029, and spring GPA, F (11, 670) = 2.56, p = .004, indicating that the error variance of utility value and GPA across groups was not equal. Implications of this violation of assumptions will be explored in the discussion. Results of the MANOVA are presented in Table 11. Overall, contrary to my hypotheses, no significant main or interactive effects were found. However, as the interaction between utility value conditions and incremental/belonging conditions had a p value of .105, and particularly considering the large number of constructs examined as outcomes, follow-up univariate comparisons were examined as an exploratory analysis to understand potential avenues for future research. Univariate comparisons (see Table 12) indicated that the interaction between utility value conditions and incremental/belonging conditions was significant for the belonging component of engineering identity, F (2, 670) = 4.92, p = .012, partial eta squared = 0.01. However, comparisons of estimated marginal means revealed no significant differences in mean levels of engineering identity (see Figure 1), as all estimated confidence intervals were overlapping. The pattern of estimated marginal means, despite non-significant differences, showed that students who participated in the control conditions for both utility value and mindset/belonging interventions appeared to have the highest mean levels of engineering identity, followed by students who participated in both treatment conditions (utility value + belonging or mindsets). Students who participated in only one of the interventions appeared to have the lowest mean levels of engineering identity. Logistic regression analyses were conducted in two steps. The first step included the main effects of each condition (Model 1), and the second included two- and three-way interactions (Model 2). Results are presented in Table 13. Similar to the MANOVA results and contrary to my hypotheses, the logistic regression analysis examining main and interactive effects of the interventions on spring semester retention in an engineering major revealed no significant effects. Each step of the analysis did not explain significantly more variance than prior steps (Step 1 χ2 (4) = 2.99, p = .56; Step 2 χ2 (5) = 7.12, p = .21; 58 Step 3 χ2 (3) = 3.49, p = .17, and overall the final model explained only a small amount of variance in the engineering major retention outcome (Cox & Snell R2 = .02, Nagelkerke R2 = .04). Moderation Analyses The potential differences in effects of the interventions based on prior achievement were examined using MANCOVA for continuous outcomes and logistic regression for the major retention outcome. The MANCOVA was conducted using the same procedure as described above with the addition of prior achievement (Math ACT scores) as a covariate and interacting with each of the main and interactive effects of the intervention. Table 14 shows the results of the multivariate tests, which indicated that no significant effects or interactions were observed among the intervention conditions and prior achievement (Math ACT), with the exception of a significant interaction between the utility value and TA conditions, as well as a significant interaction among utility value and TA conditions with Math ACT. Follow-up univariate tests (Table 15) indicated no significant univariate differences, however. Similar to the first logistic regression examining main and interactive effects of the interventions on engineering major retention, the interactions of the intervention conditions with prior achievement (Math ACT scores) on major retention were not significant (see Table 13). In addition, very large regression estimates and odds ratios indicated the results were likely untrustworthy, perhaps indicating it was unnecessary to control for prior achievement and include interactions with prior achievement, particularly for the belonging intervention. 59 CHAPTER 5: Discussion Teachers and college administrators in STEM fields have identified supporting students’ motivation as a top priority (Trygstad, 2013; National Academies of Sciences, Engineering, and Medicine, 2016). Students themselves have identified motivation problems such as loss of interest, low feelings of belonging in STEM, low perceived competence to meet learning demands, and high costs associated with course demands as key factors leading to STEM dropout in college (Seymour & Hewitt, 1997). Theory and empirical evidence suggest that classroom features can support or undermine motivation (Ames, 1992; Linnenbrink, 2005), and prior research examining the effectiveness of training instructors to support motivation has provided evidence that a relatively short training period can shift teacher behaviors almost immediately, with subsequent gains in student engagement and achievement (Reeve et al., 2004). A rich tradition of theory and research supports the idea that motivation is multifaceted, complex, and sensitive to both interpersonal and intrapersonal forces. In essence, both the quantity and the quality of motivation matter when it comes to supporting student motivation and designing optimally supportive learning environments. In an attempt to better align research with practice and to address the issue of declines in motivation, achievement gaps, and attrition from STEM fields in higher education, this dissertation study examined the effectiveness of a multifaceted motivation intervention. By combining student-level interventions to support incremental theories of intelligence and value for engineering course content, along with an instructor training component designed to support multiple forms of motivation, I aimed to promote multiple pathways to optimal motivation and success for college engineering students. The randomized field experiment involved combining three popular, student- focused social psychological interventions with a teacher-focused intervention designed to help teachers enhance their motivationally supportive practices in the classroom. 60 Main Effects of Interventions The first aim of the study was to examine main effects of each intervention (utility value, growth mindset, belonging, and the TA workshop on motivationally supportive teaching practices) on a variety of student outcomes including motivation, identity, and achievement both in the course and in the following semester courses. Despite my hypothesis that main effects would be observed in accordance with prior studies and with theoretical expectations, no main effects were observed on any of the outcomes. Utility value intervention. First, contrary to prior research finding main effects of the utility value intervention on achievement (Canning et al., 2018; Harackiewicz et al., 2016), persistence (Canning et al., 2018), interest (Hulleman et al., 2010), utility value (Gaspard et al., 2015), and expectancy for success (Canning & Harackiewicz, 2015), the utility value intervention did not show any main effects on these variables. Furthermore, it did not show main effects on any other variables. Perhaps most notably, the intervention did not appear to shift utility value itself, the theorized mechanism for the effects of this intervention. However, it is true that only in two field studies have researchers shown effects on utility value (Gaspard et al., 2015; Rosenzweig et al., 2018). This finding of no effects on any of the focal variables suggests that within the context of this introductory engineering course, the utility value intervention did not provide an additional “boost” to motivation, identity, achievement, or persistence relative to students in the control condition. There are a variety of possible explanations for these null effects. In particular, I focus here on those most relevant to the conceptual model by which the utility value intervention enacts relevant processes. Specifically, prompting students to reflect on the relevance of the material to their lives promotes positive outcomes through students’ deeper and more personal engagement with the course material in settings that may otherwise pose a challenge to recognizing relevance (Harackiewicz et al., 2014). This increased engagement, spurred by motivation that results from personally connecting to course materials, is the proposed mechanism for subsequent higher achievement. Failures in the enactment of any of these steps or in the causal links between each of these steps are potential explanations for null effects of the intervention in this setting. 61 First, the engineering course provided a variety of opportunities for students to reflect on the value or relevance of the course material to their lives. For example, the first unit of the course was about common careers in engineering and how concepts learned in engineering coursework were needed for success in these careers. This unit may have functioned similarly to a “directly communicated” utility value intervention (e.g., Canning & Harackiewicz, 2015) for the entire class. More broadly, engineering subject matter itself may more readily lend itself to naturalistic valuing of the material compared to prior subjects targeted in utility value intervention research (math and biology). In other words, utility may be readily perceived in engineering due to the hands-on, real-world applications inherent in assigned projects and career considerations embedded in engineering programs. In contrast, the more abstract nature of mathematics or biology may mean that students need additional support to perceive utility in these fields. Indeed, in support of these explanations, levels of interest value and attainment value were quite high for the overall sample, with the mean above 4 on a scale of 1 to 5, indicating potential ceiling effects. A second explanation for the lack of effects of the utility value intervention may be that students’ participation in other interventions may have “washed out” effects, such that other interventions may have directly or indirectly boosted perceptions of value or expectancy that would normally be targeted only by the utility value intervention. Thus, particularly for comparisons across differential treatment conditions, there is a possibility that fanning out of effects made it impossible to detect specific intervention effects. It is interesting to note that the intervention itself “worked” in that it prompted students in the treatment condition to reflect on the relevance of the material to their lives significantly more than did students in the control condition. Mean levels of personal connection as found by trained coders indicated moderate levels of personal connection in the treatment group (M = 2.15 on a scale of 1 to 4), and quite low levels of personal connection in the control group (M = 1.15), which provides some evidence that the course material itself did not emphasize value and relevance sufficiently to prompt students to spontaneously and explicitly make these connections. However, a third potential explanation for null effects is that increased engagement in the course was not necessary for high achievement, thus explaining the lack of effects by unlinking the chain of events from intervention to achievement 62 outcomes. Indeed, the majority of students received high grades in the course despite varying levels of value and presumably varying levels of engagement in course activities as seen in all educational settings. This suggests that in contrast to other settings, the links between value, engagement, and achievement may not be as important as other processes for promoting success. Incremental and belonging interventions. Second, the university-administered interventions to promote incremental theories of intelligence (growth mindset) or belonging did not exhibit effects on any of the indicators of motivation, identity, or achievement relative to the control group. Interestingly, these interventions also did not show significant effects on students’ self-reported growth or fixed mindset beliefs or on perceptions of belonging at the beginning of college. It is worth noting, however, that typically the effects of growth mindset interventions are only measured in terms of their mitigating effects on “fixed” mindsets (Yeager et al., 2016), rather than looking at effects of both boosting growth and/or decreasing fixed mindsets. Therefore, manipulation check used for this study may have been overly strict. Indeed, examining follow-up univariate of mindset variables revealed that fixed mindsets were significantly lower in the treatment group, although the overall MANOVA was non-significant. For the belonging intervention, the lack of effects may show that the intervention does not function via belonging, as theorized, or at least via conscious, self-reported belonging perceptions. However, it is important to consider that the majority of students in the sample may not have benefited from these interventions because they are designed specifically for students at risk for belonging threat or stereotype threat. In prior research on belonging and mindsets interventions, effects were localized to Black students in college (Aronson et al., 2002; Walton & Cohen, 2007) and women in engineering (Walton et al., 2015) rather than students with low prior achievement. These prior findings suggest that the belonging intervention may have shown effects for women and for racial/ethnic groups that are minoritized in engineering, as these groups are most likely to experience belonging threat or stereotype threat in engineering. The current study did not assess these potential interactions due to limitations of sample size. However, features of this context, such as high representation of women among the TAs, may have mitigated these threats enough that the interventions were not necessary. In order to better 63 understand the conditions and mechanisms by which social psychological interventions support underrepresented students, future research may need to assess belonging threat and stereotype threat directly, and further assess elements of the context that may also mitigate belonging threat. Future research on belonging and mindsets interventions should also consider the timing of measuring effects, as the interventions are theorized to enact recursive processes that unfold over time. Specifically, gains of the belonging and mindsets interventions may be seen much earlier or later than the assessments in the current study. However, due to the lack of specific effects on growth mindsets and belonging specifically, it is likely that these interventions simply were not effective or were not necessary within the sample for the present study. Indeed, constructs such as mindsets and belonging, mindsets in particular, have become a part of popular discourse to the extent that students may encounter support for growth mindsets in a variety of settings, and may themselves know the value of a growth mindset and so endorse the growth mindset whether or not they received the intervention. TA intervention. Lastly, the TA-focused intervention also showed no effects on student outcomes, and largely did not show effects on TAs’ motivationally supportive practices in the classroom. Contamination of effects is a possible explanation for the lack of differences in student outcomes, as TAs saw one another often and discussed teaching strategies at weekly meetings. While TAs reported that they discussed the content of the workshops with other TAs only a small to moderate amount (M = 1.86 on a scale of 1 to 4), it is unclear whether these discussions took place within or across treatment and control groups. In addition, the course of interest was designed to be a supportive course rather than a difficult “weed-out” course, the lack of observed differences in motivationally supportive practices across conditions could be due to pre-existing motivational supports already in place. Some dimensions of TA behavior, as reported by students, actually showed effects in the opposite direction of what was expected, with students reporting that treatment group TAs were actually less motivationally supportive than control group TAs. It is possible that the TA intervention somehow flattened or decreased intervention TAs’ use of motivationally supportive practices, or that the control group workshops on knowledge development somehow led to an increase in motivationally supportive 64 practices. In the presence of comparable treatment and control groups and a larger sample of TAs, these explanations would be more likely to explain these effects. However, due to the small number of TAs and the results of the T1 TA survey, the most likely explanation for these reverse effects is that by chance, the control group contained TAs who were generally already more motivationally supportive on average. In other words, due to the small sample size, randomization did not result in groups that were roughly equivalent and thus comparable at T1, and thus the differences cannot be causally attributed to the intervention. Descriptively, the results of the T1 TA survey show that the control group TAs appeared to endorse more confidence in their ability to support students’ motivation. Furthermore, my informal observations of the TAs supported students’ reports that the control group TAs were more motivationally supportive. Thus, it may actually be a promising result that student outcomes were not statistically different across groups, as those in the TA intervention group did not show worse outcomes on motivation, identity, or achievement compared to students in the control group, despite the fact that initially the TAs in the treatment group appeared to be less confident about their ability to support students’ motivation and on some dimensions, were perceived by the students to be less motivationally supportive. Interactive Effects of Motivation Interventions The second aim of the study was to explore whether motivation interventions could be combined to enhance their effects, as might be expected from theoretical conceptualizations of motivation as an amalgam of dynamic, interconnected processes. Expectancy and value, for example, are theorized to interact such that high expectancy is most beneficial when value is also high, and vice versa, and this effect has been observed in a variety of correlational studies (e.g., Trautwein et al., 2012), such that for some students, high value may actually be detrimental when expectancy is low. Further, even if there were no main effects of each individual intervention, the combined interventions may have exhibited effects, and considering prior research finding greater effects on students with low prior achievement (Canning & Harackiewicz, 2015; Harackiewicz et al., 2016; Hulleman et al., 2010), the interventions may have only had effects on particular groups but not on others. 65 Contrary to my expectations, none of the tested interactions were statistically significant, indicating that the combined interventions did not exhibit significant effects on motivation, identity, or achievement. However, exploratory follow-up examinations of the interaction between the utility value and belonging/mindsets interventions revealed a trend indicating that for the belonging component of engineering identity, the effects of the utility value intervention might be enhanced when combined with either a mindset or a belonging intervention, with seeming opposite effects of the utility value intervention alone, overall. Indeed, the control/control condition, with no utility value intervention and no belonging or mindsets intervention, actually appeared to have the highest mean of engineering identity. This may indicate that no utility value intervention was best, unless it was combined with a belonging or mindsets intervention. Importantly, no conclusions can be drawn from this trend without further research. In addition, the interactions among interventions and prior achievement were also not significant, indicating that the effectiveness of the interventions did not appear to differ for low- vs. high-achieving students. Perhaps these are all unsurprising results, particularly given that the manipulation checks indicated that the belonging, mindsets, and TA interventions showed little to no differences across treatment and control groups on proximal measures of belonging, growth mindset, or motivationally supportive practices in the classroom. Implications This study gives rise to a number of considerations for implementing and evaluating interventions in novel contexts. First, the findings of possible ceiling effects, potentially due to the existing motivational supports embedded within the curriculum, point to the importance of pilot testing and fully understanding the motivational needs and resources within the context before selecting, tailoring, and designing motivation interventions to fit a particular context (Walton & Wilson, 2018). Indeed, this may be particularly and increasingly important for mindset interventions, given the popularity of “growth mindset” messages to the public, including teachers, parents, and students, and likelihood of students’ exposure to instruction that supports a growth mindset as well as social desirability effects, wherein they 66 are unlikely to indicate on a survey that they have low growth mindset because they know this is a desirable quality. The findings related to the TA intervention have implications for the design of teacher-focused interventions, particularly for sample size and dosage considerations. First, the relatively “light touch” approach of only two brief interactive workshops may not be enough time for instructors to learn and digest the concepts and enact changes to their daily instruction. Indeed, prior research with teachers indicated that consistent effort was needed over a period of years to show real gains in teachers’ support of student engagement via motivationally supportive strategies, for example (Turner et al., 2014). Further, for an experimental design, a larger sample of instructors may be needed to ensure that randomization is successful and differences between groups can be attributed to the intervention. Alternately, with smaller groups, quasi-experimental methods such as propensity score matching could be used instead of random assignment to ensure that the treatment and control groups are approximately comparable before the intervention. Additionally, results also may indicate a mismatch between the level of measurement and the level at which interventions were administered, particularly for the utility value and teacher-focused interventions. Motivation and identity outcomes were measured at the level of the domain of engineering, whereas the utility value and TA-focused interventions may have instead more directly targeted students’ beliefs about the course or about particular activities within the course rather than their overall beliefs about themselves and engineering. For example, results may have differed if I had measured task- or course-specific self-efficacy beliefs, which are assumed to be more malleable in the short term compared to higher-order academic perceived competence (Bong & Skaalvik, 2003). While it is not unreasonable to expect that beliefs such as utility value for engineering might shift quickly in response to a targeted intervention, it is likely much more difficult to shift utility value for engineering than utility value for an engineering course. However, the overall implementation and evaluation procedures were promising for future research in this vein. Specifically, the interventions and control conditions were successfully executed 67 with minimal burden to instructors and with favorable reactions from instructors at the end of the study. TAs indicated on the post-survey that they were open to learn more about motivationally supportive practices. Thus, future iterations of complex motivational interventions appear to be feasible and desirable to instructors. Although null findings cannot be interpreted as “disproving” hypotheses, the study’s implications for theory; for example, include a suggestion that the effectiveness of these interventions may be limited in this context due to the theorized role and targets of the interventions in particular contexts and populations as outlined above. Considering the largely null effects of these interventions, replication and examination of contextual intervention moderators is vitally important for understanding how interventions function in various contexts. Indeed, moderation analyses of similar brief, social psychological interventions have provided key insights into the theorized mechanisms of motivation interventions and their appropriateness in various contexts (Borman, Grigg, Rozek, Hanselman, & Dewey, 2018; Hanselman, Rozek, Grigg, & Borman, 2016). For example, future directions arising from this study may include an examination of differential effects by student and instructor gender in alignment with the hypothesis that the belonging and mindset interventions may only be needed for women who are at risk of belonging threat or stereotype threat in male-dominated engineering contexts. In general, findings of this study support the idea from multiple theoretical perspectives that motivation is complex, multifaceted, and shaped by a variety of internal and external processes, such that a set of brief interventions may pale in comparison to the forces already at work in shaping key student outcomes in STEM. Limitations The study included a few elements that may limit the conclusions that can be drawn from the results, particularly considering the findings that the interventions did not appear to impact any of the outcomes. First and perhaps most importantly, the study may have been underpowered due to study attrition and small effect sizes, particularly considering that there may have been only small differences in outcomes when comparing one intervention to another and when comparing one intervention alone vs. 68 two or more combined interventions. While a priori power analyses indicated that a sample size of 639 was needed to detect an effect size of .20, the analytic sample was just above this cutoff. Prior studies of these motivation interventions have had wide-ranging effect sizes (-.04-1.94; Lazowski & Hulleman, 2016; Harackiewicz et al., 2016; Hulleman et al., 2010; Reeve et al., 2004; Rubie-Davies et al., 2015; Walton & Cohen, 2007), so much greater power may in fact be needed to detect potentially small effects. This may be particularly important in a context that is already at least moderately motivationally supportive, as interventions may add only incremental gains above the existing motivational resources in the classroom. Additionally, prior research indicates that the effect of motivation interventions can be moderated by students’ demographic characteristics as well as by prior achievement (Harackiewicz et al., 2016; Walton & Cohen, 2011). This is true particularly for belonging interventions, which appear to have no main overall effects but are beneficial for students from underrepresented ethnic/minority groups. However, this study only examined prior achievement as a moderator of each intervention. This is an important consideration when interpreting the findings—the lack of main effects for each intervention may have obscured subgroup effects, so follow-up studies are needed to explore this possibility. This is a particularly important consideration for interpreting findings of the belonging intervention and when considering the generalizability of the findings beyond engineering contexts and largely white and Asian student populations. The study may also be limited by the potential for contamination effects at the teacher and student levels. Though TAs were informed of the study design and asked not to communicate about the contents of the workshops, TA reports indicated that this request may not have been honored and TAs may simply have shared motivationally supportive strategies by accident. Contamination may have also taken place at the student level, with students sharing essay prompts or the contents of their essays, or simply because students who are prompted to write about the relevance of the content to their lives may also talk about these ideas with their classmates in the control group. However, this randomization at the student level 69 mirrors prior studies showing significant effects of the intervention, despite the potential for contamination (e.g., Harackiewicz et al., 2014, 2016). Another potential limitation is that initial levels of motivation constructs prior to the mindsets and belonging interventions were not obtainable. This meant that I could not test the effectiveness of random assignment to conditions or control for initial levels of motivation if randomization resulted in different levels of initial motivation by chance. However, the random assignment to conditions with a large N, examination of pre-intervention achievement indicators, and examination of motivation indicators for other interventions mitigates this concern. Additionally, not all students enrolled in the course were first-year students, which means that some students participated in the belonging or mindsets intervention a year or more before the course began. This raised concerns about whether it was reasonable to expect that the effects of these interventions would be observable so long after students’ participation. Prior interventions using similar procedures have demonstrated, in fact, that gains in achievement from social psychological interventions have persisted or actually increased across four years of college (e.g., Walton & Cohen, 2011). However, these prior studies did not assess effects on motivation variables, so in preliminary analyses I assessed whether the effect of the mindsets and belonging interventions differed for first-year students vs. upperclassmen. Based on the results of that analysis, it was not necessary to use year in school as a control variable in the analyses. However, the overall test revealed no significant differences in mindsets at T1 across treatment and control groups, even for only first-year students, indicating that even just a few days or weeks after the intervention, no effects were observable on the construct targeted by the intervention. However, as explained above, the intervention may in fact have mitigated entity beliefs to some degree. While the assumption of homogeneity of variance was met for the majority of outcome measures used in the MANOVA analyses, T2 utility value and spring grades showed evidence of heterogeneity of variance. It is remarkable, in fact, that given the unequal cell sizes in the study (see Table 1), the assumption largely appeared to be met. In addition, the assumption of normality was largely met, with the 70 exception of course grades. The results should be carefully interpreted given that these two assumptions were at least partially violated. However, small group sizes and non-normality tend to bias the F statistic toward type I error, with p values underestimated leading to an increased possibility of falsely rejecting the null hypothesis. In addition, analyses of variance are quite robust to non-normality in particular. Thus, it is very unlikely that the violations of the assumptions of normality and homogeneity of variance resulted in overly biased findings. Lastly, the examination of mechanisms by which interventions function somewhat limited by taking a MANOVA approach rather than an examination of longitudinal growth or mediation. The analytic plan using MANOVA was designed to provide a high-level account of which variables seemed to play the most prominent roles in explaining the outcomes of motivation interventions, controlling for others. However, particularly considering the null findings, follow-up studies will be needed to elaborate these mechanisms, in particular to examine short- and long-term change processes as a result of interventions, including causal ordering of the mechanisms, and to determine optimal timing, dosage, and targeting of various constructs in future interventions. Conclusion Supporting students’ motivation to pursue and succeed in science, technology, engineering, and mathematics (STEM) fields is a top national priority (National Academies of Sciences, Engineering, and Medicine, 2016). Indeed, average declines in motivation throughout college (Kosovich et al., 2017; Robinson, Lee, et al., 2018), and the assumption that students differ in their motivational needs (Pintrich, 2003; Usher, 2009) indicate that educators need a broader array of concrete and effective tools for supporting students’ motivation in specific, everyday educational contexts. Motivation theories (Linnenbrink-Garcia et al., 2016), results of correlational studies (Andersen & Ward, 2014; Cheon et al., 2012; Cheon & Reeve, 2015; Estrada et al., 2011; Larson et al., 2015; Perez et al., 2014), and a growing body of intervention research (Harackiewicz & Priniski, 2017; Lazowski & Hulleman, 2016; Rosenzweig & Wigfield, 2016) can be used to provide guidelines for supporting motivation through curricular design. 71 However, extant research does not yet translate seamlessly to practice. In particular, motivation theory and correlational research point to the need for intervention research to account for and support the dynamic, multifaceted nature of motivation as it occurs within individuals and contexts. Despite the possibility that a multifaceted approach to supporting motivation may result in additive or exponential gains for students’ learning and persistence, thus far, motivation intervention research has largely focused on one motivational process at a time, and has provided little evidence for or against the efficacy of such complex interventions. Furthermore, although motivation is assumed to be shaped by the learning environment and teachers play a pivotal role in shaping student’s motivation, teacher-focused motivation interventions are rare and mixed in their findings. Lastly, inconsistent findings as to the mechanisms of particular interventions point to the need to thoroughly assess the specific proximal and distal impacts of motivation interventions in order to more fully understand the psychological processes in play and thus maximize the effectiveness of interventions across multiple contexts. This dissertation study aimed to address these issues through a randomized field experiment combining three student-level interventions with a teacher-level motivation intervention within an introductory, gateway college engineering course. Overall, results did not support the efficacy of any of the individual interventions or of their combinations within this setting for supporting students’ motivation to pursue engineering, engineering identity, achievement, or retention in an engineering major. These results suggest that a “light touch” approach to both student-focused and teacher-focused motivational interventions is not straightforwardly scalable and effective, and that particularly for teacher- focused interventions, greater time and involvement of the instructors may be needed to successfully shift teaching behaviors with resulting gains to student outcomes. In general, it is clear that the context, students’ needs, and existing motivational climate of the setting should be carefully considered in designing and implementing motivational interventions. 72 APPENDICES 73 APPENDIX A: Tables 74 Table 1: Study Timeline Time Summer Week 0 Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 Week 13 Week 14 Semester 2 Instructors Students Mindsets/Belonging Interventions T1 Survey & Workshop 1 T1 Survey Semester Begins 1st Writing Assignment 2nd Writing Assignment 3rd Writing Assignment T2 Survey T3 Survey Workshop 2 Observations Observations T2 Survey 75 Table 2: Field Experiment Conditions Incremental No Training Teacher Training No Training Utility Value Intervention Utility Value Control Incremental + Utility n = 47 Incremental + Utility + Teacher Training n = 63 Incremental + Teacher Training n = 63 Incremental Only n = 43 Belonging Belonging + Utility n = 53 Belonging + Utility + Teacher Training Belonging + Teacher Training n = 68 n = 75 Belonging Only n = 57 Control Utility Value Only n = 39 Utility Value + Teacher Training n = 64 Teacher Training Only n = 60 Control n = 50 76 Table 3: Sample Prompts for Utility Value Treatment and Control Group Writing Assignments Relevance (Treatment) Condition Prompt Control Condition Prompt Select a concept or issue that was covered in lecture and formulate a question. Select the relevant information from class notes and the textbook, and write a 1–2 page essay. Select a concept or issue that was covered in lecture and formulate a question. Select the relevant information from class notes and the textbook, and write a 1–2 page essay. Write an essay addressing this question and discuss the relevance of the concept or issue to your own life. Be sure to include some concrete information that was covered in this unit, explaining why this specific information is relevant to your life or useful for you. Be sure to explain how the information applies to you personally and give examples. Select the relevant information from class notes and the textbook, and write a one to two page response to your question. You should attempt to organize the material in a meaningful way, rather than simply listing the main facts or research findings. Remember to summarize the material in your own words. 77 Table 4: Hypothetical TA Scenarios for Workshop Self-Assessments Autonomy Supportive TA As you plan and prepare for an upcoming lesson, you think about what your students want and need. You wonder if students will find the lesson interesting and relevant to their lives. To support their interest and valuing of the lesson, you prepare some resources in advance so that they can see how interesting and how important the lesson truly is. To better engage students in the lesson, you create a challenging activity for students to do, and you create some engaging questions to pique their interest. As the class period begins, you invite your students’ input and suggestions before finalizing the day’s lesson plan, letting your students know that you welcome and value their thoughts, ideas, and suggestions. To motivate students, you take the time to explain why the lesson is important, how it aligns with their personal goals, and why it is a truly worthwhile thing to do. When students encounter difficulties and setbacks, you display patience— giving them the time and space they need to figure out the problem for themselves. When students complain and show little or no initiative, you acknowledge and accept their negative feelings, telling them that you understand why they might feel that way, given the difficulty and complexity of the lesson. When students succeed, you give praise about their particular efforts or strategies. As you talk with your students, you resist any pressuring language such as ‘‘you should’’, ‘‘you must’’, and ‘‘you have to.’’ Instead, you communicate your understanding and encouragement. Overall, you take your students’ perspective, welcome their thoughts, feelings, and actions into the flow of the lesson, and support their developing capacity for autonomous self- regulation. Does this approach to teaching describe what you do on a daily basis to motivate and engage your students? Controlling TA As you plan and prepare for an upcoming lesson, you think about what needs to be covered. You make a step-by-step plan of what students are supposed to do and when they are supposed to do it. As the class period begins, you tell students what to do, monitor their progress closely, and when needed make it clear that there is no time to waste. To keep students on-task, you make sure they follow your directions, obey their assignments, and basically do what they are supposed to do while not doing what they are not supposed to do. When students stray off task, you correct them saying, ‘‘You should be working now.’’ To motivate students, you offer little incentives and privileges. When students encounter difficulties and setbacks, you intervene quickly to show and tell them the right way to do it. When they do what you tell them to do and when they produce right answers, you smile and praise their abilities. When they don’t do what you tell them to do and when they misbehave, you make it clear that you are in charge and that it is your responsibility to make sure that they act responsibly and complete their work. You also motivate students by comparing them to one another, highlighting individual students as examples of high or low performance. When students complain and show little or no initiative, you ignore or criticize their negative feelings, tell them they should feel differently. Overall, you take a ‘‘no-nonsense’’ attitude and make sure students do what you tell them to do, even if it means you need to push and pressure them into doing what they are supposed and required to do. Does this approach to teaching describe what you do on a daily basis to motivate and engage your students? Note: TA scenarios are adapted from Reeve et al., 2014. 78 Table 5: Construct List and Measurement Occasions Constructs Measurement Occasions Aug. 2017 Nov. 2017 Apr. 2018 Student Characteristics Engineering Academic Perceived Competence Design Self-Efficacy Interest Value Attainment Value Utility Value Engineering Identity Achievement Goals Theories of Intelligence Belonging in Engineering Classroom Characteristics (Student Perceptions) Perceived Autonomy Support Perceived Teacher Control Perceived Competence Support Mastery Goal Structure Performance Goal Structure Connection to Real Life Instructor is Personable Belonging with Instructor and Peers TA Practices and Beliefs Self-Report (TAs) Observed x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Note: Variables used in the fidelity checks, manipulation checks, and main analyses are presented in bold. 79 Table 6: Research Questions and Planned Analyses Research Question Preliminary Analyses Effects of teacher training on teacher practices UV fidelity check Mindsets manipulation check Belonging manipulation check Research Question #1 RQ 1a: Effects of brief social psych interventions on student outcomes RQ 1b: Effects of instructor training on student outcomes RQ 1c: Prior achievement as a moderator of intervention effects Research Question #2 RQ 2a: Combined effects of UV and mindsets interventions RQ 2b: Combined effects of UV, mindsets, and instructor training interventions Data Analyses Descriptive data from TA surveys, MANOVAs comparing motivationally supportive practices from student perception surveys T-test comparing number of personal connections in subsample of essays across treatment groups T-test comparing T1 theories of intelligence for treatment and control T-test comparing T1 levels of belonging for treatment and control MANOVAs for continuous outcomes, logistic regression for retention outcome MANOVA for continuous outcomes, logistic regression for retention outcome MANCOVAs with prior achievement as a covariate and interacting with interventions MANCOVA including main effects of all interventions and interactions with prior achievement MANCOVA with main effects, 2-way interactions, and 3-way interactions, including prior achievement 80 Table 7: Descriptive Statistics for Study Variables Variable N Min. Max. Mean S.D. Skewness Kurtosis 6.00 1.00 1.00 6.00 3.00 10.00 2.50 10.00 3.50 10.00 1.00 5.00 5.00 1.00 5.00 1.00 5.00 1.00 1.00 5.00 5.00 1.00 5.00 1.00 1.00 5.00 5.00 2.00 5.00 1.58 1.74 5.00 5.00 2.25 5.00 2.00 5.00 1.00 1.00 5.00 5.00 1.00 5.00 1.00 1.30 5.00 4.00 0.00 0.00 1.00 4.42 0.97 630 T1 Incremental 2.57 1.08 630 T1 Entity 7.57 1.36 631 T1 Belonging in the College 7.24 1.55 629 T1 Belonging with Classmates 7.41 1.62 623 T1 Belonging with Professors 3.70 0.82 514 T2 Perceived Autonomy Support 2.31 0.87 515 T2 Perceived TA Control 3.73 0.78 513 T2 Perceived Competence Support 3.79 0.78 511 T2 Perceived TA Mastery Goals 2.02 0.97 514 T2 Perceived TA PAP Goals 2.26 0.94 514 T2 Perceived TA PAV Goals 3.47 0.96 515 T2 Connection to Real Life 3.79 0.79 514 T2 Instructor is Personable 4.07 0.57 682 T2 Perceived Competence 4.06 0.66 682 T2 Interest Value 3.92 0.66 682 T2 Attainment Value 4.29 0.55 682 T2 Utility Value 4.17 0.57 682 T2 Mastery-Approach Goals 3.14 0.88 682 T2 Performance-Approach Goals 3.04 0.87 682 T2 Performance-Avoidance Goals 3.59 0.84 682 T3 Engineering Identity Belonging 3.60 0.80 T3 Engineering Identity Centrality 682 3.87 0.72 T3 Engineering Identity Future Self 682 3.18 0.84 682 Spring GPA 674 Spring Engineering Major 0.90 0.30 682 17.00 36.00 29.15 3.22 Math ACT 3.73 0.53 678 Course Grade α 0.89 0.92 0.87 0.87 0.92 0.91 0.84 0.85 0.93 0.89 0.89 0.87 0.94 0.88 0.94 0.85 0.87 0.90 0.92 0.86 0.81 0.84 0.89 Note: Descriptive statistics for the variables used in main analyses (T2 Perceived Competence – Math ACT) were computed using pooled estimates of the imputed data for the analytic sample only. Reliability coefficients were computed for the full sample before excluding participants for non-participation in intervention procedures and prior to imputation. 0.06 0.15 -0.31 -0.60 -0.94 1.38 0.92 1.63 1.81 1.16 0.54 -0.02 1.40 0.42 1.76 0.70 1.09 1.13 -0.24 -0.22 0.38 0.55 1.37 2.46 - -0.02 13.66 -0.29 0.63 -0.25 -0.03 -0.04 -0.89 1.02 -0.98 -0.96 1.21 0.85 -0.58 -0.96 -0.42 -0.91 -0.64 -0.80 -0.67 -0.08 -0.05 -0.56 -0.59 -0.89 -1.55 - -0.10 -3.13 0.00 4.00 81 Table 8: Correlations for Study Variables Variable 1 2 3 4 5 6 7 8 9 10 11 12 13 1. T2 PC 2. T2 Int 3. T2 Att 4. T2 Util 5. T2 MAP 6. T2 PAP 7. T2 PAV 8. T3 EIDB 9. T3 EIDC 10. T3 EIDFS 11. GPA 12. Major 13. Math ACT 14. Course Grade .59*** .56*** .58*** .64*** .25*** .16*** .34*** .36*** .40*** .13* .14** .10 .12 .72*** .74*** .75*** .30*** .19*** .59*** .61*** .64*** .01 .25*** .06 .07 .69*** .69*** .52*** .43*** .48*** .53*** .48*** .002 .19*** .02 .003 .72*** .27*** .18*** .44*** .44*** .55*** .11 .21*** .04 .18* .28*** .19*** .45*** .47*** .50*** .10 .15** .01 .12* .83*** .22*** .25*** .12** -.09 .02 -.03 -.10 .14** .16*** .08* -.07 .02 -.04 -.12* .80*** .74*** .10 .22*** .03 .11 .73*** .11* .23*** .06 .16* .11* .28*** .07 .14* .04 .20** .55*** .14** .11** .05 Note. PC = Perceived Competence, Int = Interest Value, Att = Attainment Value, Util = Utility Value, MAP = Mastery-Approach Goals, PAP = Performance-Approach Goals, PAV = Performance-Avoidance Goals, EIDB = Engineering Identity – Belonging, EIDC = Engineering Identity – Centrality, EIDFS = Engineering Identity – Future Self. 82 Table 9: Means of Teaching Assistants’ Motivation for Teaching TA Utility Value for Teaching Engineering TA Attainment Value for Teaching Engineering TA Interest Value for Teaching Engineering TA Cost for Teaching Engineering 4.13 (.72) 4.25 (.48) 4.13 (.59) 2.25 (.66) Previous teaching experience (in semesters) 1.50 (1.50) TA Motivation Self-Efficacy TA Competence Support Self-Efficacy TA Autonomy Support Self- Efficacy TA Value/Interest Support Self-Efficacy TA Learning/Ego Goals TA Belonging/Relatedness Support Self-Efficacy T1 Means (SE) Control Treatment T2 Mean (SE) Control Treatment ΔT2-T1 Control Treatment 4.70 (.200) 4.30 (.200) 4.50 (.224) 1.60 (.19) 1.40 (.60) 3.30 (.29) 3.93 (.25) 4.00 (.16) 3.93 (.36) 3.67 (.28) 4.13 (.13) 4.33 (.333) 4.83 (.17) 4.67 (.33) 2.33 (.33) 3.50 (.14) 3.89 (.29) 3.83 (.17) 4.11 (.44) 3.89 (.11) 3.67 (.33) 5.00 (.00) 4.80 (.12) 4.80 (.20) 1.63 (.32) 3.30 (.40) 4.27 (.25) 4.20 (.26) 4.07 (.32) 4.27 (.29) 4.07 (.22) 0.21 0.30 0.58 0.50 0.54 0.30 0.08 0.02 0.00 0.34 0.20 0.14 0.60 -0.06 83 Table 10: Levene’s Test for MANOVA Examining Main and Interactive Effects of Interventions Variable F (11, 670) p T2 Perceived Competence T2 Interest Value T2 Attainment Value T2 Utility Value T2 MAP Goals T2 PAP Goals T2 PAV Goals T3 Identity - Belonging T3 Identity - Centrality T3 Identity - Future Self Course Grade Spring GPA 1.391 1.775 1.687 2.032 1.355 0.943 0.972 0.498 0.707 0.681 1.532 2.559 0.196 0.061 0.097 0.029 0.243 0.511 0.520 0.898 0.691 0.743 0.116 0.004 84 Table 11: Results of Multiple Analysis of Variance Mindsets/Belonging UV TA Mindsets/Belonging x UV Mindsets/Belonging x TA UV x TA Mindsets/Belonging x UV x TA Wilks' λ F Hypothesis df Error df p 0.98 0.99 0.99 0.95 0.97 0.98 0.97 0.62 0.73 0.61 1.40 0.99 0.92 0.73 24.00 12.00 12.00 24.00 24.00 12.00 24.00 1318.00 659.00 659.00 1318.00 1318.00 659.00 1318.00 0.903 0.696 0.797 0.105 0.510 0.530 0.828 Partial η Squared 0.01 0.01 0.01 0.02 0.02 0.02 0.01 85 Table 12: Univariate Tests for the Interaction of Utility Value and Belonging/Incremental Conditions T2 Perceived Competence T2 Interest Value T2 Attainment Value T2 Utility Value T2 Mastery-Approach Goals T2 Performance-Approach Goals T2 Performance-Avoidance Goals T3 Engineering Identity - Belonging T3 Engineering Identity - Centrality T3 Engineering Identity - Future Self Course Grade Spring GPA F 0.359 0.421 0.218 0.155 0.409 0.713 0.829 4.919 3.149 2.539 1.354 2.834 p Partial η Squared 0.701 0.692 0.811 0.863 0.675 0.567 0.492 0.012 0.053 0.083 0.259 0.060 0.001 0.001 0.001 0.000 0.001 0.002 0.002 0.014 0.009 0.008 0.004 0.008 86 Table 13: Results of Logistic Regression Analyses Examining Effects of Interventions on Major Retention Variable UV Intervention TA Intervention Belonging Intervention Mindsets Intervention UV x TA UV x Belonging UV x Mindsets TA x Belonging TA x Mindsets TA x UV x Belonging TA x UV x Mindsets ACT UV x ACT TA x ACT Belonging x ACT Mindsets x ACT UV x TA x ACT UV x Belonging x ACT UV x Mindsets x ACT TA x Belonging x ACT TA x Mindsets x ACT TA x UV x Belonging x ACT TA x UV x Mindsets x ACT Model 1 B S.E. Sig. O.R. -0.36 0.26 0.168 0.70 -0.20 0.26 0.446 0.82 -0.20 0.31 0.517 0.82 -0.14 0.32 0.655 0.87 87 Model 2 B S.E. Sig. O.R. -1.23 0.82 0.134 0.29 -0.52 0.58 0.368 0.59 -0.93 0.65 0.152 0.40 0.05 0.57 0.931 1.05 1.31 1.01 0.194 3.72 1.76 1.06 0.099 5.80 0.49 1.06 0.643 1.63 0.68 0.89 0.445 1.97 0.24 0.81 0.767 1.27 -1.58 1.35 0.243 0.21 -2.79 1.62 0.085 0.06 Model 3 B S.E. Sig. O.R. 4.37 8.17 4.19 11.66 11.05 0.301 116151.88 79.12 3547.89 65.84 0.00 0.02 0.00 0.00 2.56 12.20 2.75 5.88 0.458 8.16 0.321 6.04 0.489 -14.32 12.53 0.259 -4.06 15.00 0.789 -19.33 11.42 0.091 9.76 0.538 -6.02 0.94 8.04 0.907 2.50 17.87 0.889 1.01 38.95 0.980 -0.27 -0.48 -0.17 -0.32 -0.14 0.58 0.21 0.72 0.23 -0.03 -0.14 -0.21 1.20 0.823 0.43 0.276 0.20 0.403 0.29 0.270 0.21 0.491 0.48 0.235 0.57 0.720 0.43 0.097 0.35 0.500 0.28 0.903 0.67 0.832 1.15 0.855 0.76 0.62 0.84 0.73 0.87 1.78 1.23 2.04 1.26 0.97 0.87 0.81 Table 14: Results of MANCOVA Analyses Examining Interactions of Interventions with Prior Achievement ACT Belonging/Mindsets UV TA Belonging/Mindsets x UV Belonging/Mindsets x TA UV x TA Belonging/Mindsets x UV x TA Belonging/Mindsets x ACT UV x ACT TA x ACT Belonging/Mindsets x UV x ACT Belonging/Mindsets x TA x ACT UV x TA x ACT Belonging/Mindsets x UV x TA x ACT Wilks' λ 0.926 0.960 0.975 0.981 0.973 0.958 0.961 0.961 0.961 0.974 0.981 0.976 0.958 0.961 0.961 F 4.332 1.125 1.378 1.070 0.740 1.170 2.202 1.084 1.075 1.455 1.070 0.671 1.184 2.215 1.080 Hypothesis df 12.00 24.00 12.00 12.00 24.00 24.00 12.00 24.00 24.00 12.00 12.00 24.00 24.00 12.00 24.00 Error df p 647.00 0.000 1294.00 0.354 647.00 0.236 647.00 0.463 1294.00 0.733 1294.00 0.284 647.00 0.043 1294.00 0.410 1294.00 0.394 0.189 647.00 647.00 0.465 1294.00 0.797 1294.00 0.272 647.00 0.039 1294.00 0.408 Partial η Squared 0.074 0.020 0.025 0.019 0.014 0.021 0.039 0.020 0.020 0.026 0.019 0.012 0.021 0.039 0.020 88 Table 15: Univariate Tests for the Interaction of Utility Value and TA Conditions with Prior Achievement Variable T2 Perceived Competence T2 Interest Value T2 Attainment Value T2 Utility Value T2 Mastery-Approach Goals T2 Performance-Approach Goals T2 Performance-Avoidance Goals T3 Engineering Identity Belonging T3 Engineering Identity Centrality T3 Engineering Identity Future Self Course Grade Spring GPA UV x TA F (658, 1) p Partial η Squared 0.349 0.596 0.968 0.374 1.282 0.339 2.162 0.178 0.870 0.381 0.716 0.598 0.121 0.794 1.860 0.222 0.537 0.476 0.159 0.719 1.236 0.351 0.473 0.639 0.001 0.001 0.002 0.003 0.001 0.001 0.000 0.003 0.001 0.000 0.002 0.001 UV x TA x ACT F (658, 1) p Partial η Squared 0.367 0.589 0.954 0.380 1.209 0.362 2.148 0.187 0.760 0.410 0.629 0.598 0.124 0.794 1.891 0.219 0.581 0.456 0.148 0.734 1.656 0.323 0.463 0.630 0.001 0.001 0.002 0.003 0.001 0.001 0.000 0.003 0.001 0.000 0.003 0.001 89 APPENDIX B: Figure 90 5.00 4.50 4.00 3.50 3.00 2.50 2.00 1.50 1.00 5.00 4.50 4.00 3.50 3.00 2.50 2.00 1.50 1.00 Control Mindset Belonging UV NO UV Control Mindset Belonging UV NO UV Figure 1: Estimated marginal means for belonging component of engineering identity as a function of the interaction between mindset/belonging and utility value interventions. Error bars represent confidence intervals. 91 APPENDIX C: Study Measures 92 TA PRE- AND POST SURVEYS Personal Teaching Efficacy (adapted from Midgley et al., 2000) How certain are you that you can: 1. Get through to even the most difficult student. 2. Help all students make significant improvement. Teacher Self-Efficacy, Motivation Subscale (Skaalvik & Skaalvik, 2007) How certain are you that you can: 1. Get all students in class to work hard with their schoolwork. 2. Wake the desire to learn even among the lowest achieving students. 3. Get students to do their best even when working with difficult problems. 4. Motivate students who show low interest in schoolwork. Self-Efficacy for Supporting Motivation, based on Design Principles (Linnenbrink-Garcia et al., 2016) How certain are you that you can: Competence 1. Provide students with an appropriate level of challenge. 2. Structure and pace instruction so students feel challenged but not overwhelmed. 3. Give informational and encouraging feedback. Autonomy 1. Provide students with opportunities to make choices. 2. Listen and respond to students' ideas, reactions, and feelings. Value/Interest 1. Make activities personally relevant to students. 2. Help students apply their learning to real-world situations. 3. Keep students actively involved in their learning. Learning and Understanding 1. De-emphasize performance, competition, and social comparison. 2. Give students clear criteria for success. 3. Recognize growth in individual students. Belonging 1. Build strong relationships with students. 2. Help students feel connected to each other. 3. Show enthusiasm for engineering. Please indicate your agreement with the following: Value and Cost for Teaching (Adapted from Conley et al., 2012) 1. Teaching engineering will be useful for me later in life. 2. Teaching is valuable because it will help me in the future. 3. Being someone who is good at teaching is important to me. 4. It is important for me to be someone who is good at teaching engineering. 5. I enjoy teaching engineering. 6. I like teaching engineering. 7. I’m concerned that I have to give up a lot to teach engineering. 8. For me, teaching engineering may not be worth the effort. 93 STUDENT MOTIVATION SURVEY Engineering Academic Perceived Competence (Mamaril et al., 2016) 1. I’m certain I can master the content in the engineering-related courses I am taking this semester. 2. I will be able to master the content in even the most challenging engineering course if I try. 3. I will be able to do a good job on almost all my engineering coursework if I do not give up. 4. I’m confident that I can learn the content taught in my engineering-related courses. 5. I’m certain I can earn a good grade in my engineering-related courses. Design Self-Efficacy (Mamaril et al., 2016) 1. I can design new things in engineering. 2. I can identify an engineering design need. 3. I can develop engineering design solutions. 4. I can evaluate an engineering design. 5. I can recognize changes needed for an engineering design solution to work. Task Value (adapted from Conley, 2012) Interest Value 1. I enjoy the subject of engineering. 2. I enjoy doing engineering. 3. Engineering is exciting to me. 4. I am fascinated by engineering. 5. I like engineering. Attainment Value 1. It is important for me to be a person who reasons like an engineer. 2. It is important for me to be someone who is good at solving problems that involve engineering. 3. Being someone who is good at engineering is important to me. 4. Being good in engineering is an important part of who I am. Utility Value 1. Engineering is valuable because it will help me in the future. 2. Engineering will be useful for me later in life. 3. Engineering is practical for me to know. 4. Engineering helps me in my daily life outside of school. 5. Being good in engineering will be important for my future (like when I get a job or go to graduate school). Engineering Identity (*Pugh et al., 2009; +Estrada et al., 2011) 1. I can imagine myself being involved in an engineering related career.* 2. Being involved in engineering is a key part of who I am.* 3. I consider myself an engineering person.* 4. I can see myself doing engineering in the future.* 5. I have a strong sense of belonging to the community of engineers.+ 6. I derive great personal satisfaction from being part of a team of engineers.+ 7. I have come to think of myself as an ‘engineer.’+ 8. I feel like I belong in the field of engineering.+ 9. The daily work of an engineer is appealing to me.+ Achievement Goals (PALS; Midgley et al., 2000) Mastery Approach 1. It’s important to me that I learn a lot of new concepts in engineering. 2. One of my goals in engineering is to learn as much as I can. 94 3. One of my goals in engineering is to master a lot of new skills. 4. It’s important to me that I thoroughly understand my coursework in engineering. 5. It’s important to me that I improve my skills in engineering this semester. Performance Approach 1. It’s important to me that other students think I am good at engineering. 2. One of my goals is to show others that I’m good at engineering. 3. One of my goals is to show others that engineering is easy for me. 4. One of my goals is to look smart in comparison to the other students in engineering. 5. It’s important to me that I look smart compared to others in engineering. Performance Avoidance 1. It’s important to me that I don’t look stupid in engineering. 2. One of my goals is to keep others from thinking I’m not smart in engineering. 3. It’s important to me that my professors don’t think that I know less than others about engineering. 4. One of my goals in engineering is to avoid looking like I have trouble doing the work. Theories of Intelligence (Dweck, 1999; *indicates incremental beliefs) 1. You have a certain amount of intelligence, and you can’t really do much to change it. 2. Your intelligence is something about you that you can’t change very much. 3. No matter who you are, you can significantly change your intelligence level.* 4. To be honest, you can’t really change how intelligent you are. 5. You can always substantially change how intelligent you are.* 6. You can learn new things, but you can’t really change your basic intelligence. 7. No matter how much intelligence you have, you can always change it quite a bit.* 8. You can change even your basic intelligence level considerably.* Belonging (Mendoza-Denton et al., 2002) 3 2 2 3 OK 6 Feelings of Belonging in the College of Engineering 1. Circle the number that best describes your current feelings about MSU’s College of Engineering. a) miserable 1 b) do NOT fit in 1 c) NOT welcome 1 d) very uncomfortable 1 Feelings of Belonging – Engineering Classmates/Peers 2. Circle the number that best describes your current feelings toward your classmates/peers in your 9 definitely fit in 9 very welcome 9 feel very comfortable 9 5 sort of fit in 5 sort of welcome 5 so-so 5 thrilled to be here 10 8 8 8 8 4 4 4 4 7 7 6 6 7 7 10 10 10 2 2 3 3 6 engineering courses. 4 5 3 3 7 7 6 6 2 2 4 9 8 like them 10 sort of like them 5 a) do NOT like them 1 b) do NOT feel comfortable with them sort of feel comfortable with them feel comfortable with them 1 Feelings of Belonging – Engineering Professors 3. Circle the number that best describes your current feelings toward your engineering professors. a) do NOT like them 1 b) do NOT feel comfortable with them sort of feel comfortable with them feel comfortable with them 1 sort of like them 5 like them 10 8 9 4 10 10 8 8 2 2 9 9 3 6 6 7 7 3 4 5 95 END-OF-SEMESTER SURVEY (ADDITIONAL CONSTRUCTS) Motivational Climate (Jang et al., 2016; adapted by replacing "teacher" with "EGR 100 TA") My EGR 100 lab TA… Perceived Autonomy Support 1. …provides me with choices and options. 2. …makes me feel understood. 3. … conveys confidence in my ability to do well in this course. 4. … encourages me to ask questions. 5. … listens to how I would like to do things. 6. …tries to understand how I see things before suggesting a new way to do things. Perceived Teacher Control 1. …tries to control everything I do. 2. …is inflexible. 3. … uses forceful language. 4. … puts a lot of pressure on me. Perceived Competence Support 1. … provides feedback that helps me improve my skills and knowledge. 2. … helps me develop skills for success. 3. … praises my efforts and strategies. 4. …assigns tasks that are not too hard, and not too easy. TA Mastery Goals Perceived TA Achievement Goals (PALS, Midgley et al., 2000) 1. My EGR 100 TA thinks it’s okay to make mistakes as long as you are learning. 2. My EGR 100 TA thinks it’s important to understand the work, not just memorize it. 3. My EGR 100 lab TA recognizes us for trying hard. 4. My EGR 100 lab TA wants us to understand the material, not just memorize it. 5. My EGR 100 lab TA thinks learning new ideas and concepts is very important. 6. My EGR 100 lab TA thinks how much you improve is really important. 7. My EGR 100 lab TA gives us the time to really explore and understand new ideas. TA Performance-Approach Goals 1. My EGR 100 TA points out those students who get good grades as an example to all of us. 2. My EGR 100 TA lets us know which students get the highest scores on a test or assignment. 3. My EGR 100 TA tells us how we compare to other students. TA Performance-Avoidance Goals 1. My EGR 100 TA tells us that it is important that we don’t look stupid in class. 2. My EGR 100 TA says that showing other that we are not bad at class work should be our goal. 3. My EGR 100 TA tells us it’s important to join in discussions and answer questions so it doesn’t look like we can’t do the work. 4. My EGR 100 TA tells us it’s important to answer questions in class, so it doesn’t look like we can’t do the work. Connection to Real Life (Godin et al., 2015) 1. My EGR 100 lab TA relates course material to real life. 2. My EGR 100 lab TA tells personal anecdotes related to the material taught in class. 3. My EGR 100 lab TA uses examples that relate the course material to real life. 4. My EGR 100 lab TA tells stories that relate the course material to real life. 1. My EGR 100 lab TA is enthusiastic. Instructor is Personable (Godin et al., 2015) 96 2. My EGR 100 lab TA is humorous. 3. My EGR 100 lab TA has a personal connection to students. 4. My EGR 100 lab TA is friendly. 5. My EGR 100 lab TA shows interest in students’ education. 6. My EGR 100 lab TA gives positive feedback. 7. My EGR 100 lab TA is approachable. 8. My EGR 100 lab TA is cool. 9. My EGR 100 lab TA is interesting. 10. My EGR 100 lab TA is engaging. 11. I admire my EGR 100 lab TA. Belonging (Mendoza-Denton et al., 2002) 2 2 3 3 4 4 sort of like them 5 Feelings of Belonging – EGR 100 Classmates/Peers 1. Circle the number that best describes your current feelings toward your classmates/peers in your EGR 100 lab. a) do NOT like them 1 b) do NOT feel comfortable with them sort of feel comfortable with them feel comfortable with them 1 Feelings of Belonging – EGR 100 TA 2. Circle the number that best describes your current feelings toward your EGR 100 lab TA. a) do NOT like them 1 b) do NOT feel comfortable with them sort of feel comfortable with them feel comfortable with them 1 like them 10 like them 10 sort of like them 5 6 7 6 6 7 7 9 9 8 8 4 4 3 3 9 9 2 2 8 8 10 5 5 6 7 10 97 APPENDIX D: Observation Instrument 98 Tally (count) the following practices each time they happen in the 10-minute interval. Competence 1. Instructor demonstrates a skill or concept by working through an example 2. Instructor demonstrates a skill or concept by having a student work through an example 3. Instructor gives a hint, clue, or tip rather than giving the answer 4. Instructor gives verbal correction and/or feedback: a. Verifies that an answer/action is right or wrong only (yes, no, correct, incorrect) b. Verifies that an answer/action is right or wrong and elaborates (explains why) c. Verifies that an answer/action is right or wrong and asks student to elaborate d. Provides feedback that is specific to the task and/or student e. Other (please describe): Autonomy 1. Instructor asks for students’ opinions about something procedural (need more time, what topic to cover next, how to complete an assignment, etc.) 2. Instructor asks for students’ opinions about the topic (do you agree, is this interesting, what do you think, etc.) 3. Instructor gives extra time to finish a task when student(s) want(s) to continue 4. Instructor refuses to give extra time to finish a task when student(s) want(s) to continue 5. Instructor provides a choice or option a. Note what type of choice or option (topic, procedure, order of tasks, etc.): 6. Instructor uses a controlling statement, directive, or phrase to seek compliance (You must, You have to, You’ve got to) Value/Interest 1. Instructor connects course material to a real-world situation (future job, personal example, etc.) 2. Instructor asks students to connect course material to interests, goals, or a real-world situation 3. Instructor highlights an externally regulated extrinsic reason for doing something (grade, incentives, consequences, deadlines, others’ opinions) 4. Instructor highlights a more internally regulated extrinsic reason for doing something (goals, personal importance, self-control, not letting yourself down) 5. Connects to a particular students’ interests or goals (mentions specific interest or goal in reference to a particular student) 6. Highlights an intrinsic reason for doing something (interest, enjoyment, curiosity, challenge) 7. Instructor asks a yes/no question 8. Instructor asks an open-ended question a. An open-ended question results in < 1 minute of student speech on the topic (including dialogue with instructor) b. An open-ended question results in 1-3 minutes of student speech on the topic (including dialogue with instructor) c. An open-ended question results in > 3 minutes of student speech on the topic (including dialogue with instructor 9. Student asks a yes/no question 10. Student asks an open-ended question Mastery/Performance Goals 1. Instructor compares and/or points out student(s) for correct answers 2. Instructor compares and/or points out student(s) as examples of high or low ability 3. Instructor praises student(s) for effort or strategy use (I like how you…, I saw that you worked really hard, etc.) 4. Instructor praises student(s) for improvement, growth, or mastery of a concept or skill 5. Instructor refers to previous success or failure 99 a. Instructor describes ways to build on prior success or failure b. Instructor simply praises students(s) for success c. Instructor emphasizes the need to avoid past failures Belonging/Relatedness 1. Instructor expresses enthusiasm for the material (exclaims, smiles while talking about material, verbally notes something is interesting or exciting) 2. Instructor calls a student by name 3. Instructor asks about students’ feelings (are you bored, how are you feeling today, is this exciting) 4. Student(s) verbally express negative affect (boredom, anger, frustration, hopelessness) a. Instructor appears to ignore students’ expressions of negative affect b. Instructor invalidates or tries to change negative affect c. Instructor listens (nods, looks at students while talking), verbally validates and/or shows understanding for negative affect 5. Students talk or work with each other (note start and end time): a. While students talk or work with each other, instructor does not interact with students b. While students talk or work with each other, instructor also interacts with students (one- on-one, circulates the room, etc.) 6. Instructor provides general encouragement (you can do this, keep trying, etc.) 100 APPENDIX E: Results After Removing Multivariate Outliers 101 Table E1: Results of Multiple Analysis of Variance Removing Multivariate Outliers Mindsets/Belonging UV TA Mindsets/Belonging x UV Mindsets/Belonging x TA UV x TA Mindsets/Belonging x UV x TA Wilks' λ 0.98 0.99 0.99 0.95 0.96 0.98 0.97 F 0.63 0.71 0.82 1.35 0.99 1.00 0.79 Hypothesis df 24.00 12.00 12.00 24.00 24.00 12.00 24.00 Error df p 1302.00 0.893 651.00 0.708 651.00 0.627 1302.00 0.136 1302.00 0.500 651.00 0.452 1302.00 0.749 Partial η Squared 0.01 0.01 0.01 0.02 0.02 0.02 0.01 102 Table E2: Results of Logistic Regression Analyses Examining Effects of Interventions on Major Retention, Multivariate Outliers Removed Variable UV Intervention TA Intervention Belonging Intervention Incremental Intervention UV x TA UV x Belonging UV x Incremental TA x Belonging TA x Incremental TA x UV x Belonging TA x UV x Incremental Math ACT UV x ACT TA x ACT Belonging x ACT Incremental x ACT UV x TA x ACT UV x Belonging x ACT UV x Incremental x ACT TA x Belonging x ACT TA x Incremental x ACT TA x UV x Belonging x ACT TA x UV x Incremental x ACT Model 1 B S.E. Sig. O.R. -0.32 0.27 0.235 0.73 -0.18 0.26 0.484 0.83 -0.16 0.31 0.616 0.85 -0.15 0.33 0.642 0.86 103 Model 2 B S.E. Sig. O.R. -1.10 0.83 0.187 0.33 -0.37 0.59 0.536 0.69 -0.77 0.66 0.242 0.46 0.18 0.58 0.754 1.20 1.18 1.02 0.248 3.25 1.61 1.07 0.135 4.98 0.36 1.07 0.738 1.43 0.52 0.90 0.562 1.68 -0.03 0.83 0.973 0.97 -1.44 1.36 0.289 0.24 -2.53 1.63 0.122 0.08 Model 3 B S.E. Sig. O.R. 5.37 8.64 4.72 12.20 11.23 0.288 198124.32 214.97 5631.36 112.30 0.00 0.01 0.00 0.00 0.60 17.45 10.19 6.17 0.384 8.43 0.311 6.27 0.453 -15.21 12.63 0.235 -4.52 15.10 0.767 -19.86 11.53 0.085 -6.48 10.00 0.517 -0.51 8.28 0.950 2.86 17.94 0.874 2.32 38.79 0.953 -0.28 -0.49 -0.20 -0.33 -0.16 0.60 0.22 0.73 0.24 0.01 -0.15 -0.25 1.20 0.820 0.43 0.269 0.21 0.349 0.30 0.270 0.22 0.467 0.48 0.218 0.57 0.708 0.43 0.094 0.35 0.492 0.29 0.983 0.67 0.824 1.15 0.829 0.76 0.61 0.82 0.72 0.85 1.82 1.24 2.07 1.28 1.01 0.86 0.78 Table E3: Results of MANCOVA Analyses Examining Interactions of Interventions with Prior Achievement, Multivariate Outliers Removed Wilks' λ F df Error df p Hypothesis ACT Belonging/Mindsets UV TA Belonging/Mindsets x UV Belonging/Mindsets x TA UV x TA Belonging/Mindsets x UV x TA Belonging/Mindsets x ACT UV x ACT TA x ACT Belonging/Mindsets x UV x ACT Belonging/Mindsets x TA x ACT UV x TA x ACT Belonging/Mindsets x UV x TA x ACT 0.930 0.958 0.974 0.978 0.973 0.961 0.958 0.961 0.960 0.972 0.978 0.975 0.960 0.958 0.961 4.055 1.165 1.439 1.201 0.740 1.075 2.342 1.079 1.111 1.525 1.179 0.670 1.092 2.310 1.075 12.00 24.00 12.00 12.00 24.00 24.00 12.00 24.00 24.00 12.00 12.00 24.00 24.00 12.00 24.00 639.00 0.000 1278.00 0.320 639.00 0.206 639.00 0.381 1278.00 0.739 1278.00 0.391 639.00 0.026 1278.00 0.410 1278.00 0.365 639.00 0.160 639.00 0.396 1278.00 0.807 1278.00 0.373 639.00 0.026 1278.00 0.408 Partial η Squared 0.070 0.021 0.026 0.022 0.014 0.020 0.042 0.020 0.020 0.028 0.022 0.012 0.020 0.042 0.020 104 REFERENCES 105 REFERENCES Ames, C. (1992). Classrooms: goals, structures, and student motivation. Journal of Educational Psychology, 84, 261-271. doi: 10.1037/0022-0663.84.3.261 Anderman, E. M. (1996). School reform and the transition to middle school. Paper presented at the annual meeting of the American Educational Research Association, New York, NY. Andersen, L., & Ward, T. J. (2014). Expectancy-value models for the STEM persistence plans of ninth- grade, high-ability students: A comparison between Black, Hispanic, and White students. Science Education, 98, 216-242. doi: 10.1002/sce.21092 Aronson, J., Fried, C., & Good, C. (2002). Reducing the effects of stereotype threat on African American college students by shaping theories of intelligence. Journal of Experimental Social Psychology, 38, 113–125. doi: 10.1006/jesp.2001.1491 Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64, 359-372. doi:10.1037/h0043445 Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. doi: 10.1037/0033-295X.84.2.191 Barron, K. E., & Harackiewicz, J. M. (2001). Achievement goals and optimal motivation: Testing multiple goal models. Journal of Personality and Social Psychology, 80, 706–722. doi: 10.1037/0022-3514.80.5.706 Baumeister, R., & Leary, M. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497–529. doi:10.1037/0033- 2909.117.3.497 Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78, 246–263. doi:10.1111/j.1467-8624.2007.00995.x Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15, 1-40. doi: 10.1023/A:1021302408382 Borman, G. D., Grigg, J., Rozek, C. S., Hanselman, P., & Dewey, N. A. (2018). Self-affirmation effects are produced by school context, student engagement with the intervention, and time: Lessons from a district-wide implementation. Psychological Science, 29, 1-12. doi: 10.1177/0956797618784016 Brophy, J. (2008). Developing students’ appreciation for what is taught in school. Educational Psychologist, 43, 132-141. doi: 10.1080/00461520701756511 Brown, E., Smith, J., Thoman, D., Allen, J., & Muragishi, G. (2015). From bench to bedside: A communal utility value intervention to enhance students’ biomedical science motivation. Journal of Educational Psychology, 107, 1116–1135. doi: 10.1037/edu0000033 106 Canning, E. A., & Harackiewicz, J. M. (2015). Teach it, don’t preach it: The differential effects of directly-communicated and self-generated utility-value information. Motivation Science, 1, 47-71. doi: 10.1037mot0000015 Canning, E. A., Harackiewicz, J. M., Priniski, S. P., Hecht, C. A., Tibbetts, Y., & Hyde, J. S. (2018). Improving performance and retention in introductory biology with a utility-value intervention. Journal of Educational Psychology, 110, 834-849. doi: 10.1037/edu0000244 Chemers, M. M., Zurbriggen, E. L., Syed, M., Goza, B. K., & Bearman, S. (2011). The role of efficacy and identity in science career commitment among underrepresented minority students. Journal of Social Issues, 67, 469-491. doi: 10.1111/j.1540-4560.2011.01710.x Chen, X., & Soldner, M. (2013). STEM Attrition: College students’ paths into and out of STEM fields. Statistical Analysis Report. National Center for Education Statistics. Cheon, S. H., & Moon, I. S. (2010). Implementing an autonomy-supportive fitness program to facilitate students’ autonomy and engagement. Korean Journal of Sport Psychology, 21, 175–195. Cheon, S. H., & Reeve, J. (2015). A classroom-based intervention to help teachers decrease students’ amotivation. Contemporary Educational Psychology, 40, 99-111. doi: 10.1016/j.cedpsych.2014.06.004 Cheon, S. H., Reeve, J., & Moon, I. S. (2012). Experimentally based, longitudinally designed, teacher- focused intervention to help physical education teachers be more autonomy supportive toward their students. Journal of Sport & Exercise Psychology, 34, 365–396. doi: 10.1123/jsep.34.3.365 Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing the racial achievement gap: A social- psychological intervention. Science, 313, 1307–1310. doi:10.1126/science.1128317 Cohen, G. L., Garcia, J., Purdie-Vaughns, V., Apfel, N., & Brzustoski, P. (2009). Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science, 324, 400–403. doi:10.1126/science.1170769 Conley, A. M. (2012). Patterns of motivation beliefs: Combining achievement goal and expectancy-value perspectives. Journal of Educational Psychology, 104, 32-47. doi: 10.1037/a0026042 Cook, J. E., Purdie-Vaughns, V., Meyer, I. H., & Busch, J. T. A. (2014). Intervening within and across levels: A multilevel approach to stigma and public health. Social Science & Medicine, 103, 101- 109. doi: 10.1016/j.socscimed.2013.09.023 Corrion, K, D’Arripe-Longueville, F., Chalabaev, A., Schiano-Lomoriello, S., Roussel, P., & Cury, F. (2010). Effect of implicit theories on judgement of cheating acceptability in physical education: The mediating role of achievement goals. Journal of Sports Sciences, 28, 909-919. doi: 10.1080/02640414.2010.484065 Cury, F., Elliott, A. J., Da Fonseca, D., & Moller, A. C. (2006). The social-cognitive model of achievement motivation and the 2 x 2 achievement goal framework. Journal of Personality and Social Psychology, 90, 666-679. doi: 10.1037/0022-3514.90.4.666 107 Dai, T., & Cromley, J. G. (2014). Changes in implicit theories of ability in biology and dropout from STEM majors: A latent growth curve approach. Contemporary Educational Psychology, 39, 233- 247. doi: 10.1016/j.cedpsych.2014.06.003 deCharms, R. (1968). Personal causation: The internal affective determinants of behavior. New York, NY: Academic Press. Deci, E. L. (1975). Intrinsic motivation. New York, NY: Plenum. Durik, A. M., Shechter, O. G., Noh, M., Rozek, C. S., & Harackiewicz, J. M. (2015). What if I can’t? Success expectancies moderate the effects of utility value information on situational interest and performance. Motivation and Emotion, 39, 104– 118. http://dx.doi.org/10.1007/s11031-014- 9419-0 Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. New York, NY: Psychology Press. Dweck, C. S. (2003). Ability conceptions, motivation and development. In BJEP Monograph Series II, Number 2-Development and Motivation (Vol. 13, No. 27, pp. 13-27). Leicester, UK: British Psychological Society. Dweck, C. (2016, January 11). Recognizing and overcoming false growth mindset. Retrieved from https://www.edutopia.org/blog/recognizing-overcoming-false-growth-mindset-carol-dweck Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256–273. doi: 10.1037/0033-295X.95.2.256 Dweck, C. S., & Molden, D. C. (2017). Mindsets: Their impact on competence motivation and acquisition. In A. J. Elliot, C. S. Dweck, & D. S. Yeager (Eds.), Handbook of Competence and Motivation (pp. 135-154). New York, NY: The Guilford Press. Eagan, K., Hurtado, S, Figueroa, T., and Hughes, B. (2014). Examining STEM Pathways among Students Who Begin College at Four-Year Institutions. Paper prepared for the Committee on Barriers and Opportunities in Completing 2-Year and 4-Year STEM Degrees, National Research Council. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/ dbasse_088834.pdf [April 2015]. Eccles, J. S. (1983). Expectancies, values, and academic behavior. In J. T. Spence (Ed.), Achievement and achievement motivation (pp. 75–146). San Francisco, CA: Freeman. Eccles, J. (2009). Who am I and what am I going to do with my life? Personal and collective identities as motivators of action. Educational Psychologist, 44, 78-79. doi: 10.1080/00461520902832368 Elliott, E. S. & Dweck, C. S. (1988). Goals: An approach to motivation and achievement. Journal of Personality and Social Psychology, 54, 5-12. doi: 10.1037/0022-3514.54.1.5 Estrada, M., Woodcock, A., Hernandez, P. R., & Schultz, P. W. (2011). Toward a model of social influence that explains minority student integration into the scientific community. Journal of Educational Psychology, 103, 206-222. doi: 10.1037/a0020743 108 Feng, S.-L., & Tuan, H.-L. (2005). Using ARCS model to promote 11th graders’ motivation and achievement in learning about acids and bases. International Journal of Science and Mathematics Education, 3, 463-484. doi: 10.1007/s10763-004-6828-7 Gaspard, H., Dicke, A.-L., Flunger, B., Brisson, B. M., Häfner, I., Nagengast, B., & Trautwein, U. (2015). Fostering adolescents’ value beliefs for mathematics with a relevance intervention in the classroom. Developmental Psychology, 51, 1226-1240. doi: 10.1037/dev0000028 Godin, E. A., Wormington, S. V., Perez, T., Barger, M. M., Snyder, K. E., Richman, L. S., . . . & Linnenbrink-Garcia, L. (2015). A pharmacology-based enrichment program for undergraduates promotes interest in science. CBE—Life Sciences Education, 14, 1-12. doi: 10.1187/cbe.15-02- 0043 Gonida, E. N., Voulala, K., & Kiosseoglou, G. (2009). Students’ achievement goal orientations and their behavioral and emotional engagement: Co-examining the role of perceived goal structures and parent goals during adolescence. Learning and Individual Differences, 19, 53-60. doi: 10.1016/j.lindif.2008.04.002 Good, C., Aronson, J., & Inzlicht, M. (2003). Improving adolescents’ standardized test performance: An intervention to reduce the effects of stereotype threat. Applied Developmental Psychology, 24, 645-662. doi: 10.1016/j.appdev.2003.09.002 Graham, S., & Williams, C. (2009). An attributional approach to motivation in school. In K. R. Wentzel & A. Wigfield (Eds.), Handbook of motivation at school (pp. 11-33). New York, NY: Routledge. Gross-Loh, C. (2016, December 16). How praise became a consolation prize. Retrieved from https://www.theatlantic.com/education/archive/2016/12/how-praise-became-a-consolation- prize/510845/ Grubbs, F. E. (1969). Procedures for detecting outlying observations in samples. Technometrics, 11, 1-21. doi:10.1080/00401706.1969.10490657 Guthrie, J. T., Van Meter, P., Hancock, G. R., McCann, A., Anderson, E., & Alao, S. (1998). Does Concept-Oriented Reading Instruction increase strategy use and conceptual learning from text? Journal of Educational Psychology, 90, 261–278. Guthrie, J. T., Wigfield, A., Barbosa, P., Perencevich, K. C., Taboada, A., Davis, M. H., . . ., & Tonks, S. (2004). Increasing reading comprehension and engagement through concept-oriented reading instruction. Journal of Educational Psychology, 96, 403-423. doi: 10.1037/0022-0663.96.3.403 Guthrie, J. T., Wigfield, A., Humenick, N. M., Perencevich, K. C., Taboada, A., & Barbosa, P. (2006). Influences of stimulating tasks on reading motivation and comprehension. The Journal of Educational Research, 99, 232-246. doi: 10.3200/JOER.99.4.232-246 Guthrie, J. T., Wigfield, A., & VonSecker, C. (2000). Effects of integrated instruction on motivation and strategy use in reading. Journal of Educational Psychology, 92, 331-341. doi: 10.1037/0022- 0663.92.2.331 Hanselman, P., Rozek, C. S., Grigg, J., & Borman, G. D. (2016). New evidence on self-affirmation effects and theorized sources of heterogeneity from large-scale replications. Journal of Educational Psychology, 109, 405-424. doi: 10.1037/edu0000141 109 Harackiewicz, J. M., Barron, K. E., Tauer, J. M., & Elliot, A. J. (2002). Predicting success in college: A longitudinal study of achievement goals and ability measures as predictors of interest and performance from freshman year through graduation. Journal of Educational Psychology, 94, 562-575. doi: 10.1037//0022-0663.94.3.562 Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde, J. S. (2014). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106, 375-89. doi: 10.1037/a0034679 Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Priniski, S. J., & Hyde, J. S. (2016). Closing achievement gaps with a utility-value intervention: Disentangling race and social class. Journal of Personality and Social Psychology, 111, 745-765. doi: 10.1037/pspp0000075 Harackiewicz, J. M., & Priniski, S. J. (2017). Improving student outcomes in higher education: The science of targeted intervention. Annual Review of Psychology, 69, 409-435. doi: 10.1146/annurev-psych-122216-011725 Harackiewicz, J. M., Rozek, C. S., Hulleman, C. S., & Hyde, J. S. (2012). Helping parents to motivate adolescents in mathematics and science: An experimental test of a utility value intervention. Psychological Science, 23, 899–906. doi:10.1177/0956797611435530 Hardré, P., & Reeve, J. (2009). Benefits of training corporate managers to adopt a more autonomy- supportive style toward employees: an intervention study. International Journal of Training and Development, 13, 165–184. doi:10.1111/j.1468-2419.2009.00325.x. Harter, S. (1978). Effectance motivation reconsidered: Toward a developmental model. Human Development, 21, 24-64. doi: 10.1159/000271574 Hecht, C., Canning, E. A., Tibbetts, Y., Priniski, S. J., & Harackiewicz, J. (2016, April). Connecting action to identity: Developing an attainment value intervention. Paper presented at the annual meeting of the American Educational Research Association. Hernandez, P. R., Schultz, P. W., Estrada, M., Woodcock, A., & Chance, R. C. (2013). Sustaining optimal motivation: A longitudinal analysis of interventions to broaden participation of underrepresented students in STEM. Journal of Educational Psychology, 105, 89-107. doi: 10.1037/a0029691 Huang, G., Taddese, N., & Walter, E. (2000). Entry and persistence of women and minorities in college science and engineering education. U.S. Department of Education, National Center for Education Statistics. Available: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2000601 Hulleman, C. S., & Barron, K. E. (2016). Motivation interventions in education: Bridging theory, research, and practice. In L. Como & E. M. Anderman (Eds.), Handbook of Educational Psychology (pp. 160-171). Florence, US: Routledge. Hulleman, C. S., & Cordray, D. S. (2009). Moving from the lab to the field: The role of fidelity and achieved relative intervention strength. Journal of Research on Educational Effectiveness, 2, 88- 110. doi: 10.1080/19345740802539325 110 Hulleman, C. S., Godes, O., Hendricks, B. L., & Harackiewicz, J. M. (2010). Enhancing interest and performance with a utility value intervention. Journal of Educational Psychology, 102, 880–895. doi:10.1037/a0019506 Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326, 1410–1412. doi:10.1126/science.1177067 Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2017). Making connections: Replicating and extending the utility value intervention in the classroom. Journal of Educational Psychology, 109, 387-404. doi: 10.1037/edu0000146 Hulleman, C. S., Schrager, S. M., Bodmann, S. M., & Harackiewicz, J. M. (2010). A meta-analytic review of achievement goal structures: Different labels for the same constructs or different constructs with similar labels? Psychological Bulletin, 136, 422-449. doi: 10.1037/a0018947 Hulleman, C. S., & Senko, C. (2010). Up around the bend: Forecasts for achievement goal theory and research in 2020. In T. Urdan & S. Karabenick (Eds.), The decade ahead: Theoretical perspectives on motivation and achievement. Advances in Motivation and Achievement (Vol. 16, pp. 71-104). Bingley, UK: Emerald. Jang, H., Kim, E. J., & Reeve, J. (2016). Why students become more engaged or more disengaged during the semester: A self-determination theory dual-process model. Learning and Instruction, 43, 27- 38. doi: 10.1016/j.learninstruc.2016.01.002 Kaplan, A. & Flum, H. (2009). Motivation and identity: The relations of action and development in educational contexts—An introduction to the special issue. Educational Psychologist, 44, 73-77. doi: 10.1080/00461520902832418 Kaplan, A. & Flum, H. (2012). Identity formation in educational settings: A critical focus for education in the 21st century. Contemporary Educational Psychology, 37, 171-175. doi: 10.1016/j.cedpsych.2012.01.005 Kaplan, A., & Maehr, M. (2007). The contributions and prospects of goal orientation theory. Educational Psychology Review, 19, 141-184. doi: 10.1007/s10648-006-9012-5 Kena, G., Hussar W., McFarland J., de Brey, C., Musu-Gillette, L., Wang, X., Zhang, J., Rathbun, A., Wilkinson Flicker, S., Diliberti M., Barmer, A., Bullock Mann, F., and Dunlop Velez, E. (2016). The Condition of Education 2016 (NCES 2016-144). Washington, DC: U.S. Department of Education, National Center for Education Statistics. Available: http://nces.ed.gov/pubs2016/2016144.pdf [August 2016]. Koskey, K. L. K., Karabenick, S. A., Woolley, M. E., Bonney, C. R., & Dever, B. V. (2010). Cognitive validity of students’ self-reports of classroom mastery goal structure: What students are thinking and why it matters. Contemporary Educational Psychology, 35, 254-263. doi: 10.1016/j.cedpsych.2010.05.004 Kosovich, J. J., Flake, J. K., & Hulleman, C. S. (2017). Short-term motivation trajectories: A parallel process model of expectancy-value. Contemporary Educational Psychology (advance online publication). doi: http://dx.doi.org/10.1016/j.cedpsych.2017.01.004 111 Larson, L. M., Pesch, K. M., Surapaneni, S., Bonitz, V. S., Wu, T., & Werbel, J. D. (2015). Predicting graduation: The role of mathematics/science self-efficacy. Journal of Career Assessment, 23, 399-409. doi: 10.1177/1069072714547322 Lau, S., & Nie, Y. (2008). Interplay between personal goals and classroom goal structures in predicting student outcomes: A multilevel analysis of person-context interactions. Journal of Educational Psychology, 100, 15-29. doi: 10.1037/0022-0663.100.1.15 Lazarides, R., Gaspard, H., & Dicke, A.-L. (2018). Dynamics of classroom motivation: Teacher enthusiasm and the development of math interest and teacher support. Learning and Instruction. Advance online publication. doi: 10.1016/j.learninstruc.2018.01.012 Lazowski, R. A., & Hulleman, C. S. (2016). Motivation interventions in education: A meta-analytic review. Review of Educational Research, 86, 602-640. doi: 10.3102/0034654315617832 Linnenbrink, E. A. (2005). The dilemma of performance-approach goals: The use of multiple goal contexts to promote students’ motivation and learning. Journal of Educational Psychology, 97, 197-213. doi: 10.1037/0022-0663.97.2.197 Linnenbrink-Garcia, L., Patall, E. A., & Messersmith, E. E. (2013). Antecedents and consequences of situational interest. British Journal of Educational Psychology, 83, 591-614. doi: 10.1111/j.2044- 8279.2012.02080.x Linnenbrink-Garcia, L., Patall, E. A., & Pekrun, R. (2016). Adaptive motivation and emotion in education: Research and principles for instructional design. Policy Insights from the Behavioral and Brain Sciences, 3, 228-236. doi: 10.1177/2372732216644450 Linnenbrink-Garcia, L., Perez, T., Barger, M., Wormington, S. V., Godin, E., Snyder, K., Robinson, K. A., . . ., & Schwartz-Bloom, R. (2018). Repairing the leaky pipeline: A motivationally supportive intervention to enhance persistence in undergraduate science pathways. Contemporary Educational Psychology, 53, 181-195. doi: 10.1016/j.cedpsych.2018.03.001 Linnenbrink-Garcia, L., Wormington, S. V., Snyder, K. E., Riggsbee, J., Perez, T., Ben-Eliyahu, A., & Hill, N. E. (2018). Multiple pathways to success: An examination of integrative motivational profiles among upper elementary and college students. Journal of Educational Psychology, 110, 1026-1048. doi: 10.1037/edu0000245 Maehr, M. L., & Meyer, H. A. (1997). Understanding motivation and schooling: Where we’ve been, where we are, and where we need to go. Educational Psychology Review, 9, 371-409. doi: 10.1023/A:1024750807365 Maehr, M. L., & Midgley, C. (1991). Enhancing student motivation: A schoolwide approach. Educational Psychologist, 26, 399-427. doi: 10.1080/00461520.1991.9653140 Maehr, M. L., & Midgley, C. (1996). Transforming school cultures. Boulder, CO: Westview Press. Maehr, M. L., & Zusho, A. (2009). Achievement goal theory: The past, present, and future. In K. R. Wentzel & A. Wigfield (Eds.), Handbook of motivation at school (pp. 77-104). New York, NY: Routledge. 112 Maltese, A. V., & Tai, R. H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among US students. Science Education, 95, 877–907. doi:10.1002/sce.20441 Mamaril, N. A., Usher, E. L., Li, C. R., Economy, D. R., & Kennedy, M. S. (2016). Measuring undergraduate students' engineering self-efficacy: A validation study. Journal of Engineering Education, 105, 366–395. doi:10.1002/jee.20121 Martin, A. J. (2008). Enhancing student motivation and engagement: The effects of a multidimensional intervention. Contemporary Educational Psychology, 33, 239-269. doi: 10.1016/j.cedpsych.2006.11.003 Mendoza-Denton, R., Downey, G., Purdie, V. J., Davis, A., & Pietrzak, J. (2002). Sensitivity to status- based rejection: Implications for African-American students’ college experience. Journal of Personality and Social Psychology, 83, 896-918. doi: 10.1037//0022-3514.83.4.896 Midgley, C., Kaplan, A., Middleton, M. (2001). Performance-approach goals: Good for what, for whom, under what circumstances, and at what cost? Journal of Educational Psychology, 93, 77-86. doi: 10.1037//0022-0663.93.1.77 Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., . . . , & Urdan, T. (2000). Manual for the Patterns of Adaptive Learning Scales (PALS). Ann Arbor, MI: University of Michigan. Murayama, K., & Elliot, A. J. (2009) The joint influence of personal achievement goals and classroom goal structures on achievement-relevant outcomes. Journal of Educational Psychology, 101, 432- 447. doi: 10.1037/a0014221 Musu-Gillette, L. E., Wigfield, A., Harring, J. R., & Eccles, J. S. (2015). Trajectories of change in students’ self-concepts of ability and values in math and college major choice. Educational Research and Evaluation, 21, 343-370. doi: 10.1080/13803611.2015.1057161 Nasir, N. S., Rowley, S. J., & Perez, W. (2016). Cultural, racial/ethnic, and linguistic diversity and identity. In L. Corno & E. M. Anderman (Eds.), Handbook of Educational Psychology (pp.186- 198). New York, NY: Taylor & Francis. National Academies of Sciences, Engineering, and Medicine. (2016). Supporting Students’ College Success: Assessment of Intrapersonal and Interpersonal Competencies. Board on Testing and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. doi: 10.17226/24697. National Research Council. (2000). In How People Learn: Brain, Mind, Experience, and School: Expanded Edition. Washington, DC: The National Academies Press. doi: 10.17226/9853 National Science Board. (2016). Science and Engineering Indicators 2016. Arlington, VA: National Science Foundation (NSB 2016-01). National Science Foundation. (2015). Women, minorities, and persons with disabilities in science and engineering. Special Report NSF 15-311. Arlington, VA. Available at http://www.nsf.gov/statistics/wmpd/ 113 Nelson, M. C., Cordray, D. S., Hulleman, C. S., Darrow, C. L., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. The Journal of Behavioral Health Services & Research, 39, 374-396. doi: 10.1007/s11414-012- 9295-x O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33-84. doi: 10.3102/0034654307313793 Okolo, C. M. (1992). The effects of computer-based attribution retraining on the attributions, persistence, and mathematics computation of students with learning disabilities. Journal of Learning Disabilities, 25, 327–334. doi:10.1177/002221949202500507 Patrick, H., Mantzicopoulos, P., & Samarapungavan, A. (2009). Motivation for learning science in kindergarten: Is there a gender gap and does integrated inquiry and literacy instruction make a difference. Journal of Research in Science Teaching, 46, 166-191. doi: 10.1002/tea.20276 Paunesku, D., Walton, G., Romero, C., Smith, E., Yeager, D., & Dweck, C. (2015). Mindset interventions are a scalable treatment for academic underachievement. Psychological Science, 26, 784-793. doi:10.1177/0956797615571017. Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18, 315-341. doi: 10.1007/s10648-006-9029-9 Perez, T., Cromley, J. G., & Kaplan, A. (2014). The role of identity development, values, and costs in college STEM retention. Journal of Educational Psychology, 106, 315-329. doi: 10.1037/a0034027 Perry, R. P., Chipperfield, J. G., Hladkyj, S., Pekrun, R., & Hamm, J. M. (2014). Attribution-based treatment interventions in some achievement settings. In S. A. Karabenick & T. Urdan (Eds.), Advances in motivation and achievement (Vol. 18, pp. 1-35). Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95, 667-686. doi: 10.1037/0022- 0663.95.4.667 Plewis, I., & Hurry, J. (1998). A multilevel perspective on the design and analysis of intervention studies. Educational Research and Evaluation, 4, 13–26. doi: 10.1076/edre.4.1.13.13014 Pugh, K., Linnenbrink-Garcia, L., Koskey, K. L. K., Stewart, V. C., & Manzey, C. (2009). Motivation, learning, and transformative experience: A study of deep engagement in science. Science Education, 94, 1-28. doi: 10.1002/sce.20344 Pugh, K., Linnenbrink-Garcia, L., Phillips, M., & Perez, A. C. (2015). Supporting the development of transformative experience and interest. In K. A. Renninger, M. Nieswandt, & S. Hidi (Eds.), Interest, the self, and K-16 mathematics and science learning (pp. 369-384). Washington, DC: AERA. 114 Reeve, J., Jang, H., Carrell, D., Jeon, S., & Barch, J. (2004). Enhancing students’ engagement by increasing teachers’ autonomy support. Motivation and Emotion, 28, 147–169. doi:10.1023/B:MOEM.0000032312.95499.6f Reeve, J., Vansteenkiste, M., Assor, A., Ahmad, I., Cheon, S. H., Jang, H., . . . & Wang, C. K. J. (2014). The beliefs that underlie autonomy-supportive and controlling teaching: A multinational investigation. Motivation and Emotion, 38, 93–110. doi: 10.1007/s11031-013-9367-0 Renninger, K. A., & Hidi, S. (2011). Revisiting the conceptualization, measurement, and generation of interest. Educational Psychologist, 46, 168-184. doi: 10.1080/00461520.2011.587723 Robinson, T. N. (2010). Stealth interventions for obesity prevention and control: Motivating behavior change. In L. Dube, A. Bechara, A. Dagher, A. Drewnowski, J. LeBel, P. James, . . . R. Yada (Eds.), Obesity prevention: The role of brain and society on individual behavior. New York, NY: Elsevier. Robinson, K. A., Lee, Y., Bovee, E. A., Perez, T., Walton, S. P., Briedis, D., & Linnenbrink-Garcia, L. (2018). Motivation in transition: Development and roles of expectancy, task values, and costs in early college engineering. Journal of Educational Psychology. Advance online publication. doi: 10.1037/edu0000331 Robinson, K. A., Perez, T., Nuttall, A. K., Roseth, C. J., & Linnenbrink-Garcia, L. (2018). From science student to scientist: Predictors and outcomes of heterogeneous science identity trajectories in college. Developmental Psychology, 54, 1977-1992. doi: 10.1037/dev0000567 Robnett, R. D., Chemers, M. M., & Zurbriggen, E. L. (2015). Longitudinal associations among undergraduates’ research experience, self-efficacy, and identity. Journal of Research in Science Teaching, 52, 847-867. doi: 10.1002/tea.21221 Roeser, R. W., Midgley, C., & Urdan, T. C. (1996). Perceptions of the school psychological environment and early adolescents’ psychological and behavioral functioning in school: The mediating role of goals and belonging. Journal of Educational Psychology, 88, 408-422. doi: 10.1037/0022- 0663.88.3.408 Roeser, R. W., Peck, S. C., & Nasir, N. S. (2006). Self and identity processes in school motivation, learning and achievement. In P. A. Alexander, & P. H. Winne (Eds.), Handbook of educational psychology (pp. 391–424). Mahwah, NJ: Lawrence Erlbaum. Rosenzweig, E. Q., Harackiewicz, J. M., Priniski, S. J., Hecht, C. A., Canning, E. A., Tibbetts, Y., & Hyde, J. S. (2018). Choose your own intervention: Using choice to enhance the effectiveness of a utility-value intervention. Motivation Science. Advance online publication. doi: 10.1037/mot0000113 Rosenzweig, E. Q., & Wigfield, A. (2016). STEM motivation interventions for adolescents: A promising start, but further to go. Educational Psychologist, 51, 146-163. doi: 10.1080/00461520.2016.1154792 Rubie-Davies, C. M., Peterson, E. R., Sibley, C. G., & Rosenthal, R. (2015). A teacher expectation intervention: Modelling the practices of high expectation teachers. Contemporary Educational Psychology, 40, 72-85. doi: 10.1016/j.cedpsych.2014.03.003 115 Ryan, R. M., & Deci, E. L., (2000a). Intrinsic and extrinsic motivators: Classic definitions and new directions. Contemporary Educational Psychology, 25, 54-67. doi: 10.1006/ceps.1999.1020 Ryan, R. M., & Deci, E. L., (2000b). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78. doi: 10.1037/0003- 066X.55.1.68 Ryan, R. M., & Grolnick, W. S. (1986). Origins and pawns in the classroom: Self-report and projective assessments of individual differences in children’s perceptions. Journal of Personality and Social Psychology, 50, 550-558. doi: 10.1037/0022-3514.50.3.550 Schafer, Joseph L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research, 8, 3- 15. doi: 10.1177/096228029900800102 Schunk, D. H., Meece, J. L., Pintrich, P.R. (2014). Motivation in education: Theory, research, and applications (4th Edition). Boston, MA: Pearson. Schwartz, D. L., Cheng, K. M., Salehi, S., & Wieman, C. (2016). The half-empty question for socio- cognitive interventions. Journal of Educational Psychology, 108, 397-404. doi: 10.1037/edu0000122 Seymour, E., & Hewitt, N. M. (1997). Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview Press. Skaalvik, E. M. (1997). Self-enhancing and self-defeating ego orientation: Relations with task and avoidance orientation, achievement, self-perceptions, and anxiety. Journal of Educational Psychology, 89, 71–81. doi: 10.1037/0022-0663.89.1.71 Skaalvik, E. M., & Skaalvik, S. (2007). Dimensions of teacher self-efficacy and relations with strain factors, perceived collective teacher efficacy, and teacher burnout. Journal of Educational Psychology, 99, 6611-625. doi: 10.1037/0022-0663.99.3.611 Stipek, D., Givvin, K. B., Salmon, J. M., & Macguyvers, V. L. (1998). Can a teacher intervention improve classroom practices and student motivation in mathematics? The Journal of Experimental Education, 66, 319-337. doi: 10.1080/00220979809601404 Su, Y.-L., & Reeve, J. (2011). A meta-analysis of the effectiveness of intervention programs designed to support autonomy. Educational Psychology Review, 23, 159-188. doi: 10.1007/s10648-010-9142- 7 Tabachnick, B. G., Fidell, L. S., & Ullman, J. B. (2007). Using multivariate statistics (Vol. 5). Boston, MA: Pearson. Trautwein, U., Marsh, H. W., Nagengast, B., Lüdtke, O., Nagy, G., & Jonkmann, K. (2012). Probing for the multiplicative term in modern expectancy-value theory: A latent interaction modeling study. Journal of Educational Psychology, 104, 763–777. doi: 10.1037/a0027470 Trygstad, P. J. (2013). 2012 National Survey of Science and Mathematics Education: Status of elementary school science. Chapel Hill, NC: Horizon Research, Inc. 116 Turner, J. C., Christensen, A., Kackar-Cam, H. Z., Trucano, M., & Fulmer, S. M. (2014). Enhancing students’ engagement: Report of a 3-year intervention with middle school teachers. American Educational Research Journal, 51, 1195-1226. doi: 10.3102/0002831214532515 Turner, J. C., Warzon, K. B., & Christensen, A. (2011). Motivating mathematics learning: Changes in teachers’ practices and beliefs during a nine-month collaboration. American Educational Research Journal, 48, 718-762. doi: 10.3102/0002831210385103 Usher, E. (2009). Sources of middle school students’ self-efficacy in mathematics: A qualitative investigation. American Educational Research Journal, 46, 275-314. doi: 10.3102/0002831208324517 Usher, E. L., & Pajares, F. (2008). Sources of self-efficacy in school: Critical review of the literature and future directions. Review of Educational Research, 78, 751-796. doi: 10.3102/0034654308321456 Vallerand, R. J., & Reid, G. (1988). On the relative effects of positive and negative verbal feedback on males’ and females’ intrinsic motivation. Canadian Journal of Behavioural Science, 20, 239-250. doi: 10.1037/h0079930 Vansteenkiste, M., Simons, J., Lens, W., Sheldon, K. M., & Deci, E. L. (2004). Motivating learning, performance, and persistence: The synergistic effects of intrinsic goal contents and autonomy- supportive contexts. Journal of Personality and Social Psychology, 87, 246–260. doi:10.1037/0022-3514.87.2.246 Walker, C. O., & Greene, B. A. (2009). The relations between student motivational beliefs and cognitive engagement in high school. The Journal of Educational Research, 102, 463-472. doi: 10.3200/JOER.102.6.463-472 Walton, G. M., & Cohen, G. L. (2007). A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology, 92, 82–96. doi:10.1037/0022-3514.92.1.82 Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331, 1447–1451. doi:10.1126/science.1198364 Walton, G. M, Logel, C., Peach, J. M., Spencer, S. J., & Zanna, M. P. (2015). Two brief interventions to mitigate a “chilly climate” transform women’s experience, relationships, and achievement in engineering. Journal of Educational Pscyhology, 107, 468-485. doi: 10.1037/a0037461 Walton, G. M., & Wilson, T. D. (2018). Wise interventions: Psychological remedies for social and personal problems. Psychological Review, 125, 617-655. doi: 10.1037/rev0000115 Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548-573. doi: 10.1007/978-1-4612-4948-1_6 White, R. W. (1959). Motivation reconsidered: the concept of competence. Psychological Review, 66, 297-333. doi: 10.1037/h0040934 Wigfield, A., & Cambria, J. (2010). Expectancy-value theory: Retrospective and prospective. In T. C. Urdan & S. A. Karabenick (Eds.), The decade ahead: Theoretical perspectives on motivation and achievement. Advances in motivation and achievement (Vol. 16A, pp. 35-70). Bingley, UK: Emerald Group Publishing Limited. 117 Wigfield, A., & Eccles, J. (1992). The development of achievement task values: A theoretical analysis. Developmental Review, 12, 265–310. doi:10.1016/0273-2297(92)90011-P Wigfield, A., Eccles, J. S., Schiefele, U., Roeser, R. W., & Davis-Kean, P. (2006). Development of achievement motivation. In: N. Eisenberg (Ed.), Handbook of child psychology (Vol. 3, pp. 933– 1002). New York: Wiley. Wilson, T. D., & Buttrick, N. R. (2016). New directions in social psychological interventions to improve academic achievement. Journal of Educational Psychology, 108, 392-396. doi: 10.1037/edu0000111 Yeager, D. S., Hanselman, P., Paunesku, D., Hulleman, C., Dweck, C., Muller, C., … Duckworth, A. (2018, March 1). MANUSCRIPT UNDER REVISION: Where and for whom can a brief, scalable mindset intervention improve adolescents’ educational trajectories? https://doi.org/10.31234/osf.io/md2qa Yeager, D. S., Henderson, M. D., Paunesku, D., Walton, G. M., D’Mello, S., Spitzer, B. J., & Duckworth, A. L. (2014). Boring but important: A self-transcendent purpose for learning fosters academic self-regulation. Journal of Personality and Social Psychology, 107, 559-580. doi: 10.1037/a0037637 Yeager, D. S., Romero, C., Paunesku, D., Hulleman, C. S., Schneider, B., Hinojosa, C., Lee, H. Y. . . . , & Dweck, C. S. (2016). Using design thinking to improve psychological interventions: The case of the growth mindset during the transition to high school. Journal of Educational Psychology, 108, 374-391. doi: 10.1037/edu0000098 Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81, 267–301. doi:10.3102/0034654311405999 Ziegler, A., & Heller, K. A. (2000). Effects of an attribution retraining with female students gifted in physics. Journal for the Education of the Gifted, 23, 217–243. doi: 10.4219/jeg-2000-585 118