CHARACTERIZATION OF STUDENTS’ REASONING ABOUT ATOMIC EMISSION SPECTRA – A DESIGN-BASED RESEARCH STUDY TO IMPROVE STUDENTS’ UNDERSTANDING OF LIGHT-MATTER INTERACTIONS By Christopher Joel Minter A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Chemistry – Doctor of Philosophy 2019 ABSTRACT CHARACTERIZATION OF STUDENTS’ REASONING ABOUT ATOMIC EMISSION SPECTRA – A DESIGN-BASED RESEARCH STUDY TO IMPROVE STUDENTS’ UNDERSTANDING OF LIGHT-MATTER INTERACTIONS By Christopher Joel Minter The research presented in this dissertation looks at how students in general chemistry reason about atomic emission spectroscopy. Situated within the context of a transformed general chemistry curriculum called Chemistry, Life, the Universe, and Everything (CLUE), the goal of this study was to (1) characterize the various ways in which students explain atomic emission spectra, and to (2) refine the treatment of spectroscopy concepts within CLUE to improve students’ understanding of light-matter interactions. Using a design-based research methodology, a series of evidence-based curriculum changes were implemented within CLUE from Fall 2013 to Fall 2016. The specific changes that were made during each phase of this study were directly informed by observations of student understanding from the prior year. To assess the effect that these changes to curriculum and assessment had on students’ reasoning, a robust coding scheme was developed to characterize the extent to which students link and integrate their knowledge of spectroscopy concepts. In F13, students were interviewed to gain insight into how they understand atomic spectroscopy. Findings showed that students had difficulty explaining the mechanistic process for how spectral lines are created. To address this issue, the curriculum materials (i.e. homework, recitation activities, summative assessment tasks, and instruction) were refined in F14 to provide a more explicit emphasis on the mechanistic process by which an atomic emission spectrum is created. These changes led to an improvement in the percentage of students who reasoned about electronic transitions; however, there was no improvement in the number of students who reasoned about energy quantization. Based on this observation, the formative and summative assessments were refined in F15 to emphasize the quantized nature of energy. However, these changes did not lead to any observable differences in students’ reasoning. To see if a more explicit question prompt would better elicit a more detailed explanation of atomic emission spectra, only the summative assessment task was changed in F16. Findings showed that the summative assessment task used in F16 elicited more sophisticated reasoning. Based on an analysis of four separate cohorts of general chemistry students who were enrolled in CLUE during different phases of curriculum refinement, it appears that general chemistry students’ explanations of atomic emission spectra ranged from simple descriptions of properties of light to highly complex responses in which students applied their understanding of the mechanistic process for how an atomic emission spectrum is created to explain how different colored spectral lines are produced or why each element has its own unique emission spectrum. The effect that the curriculum and assessment changes had on students’ reasoning is presented within this study. This dissertation is dedicated to my wife Hannah, and to my parents, Joel and Cheryl. Thank you for always believing in me. I am forever grateful for your endless love and support. iv ACKNOWLEDGEMENTS First and foremost, I would like to thank Dr. Melanie Cooper for all her support and guidance throughout graduate school. She truly cares for her students and does everything in her power to help them achieve their goals and dreams, and for this, I am extremely grateful to have had her as a mentor and advisor. I would also like to thank my committee members, Dr. Joe Krajcik, Dr. Lynmarie Posey, and Dr. Danny Caballero for all the guidance, help, and support they have given me throughout my time as a graduate student. I would especially like to thank Dr. Joe Krajcik for taking me under his wing when I first started as a graduate student. From common group meetings to rich class discussions about design-based research, I am grateful for everything he has taught me. I have also had the pleasure of working with numerous graduate students and post-docs throughout my time at MSU, and I am thankful for all their support. I also want to recognize all the past teachers and professors who have inspired me to pursue a career in education. I have had lots of ups and downs throughout graduate school. At times, I have felt empowered and successful, and at other times, I have felt defeated and hopeless. From these experiences, I have grown tremendously as a person. The endless support of my family and friends has been instrumental in helping me get through graduate school. I owe my success to my parents for pushing me throughout my life and always believing in me. Most importantly, I want to thank my wife Hannah. Words cannot describe the profound impact she has had on my life. She is my rock and best friend. I am truly grateful for her unwavering love and support. v TABLE OF CONTENTS LIST OF TABLES ....................................................................................................................... viii LIST OF FIGURES ....................................................................................................................... ix INTRODUCTION ................................................................................................... 1 THEORETICAL FRAMEWORKS ........................................................................ 5 Constructivism ......................................................................................................................... 5 Social Nature of Learning ........................................................................................................ 6 Scaffolding ............................................................................................................................... 7 Social Constructivism .............................................................................................................. 8 Conceptual Change ................................................................................................................ 10 Coherence and Fragmentation .......................................................................................... 11 Resources .......................................................................................................................... 13 Knowledge Integration .......................................................................................................... 14 LITERATURE REVIEW ...................................................................................... 16 Student Understanding of Quantum Concepts ...................................................................... 16 Student Understanding of Atomic Spectroscopy ................................................................... 18 METHODOLOGY & REFINEMENT OF THE CLUE CURRICULUM ............ 22 Preface ................................................................................................................................... 22 Design-Based Research ......................................................................................................... 22 Chemistry, Life, the Universe, and Everything (CLUE) ....................................................... 24 Refinement of the CLUE Curriculum .................................................................................... 28 Student Interviews (F13)................................................................................................... 29 Curriculum Changes (F14) ............................................................................................... 34 Curriculum Changes (F15) ............................................................................................... 41 Curriculum Changes (F16) ............................................................................................... 47 Summary of Curriculum Changes .................................................................................... 49 CHARACTERIZATION OF STUDENTS’ REASONING ABOUT ATOMIC EMISSION SPECTRA ................................................................................................................. 51 Preface ................................................................................................................................... 51 Introduction ............................................................................................................................ 51 Theoretical Frameworks ........................................................................................................ 53 Methods ................................................................................................................................. 56 Participants and Setting..................................................................................................... 56 Data Analysis .................................................................................................................... 59 Knowledge Integration Rubric .......................................................................................... 59 Main Ideas and Links Coding Scheme ............................................................................. 61 Refinement of the Knowledge Integration Rubric ............................................................ 67 Spectroscopy Reasoning Rubric ....................................................................................... 69 Results.................................................................................................................................... 72 vi Multiple Connected Ideas ................................................................................................. 75 Connected Ideas ................................................................................................................ 76 Descriptive Process ........................................................................................................... 77 Disconnected Ideas ........................................................................................................... 79 Off-Topic .......................................................................................................................... 80 Discussion .............................................................................................................................. 81 Conclusions............................................................................................................................ 83 Implications and Future Work ............................................................................................... 84 Limitations ............................................................................................................................. 85 ANALYZING THE EFFECT THAT THE F13-F16 CURRICULUM CHANGES HAD ON STUDENTS’ REASONING ........................................................................................ 87 Preface ................................................................................................................................... 87 Introduction ............................................................................................................................ 87 Participants ............................................................................................................................ 90 Data Analysis ......................................................................................................................... 91 Results.................................................................................................................................... 92 Association between cohort and main ideas ..................................................................... 92 Association between cohort and links............................................................................... 95 Association between cohort and reasoning ....................................................................... 98 Discussion ............................................................................................................................ 101 Implications ......................................................................................................................... 107 Conclusions.......................................................................................................................... 110 CHARACTERIZATION OF STUDENTS’ REPRESENTATIONS OF THE PROCESS FOR HOW SPECTRAL LINES ARE CREATED .................................................. 112 Diagram Assessment Tasks ................................................................................................. 112 Data Analysis ....................................................................................................................... 114 Results & Discussion ........................................................................................................... 121 Implications ......................................................................................................................... 126 Conclusions.......................................................................................................................... 127 CONCLUSIONS, LIMITATIONS, AND FUTURE WORK ............................. 129 Conclusions.......................................................................................................................... 129 Limitations and Future Work............................................................................................... 133 APPENDICES ............................................................................................................................ 136 Appendix A: Fall 2013 interview protocol .......................................................................... 137 Appendix B: Fall 2013 spectroscopy homework activity .................................................... 140 Appendix C: Fall 2014 spectroscopy homework activity .................................................... 145 Appendix D: Fall 2014 recitation worksheet ....................................................................... 152 Appendix E: Fall 2015 spectroscopy homework activity .................................................... 154 Appendix F: Copy of the American Chemical Society’s policy on dissertations ............... 162 REFERENCES ........................................................................................................................... 163 vii LIST OF TABLES Table 4.1 – Coding scheme for student descriptions of absorption and emission process ............ 31 Table 4.2 – Level of sophistication in student explanations of the absorption and emission process .............................................................................................................................................. 32 Table 5.1 – Explanation assessment tasks (F13-F16) .................................................................... 58 Table 5.2 – Knowledge integration rubric and item examples ....................................................... 60 Table 5.3 – Main ideas and links coding scheme ........................................................................... 64 Table 5.4 – Spectroscopy reasoning rubric coding scheme ........................................................... 70 Table 5.5 – Inter-rater reliability results for main ideas, links, and SRR coding scheme .............. 72 Table 5.6 – Percentage of students who incorporated the main ideas, links, and modes of reasoning within their explanations of atomic emission spectra ......................................... 74 Table 6.1 – Curriculum changes and exam prompts for F13-F16 cohorts ..................................... 88 Table 6.2 – Association between cohort and main ideas................................................................ 93 Table 6.3 – Contingency table for the post-hoc analysis of the association between cohort and main ideas ............................................................................................................................ 94 Table 6.4 – Association between cohort and links ......................................................................... 96 Table 6.5 – Contingency table for the post-hoc analysis of the association between cohort and links ...................................................................................................................................... 97 Table 6.6 – Contingency table for the post-hoc analysis of the association between cohort and mode of reasoning .............................................................................................................. 100 Table 7.1 – Diagram assessment tasks on students’ midterm exam............................................. 113 Table 7.2 – Type of representation coding scheme for students’ drawing of atomic emission spectra ................................................................................................................................ 115 Table 7.3 – Mechanism coding scheme for students’ drawings of atomic emission spectra ....... 117 Table 7.4 – Reasoning and incorrect components coding scheme for students’ drawings of atomic emission spectra ................................................................................................................. 119 viii LIST OF FIGURES Figure 4.1 – Design-based research cycle for the iterative refinement of curriculum and assessments .......................................................................................................................... 24 Figure 4.2 – Energy difference questions added to the F15 homework activity ............................ 42 Figure 5.1 – Characterization of students’ reasoning about atomic emission spectra using the spectroscopy reasoning rubric .............................................................................................. 73 Figure 6.1 – Main ideas present within student explanations of atomic emission spectra ............. 92 Figure 6.2 – Links present within student explanations of atomic emission spectra ..................... 95 Figure 6.3 – Reasoning present within student explanations of atomic emission spectra ............. 98 Figure 7.1 – Type of representation in student diagrams ............................................................. 121 Figure 7.2 – Mechanism represented in student diagrams ........................................................... 123 Figure 7.3 – Reasoning and incorrect components in student diagrams ...................................... 125 Figure B.1 – Introduction slide (F13) ........................................................................................... 140 Figure B.2 – Explanation and representation prompt for energy quantization (F13) .................. 140 Figure B.3 – Explanation and representation prompt for difference between absorption and emission spectra (F13) ....................................................................................................... 141 Figure B.4 – Explanation and representation prompt for reflection (F13) ................................... 141 Figure B.5 – Multiple choice question for wavelength of a moving electron (F13) .................... 142 Figure B.6 – Multiple choice question for the wavelength of Usain Bolt (F13) .......................... 142 Figure B.7 – Representation prompt for an s orbital (F13) .......................................................... 143 Figure B.8 – Representation prompt for a p orbital (F13) ............................................................ 143 Figure B.9 – Representation prompt for mental image of what an atom would look like (F13) . 144 Figure C.1 – Introduction slide (F14) ........................................................................................... 145 Figure C.2 – Description of spectroscopy (F14) .......................................................................... 145 Figure C.3 – Explanation prompt for how an emission spectrum is created (F14) ...................... 146 Figure C.4 – Description of spectroscopy activity (F14) ............................................................. 146 ix Figure C.5 – Representation and explanation prompt for the atomic structure of a hydrogen atom (F14) ................................................................................................................................... 147 Figure C.6 – Explanation and representation prompt for the absorption process (F14) .............. 147 Figure C.7 – Compare and contrast prompt for the relationship between student’s representation of the absorption process to an energy level diagram representing the absorption process (F14) ................................................................................................................................... 148 Figure C.8 – Explanation prompt for why an electron transitions to a higher energy level (F14) ............................................................................................................................................ 148 Figure C.9 – Multiple choice and explanation prompt for the energy required to promote an electron to a higher energy level (F14) .............................................................................. 149 Figure C.10 – Representation and explanation prompt for the emission process (F14) .............. 149 Figure C.11 – Explanation prompt for how different colors are emitted (F14) ........................... 150 Figure C.12 – Explanation prompt for how a blue line in the emission spectrum is created (F14) ............................................................................................................................................ 150 Figure C.13 – Explanation prompt for why only specific lines appear in hydrogen’s emission spectra (F14) ...................................................................................................................... 151 Figure C.14 – Explanation prompt for why each element has a unique emission spectrum (F14) ............................................................................................................................................ 151 Figure D.1 – Helium’s atomic emission spectrum ....................................................................... 153 Figure E.1 – Introduction slide (F15) ........................................................................................... 154 Figure E.2 – Description of spectroscopy (F15) .......................................................................... 154 Figure E.3 – Explanation prompt for how an emission spectrum is created (F15) ...................... 155 Figure E.4 – Description of spectroscopy activity (F15) ............................................................. 155 Figure E.5 – Representation and explanation prompt for the atomic structure of a hydrogen atom (F15) ................................................................................................................................... 156 Figure E.6 – Explanation and representation prompt for the absorption process (F15) ............... 156 Figure E.7 – Compare and contrast prompt for the relationship between student’s representation of the absorption process to an energy level diagram representing the absorption process (F15) ................................................................................................................................... 157 Figure E.8 – Multiple choice and explanation prompt for how much energy would be required for an electron to transition to a higher energy level (F15) ..................................................... 157 x Figure E.9 – Multiple choice and explanation prompt for how much energy would need to be absorbed for an electron to transition to a higher energy level (F15) ................................ 158 Figure E.10 – Representation and explanation prompt for the emission process (F15)............... 158 Figure E.11 – Explanation prompt for how much energy would be emitted as an electron transitions to a lower energy level (F15) ........................................................................... 159 Figure E.12 – Multiple choice and explanation prompt for the electronic transition that emits a photon with higher energy (F15) ....................................................................................... 159 Figure E.13 – Explanation prompt for how a blue line in the emission spectrum is created (F15) ............................................................................................................................................ 160 Figure E.14 – Explanation prompt for why only specific colors are absorbed or emitted (F15) . 160 Figure E.15 – Explanation prompt for why each element has a unique emission spectrum (F15) ............................................................................................................................................ 161 xi INTRODUCTION Understanding the role energy plays in chemical systems is central to developing a robust understanding of how atoms and molecules interact with one another. Energy is not only central to learning chemistry, but it is a core idea that spans across scientific disciplines (National Research Council, 2012). However, even though it is an important and central concept, ideas about energy are complex and unintuitive. For instance, colloquial views of energy (e.g. you get energy from food or you use energy when doing physical activity) can be more intuitive and relatable to students compared to scientific views of energy (e.g. energy conservation, transfer, and quantization). Furthermore, energy is symbolized in a variety of different ways (e.g. U, G, H, PE, ΔE), which adds an extra layer of abstraction to an already complex concept. Due to this complexity, there is a great deal of evidence students have a difficult time understanding energy (Becker & Cooper, 2014; Boo, 1998; Nilsson & Niedderer, 2014; Özmen, 2004; Teichert & Stacy, 2002). It has been famously noted by Nobel Laureate Richard Feynman that “it is important to realize that in physics today [1964], we have no knowledge of what energy is” (Feynman, Leighton, & Sands, 2011). Given that this sentiment still rings true today, it is not surprising that students find this concept confusing given the various ways in which energy is discussed both within and across disciplines. This raises an important question – how can science educators improve the teaching and learning of energy concepts? Acquiring a deep understanding of complex ideas, such as energy, cannot be achieved in a single lesson or intervention. Instead, it is a process that occurs over time as students continually grapple with and reorganize what they know into a more integrated knowledge structure. From this perspective, learning can be enhanced through coherent materials that follow a logical progression. One theoretical approach for how to move students toward a more 1 expert level of understanding is by using Learning Progressions, which are evidence-based descriptions for how students’ understanding of a core idea develops and increases in sophistication over time (Duschl, Maeng, & Sezen, 2011; Smith, Wiser, Anderson, & Krajcik, 2006). Core ideas, also called big ideas, are the overarching ideas that are fundamental to a discipline. They have broad applicability and serve as a “key tool for understanding or investigating more complex ideas and solving problems” (National Research Council, 2012). Within the context of a reformed general chemistry curriculum entitled Chemistry, Life, the Universe, and Everything (CLUE), Cooper et al. (2014) have worked to develop a scaffolded energy learning progression to help students better understand the role energy plays at the macroscopic, atomic-molecular, and the quantum-mechanical scale. Prior research in the Cooper Research Group has focused on characterizing students’ understanding of energy from the macroscopic and atomic-microscopic level (Becker & Cooper, 2014; Becker, Noyes, & Cooper, 2016; Cooper, Williams, & Underwood, 2015; Williams, Underwood, Klymkowsky, & Cooper, 2015). This study builds upon this prior research and looks at how students in the CLUE curriculum understand energy from the quantum-mechanical perspective. Specifically, this work looks at how students understand and explain atomic spectroscopy. Overall, three research questions guided this study: RQ1: How do general chemistry students reason about atomic emission spectra? RQ2: How can the general chemistry curriculum be designed/refined to improve students’ reasoning about atomic emission spectra? RQ3: What effect do the implemented curriculum and assessment changes have on students’ reasoning about atomic emission spectra? 2 The primary goal of this study was to characterize general chemistry students’ understanding of atomic spectroscopy (RQ1) and to improve the treatment of spectroscopy concepts within the CLUE curriculum (RQ2 & RQ3). Utilizing a design-based research methodology (Brown, 1992; Collins, 1992), a series of curriculum changes were made to the spectroscopy section of the CLUE general chemistry curriculum from Fall 2013 to Fall 2016, resulting in four different cohorts (F13, F14, F15, and F16) who were enrolled in CLUE at different phases of curriculum refinement. The specific changes that were implemented during each phase of the study (F13- F14, F14-F15, F15-F16) were directly influenced by evidence obtained from the previous semester. Chapter 4 describes the changes that were made to instruction, homework, recitation, and the summative assessment tasks from F13-F16. Each cohort was asked a constructed response question on their midterm exam in which they were prompted to explain the process for how an atomic emission spectrum is created. Student explanations to these assessment tasks were analyzed and used to develop a coding scheme (later referred to as the Spectroscopy Reasoning Rubric) that characterizes the connections students made between spectroscopic concepts. Chapter 5 describes the development of this coding scheme and presents findings on how students within the CLUE curriculum reasoned about atomic emission spectra. Once the curriculum changes (Ch.4), coding scheme (Ch.5), and reasoning patterns (Ch.5) have been presented, Chapter 6 discusses the effect that the changes to curriculum and assessment had on students’ reasoning over the course of this study. In addition to the explanation task, each cohort was asked to illustrate the process for how a spectral line in an emission spectrum is created. Chapter 7 describes how students represented 3 this process. Finally, Chapter 8 summarizes the findings for each research question and discusses implications for teaching and future research. 4 THEORETICAL FRAMEWORKS Constructivism Science as an enterprise is not just an accumulation of facts, but it is how these foundational ideas and ways of knowing can be used to make sense of the world (National Research Council, 2007). Thus, an overarching goal of science education is to help students understand foundational concepts and to help them apply their knowledge in meaningful ways, such as explaining a scientific phenomenon or designing solutions to problems. This requires both knowledge of the content and the practices of doing science (National Research Council, 2012). It is through these practices that deeper understanding and meaning of content knowledge is established. Knowledge cannot simply be transferred to students as if they were blank slates; instead, students need to actively construct their own understanding. This view of learning, referred to as Constructivism, focuses on the ways in which knowledge is conceptualized within the mind of the learner (Bodner, 1986; Piaget, 1964). The view that students construct their own understanding of the world was popularized by the work of Piaget, who found that children think about the world differently than adults (Murray, Hufnagel, Gruber, & Vonèche, 1979). While this may seem obvious with what is known today, at the time, this finding was revolutionary and sparked a cultural shift in education. Rather than seeing students as blank slates in which knowledge can be imparted upon, there was emerging evidence that students construct their own understanding of the world around them. Consequently, this means that learning is not only a cognitive process that happens solely within the individual, but that the surrounding environment and the socio-cultural context directly influences how learning is conceptualized. 5 Social Nature of Learning Decades of educational research has shown that “each learner develops a unique array of knowledge and cognitive resources in the of course of life that are molded by the interplay of that learner’s cultural, social, cognitive, and biological contexts” (National Research Council, 2018). Ultimately, this means that learning is not a simple process in which knowledge is acquired and put into the brain, but instead, learning is an extremely complex process which is influenced by a multitude of factors (National Research Council, 2000). As a result, learning is not just an individual endeavor, but the environment and culture that person is immersed within directly influences how a person conceptualizes and makes sense of the world. Within a classroom context, the interactions between teachers and students are directly influenced by both the physical environment and resources being used. It is important that learning environments are coherently organized in order to support and help students develop an integrated understanding (Krajcik, Slotta, McNeill, & Reiser, 2008). Because the environment and its respective entities directly influence how people learn, learning can be viewed as a community of practice in which groups of people engage in a collective process of learning through interactions with one another (Wenger, 1998). The perspective that learning occurs in communities of practice builds upon the notion that learning is “an active process that is deeply social, embedded in a particular cultural context, and enhanced by intentional support provided by more knowledgeable individuals, be they peers, mentors, or teachers.” (Honey, Pearson, & Schweingruber, 2014). This sentiment shows that learning is not just an individual endeavor, but something more, a collective aggregation of people who share and work toward a common goal, actively supporting one another – an act that helps members of the community to engage in the learning process. 6 Scaffolding With learning being inherently social in nature, the question arises, how can interactions between peers support and enhance the learning process? One potential view for how to address this issue comes from the work of Soviet psychologist Lev Vygotsky who proposed the learning theory known as the Zone of Proximal Development (ZPD), which is defined as the distance between an individual’s developmental level and their potential development from collaboration with more knowledgeable peers (Vygotsky, 1978). The following example will help unpack and explain the role the Zone of Proximal Development can play in learning. Imagine a girl in chemistry who is interested in learning about atomic structure. This student looks through her chemistry textbook and sees that that there are multiple models of the atom. She may be able to remember details about each model, but how these models relate to one another and what they are used for may not be known. This can be thought of as the extent to which she can learn about atomic structure on her own. When this girl goes to class, the teacher leads the class through an activity that focuses on how the various models are related to one another and how they have changed over time. With the support of a more knowledgeable person, the student might be better equipped to understand relationship between the various models and expand upon her prior knowledge from reading the textbook. Based on the social interactions within the classroom, the teacher may help the student gain a better understanding of atomic structure compared to what she would have learned on her own. This example illustrates how learning experiences can be enhanced with proper support and interactions with others. This act of supporting learners is commonly referred to as scaffolding. Like the name suggests, scaffolds provide temporary structure and support to students as they engage in the learning process. They are not meant to be permanent, but instead, scaffolds are removed and faded away once the fundamental structure and base has been built. 7 As a result, scaffolds within learning environments should be designed to support and help students generate a deeper understanding. However, if too much “support” is given, students may resort to memorization and algorithmic problem solving because they do not need think deeply about a concept due to over scaffolding. Decades of research has looked at how to scaffold student learning. In a meta-analysis on scaffolding, Van de Pol, Volman and Beishuizen (2010) found that the results from effectiveness studies suggest that scaffolding is an effective way to improve student learning. However, they did note that more studies are needed and that measuring changes in scaffolding is particularly difficult. Science concepts can often be difficult for students to understand, and as a result, scaffolding is important in both instruction as well as assessment. Prior research has shown that the nature of an assessment task directly influences how students’ reason about scientific phenomena (Cooper, Kouyoumdjian, & Underwood, 2016). This has direct implications on how to design and scaffold assessment tasks that elicit evidence about what we want students to know and be able to do with their knowledge – a theme which is addressed in further detail in Chapter 6. Social Constructivism The fusion of constructivist and socio-cultural perspectives of learning can be viewed through the lens of a learning theory called Social Constructivism, which proposes that the interaction of an individual and their surrounding environment influences how meaning and understanding is conceptualized within the learner (Orey, 2010). Simply put, social interactions affect how a concept is internalized within an individual. The ways in which a concept is interpreted will be impacted by others’ perspectives and views. Similarly, a person’s prior 8 knowledge and past experiences will shape the way new information is conceptualized. This is predicated on the view that students are not blank slates in which information can be transferred, but that the way a person constructs understanding and develops meaning of a concept is greatly influenced by their past experiences. To illustrate how learning occurs from a socio-constructivist perspective, imagine a group of two students, whom I will refer to as David and Sarah. They are working together to explain the photoelectric effect – a phenomenon in which electrons are ejected from a metal when light shines upon it. David begins by sketching a representation that shows light hitting a sheet of metal and electrons being ejected from the metal. In explaining how electrons are emitted, David mentions that light is a wave and that by increasing the intensity of the light, electrons will be emitted from the metal. However, the experimental results they were provided showed that increasing the intensity of the light did not result in electrons being emitted. David is confused and has trouble reconciling why his explanation is wrong. Sarah recalls that light acts as both a wave and a particle. She brings this to David’s attention and mentions that to describe the data, they need to think of light as a particle rather than a wave. Sarah goes on to explain that increasing the intensity of the light has no impact on whether an electron is emitted, but that the light must have a certain threshold frequency for electrons to be emitted. Light must be thought of as particle rather than a wave in order to explain the fact that increasing the intensity of light does not result in the ejection of an electron. David was thinking about light as having wave-like properties, and as a result, could only explain the phenomena from the perspective of light as a wave. This example highlights how learning might occur for David. In his explanation of the photoelectric effect, David draws on his prior knowledge that light acts a wave and uses this 9 perspective in his explanation of the photoelectric effect. However, because he does not know or remember that light can also act as a particle, he is unable to construct a meaningful explanation for the given data. It is through the social interaction with his partner, Sarah, who helps him make sense of the information. Sarah’s views and perspectives influence how he comes to conceptualize and understand the photoelectric effect. Even though Sarah may have offered a canonical explanation, the way in which David internalizes this information will be informed by his prior conceptions of the photoelectric effect. How deeply held his prior beliefs are will have a direct impact on how new information is integrated into his cognitive framework. If the prior conception that light acts only as a wave is a loosely held belief, he may easily assimilate the idea that light is both a wave and particle into his existing conceptual framework, but if the notion of light as only a wave is a deeply held belief, a process of conceptual change may need to occur in order for this new perspective of light to be acquired. Conceptual Change Conceptual change has been defined as the “gradual modification of one’s mental models of the physical world, achieved either through enrichment or through revision” (Vosniadou, 1994). According to Chi (2008), mental models reflect an “internal representation of a concept” that are developed through the aggregation of relevant pieces of knowledge in one’s conceptual framework. The ways in which knowledge is constructed varies for every individual, which can potentially manifest into a non-canonical understanding of a concept or relationship between concepts. Often termed misconceptions or alternative conceptions, these perspectives can impede student learning and hinder students’ ability to make meaningful connections between ideas. 10 When alternative conceptions are deeply held within an individual, conceptual change can be quite difficult. For learning that is not simply knowledge acquisition, the process of conceptual change is heavily influenced by an individual’s knowledge structure and their prior conceptions. Inspired by the work of Thomas Kuhn (1962) and Stephen Toulmin (1975), two dominant camps have emerged for how knowledge is organized within the mind of the learner. Coherence and Fragmentation Much of the early work in conceptual change focused on identifying the misconceptions students have in science under the premise that if we can identify students’ incorrect ideas, we can explicitly address these wrong ideas and replace them with correct ideas (Posner, Strike, Hewson, & Gertzog, 1982). In fact, an entire book has been written that documents the large number of misconceptions that students have about science (Confrey, 1990). How students come to understand and think about the world directly relates to how that knowledge is synthesized and organized into their conceptual framework. The question of how naïve ideas are structured within a mental framework has been described as an epistemologically fundamental idea in understanding the nature of knowledge (diSessa, 2008). In making sense of a learner’s knowledge structure, the learning theories of Knowledge-in-Pieces (1988) and Theory-Theory (1994) have played prominent roles within conceptual change research. Both theories take a constructivist approach and recognize that prior knowledge is important for new knowledge construction, but they differ in their views on how a person’s knowledge structure is conceptually organized. Theory-theory, also called framework theory, describes the structure of prior knowledge as a coherent explanatory framework (Vosniadou, 1994), whereas knowledge-in-pieces sees it as a loose connection of a variety of fine grained and fragmented bits of knowledge (diSessa, 1988). In the framework theory, 11 coherent does not refer to coherence in the canonical sense that a learner’s ideas are scientifically correct, but it refers to the fact that individuals have coherent perspectives that are naïve in nature and can be restructured (Vosniadou, 1994). Whether one views knowledge as being coherent or existing as fragmented bits of knowledge, these differing perspectives directly influence the way one views and thinks about the world in both explicit and implicit ways. Even though knowledge-in-pieces and theory- theory offer different perspectives for how knowledge is structured, both theories serve as theoretical rationales for how conceptual change can occur. If you wanted to invoke conceptual change from a coherence perspective, it may seem logical to identify the misconceptions students have, make them aware of their incorrect ideas to create cognitive dissonance, and then replace these incorrect ideas with normative scientific ideas – ultimately changing the coherently incorrect ideas to coherent correct ideas (Posner et al., 1982). When viewing student knowledge as fragmented, conceptual change could be viewed as a process of making connections between concepts more explicit so that students can reweave and integrate their fragmented ideas – ultimately trying to minimize the number of fragments within a learner’s conceptual framework. However, given these two scenarios, there is little evidence for how to bring about long-term conceptual change from either approach. This does not mean that these are not useful perspectives, but quite the contrary – these perspectives directly influence a person’s world views and they provide a lens for how to view educational initiatives. In thinking about how one might induce conceptual change within the context of chemistry, it is important to think about the factors that affect student learning. Chemistry is a field of study that is abstract and unintuitive because learners need to change their frame of reference from the macroscopic world which they have always known and 12 experienced, to the microscopic world, which is complex and unintuitive (Johnstone, 1991). This means that their prior knowledge, ways of knowing, and interactions with the macroscopic world will not be as relevant in learning compared to other disciplines that focus on phenomena at the macroscopic scale that can be directly related back to a person’s prior experiences. In order to learn chemistry, students need to not only learn foundational concepts, but they also need to learn the symbolism and models that are used to represent the atomic-molecular world. Due to this complexity, students need to take in a vast amount of new information and synthesize it into their existing conceptual framework. One way to think about this process is through the Resources Perspective of Learning, which builds upon the knowledge in pieces perspective of learning (Hammer, 2000; Hammer & Elby, 2003). Resources Rather than viewing student knowledge as either coherent or fragmented, the Resources Perspective of Learning views the mind of the learner as a conceptual schema that exists as a complex network of interrelated resources (Hammer, 2000; Hammer & Elby, 2003). These resources are the different ways students understand concepts and their relationship to one another. As students learn new material, they are continually adding new ideas, sorting ideas, and grappling with how one concept might relate to one another, either explicitly or implicitly. From this process, students develop conceptual resources that serve as the foundational knowledge that they use to make sense of the world around them. Through the lens of the Resources Framework, learning can be thought of as the culmination of many fine-grained resourced being activated and connected to one another. Students’ understanding of concepts are directly related to their understanding of other concepts, and as students learn, they develop local pockets of coherence amongst these ideas. These 13 resources can be thought of as a conceptual ecology in which a learner continually draws upon, refines, and applies when engaging in the learning process. How students activate these resources and apply their knowledge is highly dependent on the context and the framing of a task – thus, knowledge is emergent and directly influenced by a variety of contextual factors. As a result, instruction should aim to help students develop epistemological resources that are productive for learning (Hammer & Elby, 2003). Knowledge Integration One approach for how to help students develop a more integrated understanding is by explicitly focusing on the relationship and connections between concepts. This approach toward improving student learning is at the core of the Knowledge Integration Framework (KIF) (Linn, 2006; Linn & Eylon, 2011). The KIF is a learning theory that describes the knowledge integration process students go through as they make sense of science phenomena (Linn, Bell, & Hsi, 1998). Students come to our classrooms with a wide array of ideas about scientific phenomena that they have learned from both their everyday experiences and their prior schooling (National Research Council, 2000). As students engage in the learning process, they need to think of how new ideas relate to their existing knowledge and readily make connections between these ideas. Students need to “add new ideas from instruction or experience, sort out these ideas, learn how to combine them, develop a sense of coherence among ideas, and recognize new situations where these ideas apply” (Clark & Linn, 2003). In a broad sense, knowledge integration can be described as a process of integrating new knowledge into an existing conceptual framework and making connections between newly acquired knowledge and relevant prior knowledge. The importance of making connections 14 between ideas is fundamental to developing a deeper understanding of science concepts and principles. Historically, the Knowledge Integration perspective has been used to develop Knowledge Integration Environments that are designed to foster knowledge integration using technology (Linn & Eylon, 2011). In this work, the KIF was not used to develop a learning environment, but instead, it was used as lens for how to characterize the productive resources students use to make sense of spectroscopic phenomena. The KIF was used to develop a coding scheme that captures the different ideas students’ reason about and the connections they make between them when explaining atomic emission spectra. This will be discussed in more detail in Chapter 5. Decades of educational research has shown that a multitude of factors affect how people learn (National Research Council, 2000, 2007, 2012, 2018). The theoretical perspectives of learning discussed in this chapter are by no means a complete account for how students learn science, but instead, they provide a simplified model for how science learning occurs. Together, these perspectives provided a lens in which students’ understanding of spectroscopy was viewed within this study. 15 LITERATURE REVIEW Student Understanding of Quantum Concepts Spectroscopy is inherently quantum-mechanical in nature. Thus, understanding how light interacts with matter at the atomic and molecular level requires students to shift their thinking from the macroscopic level to the unseen and complex sub-microscopic world. However, navigating among the macroscopic, atomic-molecular, symbolic, and quantum-mechanical ideas and representations can be challenging for students (Johnstone, 1991; Tsaparlis, 2014). To better understand how students perceive the difficult nature of learning quantum- mechanical concepts, Tsaparlis (2016) interviewed chemistry graduate students and asked them for their opinions on the difficulties they faced in learning physical chemistry. Students unanimously reported that they found the subjects of thermodynamics, kinetics, and electrochemistry easier than quantum chemistry. Their primary reasoning was that quantum mechanics is more abstract due to its mathematical nature. They believed that classical thermodynamics is easier because it has more logic and examples that relate to their prior experiences. We know that people learn better when learning experiences leverage and build upon their prior knowledge (National Research Council, 2018), and so, it should come to no surprise that this is a major difficulty students face in learning quantum concepts. In general, students struggle to develop a meaningful understanding of concepts that are quantum-mechanical in nature (Johnston, Crawford, & Fletcher, 1998; Singh, 2001; Tsaparlis & Papaphotis, 2009). Several studies have shown that students have difficulties understanding the quantum concepts of atomic and molecular orbitals (Barradas-Solas & Sánchez Gómez, 2014; Nakiboglu, 2003; Stefani & Tsaparlis, 2009; Taber, 2002a, 2002b, 2005; Tsaparlis & Papaphotis, 1997, 2002), hybridization (Nakiboglu, 2003; Stefani & Tsaparlis, 2009), ionization energy 16 (Taber, 2003), quantum numbers (Niaz & Fernández, 2008), and quantization (Didiş, Eryılmaz, & Erkoç, 2014; Didiş Körhasan & Wang, 2016; Park & Light, 2009; Savall-Alemany, Domènech-Blanco, Guisasola, & Martínez-Torregrosa, 2016). For example, a study by Park and Light (2009) provided insight into the difficulties that chemistry students have in understanding energy quantization when learning about atomic structure. These authors were interested in characterizing students’ mental models of atomic structure and identifying threshold concepts that can hinder students understanding of quantum models of the atom. They conducted interviews with general chemistry students and based on their findings developed a two-part coding scheme to characterize the different mental models students have about atomic structure. They described four separate mental models that students have about atomic structure (Particle, Nuclear, Bohr, and Quantum), and within those models, they identified 13 hierarchical levels. By looking at three-high achieving students who were described as having a “single coherent model of atomic structure”, the authors proposed that the concepts of probability and energy quantization are threshold concepts that are fundamental to developing a quantum model of atomic structure. Several suggestions have been proposed for how to help students better understand quantum concepts. Some authors have proposed that teaching of quantum-mechanical concepts should be grounded in the History and Philosophy of Science (Greca & Freire, 2014; Justi & Gilbert, 2000; Rittenhouse, 2015; Sánchez Gómez & Martín, 2003). For example, Greca and Freire (2014) argued that current textbooks provide an oversimplified view of Bohr’s model of the atom that has “little or no explanation of the logic by which it was obtained.” Instead, they 17 proposed that students should learn about key discoveries and derive the equation for the potential energy levels within a hydrogen atom. A variety of other suggestions and approaches have been proposed for how to help students develop a deeper understanding of quantum-mechanical concepts, such as the development of new labs and activities (Armstrong, Burnham, & Warminski, 2017; Mowry, Milofsky, Collins, & Pimentel, 2017; Zollman, Rebello, & Hogg, 2002), using different teaching methods (Deslauriers & Wieman, 2011), utilizing technology (Zollman et al., 2002), developing concept maps (Aguiar & Correia, 2016), and the use of visual-conceptual tools to promote conceptual understanding (Dangur, Avargil, Peskin, & Judy Dori, 2014). In general, there has been a growing emphasis on the need for a more conceptual approach to teaching quantum- mechanical concepts (deSouza & Iyengar, 2013; Kalkanis, Hadzidaki, & Stavrou, 2003; Pepper, Chasteen, Pollock, & Perkins, 2012). Student Understanding of Atomic Spectroscopy Research on students’ understanding of light-matter interactions has been conducted within the context of chemistry, physics, and astronomy classrooms. Together, these studies have primarily focused on identifying the misconceptions and difficulties students have in understanding light-matter interactions. For example, a study by Körhasan & Wang (2016) characterized physics students’ mental models of atomic spectra. Through semi-structured interviews (N=9), the authors asked students to explain various aspects of atomic spectra to reveal how they were thinking about spectroscopy concepts and to identify any threshold concepts that may hinder their understanding. These authors identified six different concepts that were present within student responses: bound electron, discrete energy levels, spectral lines, photon energy, electronic transition, and orbit. 18 Then, based on the concepts present within a student’s explanation of atomic emission spectra, the authors coded students’ responses as having one of four possible mental models about atomic spectra. The authors reported that seven out of the nine students interviewed had unscientific models of atomic spectra, and they identified the concepts of electronic transition and energy of photons as two threshold concepts that students struggled with in developing a more appropriate model. Together, these two concepts are important for understanding the underlying mechanistic process for how spectral lines are created. In a different study, Ivanjek and colleagues (2014) administered a series of questions to physics students at the end of the spectroscopy unit to investigate their understanding of the relationship between electronic transitions and energy levels. Seven incorrect ideas were identified including the idea that a spectral line is equal to an energy level and that the ground state is always involved in a transition. They reported that “many students did not seem to recognize that each spectral line is a result of a transition of an electron between two energy levels.” This finding suggests that physics students, just like chemistry students (Park & Light, 2009), had difficulty understanding and explaining the mechanistic process for how an atomic emission spectrum is created. Based on their findings, Ivanjek and colleagues (2015) developed a spectroscopy tutorial designed to elicit students’ ideas, confront their wrong ideas, and help “students recognize the conflict between their initial ideas … and come to understand the reasoning required for resolution.” Ultimately, the tutorial was designed to make students aware of the wrong ideas they have about atomic emission spectra and help them overcome their incorrect understanding based on the cognitive dissonance instigated by the tutorial. The tutorial was administered in various contexts at two different universities. To assess how the tutorial affected students’ understanding, 19 the authors collected data both before and after the tutorial and reported that on the post-test, 85- 95% of the participants were able to correctly identify the number of energy levels within the atom based on the number of spectral lines present, whereas on the pretest, only 15% of the introductory physics students and 45% of the honors students could accurately perform the task. While they found that the tutorial improved students’ immediate responses about the relationship between energy levels and spectral lines, more research would need to be done to see how the tutorial effects students’ reasoning of other important spectroscopic concepts. Taking a different approach to improving students’ understanding of atomic spectroscopy, Moon et al. (2018) designed a Writing-to-Learn (WTL) activity in which physical chemistry students applied their understanding of quantum concepts to explain the composition of the Orion Nebula. The participants were instructed to summarize an astronomy professor’s research and to write an article for a university research newsletter. To do this, students were prompted to explain three features: (1) the quantum nature of light, (2) how light interacts with matter at the atomic-molecular level, and (3) how these interactions relate to chemical composition. The authors report that the WTL activity improved students’ conceptual understanding of spectroscopy concepts. They also found that both students’ confidence and ability to explain electronic transitions improved from the WTL activity. These findings are promising and provide evidence that students’ conceptual understanding of spectroscopy can be improved by having students construct rich explanations of scientific phenomena. Finally, two concept inventories have been developed that can be used to diagnose students’ understanding of atomic emission spectra (Bardar, Prather, Brecher, & Slater, 2006; Bretz & Murata Mayo, 2018). Bretz and Mayo (2018) developed a flame test concept inventory that was designed to identify students’ misconceptions about atomic emission. Based on 20 interviews with general chemistry students, they identified seven categories of alternative conceptions students have about atomic spectra, including misrepresentations of atomic emission, electrons being gained/lost, and ideas about breaking/forming bonds. A light and spectroscopy concept inventory (LSCI) was developed by Bardar et al. (2006) within the context of an introductory astronomy classroom. Consisting of 26 items, the LSCI was designed to identify students’ conceptual understanding of light-matter interactions. Together, these studies show that both chemistry and physics students struggle to understand the various quantum concepts that underpin atomic spectroscopy. While this research on students’ misconceptions or incorrect ideas illuminates the difficulties students have in understanding this complex phenomenon, there is a need for more research that explores the various ways in which students integrate and apply their knowledge. 21 METHODOLOGY & REFINEMENT OF THE CLUE CURRICULUM Preface In this chapter, I describe how a design-based research methodology was used to refine the treatment of spectroscopy concepts within a general chemistry curriculum called Chemistry, Life, the Universe, and Everything (CLUE). First, I describe design-based research and discuss how it can be used to make evidence-based curriculum changes. Next, I describe the CLUE curriculum and how spectroscopy is taught within this course. The second half of this chapter explains the curriculum and assessment changes that were made within CLUE over the course of this study. In total, the CLUE curriculum was refined three times (F13-14, F14-F15, and F15-F16). To provide context for why specific changes were made during each phase of this study, I provide a brief description of the findings that influenced these changes. However, these findings are discussed in more detail in Chapters 5 and 6. Design-Based Research Prior research on students’ understanding of atomic emission spectroscopy provides evidence that students have difficulties understanding and making sense of spectroscopic phenomena (Bardar et al., 2006; Bretz & Murata Mayo, 2018; Didiş Körhasan & Wang, 2016; Ivanjek et al., 2014, 2015). While this research characterizes what students do not know about light-matter interactions, there is a need for more research on the productive resource’s students use to make sense of spectroscopic phenomena. Thus, the primary goal of this dissertation was to (1) identify the productive ways students’ reason about atomic spectra and to (2) implement evidence-based changes within the general chemistry curriculum aimed at improving students’ understanding of how light interacts with matter. To address these two goals, a design-based 22 research methodology was used to refine the treatment of spectroscopy concepts within the general chemistry curriculum. Design-Based Research (DBR) is a methodological approach, or more so, series of approaches for how to study the effect an implemented design may have on student learning in real world contexts (Collins, Joseph, & Bielaczyc, 2004).The goal of DBR is to develop evidence-based claims about how people learn and to identify design features that have the potential to improve teaching and learning (Barab & Squire, 2004). Originating from the work of Ann Brown (1992) and Alan Collins (1992), design research was proposed as an approach for how to test and generate theory of how students learn in more naturalistic settings. Historically, a large proportion of educational research has focused on identifying single variables and measuring the effect that these factors have on student learning (Collins et al., 2004). Instead of isolating specific factors that influence student learning, DBR embraces the messiness of learning environments. Decades of research has shown that learning environments directly influence teaching and learning (National Research Council, 2018), and so, DBR has emerged as a methodologic approach for how to investigate the effect that different learning environment designs have on student learning. At its core, DBR takes a pragmatic approach to educational research in which educational designs (i.e. interventions, activities, assessments, curriculum, etc.) are developed, implemented, and assessed within the context of real-world learning environments. Within DBR, design features are progressively refined as new evidence emerges on the effect that the implemented changes had on student learning. An example of this iterative design-based research cycle is illustrated in Figure 4.1. 23 Figure 4.1 – Design-based research cycle for the iterative refinement of curriculum and assessments Implementation of DBR is often done through an iterative design research process where student learning is assessed and then used as evidence to design or refine curricular materials. Once a change has been made, student learning is reassessed to see how the implemented design effected students’ understanding. If warranted, further changes may need to be implemented to better help students reach a desired learning goal. Chemistry, Life, the Universe, and Everything (CLUE) This methodological approach has been used to develop and refine a transformed general chemistry curriculum – Chemistry, Life, the Universe, and Everything (CLUE) (Cooper & Klymkowsky, 2013) which serves as the context for research discussed in within this dissertation. The CLUE curriculum was originally developed around an interconnected learning progression of three big ideas – molecular level structure, macroscopic properties, and energy. Designed to help students “develop a molecular-level understanding of core chemical 24 principles”, the CLUE curriculum emphasizes the role energy plays in chemical systems and how this affects structure-property relationships (Cooper & Klymkowsky, 2013). Since its inception in 2010, a variety of studies have looked at the effect that the CLUE curriculum has had on students’ understanding of different chemistry concepts, such as structure- property relationships (Cooper, Corley, & Underwood, 2013; Cooper, Underwood, Hilley, & Klymkowsky, 2012; Kararo, Colvin, Cooper, & Underwood, 2018; Kohn, Underwood, Cooper, & Loertscher, 2018; Underwood, Reyes-Gastelum, & Cooper, 2015, 2016), potential energy (Becker & Cooper, 2014), intermolecular forces (Becker et al., 2016; Cooper et al., 2015; Williams et al., 2015), and acid-base reactions (Cooper et al., 2016; Crandell, Kouyoumdjian, Underwood, & Cooper, 2018). Findings from this research has been used to make evidence- based changes within CLUE, thus resulting in an ever-evolving curriculum that is continually being altered as new evidence about students’ understanding emerges. Currently, the CLUE curriculum emphasizes the relationship between four core interconnected concepts: electrostatic and bonding interactions, atomic-molecular structure and properties, change and stability in chemical systems, and energy (Cooper, Posey, & Underwood, 2017). Centered around these four overarching concepts, the CLUE curriculum emphasizes how these ideas can be used to explain and predict chemical phenomena. This study builds upon the prior research that has been conducted using the CLUE curriculum and looks at how students’ reason about spectroscopic concepts within this reformed general chemistry course. The CLUE curriculum has been implemented at multiple universities; however, this study takes place within the context of the CLUE curriculum as enacted at Michigan State University (MSU). At MSU, general chemistry students attend lecture, which consists of three 50-minute lectures or two 80-minute lectures, and recitation, which meets for 50 minutes once a week. The 25 CLUE textbook is provided to students free of charge, and it directly aligns with the lecture materials. A traditional lecture period begins with the instructor reviewing student’s homework. Students complete their homework on an online platform called beSocratic (Bryfczynski, 2012). BeSocratic is used to generate formative assessment activities that engage students in the construction of freeform drawings, graphs, models, and explanations of scientific phenomena. These activities are designed to make student thinking and understanding visible, which in turn, the lecturer reviews and uses to tailor their instruction within the lecture period. Once the lecturer reviews the homework activity, students learn new concepts through a combination of direct instruction, iClicker questions, and small group discussions. Graduate teaching assistants attend lecture and provide guidance to students during discussion periods. Students attend recitation once a week and work in groups of three or four on a recitation worksheet. The recitation materials consist of discussion focused questions that require students to construct models, representations, and explanations. A graduate teaching assistant supports students as they work through the recitation activity and provides personal feedback to each group of students. Recitation is designed to be safe environment where students feel free to express their thinking by engaging in rich discussions with their peers. Within the CLUE curriculum, students learn about light-matter interactions early in the first semester, where these ideas are introduced as evidence for quantum models of the atom. Properties of electromagnetic radiation are discussed, and students learn about the relationship between energy, wavelength, and frequency. To provide evidence that light can act as a wave, the double-slit experiment is discussed. To provide evidence that light can also act as a particle, the photoelectric effect is presented. Once students have learned that electromagnetic radiation 26 can act as a both a wave and a particle, atomic spectroscopy is discussed as evidence for the quantized nature of energy levels within the atom. Students learn about the mechanistic process by which an atomic absorption and emission spectrum is created. Energy level diagrams, as opposed to a planetary Bohr Model, are used to illustrate this process. After explaining the mechanism, students are asked to reason about important spectral features – such as how different colored spectral lines are created, or why each element has its own unique atomic spectrum. Together, these guiding questions are used to help students synthesize and apply their understanding of how light interacts with matter at the atomic level. Students first learn light-matter interactions in lecture. During lecture, students break into small groups to discuss the relationship between spectroscopic concepts and they answer iClicker questions related to the content. After students have attended lecture, they work through a beSocratic homework activity that is designed to help them make connections between the different spectroscopic concepts they have learned. Here, students explain various properties of atomic emission spectra and describe the mechanistic process for how an atomic emission spectrum is created. The goal of the activity is to help students synthesize their understanding of important spectroscopic concepts by having them actively apply their knowledge. This activity was refined throughout the course of this study, which will be described in more detail later in the chapter. After completing the homework activity, the instructor goes over students’ responses at the beginning of the next lecture. This is designed to address common difficulties students had, but more importantly, it is used as a platform to discuss the productive modes of reasoning students used within their explanations. 27 Finally, students attend recitation where they work in a small group on a spectroscopy problem under the guidance of a graduate teaching assistant. Refinement of the CLUE Curriculum Following the design-based research cycle illustrated in Figure 4.1, the spectroscopy portion of the CLUE general chemistry curriculum was iteratively refined from Fall 2013 to Fall 2016. These changes were designed to improve students’ understanding atomic spectroscopy. It is important to note that the curriculum changes were not planned from the onset, but instead, they were made based on observations and evidence obtained from students work throughout the course of this study. Each cohort was asked a similar, yet different constructed response question on their midterm exam where they were prompted to explain the process for how an atomic emission spectrum is created. Student responses to these assessment tasks were analyzed and the findings directly influenced the changes that were made within the CLUE curriculum for the following Fall semester. The curriculum changes are broken down into four separate phases – Student Interviews (F13), Curriculum Changes (F14), Curriculum Changes (F15), and Curriculum Changes (F16). The second half of this chapter describes the curriculum changes that were implemented from F13-F16 and the rationale for why these changes were made. To provide context for the changes that were implemented during each phase of this study, a brief description of the coding scheme and the findings will be discussed, however, a more in-depth analysis of how the coding scheme was developed (Chapter 5) and the findings that emerged (Chapter 5 & 6) are discussed in later chapters. 28 Student Interviews (F13) In Fall 2013, semi-structured interviews were conducted with students (N=12) enrolled in the first semester of general chemistry. The interviews ranged from 30-60 minutes and were conducted by a post-doctoral researcher who was not affiliated with the general chemistry courses and a graduate student who was a teaching assistant in the course. Participants were recruited on a voluntary basis and each student signed an informed consent form agreeing to participate in the research study. A Livescribe pen was used to record both audio and written work that students generated during their interviews. The interviews were designed to elicit students’ understanding of energy. The first part of the interview focused on how students understand potential energy in the context of atoms and molecules interacting. The second section of the interview was an exploratory study to investigate how students understand light-matter interactions. The interview protocol is in Appendix A. For the first part of the spectroscopy section of the interview, students were asked to describe their understanding of electromagnetic radiation. This question was designed to elicit their understanding of properties of light and the relationship between color, energy, frequency, and wavelength. This then led into a discussion about what happens when a hydrogen atom absorbs or emits electromagnetic radiation. To see how they understand the absorption process, the interviewer asked the participants to explain what happens when a hydrogen atom absorbs light. Then, to see how they understand the emission process, students were asked to explain how the atomic emission spectrum for hydrogen is created. Finally, students were asked to explain the concept of energy quantization. Overall, the goal of the spectroscopy section of the interview was to gain insight into how students understand light-matter interactions within the context of atomic spectroscopy. 29 Once complete, the interviews were transcribed verbatim and each student was assigned a pseudonym to protect their identity. The transcripts and their associated drawings were analyzed using a qualitative software program, Nvivo11. The interviews were analyzed using an inductive, open-coding approach to capture the different ways students reasoned about spectroscopy concepts. After going through this process, I found that students explained the mechanistic process for how spectral lines are created in a variety of ways. It also appeared that students who correctly explained both the absorption and emission process were able to better reason about energy quantization. Based on these observations, I worked with a post-doctoral researcher to develop a coding scheme that captured the different ways in which students’ reason about these concepts. This led to the development of a levels-based coding scheme that characterizes students’ explanations of the absorption and emission process (Table 4.1). Before describing this coding scheme, I would like to note that the “Levels of Sophistication” were disregarded as the study progressed; however, I include this initial coding scheme to provide insight into how students explanations were initially analyzed and how these results influenced the first set of curriculum changes implemented from F13-F14. 30 Table 4.1 – Coding scheme for student descriptions of absorption and emission process Level of Sophistication Description 0 1 2 *X 1a 1b • Student does not claim that an electron changes energy levels • Student claims that an electron can move up or down energy levels • Student asserts that an electron only moves to a higher or lower energy level, respectively for absorption and emission • Student states that an electron only moves to a higher or lower energy level and supports with appeal to causality • Atom absorbs or emits a specific amount of energy *X is an independent code that is applied to student responses that mention the specific amount of energy that is absorbed or emitted during these two processes. Each participant was attributed an absorption and emission level (0-2) based on the degree of sophistication in their response. If the participant did not discuss electron movement within their description of the absorption or the emission process, they were coded as L0. If the participant described general electron movement but did not state that the absorption process is when the electron transitions to a higher energy level or that the emission process is when an electron transitions to a lower energy level, they were coded as L1a. If the student described electronic transitions and made this correct relationship, they were coded as L1b. Finally, if the participant provided a correct description of the absorption or emission process and provided a more in-depth explanation, such as stating that an electron transitions to a lower energy level because it is unstable, they were coded as L2. An additional code (X) was created for when students mentioned that only specific amounts of energy can be absorbed or emitted within an atom. To test reliability of the codes, two coders independently coded absorption and emission levels for each participant. The results showed a percent agreement of 90% and 81% respectively 31 for absorption and emission. After discussing discrepancies in coding, the researchers recoded the interviews where they saw 100% agreement in their coding. The participants explained the absorption and emission processes to varying degrees, ranging from non-canonical descriptions of electrons being emitted to complex explanations that incorporated ideas of energy quantization. It is important to note that these explanations are context dependent and provide only a single snapshot into how students reason about spectroscopy concepts. The level of sophistication of the different participants explanations can be found below in Table 4.2. Verbal ticks (such as umm, err, and like) were removed from the student examples discussed below. Table 4.2 – Level of sophistication in student explanations of the absorption and emission process Participants Absorption Emission Level Level Kelsey Macayla Madelyn Kent Caelyn Shiloh Devon Ryan Octavia Yancy Irene Jill 2, X 2, X 1B, X 1B 1B 1B 1A 0 2 1A 0 0 2, X 2 2, X 2 1B 1B 1A 2 0 0 0 0 I found that students explained the mechanistic process for how an atomic emission spectrum is created in various ways. For example, when asked to explain how spectral lines are created, Irene responded by saying that “As a particle, as light hits a surface, then photons are emitted, but you have to reach a certain energy in order to even have particles being emitted, 32 otherwise, electrons will just be like shot down and nothing will happen. You have to hit a certain potential for particles to be reflected off.” It appears that Irene is conflating the photoelectric effect and the mechanistic process for how spectral lines are created. She is confused about the differences between these two spectroscopic phenomena, and as a result, is unable to offer a coherent explanation of atomic spectra. Of the twelve participants, five of the students had difficulties explaining that spectral lines are created by electrons transitioning between energy levels. Additionally, it appeared that students who had a better understanding of the mechanism for how an atomic spectrum is created were more likely to incorporate the concept of energy quantization within their explanations. For example, Madelynn explained hydrogen’s atomic emission spectrum by stating that “if it absorbed a certain amount, a certain wavelength of electromagnetic radiation, it would be promoted to a different energy level. So it would be promoted up, and then since that would be unstable, the electron would then fall back down and then emit that same, well that quantity of electromagnetic radiation as light.” She then goes on to explain why each element has its own unique spectra, “The orbitals or the electron density, the level, energy levels are different in different elements. So the amount of energy released from a sodium, like an electron falling in the sodium atom, that amount of energy is different from an electron falling in the calcium falling down in energy levels. So it emits different wavelengths.” Madelyn provided a coherent and accurate explanation of the emission process. She stated that different elements release different amounts of energy when electrons fall energy levels, however based on her response, it is difficult to know if she is thinking about the quantized amount of energy being released as being equal to the difference in energy levels that the electron transitioned between. 33 These examples highlight how students reasoning about atomic emission spectra varied in sophistication, ranging from off-topic descriptions of the photoelectric effect (Irene), to more detailed explanations for how light is absorbed and emitted within the atom (Madelynn). The participants who explained the absorption and emission process were more likely to make connections between ideas providing evidence of a more integrated understanding of spectroscopy, whereas students who had difficulties explaining this process appeared to have a rather fragmented understanding of spectroscopy concepts. These findings suggest that providing a more explicit emphasis on the process for how spectral lines are created within the general chemistry curriculum may be a promising approach for improving students’ understanding of atomic spectroscopy. Curriculum Changes (F14) Based on these interview findings, the spectroscopy section of the CLUE curriculum was refined for Fall 2014. Changes were made to instruction, homework, recitation, and summative assessments. Overall, these changes were designed to help students develop a deeper understanding of how atomic spectra are created by emphasizing the mechanistic process for how spectral lines are created. Findings from the F13 Interviews suggested that some of the students conflated their understanding of the photoelectric effect and atomic spectroscopy rather than seeing them as different spectroscopic phenomena. To address this issue in F14, a more explicit emphasis than in F13 was made during instruction on the similarities and differences between the photoelectric effect and atomic spectroscopy. During lecture, the professor had students break into small groups to discuss what was similar between these two phenomena and what was different. After 34 talking with their peers, the professor had students come together as a class and they discussed how these phenomena related to one another. The other main change focused on helping students better understand how electronic transitions give rise to spectral lines when specific amounts of energy are absorbed or emitted. Accordingly, the F13 curriculum materials were analyzed to see what materials might need to be developed or refined to help students better understand this process. In F13, the main emphasis in lecture was on the fact that atomic spectra provide evidence of the quantized nature of energy levels within the atom. To reinforce this important concept, instruction focused on the mechanistic process for how an atomic spectrum is created and the instructor discussed how different colored spectral lines are created. The F13 beSocratic homework activity pertaining to spectroscopy (Appendix B) consisted of five questions related to spectroscopic concepts. The first questions prompted students to draw and explain why atomic absorption spectra provide evidence for the existence of quantized electron energy levels in the atom. Question 2 had them explain and draw the differences between absorption and emission spectra. For Question 3, students were asked to explain why a green T-shirt appears green. Finally, Questions 4 and 5 had students calculate wavelength using velocity and mass. I found that there were few opportunities within this activity for students to explain and reason about spectroscopic concepts. Students also attended recitation once a week where they discussed concepts in small groups to reinforce the material covered in lecture. After reviewing the recitation worksheets, I found that there were no questions that asked students to explain atomic spectroscopy. The Fall 2013 midterm exam had multiple choice questions related to properties of light and a constructed response question where they were asked to describe how spectral lines are created in an 35 emission spectrum. For the constructed response question, students responded to the following assessment task: 1. The spectra above shows the emission spectra in the visible region of the electromagnetic spectrum for a number of elements. What process is responsible for the emission of these wavelengths of light? Draw a diagram to illustrate your thinking 2. Please explain why only certain wavelengths of light appear in the spectra for each element 3. Draw a picture to show what the absorption spectrum of sodium would look like? In F13, students within CLUE were taught in lecture the process for how an atomic spectrum is created and were asked to explain this process on their exam, however, there were few opportunities outside of lecture for students to synthesize and develop their own understanding of this complex phenomenon. Because of this, it was not surprising that approximately half of interview participants in F13 had difficulties explaining atomic spectra. To address this issue, a scaffolded homework activity and new recitation materials were developed for F14. The F14 homework activity (Appendix C) consisted of a series of scaffolded questions that were designed to help students better understand the mechanistic process for how an atomic emission spectrum is created. The activity is broken into three parts: 1. Students’ prior knowledge of spectroscopy concepts 2. Explaining the mechanistic process for how an atomic emission spectrum is created 3. Applying understanding of this process to explain key features of atomic spectra The first part of the activity was designed to elicit students’ prior understanding of spectroscopy concepts and to bring their thinking about light-matter interactions to the forefront. Students had 36 previously learned about atomic spectroscopy in lecture, and thus, the first few questions of the activity were designed to see how students conceptualized this process. The first question on the homework activity asked students to explain (to the best of their ability) how they think an emission spectrum is created. This complex question was asked at the beginning of the homework activity to see what they had remembered from the spectroscopy lecture. Next, they were asked to construct a model that illustrates what would happen when a hydrogen atom absorbs electromagnetic radiation. An energy diagram that illustrated the absorption process was then given to students on the next slide in the activity and they were asked to compare the model they drew to the given energy level diagram. I suspected that many of the students would illustrate the absorption process using the Bohr Model of the atom, even though the Bohr Model was explicitly de-emphasized within lecture due its problematic nature. The Bohr model of the atom is persistently used by students, and once learned (often in middle or high school), it can make future learning about a quantum model of the atom quite difficult. Instead of using the Bohr Model, which promotes the idea that electrons exist in discrete orbits and move specific distances between them when absorbing or emitting energy, I used an energy level diagram to illustrate the mechanism for how an atomic spectrum is created. By asking students to compare their model (which was expected be the Bohr Model) to an energy level diagram, I wanted to make the similarities and differences between the two models more explicit to students. After students were shown an energy level diagram that illustrates the absorption process, they were asked to explain why they think that the electron (as shown in the energy level diagram) moves to a higher energy level. To see how students were thinking about the role energy plays in this process, the next question asked students to explain how much energy would 37 need to be transferred to the electron to excite the electron to a higher energy level for a transition from n=1 to n=3. These questions were designed to help students understand that electronic transitions only occur when a specific amount of energy is absorbed or emitted and that that amount of energy which is equal to the difference between energy levels within the atom. The next two questions on the homework activity related to the emission process. Students were asked to draw a representation that shows the emission process and to explain what is happening during the emission process. Then, to see how they related the emission process to the color of the emitted light, they were asked to explain how different colors are emitted. Together, these questions probed students’ understanding of the emission process and how different colored spectral lines are created. The final three questions asked students to apply the concepts they had been learning about to explain important features of atomic spectra. First, students were asked to explain how a blue line in the emission spectrum is created. To explain this phenomenon, students need to understand how different colored spectral lines are created during the absorption and emission process (i.e. how electronic transitions relate to properties of light). The second question asked students to explain why only specific lines show up in hydrogen’s atomic emission spectrum (i.e. how electronic transitions relate to energy quantization). The final question on the homework activity asked students to explain why the atomic emission spectrum of hydrogen is different than the atomic emission spectrum of a sodium atom. To answer this question, students need to integrate their understanding of electronic transitions, properties of light, and energy quantization. 38 To supplement this homework activity, a new recitation worksheet was designed for F14 (Appendix D). The atomic spectroscopy question on the recitation worksheet prompted students to explain the process for how red and blue lines are created within hydrogen’s atomic emission spectrum. Working in small groups of three or four, students discussed the similarities and differences for how different colored spectral lines are created. A graduate teaching assistant provided guidance and support to students as the engaged with this activity. Finally, the spectroscopy summative assessment task on the F14 midterm exam was refined. The new assessment task prompted students to explain the process for how different colored spectral lines are created. Recall, the F13 assessment prompted students to state the process for how spectral lines are created and to explain why each element has a unique emission spectrum. Based on the findings from the F13 interviews, I wanted to simplify the F14 question prompt to better target students’ understanding of the mechanistic process for how spectral lines are created. The F13 assessment task prompted students to state the emission process, not to explain it. Thus, I wanted to see if asking students to describe the process for how red and violet spectral lines are created would result in more detailed explanations of the absorption and emission process compared to the F13 assessment task. After making changes to instruction, recitation, and assessments (formative and summative) based on evidence from students’ interviews, I wanted to see what effect the implemented curriculum changes from F13-F14 had on students’ reasoning about atomic spectra. To do this, I selected a random sample of students from each cohort and analyzed their explanations from their exams. I began by applying the coding scheme developed during the F13 interviews, but quickly found that it did not adequately capture students’ reasoning. 39 To make sense of students’ explanations on these two different assessment tasks, I open- coded student responses from both the F13 and F14 cohort. To identify common themes in students’ reasoning, I drew upon an existing assessment rubric called the Knowledge Integration Rubric (KIR) to characterize the main ideas and links students made within their responses (Liu, Lee, Hofstetter, & Linn, 2008). The development of this coding scheme, as well as the findings that are presented below, are discussed in more detail in Chapters 5 and 6. Once developed, the KIR was used to characterize students’ reasoning on the F13 and F14 exams. The findings showed that the F13-F14 curriculum changes improved the number of students who were able to explain the mechanistic process for how spectral lines are created. Similarly, there was a decrease in the number of students who conflated the photoelectric effect with atomic spectroscopy. Together, these findings suggested that the curriculum changes implemented from F13-F14 improved students’ understanding of the mechanistic process for different colored spectral lines are created within an atomic emission spectrum. While there was an improvement in the number of students who applied their understanding of electronic transitions to explain different properties of light, many of the students in F14 still had difficulty linking together their understanding of these two concepts. These findings will be discussed in more detail in Chapters 5 and 6. Furthermore, only a small number of students in both cohorts incorporated the concept of energy quantization within their explanations, suggesting that the implemented curriculum changes did not improve students’ understanding of this concept. Based on these results, a second iteration of curriculum changes were implemented within the spectroscopy section of the CLUE curriculum for Fall 2015. 40 Curriculum Changes (F15) Based on these findings, the F14 curriculum materials were reviewed to see how they could be further refined and improved upon for the F15 semester to address the difficulties students had in understanding energy quantization. After reviewing the F14 materials, I found that students had few opportunities in the curriculum to reason about energy quantization and that the main emphasis of the F14 curriculum materials was on the relationship between electronic transitions and properties of light (as it had been designed based on the F13 interview findings). To address this issue, a series of changes were implemented in F15 aimed at helping students understand that the specific amount energy absorbed or emitted by an atom is equal to the difference in energy levels that the electron transitions between. This concept will be referred to as the energy difference idea. I began by refining the F14 homework activity to specifically target students’ understanding of the energy difference idea. In total, four new questions were added, and three questions were removed from the F14 activity. The complete F15 homework activity can be found in Appendix E. The new questions that focus on energy can be found below in Figure 4.2. 41 Figure 4.2 – Energy difference questions added to the F15 homework activity For the first question, students were given an energy diagram that showed an electron transitioning to an excited state (from E1 to E3), and they were asked to explain how much energy would need to be absorbed to cause this transition. This question was designed to see if students think that any amount of energy can be absorbed, or if they think only specific amounts of energy can be absorbed. The next slide told students that only specific amounts of energy can be absorbed and emitted by an atom and that this amount of energy is equal to the difference between energy levels. To see if they understood this concept, they were given an energy level diagram that had five discrete energy levels with an electron in the ground state (E1). They were then asked a 42 multiple-choice question where they predicted the number of possible electronic transitions that could occur if an electron in the ground state moved to an excited state. Then, they were prompted to explain how much energy would need to be absorbed for the electron in E1 to enter an excited state of E4. These two questions focused on how students understand the energy difference idea in the context of absorption. The third slide focused on the relationship between the emission process and the energy difference idea. Students were prompted to explain how much energy is released if an electron transitions from an excited state (E3) to the ground state (E1) and emits a photon. The goal of this question was to have students recognize that the amount of energy released during the emission process is equal to the difference between the two energy levels (E3-E1). The final question was designed to address two common ways students may incorrectly understand the energy difference idea. During the analysis of the F13-F14 curriculum changes, I observed that some of the students in the F14 cohort related the color and energy of the emitted photon to the relative height of the energy level, for example, by stating that higher energy photons are emitted when an electron moves from a higher energy level. In a similar fashion, I saw that some students incorrectly conceptualized the energy difference idea by thinking that the relative amount of energy released corresponds to the number of energy levels the electron transitioned between. For example, an electron that transitions between n=6 to n=4 would release more energy than an electron that transitions between n=2 to n=1 because the initial electron moved more energy levels (two versus one). To see how pervasive these ideas might be for students, the final question focusing on the energy difference idea was specifically designed to address these two incorrect ideas. Slide 4 has an energy level diagram with two electronic transitions. One of the transitions showed an 43 electron moving from n=6 to n=2 and the other transition showed an electron moving from n=4 to n=1. Using this model, students were asked to explain which transition they think would release a higher energy photon. If students had an incorrect understanding of the energy difference idea, they would select the n=6 to n=2 transition because the electron started in a higher energy level (n=6 vs. n=4) and it moved more energy levels compared to the n=4 to n=1 transition (4 vs. 3). Overall, four new slides were implemented within the homework activity, and they were designed to help students better understand that only discrete amounts of energy (equal to the difference between energy levels) can be absorbed or emitted from an atom. After refining the homework activity, I analyzed the F14 instructional materials to see if they could be further improved upon. I found that the primary lecture on spectroscopy explicitly covered the energy difference idea, and therefore, no changes were made to the primary lecture on spectroscopy. However, I did find that the energy difference idea was not emphasized in the subsequent lecture where the professor went over student’s homework. The F14 homework activity did not have any questions that explicitly focused on energy quantization, and thus, it was not surprising that coverage of this concept during homework review was minimal. And so, for F15, the instructor explicitly discussed the various ways students conceptualized the energy difference idea in their homework. To facilitate this discussion, the professor showed students examples of both good and bad explanations and asked the students to discuss with their neighbors how the explanations differed from one another. The final change that was made in F15 was altering the wording of the summative assessment task on students’ exam. Recall, in F14 students were asked to describe the process for how red and violet lines are created. The wording of this question explicitly prompted students to explain the mechanism for how an atomic emission spectrum is created, but only implicitly 44 prompted them to relate their understanding of this process to reason about how different colored spectral lines are created. To address the implicit nature of this question prompt, I broke the question prompt into four separate questions to see if it would improve the relationships students make between electronic transitions, properties of light, and energy quantization. The F15 summative assessment task is shown below: 1. Construct energy diagrams that illustrate the processes that cause the emission of the red line and the blue line in the spectrum. Make sure to label all the components within your diagram. (two boxes for students to draw in, one labeled “Process that causes the red line” and other labeled “Process that causes the blue line”) 2. Using your diagrams, describe in words the processes that cause the emission of a line in an emission spectrum 3. What is similar between the processes that cause the red and blue line? 4. What is different between the processes that cause the red and blue line The first question had students construct an energy diagram to show how the red and blue lines in the atomic emission spectrum are created. Then, they were prompted to use their diagram to describe the process how for how an emission line is created. Once they described this process, they explained what is similar (Q3) and different (Q4) in how different colored spectral lines are created. Both the F14 and F15 assessment tasks are similar because they prompted students to reason about different colored spectral lines, however, the F15 assessment task was broken into separate parts. I wanted to see if a more scaffolded question prompt, in addition to the other curriculum changes that were implemented in F15, would increase the number of students who reasoned about energy quantization within their explanations. 45 Overall, the changes to instruction and assessments in F15 were designed to improve students’ understanding of the energy difference idea and how it relates to other spectroscopic concepts. These changes were made based on observed difficulties that the F13 and F14 cohorts had in reasoning about atomic emission spectra. To see if this second iteration of curriculum changes had any effect on students’ reasoning, a random sample of students from the F15 cohort were selected and their explanations were analyzed using the KIR rubric that had previously been developed in F14. At this point in the study, I found that the KIR did not adequately capture the nuanced ways in which students reasoned about atomic emission spectra. This led to the development of a new coding scheme called the Spectroscopy Reasoning Rubric (SRR), which is based on the KIR. The development of the SRR is discussed in Chapter 5. Once developed, the SRR was used to characterize students’ explanations on the F13, F14, and F15 exams. From this analysis, I found that both the F14 and F15 cohorts had similar reasoning within their explanations, which suggested that the F14-F15 curriculum changes did not improve students’ reasoning. Recall, multiple changes were made within the CLUE curriculum from F14-F15. The homework activity was refined to provide a more explicit emphasis on the relationship between energy quantization, electronic transitions, and properties of light. The energy difference idea was explicitly discussed in lecture after students completed their homework activity, which had not previously been done in F14. Finally, the summative assessment was broken into four separate parts and students were prompted to describe the similarities and differences for how different colored spectral lines are created. It may be that the F14-F15 changes to instruction and homework helped students better integrate their knowledge of spectroscopy concepts, but because the F15 assessment task did not 46 adequately prompt students to reason in a more sophisticated way, there was no observable changes in students’ reasoning. To test this hypothesis, only the summative assessment task was changed in F16. Curriculum Changes (F16) A variety of studies being conducted in the Cooper Research Group, including this study, have indicated that the nature of the assessment task directly influences students’ reasoning (Cooper et al., 2016; Crandell et al., 2018). Consequently, this means that the different assessment tasks used in F13, F14, and F15 directly influenced the ways in which students reasoned about atomic emission spectra. The main changes to instruction, homework, and recitation occurred from F13-F14. However, the F14 summative assessment task was less explicit than the F13 question prompt. These conditions (main curriculum changes and less explicit question prompt) led to improvements in students’ reasoning from F13-F14. From F14-F15, minor changes were made within instruction and homework to emphasize the energy difference idea. The F15 assessment task was more detailed than the F14 assessment task because it had students explain the process for how spectral lines are created, as well as explain the similarities and differences between different colored spectral lines. These conditions (minor curriculum changes and compare/contrast question prompt) did not lead to any improvements in students’ reasoning. For F15-F16, I wanted to see if the F14-F15 curriculum changes coupled with a more explicit question prompt would better elicit students’ understanding of atomic emission spectroscopy. The F16 summative assessment task consisted of four separate questions: 47 1. Draw an energy diagram that illustrates the process for how one spectral line is created in an atomic emission spectrum. Make sure to label all the components within your diagram. 2. Describe what happens at the atomic level that causes the emission lines to be created. 3. Explain how this process produces different colored lines. 4. Explain why only certain wavelengths of light appear in the spectra for each element. The first question prompted students to construct an energy diagram to illustrate the process for how a spectral line is created. Question 2 prompted students to describe the mechanistic process for how electronic transitions give rise to atomic spectra. Question 3 asked students to apply their understanding of the emission process to explain how different colored spectral lines are created. Finally, the last question asked students to explain why each element has its own unique atomic emission spectrum. Each of these questions were incorporated (either implicitly or explicitly) within the F13, F14, and F15 summative assessment tasks, however, the F16 task was designed to make the relationship between these different questions more explicit. Based on the curriculum changes that were made to instruction, recitation, and homework from F13-F15, I wanted to see if a more scaffolded assessment task would prompt students to provide more detailed explanations. Once the F16 assessment task was implemented, a random sample of students in the F16 cohort were selected and their explanations on their exams were analyzed used the Spectroscopy Reasoning Rubric. Findings showed that the changes to the summative assessment task in F15- F16 significantly improved students’ reasoning, thus providing evidence that the nature of the question prompt directly influenced the sophistication of students’ reasoning. This finding, along 48 with a more in-depth analysis of the effect that the F13-F16 curriculum changes had on students’ reasoning, is discussed in Chapter 6. Summary of Curriculum Changes Following an iterative design-based research cycle, the spectroscopy section of the CLUE curriculum was refined from F13-F16. During each phase of the study, students’ understanding of spectroscopy concepts was assessed and these findings directly informed the curriculum changes that were implemented the following fall semester. These evidence-based changes were designed to improve students’ understanding of atomic spectroscopy and the relationship between three overarching spectroscopy concepts – electronic transitions, properties of light, and energy quantization. The first set of curriculum changes (F13-F14) were based on interviews that were conducted in Fall of 2013. Findings from the interviews showed that general chemistry students explained the mechanistic process for how an atomic spectrum is created in a variety of different ways. To address this finding, the F13-F14 curriculum changes focused on the mechanistic process for how an atomic emission spectrum is created. This led to the development of a new homework activity which gave students opportunities to actively apply and synthesize their knowledge of spectroscopy concepts. To see if students gained a better understanding of this process, the F14 assessment task was designed to explicitly target students’ understanding of this construct. The F13-F14 curriculum changes improved students’ reasoning about electronic transitions and properties of light, but there were minimal improvements in students’ understanding of energy quantization. This finding influenced the F14-F15 curriculum changes, where instruction and the homework activity were refined. Four additional slides were added to 49 the homework activity to emphasize the role energy quantization plays in the creation of an atomic emission spectrum. The F15 assessment task was broken into separate questions to highlight the similarities and differences in how different colored spectral lines are created. Overall, I found that the changes made in F14-F15 did not improve students’ reasoning. From F15-F16, no changes were made to instruction, recitation, or the homework activity. However, a more explicit summative assessment task was designed to see if it would prompt students to provide a more detailed explanation of atomic emission spectra. Indeed, this change to the question prompt improved students’ reasoning from F15-F16. Overall, three iterations of curriculum changes were implemented from F13-F16, resulting in four separate cohorts of students who were enrolled in the CLUE general chemistry curriculum at various stages of refinement. Throughout this process, a coding scheme (the SRR) was developed to characterize students’ reasoning about atomic emission spectra. Chapter 5 describes the development of the SRR and discusses the different modes of reasoning students employed within their explanations of atomic emission spectra. After discussing the development of the SRR, Chapter 6 presents findings for how each iteration of curriculum changes (F13-F14, F14-F15, and F15-F16) influenced students’ reasoning. 50 CHARACTERIZATION OF STUDENTS’ REASONING ABOUT ATOMIC EMISSION SPECTRA Preface This chapter describes the development of the Spectroscopy Reasoning Rubric and discusses findings from my research into how students’ reason about atomic emission spectra. The research presented in this chapter has been submitted to the Journal of Chemical Education and is currently under review. This chapter is “reproduced with permission from the Journal of Chemical Education, submitted for publication. Unpublished work copyright (2019) American Chemical Society.” A copy of the Journal of Chemical Education publishing agreement can be found in Appendix F. Introduction Spectroscopy is an indispensable tool that scientists use to probe the atomic and molecular structure of matter. Understanding how light interacts with matter is central to not only learning chemistry, but it is a foundational concept across scientific disciplines (National Research Council, 2012). Typically, in chemistry, one of the first spectroscopic phenomena students learn about is atomic absorption and emission spectroscopy. Atomic spectroscopy provides evidence of the quantized nature of energy levels within the atom and can be used to introduce students to quantum models of the atom. To understand and explain atomic spectra, students need to develop an understanding of several spectroscopic concepts, including properties of light, energy quantization, and atomic and molecular structure. Furthermore, students need to make relevant connections between these concepts in order to explain important features of atomic spectra. 51 However, developing a deep understanding of how light interacts with matter at the atomic and molecular level can be difficult for students due to the complex and unintuitive nature of the quantum world (Johnston et al., 1998; Singh, 2001; Tsaparlis, 2016). This is evidenced by the fact that students have difficulties developing a conceptual understanding of spectroscopy concepts (Didiş et al., 2014; Didiş Körhasan & Wang, 2016; Ivanjek et al., 2014; Park & Light, 2009; Savall-Alemany et al., 2016). Prior studies have shown that students have a wide array of non-canonical ideas about atomic spectroscopy, such as the view that energy levels within an atom correspond to the different colors within an atomic emission spectrum (Ivanjek et al., 2014). While these studies provide insight into the specific difficulties students have in learning about light-matter interactions, there is a need for more research that identifies the productive ways students reason about and conceptualize spectroscopic concepts. It is our contention that if we can identify the productive resources students use to make sense of spectroscopic phenomena, we can gain a deeper understanding of what students know and can do with their knowledge. Once we have a better understanding of the productive ways students’ reason about light-matter interactions, we will be better equipped to address the difficulties students face in learning spectroscopy. Thus, the goal of this study was to characterize the ways general chemistry students’ reason about and explain atomic emission spectra. Specifically, we wanted to characterize the connections students make between spectroscopic concepts when they are asked to explain how an atomic emission spectrum is created. This study was guided by the following research question – How do general chemistry students’ reason about atomic emission spectra? 52 Theoretical Frameworks In general, there are two approaches to thinking about how students undergo conceptual change (or learning). One approach is that students may have incorrect ideas, theories, or mental models that lead to misconceptions that must be “corrected” or overridden in some way by appropriate instruction (Posner et al., 1982). Another, and we believe more fruitful, approach for most complex ideas, is to consider learners’ knowledge as fragmentary – the so called “knowledge in pieces” theory (A. A. diSessa, 1993; A.A. diSessa, 2006). In this framework, students’ knowledge is believed to consist of productive and unproductive mental resources that may be activated in response to a particular prompt or context (Hammer, 2000). The work described in this report is grounded in two theoretical frameworks. The first is the Resources perspective (Hammer, 2000; Hammer & Elby, 2003), which has emerged from the knowledge in pieces perspective of learning: it describes the cognitive resources students use and develop throughout their learning experiences. However, because even productive resources are not particularly useful without being connected, we also use the Knowledge Integration perspective of learning (Linn, 2006; Linn & Eylon, 2011), which describes the process by which students integrate knowledge into their existing conceptual framework. The Resources perspective of learning views students’ knowledge as resources which may or may not be activated within a given context and may or may not be fruitful for making sense of a phenomenon (Hammer, 2000; Hammer & Elby, 2003). These resources are not constant, but continually changing as students learn new content and build upon their existing conceptual framework. As a result, learners can develop clusters of resources by understanding the relationship between different concepts (Hammer & Elby, 2003). Both teaching and research from this perspective should aim to identify and support student’s development of productive 53 resources (Hammer & Elby, 2003). For example, a study by Becker et al. (2017) characterized the resources students use to reason about chemical kinetics. While identifying productive resources is important, those resources must also be woven and connected if students are to be able to provide coherent and scientifically appropriate explanations for phenomena. One approach to helping students integrate their knowledge and develop coherent explanations is by viewing learning as a process of knowledge integration. The Knowledge Integration perspective of learning describes the process a learner goes through as they explain and make sense of scientific phenomena (Linn, 2006). Every person has prior knowledge in which new knowledge is built upon. Thus, as students engage in the learning process, they are continually tasked with linking new concepts and ideas to their existing ideas (Linn, 2006; Linn & Eylon, 2011). For students to have meaningful learning experiences, it is important that students not only accumulate new knowledge, but that they readily link and integrate their knowledge (National Research Council, 2018). Knowledge integration, when viewed from the resource’s framework, can be thought of as a process of making connections between ideas to develop useful resources. Together, these two theoretical frameworks have direct implications on how one approaches teaching and learning. This sentiment is echoed within the work of Hammer and Elby (2003), where they stated that “teachers who ascribe coherent epistemological beliefs to their students will incline toward instructional strategies that differ from those of teachers who ascribe context-sensitive resources.” Similarly, by viewing student’s knowledge as resources that may or may not be activated within a given context, it shifts our perspective and approach on how to address the difficulties students face in learning spectroscopy. 54 As highlighted in Chapter 3, research on students’ understanding of light-matter interactions has primarily focused on identifying the incorrect ideas students have about spectroscopy concepts and characterizing their mental models. However, by viewing students’ understanding through the lens of the resources/knowledge-integration frameworks, we shift the emphasis away from what students do not know, and their apparent mental models about spectroscopy, and instead, focus on what students know and can do with their knowledge. Thus, we believe that students’ understanding is highly dependent on the context and the way in which they are being prompted, as opposed to what their existing mental model of atomic spectroscopy may appear to be. This dichotomy can also be viewed as a deficit versus a capability model of learning. Depending on which view one takes, it has direct implications on how educational initiatives are conceptualized and implemented. For example, Ivanjek et al. (2015) designed an intervention to elicit students’ wrong ideas about spectroscopy concepts so that “students recognize the conflict between their initial ideas … and come to understand the reasoning required for resolution.” However, a different intervention developed by Moon et al. (2018) engaged students in a writing-to-learn activity in which they were prompted to explain how the chemical composition of a celestial body is determined, a task which required students to integrate their knowledge of multiple spectroscopic concepts and make connections between them. While both studies aimed to improve students’ understanding of spectroscopic concepts, they took vastly different approaches, which we would argue, are heavily influenced by their underlying perspectives of learning. In this work, we view students’ understanding of spectroscopy through a capability model of learning in which students have a variety of resources that they can draw upon when 55 reasoning about spectroscopic phenomena. Based on this perspective, the primary goal of this study was to characterize the different ways in which students explain atomic emission spectra. Participants and Setting Methods The participants in this study consisted of a random subsample of students (N=576) who were enrolled within the first semester of general chemistry at a Large Midwestern Research Intensive University. Approximately 43% of the participants identified as Male and 57% identified as Female. These students had a mean ACT score of 26 and about a quarter of these students (26%) were First Generation College Students. About two-thirds of the participants identified as white (68%), while the next most commonly reported ethnicities were International Student (10%), Asian (7%), African American (6%), and Hispanic (4%). This study took place within the context of a transformed general chemistry curriculum: Chemistry, Life, the Universe, and Everything (CLUE) (Cooper & Klymkowsky, 2013, 2015). The CLUE curriculum is based on an interconnected set of learning progressions around four core ideas – atomic/molecular structure and properties, electrostatic and bonding interactions, energy, and change/stability in chemical systems (Cooper et al., 2017). Students engage with these ideas in the context of the scientific and engineering practices, such as constructing explanations of phenomena or engaging in argumentation (National Research Council, 2012). By emphasizing these practices, there is a focus not only on concepts and ideas, but also on the use of that knowledge. Within the CLUE curriculum, spectroscopy is taught near the beginning of the first semester to introduce students to quantum models of atomic structure. Instruction is situated around phenomena that provide evidence about the nature of light and its interactions with 56 matter. The double-slit experiment provides evidence that light can act as a wave, the photoelectric effect provides evidence that light acts a particle, and atomic spectroscopy provides evidence about the quantized nature of electron energy levels within the atom. To help students understand and explain atomic absorption and emission spectroscopy, students learn the mechanism by which an atomic absorption or emission spectrum is created. The relationships between energy, electronic transitions, and properties of light are used to explain how discrete spectral lines are created within atomic absorption and emission spectra. The primary emphasis of this unit is on the quantized nature of electromagnetic radiation, electronic energy levels, and the relationship to atomic structure. The data collected for this study comes from four separate cohorts of students who were enrolled within the CLUE general chemistry curriculum from Fall 2013 to Fall 2016. At the beginning of each semester, students gave informed consent for their work to be included in this study. Each cohort was asked a constructed response question on their midterm exam in which they were prompted to explain the process for how different colored spectral lines are created within an atomic emission spectrum. In total, four separate question prompts were administered (Table 5.1). 57 Table 5.1 – Explanation assessment tasks (F13-F16) Cohort Exam Explanation Prompt Fall 2013 Fall 2014 Fall 2015 State the Process & Explain • What process is responsible for the emission of the wavelengths of light in the spectrum? • Explain why only certain wavelengths of light appear in the spectra for each element Describe Process • Describe the process that is responsible for the emission of the red line and violet line in the spectrum Describe Process & Compare/Contrast • Describe in words the processes that cause the emission of a line in an emission spectrum • What is similar between the processes that cause the red and blue line? • What is different between the processes that cause the red and blue line? Describe Process & Explain (x2) • Describe what happens at the atomic level that causes the emission lines to be created Fall 2016 • Explain how this process produces different colored lines • Explain why only certain wavelengths of light appear in the spectra for each element Students’ responses to these assessment tasks were analyzed and used as the primary data source for this study. This chapter highlights the common ways in which students reasoned about atomic emission spectra based on their responses to these similar, yet different assessment tasks. For more information on the effect that these different assessment tasks had on students’ reasoning (in conjunction with the curriculum changes that were implemented), please see Chapter 6. 58 Data Analysis We began our analysis by open-coding student explanations (Corbin & Strauss, 2008) to identify common themes in student reasoning. However, due to the complexity of the phenomenon, organizing the different ideas into overarching categories proved to be quite difficult. Ultimately, we drew upon an existing assessment rubric called the Knowledge Integration Rubric (KIR) (Liu et al., 2008) to characterize students’ reasoning about atomic emission spectra. The KIR was designed to capture the increasingly sophisticated ways in which students link and integrate their knowledge, and because of this, we found it to be a useful tool in analyzing students’ explanations. Knowledge Integration Rubric The KIR is an assessment rubric that measures the extent to which students make meaningful connections between ideas when explaining a scientific phenomenon (Liu et al., 2008). It ranks, into five different levels, the relative sophistication of a student’s explanation based on number of normative ideas and scientifically valid links within their response. The levels and their descriptions can be found below in Table 5.2. 59 Table 5.2 – Knowledge integration rubric and item examples Knowledge Integration Categories Description Score Response Characteristics Systemic link Students have a systemic understanding of science concepts Not Applied Complex link Students understand how more than two science concepts interact in a given context Full link Students understand how two scientific concepts interact in a given context Partial link No link Off task No answer Students recognize potential connections between concepts but cannot elaborate the nature of the connections specific to a given context Students have non-normative ideas and/or make scientifically invalid links in a given context Students make statements about non- scientific situations No response *Adapted from Liu et al. (2008) 5 4 3 2 1 0 Compare similarities and differences between contexts, and apply concepts relevant to each context Elaborate two or more scientifically valid links among ideas relevant to a given context Elaborate a scientifically valid link between two ideas relevant to a given context Indicate a link between relevant ideas but do not fully elaborate the link in a given context Make links between relevant and irrelevant ideas Have non-normative ideas Blank The levels within the KIR reflect the variability in how students reason about multiple scientific concepts when engaging in tasks that require them to synthesize their knowledge. For example, students who describe non-scientific ideas would be coded as Off Task (L1). If students have non-normative ideas about science or make invalid links between ideas within their response, they would be classified as No Link (L2). The next category, Partial Link (L3), is when students describe relevant scientific ideas, but do not explicitly describe how these ideas relate to one another. Finally, if students make an explicit connection between two scientific concepts they would be categorized as a Full Link (L4), and if they make multiple connections between concepts, they would be classified as a Complex Link (L5). The KIR provides insight into how students’ reasoning varies in complexity, and it captures the increasingly sophisticated ways in which students link and integrate their knowledge. 60 Liu et. al. (2008) proposed a four-step process for how to develop a knowledge integration scoring scheme: 1. Identify the different ideas that are present within a student’s response and decide if these ideas are relevant to the scientific phenomenon 2. Identify potential links students make between these ideas 3. Examine each link to see if its scientifically valid and fully elaborated 4. Determine the overall linking structure of a response After the main ideas and links have been identified for a given construct, one can then use the KIR to rank the relative sophistication of an explanation based on the linking structure. Main Ideas and Links Coding Scheme To identify the main ideas and links within students’ explanations of atomic emission spectra, we began by reviewing the previous open-coding to identify the main ideas that students commonly discussed in their responses. By re-analyzing this data through the lens of the KIF, we found that students primarily discussed three main topics– electronic transitions, properties of light, and energy quantization. However, students reasoned about these concepts and made connections between them in a variety of ways. For example, Mary explained that “the wavelength of the light being emitted changes, which causes a change in color” whereas another student, Jameson, talked about both electronic transitions and properties of light within his response by stating that “when energy is released from the atom, electrons drop to lower energy levels and photons are released. Violet will appear when more energy is released and the electrons drop to much lower energy levels. Red will appear on an emission spectrum when not as much energy is released and the electrons drop fewer energy levels.” In both explanations, each student described a property of light, 61 however, the extent to which they applied their understanding of this concept to explain the given phenomenon varied considerably. The first explanation merely stated the relationship between wavelength and color, whereas the second student explained how different colored spectral lines are related to the energy of the emitted photon. To characterize the productive ways students reasoned about properties of light, electronic transitions, and energy-quantization within their explanations, we drew upon Evidence-Centered Design (ECD) (Mislevy, Almond, & Lukas, 2003). ECD is an approach to developing assessment tasks in which assessment is viewed as an evidentiary argument (Mislevy & Haertel, 2006). That is, assessments should be developed so that they have the potential to elicit evidence about what students know and can do. Using this evidence, the researcher is able to make an argument about student understanding. The principles of ECD allow us to design assessments that have such potential. For example, if we want to know if students understand that light has a dual nature and can behave as a wave or a particle, asking students to calculate the energy of a particular wavelength would not elicit such evidence. Using ECD allows us to identify what students should know and what will be accepted as evidence that students understand a desired construct. While ECD is traditionally used in assessment development, we used ECD as a lens to (1) identify what we want students to know and do with their knowledge within the context of explaining atomic emission spectra, and to (2) identify what will serve as evidence that students understand these constructs. For example, we want students to know that a spectral line within an atomic emission spectrum is created by the process of an electron transitioning to a lower energy level and emitting a photon whose energy corresponds to the wavelength of light seen within the spectrum. This explanation requires students to connect their understanding of electronic 62 transitions and properties of light; thus, we looked for evidence within students’ explanations that they are making this relationship. For example, Shelby stated that “the emission of a line is caused by the electron moving from a higher energy level to a lower energy level that emits a particular wavelength.” This response provides evidence that she is making a connection between her understanding of electronic transitions and properties of light to describe how spectral lines are created. Furthermore, we want students to know that the amount of energy emitted during this process is equal to the difference in energy levels that the electron transitioned between. To explain this concept, students need to integrate their understanding of electronic transitions and energy quantization. Thus, a response should contain evidence to show the student understands that only a specific amount of energy can be emitted that is equal to the difference in energy levels. For example, Jackson stated that “An excited electron emitted a photon of light equal to the differences between the two levels (i.e. n=3 and n=1)”. This response provides evidence that he is thinking about energy quantization within the context of electronic transitions. After identifying what we want students to know and do with their knowledge (as well as the evidence of their understanding), we re-analyzed their explanations to identify the productive ways in which students reasoned about electronic transitions, properties of light, and energy quantization. Ultimately, we identified three main ideas and three links that students reasoned about within their explanations Table 5.3. 63 Table 5.3 – Main ideas and links coding scheme Code Description Example Absorption Process Electron absorbs a photon and transitions to a higher energy level electron, the electron gains enough energy to move to another energy “When a photon is fired at an Emission Process Electron transitions to a lower energy level and emits a photon Main Ideas Partial Emission Process* Nonspecific or general description of electronic transitions Color/Energy The energy/wavelength/frequency of a photon determines its color Absorption – Emission of the absorption and emission Student connects their description processes Links Emission – Color/Energy Emission – Energy Difference Student relates the color and relative energy of spectral lines to the energy of the photon that was released during the emission process Student explains that the amount of energy released during the emission process is equal to the difference in energy levels the electron transitioned between level.” “The emission of the photon of light is due to electrons moving down from higher energy levels to lower energy levels.” “Each element requires different wavelengths or energy for the electrons to move levels.” “The higher energy will result in violet and a lower energy will result in red.” “Light is absorbed and an electron moves up in energy level. The electron wants to return to its previous energy level. So the electron jumps back down to it original level and release a photon light.” “In order for photons to be emitted electrons must drop down an orbital (or multiple orbitals). A specific amount of energy will be released when photons are emitted and that amount of energy is directly related to the wavelength/colors emitted” “An electron is moving from a high energy level to a low energy level and emits a photon When an electrons moves from a high energy to a lower one, the difference in these energy changes results in a different frequencies of the photons emitted.” *Code for partial emission process was added during the refinement of the KIR We found that some students reasoned about electronic transitions by explaining the absorption and emission processes separately. For example, Ryan described the absorption process by stating that “when a photon is fired at an electron, the electron gains enough energy to move to another energy level.” While Ryan invoked the idea of movement within his response (as opposed to it being a transition), we believe that this response provided evidence that he was thinking about the absorption process in a productive way. To explain the emission process, 64 Heather stated that “electrons are moving down from a higher energy level to a lower energy level. Therefore photons are being emitted (producing light) causing an emissions spectrum.” Like Ryan’s explanation of the absorption process, Heather invoked the idea of electron movement within her response, but even so, she was able to apply her understanding of electronic transitions to explain the process for how an atomic emission spectrum is created. Some of the participants linked their understanding of the absorption and emission process. For example, Elena connected these two ideas together by stating that “light is absorbed and an electron moves up in energy level. The electron wants to return to its previous energy level. So the electron jumps back down to it original level and release a photon light.” Elena described both the absorption and emission process to provide a more detailed explanation for how spectral lines are created. This explanation differs from the previous two examples in which the students discussed the absorption or emission process in isolation. In addition to reasoning about electronic transitions, the participants also discussed various properties of light, such as the relationship between color, energy, frequency, and/or wavelength. For example, Anar described the relationship between color and energy by stating that “the higher energy will result in violet and a lower energy will result in red.” While this relationship is important and relevant to the phenomenon, it does not explain how different colored spectral lines are created. This type of explanation provides evidence that a student understands the relationship between color and energy, however, it does not provide evidence that they are able to relate this understanding to the underlying mechanism by which spectra are created. Conversely, some of the participants linked their understanding of color/energy relationships to electronic transitions. For instance, Maggie explained that “the process that is 65 responsible for the emission of the red and violet colors is when an electron moves from a high energy level to a lower level causing energy to be released in the shape of emitting light. Different colors were emitted because each emitted amount of color corresponds to a certain wavelength and amount of energy, therefore different colors were emitted due to the energy released by each electron.” Maggie began her explanation by stating that the red and violet lines within the spectrum are caused by a photon being emitted during the emission process. She then went on to explain that the wavelength and energy of the emitted photon determines the color seen on the spectrum. Not only did Maggie describe the emission process and the relationship between color and energy, but she was able to link these two ideas together. The final spectroscopy concept that students reasoned about was the Energy Difference link, which is an important concept related to the quantized nature of energy levels within the atom. Because electrons exist in quantized energy levels, this means that an electron can only absorb specific amounts of energy that are equal to the difference in energy levels. Upon absorbing or emitting this specific amount of energy, the electron will either transition to a higher energy level (Absorption) or transition to a lower energy (Emission). This makes the Energy Difference link unique compared to the other concepts discussed thus far because it is inextricably connected to how students understand the process by which electronic transitions occur. For example, Matt explained that “the line straight down is an electron falling from a higher energy level to a lower one, causing an emission of a photon of light equal to the difference in energy levels.” Matt explained the emission process by stating that an electron falls from a higher energy and emits a photon. He then connected his understanding of this process to explain that the energy of the emitted photon is determined by the difference in energy levels that 66 the electron transitioned between. This example provides evidence that Matt linked his understanding of the emission process and the quantized amount of energy that is emitted when an electron transitions to a lower energy level. After we identified the main ideas and links that were present within students’ explanations of atomic emission spectra, we used the Knowledge Integration Rubric (Table 5.2) to characterize students’ explanations based on the linking structure of their response. Students who had two or three links within their explanation were coded as a Complex Link (L5), and those that had only a single link were classified as Full Link (L4). If a student described one or more of the four main ideas, but did not make a link between them, they were categorized as a Partial Link (L3). Students who described non-normative ideas and/or made incorrect links between ideas were coded as No Link (L2). And finally, students that discussed unrelated content were listed as Off-Topic (L1). Refinement of the Knowledge Integration Rubric Once students’ explanations were grouped using the KIR, it became clear that some of the levels within the KIR needed to be refined to better characterize the nuanced ways in which students reason about atomic emission spectra. For example, one student might describe the process for how an atomic emission spectrum is created by saying that an electron moves to a lower energy level and emits a photon (Emission Process Idea), and another student might respond by saying that blue light has a higher energy than red light (Color/Energy Idea). While each student described an important spectroscopy concept in their response, the first student explained the mechanism for how an atomic emission spectrum is created, whereas the second student did not provide any evidence of a mechanistic understanding, but listed a fact about light that they knew. These two responses are qualitatively different, but because each response 67 included a single, unconnected idea, they had both been coded as a Partial Link (L3) using the original KIR. However, based on the relative sophistication of these responses, we believe that an explanation that correctly describes the mechanism for how spectral lines are created should be classified differently from a response that described a property of light. Similarly, we saw differences in the links students made within their explanations. Students who made the Absorption-Emission Link provided a more detailed mechanism for how spectral lines are created (compared to having the Emission Process Idea), whereas students who made the Emission Process – Color/Energy Link or the Energy Difference Link applied their understanding of electronic transitions to reason about important properties of atomic spectra. We found that students who had the latter two links within their response provided more complex reasoning within their explanations compared to students who had only the Absorption- Emission Link. However, within the KIR, these two different types of explanations were classified within the same category. We also found that some of the students who did not have the Absorption or Emission process idea within their response provided a general description of electronic transitions. For example, Emily stated that “what causes emission lines to be created is the jump of electron from one energy level to another.” Responses such as this did not have enough information to be coded as Absorption or Emission, but because they included generic reasoning about electronic transitions, we created a separate main idea code which we refer to as the Partial Emission Process. This additional code can be found in Table 5.3, along with the other main ideas and links. 68 Spectroscopy Reasoning Rubric Based on these observations, we refined the different levels within the KIR to better align with the different modes of reasoning students used within their explanations. Ultimately, this led to the development of the Spectroscopy Reasoning Rubric (SRR). The SRR consists of five categories, or modes of reasoning that are based on connections students make between spectroscopic concepts. Analogous to the KIR, the SRR characterizes students’ explanations based on the extent to which students have integrated their knowledge of multiple concepts and ideas. However, it differs from the KIR in that some types of reasoning (i.e. main ideas and links) are more sophisticated than others, which results in distinct differences in how students’ reasoning is classified. The SRR coding scheme can be found below in Table 5.4. 69 Table 5.4 – Spectroscopy reasoning rubric coding scheme Characterization Description Example Student’s response contains the Emission Process – Color/Energy Link and the Energy Difference Link Student’s response contains either the Color/Energy Link or Energy Difference Link Student attributes the emission process as the cause for how spectral lines are created Student describes properties of light (Color/Energy Idea) and/or provides a partial mechanism for how spectral lines are created Student provides an off-topic response that does not have any main ideas Multiple Connected Ideas Connected Ideas Descriptive Process Disconnected Ideas Off Topic/No Answer “The emission of a line is caused by the electron moving from a higher energy level to a lower energy level that emits a particular wavelength in correspondence to the change in energy level. The electron goes from a higher energy level/state to a lower energy level/state. The blue line is of a higher energy and therefore has a greater energy change (n=4 to n=1 in my diagram) While red has longer wavelengths and lower energy, thus has a smaller energy drop. (n=2 to n=1).” “The electrons are falling down energy levels, which emits a photon of energy. The violet electron falls from a higher energy level because violet has a higher energy than red. Red falls from a low energy level because it has low energy.” “Energy is released from the system which causes the electron that was on a higher energy level to move down the energy levels.” “The higher energy will result in violet and a lower energy will result in red. If a photon is ejected while violet is visible and about 300 nm is lost the energy level moves down and red is now visible.” “Hydrogen absorbs all wavelengths of light except for violet and red.” If a student described unrelated content or did not provide an explanation, they were classified as Off Topic/No Answer. When a student described properties of light (Color/Energy Idea), and/or provided a partial mechanism for how an atomic emission spectrum is created (Absorption or Partial Emission Process), but did not make any link between these ideas, they were coded as Disconnected Ideas. The next mode of reasoning, Descriptive Process, is when students provided a correct description of the emission process (Emission Process or Absorption- Emission Link) but did not further reason about the color or relative energy of spectral lines. 70 The final two modes of reasoning are when students apply their understanding of the Emission Process to explain the color of an emitted photon (Emission – Color/Energy Link), or to further reason about the amount of energy released during this process (Energy Difference Link). If a student had one of these links in their response they were coded as Connected Ideas, and if they make both links in their response, they were coded as Multiple Connected Ideas. During the coding development process, inter-rater reliability (IRR) studies were performed on a subset of the total student population. An initial round of IRR was conducted after the first KIR coding scheme was developed. Two researchers coded a small sample of students’ explanations and found that the KIR did not capture the increasingly sophisticated ways in which students reasoned about spectroscopy concepts. This finding led to the refinement of the KIR as described above. A second round of IRR was conducted after the main ideas, links, and SRR coding scheme was developed. Two researchers independently coded a small sample of student’s explanations using the main ideas, links, and SRR coding scheme. Upon completion, they compared their results and discussed any discrepancies in application of the coding scheme. Then, they conducted another round of IRR in which they both independently coded ~20% of student’s responses. Overall, they established a high level of inter-rater agreement with a Cohen’s Kappa greater than 0.7 for the main ideas, links, and SRR coding scheme (Table 5.5) (Landis & Koch, 1977). Any discrepancies in their coding were discussed, and they reached 100% agreement upon completion. 71 Table 5.5 – Inter-rater reliability results for main ideas, links, and SRR coding scheme Code Kappa 0.96 0.91 0.70 0.91 0.94 0.80 0.82 0.84 Main Ideas Absorption Process Emission Process Partial Emission Process Color/Energy Absorption - Emission Links Emission - Color/Energy Energy Difference SRR Reasoning (1-5) Once reliability of the SRR was established, the remaining explanations were coded. Student responses that appeared ambiguous, such as a response that may or may not have a link within it, were set aside and discussed as a group to determine the main ideas and links present within the response. Results Our findings suggest that the sample of general chemistry students included in this study reason about atomic emission spectra in a variety of different ways, ranging from simple descriptions of spectroscopy concepts to highly complex explanations in which students integrate their understanding of multiple concepts and ideas. Based on our analysis of 576 general chemistry students’ explanations of atomic emission spectra, we see that the two most common modes of reasoning were the disconnected and connected ideas categories (Figure 5.1). 72 s e s n o p s e R t n e d u t S f o % 35% 30% 25% 20% 15% 10% 5% 0% Off-Topic Disconnected Descriptive Connected Ideas Process Ideas Multiple Connected Ideas Figure 5.1 – Characterization of students’ reasoning about atomic emission spectra using the spectroscopy reasoning rubric As can be seen from Figure 5.1, a large group of students (42% N=241) provided explanations in which they were able to link two or more productive ideas together (Connected Ideas & Multiple Connected Ideas). A slightly smaller group (39%, N=223) provided explanations that did not make connections between spectroscopic concepts (Off-topic and Disconnected Ideas). The remaining students (19%, N=112) provided a descriptive mechanism for the emission process but did not link it to any other ideas. The number of participants who provided explanations with these modes of reasoning are listed in Table 5.6. The frequencies for the number of students’ who reasoned about the main ideas and links can also be found in Table 5.6. 73 Table 5.6 – Percentage of students who incorporated the main ideas, links, and modes of reasoning within their explanations of atomic emission spectra Main Ideas Links Reasoning Explanation Feature Percentage of Students (N=576) Absorption Process Emission Process Partial Emission Process Color/Energy Absorption – Emission Emission – Color/Energy Energy Difference Off-Topic Disconnected Ideas Mechanistic Process Connected Ideas Multiple Connected Ideas 22% (N=124) 61% (N=353) 18% (N=103) 64% (N=367) 16% (N=93) 39% (N=225) 15% (N=86) 12% (N=70) 27% (N=153) 19% (N=112) 30% (N=170) 12% (N=71) The Color/Energy idea was the most common idea present within students’ responses, with just under two thirds of the participants (64%, N=367) including it within their explanation. The next most common idea students reasoned about was the Emission Process (61%, N=353). However, only 39% (N=225) of the participants were able to link their understanding of these two concepts (Emission Process-Color/Energy Link). The Absorption Process (22%, N=124) and Partial Emission Process (18%, N=103) were not as prevalent as the Emission Process and Color/Energy ideas. Finally, we found that 16% (N=93) of the participants responses contained the Absorption-Emission Link, and 15% (N=86) of the participants had the Energy Difference link. Below, we describe how students reasoned about these main ideas and links within the five different categories in the Spectroscopy Reasoning Rubric. 74 Multiple Connected Ideas We begin by looking at exemplar responses where students explain how atomic emission spectra are created and apply that understanding to reason about the relative energy of the emitted photon (Energy Difference Link). They also explain how different colored spectral lines are produced (Emission Process-Color/Energy Link). For example, Jack explained the process for how an atomic emission spectrum is created by stating that “an emission of a line in an emission spectrum is created when an electron drops energy levels and emits a photon of light”. He then went on to explain how different colored spectral lines are created: “the difference is the amount of energy released which is dependent on the distance between the energy levels. The higher the gap between the energy levels the higher frequency the photon will have. The blue light is a result of a larger gap between energy levels.” Here, Jack linked his understanding of the emission process and energy quantization to explain that the energy of the emitted photon is equal to the difference in energy levels that the electron transitioned between (Energy Difference Link). Within his explanation, Jack also related the energy of the photon emitted to the frequency and resultant color seen within the spectrum (Color/Energy Link). Overall, Jack explained the mechanistic process how an atomic emission spectrum is created and then applied his understanding of this process to reason about important spectral features by linking together his understanding of electronic transitions, properties of light, and energy quantization. While some might focus on the ideas that are not canonically correct in this explanation, our focus was on identifying what students know and can do with their knowledge. For example, Jack stated that the energy released is dependent on the distance between the energy levels, that is, he may be conflating distance and energy levels. However, we would need to probe further to determine whether Jack really misunderstands this, or whether he is simply using the term distance (which many instructors do) without thinking deeply about it. The question prompt did 75 not ask him to explain how energy levels in atoms are arranged, and for that reason, we do not “penalize” Jack for this stray terminology. In this study, 12% (N=71) of the students constructed explanations that contained multiple connected ideas, all of which described the energy difference link with their response. We believe this concept is crucial to students’ understanding of spectroscopy, and as noted earlier, is a very difficult concept for students (Didiş Körhasan & Wang, 2016; Ivanjek et al., 2014; Park & Light, 2009). This finding shows that some general chemistry students can link together their understanding of electronic transitions, energy quantization, and properties of light to provide a coherent explanation of atomic emission spectra. Connected Ideas The connected ideas classification was applied when students only provided one link from the emission process to either explain how different colors are produced (Emission Process- Color/Energy Link) or to reason about the relative energy of the emitted photon (Energy Difference Link). While a student could have either of these links in their response, we found that the vast majority (91% , N=155) of students in this category related the energy and wavelength of the emitted photon to the color seen on the spectrum, whereas only 9% (N=15) of these students explained that the amount of energy emitted is equal to the difference in energy levels the electron transitioned between. As noted earlier, most of the students who included the energy difference link constructed more sophisticated explanations and were classified as Multiple Connected Ideas. We begin by looking at a student’s response that relates the emission process to the energy and resultant color of the emitted photon. For example, Sarah responded to the exam prompt that asked her to explain what happens at the atomic level that causes emission spectra 76 lines by saying that “at the atomic level an electron is jumping from a higher energy level to a lower energy level, which leads to a photon being emitted. Depending on how much energy was released a certain wavelength will show up on the emission spectra.” Sarah describes the mechanistic process for how a photon is emitted and explains how the energy of the emitted photon corresponds to a certain wavelength which is what is seen on an emission spectrum (Emission Process-Color/Energy Link). Sarah goes on to provide further reasoning about how she understands the relationship between these concepts in the following prompt that asked her to explain how this process produced different colored lines. She responded by saying that “different colors have different wavelengths and energy. So depending on how much energy was released you would be able to determine what the wavelength is which will determine the color that will show up. Less energy means longer wavelength and higher energy means shorter wavelength.” Sarah demonstrated her understanding of how the energy of the emitted photon corresponds to specific color in the first question prompt and provided further elaboration in the second prompt that showed how she conceptualizes the relationship between these concepts. However, Sarah did not explicitly discuss the idea that it is the energy difference between the two states that results in the overall energy of the photon, and therefore, the color of the light. Descriptive Process 19% (N=112) of the participants in this study were classified as Descriptive Process. Students with this type of reasoning described the mechanistic process for how spectral lines are created (Emission Process Idea) but did not connect their understanding of this process to explain how different colored spectral lines are created (Emission Process – Color/Energy Link) or to explain the quantized amount of energy that is emitted during this process (Energy Difference 77 Link). Students with this type of reasoning included either the Emission Process Idea or the Absorption-Emission Link within their explanation. For example, Elizabeth described the process for how spectral lines are created by stating that “electrons are moving down from a higher energy level to a lower energy level, therefore photons are being emitted (producing light) causing an emissions spectrum.” Elizabeth offered a clear and concise description of the process by which a spectral line is created, however, she did not explain how this process produces different colored spectral lines. In fact, she may understand and be thinking about this relationship, but based on her reasoning within her explanation, there is no evidence that she is making this relationship. Some of the participants provided more detailed explanations for how an atomic emission spectrum is created. For example, Jared stated that “First an electron is absorbing a photon light (hv). This causes the electron to move from its lower energy level to a higher energy level. Then in the emission process the electron is emitting a photon of light causing the electron to move to a lower energy level. Emission lines are created because of the electrons moving to a lower energy level from a higher E level”. Jared explained that when energy is absorbed by the hydrogen atom, an electron moves to a higher energy level. Then when that electron returns to a lower energy level, it emits a photon. He concluded by explaining that it is specifically the emission process that is responsible for creating the lines within an emission spectrum. By connecting his understanding of the absorption and emission process, Jared provided a more detailed explanation of the mechanistic process for how a spectral line is created. While Jared’s explanation was more detailed than Elizabeth’s, they both accurately explained that spectral lines on an atomic emission spectrum are created by an electron transitioning to a lower energy level and emitting a photon. Even though we did not explicitly 78 ask students to describe the Absorption Process in their explanations, we saw that 16% (N=93) of students made the Absorption-Emission Link within their response. Disconnected Ideas This category captures the various ways students described relevant, yet disconnected facts about light-matter interactions. When students provided incomplete mechanisms (Absorption Process Idea or Partial Emission Process Idea) and/or talked generically about properties of light (Color/Energy Idea), they were characterized as Disconnected Ideas. Approximately 27% (N=153) of the participants in this study had this type of reasoning. These responses had two common types of reasoning: (1) Students described general electron movement or provided incomplete mechanisms; and/or (2) Student’s described the relationship between color, energy, frequency, and/or wavelength without understanding the mechanism for how spectral lines are created. For example, Maggie described the process for how spectral lines are created as follows: “The emission of a line on the spectrum signifies the emission of a photon when an electron goes up or down an energy level”. Maggie explained that light is emitted when electrons move up or down energy levels (Partial Emission Process Idea). However, based on her response, we do not know if she understands that spectral lines are created from the emission process or if they are created from general electron movement within the atom. Maggie then went on to explain how different colored spectral lines are created, “The difference between the processes is that emission of a photon caused the red line and absorbance of a photon caused the blue line”. From her response, we get evidence that Maggie was in fact thinking that spectral lines are created when electrons move up or down energy levels. Furthermore, she related the creation of red light to the emission process and blue light to the absorption process. This example illustrates how students incomplete understanding of the 79 underlying mechanism for how spectral lines can result in nonnormative descriptions for how different colored spectral lines are created. The other common type of response that was characterized as Disconnected Ideas were when students described generic properties of light. For example, Jessica stated that “violet light has a shorter wavelength therefore a higher energy and can drop further than red light could”. Jessica explained that violet light has a higher energy than red light because it has a shorter wavelength. She applied her understanding of the relationship between wavelength and color to describe the relative energy of violet and red light. She also explained that violet light drops further than red light. Based on her response, it is unclear if she is thinking about the “color” dropping or if she is thinking about electrons transitioning to lower energy levels. Also, there is no explicit evidence within her response on how she is thinking about energy quantization when she stated that violet light “can drop further than red light could”. Off-Topic We found that 12% (N=70) of the participants had off-topic responses or did not provide an explanation within their response. While some students included off-topic non-useful responses such as “quantum mechanics”, we found that many of students in this category were conflating the photoelectric effect and atomic emission spectroscopy. For example, Travis stated that “the photoelectric effect states light is a wave and a particle. Electrons are ejected and emit a certain color based on the certain amount of energy an element has. In this case, Hydrogen fell between violet & red section on the emission spectra.” In his explanation, Travis drew upon his understanding of multiple spectroscopic concepts and made nonnormative connections between related, yet different spectroscopic phenomena. He began by describing the photoelectric effect as a process of electrons being emitted and related 80 this to the color and energy emitted. Travis concluded by making a nonsensical relationship between the hydrogen atom and where it would “fall” between colors within the atomic emission spectrum. While this explanation has a variety of nonnormative ideas, it illustrates how students can combine their understanding of the photoelectric effect and atomic spectroscopy to reason about atomic emission spectra in a nonmeaningful way. Discussion Students’ reasoning about atomic emission spectra ranged from simple descriptions of electronic transitions and disconnected facts about light, to complex explanations that provided reasoning for how and why different colored spectral lines are produced. Explaining how this process occurs requires students to (1) recognize what concepts and ideas are needed to explain the phenomenon, (2) make relevant connections between those ideas, and (3) make their thinking visible by constructing a coherent mechanistic explanation. Based on students’ linking structure, our findings suggest that the Emission Process Idea is an important concept in which students anchor their knowledge of other spectroscopy concepts. Whether it is providing a more detailed description of the emission process (Absorption-Emission Link), explaining how different colored spectral lines are created (Emission – Color/Energy Link), or explaining why only discrete amounts of energy are emitted (Energy Difference Link), the emission process idea is central to each type of link seen in student responses. These results echo the work of Körhasan and Wang (2016) who identified electronic transitions and energy of photon as threshold concepts related to students’ understanding of atomic spectroscopy. However, because of our theoretical perspective, our interpretation of the importance and application of these concepts is different. In their work, Körhasan and Wang 81 analyzed students’ explanations of atomic emission spectroscopy through a mental modelling approach in which they viewed students’ knowledge structures as being coherently organized. They found that students had four distinct mental models of atomic spectra and that “students’ wrong understanding of the concepts results in the construction of unscientific mental models of atomic spectra” (Didiş Körhasan & Wang, 2016). Through this lens, they viewed the concepts of electronic transitions and energy of photon as concepts that hinder student’s development of scientifically correct mental models. In contrast, our findings suggest that students understand and explain spectroscopic concepts in a variety of ways by linking ideas together in different ways. Students who integrated their understanding of electronic transitions, energy quantization, and properties of light provided complex explanations of atomic emission spectra. We do not view the Emission Process Idea as a threshold concept that hinders students learning, but instead, it is an important concept that is fundamental to understanding and explaining spectroscopic phenomena. Prior research by Park and Light (2009) identified energy quantization as a threshold concept that hinders student learning about quantum models of atomic structure. Similarly, we found that students had difficulty applying this concept within their explanations of atomic emission spectra. Throughout the spectroscopy section within the general chemistry curriculum, atomic emission spectroscopy was presented as the evidence for the quantized nature of energy levels within the atom. Students were explicitly taught that elements can only absorb or emit specific amounts of energy that are equal to the energy difference between the electron’s energy levels, yet we found that only 15% of students explicitly incorporated this concept within their explanations, even though it had been highly emphasized in both instruction and formative assessments. 82 However, it may be that some of the students who were not coded as having the Energy Difference Link may have implicitly incorporated the concept within their explanation. For example, Matt provided the following explanation: “In this spectrum, the electrons are dropping down to a lower energy level. The electron that drops from a higher energy level produces a light with higher energy (shorter wavelength). The electron that drops from a lower energy level produces light with a lower energy level (longer wavelength).” Matt provided an explanation of the emission process and related the color to the amount of energy released (Emission Process/Color-Energy Link), and he explained that the electron that transitioned from a higher energy level emitted a photon of higher energy. It is not possible to ascertain from his response whether he is implicitly relating the energy of the emitted photon to the difference in energy levels or if he is thinking about energy based on the height of the energy level. Prior research by Ivanjek et al. (2014) found that physics students equated the energy of spectral lines to the energy levels within the atom, so it may be that Matt was thinking about atomic emission spectra in a similar way by relating blue light to higher energy levels and red light to lower energy levels. Therefore, our analyses may undercount the number of students who have an Energy Difference Link, but even so, this is clearly a difficult concept for students. Conclusions Our results show that general chemistry students can link their knowledge of multiple spectroscopy concepts and construct sophisticated explanations of atomic emission spectra. By shifting the focus away from what students do not know (which has been the primary emphasis in prior research on students’ understanding of atomic spectroscopy) to what they do know, we were able to capture the productive ways in which students integrated and applied their understanding of electronic transitions to make sense of spectroscopic phenomena. If we can 83 foster and help students integrate their knowledge, we believe that students have the potential to develop a more robust understanding of important concepts in science. The results from this study indicate that students who provided more complex explanations of atomic emission spectra understand the underlying mechanism by which they are created. Students who reasoned mechanistically about atomic spectra integrated their understanding of how changes in energy relate to changes in atomic structure. These findings suggest that the Emission Process Idea is an important concept that students use to make sense of spectroscopic phenomena. Once students understood the underlying mechanism for how an atomic emission spectrum is created, they were better equipped to make meaningful connections between different spectroscopic concepts. Conversely, we found that students who did not understand that electronic transitions give rise to spectral lines often listed disconnected facts and were unable to integrate their knowledge. Implications and Future Work As noted earlier, the instructional approach to support students’ understanding of any scientific phenomenon should depend on how one thinks about the ways in which students learn. Much of the earlier research on students’ understanding of light-matter interactions has focused on characterizing problematic ideas students have about spectroscopy concepts and ascribing various mental models to students. In contrast, we believe that our work has provided evidence for a more fluid situation, in which students use ideas and connect them in different ways. This has direct implications for both instruction and future research on this particular topic. General recommendations include the idea that students should have opportunities to construct complex explanations about phenomena that require them to link several ideas together. In practice, this approach can be difficult to accomplish, especially in large enrollment 84 classes where multiple choice assessments or computerized homework tends is more commonly used. However, for students to develop a more robust and integrated understanding of the big ideas in science, they should be given the opportunity to develop and connect them for themselves. That is, in formative assessment and tutorials, students should be given the opportunity to engage in analysis and interpretation of data, argumentation, and explanation. For example, students can construct drawn and written explanations for homework, recitation, or group activities in class. More specific implications for how to help students better understand atomic emission spectroscopy can be based on the findings reported here. For example, it seems clear that the emission process idea is a central concept from which students link their knowledge. Therefore, a more explicit emphasis on the mechanism by which spectral lines are created in both instruction and assessment should be an important take-away. If we can foster opportunities in and out of the classroom for students to actively engage in this knowledge integration process, we believe that students can construct a deeper understanding of how light interacts with matter at the atomic and molecular level. Ideally, findings such as these would be integrated back into a curriculum design cycle. Future work in this area will focus on the ways that instructional changes, as described above, affect student understanding. We also plan to investigate how the nature of the prompt affects student responses, since we have found in previous work that the design of the task prompt can elicit different types of responses from students (Crandell et al., 2018). Limitations Finally, we would like to note that the students who participated in this study were enrolled in a transformed general chemistry curriculum (CLUE) in which atomic emission 85 spectroscopy may have been discussed in more detail compared to a more traditional general chemistry course. Additionally, students within the CLUE curriculum may have more experience and training in constructing explanations of phenomena compared to a traditional general chemistry curriculum. As a result, caution needs to be taken in generalizing these results based on these potentially confounding factors. Another limitation of the research presented in this chapter is that the students’ reasoning is directly influenced by both the CLUE curriculum and the summative assessment tasks. Recall, each cohort (F13-F16) had a different assessment task on their midterm exam; thus, the coding scheme and findings presented in this chapter do not address the various ways in which the individual assessment tasks may have influenced students’ reasoning in different ways. The effect that the curriculum and assessment changes had on students’ reasoning is discussed in more detail in Chapter 6. 86 ANALYZING THE EFFECT THAT THE F13-F16 CURRICULUM CHANGES HAD ON STUDENTS’ REASONING Preface The previous chapter (Chapter 5) described the development of the Spectroscopy Reasoning Rubric (SRR) and discussed the findings on how general chemistry students within the CLUE curriculum reasoned about atomic emission spectra. While these findings illustrated the various ways in which students integrated and connected their knowledge, there was no discussion of the how the curriculum and assessment changes presented in Chapter 4 may have impacted their explanations. In this chapter, I discuss the effect that the curriculum and assessment changes had on the reasoning patterns used by each cohort (F13-F16). Introduction Throughout the course of this study, the spectroscopy section of the CLUE general chemistry curriculum was iteratively refined using a design-based research methodology (Brown, 1992; Collins, 1992), which resulted in a series of changes being made to instruction, homework, recitation, and summative assessment tasks. Together, these changes were designed to help students better understand and explain atomic emission spectroscopy. A summary of the curriculum and assessment changes that were implemented from F13-F16 can be found below in Table 6.1. For a more in-depth explanation of the changes that were made, please refer to Chapter 4 where they are discussed in more detail. 87 Table 6.1 – Curriculum changes and exam prompts for F13-F16 cohorts Cohort Curriculum Changes Exam Explanation Prompt Pre-Curriculum Changes State the Process & Explain • Emphasis of atomic spectroscopy was on the • What process is responsible for the evidence it provides about the quantized nature of energy levels within an atom emission of the wavelengths of light in the spectrum? Fall 2013 • Students learned about how atomic spectra are • Explain why only certain created in lecture, but they did not have any additional activities where they had to apply their understanding of spectroscopy concepts wavelengths of light appear in the spectra for each element Major Curriculum Changes Describe Process • Explicitly discussed differences between the • Describe the process that is Fall 2014 Fall 2015 responsible for the emission of the red line and violet line in the spectrum photoelectric effect and atomic spectra • Emphasized the mechanistic process for how • an emission spectrum is created Implemented a scaffolded homework activity where students work to explain atomic spectra and reason about key spectral features • Developed new recitation materials that build upon the homework activity and have students discuss concepts in small groups • Altered exam question prompt Minor Curriculum Changes Describe Process & Compare/Contrast • Altered instruction to provide more explicit • Describe in words the processes emphasis on why only discrete energy changes can occur and the impact this has on the emitted color from an atom • Refined the homework activity to provide a more explicit focus on how the energy of the emitted photon is equal to the difference in energy levels the electron transitioned between • Altered exam question prompt that cause the emission of a line in an emission spectrum • What is similar between the processes that cause the red and blue line? • What is different between the processes that cause the red and blue line? Minor Curriculum Changes • No changes were made to instruction or formative assessment • Altered exam question prompt Fall 2016 Describe Process & Explain (x2) • Describe what happens at the atomic level that causes the emission lines to be created • Explain how this process produces different colored lines • Explain why only certain wavelengths of light appear in the spectra for each element 88 The first set of curriculum and assessment changes were implemented from F13-F14 based on interviews conducted in F13. These changes focused on helping students understand the mechanistic process for how spectral lines are created within an atomic emission spectrum. One of the main changes was the implementation of a new homework activity, which consisted of a series of scaffolded questions where students applied their knowledge of electronic transitions, properties of light, and energy quantization to explain the process for how different colored spectral lines are created. These concepts and their relationship to one another were further emphasized in lecture and in a subsequent recitation activity. From F13-F14, the summative assessment task was reduced to a single question prompt which asked students to describe the process for how a red and violet line are created within an atomic emission spectrum. The F14-F15 curriculum changes explicitly emphasized the energy difference link, which is the concept that electronic transitions can only occur when a specific amount of energy is absorbed/emitted which is equal to the difference between two energy levels. To provide a more explicit emphasis on this concept, four additional questions pertaining to the energy difference link were added to the homework activity. This concept was also discussed more explicitly within lecture in F15 compared to F14. The F14 summative assessment task was broken down into three separate question prompts in F15. First, students were asked to describe the process that causes the emission of a line and then they were prompted to explain what is similar and different between the processes that cause the red and blue line. From F15-F16, no changes were made to instruction, homework, and recitation. However, the summative assessment task was refined. Consisting of three clustered explanation prompts, the F16 summative assessment task asked students to (1) describe what happens at the atomic level that causes the emission lines to be created, (2) explain how this process produces 89 different colored lines, and to (3) explain why only certain wavelengths of light appear in the spectra for each element. Participants The curriculum and assessment changes described above were implemented over a four- year timespan, from Fall 2013 to Fall 2016. In total, the general chemistry curriculum was refined a total of three times, resulting in four separate cohorts of students (F13, F14, F15, & F16) who were enrolled in General Chemistry I (GC1) at various stages of curriculum refinement. The same participants described in Chapter 5 were used in the analysis presented in this chapter. In total, 576 students participated in this study. Each cohort had a similar number of participants: F13 (N=145), F14 (N=141), F15 (N=145), and F16 (N=145). To ensure that the random sample of students were similar, I performed a variety of statistical tests based on available demographic data and assessment measures. The threshold used to determine statistical significance was a p  0.01. A Mann-Whitney test was used to compare students’ course grades in GC1 and ACT scores. No statistical differences were found between student’s course grades (p=0.06) or ACT scores (p=.033). A chi-square analysis was used to compare the ratio of males to females and the number of first-generation students. I found that there were no statistical differences (p>.01) between the cohorts for these two measures. Each cohort also had similar characteristics regarding ethnicity and class standing. From these analyses, I found that the four cohorts of students were similar based on available demographic data and assessment measures. 90 Data Analysis To investigate the effect that the curriculum and assessment changes had on students’ understanding of spectroscopy concepts, this chapter looks at the association between the F13- F16 cohorts and the reasoning patterns they employed within their explanations of atomic emission spectra. Analysis of student explanations (to the four separate question prompts shown in Table 6.1) was conducted using the main ideas, links, and reasoning coding scheme presented in the previous chapter. To make inferences about the differences in the reasoning patterns used by each cohort, several Pearson Chi-squared tests of association were conducted to see if there were any significant differences in how the different cohorts explained atomic emission spectra (Green & Salkind, 2016). The threshold used to determine significance within this study was a p  0.01. Whenever a significant association was found between cohort and an explanation feature (main ideas, links, or mode of reasoning), Cramer’s V was calculated and interpreted using Cohen’s guidelines, in which a small effect size corresponds to Cramer’s V of 0.1, a medium effect size is 0.3, and a large effect size is 0.5 (Cohen, 1988). If a significant association was found between cohort and an explanation feature, a post-hoc analysis was conducted to identify the main driver(s) of significance. This process consisted of identifying the standardized residual (which is a measure of the difference between an observed and expected value) and comparing that value to the critical value of 2.58 (which is based on a p  0.01). If a standardized residual was greater in magnitude than the critical value of 2.58, they were marked in bold and deemed as a primary driver for the observed association. The sign of the standardized residual indicates whether the observed value is greater or less than the expected value. 91 Association between cohort and main ideas Results The main ideas, links, and overall reasoning (SRR) present within a students’ response varied across the four cohorts. Here, I report the percentage of students from each cohort who included main ideas, links, and the overall reasoning of their response. Findings on the association between cohort (F13-F16) and the different explanation features (main ideas, links, and reasoning) are also presented. A more in-depth analysis on how these findings relate to the curriculum changes implemented from F13-F16 can be found below in the discussion section. The percentage of students in each cohort who reasoned about main ideas within their response is indicated in Figure 6.1. Main Ideas s t n e d u t S f o e g a t n e c r e P 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Absorption Emission Process Partial Emission Color/Energy Process Process F13 (N=145) F14 (N=141) F15 (N=145) F16 (N=145) Figure 6.1 – Main ideas present within student explanations of atomic emission spectra For three of the four main ideas, there was an increase in the number of students from F13-F16 who reasoned about the absorption process, emission process, and color/energy within their explanation. The number of students who reasoned about the absorption process decreased (19% to 13%) from F13-F14, however more students in F15 (23%) and F16 (33%) reasoned 92 about the absorption process. The percentage of students who reasoned about the emission process steadily increased from F13-F16. In F13, less than half of the participants reasoned about the emission process and by F16, after multiple changes had been made to the curriculum and the assessment task, over three-quarters of the students reasoned about the emission process. Similarly, there was an increase in the number of students from F13-F16 who included the color/energy idea within their explanation. Only 20% of the participants reasoned about the color/energy idea in F13, but that percentage improved in F14 when 68% of students included it within their response. This percentage continued to improve in F15 (80%) and again in F16 (87%). Finally, the partial emission process idea was the least predominant idea across the four cohorts, with less than 20% of each cohort including it within their response. The association between cohort and the four main ideas is illustrated below in Table 6.2. Table 6.2 – Association between cohort and main ideas Main Ideas 2 Value df p-Value Cramer’s V Absorption Process Emission Process 13.82 33.15 Partial Emission Process 0.45 Color/Energy 171.36 3 3 3 3 0.003 <0.001 0.930 <0.001 0.16 0.24 N/A 0.55 Findings show that there is a significant relationship (p  0.01) between cohort and three of the four main ideas – the absorption process had a small effect size (p=0.003, V=0.16), the emission process had a small effect size (p<0.001, V=0.24), and the color/energy idea had a large effect size (p<0.001, V=0.55). No significant differences were found between cohort and the partial emission process (p=0.930). A post-hoc analysis was conducted to determine the driver of significance between cohort and main ideas present (Table 6.3) 93 Table 6.3 – Contingency table for the post-hoc analysis of the association between cohort and main ideas Cohort F13 F14 F15 F16 Absorption Process Emission Process Color/Energy Present -0.6 Not Present 0.3 Present -2.5 Not Present 3.2 Present -6.6 Not Present 8.7 Expected: 31.2 Observed: 28 Expected: 113.8 Observed: 117 Expected: 88.9 Observed: 65 Expected: 56.1 Observed: 80 Expected: 92.4 Observed: 29 Expected: 52.6 Observed: 116 -2.2 1.2 -0.6 0.7 0.7 -0.9 Expected: 30.4 Observed: 18 Expected: 110.6 Observed: 123 Expected: 86.4 Observed: 81 Expected: 54.6 Observed: 60 Expected: 89.8 Observed: 96 Expected: 51.2 Observed: 45 0.5 -0.3 0.8 -1 2.5 -3.3 Expected: 31.2 Observed: 34 Expected: 113.8 Observed: 111 Expected: 88.9 Observed: 96 Expected: 56.1 Observed: 49 Expected: 92.4 Observed: 116 Expected: 52.6 Observed: 29 2.3 -1.2 2.3 -3 3.5 -4.6 Expected: 31.2 Observed: 44 Expected: 113.8 Observed: 101 Expected: 88.9 Observed: 111 Expected: 56.1 Observed: 34 Expected: 92.4 Observed: 126 Expected: 52.6 Observed: 19 *In each cell, the standardized residual, expected value, and the observed value is reported. Standardized residuals ( 2.58) are bolded Post-hoc analysis examining the association between cohort and absorption process showed that none of the standardized residuals were larger than the critical value of ±2.58, suggesting that none of the cohorts were a main driver for significance. For the emission process, there was a positive association between the F13 cohort and the absence of the emission process within their explanation. This finding suggests that the absence of the emission process idea within F13’s explanations is the main driver of significance for the relationship between cohort and emission process idea. For the relationship between cohort and the color/energy idea, the main driver of significance was the negative association between F13 and color/energy, and a positive association between F16 and the color/energy. This means that the F13 cohort had fewer students than expected who reasoned about the color/energy idea, whereas the F16 cohort had more students than expected. Additionally, there was a negative association between the F15 cohort and the absence of the color/energy idea, which means that fewer students than expected did not reason about the color/energy idea within their response. 94 Association between cohort and links Students’ made three different linkages within their explanations. The percentage of students from each cohort who included the absorption-emission, emission-color/energy, and energy difference link within their response is illustrated in Figure 6.2 – Links present within student explanationsFigure 6.2. Links s t n e d u t S f o e g a t n e c r e P 70% 60% 50% 40% 30% 20% 10% 0% Absorption - Emission Emission - Color/Energy Energy Difference F13 (N=145) F14 (N=141) F15 (N=145) F16 (N=145) Figure 6.2 – Links present within student explanations of atomic emission spectra Just over 10% of the participants in the F13 and F14 cohorts reasoned about the absorption-emission link and approximately 20% of the F15 and F16 cohorts had this link present within their response. For the next category, the emission-color/energy link, there was an increase in the number of students from F13-F16 who linked their understanding of these two concepts. Only 11% of participants from F13 reasoned about the emission-color/energy link and approximately 40% of the students in F14 and F15 had this link within their response. There was another increase in F16, where 65% of participants made this relationship. The final link that students made within their explanations was the energy difference link, which was almost non-existent in F13 (4%). This number improved from F13-F16, resulting in 95 28% of the students in F16 having the energy difference link within their explanations. The association between cohort and links within their explanations is illustrated below in Table 6.4. Table 6.4 – Association between cohort and links Links 2 Value df p-Value Cramer’s V Absorption – Emission 9.02 Emission – Color/Energy 89.25 Energy Difference 34.07 3 3 3 0.029 <0.001 <0.001 N/A 0.39 0.24 No significant differences were found between cohort and the absorption-emission link (p=0.029). Findings show that there is a significant relationship between cohort and the emission-color/energy link (p<0.001, V=0.39), which corresponds to a medium effect size. There is also a significant relationship between the cohort and the energy difference link (p<0.001, V=0.24), which corresponds to a small effect size. A post-hoc analysis was conducted to determine the driver of significance between cohort and the presence of links within students’ responses (Table 6.5). 96 Table 6.5 – Contingency table for the post-hoc analysis of the association between cohort and links Cohort F13 F14 F15 F16 Emission – Color/Energy Energy Difference Present -5.4 Not Present 4.3 Present -3.4 Not Present 1.4 Expected: 56.6 Expected: 88.4 Expected: 21.6 Expected: 123.4 Observed: 16 Observed: 129 Observed: 6 Observed: 139 -0.3 0.2 -0.2 0.1 Expected: 55.1 Expected: 85.9 Expected: 21.1 Expected: 119.9 Observed: 53 Observed: 88 Observed: 20 Observed: 121 0.7 -0.6 -0.6 0.2 Expected: 56.6 Expected: 88.4 Expected: 21.6 Expected: 123.4 Observed: 62 Observed: 83 Observed: 19 Observed: 0.2 5.0 -4.0 4.2 -1.7 Expected: 56.6 Expected: 88.4 Expected: 21.6 Expected: 123.4 Observed: 94 Observed: 51 Observed: 41 Observed: 104 *In each cell, the standardized residual, expected value, and the observed value is reported. Standardized residuals ( 2.58) are bolded The post-hoc analysis examining the association between cohort and the emission- color/energy link showed that there was a negative association for the F13 cohort and that there was a positive association for the F16 cohort. This finding suggests that fewer students in F13 had the emission-color/energy link in their response than expected, whereas in F16, more students made the emission-color/energy link than expected. These two factors are the main drivers of significance for the relationship between cohort and emission-color/energy. A similar association was found for the relationship between cohort and energy difference link. There was a negative association between the F13 cohort and the energy difference link, whereas the F16 cohort had a positive association to the energy difference link. Together, these findings suggest that there is a negative association between the F13 cohort and links present, whereas there is a positive association for the F16 cohort. 97 Association between cohort and reasoning Based on the main ideas and links present, students’ responses were coded using the Spectroscopy Reasoning Rubric, which consists of five different modes of reasoning. The percentage of students from each cohort who employed the various modes of reasoning is illustrated in Figure 6.3. Modes of Reasoning s t n e d u t S f o e g a t n e c r e P 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Off-Topic Disconnected Descriptive Connected Ideas Process Ideas Multiple Connected Ideas F13 (N=145) F14 (N=141) F15 (N=145) F16 (N=145) Figure 6.3 – Reasoning present within student explanations of atomic emission spectra The number of students who had off-topic reasoning decreased from F13-F16. Nearly a third of the F13 cohort (29%) had off-topic reasoning. This percentage decreased in F14 (12%), F15 (6%), and was almost non-existent in F16 (1%). As discussed in Chapter 4, many of the students in the F13 cohort thought that the photoelectric effect is the process by which an atomic emission spectrum is created. It is for this reason that several of the students in F13 were coded as having off-topic reasoning. From F13-F16, the percentage of students who had disconnected ideas ranged from 22% (F16) to 30% (F14). The frequency in which students had disconnected ideas was similar across the four cohorts. 98 Approximately thirty percent of the F13 cohort described the descriptive process within their responses. This percentage decreased in F14 (17%), slightly increased in F15 (20%), and dropped again in F16 (10%). Overall, the number of students who were coded as descriptive process decreased from F13-F16. The final two categories, connected ideas and multiple connected ideas, were used to code responses in which students linked together their knowledge. In F13, only 13% of the participants had connected ideas. This number increased to 29% in F14, 37% in F15, and 40% in F16. Each year the number of students who had connected ideas increased, with the largest increase happening from F13-F14. Only 1% of the participants in F13 had multiple connected ideas within their response, whereas approximately 10% of participants from F14 and F15 had this type of reasoning. This percentage increased in F16, where 27% of the participants were able to reason about multiple ideas and make connections between them. To see if there is significant relationship between the F13-F16 cohorts and the modes of reasoning they employed, a chi-square test of association was conducted. However, the data did not meet the required assumption that each cell needs to have at least 5 counts. The F13 cohort only had two students who had multiple connected ideas and the F16 cohort had only two students who had off-topic responses. In order to run a chi-square test, I reduced the coding scheme from five bins (which are the five different modes of reasoning) to three bins in which the off-topic and disconnected ideas categories were combined and the connected ideas and multiple connected ideas were combined. A chi-square test was then conducted to compare the relationship between cohort and reasoning (3 bins). 99 Findings showed that a signification association exists between the F13-F16 cohorts and the mode of reasoning they employed (2 (3) = 87.68, p < 0.001, V = 0.28), which corresponds to a small to medium effect size. A post-hoc analysis was conducted to identify the main drivers of significance (Table 6.6). Table 6.6 – Contingency table for the post-hoc analysis of the association between cohort and mode of reasoning Cohort Off-Topic + Disconnected Ideas Descriptive Process Connected + Multiple Connected Ideas F13 F14 F15 F16 3.2 3.2 -5.2 Expected: 56.1 Expected: 28.2 Expected: 60.7 Observed: 80 Observed: 45 Observed: 20 0.7 -0.7 -0.3 Expected: 54.6 Expected: 27.4 Expected: 59 Observed: 60 Observed: 24 Observed: 57 -1.0 0.2 0.8 Expected: 56.1 Expected: 28.2 Expected: 60.7 Observed: 49 Observed: 29 Observed: 67 -3.0 -2.7 4.7 Expected: 56.1 Expected: 28.2 Expected: 60.7 Observed: 34 Observed: 14 Observed: 97 *In each cell, the standardized residual, expected value, and the observed value is reported. Standardized residuals greater than  2.58 are bolded. The post-hoc analysis of the chi-square test examining the association between cohort and mode of reasoning showed that the main drivers of significance were the F13 and F16 cohort’s relationship to the different modes of reasoning. The F13 had a positive association with the Off-Topic + Disconnected Ideas category, and to the Descriptive Process category. There was a negative association between the F13 cohort and the Connected + Multiple Connected ideas category. 100 An opposite trend was observed between the F16 cohort and the different modes of reasoning. The F16 cohort had a negative association with the Off-Topic + Disconnected Ideas category and the Descriptive Process category. A positive association was observed between the F16 cohort and the Connected + Multiple Connected ideas category. Together, these findings show that the F13 cohort had a positive association with less sophisticated modes of reasoning and that the F16 cohort had a positive association with having more complex reasoning. Discussion Overall, these findings showed that there was a significant association between the different cohorts (F13-F16) and the reasoning patterns they employed within their explanations of atomic emission spectra. Specifically, the main driver of significance was that students’ reasoning in F13 was less sophisticated than the reasoning patterns used by the F16 cohort. This finding provides evidence that the curriculum and assessment changes made from F13-F16 had a significant impact on how students within the CLUE curriculum reason about atomic emission spectra. However, because multiple changes were made from F13-F16, it is difficult to identify the effect that any specific change had on students’ reasoning. Instead, I highlight the main curriculum changes that were made from F13-F16 and discuss how these changes relate to observed findings presented above. Recall, in Fall 2013 the CLUE curriculum emphasized the fact that atomic spectra provide evidence for the quantized nature of electronic energy levels within the atom. During instruction, this idea was explicitly addressed, and students were taught the mechanistic process for how spectral lines are created. The F13 homework and recitation activities primarily focused on properties of light, where students performed calculations of wavelength or energy using Planck’s equation (E=hυ). There were few opportunities in F13 for students to practice 101 constructing explanations of the mechanistic process for how spectral lines are created. Finally, the F13 assessment task consisted of two parts: (1) what process is responsible for the emission of wavelengths of light, and (2) explain why only certain wavelengths of light appear in the spectrum for each element. Based on these curriculum materials and the given assessment prompt, I found that nearly 30% of the F13 cohort had Off-Topic reasoning. These explanations contained incorrect ideas that were often woven together in a variety of ways, such as the example below. Zoe: The photoelectric effect specific wavelengths provide the necessary energy for the electrons to be released, or break away from the atom, in this the wavelength of light is also reflected. The emission spectra only shows those wavelengths which are reflected by an element, the wavelength which appear here and those wavelengths, all the others (which are displayed in black) are absorbed. Within her explanation, Zoe stated that electrons are released when specific wavelengths are absorbed. She then explained that this wavelength is reflected and ultimately seen within the emission spectrum. Furthermore, she mentioned that the black lines in the spectrum are the wavelengths that are absorbed. From her response, it appears that Zoe was doing a “brain dump” in which she tried to incorporate all the spectroscopic concepts previously learning within her explanation. Because she had not developed an understanding of how these concepts related to one another, she made multiple incorrect connections within her explanation. In addition to the 30% of students in F13 who provided off-topic explanations, 26% of the F13 cohort was coded as having Disconnected Ideas. This means that over 50% of the participants in F13 were unable to explain the mechanistic process (i.e. an electron transitioning to a lower energy level and emitting a photon) for how a spectral line within an emission spectrum is created. 102 The curriculum changes that were implemented from F13-F14 were explicitly designed to emphasize the mechanistic process for how spectral lines are created. In F14, a new beSocratic homework activity was developed. It consisted of a series of scaffolded questions that emphasized the process for how an atomic emission spectrum is created. During instruction, there was a more explicit discussion on how the photoelectric effect relates to the absorption/emission process. A new question on atomic spectroscopy was added to the recitation worksheet to give students an opportunity to discuss spectroscopic concepts with peer support. Finally, the F14 assessment task was changed. The F14 question prompt asked students to describe the process that is responsible for the emission of the red and violet line within an emission spectrum. Based on these curriculum and assessment changes, I found that the number of students who provided Off-Topic responses decreased from F13-F14 (29% to 12%). This finding is directly related to the decrease in the number of students in F14 who thought that atomic spectra are created from the photoelectric effect. However, the percentage of students who were coded as having Disconnected Ideas were similar in F13 (26%) and F14 (30%). Overall, students in F14 provided more detailed explanations than the F13 cohort. This finding is supported by the increase in the number of students from F13-F14 who were coded as Connected or Multiple Connected Ideas, which are the two modes of reasoning in which students integrate their understanding of different spectroscopic concepts. The percentage of students who were coded as having Connected Ideas (12% to 37%) and Multiple Connected Ideas (1% to 11%) improved from F13-F14. Based on the decrease in the percentage of students who had Disconnected Ideas, and the increase in the percentage of students who had Connected and Multiple Connected Ideas, it 103 appears that the F13-F14 curriculum changes had a positive impact on student reasoning. The statistical analysis discussed in the Results section also provided additional support for this finding: the F13 cohort was identified as having a positive association with less sophisticated modes of reasoning (Off-topic + Disconnected Ideas, and Descriptive Process) and a negative association with more sophisticated modes of reasoning (Connected Ideas + Multiple Connected Ideas). However, it is important to note that these statistical tests are based on the reasoning patterns of each cohort (F13-F16) and do not directly compare the F13 and F14 cohort. As discussed in Chapter 4, the curriculum changes that were implemented from F14-F15 were explicitly designed to target students’ understanding of the energy difference link, which describes the quantized amount of energy that is absorbed and emitted from the atom. When electrons absorb energy and transition to a higher energy level or emit energy when transitioning to a lower energy level, the amount of energy absorbed/emitted during this process is equal to the difference in energy levels that the electron transitions between. To help students better understand this concept, the spectroscopy homework activity was refined from F14-F15 to have a more explicit emphasis on the energy difference link. This concept was also discussed more explicitly within lecture, where they instructor directly addressed the common ways in which students may incorrectly think about this concept. The F15 summative assessment task was also refined. In F15, students were asked to (1) describe the processes that cause the emission of a line in an emission spectrum, (2) explain what is similar between the processes that cause the red and blue line, and to (3) explain what is different between the processes that cause the red and blue line. Looking at how these changes impacted students’ explanations, I found that the F14 and F15 cohorts employed similar reasoning patterns (main ideas, links, and SRR) within their 104 explanation of atomic emission spectra. There appeared to be no discernable differences in the percentage of students who were coded for each mode of reasoning within the Spectroscopy Reasoning Rubric. This finding is supported by the statistical tests in the results section, which showed that the F14 and F15 cohorts were not a main driver in the association between cohort (F13-F16) and reasoning. The curriculum changes that were implemented in F14-F15 were designed to improve students’ understanding of the energy difference link. In hindsight, the F15 assessment task did not adequately elicit students’ understanding of this concept. The F15 task asked students to explain what is similar and different between the processes that create a blue and red line. This prompt explicitly focused on the relationship between the emission process and the color of emitted light. While students would need to implicitly think of the energy difference link to provide a detailed explanation of this phenomenon, the question did not directly address this concept; thus, it is not surprising that students did not reason about a concept in which they were not explicitly asked to explain. To see if a more explicit question prompt would in fact improve students’ reasoning, the summative assessment task was changed in F16. No changes were made to instruction, homework, or recitation. The F16 assessment task consisted of a clustered set of question prompts: 1. Draw an energy diagram that illustrates the process for how one spectral line is created in an atomic emission spectrum 2. Describe what happens at the atomic level that causes the emission lines to be created 3. Explain how this process produces different colored lines 4. Explain why only certain wavelengths of light appear in the spectra for each element 105 The first question had students construct an energy diagram. Then, students were asked to describe what happens at the atomic level that causes emission lines to be created. This prompt was designed to elicit students’ understanding of the mechanism (the emission process idea) for how spectral lines are created. Next, students were asked to explain how this process produces different lines. This question is similar to the F14 and F15 prompts in which students were asked to reason about the similarities and differences between a red and blue line. It was designed to elicit students’ understanding of the relationship between the emission process and properties of light (emission-color/energy link). Finally, the last question asked students to explain why only certain wavelengths of light appear in the spectra for each element. This prompt was designed to elicit students’ understanding of the energy difference link. In fact, this question was the same question prompt that was asked in the F13 assessment task. I found that that there was an increase in the number of students from F15-F16 who reasoned about the Emission-Color/Energy link (43% to 65%) and the Energy Difference link (13% to 28%). The F16 cohort were more likely to make linkages within their explanations, and the percentage of students from F15-F16 who were coded as having Multiple Connected Ideas increased from 10% to 27%. That is, more students in F16 were able to integrate their understanding of spectroscopic concepts to reason about atomic emission spectra in a more sophisticated manner. This inference is supported by findings presented in the Results section, in which the F16 cohort was identified as having a negative association with less sophisticated modes of reasoning (Off-topic + Disconnected Ideas, and Descriptive Process) and a positive association with more sophisticated modes of reasoning (Connected Ideas + Multiple Connected Ideas). Based on the differences in the reasoning patterns used by the F15 and F16 cohorts, it 106 appears that the F16 assessment task was able to better elicit students’ understanding of atomic emission spectra. In summary, the changes that were made to instruction, homework, recitation, and the summative assessment task over the course of this study (F13-F16) had a positive impact on students’ reasoning about atomic emission spectra. However, the curriculum and assessment changes implemented during each phase of the study (F13-F14, F14-F15, and F15-F16) may have influenced students’ reasoning in different ways; which has direct implications for how curricular designs can impact student learning. Implications The curriculum changes implemented from F13-F16 were designed to improve students’ understanding of atomic emission spectroscopy. Specific changes were made to instruction, homework, recitation, and the summative assessment task based on observations of how students reasoned about this phenomenon the prior fall semester. These changes were designed to bring coherence within the CLUE curriculum by aligning the treatment of spectroscopy concepts across curricular materials and assessments. By looking at the changes that were implemented during each phase of the study, it provides insight into how different components of the curriculum (instruction, homework, recitation, and summative assessment tasks) can influence (or not influence) students’ reasoning. The first set of curriculum changes (F13-F14) were based on interviews that were conducted in Fall of 2013. Interview findings showed that general chemistry students had difficulty explaining the mechanistic process for how an atomic spectrum is created; thus, the F13-F14 curriculum changes were designed to address this issue. This led to the development of a new homework and recitation activity, which provided students with multiple opportunities for 107 them to actively develop and apply their understanding of the process for how an atomic spectrum is created. The F14 summative assessment task was refined to explicitly focus on students’ understanding of this mechanistic process. In fact, the F14 assessment task was less complex than the F13 task, which asked students to explain why only certain wavelengths of light appear in the spectra for each element. As discussed above, the F13-F14 curriculum changes led to improvements in students’ reasoning. This shift toward more sophisticated reasoning is more likely a result of the increased emphasis within the course (instruction, homework, recitation) as opposed to the summative assessment task because The F13 question prompt was more detailed than the F14 assessment task. The F14 question prompt only asked students to describe the process for how different colored spectral lines are created, whereas the F13 prompt had an additional question pertaining to the quantized amount of energy that each element can absorb or emit. However, because the F13 cohort had not developed a deep understanding of spectroscopic concepts, they were not able to respond to this question prompt in a meaningful way. Based on these observations, it appears that the increase in students’ reasoning from F13-F14 can be mainly attributed to the changes made to instruction, homework, and recitation. From F14-F15, changes were made to instruction, homework, and the summative assessment task to provide a more explicit emphasis on the energy difference link, where only specific amounts of energy can be absorbed or emitted from the atom which are equal to the difference in energy levels in which electrons transitions between. The homework activity was refined, and four additional slides were added that focused on the energy difference link. For F15, the F14 summative assessment task was broken into multiple parts, in which students were asked to explain the similarities and differences in how different colored spectral lines are 108 created. As described above, I found that the F14-F15 changes did not improve students’ reasoning. Based on the different factors that were changed from F14-F15, it was unclear if changes to instruction and homework had no impact on student learning, whether the new assessment task did not elicit the types of reasoning I had hoped for, or if it was a combination of factors that resulted in no observable differences in student reasoning. To see if a more explicit assessment task would result in more detailed explanations of atomic emission spectra, the F16 question prompt was changed. This new task incorporated components from each cohort. Students were asked to describe the process for how spectral lines are created, to explain how different colored lines are created, and to explain why each element has its own unique spectrum. If the changes made to instruction, homework, and recitation from F13-F15 had helped students develop a more robust understanding of atomic spectroscopy, I had hoped that the F16 question prompt would better elicits students’ reasoning compared to the F15 prompt. Indeed, I found that students’ reasoning improved from F15-F16, which provided evidence that the nature of the assessment task directly influenced students’ reasoning. Together, these findings have important implications on how to design learning environments that support student learning, and they illustrate the need for coherence across curriculum materials. Question prompts that have the capability of eliciting complex reasoning are of little use if students have not had proper support and scaffolding to help them develop an integrated understanding of foundational concepts and ideas in science. Similarly, if there is incoherence between what is being valued in the classroom and the question prompts used to assess student understanding, it creates an environment where students are being sent mixed- messages on what they should know and be able to do with their knowledge. Ultimately, students should have clear expectations of what types of knowledge and reasoning are valued within a 109 given learning environment, and design features (i.e. instruction, formative assessments, worksheets, summative assessment tasks, etc.) should be aligned to better promote student learning. Conclusions Overall, the findings presented in this chapter illustrate how changes to curriculum and assessment tasks can impact the reasoning patterns students use when explaining scientific phenomena. From F13-F16, a series of changes were made to the spectroscopy section of the CLUE general chemistry curriculum talk at Michigan State University. These curriculum changes were designed to improve students’ understanding of atomic emission spectroscopy by emphasizing the relationship between electronic transitions, properties of light, and energy quantization. Each set of curriculum changes (F13-F14, F14-F15, and F15-F16) was directly influenced based on an analysis of students’ reasoning from the prior semester. Results show that the changes made over the course of this study improved students’ reasoning about atomic emission spectra. The F16 cohort more readily constructed explanations in which they connected their understanding of multiple concepts compared to the F13 cohort, which was more likely to have disconnected ideas. Evidence for this relationship is observed in the statistical analysis which looked at the relationship between cohort (F13-F16) and the modes of reasoning they employed. A positive association was identified between the F16 cohort and the presence of links within their response, whereas the F13 cohort had a negative association with the presence of links within their explanation. These linkages directly influenced the overall reasoning patterns each cohort employed, resulting in more complex explanations by the F16 cohort; thus, providing evidence that that changes made to instruction, homework, recitation, and 110 the summative assessment task helped students develop a deeper understanding of atomic spectroscopy. 111 CHARACTERIZATION OF STUDENTS’ REPRESENTATIONS OF THE PROCESS FOR HOW SPECTRAL LINES ARE CREATED The overarching goal of the broader research study was to implement curriculum changes that improve students’ understanding of atomic emission spectroscopy. Specifically, the curriculum changes that were made to instruction, homework, recitation, and summative assessment tasks from F13-F16 were designed to improve students’ explanations of atomic emission spectra. While the primary emphasis of this study has focused on characterizing student’s reasoning, I wanted to see if these changes indirectly influenced the ways in which students illustrated the process for how an atomic emission spectrum is created. Recall, each cohort had a constructed response question on their midterm exam in which they were asked to both draw and explain the mechanistic process for how an atomic emission spectrum is created. Chapters 5 described how students’ explanations were analyzed, and Chapter 6 discussed the effect that the curriculum changes implemented from F13-F16 had on students’ reasoning. This chapter describes how students’ diagrams were analyzed and presents the findings for how students represented the process for how an atomic emission spectrum is created. Diagram Assessment Tasks After students’ explanations on their midterm exams were analyzed using the Spectroscopy Reasoning Rubric (as described in Chapters 5 and 6), I worked with an undergraduate researcher to develop a diagram coding scheme to characterize the representations that accompanied each participants explanation. The different assessment tasks that each cohort (F13, F14, F15, and F16) had on their midterm exam is provided in Table 7.1. 112 Table 7.1 – Diagram assessment tasks on students’ midterm exam Cohort Diagram Assessment tasks The spectra above show the emission spectra in the visible region of the electromagnetic spectrum for a number of elements. What process is responsible for the emission of these wavelengths of light? Draw a diagram to illustrate your thinking Describe the process that is responsible for the emission of the red line in the spectrum and the violet line in the spectrum? Draw an energy diagram to illustrate your thinking Construct energy diagrams that illustrate the processes that cause the emission of the red line and the blue line in the spectrum. Make sure to label all the components within your diagram. (Two boxes were provided for students to draw in, one labeled “Process that causes the red line” and other labeled “Process that causes the blue line”) Draw an energy diagram that illustrates the process for how one spectral line is created in an atomic emission spectrum. Make sure to label all the components within your diagram. F13 F14 F15 F16 The diagram assessment tasks are directly related the explanation question prompts. The F13 assessment task asked students to draw a diagram that illustrates how they are thinking about the process that is responsible for the emission of different wavelengths of light. The F14 assessment task asked students to draw an energy diagram that illustrates how they are thinking about the process that is responsible for the red and violet light in the spectrum. These two prompts are similar in that they both prompt students to draw a diagram to illustrate their thinking of the emission process, however, the F14 assessment task explicitly asked students to draw an energy diagram, whereas the F13 task does not specifically ask for an energy diagram. The F15 diagram assessment task asked students to construct energy diagrams that illustrate the process for how a blue and red line are created. They were also prompted to label the different components in their diagram. Two boxes were given on the exam for the students to draw their energy diagrams. The F16 diagram assessment task asked students to draw an energy diagram that illustrates the process for how one spectral line is created in an atomic emission 113 spectrum. The F15 and F16 assessment tasks were similar in that they both prompted students to draw an energy diagram for how a spectral line is created, however they were different because the F15 task explicitly had students construct two energy diagrams for how two different colors are created, whereas the F16 task only asked students construct a single energy diagram to show how a spectral line is created. While these assessment tasks are similar, the specific prompting is different for each cohort. The main difference is that the F13 cohort was asked to construct a diagram, whereas the F14, F15, and F16 cohorts were explicitly prompted to construct energy diagrams. The F15 and F16 cohorts were also told to label all the components within their diagram. Data Analysis I began by open-coding a random sample of representations from each cohort to identify common ways in which students illustrated their knowledge. During this process, I found that the F13 cohort constructed a variety of different diagrams (i.e. energy level diagram, bohr model of the atom, and photoelectric effect) whereas the F14, F15, and F16 cohorts primarily drew energy level diagrams. This finding led to the development of an initial coding scheme that characterizes the type of diagram a student represents. The four common diagrams students constructed are shown in Table 7.2 114 Table 7.2 – Type of representation coding scheme for students’ drawing of atomic emission spectra Type of Representation Coding Scheme Code Example Description Energy Level Diagram Bohr Model Photoelectric Effect Other Apply mechanism/reasoning codes if student draws energy diagram If student draws Energy diagram and another diagram, code each model present Apply mechanism/reasoning codes if student draws Bohr Model If student draws Bohr and energy level diagram, only apply mech/reasoning codes once When PE is coded, also code for electron emitted Other possible examples are a reaction progress diagram, an experimental set-up, or random wavelengths 115 The most common representation that students constructed was an energy level diagram, which consists of discrete energy levels that represent the quantized amount of energy that electrons have within the atom. I expected that most of the students would construct an energy level diagram because they were emphasized in lecture and the F14, F15, and F16 assessment tasks directly asked students to construct an energy diagram within their response. Students also constructed representations using the Bohr model of the atom, which represents electrons orbiting around a nucleus. Some of the students constructed representations of the photoelectric effect within their response, which is a representation that is not relevant to the given phenomenon. The final type of diagram in the representation coding scheme is the other category, which is a code that is applied when students drew representations that typically showed some macroscopic phenomenon, such as light shining through a prism. If a student constructed multiple diagrams, each diagram was coded. Both energy level diagrams and the Bohr model of the atom were used to illustrate various electronic transitions within the atom, whereas students who constructed irrelevant representations (Photoelectric Effect and Other Category) did not represent electronic transitions within their diagrams. To capture the mechanisms students represented using an energy level diagram or a Bohr Model of the atom, a mechanism coding scheme for drawings was developed (Table 7.3). 116 Table 7.3 – Mechanism coding scheme for students’ drawings of atomic emission spectra Mechanism Coding Scheme Code Example Description Emission Process Partial Emission Process Absorption Process Partial Absorption Process Multiple Transitions or General Electron Movement Electron transitions from higher to lower energy level and a photon is emitted Electron transitions from higher to lower energy level, but representation does not show a photon being emitted Electron absorbs a photon and electron transitions from lower to higher energy level Electron transitions from lower to higher energy level, but representation does not show absorption of a photon Response contains various transitions represented by multiple arrows or arrow points both up and down Example would be coded as partial emission and multiple transitions 117 I identified five common ways in which students represented electronic transitions in an energy level diagram or a Bohr model of the atom. The first code is the Emission Process, in which a student shows an electron transitioning to a lower energy level and emitting a photon. To be coded as emission process, student’s representation had to show both an electron transitioning to a lower energy level and a photon being emitted. If a response showed a transition from a higher to a lower energy level, but did not show the emission of a photon, it was coded as Partial Emission Process. Similarly, students were coded as Absorption Process if they showed an electron absorbing a photon and moving to a higher energy level. If the representation showed an electron transitioning to a higher energy level, but did not show the absorption of a photon, it was coded as Partial Absorption Process. The final way in which students represented electronic transitions within their drawings was by showing Multiple Transitions or General Electron Movement. For example, the illustration in Table 7.3 shows multiple arrows pointing down. Because this response has three different arrows illustrating electronic transitions, it would be coded as Multiple Transitions. It would also be coded as partial emission process because the arrows signify the transition of an electron from higher to lower energy level without the release of a photon. Students illustrated electronic transitions in five different ways, however, more than one code could be applied to a particular response. For example, if a student showed the absorption process followed by the emission process, they would be coded for both processes within the coding scheme. In addition to the representation and mechanism coding schemes, one final diagram coding scheme was developed to characterize additional types of reasoning or incorrect components that were present within students’ representations (Table 7.4). 118 Table 7.4 – Reasoning and incorrect components coding scheme for students’ drawings of atomic emission spectra Reasoning and Incorrect Components Code Example Description Properties of Light (E=hυ=hc/λ) Energy Difference (ΔE) Electron Emitted Incorrect Ground State Mixed Up Absorption & Emission Student labels the energy of the absorbed or emitted photon as being equal to hυ or hc/λ Student clearly labels that ΔE is the difference in energy levels the electron transitioned between Student shows an electron being emitted within their diagram Every time student draws photoelectric effect, it is coded as electron emitted Students diagram has ground state listed as n=0 or it has an unlisted ground state below n=1 Student shows emission occurring when a photon is absorbed or absorption occurring when a photon is emitted 119 The Properties of Light and the Energy Difference codes in Table 7.4 represent two ways in which students productively expanded upon their representation of electronic transitions. Student diagrams were coded as Properties of Light when the photon being absorbed (absorption process) or emitted (emission process) was labeled as hυ or hc/λ, which indicated the wavelength, frequency, and resultant energy of a photon. If students labeled the electronic transition as being equal to the difference in energy levels that the electron transitioned between it was coded as Energy Difference. Both codes are analogous to the Color/Energy idea and Energy Difference link used within the explanation coding scheme. Students also incorporated three incorrect ideas within their representations. The electron emitted code was applied when a student indicated within their representation that an electron is released from the atom. This code was also applied whenever a student represented the photoelectric effect, which is a phenomenon in which electrons are released. The Incorrect Ground State code was applied whenever a student labeled the ground state in their diagram as n=0 or if their diagram had an unlisted ground state that was below n=1. The final incorrect idea students had in their diagrams was when they mixed up the absorption and emission process. For example, if a student drew an electron absorbing a photon and moving to a lower energy level they would be coded as Mixed Up Absorption and Emission. To summarize: three coding schemes were developed to characterize students’ diagrams of the process for how atomic emission spectra are created. 1) The Type of Representation coding scheme was used to characterize the type of diagram that a student constructed, 2) the Mechanism coding scheme was used to capture the various ways in which students represent electronic transitions if they constructed an energy level diagram or a Bohr model of the atom, and 3) the Reasoning and Incorrect Components coding scheme was used to characterize both 120 the productive ways students expand upon their representations of electronic transitions and the incorrect components they have within their diagrams. To establish reliability of the three diagram coding schemes, I conducted a round of inter- rater reliability with the undergraduate researcher who helped in the development of the coding scheme. A random sample of approximately 15% of the participants from each cohort were selected and both of us independently coded students’ representations using three coding schemes described above. Upon completion, I calculated the inter-rater agreement and found that each code had a high inter-rater reliability (κ >0.7). Once reliability of the coding schemes was established, the undergraduate researcher coded the remaining responses. Any responses that were unclear were discussed as a group. Results & Discussion Type of Representation s t n e d u t S f o e g a t n e c r e P 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Energy Diagram Bohr Model Photoelectric Other Effect F13 (N=145) F14 (N=141) F15 (N=145) F16 (N=145) Figure 7.1 – Type of representation in student diagrams The most common representation that students constructed was an energy diagram, which is unsurprising given the fact that the F14, F15, and F16 cohorts were explicitly prompted to construct such a representation. Over 90% of these participants (F14-F16) constructed an energy 121 diagram, whereas only 53% of the students in F13 drew an energy diagram to illustrate their understanding of the process for how an atomic emission spectrum is created. The F13 cohort was asked to construct a diagram to illustrate their thinking, whereas the F14-F16 cohorts were explicitly asked to construct an energy level diagram. It appears that without an explicit prompt saying which type of diagram students should draw, the F13 participants constructed a variety of other representations. Approximately a third of the F13 cohort drew a Bohr model of the atom within their response. Although the Bohr model of the atom was explicitly de-emphasized in lecture because it promotes the idea that electrons move a certain distance between energy levels, never the less, over 30% of students drew this representation. Additionally, 17% of the F13 cohort illustrated the photoelectric effect which showed electrons being emitted from a metal plate. This finding aligns with the results from the explanation coding where I found that many of the students in the F13 cohort had off-topic responses because they explained the photoelectric effect rather than the emission process. Together, the findings from the explanation and diagram coding suggests that the F13 cohort had trouble understanding the difference between these two similar, yet different spectroscopic phenomena. The students who constructed an energy diagram or a Bohr model of the atom were then analyzed to see how students illustrated the mechanism for how the spectral lines in an emission spectrum are created. In total, students illustrated five different types of mechanism within their representations (Figure 7.2). 122 Mechanism Represented 80% 70% 60% 50% 40% 30% 20% 10% 0% Emission Process Partial Emission Process Absorption Partial Multiple Process Absorption Transitions Process F13 (N=145) F14 (N=141) F15 (N=145) F16 (N=145) Figure 7.2 – Mechanism represented in student diagrams Students that illustrated the emission process constructed a correct mechanism. Just under 40% of the F13 cohort illustrated the emission process. This percentage slightly increased in F14 (43%) and again in F15 (50%). Then, in F16, nearly three quarters of the participants (73%) illustrated the emission process within the energy diagram they constructed. I looked back at the F15 and F16 diagram assessment tasks to see if there were any discernable differences between the question prompts that would lead to this large increase from F15-F16 in students drawing the emission process. The F15 task provided students with two boxes and they were asked to “construct energy diagrams that illustrate the processes that cause the emission of the red line and the blue line in the spectrum” and the F16 task asked students to “draw an energy diagram that illustrates the process for how one spectral line is created in an atomic emission spectrum.” Both of these prompts asked students to illustrate the process for how spectral lines are created, however the F15 task focused on the creation of two different colors (red and blue) whereas the F16 task did not invoke the specific color created. It may be that the F15 assessment tasks focus on the 123 creation of two specific colors shifted students thinking away from the emission process, but without any more information (such as follow up interviews or additional representations), it is difficult to make inferences as to why a larger percentage of the F16 cohort drew the emission process in their representation compared to the other cohorts. Another mechanism the participants illustrated was the partial emission process. There was a large increase in the number of students from F13 (21%) to F14 (44%) who drew the partial emission process. Additionally, 39% of the students in F15 and 19% of the students in F16 illustrated the partial emission process using an energy level diagram. This finding aligns with one of main changes that was implemented from F13-F14, where there was an explicit distinction made between the photoelectric effect and the process for how atomic spectra are created. As discussed earlier, 17% of the students in F13 illustrated the photoelectric effect and in F14 only 1% of the students drew the photoelectric effect. The increase in the number of students who drew the partial emission process from F13-F14 (21% to 44%) may be related to the decrease in the number of students from F13-F14 (17% to 1%) who illustrated the photoelectric effect. The final three types of mechanisms were not as prevalent as the emission process and the partial emission process. Less than 20% of participants from F13-F16 drew the absorption process and a smaller amount (<12%) illustrated the partial emission process. Under 10% of participants from F13-F16 drew multiple transitions within their diagram. A final coding scheme was used to characterize the additional components (both correct and incorrect) that students included within their representations (Figure 7.3). 124 Reasoning & Incorrect Components s t n e d u t S f o e g a t n e c r e P 30% 25% 20% 15% 10% 5% 0% Properties of Energy Light Difference Electron Emitted Incorrect Mixed-up Ground State Absorption & Emission F13 (N=145) F14 (N=141) F15 (N=145) F16 (N=145) Figure 7.3 – Reasoning and incorrect components in student diagrams Approximately a quarter (24%) of the F13 cohort illustrated Properties of Light (student labeled the energy of the absorbed/emitted photon as E=hυ=hc/λ) within their representations. In F14 (11%) and F15 (14%), there was a decrease in the number of students who included this component within their energy diagram, and in F16 (27%) nearly a third of the participants labeled the energy of the absorbed/emitted photon within their representation. Very few of the F13-F16 participants labeled electronic transitions as having energy equal to the difference between two energy levels (Energy Difference). Only 1% of the participants in the F13 and F15 cohort included this component within their representation, and none of the students in F16 illustrated the energy difference. This finding is surprising because 28% of the participants in F16 included the Energy Difference link with their explanation. The F14 cohort had the largest percentage of students (10%) who illustrated the energy difference. The other three codes (Electron Emitted, Incorrect Ground State, and Mixed-up Absorption &Emission) are the incorrect components students included within their 125 representations. 21% of the F13 cohort drew an electron being emitted within their diagram. Less than 3% of the F14, F15, and F16 cohort illustrated electrons being emitted. Participants also labeled the ground state incorrectly (e.g. n=0). There was an increase in the percentage of students who drew incorrect ground states from F13-F16 (4% to 20%). Less 6% of students from each cohort mixed-up the absorption and emission process. Implications While these findings show the different ways in which students represent their understanding of the process for how an emission spectrum is created, there are similarities between the representations each cohort constructed and the explanations they provided. For example, from F13-F16 the percentage of students who reasoned about the emission process steadily improved from F13 (45%) to F16 (77%). Similarly, there was a steady in increase in the number of students who represented the emission process from F13-F16 (39% to 73%). Based on this observation, it appears that the curriculum and assessment changes implemented throughout the course of this study improved both students’ explanations and their representations of the emission process even though the changes were not explicitly designed to improve students’ representations of the emission process. From F13-F16, there was an increase in the percentage of students who reasoned about the emission – color/energy link (11% to 65%) and the energy difference link (4% to 28%). However, there was a different trend in the percentage of students who represented these concepts (properties of light and energy difference) within their energy level diagrams. The percentage of students who represented properties of light within their representations was similar in F13 (24%) and F16 (27%). Almost none of the students in F13 (1%) and F16 (0%) represented the energy difference within their representation. 126 Together, these findings show that while the F13-F16 increased the percentage of students who reasoned about the emission – color/energy link and the energy difference link within their explanations, there was not an improvement in the percentage of students who illustrated this concept within their energy level diagrams. Interestingly, 28% of the F16 cohort reasoned about the emission link and none of these students represented the energy difference within their representations. This observation shows how difficult it is for students to represent their understanding of energy quantization. To better understand how students represent the energy difference link, it would be important to design more explicit question prompts that ask students to include this concept within their representations. Conclusions Overall, general chemistry students represented the process for how an atomic emission spectrum is created in various ways. To characterize the different models and components students included within their representations, three coding schemes were developed. The first scheme characterized the Types of Representations students drew, which consisted of energy level diagrams, the Bohr Model of the atom, the photoelectric effect, and random/off-topic models. The energy level diagram was the most common representation used by the F13-F16 cohorts. However, the F13 cohort was much more likely to draw different types of representations, such as the Bohr model of the atom and the photoelectric effect. If students constructed an energy level diagram or a Bohr model of the atom, they were coded using a Mechanism Coding Scheme, which characterizes the different ways in which they illustrated electronic transitions. Findings show that from F13-F16 there was an increase in the percentage of students who illustrated the emission process. Other ways of representing 127 electronic transitions (Absorption Process, Partial Absorption, Partial Emission, and Multiple Transitions) were less common across all four cohorts. The final diagram coding scheme captured the correct and incorrect components students included within their representation. The percentage of students who reasoned about Properties of Light (E=hυ=hc/λ) ranged from 11% in F14 to 27% in F16. Very few of the students across all four cohorts labeled the Energy Difference (energy between two energy levels labeled as ΔE) within their representation. Approximately 20% of the students in the F13 cohort showed that electrons are emitted from the atom, whereas less than 3% of the F14-F16 cohorts included this component within their representation. Two other incorrect components students included within their representations was drawing an inaccurate ground state (n=0), or they mixed up the absorption- emission process. Together, these findings show that students represent their knowledge in different ways. Future work on how the representations students construct align with the explanations they construct would provide a deeper insight into how students conceptualize the mechanistic process for how spectral lines are created. 128 CONCLUSIONS, LIMITATIONS, AND FUTURE WORK The research presented in this dissertation has focused on characterizing the different ways general chemistry students conceptualize and reason about atomic emission spectroscopy. Based on these findings, a variety of curriculum and assessment changes were implemented within the context of a reformed general chemistry curriculum (CLUE) over a four-year timespan (F13-F16). These changes were designed to help students integrate their understanding of three overarching spectroscopic concepts – electronic transitions, properties of light, and energy quantization. To see what effect the curriculum and assessment changes had on students’ reasoning, I developed a coding scheme (referred to as the Spectroscopy Reasoning Rubric (SRR)) to characterize the extent to which students integrate and connect their knowledge of spectroscopy concepts. Using the SRR, I analyzed students’ explanations on their midterm exams to see if there were any differences in students’ reasoning across the four cohorts of students. Below, I summarize the impact that these changes had on students’ reasoning about atomic emission spectra and describe the design-features that emerged from this work. Conclusions The spectroscopy section of the CLUE general chemistry curriculum was iteratively refined from F13-F16 using a design-based research methodology (Brown, 1992; Collins, 1992). Throughout the course of this study, evidence-based changes were made to instruction, homework, recitation, and the summative assessment tasks. Interview findings from Fall 2013 showed that general chemistry students had difficulties understanding the process by which an atomic emission spectrum is created. Almost half of the students interviewed described the photoelectric effect when reasoning about atomic 129 spectroscopy. To help students better understand the relationship between the photoelectric effect and atomic spectroscopy, instructional changes were implemented in F14 that explicitly addressed the similarities and differences between these two quantum phenomena. Additionally, formative assessment tasks (homework and recitation materials) within the course were refined to provide students with more opportunities for them to actively develop and apply their understanding of electronic transitions to reason about the process by which an atomic emission spectrum is created. Finally, a simpler question prompt was administered on the F14 summative assessment task. Findings showed that the F13-F14 curriculum and assessment changes resulted in a decrease (27% to 9%) in the percentage of students who thought that the photoelectric effect was the process by which an atomic emission spectrum is created. These results point to a more generalizable design feature for how to improve student learning in science, which is to provide students with explicit opportunities within the classroom to compare similar phenomena. As students learn new material and progress throughout their education, they need to continually think of how new concepts and phenomena relate to their prior knowledge. However, if connections between concepts and phenomena are not made explicitly, students will have to implicitly connect these ideas together on their own. The findings within this work show that providing a more explicit emphasis on the similarities and differences between two similar phenomena (the photoelectric effect and atomic spectroscopy) can help students develop a better understanding of the relationship between related phenomena. The F13-F14 curriculum and assessment changes led to an improvement in the percentage of students who reasoned about electronic transitions and properties of light, however, only a small percentage of students in both cohorts reasoned about energy quantization within their explanations of atomic emission spectra. To address this observation, the 130 spectroscopy homework activity was refined in F15 to emphasize the fact that only specific amounts of energy can be absorbed or emitted, which are equal to the difference in energy levels that electrons transition between. Additionally, the summative assessment task was changed in F15. Students were asked to explain what is similar and different between the process that causes a red and blue line to be created, whereas in F14, students were prompted to explain the process for how red and violet lines are created. Findings showed that there were no observable differences in students’ reasoning from F14-F15. To see if a more explicit question prompt would elicit more sophisticated reasoning, only the summative assessment task was changed from F15-F16. The F16 assessment task consisted of three clustered question prompts – (1) describe what happens at the atomic level that causes the emission lines to be created, (2) explain how this process produces different colored lines, and (3) explain why only certain wavelengths of light appear in the spectra for each element. The more explicit question prompt resulted in an increase in the percentage of students from F15-F16 who provided more sophisticated reasoning within their explanations. This finding shows that the nature of the assessment task directly influences students’ reasoning, which is an important design feature that one must think about when assessing student learning. Together, the curriculum and assessment changes implemented from F13-F16 show how summative assessment tasks can influence students’ reasoning in different ways when paired with or without curriculum changes. For instance, the main changes to instruction and formative assessment were implemented from F13-F14. However, a less explicit summative assessment task was used in F14 compared to F13. These changes led to an improvement in students’ reasoning. 131 From F15-F16, no changes were made to instruction or formative assessments, but a more explicit summative assessment task was implemented. This led to an improvement in the percentage of students who had multiple connected ideas within their explanations. The F13-F14 findings showed that even with a less explicit question prompt particular aspects of reasoning about the spectroscopic process can be improved when students have more opportunities within the classroom to actively integrate their knowledge, whereas the F15-F16 findings showed that a more explicit question prompt can improve students’ reasoning without making any other curriculum changes. Together, these findings show the dynamic relationships between instructional materials, formative assessments, and summative assessments; thus, showing the importance of having coherence across the curriculum. Overall, the F13-F16 curriculum and assessment changes were designed to improve students’ understanding of atomic spectroscopy. Findings show that over the year years of this study the curriculum and assessment changes had a positive effect on students’ reasoning. The F13 cohort had a positive association with less sophisticated reasoning patterns (Off-topic, Disconnected Ideas, and Descriptive Process) and a negative association with more sophisticated modes of reasoning (Connected Ideas and Multiple Connected Ideas). Conversely, the F16 cohort had a negative association with less sophisticated reasoning patterns (Off-Topic, Disconnected Ideas, and Descriptive Process) and a positive association with more sophisticated modes of reasoning (Connected Ideas and Multiple Connected Ideas). Together, these findings provide evidence that that curriculum and assessment changes implemented throughout the course of this study improved student’s explanations of atomic emission spectroscopy. 132 Limitations and Future Work A direct limitation of this study is that it is impossible to know how each individual change within the CLUE curriculum may have effected students’ reasoning. Instead, it was the combination of changes to instruction, recitation, homework, and the nature of the assessment task that led to the observed findings discussed throughout this work. The effect that these design features had on students’ reasoning is directly tied to the specific learning environment (the CLUE curriculum) in which they were implemented. This is a distinguishing feature of design- based research and raises some important questions. How would students from different general chemistry contexts reason about atomic emission spectra? How would the design features implemented in CLUE translate to a new context? These are testable questions and could be the focus of future work. I think it would be particularly interesting to see how different learning environments influence the ways in which students integrate and link their knowledge. For example, within the CLUE curriculum students are readily asked to apply their knowledge by constructing explanations, arguing from evidence, and using models to make sense of chemical phenomena. This emphasis on putting knowledge to use by engaging in scientific practices is emphasized throughout CLUE and explicit support is given to students for how to reason about scientific phenomena in a more meaningful way. For example, one way to help students in constructing explanations is by explicitly discussing the components that make up a coherent explanation. CLUE uses the Claim, Evidence, Reasoning (CER) framework (McNeill & Krajcik, 2008) as a scaffolding tool to help students construct more in-depth explanations. By helping students identify their claim, the evidence that supports their claim, and the reasoning for why the evidence supports their claim, CLUE provides instruction to students of the different components that are essential for a coherent explanation. This approach is used because it has been previously observed that 133 students can often make a claim, but they are unable to connect their claim to their evidence with proper reasoning. Building on the CER framework, CLUE focuses on helping students understand both how and why phenomena occur (as opposed to focusing on surface-level features). Consequently, the assessment tasks within CLUE are designed to elicit how students understand the underlying components of a phenomenon. These features are embedded within the CLUE curriculum and have a direct impact on the question’s students are asked, the knowledge that is valued, and the ways in which students engage in the process of learning chemistry. It would be interesting to see how different learning environments impact the ways in which students’ reason about atomic emission spectroscopy to gain insight into how different curricular designs influence student learning. Additionally, the scope of this work is limited in that it only looks at how students understand a single spectroscopic phenomenon – atomic emission spectroscopy. More research would be needed to see how students’ reason about other spectroscopic phenomena such as UV- VIS, IR, 13C NMR, and 1H NMR spectroscopy. Research looking at how students’ reason about different types of spectroscopy would provide a deeper insight into how they conceptualize and think about light-matter interactions. This work would also touch on another area of future research that looks at how students’ understanding of spectroscopy changes over time as they move from introductory general chemistry to upper-division classes such as quantum mechanics – a course in which principles of spectroscopy are discussed in more depth. Based on the findings from the F15-F16 curriculum changes in which only the summative basement task was refined, it is clear the nature of the question prompt has a direct impact on student reasoning. If we want students to reason about multiple concepts (which is 134 often required when explaining both how and why phenomena occur), it is important that assessment prompts are carefully structured so that they directly elicit students’ understanding of a desired construct(s). However, writing question prompts that provide an adequate amount of scaffolding, yet do not over simplify the task, can be difficult to write. There is a need for more research that looks at how different assessment tasks influence students’ explanations of scientific phenomena. While the design of better assessment tasks has the potential to provide a deeper insight into how students understand science, it is impossible to truly know what students know. Instead, all we can do is take a brief snapshot of what students appear to know within a given context. A single picture, or even a collection of pictures, provides only a glimpse into how students’ understanding in science grows and changes over time. However, these pictures tell a story of how students connect and integrate their knowledge, and it is through these stories that we can make evidence-based changes that have the potential to improve the teaching and learning of science. 135 APPENDICES 136 Appendix A: Fall 2013 interview protocol Intro: Today I want to ask you some questions about your ideas of energy in chemical systems. Some of these questions might be related to things you’ve learned about in your courses, but some of them will probably be about things that you haven’t learned yet. So you’re probably not going to be very sure about many of the answers. That’s okay. We’re really just interested in how you think about these things; we’re not really interested in whether you get answers right or wrong. So, I’m hoping you’ll tell me as much as you can about what you think about the questions that I’m going to ask. Just talk, and I’ll listen and ask questions. I won’t really give you any feedback like “yes, that’s right” or “no, that’s not right” since I really just want to know how you think about things. But, if you’d like to talk more about the questions afterwards, we can do that. Background and demographics • Major • Year in school, • Prior courses in chemistry, physics, other The first part of the interview relates to how atoms and molecules interact. 1. Tell me what you know about helium atoms. a. What does the structure look like? b. How would two helium atoms interact? c. Are there any forces that would act between the two atoms? If so, please describe. i. Why do you think these forces are present here? d. What would happen to the amount of force between them if the atoms moved closer to one another? 2. Please describe what you know about potential energy in the context of this atomic-level system (HELIUM) a. What does the term potential energy mean to you in the context of an atomic- molecular system? b. What happens to the potential energy as the two helium atoms approach? c. Why does PE change? d. Please try to draw a graphical representation of the relationship between potential energy and the distance between the helium atoms. i. What does the representation you constructed mean in terms of the interactions between atoms? (what’s going on a the molecular level?) e. At the minimum point in PE, it is sometimes said that the system is “stable”. What does this mean to you? 137 3. Please describe what you know about kinetic energy in the context of this system. a. What happens to the kinetic energy as the atoms move closer together? i. How does KE change as the two atoms approach? ii. Why does KE change as the two atoms approach? 4. What happens to the energy of the system when you ‘break’ a stable interaction (LDF bond in case of helium) and separate the helium atoms. a. Why? b. If they identify that energy must be added to the system—where does that energy come from? c. If bond formation releases energy—what happens to that energy? 5. Repeat questions on PE, KE with water molecules 6. Repeat questions on PE, KE, with hydrogen atoms 7. As you noted, hydrogen atoms form a covalent bond, while water molecules and helium atoms form intermolecular interactions. a. If they haven’t yet described each -- Please describe your understanding of these types of interactions. b. How are they similar? Different? i. In terms of magnitude? ii. Why do they occur? iii. Are there any similarities between them? iv. Differences? c. If they drew similar representations of PE vs. r-- You drew graphical representations to represent how the potential energy of the system would change for helium atoms and hydrogen atoms. i. How do the two representations compare? What’s similar/different. Why? ii. Can ask them to draw all on same axis and compare magnitudes, explain why. The next part of the interview relates to how electromagnetic radiation and atoms 8. Describe your understanding of electromagnetic radiation. 9. Since it’s the simplest element, think of a hydrogen atom again. a. Atoms (and molecules) like hydrogen can absorb and emit electromagnetic radiation under certain circumstances. What would happen if this atom were to absorb electromagnetic radiation? i. Draw a picture of what’s going on when this happens at the atomic- molecular level. ii. What wavelengths could be absorbed? Why? b. What about emitting electromagnetic radiation? How might that happen? i. What wavelengths could be absorbed? Why? ii. What part of the atomic structure would be affected by absorption or emission of radiation (what happens to the electrons?) 10. Here’s an example of some atomic emission spectra for different atoms that show the 138 emission spectra in the visible region of the electromagnetic spectrum. a. What does this representation mean to you? i. Explain the atomic process by which this emission takes place. (draw as necessary to explain thinking) ii. Why do only certain wavelengths of light appear for each element? b. Tell me about your understanding of the term “energy quantization”. 139 Appendix B: Fall 2013 spectroscopy homework activity The spectroscopy homework activity for Fall 2013 is presented below and the figures illustrate what the students would have seen as they completed the activity. Figure B.1 – Introduction slide (F13) Figure B.2 – Explanation and representation prompt for energy quantization (F13) 140 Figure B.3 – Explanation and representation prompt for difference between absorption and emission spectra (F13) Figure B.4 – Explanation and representation prompt for reflection (F13) 141 Figure B.5 – Multiple choice question for wavelength of a moving electron (F13) Figure B.6 – Multiple choice question for the wavelength of Usain Bolt (F13) 142 Figure B.7 – Representation prompt for an s orbital (F13) Figure B.8 – Representation prompt for a p orbital (F13) 143 Figure B.9 – Representation prompt for mental image of what an atom would look like (F13) 144 Appendix C: Fall 2014 spectroscopy homework activity The revised spectroscopy homework activity for Fall 2014 is presented below and the figures illustrate what students would have seen as they completed the activity. Figure C.1 – Introduction slide (F14) Figure C.2 – Description of spectroscopy (F14) 145 Figure C.3 – Explanation prompt for how an emission spectrum is created (F14) Figure C.4 – Description of spectroscopy activity (F14) 146 Figure C.5 – Representation and explanation prompt for the atomic structure of a hydrogen atom (F14) Figure C.6 – Explanation and representation prompt for the absorption process (F14) 147 Figure C.7 – Compare and contrast prompt for the relationship between student’s representation of the absorption process to an energy level diagram representing the absorption process (F14) Figure C.8 – Explanation prompt for why an electron transitions to a higher energy level (F14) 148 Figure C.9 – Multiple choice and explanation prompt for the energy required to promote an electron to a higher energy level (F14) Figure C.10 – Representation and explanation prompt for the emission process (F14) 149 Figure C.11 – Explanation prompt for how different colors are emitted (F14) Figure C.12 – Explanation prompt for how a blue line in the emission spectrum is created (F14) 150 Figure C.13 – Explanation prompt for why only specific lines appear in hydrogen’s emission spectra (F14) Figure C.14 – Explanation prompt for why each element has a unique emission spectrum (F14) 151 Appendix D: Fall 2014 recitation worksheet The questions on the revised spectroscopy recitation activity for Fall 2014 are presented below. On the actual recitation worksheet, students were given extra space between the questions. Question 5 is the question prompt that focuses on students’ understanding of atomic emission spectra Question 1 a. What is the wavelength of a light with a frequency of 7.26 x 1014 Hz? What region of the EM spectrum would this lie in? b. What is the frequency of radiation with a wavelength of 442 nm. What region of the EM spectrum would this lie in? c. What is the energy of a photon with a frequency of 7.26 x 1014 Hz. What region of the EM spectrum would this lie in? d. What is the wavelength of a photon of energy 2.4 x 10-16J? What region of the EM spectrum would this lie in? Question 2 The energy to break 1 mol of C-C bonds (that is 6.022x 1023 C-C bonds) is 348 kJ/mol a. What would be the minimum frequency of a photon that would break a single C-C bond? b. What region of the EM spectrum would this lie in? Question 3 Make an argument that light (electromagnetic radiation) is a wave Claim: Light is a wave Evidence: Reasoning: Question 4 Make an argument that light (electromagnetic radiation) is a particle Claim: Light is a particle Evidence: Reasoning: 152 Question 5 The emission spectrum for helium is given below Yellow Red Blue Figure 0.1 – Helium’s atomic emission spectrum a. Draw energy diagrams to help you construct a step-by-step description for the processes involved in the production of an emission spectrum. Be sure to include both the processes by which energy is absorbed and emitted. b. Draw an energy diagram to compare the possible electron transitions that would produce the red emission line at the left of the spectrum and the blue line at the far right of the diagram. 153 Appendix E: Fall 2015 spectroscopy homework activity The revised spectroscopy homework activity for Fall 2015 is presented below and the figures illustrate what students would have seen as they completed the activity. Figure E.1 – Introduction slide (F15) Figure E.2 – Description of spectroscopy (F15) 154 Figure E.3 – Explanation prompt for how an emission spectrum is created (F15) Figure E.4 – Description of spectroscopy activity (F15) 155 Figure E.5 – Representation and explanation prompt for the atomic structure of a hydrogen atom (F15) Figure E.6 – Explanation and representation prompt for the absorption process (F15) 156 Figure E.7 – Compare and contrast prompt for the relationship between student’s representation of the absorption process to an energy level diagram representing the absorption process (F15) Figure E.8 – Multiple choice and explanation prompt for how much energy would be required for an electron to transition to a higher energy level (F15) 157 Figure E.9 – Multiple choice and explanation prompt for how much energy would need to be absorbed for an electron to transition to a higher energy level (F15) Figure E.10 – Representation and explanation prompt for the emission process (F15) 158 Figure E.11 – Explanation prompt for how much energy would be emitted as an electron transitions to a lower energy level (F15) Figure E.12 – Multiple choice and explanation prompt for the electronic transition that emits a photon with higher energy (F15) 159 Figure E.13 – Explanation prompt for how a blue line in the emission spectrum is created (F15) Figure E.14 – Explanation prompt for why only specific colors are absorbed or emitted (F15) 160 Figure E.15 – Explanation prompt for why each element has a unique emission spectrum (F15) 161 Appendix F: Copy of the American Chemical Society’s policy on dissertations 162 REFERENCES 163 REFERENCES Aguiar, J. G., & Correia, P. R. M. (2016). Using concept maps as instructional materials to foster the understanding of the atomic model and matter–energy interaction. Chemistry Education Research and Practice, 17(4), 756–765. https://doi.org/10.1039/C6RP00069J Armstrong, C., Burnham, J. A. J., & Warminski, E. E. (2017). Combining Sustainable Synthesis of a Versatile Ruthenium Dihydride Complex with Structure Determination Using Group Theory and Spectroscopy. Journal of Chemical Education, 94(7), 928–931. https://doi.org/10.1021/acs.jchemed.6b00874 Barab, S., & Squire, K. (2004). Design-Based Research: Putting a Stake in the Ground. Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1 Bardar, E. M., Prather, E. E., Brecher, K., & Slater, T. F. (2006). Development and Validation of the Light and Spectroscopy Concept Inventory. Astronomy Education Review, 5(2), 103– 113. https://doi.org/10.3847/AER2006020 Barradas-Solas, F., & Sánchez Gómez, P. J. (2014). Orbitals in chemical education. An analysis through their graphical representations. Chemistry Education Research and Practice, 15(3), 311–319. https://doi.org/10.1039/C4RP00023D Becker, N. M., & Cooper, M. M. (2014). College chemistry students’ understanding of potential energy in the context of atomic-molecular interactions. Journal of Research in Science Teaching, 51(6), 789–808. https://doi.org/10.1002/tea.21159 Becker, N. M., Rupp, C. A., & Brandriet, A. (2017). Engaging students in analyzing and interpreting data to construct mathematical models: an analysis of students’ reasoning in a method of initial rates task. Chemistry Education Research and Practice, 18(4), 798– 810. https://doi.org/10.1039/C6RP00205F Becker, N., Noyes, K., & Cooper, M. (2016). Characterizing students’ mechanistic reasoning about London Dispersion Forces. Journal of Chemical Education, 93(10), 1713–1724. https://doi.org/10.1021/acs.jchemed.6b00298 Bodner, G. M. (1986). Constructivism: A theory of knowledge. Journal of Chemical Education, 63, 873–878. https://doi.org/10.1021/ed063p873 Boo, H. K. (1998). Students’ understandings of chemical bonds and the energetics of chemical reactions. Journal of Research in Science Teaching, 35, 569–581. https://doi.org/10.1002/(SICI)1098-2736(199805)35:5<569::AID-TEA6>3.0.CO;2-N Bretz, S. L., & Murata Mayo, A. V. (2018). Development of the Flame Test Concept Inventory: Measuring Student Thinking about Atomic Emission. Journal of Chemical Education, 95(1), 17–27. https://doi.org/10.1021/acs.jchemed.7b00594 164 Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2, 141–178. Bryfczynski, S. P. (2012). BeSocratic: An intelligent tutoring system for the recognition, evaluation, and analysis of free-form student input (Doctoral Dissertation). Clemson University. (UMI No. 3550201). Chi, M. T. H. (2008). Three types of conceptual change: Belief revision, mental model transformation, and categorical shift. In S. Vosniadou (Ed.), Handbook of research on conceptual change. Hillsdale, NJ: Erlbaum. Clark, D., & Linn, M. C. (2003). Designing for Knowledge Integration: The Impact of Instructional Time. Journal of the Learning Sciences, 12(4), 451–493. https://doi.org/10.1207/S15327809JLS1204_1 Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2 edition). Hillsdale, N.J: Routledge. Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New Directions in Educational Technology (pp. 15–22). https://doi.org/10.1007/978-3- 642-77750-9_2 Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design Research: Theoretical and Methodological Issues. Journal of the Learning Sciences, 13(1), 15–42. https://doi.org/10.1207/s15327809jls1301_2 Confrey, J. (1990). A review of the research on student conceptions in mathematics, science, and programming. Review of Research in Education, 16, 3–56. Cooper, M. M., Corley, L. M., & Underwood, S. M. (2013). An investigation of college chemistry students’ understanding of structure–property relationships. Journal of Research in Science Teaching, 50(6), 699–721. https://doi.org/10.1002/tea.21093 Cooper, M. M., & Klymkowsky, M. W. (2013). Chemistry, Life, the Universe and Everything: A new approach to general chemistry, and a model for curriculum reform. Journal of Chemical Education, 90, 1116–1122. https://doi.org/10.1020/ed300456y Cooper, M. M., & Klymkowsky, M. W. (2015). CLUE: Chemistry, Life, the Universe & Everything. Retrieved July 18, 2017, from Chemistry, Life, the Universe and Everything website: http://clue.chemistry.msu.edu/ Cooper, M. M., Klymkowsky, M. W., & Becker, N. M. (2014). Energy in Chemical Systems: An Integrated Approach. In R. F. Chen, A. Eisenkraft, D. Fortus, J. Krajcik, K. Neumann, J. Nordine, & A. Scheff (Eds.), Teaching and Learning of Energy in K – 12 Education (pp. 301–316). https://doi.org/10.1007/978-3-319-05017-1_17 165 Cooper, M. M., Kouyoumdjian, H., & Underwood, S. M. (2016). Investigating Students’ Reasoning about Acid–Base Reactions. Journal of Chemical Education, 93(10), 1703– 1712. https://doi.org/10.1021/acs.jchemed.6b00417 Cooper, M. M., Posey, L. A., & Underwood, S. M. (2017). Core ideas and topics: Building up or drilling down? Journal of Chemical Education, 94(5), 541–548. https://doi.org/10.1021/acs.jchemed.6b00900 Cooper, M. M., Underwood, S. M., Hilley, C. Z., & Klymkowsky, M. W. (2012). Development and assessment of a molecular structure and properties learning progression. Journal of Chemical Education, 89, 1351–1357. https://doi.org/10.1021/ed300083a Cooper, M. M., Williams, L. C., & Underwood, S. M. (2015). Student Understanding of Intermolecular Forces: A Multimodal Study. Journal of Chemical Education, 92(8), 1288–1298. https://doi.org/10.1021/acs.jchemed.5b00169 Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: Sage Publications. Crandell, O. M., Kouyoumdjian, H., Underwood, S. M., & Cooper, M. M. (2018). Reasoning about Reactions in Organic Chemistry: Starting It in General Chemistry. Journal of Chemical Education. https://doi.org/10.1021/acs.jchemed.8b00784 Dangur, V., Avargil, S., Peskin, U., & Judy Dori, Y. (2014). Learning quantum chemistry via a visual-conceptual approach: students’ bidirectional textual and visual understanding. Chemistry Education Research and Practice, 15(3), 297–310. https://doi.org/10.1039/C4RP00025K Deslauriers, L., & Wieman, C. (2011). Learning and retention of quantum concepts with different teaching methods. Physical Review Special Topics - Physics Education Research, 7(1). https://doi.org/10.1103/PhysRevSTPER.7.010101 deSouza, R. T., & Iyengar, S. S. (2013). Using Quantum Mechanics To Facilitate the Introduction of a Broad Range of Chemical Concepts to First-Year Undergraduate Students. Journal of Chemical Education, 90(6), 717–725. https://doi.org/10.1021/ed400015y Didiş, N., Eryılmaz, A., & Erkoç, Ş. (2014). Investigating students’ mental models about the quantization of light, energy, and angular momentum. Physical Review Special Topics - Physics Education Research, 10(2). https://doi.org/10.1103/PhysRevSTPER.10.020127 Didiş Körhasan, N., & Wang, L. (2016). Students’ mental models of atomic spectra. Chemistry Education Research and Practice, 17(4), 743–755. https://doi.org/10.1039/C6RP00051G diSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction, 10(2 & 3), 105–225. 166 DiSessa, A. A. (2008). A Bird’s-Eye View of the “Pieces” vs. “Coherence” Controversy (From the “Pieces” Side of the Fence). diSessa, A.A. (2006). A History of Conceptual Change Research: Threads and Fault Lines. In K. R. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 265–282). Cambridge: Cambridge University Press. diSessa, Andrea A. (1988). Knowledge in pieces. In Forman, George E. & Pufall, Peter B. (Eds.), Constructivism in the computer age (pp. 49–70). Hillsdale, NJ: Lawrence Erlbaum Associates. Duschl, R., Maeng, S., & Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis. Studies in Science Education, 47(2), 123–182. Feynman, R. P., Leighton, R. B., & Sands, M. (2011). The Feynman Lectures on Physics, Vol. I: The New Millennium Edition: Mainly Mechanics, Radiation, and Heat. Basic Books. Greca, I. M., & Freire, O. (2014). Teaching introductory quantum physics and chemistry: caveats from the history of science and science teaching to the training of modern chemists. Chemistry Education Research and Practice, 15(3), 286–296. https://doi.org/10.1039/C4RP00006D Green, S., & Salkind, N. (2016). Using SPSS for Windows and Macintosh. Pearson. Hammer, D. (2000). Student resources for learning introductory physics. American Journal of Physics, 68, S52–S59. https://doi.org/10.1119/1.19520 Hammer, D., & Elby, A. (2003). Tapping Epistemological Resources for Learning Physics. Journal of the Learning Sciences, 12(1), 53–90. https://doi.org/10.1207/S15327809JLS1201_3 Honey, M., Pearson, G., & Schweingruber, H. (2014). Implications of the Research for Designing Integrated STEM Experiences (pp. 77–105). National Academies Press. Ivanjek, L., Shaffer, P. S., McDermott, L. C., Planinic, M., & Veza, D. (2014). Research as a guide for curriculum development: An example from introductory spectroscopy. I. Identifying student difficulties with atomic emission spectra. American Journal of Physics, 83(1), 85–90. https://doi.org/10.1119/1.4901977 Ivanjek, L., Shaffer, P. S., McDermott, L. C., Planinic, M., & Veza, D. (2015). Research as a guide for curriculum development: An example from introductory spectroscopy. II. Addressing student difficulties with atomic emission spectra. American Journal of Physics, 83(2), 171–178. https://doi.org/10.1119/1.4902222 167 Johnston, I. D., Crawford, K., & Fletcher, P. R. (1998). Student difficulties in learning quantum mechanics. International Journal of Science Education, 20(4), 427–446. https://doi.org/10.1080/0950069980200404 Johnstone, A. H. (1991). Why is science difficult to learn? Things are seldom what they seem. Journal of Computer Assisted Learning, 7(2), 75–83. https://doi.org/10.1111/j.1365- 2729.1991.tb00230.x Justi, R., & Gilbert, J. (2000). History and philosophy of science through models: some challenges in the case of “the atom.” International Journal of Science Education, 22(9), 993–1009. https://doi.org/10.1080/095006900416875 Kalkanis, G., Hadzidaki, P., & Stavrou, D. (2003). An instructional model for a radical conceptual change towards quantum mechanics concepts. Science Education, 87(2), 257– 280. https://doi.org/10.1002/sce.10033 Kararo, A. T., Colvin, R. A., Cooper, M. M., & Underwood, S. M. (2018). Making Predictions and Constructing Explanations: An investigation into introductory chemistry students’ understanding of structure-property relationships. Chemistry Education Research and Practice. https://doi.org/10.1039/C8RP00195B Kohn, K. P., Underwood, S. M., Cooper, M. M., & Loertscher, J. (2018). Connecting Structure– Property and Structure–Function Relationships across the Disciplines of Chemistry and Biology: Exploring Student Perceptions. CBE—Life Sciences Education, 17(2), ar33. https://doi.org/10.1187/cbe.18-01-0004 Krajcik, J. S., Slotta, J. D., McNeill, K. L., & Reiser, B. J. (2008). Designing Learning Environments to Support Students’ Integrated Understanding. In Designing coherent science education: Implications for curriculum, instruction, and policy (pp. 39–64). Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press: Chicago. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. Linn, M. C. (2006). The Knowledge Integration Perspective on Learning and Instruction. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 243–264). Retrieved from http://dx.doi.org/10.1017/CBO9780511816833.016 Linn, M. C., Bell, P., & Hsi, S. (1998). Using the Internet to Enhance Student Understanding of Science: The Knowledge Integration Environment. Interactive Learning Environments, 6(1–2), 4–38. https://doi.org/10.1076/ilee.6.1.4.3606 Linn, M. C., & Eylon, B.-S. (2011). Science Learning and Instruction : Taking Advantage of Technology to Promote Knowledge Integration. https://doi.org/10.4324/9780203806524 168 Liu, O. L., Lee, H.-S., Hofstetter, C., & Linn, M. (2008). Assessing Knowledge Integration in Science: Construct, Measures, and Evidence. Educational Assessment, 13(1), 33–55. https://doi.org/10.1080/10627190801968224 McNeill, K. L., & Krajcik, J. (2008). Inquiry and scientific explanations: Helping students use evidence and reasoning. Science as Inquiry in the Secondary Setting, 121–134. Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A Brief Introduction to Evidence-centered Design. ETS Research Report Series, 2003(1), 1–29. https://doi.org/10.1002/j.2333- 8504.2003.tb01908.x Mislevy, R. J., & Haertel, G. D. (2006). Implications of Evidence-Centered Design for Educational Testing. Educational Measurement: Issues and Practice, 25(4), 6–20. https://doi.org/10.1111/j.1745-3992.2006.00075.x Moon, A., Zotos, E., Finkenstaedt-Quinn, S., Ruggles Gere, A., & Shultz, G. (2018). Investigation of the role of writing-to-learn in promoting student understanding of light– matter interactions. Chemistry Education Research and Practice, 19(3), 807–818. https://doi.org/10.1039/C8RP00090E Mowry, C., Milofsky, R., Collins, W., & Pimentel, A. S. (2017). Laser-Induced Breakdown Spectroscopy for Qualitative Analysis of Metals in Simulated Martian Soils. Journal of Chemical Education, 94(10), 1507–1511. https://doi.org/10.1021/acs.jchemed.7b00133 Murray, F. B., Hufnagel, P., Gruber, H. E., & Vonèche, J. (1979). The Essential Piaget. Educational Researcher, 8(11), 20. https://doi.org/10.2307/1174291 Nakiboglu, C. (2003). Instructional Misonceptions of Turkish Prospective Chemistry Teachers About Atomic Orbitals and Hybridization. Chem. Educ. Res. Pract., 4(2), 171–188. https://doi.org/10.1039/B2RP90043B National Research Council. (2000). How People Learn: Brain, Mind, Experience, and School: Expanded Edition. https://doi.org/10.17226/9853 National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8 (Duschl, R. A., H. A. Schweingruber, & Shouse, A. W., Eds.). Washington, DC: National Academies Press. National Research Council. (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. https://doi.org/10.17226/13165 National Research Council, E., and Medicine. (2018). How People Learn II: Learners, Contexts, and Cultures. https://doi.org/10.17226/24783 169 Niaz, M., & Fernández, R. (2008). Understanding Quantum Numbers in General Chemistry Textbooks. International Journal of Science Education, 30(7), 869–901. https://doi.org/10.1080/09500690701217337 Nilsson, T., & Niedderer, H. (2014). Undergraduate students’ conceptions of enthalpy, enthalpy change and related concepts. Chemistry Education Research and Practice, 15(3), 336– 353. https://doi.org/10.1039/C2RP20135F Orey, M. (2010). Emerging Perspectives on Learning, Teaching, and Technology. Özmen, H. (2004). Some student misconceptions in chemistry: A literature review of chemical bonding. Journal of Science Education and Technology, 13, 147–159. Park, E. J., & Light, G. (2009). Identifying Atomic Structure as a Threshold Concept: Student mental models and troublesomeness. International Journal of Science Education, 31(2), 233–258. https://doi.org/10.1080/09500690701675880 Pepper, R. E., Chasteen, S. V., Pollock, S. J., & Perkins, K. K. (2012). Observations on student difficulties with mathematics in upper-division electricity and magnetism. Physical Review Special Topics - Physics Education Research, 8(1). https://doi.org/10.1103/PhysRevSTPER.8.010111 Piaget, J. (1964). Part I: Cognitive development in children: Piaget development and learning. Journal of Research in Science Teaching, 2(3), 176–186. Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66(2), 211–227. https://doi.org/10.1002/sce.3730660207 Rittenhouse, R. C. (2015). Understanding Atomic Structure: Is There a More Direct and Compelling Connection between Atomic Line Spectra and the Quantization of an Atom’s Energy? Journal of Chemical Education, 92(6), 1035–1039. https://doi.org/10.1021/ed5007234 Sánchez Gómez, P. J., & Martín, F. (2003). Quantum vs “Classical” Chemistry in University Chemistry Education: A Case Study of the Role of History in Thinking the Curriculum. Chemistry Education Research and Practice, 4(2), 131–148. https://doi.org/10.1039/b2rp90042d Savall-Alemany, F., Domènech-Blanco, J. L., Guisasola, J., & Martínez-Torregrosa, J. (2016). Identifying student and teacher difficulties in interpreting atomic spectra using a quantum model of emission and absorption of radiation. Physical Review Physics Education Research, 12(1), 010132. https://doi.org/10.1103/PhysRevPhysEducRes.12.010132 Singh, C. (2001). Student understanding of quantum mechanics. American Journal of Physics, 69(8), 885–895. https://doi.org/10.1119/1.1365404 170 Smith, C. L., Wiser, M., Anderson, C. W., & Krajcik, J. (2006). FOCUS ARTICLE: Implications of Research on Children’s Learning for Standards and Assessment: A Proposed Learning Progression for Matter and the Atomic-Molecular Theory. Measurement: Interdisciplinary Research & Perspective, 4(1–2), 1–98. https://doi.org/10.1080/15366367.2006.9678570 Stefani, C., & Tsaparlis, G. (2009). Students’ levels of explanations, models, and misconceptions in basic quantum chemistry: A phenomenographic study. Journal of Research in Science Teaching, 46(5), 520–536. https://doi.org/10.1002/tea.20279 Taber, K. S. (2002a). Compounding Quanta: Probing the Frontiers of Student Understanding of Molecular Orbitals. Chemistry Education Research and Practice, 3(2), 159–173. https://doi.org/10.1039/B2RP90013K Taber, K. S. (2002b). Conceptualizing Quanta: Illuminating the Ground State of Student Understanding of Atomic Orbitals. Chemistry Education Research and Practice, 3(2), 145–158. https://doi.org/10.1039/B2RP90012B Taber, K. S. (2003). Understanding Ionisation Energy: Physical, Chemical and Alternative Conceptions. Chemistry Education Research and Practice, 4(2), 149–169. https://doi.org/10.1039/B3RP90010J Taber, K. S. (2005). Learning quanta: Barriers to stimulating transitions in student understanding of orbital ideas. Science Education, 89(1), 94–116. https://doi.org/10.1002/sce.20038 Teichert, M. A., & Stacy, A. M. (2002). Promoting understanding of chemical bonding and spontaneity through student explanation and integration of ideas. Journal of Research in Science Teaching, 39, 464–496. https://doi.org/10.1002/tea.10033 Toulmin, S. (1975). Human Understanding. Mind, 84(334), 299–304. Tsaparlis, G. (2014). Linking the Macro with the Submicro Levels of Chemistry: Demonstrations and Experiments that can Contribute to Active/Meaningful/Conceptual Learning. In Learning with understanding in the chemistry classroom (pp. 41–61). https://doi.org/10.1007/978-94-007-4366-3_3 Tsaparlis, G. (2016). The logical and psychological structure of physical chemistry and its relevance to graduate students’ opinions about the difficulties of the major areas of the subject. Chemistry Education Research and Practice, 17(2), 320–336. https://doi.org/10.1039/C5RP00203F Tsaparlis, G., & Papaphotis, G. (1997). Atomic orbitals, molecular orbitals and related concepts: Conceptual difficulties among chemistry students. Research in Science Education, 27(2), 271–287. https://doi.org/10.1007/BF02461321 171 Tsaparlis, G., & Papaphotis, G. (2002). Quantum-Chemical Concepts: Are They Suitable for Secondary Students? Chemistry Education Research and Practice, 3(2), 129–144. https://doi.org/10.1039/B2RP90011D Tsaparlis, G., & Papaphotis, G. (2009). High‐school Students’ Conceptual Difficulties and Attempts at Conceptual Change: The case of basic quantum chemical concepts. International Journal of Science Education, 31(7), 895–930. https://doi.org/10.1080/09500690801891908 Underwood, S. M., Reyes-Gastelum, D., & Cooper, M. M. (2015). Answering the questions of whether and when student learning occurs: Using discrete-time survival analysis to investigate how college chemistry students’ understanding of structure-property relationships evolves. Science Education, 99, 1055–1072. https://doi.org/10.1002/sce.21183 Underwood, S. M., Reyes-Gastelum, D., & Cooper, M. M. (2016). When do students recognize relationships between molecular structure and properties? A longitudinal comparison of the impact of traditional and transformed curricula. Chemistry Education Research and Practice, 17(2), 365–380. https://doi.org/10.1039/C5RP00217F Van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in Teacher–Student Interaction: A Decade of Research. Educational Psychology Review, 22(3), 271–296. https://doi.org/10.1007/s10648-010-9127-6 Vosniadou, Stella. (1994). Capturing and modeling the process of conceptual change. Learning and Instruction, 4(1), 45–69. https://doi.org/10.1016/0959-4752(94)90018-3 Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes (Revised ed. edition; M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds.). Cambridge, Mass.: Harvard University Press. Wenger, E. (1998). Communities of practice: learning, meaning, and identity. New York, NY: Cambridge University Press. Williams, L. C., Underwood, S. M., Klymkowsky, M. W., & Cooper, M. M. (2015). Are noncovalent interactions an Achilles heel in chemistry education? A comparison of instructional approaches. Journal of Chemical Education, 92, 1979–1987. https://doi.org/10.1021/acs.jchemed.5b00619 Zollman, D. A., Rebello, N. S., & Hogg, K. (2002). Quantum mechanics for everyone: Hands-on activities integrated with technology. American Journal of Physics, 70(3), 252–259. https://doi.org/10.1119/1.1435347 172