STUDENT UNDERSTANDING OF STRUCTURE - PROPERTY RELATIONSHIPS AND THE ROLE OF INTERMOLECULAR FORCES By Leah Corley Williams A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Chemis try - Doctor of Philosophy 2015 ABSTRACT STUDENT UNDERSTANDING OF STRUCTURE - PROPERTY RELATIONSHIPS AND THE ROLE OF INTERMOLECULAR FORCES By Leah Corley Williams The connection between the molecular - level structure of a substance and its physical and c hemical properties (such as boiling point or relative acidity) is an integral chemistry concept and a thorough understanding of this relationship is key to understanding larger and more complex chemistry ideas. Previous research has shown that students pos sess a wide range of non - normative ideas about chemical and physical properties. Student difficulties with the connection between a chemical structure and the properties of the compound, however, are far more complex than a series of misconceptions. Using a qualitative approach, we interviewed seventeen students enrolled in either general or organic chemistry courses. We found that, while many students could correctly predict and rank melting and boiling points of various compounds, few successfully used th e molecular level structure of each compound to predict and explain its properties. Instead, we identified several emergent themes that categorize the ways in which students tried to explain these trends. While some students discussed similar individual id eas, no two students connected these ideas in the same manner, resulting in a wide range of interconnected, albeit fragmented, ideas. Intermolecular forces (IMFs), the forces that govern interactions between molecules based on differences in polarity and electronegativity, play an important role in this connection between structure and properties. Because few students discussed IMFs when describing the connection between structure and properties in our interviews, we designed the Intermolecular Forces Asse questions requiring both written responses and drawn representations. This assessment was given to several groups of students at various time points at two different universities. We found that with their provided representations and analyzing the writing alone would have given a false impression of their understanding. Constructed representations, however, often provided crucial spatial information needed to determine if the students understood IMFs as occurring between molecules rather than wi thin molecules. Using the IMFA, we also studied the effect of an alternative general chemistry of IMFs. Using two matched groups of students, those enrolled in t he CLUE course and those enrolled in a traditional general chemistry course, we found that CLUE students most often correctly represented IMFs as occurring between molecules unlike the majority of students enrolled in traditional general chemistry courses who incorrectly represented IMFs as forces within a single molecule . We replicated these findings with an additional cohort of students the following year and have preliminary data that suggest these results extend to an additional university and are more generalizable. understanding, analysis of student drawings is not always practical in terms of assessment. We onses to explore its student would receive for their corresponding con structed representation of a specific IMF. iv This dissertation is dedicated to my loving husband Benjamin Williams. He was there to see me off as I started my graduate school journey in South Carolina, he supported me from afar when my educ ational pursuits sent me to Michigan, and he has patiently listened to me bemoan writing this dissertation (even though I am certain that my whining was extremely annoying). He has provided unwavering love, support, and understanding throughout this entire process and I am forever grateful. I would also like to dedicate this dissertation to my wonderful family. I certainly would not be the person I am today if not for them. They have always supported my pursuits, academic or otherwise, and have provided lov e and guidance in the best and worst of times. This v ACKNOWLEDGEMENTS I would like to first and foremost thank my advis or, Dr. Melanie Cooper, for her encouragement and guidance over the past five years. She demands only the best of her s tudents and, while she sets the bar high, she always provides unwavering support and understanding. I am extremely grateful to have had her as my mentor. I would like to acknowledge my second reader, Dr. Lynmarie Posey, for all of her wonderful feedback an d helpful edits . I would also like to thank my other committee members, Dr. Joe Krajcik and Dr. Gary Blanchard for agreeing to fill in where my Clemson committee left off . I owe a large thank you to my Clemson committee members and supporters: Dr. Gautam B hattacharyya, Dr. Geoff Potvin, Dr. Lisa Benson, and Dr. Zahra Hazari. I would like to acknowledge all of our past group members at Clemson who helped guide me through my early years in ChemEd and taught me the ropes. At the same time, r current group members who have provided wonderful insight and discussion, while including a healthy dose of laughter. The work presented here would not have been possible if not for the wonderful Barbara Lewis who gave us more free reign than any sane person would allow. To Sonia gether, written together, travel led together, and formed a close bond that has extended outside of the office. I will be forever grateful for your guidance. Thank you to my Clemson ladies, Sara, Melissa, and Evy, who were always there with nail polish, crafts, and The Bachelor when work was too much. Lastly I owe a big thank you to Katie and Kate. I would be lying if I s aid moving to Michigan was easy. But I know for a fact that your friendship (game nights, paintings, and chats at HopCat included) made Michigan feel like home away from home. vi TABLE OF CONTENTS LIST OF TABLES ................................ ................................ ................................ ...................... xi LIST OF FIGURES ................................ ................................ ................................ .................. xiv CHAPTER I: INTRODU CTION ................................ ................................ ................................ .. 1 Main goals and research questions ................................ ................................ ............. 2 REFERENCES ................................ ................................ ................................ ........... 7 CHAPTER II: THEORETICAL FRAMEWORKS ................................ ................................ ...... 9 Constructivism and meaningful learning ................................ ................................ .... 9 Identifying prior knowledge: Moving beyond misconceptions to p - prims .............. 13 How we learn: Working memory and info rmation processing ................................ . 14 How we reason and make decisions: Heur istics and dual - process theory ................ 16 Difficulties in the chemistry classroo m: Representational competency ................... 20 Concluding remarks ................................ ................................ ................................ .. 21 REFERENCES ................................ ................................ ................................ ......... 24 CHAPTER III: REVIEW OF THE LITERATURE ON STR UCTURE - PROPERTY RELATIONSHIPS AND INTERMOLECULAR FORCES ................................ .... 28 The relationship between structure and property ................................ ...................... 28 ysical and chemical properties ............................ 30 Intermolecular forces ................................ ................................ ................................ 31 Studies exploring s MFs ................................ ........... 33 bonding ................................ ................................ ................................ .......... 33 Student diff iculties with hydrogen bonding ................................ ........................ 34 and multiple - choice i tems ................................ ................................ .............. 35 Drawing intermolecular forces ................................ ................................ ........... 37 Intermolecular forces and their relat ionship to physical properties .................... 38 Attempts to improve s ................................ ............ 40 Thinking bigger: Addressing the relationship between structure and pr operties ................................ ................................ ................................ ........ 42 REFERENCES ................................ ................................ ................................ ......... 46 CHAPTER UNDERSTANDING OF STRUCTURE - PROPERTY RELATIONSHIPS ................................ ................................ ................................ .... 50 Preface ................................ ................................ ................................ ....................... 50 Introduction ................................ ................................ ................................ ............... 50 Background Misconceptions, conceptual change, and dual processing ................ 51 M isconceptions ................................ ................................ ................................ ... 51 vii Conceptual change ................................ ................................ .............................. 53 Heuristics and dual processing ................................ ................................ ............ 54 Purpose and significance of the study ................................ ................................ ....... 57 Methods ................................ ................................ ................................ ..................... 59 Settings and participants ................................ ................................ ..................... 59 Interview protocol ................................ ................................ ............................... 61 Data collections ................................ ................................ ................................ ... 62 Data analysis ................................ ................................ ................................ ....... 62 Findings ................................ ................................ ................................ ..................... 63 Emerg - property relationship s ................................ ................................ ................................ ... 63 Mod els of phase and phase changes ................................ ................................ ... 64 Use of representations ................................ ................................ ......................... 66 Language and terminology ................................ ................................ ................. 68 Use of h euristics in student reasoning ................................ ................................ 72 Heuris tics Instru ctionally derived ................................ ................................ .... 73 Heuristics Personal ................................ ................................ ........................... 73 st ructure - property relationships ................................ ................................ ..... 75 Discussion ................................ ................................ ................................ .................. 79 Conclusions ................................ ................................ ................................ ................ 81 Questions and implications for teaching ................................ ............................. 83 Limitations of this stud y ................................ ................................ ..................... 86 REFERENCES ................................ ................................ ................................ ........... 87 CHAPTER V: DESIGN AND VALIDATION OF THE INTERMOLECULAR FORCES ASSESSMENT ................................ ................................ ......................... 92 IMFs interviews: Protocol and findings ................................ ................................ .... 92 Designing the initial version of the Intermolecular Forces Assessment (IMFA) ................................ ................................ ................................ ................ 95 Redesigning the Intermol e cular Forces Assessment (IMFA) ................................ ... 98 Pil ot testing the redesigned IMFA ................................ ................................ .......... 101 Finalizing the IMFA design ................................ ................................ .................... 101 IMFA coding and validation ................................ ................................ ................... 104 Student drawings (Items 7 - 9) ................................ ................................ ............ 104 Student text (Items 2, 4 - 6) ................................ ................................ ................ 109 IMFA data collections ................................ ................................ ............................. 111 REFERENCES ................................ ................................ ................................ ....... 112 CHAPTER VI: STUDENT UNDERSTANDING OF INTERMOLECULAR FORCES: A MULTIMODAL STUDY ................................ ................................ ................... 114 Preface ................................ ................................ ................................ ..................... 114 Introduction ................................ ................................ ................................ ............. 115 The impor tance of intermolecular forces ................................ .......................... 115 Prior work on student understanding of IMFs ................................ .................. 117 Theoretical perspective ................................ ................................ ..................... 119 viii Met hods ................................ ................................ ................................ ................... 120 Student population ................................ ................................ ............................ 120 Development of the Intermole cular Forces Assessment (IMFA) ..................... 121 Data analysis: Drawings (Items 6 - 9 ) ................................ ................................ . 123 Data analysis: Text responses (Items 2, 4 - 6) ................................ .................... 126 Results and discussion ................................ ................................ ............................ 128 RQ1: How do students represe nt IMFs in free - form drawings? ....................... 128 RQ2: How do students discuss and describe IMFs in open - ended written responses? ................................ ................................ ................................ .... 130 representations? ................................ ................................ ............................ 133 Summary ................................ ................................ ................................ ................. 137 Conclusions ................................ ................................ ................................ ............. 139 Implication s for teaching and future work ................................ ........................ 140 Limitations of the study ................................ ................................ .................... 142 REFERENCES ................................ ................................ ................................ ....... 144 CHAPTER VII: ARE NON - COVALENT INTERACTIONS AN ACHILLES HEEL IN CHEMISTRY? A COMPARISON OF INSTRUCTIONAL APPROACHES ................................ ................................ ................................ ....... 148 Introduction ................................ ................................ ................................ ............. 148 Intermo lecular forces ................................ ................................ ........................ 149 Developing improved understa nding in a reformed curriculum ....................... 151 Research questions ................................ ................................ ............................ 152 Methods ................................ ................................ ................................ ................... 152 Student population s and study design ................................ ............................... 152 Study 1: A comparison of CLUE and traditional students at Clemson University ................................ ................................ ................................ ..... 154 Study 2: A comparison of CLUE and traditional students from different Universities ................................ ................................ ................................ .. 156 Study 3: A longitudinal study of stu ................ 156 Data coding and analysis ................................ ................................ .................. 157 Results and discussion ................................ ................................ ............................ 158 Study 1: Resul ts and discussion Clemson University, Cohorts 1 and 2, GC2 ................................ ................................ ................................ .............. 158 Consist ency of responses for Cohort 1 ................................ ............................. 161 Study 2: Results and discussion Michigan State University, Cohort 3, end of GC2 ................................ ................................ ................................ ... 163 Study 3: Longitudinal study results an d discussion Clemson, Cohort 1, GC2 through OC2 ................................ ................................ ........................ 166 Conclusions ................................ ................................ ................................ ............. 169 Limitations ................................ ................................ ................................ ........ 170 Implications for teachings ................................ ................................ ................. 170 REFERENCES ................................ ................................ ................................ ....... 172 ix HYDROGEN BONDING ................................ ................................ ....................... 176 Introduction ................................ ................................ ................................ ............. 176 Automated computer scoring ................................ ................................ ............ 179 Purpose of study ................................ ................................ ................................ 181 Methods ................................ ................................ ................................ ................... 183 Lexical category creation ................................ ................................ .................. 184 Applying discriminant analysis ................................ ................................ ......... 187 Categorizin g responses using web diagrams ................................ .................... 188 Results and discussion ................................ ................................ ............................ 189 Using discriminant analysis with lexical categories to predict the hydrogen bonding drawing code ................................ ................................ .. 189 Exploring differences in CLUE and traditional student responses using web diagrams ................................ ................................ ............................... 194 Using discriminant analysis of student responses to predict GC course enrollment ................................ ................................ ................................ .... 199 Conclusions ................................ ................................ ................................ ............. 202 Limitations ................................ ................................ ................................ ........ 205 REFERENCES ................................ ................................ ................................ ....... 206 CHAPTER IX: CONCLUSIONS, IMPLICATIONS, AND FUTURE WORK ...................... 210 Conclusions ................................ ................................ ................................ ............. 210 Main s ructure - property relationships ..... 210 Main s rstanding of intermolecular forces through writing and drawing ................................ ................................ ................................ .. 212 Main s understa nding of intermolecular forces ................................ ....................... 213 Main s tudy 4: Automated text analysis of IMFA respo ns es ............................. 215 Implications ................................ ................................ ................................ ............. 216 The positive impact of the CLUE curriculum: Scaffolding structure - property concepts ................................ ................................ ................................ ........ 216 The positive impact of the CLUE curriculum: Engaging in scientific practices ................................ ................................ ................................ ....... 222 Implications for assessment ................................ ................................ .............. 223 F uture work ................................ ................................ ................................ ............. 225 REFERENCES ................................ ................................ ................................ ....... 228 APPENDICES ................................ ................................ ................................ .......................... 231 Appendix A: Copy of permissions from the Journal of Research in Science Teaching ................................ ................................ ................................ ............ 232 Appendix B: Full structure - property interview protocol ................................ ........ 239 Appendix C: Additional examples of structure - property interview themes ........... 240 ns and representations of IMFs ......... 243 Appendix E: responses ................................ ................................ ................................ ........... 245 Appendix F: dipole - dipole interactions and x London dispersion forces ................................ ................................ .................. 247 Appendix G: theses and dissertations ................................ ................................ ..................... 249 Appendix H: C Agreement ................................ ................................ ................................ ......... 25 0 Appendix I: Demographic and pre - instruction assessment data for Cohorts 1 and 2 ................................ ................................ ................................ .................. 253 Appendix J: A comparison of all drawing and text code frequencies for Cohorts 1 and 2 ................................ ................................ ................................ ............... 255 Appendix K: Demographic and pre - instruction assessment data for CL UE and traditional students in all three cohorts as well as statistical comparisons for Cohorts 1 and 2 ................................ ................................ ................................ . 256 Appendix L: Chi - square analyses of all drawing code frequencies for CLUE and tradition al students in Cohorts 1 and 2 ................................ ............................. 260 Appendix M: ng codes across all three IMFs ........ 261 Appendix N: Expansion of th ................................ ..... 262 REFERENCES ................................ ................................ ................................ ....... 263 xi LIST OF TABLES Table 4.1 : Pairs of compounds presented to students, the reasons for choosing this co mparison, a nd the expected student reasoning ................................ ................ 62 Table 4.2 : at arose during their interview ................................ ................................ ................................ ............. 64 Table 4.3 : Student prediction for highest boiling point in each comparison from the second half o f the interview protocol ................................ ................................ .............. 79 Table 6.1 : Coding examples for student drawings demonstrating understanding of s elected types of intermolecular f orces ................................ ................................ ........... 125 Table 6.2 : Examples of text codes a pplied to Cohort 1 esponses for e ach IMF ................................ ................................ ................................ ................... 127 T able 6.3 : Examples of terminology issues in student r esponses ................................ ...... 133 Table 6.4 : Drawing and text c omparisons for Tobias and Maeby ................................ ..... 135 Table 6.5 : Drawing and text comparisons for Oscar and Rita ................................ ........... 136 Table 6.6 : Drawing and text c omparisons for Trisha and Gene ................................ ........ 137 Table 7.1 : Student populations for each cohort at Clemson University and Michigan State University ................................ ................................ ................................ .......... 154 Table 7.2 : Coding examples for student drawings of selected types of intermolecular f orces ................................ ................................ ................................ ................. 157 Table 7.3 : Statistical results for compar ing code frequencies for CLUE and Traditional and Cohort 2 .............. 161 Table 7.4 : Statistical results for longitudinal comparison of code frequencies for CLUE and Traditional stude g drawings ................................ ..... 167 Table 8.1 : Most commonly assigned lexical categories (those applied to g reater than 20% of the total sample) ................................ ................................ ............................ 190 Table 8.2 : Lexical categories used as standardized canonical discrim inant function coefficients for each drawing cod e model ................................ ........................ 192 xii Table 8.3 : Agreement and classifications of the between and within drawing code models ................................ ................................ ................................ ............... 194 Table 8.4 : traditional GC ................................ ................................ ................................ .... 200 Table 8.5 : Classification and agreeme nt for the GC course model ................................ .... 201 Table 9.1: Table of contents for the material covered in the first semester of each general chemistry course ................................ ................................ ............................... 218 Table 9.2: Table of contents for the material covered in the second semester of each general chemistry course ................................ ................................ ................... 219 Table A.1 : Written descriptions and drawn representatio ns of IMFs from our interviews ................................ ................................ ................................ .......... 243 Table A.2 : Sex and most common majors for Cohor t 1 and Cohort 2 ............................... 253 Table A.3 : Pre - Instruction scores on SAT composite, MCAI, and SUMS for Cohorts 1 and 2 ................................ ................................ ................................ .................. 253 Table A.4 : Pre - Instruction performance on all 17 I IL SI items for Cohorts 1 and 2 ........... 254 Table A.5 : Sex and most common majors for Cohort 1 traditional and CLUE groups ...... 257 Table A.6 : Pre - instruction scores on SAT composite, MCAI, and SUMS for Cohort 1 traditional and CLUE groups ................................ ................................ ............ 257 Table A.7 : Pre - instruction scores on all 17 IILSI items for Cohort 1 traditional and CLUE groups ................................ ................................ ................................ .... 257 Table A.8 : Sex and most common majors for Cohort 2 traditional and CLUE groups ...... 258 Table A.9 : Pre - instruct ion scores on SAT composite and MCAI scores for Cohort 2 traditional and CLUE groups ................................ ................................ ............ 258 Table A.10 : Pre - instruction performance on all 17 IILSI items for Cohort 2 traditional and CLUE groups ................................ ................................ ................................ .... 258 Table A.11 : Sex, most common majors, and ACT compo site scores for Cohort 3 traditional and CLUE group ................................ ................................ ............................... 259 Table A.12 : Chi - square statistical analysis results for comparing all drawing code frequencies for CLUE and Traditional students at the end of GC2 in Cohort 1 and Cohort 2 ................................ ................................ ................................ ............ 260 x iii Table A.13 : Consistency of students receiving a particular draw ing code across all three IMFs ................................ ................................ ................................ .................. 261 xiv LIST OF FIGURES Figure 1.1 : The sequence of topics required to connect a given molecular formula of a compound to its resulting physical and chemical prope rties ................................ 2 Figure 2.1 : Concept map of the req uirements for meaning learning ................................ ..... 12 Figure 2.2 : ssing model and working memory ..................... 15 Figure 2.3 : Card experiment used by Wason and Evans to explore dual processes theory ................................ ................................ ................................ .................. 18 Figure 2.4 : thought ................................ ............ 20 Figure 3.1 : Representations of (a) hydrogen bonding, (b) dipole - dipole, and (c) LDFs from General Chemistry: Atoms First by McMurry and Fay ................................ ...... 33 Figure 3.2 : three core ideas throughout the first semester ................................ ................................ ................................ ............... 44 Figure 4 .1 : phase and (d) water going from the liquid phase to the gaseous phase .............. 65 Figure 4 .2 : ( dimethyl ether ................................ ................................ ................................ ..... 67 Figure 4.3 : s depiction of hydrogen bonding in ammonia, both within a molecule and between two m olecules ................................ ................................ ....................... 70 Figure 4.4 : depiction of hydrogen bonding between two water molec ules indicated by the arrows showing an electron pair le aving and a hydrogen attaching ................... 77 Figure 4.5 : ......... 81 Figure 5.1 : nding of intermolec ular forces ........... 94 Figure 5.2 : - dipole bet ween oxygen and nitrogen atoms ..... 95 Figure 5.3 : Frequencies of students who did and did not mention IMFs in their pre - instruction in termolecular forces assessment ................................ ...................... 97 xv Figure 5.4 : An exa mple of item 7 on the IMFA ................................ ................................ .. 100 Figure 5.5 : In termolecular Forces Assessment ................................ ................................ ... 102 Figure 5.6 : IMFA data collections ................................ ................................ ....................... 103 Figure 5.7 : (a) and (b) bonding (IMFA item 7) ................................ ................................ ..................... 105 Figure 5.8 : (a) early reorganization of location codes into categories; (b) addition and organization of structure codes; (c) addition of the number of molecules codes ................................ ................................ ................................ ................. 106 Figure 5.9 : Student representations of hydrogen bonding that received (a) between code, (b) within code, (c) ambiguous code, (d) not present code, and (e) student DK code ................................ ................................ ................................ ................... 108 Figure 5.10 : coding scheme ................................ ................................ ................................ ... 110 Figure 6.1 : Items included on the In termolecular Forces Assessment ................................ 122 Figure 6.2 : Cohort 2 ................................ ................................ ................................ ............ 129 Figure 6.3 : e ntations of IMFs from Cohort 1 .......... 131 Figure 6.4 : Comparison of Cohort 1 students IMFs ................................ ................................ ................................ .................. 134 Figure 7.1 : bonding, dipole - dipole, and LDFs from Cohort 1 and Cohort 2 ...................... 159 Figure 7.2 : Flow chart representing the cons istency of code frequenci es applied to student drawings in Cohort 1, both traditional a nd CLUE, across all three IMFs ........ 162 Figure 7.3 : cohorts collected at two universiti es ................................ ................................ . 164 Figure 7.4 : Code frequencies for traditional three cohorts collected at two universities ................................ ........................ 165 Figure 7.5 : for represent ations of hydrogen bonding, dipole - dipole, and LDFs from GC2 to OC2 (Cohort 1) ................................ ................................ ................................ . 167 xvi Figure 7.6 : Code frequencies applied to Cohort 1 CLUE and traditiona representations of hydrogen bonding at the end of G C2 and again at the end of OC 2 ................................ ................................ ................................ ................... 168 Figure 8.1 : a) Examples of terms (concepts) extracted by Modeler using the Text analysis node; b) Examples of lexical categories created for terms extracted by Modeler ................................ ................................ ................................ ............. 185 Figure 8.2 : Web diagr ................................ ..................... 197 Figure 8.3 : Web diagram of t ................................ ............... 197 Figure 9.1: The progression of topics discussed in CLUE to connect molecular structure and properties ................................ ................................ ................................ .... 221 Figure A.1 : ......... 245 Figur e A.2 : Student representations of dipole - dipole that received (a) between code, (b) within code, (c) ambiguous code, (d) not present code, and (e) student DK code ................................ ................................ ................................ ............ 247 Figure A.3 : Student representations of LDFs that received (a) between code, (b) within code, (c) ambiguous code, (d) not present code, (e) student DK code, and (f) always present code ................................ ................................ ................................ ...... 248 Figure A.4 : Comparison of all drawing code frequencies for Cohort 1 and 2 ..................... 255 Figure A.5 : for all three traditional student cohorts ................................ ................................ ................................ ............... 262 1 CHAPTER I : INTRODUCTION Isolated material particles are abstraction, their properties being definable and observable only through their interaction with other systems. - Niels Bohr 1 Understanding the relationship between a structure and its resulting function is a key concept for various STEM (science, technology, eng ineering, and mathematics) disciplines. The National Research Council has repeatedly highlighted the importance of emphasizing this relationship in their previous science education standards 2 as well as listing it as one of seven crosscutting concepts in their recently released Framework for K - 12 science education. 3 This understanding of structure - property relationships plays a particularly i mportant role in the study of chemistry. Much of what we expect students to learn in their introductory chemistry courses relates to and builds off of this understanding. Specifically, the knowledge that the molecular structure can influence the properties we experience at the macroscopic level, such as boiling and melting point or reactivity, serves as a solid foundation that students can use as they expand their knowledge of chemical phenomena. Unfortunately, we know from previous research that students struggle to make this connection between the molecular structure of a substance and its resulting properties. 4 6 Understanding this relationship requires thorough knowledge of a variety of interconnected topics and rules t hat build off of each other, as shown in Figure 1.1 . 6 The complexity of this connection can m ake it difficult for students to see the molecular structure as a tool to predict and explain properties. Instead, they often resort to memorized rules and heuristics to determine the properties of a substance without considering the key underlying chemica l concepts. 4,7,8 2 Figure 1.1: The sequence of topics required to connect a given molecular formula of a compound to its resulting physical and chemical properties . Repr oduced with permission of The Royal Society of Chemistry. 6 Intermolecular forces (IMFs) pla y an important role in bridging the connection between molecular structure and the properties of a substance. Resulting properties, such as relative boiling or melting points, often depend on the type and strength of IMFs exhibited by a particular compound . These IMFs are influenced by the arrangement and electronegativity of elements within a given structure. Previous research has shown that students possess a range of alterative ideas and conceptions about IMFs, where they are located, and how they affect properties. 9 11 It should come as no surprise then that students who struggle to understand IMFs also experien ce difficulties in explaining and predicting chemical and physical properties. 4 Main G oals and Research Questions The research presented here is a continuation of a series of projects centered on understanding of the connection between structure and properties. 5,6,12,13 Our previous work has shown that students struggle to construct Lewis structures 12 and often do not understand their 3 purpose as tools to predict the properties of a given substance. 5,6 Continuing in this vein of res earch, our first main goal , explored in Study 1, was to determine if and how students connect the molecular - level structure of a substance to its properties. Specifically, our research questions were: Study 1 RQ1 . In what ways d o students use molecular - le vel structures to make predictions about the macroscopic properties of a substance? RQ2 . How do students enrolled in general and organic chemistry use representations of chemical structures to make predictions about macroscopic properties of substances? B ased on the broad nature of our research questions and the need for more detailed student responses, our data derives from a series of seventeen semi - structured interviews conducted with students enrolled in either general chemistry or organic chemistry. W e collected both audio recordings of the interviews as well as any student drawings or written work for analysis. These interviews allowed us to approach the topic from a variety of angles and ask follow - ity. Through open coding of our interview data and the refinement of our codes, we were able to identify several main themes (including representational difficulties and use of heuristics in student reasoning) that governed the ways in which students thoug ht about and discussed the relationship between structure and properties. 4 4 IMFs, however the majority of students d id not use these IMFs to help explain relevant physical and chemical properties. Instead they often relied on heuristics and memorized trends. The students who did discuss IMFs often did not elaborate on their understanding or provided superficial descript ions. Therefore, we wanted to collect responses from a larger group of students to further explore this topic as well as improve the ge neralizability of our findings. The main goal s of this research were to develop an understanding of how students discuss and represent IMFs (Study 2) as well as to study the effects of an alternative general chemistry curriculum , Chemistry, Life, the Universe, and Everything (CLUE) 14,15 , IMFs (Study 3) . Our specific research questions were: Study 2 RQ1 . How do students represent IMFs in free - form drawings? RQ2 . How do students discuss and describe IMFs in open - ended written responses? RQ3 . How do stud representations? Study 3 RQ1 enrolled in a traditional general chemistry course? RQ2 . How do stud ents at different institutions compare i n their representations of IMFs? 5 RQ3 . over the course of the subsequent organic chemistry course ? The data for Study 2 and Study 3 come from the Intermolecular Forces Assessment (IMFA), a nine question assessment requiring students to both answer open - ended questions about their understanding of IMFs as well as construct representations of specific IMFs. We developed this assessment, as outlined in Chapter 5 , based on our previous findings from Study 1 and our desire to incorporate both drawing and writing into a single assessment to explore across two universities for comparison. We also follow ed a small group of students longitudinally though two years of introductory chemistry courses, from general chemistry to organic chemistry, to study the change, or lack thereof, in their understanding of IMFs over time. rawing and writing, we were able to compare their responses in both modalities to assess the effectiveness of each in determining their understanding of molecular interactions. 9 a wea lth of information about their understanding, the process of coding hundreds of student drawn responses by hand is not entirely practical. Often instructors do not have the time or resources to dedicate to the analysis of open - ended written or drawn respon ses, thus the appeal of multiple - choice assessment items. Our third main goal for this research, explored in Study 4, was to investigate ways of expediting the analysis of IMFA written responses to glean as much information as possible without tedious and time - consuming hand coding. Specifically, our research questions were: 6 Study 4 RQ1 bonding predict the location of hydrogen bonding in their constructed representation? RQ2. W hat impact does an alternative general chemistry curriculum have on responses differentiate between students enrolled in different curricula? The data for Study 4 come from studen responses were identified using lexical analysis and then used in a text analysis software system to build a mode either the CLUE general chemistry curriculum or the traditional general chemistry cu rriculum by using discriminant analysis of their written responses. 7 REFERENCES 8 REFERENCES (1) Bohr, N. Atomic physics and the description of nature ; Cambridge University Press: London, 193 4. (2) National Research Council. National science education standards ; National Academy Press: Washington, DC, 1996. (3) National Research Council. A framework for K - 12 science education: Practices, crosscutting concepts, and core ideas ; National Acad emies Press: Washington, DC, 2012. (4) Cooper, M. M.; Corley, L. M.; Underwood, S. M. J. Res. Sci. Teach. 2013 , 50 , 699 721. (5) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. J. Chem. Educ. 2012 , 89 , 1351 1357. (6) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Chem. Educ. Res. Pract. 2012 , 13 , 195 200. (7) Maeyer, J.; Talanquer, V. Sci. Educ. 2010 , 94 , 963 984. (8) McClary, L.; Talanquer, V. Int. J. Sci. Educ. 2011 , 33 (10), 1433 1454. (9) Cooper, M. M.; Williams, L. C.; Und erwood, S. M. J. Chem. Educ. 2015 . (10) Schmidt, H. - J.; Kaufmann, B.; Treagust, D. F. Chem. Educ. Res. Pract. 2009 , 10 , 265 272. (11) Henderleiter, J.; Smart, R.; Anderson, J.; Elian, O. J. Chem. Educ. 2001 , 78 (8), 1126 1130. (12) Cooper, M. M.; Gro ve, N.; Underwood, S. M.; Klymkowsky, M. W. J. Chem. Educ. 2010 , 87 , 869 874. (13) Underwood, S. M.; Reyes - Gastelum, D.; Cooper, M. M. Sci. Educ. in press . DOI: 10.1002/sce.21183. (14) Cooper, M. M.; Klymkowsky, M. W. J. Chem. Educ. 2013 , 90 , 1116 1122 . (15) Cooper, M. M.; Klymkowsky, M. W. CLUE: Chemistry, Life, the Universe & Everything http://besocratic.colorado.edu/CLUE - Chemistry/ (accessed Apr 28, 2015). 9 C HAPTER II : THEORETICAL FRAMEWORK S Nobody thinks clearly, no matter what they pretend. Th a matter of catching as many of those foggy glimpses as you can and fitting t hem together the best you can. - Dashiell Hammett , The Dain Curse 1 The history of educational research reaches back to the nineteenth century when researchers began to use scientific methods to systematically explore how people learn. 2 These early studies have their roots in behaviorism, spearheaded by the works of Edward Thorndike. Learning was viewed as behavioral in that it could be manipulated and controlled based on applied influences, what Thorndike called the Law of Effect. 3 While behaviorism in the early twentieth century studied learning throu gh the creation and alteration of engrained behaviors, researchers at the time had difficulty expanding these studies to explore thought and reasoning emphasis was placed on understanding human thought processes through new rigorous methodologies. Using these new methodologies, researchers were able to formulate and test theories of learning. 2 Constructivism and meaningful learning With the ris e of studies on learning and human understanding came an additional emphasis on the process of knowing , that is, what factors influence the gaining of new knowledge and how its subsequently applied to novel situations. To better understand this focus on th e process of knowing, we have to consult the work of researchers such as Piaget and Vygotsky. Piaget provided the foundation on which much of constructivism, as we know it 10 today, was built (although he was not the first to suggest the idea 4 ). He reasoned that intelligence in children develops ontologically, and he should therefore be able to observe and resea rch its development. 5 stimulus - response theory. He studied children over several years and concluded that children were actively involved and responsible for their own mental developm ent as they tried to make sense of the world around them. 6 individual interpretations of those actions: Learning is possible only when there is active assimilation. It is this activity on the part of the subject which seems to me to be underplayed in the stimulus - f the subject himself, and I think that without this activity there is no possible didactic or pedagogy which significantly transforms the subject. 5 Vygotsky held similar views but also emphasized his idea of the zone of proximal development, which represented t and the level they could achieve through instructor guidance. 7 He argued that learners could only truly understand material appropriate for their developmental level. For example, it may be appropriate for high school students with a general science background to take an in troductory chemistry course, but place them in a graduate - level physical chemistry course and they most likely would understand little to none of the material. The most effective learning would then which is entirely dependent on their current knowledge level. 11 Constructivism has since expanded into a variety of theories of knowledge 8 such as social constructivism 9 and radical constructivism. 10 In its most basic form, however, it can best be 6 One major critique, however, is that constructivism is a theory of learning rather than a theory of instruction. 2 That is, pure constructivism discounts the role o f instructors and their influence when it comes to the development of new knowledge. 8 W ith this in mind, it is useful to consider the theory of meaningful learning. Meaningful learning finds its origins in the works of David Ausubel and his assimilation theory. 11 He first described meaningful learning as occurring when the learner chooses to incorporate new knowledge purpose fully into their already existing framework of knowledge - approaching knowledge as small, isolated units of information that have no explicit anchoring to prior knowle dge. 12,13 Because individuals differ greatly in their knowledge structures and efforts to integrate new knowledge, however, meaningful learning and rote learning should be considered as a continuum rather than dichotomous. 14,15 Unfortunately, much of traditional school learning occurs at the rote - learning end of the continuum. While rote learning can be preferable in certain circumstances, such as learning a new language or memorizing multiplication tables, there is often not enough emphasis placed on meaningful learning. A beneficial example of meaningful learning in the class room would be an emphasis on the relationship between two concepts. 16 In order for meaningful learning to occur, three basic tenets must be met: the learner must possess some relevant knowledge to which they can connect the new knowledge to, the new knowledge must be meaningful to the learner and presented in a meaningful way by the 12 instructor, and the learner must then choose to lear ning meaningfully as opposed to undergoing rote learning. 17 These requirements for meaningful learning are outlined below in Figure 2.1. Students who undergo meaningful learning often remember information long er due to the increase in neural connections. Studies have shown that information learned via rote learning is typically forgotten within six to eight weeks. 16 Figure 2.1: Concept map of the re quirements for meaning learning 17 13 Identifying prior knowledge: Moving beyond misconceptions to p - prims We certainly understand that students ent er the classroom, be it science or otherwise, with a set of ideas and preconceived notions that are often at odds with what we understand to be true. 18 One prevalent theory in education research is the idea that all students possess misconceptions, or deeply entrenched and stable false ideas and beliefs. 19,20 To o vercome misconceptions, instructors should help students confront their misconceptions with contradictory knowledge and facts in order to produce conceptual change, that is, the reorganization or replacement of central concepts. 21 In order for conceptual change to be o be doubtful that confronting these ideas would produce the desired result. 22 Additionally, diSessa fact, no widely accepted, well - 23 While most misconceptions literature typically focuses on the coherency o f student understanding, researchers like diSessa argue that learners possess fragmented pieces of knowledge, called phenomenological primitives (p - prims) or facets. These facets of knowledge are phenomenological in nature, meaning they arise from an effor t to make sense of natural events; they are primitive in that they are often offered up as self - explanatory and require no justification. 22 Unlike the coherency stressed in theories of conceptual change, p - prims are loosely woven and highly fragmented. 24 diSessa describes a variety of p - prims, most based in physics, in his work. For example, x begets more y - prim. 22 While the bells vary in pitch, they all appear to be the same size and made of the same material. So how do the bells produce different pitches? Most people will s ay 14 that the bells must vary in thickness. While this is true, the question then becomes which results in a higher pitch, the thicker or the thinner bells? Here is where p - prims tend to come into play as intuitive reasoning. The most common answer is that t hicker bells produce lower pitches (which is the opposite of what actually occurs), however few can explain why they believe this to be true. Intuitively, it feels like the right answer; thicker/bigger/heavier things produce lower pitches. diSessa lists th is idea under the larger p - x begets more y 22 In this sense, the p - prim becomes mor e generalizable to other areas of study. For an expert, diSessa states that a discussion of pitch would trigger the concept of vibration, which would trigger the knowledge of a simple harmonic oscillator. As a result, the expert may conclude that, while a thicker bell would be heavier, it would also be stiffer. diSessa argues that experts also use intuitive p - these ideas as guides, and they can justify them if need be with additional content knowledge. 2 2 Novices, however, have more difficulty integrating new knowledge and, therefore, it is with an understanding of these processes that we can continue to inform and influence theories of learning. How we learn: Working memory and information processing Alan Baddeley was one of the first to suggest the theory of working memory during the - term memory, refers to a system of the brain used for temporary information sto rage and manipulation . 25 This system is vital for cognitive processes such as learning and comprehens ion. Unfortunately, this space is rather limited and can easily be overwhelmed. 26 For instance, 15 extreme performance pressure, such as completing mathematics problems on an exam, can consume working memory space and make it more difficult to perform well. 27 Johnstone provides a succinct model of working memory as part of the information processing model, shown in Figure 2.2, that allows us to analyze the factors that affect working memory and how it relates to long - term memory (LTM). 28,29 Figure 2.2: essing model and working memory 28,29 Learners are constantly ex periencing external information and events, which can certainly be overwhelming. This is best explained in terms of multimedia learning theory, which describes the human information processing system as having dual channels for sensory input: verbal and pi ctorial. 30 These two inputs are proces sed in the working memory space, resulting in limited capacity and the learner can switch a representation from one format to another for processing. 31 For instance, when sitting in a chemistry lecture course students are often exposed to a verbal lecture from the professor, a visual display of information (typically through Powerpoint slides), and even their own written notes or textbooks for additional information. A student may listen to the lecturer describe the interactions of molecules and mentally convert the words to a picture, changing the processing type from verbal to pictorial. Multimedia learning 16 learners are able to create a deeper understanding than from words or pi 31 W hen presented with a variety of information and sensory inputs, learners must filter out extraneous information in order to take in relevant information. Often what they decide to keep or filter out is influenced by knowledge already stored in their LTM. O bviously, this arrangement benefits experts and those with a more robust and knowledgeable LTM who are better able to recognize and filter out extra, unnecessary information. Once new information enters the working memory space, it can interact with inform ation stored in the LTM to facilitate learning and successful incorporation into the LTM. As mentioned previously, this space is limited, easily overwhelmed, and, as a result, can make learning difficult. 26 Cognitive load theory (CLT) combines the discussion of LTM and working memory with the idea of automatic processing. The theory describes how schema developed in LTM can be used in working memory to allow for more efficient processing of data. 32,33 For example, continued practice solving sets of algebraic problems (like (a+b)/ c=d, solve for a) can lead to the development of schema that make addressing these problems easier and faster over time. The learner will become more familiar with these types of algebraic problems and will be better able to recognize and solve increasingl y more complex versions of these problems. 34 How we reason and make decisions: H euristics and dual - process theory There has been considerable evidence over the years that people employ heuristics to make quick, efficient decisions and avoid overwhelming working memory space. The idea of bounded rationality. 35 He proposed that individuals work within the confines of a given task as well as their own cognitive constraints. 17 Heuristics, therefore, are used to reduce the amount of cognitive effort expended and to simplify the decision - making process. 36 Unfortunately, while heuristics are certainly useful, they do not always lead to the c orrect answer. Shah and Oppenheimer have hypothesized that all heuristics, regardless of domain, most likely fall under a handful of larger heuristic types. 36 That is, even though some heuristics are task - specific, they are rarely domain - specific. The reasoning behind heuristic use becomes appa rent when considering dual - process theory. Dual - memory. 37 The theory describes two systems of mind, System 1 and System 2, that explain how we reason. 38 System 1 processes are largely characterized by their autonomy, often resulting in these processes being described as reflexi ve and intuitive. System 2 requires deeper thought processes and can be associated with reasoning. It also typically invokes the working memory and is involved in hypothetical thinking. Cognitive decoupling, or the ability to separate real world representa tions from imaginary ones, is strongly linked to System 2 reasoning. 39 System 2 is correlated with higher performance on intelligence measures, unlike System 1, which appears to be independent of such measures. Stanovich 40 succinctly described in problem solving: . This so - called heuristic processing is designed to get you into the right ballpark when solving a problem or making a decision, but it is not designed for the type of fine - grained analysis called for in 18 Wason and Evans 37 first introduced the term dual - process theory in a n attempt to explain why their research subjects seemed to be making specific choices based on a matching bias. They provided participants with four cards, like those shown below in Figure 2.3. Participants were asked which card or cards they would flip ov er to determine if the following statement was 39 Only 10% of their participants answered correctly (flipping card A and card 7). Instead, most participants elected to flip cards A and 3, essentially matching the terms discussed in the prompt rather than considering which cards could potentially negate the original statement. This experiment has been repeated by other researchers with the same results. 41 Figure 2.3: Card experiment used by Wason and Evans t o explore dual processes theory 39 Additio nal studies have shown, however, that when the context of the question changes and becomes more realistic, like comparing ages on one side of the card with alcoholic or non - alcoholic beverage choices on the other side, the answer becomes significantly more obvious to participants. 4 2 In terms of dual - process theory, it is hypothesized that abstract versions of the many individuals do not perform well. Revised versions of the question set in a familiar context, however, appear to lead participants to the correct conclusion automatically without much reasoning (evoking System 1). 43 19 Often, the way a question or problem is worde d can prompt an automated, System 1 response over the use of System 2. For example, take the following problem from the Cognitive Reflection Test 44 (Version 1) and a rewritten version 45 (Version 2): Version 1 If it takes 5 machines 5 minutes to make 5 widge ts, how long would it take 100 machines to make 100 widgets? Version 2 If it takes 5 machines 2 minutes to make 10 widgets, how long would it take 100 machines to make 100 widgets? Each version of the question, in theory, is the same, but the variation i n wording alters the approach that readers often take in solving the problem. In Version 1, a common answer is 100 minutes. 44 This response is considered a heuristic, System 1 response arising from an innate 45 If the feeling of rightn ess is particularly strong, a System 2 override is deemed unnecessary, resulting in the instinctual answer prevailing. Even when participants are given extra time to solve a problem, shown using the Wason card experiment, participants spend more time tryin g to justify their initial choice (in this case the cards they were going to select) than they did considering the reasons for rejecting the remaining choices. 46 There has been some debate as to the terms System 1 and System 2 as they imply two individual, distinct systems. Rather, Sys tem 1 should ideally be plural as it encompasses several systems working in tandem, sometimes referred to as the autonomous set of systems (TASS). 39 In fact, many theories and terms for System 1 and System 2 have been subsumed under dual - process theory over the years as the theory has gained prominence. Evans lists several labels used instead of System 1 and System 2 in the literature that range from experiential, heuristic, and impulsive to systematic, analytic, and higher order. 47 20 Difficulties in the chemistry classroom: Representational competenc y Up until now, the frameworks presented in this chapter have been broadly applicable to a range of fields of study. The majority of chemistry lies in the abstract, atomic realm where we cannot physically see the interactions and reactions taking place, only the resulting macroscopic type of concepts with which children and adults are familiar are made up of tangible 28 Because of its abstract nature, chemistry can be described using three levels of thought: macroscopic, microscopic, and symbolic (shown below in Figure 2.4). Figure 2.4: ht. Reproduced with permission of John Wiley & Sons, Inc. 28 The macr oscopic level encompasses that which we can see and feel, like changes in physical state. For chemistry, Johnstone described sub - microscopic as including molecular and atomic levels as well as forces and interactions. 48 Symbolic includes formulas and equations along with chemical structures and graphs. While understanding each level alone can be complicated, most often chemistry concepts reference all three levels simultaneously with 21 different degrees of emphasis. For example, hydrogen bonding is often introduced by discussing the macroscopic concept of boiling water and explaining that the temperature at which water boils is directly related to the strength of the attractive forces between water molecules (sub - microscopic), which can be determined based on the molecu lar structure (symbolic) and geometry. While experts may be able to fluidly transition between these levels, for students this can be a particularly difficult challenge. Some believe that chemistry places too much emphasis on the interplay of the symbolic and submicroscopic levels, with less reference to the macroscopic level. 49 Because of the abstract nature of chemistry, there is a heavy reliance on the use of representations to convey ideas that cannot be experienced first - hand. Kozma and Russell 50 set of skills and practices that allow a person to reflectively use a variety of representations or think about, communicate, and act on chemical phenomena in terms of Concluding remarks Chemistry is notoriously considered by students to be one of the most difficult subjects of study. Its reput ation is not necessarily unfounded. Many students struggle to grasp the complex and abstract nature of chemistry and often, introductory chemistry courses sacrifice depth of material for breadth. The ways in which chemistry has been taught over the years h as not aligned well with what we know about learning and integration of new knowledge. Take the complex relationship between structure and properties. In order to understand this connection, students hey should understand that representations could be used to show molecules and interactions as well as the macroscopic changes that result. 22 Meaningful learning tells us that instructors and the curriculum they use should be making these connections explici t for students, relating each step in the process back to the previous steps as well as highlighting the usefulness of each step in the overall process to determine macroscopic properties. Instructors should also consider the prior knowledge that students bring with them to the classroom. Thinking is messy and as students attempt to sort through the plethora of new knowledge that they are exposed to, these ideas can become fragmented and loosely woven together in an attempt to build reasonable explanations . While these ideas at times can be persistent and difficult to change, providing a strong base of essential chemical concepts, perhaps through learning progressions 51,52 , can result in a solid foundation from which students can build and integrate new knowledge. Assessment plays an essential role in determining have learned. By understanding heuristic use and dual - process theory, instructors should be mindful to create questions that force students to override their impulsive, System 1 processes and use System 2 to develop rational and analytical r esponses. Assessment items that can be answered through the use of heuristics and other short cuts fail to engage System 2 processes connecting new concepts to found ational knowledge, and designing effective assessments, Several of these frameworks have guided the research presented in this dissertation. Because of the complexity properties, we chose to analyze and dissect our structure - property interviews through the lens of dual process theory and p - terviews and with the Intermolecular Forces Assessment (IMFA) was influenced by studies on 23 representational competence in chemistry and the interplay of words and pictures outlined by multimedia learning theory. Finally, our exploration of the effects of a reformed curriculum on student understanding highlight the need to thoughtfully design curricula to reflect the tenets of meaningful learning. 24 REFERENCES 25 REFERENCES (1) Hammett, D. The D ain Curse ; Alfred A. Knopf, Inc.: New York, 1929. (2) National Research Council. How people learn: Brain, mind, experience, and school. ; National Academies Press: Washington, DC, 1999. (3) Thorndike, E. L. Educational psychology ; Teachers college, Colu mbia University: New York, 1913; Vol. 2. (4) Von Glasersfeld, E. Synthese 1989 , 80 , 121 140. (5) Piaget, J. J. Res. Sci. Teach. 1964 , 2 (3), 176 186. (6) Bodner, G. M. J. Chem. Educ. 1986 , 63 , 873 878. (7) Vygotsky, L. S. Mind in society: The devel opment of higher psychological processes ; Harvard University Press: Cambridge, MA, 1980. (8) Bodner, G.; Klobuchar, M.; Geelan, D. J. Chem. Educ. 78 , 1107. (9) Solomon, J. Stud. Sci. Educ. 1987 , 14 (1), 63 82. (10) Von Glasersfeld, E. Radical constru ctivism: A way of knowing and learning ; Studies in Mathematics Education; The Falmer Press: Bristol, PA, 1995. (11) Novak, J. D. Learning, Creating, and Using Knowledge: Concept Maps As Facilitative Tools in Schools and Corporations ; Lawrence Erlbaum Ass ociates: Mahwah, NJ, 1998. (12) Ausubel, D. P. The psychology of meaningful verbal learning ; Grune & Stratton: New York, 1963. (13) Ausubel, D. P. Educational Psychology: A Cognitive View ; Holt, Rinehart and Winston: New York, 1968. (14) Novak, J. D. Sci. Educ. 2002 , 86 , 548 571. (15) Grove, N. P.; Bretz, S. L. Chem. Educ. Res. Pract. 2012 , 13 (3), 201 208. (16) Novak, J. D. A theory of education ; Cornell University Press: Ithaca, NY, 1977. (17) Bretz, S. L. J. Chem. Educ. 2001 , 78 , 1107 1117. 26 (18) Locke, J. An essay concerning human understanding ; Adamant Media Corporation: Boston, MA, 2001. (19) Confrey, J. Rev. Res. Educ. 1990 , 16 , 3 56. (20) Hammer, D. J. Learn. Sci. 1996 , 5 (2), 97 127. (21) Posner, G. J.; Strike, K. A.; Hewson, P. W .; Gertzog, W. A. Sci. Educ. 1982 , 66 (2), 211 227. (22) diSessa, A. A. Cogn. Instr. 1993 , 10 (2 & 3), 105 225. (23) diSessa, A. A. In The Cambridge Handbook of the Learning Sciences ; Sawyer, K. R., Ed.; Cambridge University Press: Cambridge, 2006; pp 265 282. (24) diSessa, A. A. In International handbook of research on conceptual change ; Vosniadou, S., Ed.; Routledge: New York, 2008; pp 35 60. (25) Baddeley, A. Science 1992 , 255 (5044), 556 559. (26) Johnstone, A. H.; Sleet, R. J.; Vianna, J. F. Stud. High. Educ. 1994 , 19 (1), 77 87. (27) Beilock, S. L.; Carr, T. H. Psychol. Sci. 2005 , 16 (2), 101 105. (28) Johnstone, A. H. J. Comput. Assist. Learn. 1991 , 7 , 75 83. (29) Johnstone, A. H.; Selepeng, D. Chem. Educ. Res. Pract. 2001 , 2 (1), 19 2 9. (30) Mayer, R. E.; Moreno, R. Educ. Psychol. 2003 , 38 (1), 43 52. (31) Mayer, R. E. Multimedia Learning New York, 2001. (32) Moreno, R. Learn. Instr. 2006 , 16 , 170 181. (33) Paas, F.; Renkl, A .; Sweller, J. Educ. Psychol. 2003 , 38 (1), 1 4. (34) Sweller, J. Learn. Instr. 1994 , 4 , 295 312. (35) Simon, H. A. Annu. Rev. Psychol. 1990 , 41 (1), 1 20. (36) Shah, A. K.; Oppenheimer, D. M. Psychol. Bull. 2008 , 134 (2), 207 222. (37) Wason, P. C .; Evans, J. S. B. Cognition 1975 , 3 (2), 141 154. 27 (38) Stanovich, K. E. Who is rational?: Studies of individual differences in reasoning ; Lawrence Erlbaum Associates: Mahwah, N.J., 1999. (39) Evans, J. S. B.; Stanovich, K. E. Perspect. Psychol. Sci. 2 013 , 8 (3), 223 241. (40) Stanovich, K. E. What intelligence tests miss: The psychology of rational thought ; Yale University Press: New Haven, 2009. (41) Evans, J. S. B. Think. Reason. 1998 , 4 (1), 45 110. (42) Griggs, R. A.; Cox, J. R. Br. J. Psycho l. 1982 , 73 (3), 407 420. (43) Evans, J. S. B. In The Cambridge handbook of thinking and reasoning ; Holyoak, K. J., Morrison, R. G., Eds.; Cambridge University Press: New York, 2005; pp 169 184. (44) Frederick, S. J. Econ. Perspect. 2005 , 19 (4), 25 42 . (45) Thompson, V. A. In In two minds: dual processes and beyond ; Evans, J. S. B., Frankish, K., Eds.; Oxford University Press: Oxford, 2009; pp 171 195. (46) Lucas, E.; Ball, L. Think. Reason. 11 (1), 35 66. (47) Evans, J. S. B. Annu. Rev. Psychol. 2008 , 59 , 255 278. (48) Johnstone, A. H. J. Chem. Educ. 1993 , 70 (9), 701 705. (49) Talanquer, V. Int. J. Sci. Educ. 2011 , 33 (2), 179 195. (50) Kozma, R.; Russell, J. In Visualization in science education ; Gilbert, J. K., Ed.; Springer: Netherlands , 2005; pp 121 146. (51) Duschl, R.; Maeng, S.; Sezen, A. Stud. Sci. Educ. 2011 , 47 (2), 123 182. (52) Corcoran, T.; Mosher, F. A.; Rogat, A. Learning progressions in science: An evidence based approach to reform ; RR - 63; Consortium for Policy Research in Education: Teachers College - Columbia University, 2009. 28 CHAPTER III : A REVIEW OF THE LITERATURE ON STRUCTURE - PROPERTY RELATIONSHIPS AND IN TERMOLECULAR FORCES A ll things are made of atoms little particles that move around in perpetual motion, a ttracting each other when they are a little distance apart, but repelling upon being squeezed into one another. In that one sentence, you will see, there is an enormous amount of information about the world, if just a little imagination and thinking are ap plied. - Richard Feynman, 1977 1 The relationship between structure and property The idea that the structure directly influences function is an integral scientific concept. I say scientific rather than chemical because that single statement applies to a wide range of scientific fields. In biochemistry, the shape of a protein determines its function, and in biology the structure and composition of leaves allow them to collect sunlight for photosynthesis; in physics (and even engineering) the structure and design of wings allow for flight. While this concept can be succinctly described in a few sentences, it encompasses a vast array of information. In terms of chemistry, the r elationship between structure and function is best highlighted by the idea that the physical and chemical properties of a substance can be traced back to the molecular level structure of the molecules that make up that substance and their interactions. We know that students struggle with this relationship and understandably so. 2 5 In order to make this connection, students have to complete a long c hain of inferences. They must be able to construct a representation of the molecule (typically a Lewis structure), identify its geometry and shape, 29 use their understanding of electronegativity and bond polarity along with shape to determine molecular polar ity, identify the type and strength of intermolecular forces that the molecule is capable of, and finally combine all of this information to predict the physical and chemical properties the substance might exhibit. 2 Each of these steps alone can be daunting for students. We know, for instance, that students have trouble constructing Lewis structu res and often are not aware that they can be used to predict chemical and physical properties. 2,6,7 Shane and Bodner interviewed three students and described the inability of one student to view Lewis structures as symbolic rather than verbo - linguistic (a set of lines, letters , and dots with no symbolism). 8 If students cannot construct the correct molecular structure, or do not know what Lewis structures are for, then we should not be surprised when they cannot use the structure to predict properties. DeFever and colleagues have reported that stu dents rarely discuss shape and geometry when determining the polarity and resulting solubility of different compounds. 5 A study by Birk and Kurtz as well as one by Peterson a nd colleagues both identified similar student misconceptions related to polarity. They both found that students often ignored molecular shape and felt molecules could only be nonpolar if the atoms within the molecule had the same electronegativities. 9,10 While identifying the elements present can help students determine the polarity of a molecule, the geometry and shape of the molecule play an important role. For instance, while carbon dioxide contains polar bonds, its linear shape allows for the resulting nonpolar molecule. 30 Several studie properties of various compounds. Talanquer and colleagues investigated how students predict reactivity and acidity for a group of compounds and found that they relied on heuristics and trends to inform their decisions. 11,12 predicting the reactivities of compounds when the structures provided were less familiar to them. They reported that students often tackled these structures by comparing them to other structures structure that they believed to be a suitable match. 5 well documented in the literature. For instance, we know that students have difficulties conceptualizing the idea of boiling or melting and often describe the process in terms of breaking covalent bonds within a molecule. 4,10,13 15 Many studies have reported participants describing the composition of bubbles in boiling water as oxygen and hydrogen, indicating that students b elieve water molecules break up during the boiling process. 16 19 Smith and Nakhleh explored students understanding of melting and dissolving processes for a variety of compounds (including salt, butter, and sugar) and found similar misconceptions of students breaking bonds or incorrect intermolecular forces during phase changes. 13 Other studies have identified additional student ideas about phase and phase changes under the view of the particulate nature of matter, outlined by Nakhleh. 20 Griffiths and Preston interviewed grade 12 students and found several alternativ e ideas including the notion that water molecules change size depending on the phase of matter, that adding heat causes atoms to expand, and that water molecules in the solid phase are not bonded together in any specific 31 pattern. 21 Othman and colleagues identified students who believed that the difference in a molecule. 14 Other studies have explored how students explain physical processes such as boiling or melting in terms of IMFs, but before we discuss them, it would be beneficial to outline the Intermolecular Forces Early chemical theory, suggested by Empedocles and refined by Aristotle, stated that amental forces to account for all natural phenomena. One was Love, 22 As romantic as this notion may be, we now understand there to be four fundamental forces: strong, weak, electromagnetic, and gravitational. The idea that there are attractive forces that exist between particles and govern macroscopic properties was not addressed quantitatively until in the mid - 23 There was little discussion as to what mig ht be causing these hypothetical interactions. The concept of intermolecular forces (IMFs) has expanded and morphed since then, but the original idea remains the same: there exist non - covalent interactions between molecules (and sometimes within very larg e molecules) that are governed by differences in charge. The most common intermolecular forces discussed in introductory chemistry courses are hydrogen bonding, dipole - dipole, ion - dipole, ion - induced dipole, dipole - induced dipole, and London dispersion for ces (LDFs). The term van der Waals forces refers to a broader category of interactions, defined by the International Union of Pure and Applied Chemistry 24 as: 32 T he attractive or repulsive forces between molecular entities (or between groups within the same molecular entity) other than those due t o bond formation or to the electrostatic interaction of ions or of ionic groups with one another or with neutral molecules For our purposes, we will focus mainly on hydrogen bonding, dipole - dipole, and LDFs. These three IMFs are predominantly emphasized i n general chemistry curricula and are, therefore, of more interest to us. The key unifying idea behind these interactions, and the idea that we would hope most students would leave their introductory chemistry courses knowing, is that these interactions oc cur between small molecules and are responsible for a wide array of macroscopic physical and chemical properties. Figure 3.1(a) - (c) shows common representations of the three main IMFs, hydrogen bonding, dipole - dipole, and LDFs , similar to those seen in General Chemistry: Atoms First by McMurry and Fay . 25 Hydrogen bonding is often represented as a dashed line between the lone pair of an extremely electronegative element and a hydrogen covalently bonded to another extremely electronegative element. Dipole - dipole interactions are sometimes also repre sented using a dashed line (as hydrogen bonding can be considered a stronger version of dipole - dipole) and often include partial positive and partial negative charges to indicate areas where these interactions occur. A variety of representations exist for LDFs, but most show either electrostatic potential maps or the distorted electron clouds of noble gas elements like helium or diatomic halogen molecules like iodine. These representations typically show one molecule with an instantaneous dipole that then i nduces a dipole on a neighboring molecule. Again representations 33 of LDFs often use the addition of partial positive and negative charges to show the dipoles responsible for the interaction. (a) (b) (c) Figure 3.1: Representa tions of (a) hydrogen bonding, (b) dipole - dipole, and (c) LDFs two categories: studies about general IMFs 10,26,27 or (more predominantly) studies about hydrogen bonding. 28 33 For those studies that only generally explore IMFs, the main focus is IMFs. Additionally, while many s - dipole interactions and LDFs. 34,35 One possible reason for the focus on hydrogen bonding could be its unique role in biological systems or its association with the properties of water. The next sections will go into greater detail about these studies and their findings. xt of bonding, as outlined by Özmen in his review of the chemical bonding literature. 36 Peterson and Treagust developed a two - tiered diagnostic instrument, which included two items about inter - and intramolecular forces. For these two items, 23% of students in g rade 12 identified IMFs as 34 forces within a molecule (rather than between molecules) and 33% of students cited IMFs as forces within a continuous covalent solid. 10 Goh and colleagues administered the same diagnostic instrument to grade 12 students in Singapore and found similar results: 35% of students indicated that IMFs occurred within a molecule and 46% stated that they existed in a network covalent solid. 26 Coll and Taylor interviewed students at various levels (from high school to graduate school) and found additional alternative conceptions, such as the involvement of IMFs in ionic and metallic bonding as well as the 27 Griffiths and Preston also found students who hypothesized that water molecules may be held together by gravity or air pressure. 21 Taber points out that when instructors discuss ionic and covalent bonding as two dichotomous ideas, there is little room for students to incorporate the concept of intermolecular forces. They are often either considered bonds or just a force. 37 While Taber and other s argue for IMFs to be accepted as bonding, the difference between the two is still important to emphasize especially in terms of what happens on the molecular level during a phase change. 37,38 Student difficulties with hydrogen bonding Other researchers have taken the exploration of student between intermolecular and intramolecular forces one step further and focused specifically on the conflation of hydrogen bonding with covalent bonding. 28 30,32,33 Nico ll, in her interviews on Lewis structures and covalent bonding, found students who incorrectly identified covalent bonds within a formaldehyde molecule as hydrogen bonding. 39 Taber discussed this particular ed that students are 35 first introduced to the idea of hydrogen bonding in biology, rather than in their chemistry course, and when instructors only mention the term without further discussion of its meaning, they can lead students astray. 28 While this may be true, it can also be argued that the term hydrogen bonding itself is misleading. St 40 41 have been previously reported. It would certainly come as no surprise that intermolecular, intramolecular, and hydrogen bonding would lead to confusion as well. - choice items Most studies on IMFs, and hydrogen bonding specifically, rely on the use of either interviews or multiple - choice and written assessments to elicit student understanding. Schmidt and colleagues administered a multiple - choice assessment with follow - up short answer questions to over 3500 high school students. Their items relat ed to IMFs required students to identify which compounds out of those provided (e.g. acetic acid, methyl fluoride, and dimethyl ether) would exhibit hydrogen bonding. 29 The assumption, however, is that students understand where hydrogen bonding occurs. Students are never asked to elaborate on the location of the IMF, but rathe r to identify compounds capable of hydrogen bonding. Henderleiter and colleagues made a similar assumption in their interviews of 22 organic chemistry students on hydrogen bonding, asking students to show were hydrogen bonds would form between given molecu les. Again, there is an assumption that students understand IMFs as interactions between molecules. 30 Both studies explored the strategies and structural features used by students to determine hydrogen bonding. Schmidt and colleagues noted that some students determined if a compound 36 was capable of hydrogen bonding by simply identifying the presence of oxygen and hydrogen or by identifying the structural similarities to a kn own compound (like comparing dimethyl ether to water). 29 Henderleiter a nd colleagues found students who listed additional atoms as capable of hydrogen bonding (like carbon and sulfur) and students who confused intramolecular hydrogen bonding with a chemical reaction. 30 Similarly, Barker and Millar provided 250 students with Lewis structures of water interacting via dotted lines and asked them to explain both the dotted line between molecules and the solid line within a water molecule. At the end of their study, 24% of the population indicated that the dotted line was an attractive force (not a real bond). The researchers considered this statement inaccurate and coded it as evidence of partial misunderstanding, whereas referring to the dotted line as van der Waals or dipole - dipole bonds was considered evidence of partial understanding and identifying the interaction as hydrogen bonding with no explanation was considered evidence of understanding. While their coding scheme is somewhat unclear, t hey found that 68.4% of their population correctly identified the dotted line as hydrogen bonding by the end of the course. 31 In a similar fashion, Villafañe and colleagues designed an ins trument to uncover provided students with Lewis structures of carboxylic acids or amines with water and asked them to identify between which atoms hydrogen bon ding would exist. Options included atoms within the same molecule (such as a covalent bond) and atoms between molecules (like oxygen on one molecule and hydrogen on another). Unlike other studies that explore this concept using multiple - choice exams, the a uthors did provided students the option to indicate hydrogen bonding as a bond within a molecule. Significantly, the questions relating to hydrogen bonding 37 resulted in the lowest Cronbach alpha value (0.306) out of all of their item sets indicating a weak 34 In a similar fashion, Nahum and colleagues asked students to indicate on a given diag ram of several water molecules where hydrogen bonding would occur. Several students incorrectly identified where hydrogen bonding would occur, indicating interactions between incorrect atoms or covalent bonds within the molecule. 38 Assessment items, like the ones outlined here, most often require students to identify hydrogen bonding in a given repr esentation or identify compounds capable of hydrogen bonding in the form of multiple - choice questions. They typically make several assumptions as to what a student does or does not know and are not g of IMFs. Drawing intermolecular forces A small number of studies have required students to draw representations of hydrogen bonding to explore their understanding of this IMF. Pereira and Pestana were some of the first understanding of hydrogen bonding through drawings. The researchers asked Spanish high school students to construct representations of water as a solid, liquid, and gas. They found that most students failed to include a representation of hydrogen bonding i n their drawings. For those students that did draw hydrogen bonding, some provided representations with bonds between hydrogen atoms on different molecules or even double bonds between molecules. Only eleven students out of the entire population ( N =227) pr ovided an however, that the authors placed a large emphasis on the portrayal of bond lengths and bond epresentation of hydrogen bonding 38 between molecules but the length of the IMF was shorter than the bond length within the molecule, then the answer would be marked incorrect by the researchers. 32 Taagepera and colleagues asked general and organic chemistry stu dents to draw representations of hydrogen bonding as part of a combination constructed - response and multiple - choice assessment. They found that the two most difficult items, averaging 36% and 40% correct responses, required students to draw IMFs; the first interacting with water molecules. Because the researchers were more interested in the connections students made between bondin g concepts (like electronegativity and bond polarity), however, that many students confused hydrogen bonding with a covalent bond and some inaccurately repres ented it as an interaction between hydrogens on different molecules. 33 tudies, little research has Many studies make assumptions as to what students do and do not understand through the format and design of their assessment items a nd interview questions. With so few studies on dipole - dipole and LDFs, additional research is still needed to explore these ideas in greater detail. Intermolecular forces and their relationship to physical properties eas and alternative conceptions about IMFs, several researchers have studied how students relate these IMFs to their understanding of physical properties, like boiling and melting points, and phases of matter. Ideally, students should be able to relate the type and strength of IMFs present to help them determine relative properties. For 39 example, stronger IMFs like hydrogen bonding require more energy to overcome resulting in higher melting and boiling points of a substance. Schmidt and colleagues asked high school students to predict which alkane, out of a series of increasingly branched alkanes, would have the lowest boiling point. Only 15% of the students who provided an explanation were able to connect surface area and van der Waals forces to the differen ce in relative boiling points. Other students explained the how the various structures effected the boiling point in terms of breaking + ions. The researchers acknowledge tha t, while they were able to identify alternative ideas about boiling, 29 Henderleiter and colleagues explicitly asked students in their interviews to identify if the propanol is because of students interviewed correctly related the trend in increasing boiling points to the molar mass or chain length of the compounds. Other students only memorized the correct trend or explained it never address if students effectively discussed that all three structures were capable of hydrogen bonding an d that the increase in molar mass or chain length corresponds to more electrons, increased polarizability, and thus stronger LDFs which result in a higher boiling point. This connection is crucial and an indicator of coherent understanding, but it is uncle ar from the published study if students made this distinction. 30 Barker and Millar provided two substances to students, magnesium chloride and titanium (IV) chloride, and i nformed them that the first was ionic and the second was covalent. They then 40 ts to explain that the IMFs between titanium (IV) chloride molecules are considerably weaker than the ionic bonds present in magnesium chloride and thus less energy would be needed to overcome them. The authors found that many students failed to discuss IM Fs in their explanations of the boiling point of titanium (IV) chloride and instead readily attributed the lower boiling point to the presence of covalent bonds, sometimes indicating that the covalent bonds themselves would break. Barker and Millar also no ted that the format of the question might have influenced may not have been apparent to students that a discussion of IMFs was necessary to successfully answ er the question. 31 of IMFs in determining various physical properties. For those that have, respon ses alluding to LDFs in determining boiling and melting points. There was little discussion of how chain length and boiling point are connected. Additionally, as question wording can affect the ways in which students respond in their explanations. More work is certainly needed to explore if students understand that IMFs, not chain length or covalent bonds, affect the relative boiling and melting points of compounds. structure and properties and the role of intermolecular forces, it is al so important to discuss evidence - based solutions to address these problems. Proposed solutions reported in the literature 41 range from small intervention activities and lab projects to full redesign of chemistry curricula. Many authors have suggested short e xamples to help students better understand IMFs, such as d 42 , using magnets to represent dipole - dipole interactions 43 , or even using a structural database to model intermolecular interactions 44 . Most of these ex amples, however, are intentionally designed as helpful suggestions with no evidence to support their effectiveness. issues with methodology or analysis. Tarhan and colleagues s hydrogen bonding, dipole - dipole and LDFs by incorporating problem - based learning (PBL) activities into the classroom. The researchers found statistically significant differences between th eir treatment and control groups on post - post - 2 O and CH 4 mole cules does hydrogen bonding 2 , HCl, and NH 3 molecule/molecules have dipole - dipole capable of IMFs may not accurately reflec t student understanding of IMFs. It was also unclear how researchers determined correct or partially correct responses. 35 Problem - based learning has been used in other areas to aid student understanding 45 , and that may certainly be th e case here, Ealy attempted to use molecular modeling in the IMFs. T he author found statistically significant differences between treatment and con trol groups on post - test assessment items. Ealy chose to use multiple - choice assessment items to test , like Tarhan and colleagues, included questions that asked students 42 to identify the strongest IMF present for three different compounds. Ealy also conducted statistical analyses on rather small sets of treatment and control groups (Group 1: N =23 and N= 23; Group 2: N =33 and N= 33) and failed to include effect sizes , which would be a better indicator of the magnitude of the effect. 46 Again, the assessment items included here may not be successful at uncovering actual student understanding of IMFs, and thus it is difficult to determine the effectiveness of these molecular modeling activities. Thinking bigger: Addressing the relationship between structure and properties bigger picture of structure - understanding of the various concepts related to the connection between structure and properties has focused on identifying and diagnosing difficulties. Significantly less wor k has been done to address how we can improve student understanding of this core chemistry concept. One possible solution could be the use of learning progressions to describe the progression of ideas and topics needed to achieve a thorough understanding of the structure - property relationship. 47 49 While there has been much debate as to the definition of learning pothetical model progress to more sophisticated levels of knowledge. 48 Several researchers have noted that traditional chemistry curricula often sacrifice breadth for depth; that is, they emphasize a large n - integrating and connecting a small number of core ideas. 47,50,51 Corcoran and colleagues stress 43 erstanding of the relevant concepts and how they learn to clearly define an appropriate sequence of topics needed to develop a deep understanding over time. Additionally, learning progressions should be tested for earning progression produce better results for most 49 Learning progressions have already been developed for a variety of scientific topics including the nature of matte r 47 , scientific modeling 52 , energy 53 , and chemical thinking 54 . These progressions can be used to aid the structure of chemistry curricula, informing how core concepts are addressed through t he course and the design of effective assessments to highlight improvements in student understanding. Cooper and Klymkowsky have designed a general chemistry curriculum, Chemistry, Life, the Universe, and Everything (CLUE) 51,55 , to address student difficulties, not only with the relationship between structure and properties, but with a wide array of foundational chemistry concepts. The curriculum continuall y revisits and emphasizes three core ideas, structure, properties, and energy, while also focusing on the interconnectivity of these ideas and the role of forces. 51 The interplay of these concepts throughout the first semester of the course can be seen in Figure 3.2. 44 Figure 3.2: first semester . Reprinted with permission from Cooper, M. M.; Klymkowsky, M. W. J . Chem . Educ. 2013 , 90 , 1116 1122. 51 Copyright 2015 American Chemical Society. Cooper and Klymkowsky designed the curriculum using a purposefully selected progression of topics t hat build off of each other and reflect the tenants of meaningful learning. 56 They have incorporated a learning progression for the structure - property relationship 3 , and have students enrolled in the CLUE course experienced less difficulties constructing appropriate Lewis structures and were better able to iden tify of the chemical information encoded in these structures than their traditional curriculum counterparts. 2,3,7 We will explore additional evidence of CLUE stu We know that students possess a wide range of alternative ideas and misunderstandings when it comes to intermolecular forces and the relationship between structure and properties. 45 While much work has been done, there is certainly room for additional research into these ideas. between structure and properties through interviews. Previous work fro m our research group indicated that many students did not believe that physical and chemical properties could be determined from a Lewis structure. 2 We were interested to see if students could effectively use structures of various compounds to predict physical and chemical properties and, if not, then how did they determine the properties of a s ubstance? 46 REFERENCES 47 REFERENCES (1) Feynman, R. P.; Leighton, R. B.; Sands, M. The Feynman Lectures on Physics, Desktop Edition Volume I ; Addison Wesely: Boston, MA, 1977; Vol. Vol. 1. (2 ) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Chem. Educ. Res. Pract. 2012 , 13 , 195 200. (3) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. J. Chem. Educ. 2012 , 89 , 1351 1357. (4) Cooper, M. M.; Corley, L. M.; Underwood, S. M. J. Res. Sci. Teach. 2013 , 50 , 699 721. (5) DeFever, R. S.; Bruce, H.; Bhattacharyya, G. J. Chem. Educ. 2015 , 92 , 415 426. (6) Cooper, M. M.; Grove, N.; Underwood, S. M.; Klymkowsky, M. W. J. Chem. Educ. 2010 , 87 , 869 874. (7) Underwood, S. M.; Reyes - Ga stelum, D.; Cooper, M. M. Sci. Educ. in press . DOI: 10.1002/sce.21183. (8) Shane, J. W.; Bodner, G. M. Chem. Educ. 2006 , 11 , 130 137. (9) Birk, J. P.; Kurtz, M. J. J. Chem. Educ. 1999 , 76 (1), 124 128. (10) Peterson, R. F.; Treagust, D. F.; Garnett, P. J. Res. Sci. Teach. 1989 , 26 , 301 314. (11) Maeyer, J.; Talanquer, V. Sci. Educ. 2010 , 94 , 963 984. (12) McClary, L.; Talanquer, V. Int. J. Sci. Educ. 2011 , 33 (10), 1433 1454. (13) Smith, K. C.; Nakhleh, M. B. Chem. Educ. Res. Pract. 2011 , 12 (4) , 398 408. (14) Othman, J.; Treagust, D. F.; Chandrasegaran, A. L. Int. J. Sci. Educ. 2008 , 30 (11), 1531 1550. (15) Pierri, E.; Karatrantou, A.; Panagiotakopoulos, C. Chem. Educ. Res. Pract. 2008 , 9 (3), 234 239. (16) Osborne, R. J.; Cosgrove, M. M. J. Res. Sci. Teach. 1983 , 20 (9), 825 838. (17) Bodner, G. M. J. Chem. Educ. 1991 , 68 (5), 385 388. (18) Chem. Educ. Res. Pract. 2008 , 9 , 219 224. 48 (19) Papageorgiou, G.; Stamovlasis, D.; Johnson, P. M. Int. J. Sci. Educ. 2010 , 32 (5), 629 652. (20) Nakhleh, M. B. J. Chem. Educ. 1992 , 69 (3), 191. (21) Griffiths, A. K.; Preston, K. R. J. Res. Sci. Teach. 1992 , 29 , 611 628. (22) Israelachvili, J. N. Intermolecular and surface forces , Third.; Academic Press: San Diego, CA, 2011. (23) Margenau, H.; Kestner, N. R. Theory of intermolecular forces ; Pergamon Press: Elmsford, NY, 1969. (24) Compendium of Chemical Terminology , 2nd ed.; McNaught, A. D., Wilkinson, A., Eds.; Blackwell Scientific Publications: Oxford, 1997. (25) McMurry, J. E.; Fay, R. C. General chemistry: Atoms first ; Pearson Education, Inc: Upper Saddle River, NJ, 2010. (2 6) Goh, N. K.; Khoo, L. E.; Chia, L. S. Aust. Sci. Teach. J. 1993 , 39 (3), 65 68. (27) Coll, R. K.; Taylor, N. Res. Sci. Technol. Educ. 2001 , 19 (2), 171 191. (28) Taber, K. S. Chemical misconceptions: Prevention, diagnosis and cure ; Royal Society of Chemistry: London, 2002; Vol. 1. (29) Schmidt, H. - J.; Kaufmann, B.; Treagust, D. F. Chem. Educ. Res. Pract. 2009 , 10 , 265 272. (30) Henderleiter, J.; Smart, R.; Anderson, J.; Elian, O. J. Chem. Educ. 2001 , 78 (8), 1126 1130. (31) Barker, V.; Millar, R. Int. J. Sci. Educ. 2000 , 22 , 1171 1200. (32) Pereira, M. P.; Pestana, M. E. M. Int. J. Sci. Educ. 1991 , 13 , 313 319. (33) Taagepera, M.; Arasasingham, R.; Potter, F.; Soroudi, A.; Lam, G. J. Chem. Educ. 2002 , 79 , 756 762. (34) Villafañe, S. M.; Ba iley, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Biochem. Mol. Biol. Educ. 2011 , 39 , 102 109. (35) Tarhan, L.; Ayar - Kayali, H.; Urek, R. O.; Acar, B. Res. Sci. Educ. 2008 , 38 , 285 300. (36) Özmen, H. J. Sci. Educ. Technol. 2004 , 13 , 147 159. (37) Taber, K. S. Chem. Educ. Res. Pract. 2001 , 2 (2), 123 158. 49 (38) Nahum, T. L.; Mamlok - Naaman, R.; Hofstein, A.; Krajcik, J. Sci. Educ. 2007 , 91 (4), 579 603. (39) Nicoll, G. Int. J. Sci. Educ. 2001 , 23 (7), 707 730. (40) Cokelez, A.; Dumon, A. C hem. Educ. Res. Pract. 2005 , 6 (3), 119 135. (41) Johnstone, A. H.; Selepeng, D. Chem. Educ. Res. Pract. 2001 , 2 (1), 19 29. (42) Wilcox, C. J. J. Chem. Educ. 1998 , 75 (10), 1301. (43) Nicholls, L. J. J. Chem. Educ. 1983 , 60 (11), 993. (44) Battle, G. M.; Allen, F. H. J. Chem. Educ. 2012 , 89 (1), 38 44. (45) Eberlein, T.; Kampmeier, J.; Minderhout, V.; Moog, R. S.; Platt, T.; Varma - Nelson, P.; White, H. B. Biochem. Mol. Biol. Educ. 2008 , 36 (4), 262 273. (46) Sullivan, G. M.; Feinn, R. J. Grad. Med. Eduation 2012 , 4 (3), 279 282. (47) Stevens, S. Y.; Delgado, C.; Krajcik, J. S. J. Res. Sci. Teach. 2010 , 47 (6), 687 715. (48) Duschl, R.; Maeng, S.; Sezen, A. Stud. Sci. Educ. 2011 , 47 (2), 123 182. (49) Corcoran, T.; Mosher, F. A.; Rogat, A. Learning progressions in science: An evidence based approach to reform ; RR - 63; Consortium for Policy Research in Education: Teachers College - Columbia University, 2009. (50) Schmidt, W. H.; Wang, H. C.; McKnight, C. C. J. Curric. Stud. 2007 , 37 (5), 525 559. (51) Cooper, M. M.; Klymkowsky, M. W. J. Chem. Educ. 2013 , 90 , 1116 1122. (52) Schwarz, C. V.; Reiser, B. J.; Davis, E. A.; Kenyon, L.; Achér, A.; Fortus, D.; Shwartz, Y.; Hug, B.; Krajcik, J. S. J. Res. Sci. Teach. 2009 , 46 , 632 654. (53) Neum ann, K.; Viering, T.; Boone, W. J.; Fischer, H. E. J. Res. Sci. Teach. 2013 , 50 , 162 188. (54) Sevian, H.; Talanquer, V. Chem. Educ. Res. Pract. 2014 , 15 (1), 10 23. (55) Cooper, M. M.; Klymkowsky, M. W. CLUE: Chemistry, Life, the Universe & Everything http://besocratic.colorado.edu/CLUE - Chemistry/ (accessed Apr 28, 2015). (56) Bretz, S. L. J. Chem. Educ. 2001 , 78 , 1107 1117. 50 CHAPTER IV: AN INVESTIGATION OF COLLEGE CHEMISTRY ST UNDERSTANDING OF STR UCTURE - PROPERTY RELATIONSHI PS Preface This chapter discusses findings from our research into how students understand the relationship between structure and properties. This research has been previously published in the Journal of Research in Science Teaching and is reproduced here in full with perm ission from publisher John Wiley and Sons: Cooper, M. M.; Corley, L . M.; Underwood, S. M . An Investigation of College Chemistry - Property Relationships. J. Res . Sci. Teach . 201 3 , 50(6) , 699 721 . DOI: 10.1002/tea.21093 A copy of permissions o btained via RightsLink is included in Appendix A. Introduction The rel ationship between the molecular - level structure of a substance and its properties is a core concept of chemistry and a vital skill for understanding a subject like organic chemistry. Th e foundational idea that the arrangement of atoms and electrons in a substance directly affects the macroscopic, observable properties of that substance is powerful and can provide students with a scaffold on which to build their understanding of a wide ra nge of chemical phenomena. O ne of the major goals of effective chemistry instruction , therefore, must be to help students learn the knowledge and skills that will allow them to make th e connection between molecular - l evel structure and macroscopic behaviors in a meaningful way. Without a robust understanding of the underlying ideas that allow the structure - property connection, there is no organizing framework for most of chemistry and students , out of necessity , resort to memorization and 51 generation of heur istics. Nowhere is this more true than in organic chemistry where literally hundreds of seemingly different reactions and interactions can be introduced within one course. If students are unable to use structural cues to determine how and why molecules int eract , we cannot be surprised when organic chemistry is thought to be all about memorization. Unfortunately, the road from structure to properties (and back) requires a long chain of inferences and the application of sets of rules that may appear to be unc onnected to the goal at hand. We have previously reported that students have great difficulty with many of the tasks required along the road from structure to properties, including drawing structures themselves and using structures to predict both physical and chemical properties. 1 3 Our goal in the work discussed here was to delve m ore deeply into the ways in which col lege students use the molecular - level structure of a substance to predict its macroscopic properties. Before we can begin to address the difficulties that students clearly have, it is important to identify how these pro blems arise so that more effective curricula and pedagogical approaches may be developed. Background Misconceptions, c onceptual change, and dual p rocessing Misconceptions Most educators would agree that the development of conceptual understanding is a major, yet somewhat elusive, goal of all science education. As the NRC committee on Discipline Based Science Ed ucation reports , it is important to begin by identifying what students know, how their ideas align with normative scientific and engineering exp lanations and practices (i.e., expert knowledge), and how to change those ideas that are not aligned . 4 Indeed conceptual (sometimes known as alternate con ceptions, or 52 naïve ideas) that would hinder student understanding of chemical concepts is a major , active research area in chemistry education. While much of the work on conceptual understanding focuses on younger children 5,6 , it is clear that college students also have a wide range of misconceptions . 7 13 I ndeed , over 120 papers on conceptual understanding in chemistry have been published in the last decade. 4 In this research , we investigate college students understanding about how structure affect s physical properties. W hile this is a core concept of chemistry , it has not been the focus of many studies. For example, Smith & Nakhleh report that many college chemistry students retain the well - documente d 14,15 misconception that when a substance is melted, covalent bonds are broken ( rather than intermolecular forces being overcome ). 16 T he focus of the ir study was no t the structure of the compounds but rather the process of melting or dissolution. To date , in fact, there has been little research about the origin of such ideas or how students understanding of structure impacts their models of phases or phase changes. While there are hundreds of different ideas that have been categorized as misconceptions, their origin and extent differ widely. Ch i has proposed a tripartite classification of incorrect student beliefs, ranging from the level of a single idea to ideas th at are robust, pervasive, and stem from multiple sources . 17 Perhaps what is most relevant to chemistry instruction is that the deep underlying ideas of chemistry, upon which the rest of the subject is scaf folded, are rarely based on a single concept, idea, representation, or definition. Even a seemingly simple task requires students to organize and synthesize a huge amount of information. For example , the cules 18 and that bond breaking releases energy 19,20 are widely prevalent, but the sour ces of confusion are complex. A correct explan ation of why sodium chloride does not form molecules or why bond breaking is endothermic , is 53 complex and would require an understanding of a range of ideas and a great deal of cognitive effort . Similarly, the focus of this paper, structure - property relatio nships, requires students to concatenate a sequence of inferences and apply several sets of rules before they can provide a meaningful prediction about structure - property relationships. Conceptual c hange While a misconception at the level of a single fa ct may be addressed by revising or rebuilding the idea itself, overcoming flawed mental models involves conceptual change. Mental models are students internal representations of phenomena and , while they need not be accurate, they must be functional and m may be restricted by their previous experiences with a similar task, technical background , and how they think about the system . 21 23 Constructing appropriate mental models is particularly important in ch emistry since much of chemistry deals with scales that are not visible . While much has been written about teaching for conceptual change , there are, in fact, no evidence - based, well - tested theories of conceptual change that are widely accepted. 24 Most researchers agree that students bring a collection of assumptions, ideas, and skills with them. The researchers differ , however, in that some interpret student ideas about concepts as fairly co herent (if naïve or mistaken) explanatory frameworks 17,25 , while others sup port an approach in which students construct loosely woven explanations of phenomena from smaller fragments . 26,27 As diSessa has pointed out, these diff erent theories may necessitate quite different instructional approaches to enact conceptual change. 24 If students have a somewhat coherent ( but incorrect ) theory about a particular concept t hen it theory through 54 dialog and appropriate instruction . 28 construc t explanations that are loosely woven , highly contextualized, and , then different instructional strategies may be needed. For example , Linn has propo sed scaffolded knowledge integration frameworks that may promote more robust and coherent models. 29 Another possible approach to t he development of coherent conceptual development involves l earning progressions that explicitly develop difficult ideas in a way that allows students to integrate the fragments and ideas into a coherent whole are also proposed as a way to help students de velop more robust and self - consistent conceptual frameworks . 30 34 It may be the case that both approaches are valid in different situations. In either case, it is important to ascertain the knowledge and assumptions (both explicit and implicit) that students bring with them before any attempt to develop instructional materials designed to improve student under standing. With these ideas in mind, our initial goals were to elicit student ideas about the connection between structure and function, to investigate the origins of these ideas and to see how coherent they were. However, as the interviews progressed, we r ealized that another factor was emerging: instead of using the methods that they had been taught, to elicit structure - property connections, many students were using self - generated (personal) shortcuts or heuristics. Heuristics and dual p rocessing Most o f the earlier research and proponents of both approaches for promoting conceptual change have been focused on systems with macroscopically observable behavior that are often encountered in physics and physical science instruction or macroscopic biological systems. However, in subjects like chemistry that encompass not only the macroscopic level but also the 55 molecular - level, there is an additional level of abstraction since students are unable to directly observe phenomena and therefore must rely on increasi ngly complex representational systems to understand concepts, models, and ideas. Because of this abstraction, molecular - level The difficulty in navigating be tween molecular, symbolic and macroscopic domains has long been understood, but it is exacerbated as the representational systems that must be used to encode information increase in complexity. 35 Experienced chemists can look at a chemical structure and determine the shape, areas of high or low electron density, types of intermolecular forces (IMFs), acidic hydrogens, and rea ctive centers almost automatically. But beginning students , ideally, must go though a long sequence: 1. Construct an appropriate structure that contains enough information to make further inferences (typically taught using a set of rules) showing where all the bonds and non - bonding electrons are located, 2. Translate the two - dimensional structure to a three - dimensional structure (using another set of rules), 3. Use knowledge of relative electronegativities of atoms to predict bond polarities, 4. Use the thr ee - dimensional structure and bond polarity information to make inferences about the overall polarity of the molecule, 5. Use this information to determine the types of IMFs that cause interactions between molecules, and 6. Use all this information to predi ct how molecules will interact . 2 So , while the co ncept that the m olecular - level structure can be used to predict properties is central, its application is complex and difficult and we should not be surprised when students struggle, even after several years of college chemistry courses. In fact, instruct ors have implemented a range of heuristics that are taught to students to help them construct molecular representation s and use them to predict properties . For example, the er how or why 56 substances will be soluble in a given solvent. Such heuristics allow rapid decisions and predictions to be made without considering the ideas that allowe d their development. While th ey it is important to remember that they are not explanations for a particular phenomenon or concept. There are only a few studies that have investigated the use and development of such heuristics in chemistry. Taber , for example, has reported on the problems arising from the use of . 11,36 In addition to heu ristics that are explicitly taught in classroom contexts (instructionally derived) , it has been shown that students also develop their own heuristics to help them simplify the reasoning that must be used to answer complex questions. 37,38 Maeyer and on typical general chemistry examinations one of four heuristics: recognition, representativeness, one reason decision - making, or arbitrary trend. These researchers did not explicitly ask students to use the structure of the substance to make predictions and rankings, but rather asked them to dis cuss the criteria they used to make their , in part, to the complexity of the task. 39 The extensive use of such heuristics has been explained by dual process theories of human cognition, which have been developed in a number o f disciplines. 40 For example , Stanovich and We st introduced the idea of System 1 and System 2 types of thinking , where System 1 thinking processes. 41 System 1 thinking is the source of the well - documented literature on cognitive biases and simple errors . 42 Most people use S ystem 1 the major ity of the time: it allows for the 57 perform ance of multiple tasks simultaneously , and does not require a great deal of cognitive effort . On the other hand, System 2 thinking is sequent ial, deeper, and requires effort and attention, resulting in the thinker bearing down on the idea at hand and concentrating hard. One of the difficulties in learning science (or learning anything) is that it is almost always necessary to use System 2 think ing processes and to consciously over - ride System 1. The use of heuristics or shortcuts allows us to use System 1, the default mode, when considering complex problems , and the heuristics we teach are designed to do just this. I t is not surprising , therefor e, in situations where an extensive chain of inference is required, such as relating molecular structure to properties, that students may also develop their own heuristics to answer questions rather than rely on their knowledge of scientific principles . Wh ile t he use of heuristics becomes ever more necessary as the chemistry becomes more complex, and experienced chemists automatically default to them to lessen the cognitive load of a particular task, it is important to remember that they are not explanation very helpful when determining how to draw correct structures, however it tells us nothing about no insight into the molec ular level processes, energy and entropy changes that are associated with the formation of a solution. Purpose and significance of the study In this study , we interviewed students from general and organic chemistry . W e specifically asked h ow they would us e the molecular - level structure to determine physical properties suc h as melting and boiling point. Our focus was on organic chemistry students since a robust understanding of the principles of organic chemistry is predicated on the idea that students can 58 predict how a substance will behave from an inspection of its molecular structure. We included general chemistry students because it is in general chemistry where these skills are first developed, and we wanted to see how (whether) these skills change over time. Our goals were to better understand the process by which students determined properties such as relative melting or boiling points from a structure and what factors they took into consideration (i.e. molecular geometry or polarity). While other stud ies have looked at student reasoning about relative phase change temperatures or solubilities 16,39 , none have explicitly hese properties. Therefore, we decided to use simple structures so that we did not overwhelm the students. For example, while some common substances like fats and sugars may be familiar to students, their structures may be too complex for a novice to analy ze. We believe this study is important because it probes a fundamental construct of chemistry that all students should have mastered by the time they finish general chemistry. In most general chemistry courses (and certainly the courses that these students were enrolled in) the topics covered by our interviews make up about 25% (five chapters out of 20 chapters that are taught), and approximately 50% of the material in a first semester general chemistry course. Indeed, by the time students reach organic che mistry most instructors spend little time on the development of these skills because they are such an integral part of the prior knowledge that is expected. While most organic chemistry textbooks do briefly review structure property relationships in the fi rst chapter, the majority of the course is taken up with more advanced concepts that build on these ideas. For example, how molecules interact to produce new products, how changes in molecular structure and interactions are related to energy changes, and h ow the three - dimensional structure can be represented and understood in 59 two - dimensional drawings. All of these ideas and skills are predicated upon the kind of understanding that we were probing in this study. This study is part of a larger series of stu dies in which we have use d a variety of methods to investigate student understanding of molecular structure and properties. In our earlier studies , we investigated whether students were able to draw and use chemical structures to make predictions about pro perties. 1 3 The study on which we report here aims to elucidate why students hav e such trouble with this concept . Using a basic , qualitative research design , a semi - structured interview protocol was implemented - property relationships. Taken together, these studies consist of a mix ed - methods sequential explanatory study. 43 Our study focused on two research questions: RQ 1: Do students use molecular - level structures to make predictions about the macroscopic properties of a substance, and if so how? RQ 2: How do students enrolled in general and organic chemistry use representations of chemical structure s to make pr e dictions about macros copic properties of substances ? Methods Setting and p articipants This study was conducted at a public southeastern research university of about 20,000 students. At this university, g eneral chemistry and organic chemistry are taught in lecture sections of between 100 and 150 students. Approximately 1500 students enroll in general chemistry per semester and 6 00 in organic chemistry. P articipants were volunteers who were 60 solicited by ema il from second - semester general chemistry students (GC2, N = 7), first - semester organic chemistry students (OC1, N = 5), and second - semester organic chemistry students (OC2, N = 5). In order to participate, students must have completed at least on e semeste r of general chemistry; t his was to prevent undue confusion for students who had not yet been exposed to topics relevant to understanding the relationship of structure and property such as polarity and intermolecular forces. All of these students signed in formed consent forms. Of the 17 participants, 5 were male and 12 were female; 11 participants pursued biology - related majors, 3 were chemistry majors, and 3 were engineering majors. All participants received either an A or a B in prior chemistry coursework . All of the students were enrolled in knowing these ideas (organic chemistry). The students came from different courses taught by different instructors, usin g different pedagogies. In general, most students in these courses completed on - line homework assessments and in class written quizzes where they would write or draw a response, often after group discussion. In general chemistry, all the course examination s were multiple choice, but in organic chemistry typically about half the examination was composed of student constructed r esponses. All students in these courses took final examinations in the form of American Chemical Society normalized examinations 44 and, on average, scored above the national norm. It is important to state here that these are student s who have done everything that is asked of them, and who appear to have a firm command of the material when traditional assessments are used. What follows is in no way intended to imply that the problems we uncover lie with the students. As we will discus s later, we believe it is the structure of the curriculum and the accompanying assessments that do not provide an appropriate learning environment in which students can be expected to develop these complex ideas. 61 I nterview protocol The semi - structured inte rview protocol began by asking students what kinds of tests they might use in a chemistry laboratory to identify a substance. This was to help students recall chemical and physical properties with which they might be familiar. If students did not spontaneo usly respond with melting point or boiling point tests, the interviewer suggested that such properties were measurable in the laboratory. Students were then asked specifically, in reference to water, ammonia, and ethane, about the types of properties the c ompound might exhibit. These were compounds that were (or should have been) familiar to the students. The interviewee was then asked to construct a Lewis structure or other structural representation for these compounds and asked how they might use that str ucture to help them explain the properties (particularly melting point or boiling point) of that compound. The second portion of the interview was designed to reflect the types of questions the students would typically experience in their chemistry course . Students were given several pairs of compounds and asked to pick, for each pair, the compound that would have the higher boiling point and explain why. Table 4. 1 lists each of the pairs given and the reason why they were chosen. The table also includes r easoning that we would expect a student to use when explaining why one compound would have a higher boiling point than the other. The compounds chosen for discussion were simple structures containing no more than two carbon atoms . It should be noted that i f students had difficulty constructing any of these structures throughout the protocol, the interviewer would provide the student with structural cues such as clarifying dimethyl ether as CH 3 OCH 3 . Additional methods outlining the full interview protocol ar e available in Appendix B . 62 Table 4.1: Pairs of compounds presented to students, the reasons for choosing this comparison, and the expected student reasoning Pairs of compounds Reason for our choice Expected student reasoning CH 3 CH 3 and CH 3 CH 2 OH Different molecular weights, different types of IMFs Ethanol has a higher boiling/melting point because it has stronger intermolecular forces (specifically hydrogen bonding and dipole - dipole), which require more energy to overcome during a phase change. CH 3 OH and CH 3 CH 2 OH Different molecular weights, same types of IMFs Ethanol has a higher boiling/melting point because it has stronger London dispersion forces, which require more energy to overcome during a phase change. CH 3 CH 2 OH and CH 3 OCH 3 Structural isomers, sa me molecular weights, different types of IMFs Ethanol has a higher boiling/melting point because it has stronger intermolecular forces (specifically hydrogen bonding), which require more energy to overcome during a phase change. Data collection A post - d octoral researcher conducted the first five interviews and was then joined by a graduate student. After co - conducting four interviews, the remaining eight interviews were conducted by the graduate student (the second author on this paper) . The length of th e interviews varied from 30 to 60 min depending on the amount of information that the students provided. For the interviews, audio and student - constructed representations were collected using a LiveScribe pen, which can replay both the audio and student dr awings in real - time. 45 Audio was also recorded using a digital voice recorder . Data analysis After t he interviews were conducted, a post - doctoral researcher, an undergraduate research assistant , and a graduate student transcribed them verbatim. Using a qualitative approach based on grounded theory techniques , a graduate student, a post - doctoral researcher , and a faculty member analyzed the interview transcripts and LiveScribe data for emergent themes and 63 commonalities using o pen coding . 46 Initial codes were created and revised via constant comparison . 47 Multiple revisions were required in order to address the complex nature of student knowledge of the structure - function relationship. After several iterations of coding , four over - arching themes were identified that encompassed the major issues experienced by students in their explanations of this relationship. Although these themes stemmed from widely differing sources, each was identified as c ontributing to student difficulties and emerged during th e reasoning tasks. These themes are: 1) i nappropriate models of phases/phase change, 2) representational difficulties, 3) l anguage and terminology issues, a nd 4) u se of heuristics in student reasoning (whether appropriate or not). Each of these themes in s tudent difficulties consists of a number of subcategories that we collapsed together to produce the major concept. Findings - property relationships In response to our first research question (RQ 1), we present examples of each overarching theme that emerged from the interviews and then illustrate how these themes combined and were used by students to make predictions about melting and boiling points of various substances . Table 4. 2 presents e their level of chemistry, and which of the four main over - arching themes were present for their reasoning . 64 Table 4.2: interview. A count is shown for a sp ecific category if the student showed at least one instance of the code during their interview . Pseudonym Course Phase/ Phase Change Represen - tations Terminology Heuristics Personal Instruc - tional More Means More Noah GC2 1 0 1 1 1 1 Brittany GC2 1 0 1 0 1 1 Tina GC2 1 1 1 1 1 0 Susan GC2 0 0 1 1 0 0 Erin GC2 0 1 0 0 1 0 Lucy GC2 0 1 1 1 1 0 Justin GC2 0 1 0 1 0 1 Robin OC1 0 1 0 1 1 1 Ted OC1 0 0 1 1 1 1 Lily OC1 1 1 1 1 1 1 Marshall OC1 1 0 1 1 0 1 Victoria OC1 0 0 1 0 0 1 Daisy OC2 1 0 1 1 0 1 Joy OC2 0 1 1 1 1 1 Jill OC2 1 1 1 1 0 1 Jane OC2 1 0 1 1 0 1 Joe OC2 1 1 1 0 1 1 Models of phases or phase c hange Eight of our interviewees did not possess a coherent model of the structure of solid, liquid, and gaseous simple molecular co mpounds, which typically emerged when students were asked to draw structures representing different phases. Joe (OC2) struggled with drawing a molecular - level depiction of a solid. H e seemed concerned about the idea that ethane might form a solid because, if ethane did form a solid, this I would say they, if compact He attempted to draw his id ea of what bonded ethane would look like in the solid 65 phase , with molecules bonded together, as seen in Figure 4. 1 (a) . Figure 4.1: representation of (c) water going from the solid phase to the liquid phase and (d) water going from the liquid phase to the gaseous phase Similarly, Jill (OC2), experienced difficulties depicting solids and liquids on the molecular - level. Her representations of solids and liquids appear to show that they are covalently bonded in a network structure rather than held together via intermolecular forces. Jill was quite consistent with her depictions of solids as networks, as seen in Figure 4.1(b ) with solid ethane , using the same idea for wat er, ammonia, and dimethyl ether . While Jill and Joe experienced difficulties explaining their model of phases, Brittany (GC2) struggled to explain the process of a phase change. When asked explicitly to describe the process of ice melting on the molecular - level, she responded , . 66 like movable I guess. Malleable . It is well documented that students often confuse intermolecular forces with covalent bon ds 14,48,49 , and i t is entirely possible that Brittany was confused about the difference betw een intermolecular forces and covalent bonds , but her structural representations of water in the solid and liquid state still contain water molecules. As she discussed the transition to a gas, however, Brittany broke the covalent bonds within the water mol ecules; she drew structures for us that clarified what she meant, clearly showing interactions between molecule s breaking from solid to liquid and H O covalent bonds breaking from liquid to gas as s een in Figure 4.1(c) and (d) . Each of these students strug gled with their understanding of either phase or phase changes. They each provided different representations in an attempt to explain their model, but in doing so, they became aware that something was wrong. Their inability to construct appropriate repres entations and extract meaning from them severely hindered their understanding. If students cannot provide an appropriate representation for each phase, it is unlikely that they will be able to make predictions about relative phase change temperatures or th e factors that affect phase changes. Use of representations Nine participants experienced some form of representational difficulty during their interview, although not all were directly related to phase or phase changes. network m od el of solid ethane in Figure 4.1(b ), it is understandable that she explained the process of boiling in terms of bond breaking since her representations made no reference to intermolecular forces. She indicated in her interview that, as a solid melted, so me of the bonds 67 were broken. Then, as the substance moved into the gas phase, the individual molecules separated. Her difficulty with this concept was apparent, since there were numerous false starts in her attempt to explain the bonds that break during th e boiling process of dimethyl ether: Like if you broke this (C - H bond in Figure 4.2( a )) versus (Figure 4.2( b) ) one dimethyl ether and one anhydrous, or one with a partial negative charge sounds better (pointing to Figure 4.2(b)) . Figure 4.2: from Her first approach, Figure 4.2(a), is consistent with the lattice form she drew for previous compounds in the interview. With this representation, she realizes that, by breaking bonds during the boiling phase change, she would have resulting dimethyl ether molecules missing hydrogens. 68 In her second approach, Jill redraws her lattice structures, connecting dimethyl ethers with covalent bonds between the hydrogens. While this solves the problem of losing hydrogens in the boiling process, she voices concern that now each hydrogen would have two bonds and hydrogen It is important to note that she refers to the tendency for hydrogen to only form one bond as the rule of hydrogen bonding (a possible terminology issue). She understood that network structures despite the fact that she is in her fourth semester of college level chemistry. A subtle r problem emerged from Lucy (GC2), who knew that the strength of the IMFs determines boiling points. Her problem , however, originated with her difficulties in translating the two - dimensional Lewis structure into a three - dimensional shape. Since she drew dimethyl ether as linear (Fi gure 4.2(c)) , she believed that it was non - polar because the bond polarities cancelled : So if you were gonna kind of split this up , oxygen obviously has the slight going this, umm opposite ways, it goes towards the negative and then if this is going (drawing) towards the negative then these tw o arrow s cancel each other out. In fact, only three of the students, all of whom were in general chemistry, used bond polarity vectors to determine molecular polarity and the resultant types of IMFs. Language and terminology It has been well documented that st udents often struggle with the use of scientific language in their chemistry courses. 50 Some of this difficulty stems from the use of words that have not only 69 a specific meaning in chemistry , but also a more colloquial use. For example, students may say that a r eaction has come to equilibrium but not understand that the process is still ongoing or that the use of the term volatile, commonly meaning explosive or unstable, is used in science to indicate a substance that is easily vaporized. Most of the participant s (14 out of 17) experienced some form of terminology and language problems . During our interviews, i t became clear that many were confused about the meaning of words that describe interactions such as bonds , intramolecular forces, and intermolecular force s (such as hydrogen bonding, dipole - dipole, and London dispersion forces) . Jane (OC2) was a ware that these terms are easily confused : U mm hydrogen bonding is between if, is it intermolecular force? I always get that shows lik e the attractions. That is, Jane, while not sure of the name for intermolecular forces , did understand that these inter actions are between two molecules. Joy (OC2) , on the other hand, illustrated hydrogen bonding as both within and between molecules while drawing her structure for ammonia . Interviewer : Ok. So could you just show me how it hydrogen bonds? Joy ht there between the H and the N. Interviewer : So if you had another ammonia molecule could you draw another one for me? Joy : (drawing additional ammonia with dashed line between the molecules ) Interviewer : Ok so umm what wou ld this be? Like that , you just dre w dotted? 70 Joy : Oh this is a hydrogen bond. Interviewer : Ok. And you said that this (indicating the N - H bond) is also a hydrogen bond? Joy : Yeah. Her depiction of hydrogen bonding , seen in Figure 4. 3, showed both an inter action between the nitrogen of one ammonia molecule and the hydrogen of another as well as the N - H bond within one molecule (indicated by hydrogen bonding is understandable, since , typically, t he term bond is used between two atoms in a molecule. At the same time, she also remembered hydrogen bonding as existing between two molecules. To compromise these two ideas, she decided that it could be both. Figure 4.3: ing in ammonia, both within a molecule and between two molecules Unfortunately, the idea that intermolecular forces are what most textbooks refer to as covalent bonds was quite pervasive. Ted (OC1) also struggled with this idea. At the beginning of his i nterview, Ted refer r ed to hydrogen bonding as the bond between oxygen and hydrogen within the water molecule rather than an interaction between two water molecules. This became 71 a significant terminology problem that followed him throughout his interview as seen when he later compared the relative boiling points of ethanol and ethane: lot stron lot stronger bond so you automatically have higher, a higher bo iling point than this (ethane). After further questioning from the interviewer about what specifically breaks when a substance boils, Ted realized there was a n inconsistency in his prior reasoning and the bonds within a molecule should not be breaking. He then proceeded to describe an attractive force present between the molecules, but , since he had already allocated the term hydrogen bonding to the O - H bond, he called this force a Van der Waals interaction: s) but there is like a since interactions . While Ted correctly identified that there are attractive forces between two ethanol molecules and had an understanding of the underlying concept of intermolecular forces, his difficulties with terminology created challenges for him in communicating this knowledge. Presumably , this terminology issue could also create problems in making sense of lectures, notes, and textbook 72 content. It should be noted that he described hydrogen bonds and London dispersion forces as within molecule interactions and Van der Waals forces as between molecules interactions. Thes e examples of student difficulties related to representing and communicating phases and with phases occur r ed i n combination with their difficulties in producing appropriate representations or terminology issues. Each of these problematic areas combined with others in Use of heuristics in student r ea soning In the second half of the interviews, students were given the three prediction tasks shown in Table 4. 1 and were asked to explain their reasoning. We anticipated that students would construct structures, use them to predict the types and strengths o f intermolecular forces present, and then use this information to predict which compound would take more energy to separate the molecules, which they would relate to the boiling or melting point. What emerged for all students, on at least one occasion, was a heuristic that had either been instructionally derived or personal . Interestingly almost all of the heuristics used some version of what may be akin to - prim 51,52 , that is, they almost all involved counting something, be it oxygens, carbons, hydrogens, bonds or intermolecular forces, and used these surface level characteristics to predict properties. While some of t hese heuristics lead to a correct prediction, many of them did not and several were complicated by problems with representations and language that we have already discussed. 73 Heuristics I nstructionally derive d Victoria (OC1) is an example of one stude nt who used a heuristic that she had been taught for reasoning about the differences in boiling points of ethanol and dimethyl ether. She discussed how the presence of oxygen is important to compare the boiling points: Yeah because they both have one oxyg cause of the hydrogen bonding. In this instance, Victoria re lied on sets of heuristics to rationalize the difference in boiling points. While she arrived at a correct conclusion, she did not explain the differences in terms of intermolecular forces. Her reasoning is surface - level, as with many other students, in th at it focused only on the elements and functional groups present, but it did bring her to the correct conclusion. Heuristics P ersonal Interestingly, more students ( N = 10) used a personal heuristic in comparison to a n instructionally derived heuristic ( N = 7). For example, Joy (OC2) developed a similar heuristic to Victoria that relied on the presence of oxygen in organic compounds to determine the relative lower boiling point when oxygen is present. I wanna say this one (ethanol) has an O, a hydroxyl group on it so I feel like it would bo il quicker than the hydrocarbon . While she never explained her reasoning for this relationship, she later changed her explanation (an d her prediction) to include hydrogen bonding when asked to draw out the Lewis structure (including drawing out the O H 74 group) for ethanol. When questioned about why she changed her reasoning, she replied s t looked . In this situation, Joy was even aware herself that she defaulted to a System 1 level thinking and that only after further prompting did she take it a step further to discuss hydrogen bonding. A few students used a phenomenological approach, ind icating that heavier molecules would have a higher boiling point because heavy molecules would be harder to get into the gas phase. Justin (GC2) invoked more he compared the boiling point of methanol a nd ethanol. . Robin (OC1) , however, used a more : (methano ideas may stem from an inappropriate application of gravitational effects , which are negligible at the molecular scale, or may simply be another p - prim heavier things are hard to move . 53 Robin was incorrect in her predictions for ethanol and dimethyl ether, presumably because her heuristic was not useful for molecules with the same molecul ar weight. In a number of cases, the representational or language issues that led to difficulties with student models of phases were also folded into the reasoning that students used to support their heuristics. For example, some explanations were predicat ed on a model in which covalent bonds means more the number of bo nds within a molecule directly a ffected the boiling point: is more substituted so I wo ss bonds to break . Here, Joe indicated that ethanol had more bonds, which he believed would be 75 broken as the ethanol was vaporized . Thus, his understanding seemed to be that ethanol had a h igher boiling point because more bonds require more energy to break. Jane (OC2) also used a similar personal means more points of ammonia and water, but she used the number of hydrogen bonds as part of her reasoning . Jane (as discussed earlier) used the number of hydrogen bonds to predict that amm onia has a higher melting point than water. Her reasoning still involve d breaking bonds during a phase change (which she call ed intermolecular forces). Additional examples for all themes are available i n Appendix C . - property relationships In reference to our second research question (RQ 2), i t is clear that organic students performed no better than general chemistry students on the se tasks, as seen in Table 4.2, despite their extra semester or two of chemistry. While this may be understandable, since instruction in organic courses typically does not dwell on material that students have presumably mastered in earlier courses, it mean s that organic students do not have a stable foundation on which to build their new knowledge. The major difference we found between the general chemistry and organic more convoluted because they brought extraneous information to their discussions. For example, while comparing the relative boiling points of methanol and ethanol, Jill (OC2) originally determined that ethanol had the higher boiling point but then changed her m ind: 76 Jill Interviewer : Ok . How would you break it? What do you mean when you say break? Jill that are kind of crammed to Jill stated caused ethanol to be less stable than methanol. It is unclear whether she was confusing steric hindrance with steric strain, but she believed that when these substances boil ed, bonds we re broken. She again invoked steric hindrance to correctly predict that ethanol would have a higher bo iling point than dimethyl ether: Cause th ese bonds are gonna be stronger than a carbon, carbon like these two bonds (C O and O H bonds) are gonna be stronger than a carbon hydrogen bond and then this one (dimethyl ether) has got uhh, even more steric hindrance than that one (ethanol) so these bon ds are going to break easier . Jill offered up yet another organic chemistry idea to explain why ethanol would have a higher boiling point than ethane: Because alcohols, the alcohol group is a poor leaving group going to be broken as oxygen bond would be umm more difficult to bre ak than a carbon hydrogen bond. 77 Figure 4.4: depiction of hydrogen bonding between two water molecules indicated by the arrows showing an electron pair leaving and a hydrogen attaching For her explanation, Jill continued to discuss the idea of bond break ing when ethanol and ethane underwent the boiling process. Instead of explaining in terms of steric hindrance, however, she used the idea that - OH (alcohol) groups are poor leaving groups and, as a result, would make the carbon - oxygen bond harder to break. Here we have a confluence of a problematic model of phase chang e involving bond breaking com bined with extraneous knowledge and a lack of distinction between a chemical reaction and a phase change . Although Jill clearly has many problems, Lily (OC1) was the only student interviewed who exhibited at least one instance of every theme outlined in Table 4. 2. When asked about water and how two water molecules would interact: the hydrog en being the partially positive like this bond right here (electron pair bonded to the oxygen) is going to break so that this proton (hydrogen on the interacting water) can come in and so like they are going to bond like that . It q uickly became apparent that Lily saw hydrogen bonding as a reaction in which the electron pair on the oxygen would leave so that the hydrogen on another water could come in and form a hydrogen bond , as seen in Figure 4.4(b) . She had stated earlier in the interview that 78 hydrogen bonding was a bond within the molecule so her confusion is understandable. Inspection of her drawing showed that Lily was using the arrow notation learned in organic chemistry to mean two different things (neither of which was correct). The top arrow showe d It was here that her confusion about the notation used in organic chemistry became apparent. This problem then led to her unique explanation of what happens to water as it changes phase from liquid to gas : Interviewer : Why would it (water) have a high boiling point? Student : Because it takes a lot of energy to break the strong hydrogen bond hold. Interviewer : So which, can you just point to on the paper which bo nd you would be breaking? Student so that they can go and that these protons can come in. Daisy (OC2) used the concept of stability of different types of carbons for her reasoning that ethanol had a higher melting point than methanol: each other so it, and over - methanol . When prompted by the interviewer to explain, Daisy elaborated: mm well you typically think of it as . It became clearer that she did not think the bonds were breaking, but that carbon - carbon bonds stabilized the molecule somehow, so that it took more energy to boil. She li nked this stabilization to what she had learned about stabilization of carbocations in organic chemistry. attached to it. And I know that as you increase that like methyl, primary, secondary, ter tiary your 79 Discussion As T able 4. 2 shows, none of the students provided a completely coherent view of how to predict properties from structures. Interestingly though, most students were able to correctly predict which of each pair would ha ve the h ighest melting or boiling point as shown in T able 4. 3, even though they used some rather surprising reasoning strategies. Table 4.3: Student prediction for highest boiling point in each comparison from the second half of the interview protocol Pse udonym Ethanol vs. Ethane Ethanol vs. Methanol Ethanol vs. Dimethyl ether Noah Ethanol Similar Ethanol Brittany Ethanol Ethanol Ethanol Tina Ethanol Not sure Dimethyl ether Susan Ethanol Methanol (but almost the same) Ethanol Erin Ethanol Ethanol (but almost the same) Ethanol Lucy Ethanol Ethanol Ethanol Justin Ethanol Ethanol Ethanol Robin Ethanol Ethanol Dimethyl ether Ted Ethanol Ethanol Ethanol Lily Ethanol Ethanol Dimethyl ether Marshall Ethanol Ethanol First dimethyl ether, then ethanol Vi ctoria Ethanol Ethanol Ethanol (but almost the same) Daisy Ethanol Ethanol Dimethyl ether Joy First ethane, then ethanol Ethanol Dimethyl ether Jill Ethanol First ethanol, then methanol Ethanol Jane Ethanol Ethanol Dimethyl ether Joe Ethanol Ethanol D imethyl ether The only pair of compounds that more than two students predicted incorrectly was ethanol and dimethyl ether, presumably because th more was not applicable. Therefore, students were forced to move to expla nations involving IMFs (typically hydrogen bonding) to provide a reason. When intermolecular forces were discussed, 80 terms like hydrogen bonding and London dispersion forces were often used incorrectly. Even students who seemed to have a robust understandin g of structu re - property relationships had some discrepancies in their reasoning process. Erin (GC2, a bio engineering major) was one of the most artic ulate and accomplished students; she was able to correctly predict most of the properties of a compound by considering polarity and intermolecular forces. Interestingly, she used her prior knowledge in biology to reason through less familiar topics. For example, she provided a spontaneous (and quite sophisticated) discussion of London dispersion forces using a phospholipid bilaye r (Figure 4.5) as an example: points of time, which is not very often but it does occur, that uhh by chance the electrons line up on one side just because the then that creates a very slight negative charge and that influences this one. So then that causes the electrons to be repelled and it causes a slight positive charge and because of partial negative and partial positive. But they only occur a very few time periods. Even though Erin had a robust understanding of IMFs, she was not clear about the relationship between polarity and shape. Rather than the shape of the molecule influencing its mentioned that . This caused her problems when later reasoning about dimeth yl ether since she thought it was nonpolar, thus making it linear. 81 Figure 4.5: From our interviews, with students who ha ve been successful in their chemistry courses, it is clear that most have significant issues that impede their understanding of the relationship between structure and properties and that the situation does not appear to improve for students who have taken organic chemistry. Despite being specifically asked how they determine the properties of a substance from its structure, few students were able to extend their ideas to predicting and almost invariably invoked a heuristic. It was striking that , even though some students used heuristics that appeared to be very similar on t means more Conclusions What emerged from our interviews was a diverse tapestry of student thinking. Some students based their predictions on one overarching idea (more means more ), some wove their model together from disparate facts and ideas, some students were hindered by their inability to construct and use structural representations, and so me were hindered by language either 82 misremembered or misunderstood. Each student constructed a different set of explanations and even those who had one overarching theme used it differently to come to different conclusions. The major findings of this st udy are: 1. Each student individually constructed a different approach to the task posed. These approaches were hindered by other factors that interacted with each other in different ways. 2. Even students who used what appeared to be a ore means more emerged during the course of the interview. 3. Students in organic or higher - level courses seemed no mo re able to make the structure - property connect ion even though some students could answer questions correctly without consciously reasoning through the process. 4. Organic students sometimes used their ex tra knowledge inappropriately, for example , ratures. What seems clear is that , as we move forward , simply categorizing enough. The ideas and reasoning that students constructed were a result of the interactions between their understanding of what words mean, what structures mean, their models of how phase changes occur, and their willingness and ability to delve deeply into the underlying concepts. As Kahnema n has written, the automatic operations of System 1 generate surprisingly complex patterns of ideas, but only the slow er System 2 can construct thought s in an orderly series of steps . 54 Much of what the students had to say was not self - consistent and a number of 83 students, on reflection or in response to a later prompt , changed their answer to a more scie ntifically reasonable one (see T able 4. 3). Most students seemed to be relying on System I type thinking, rather than going back to first principles. Wh at became clear was that there was no singl e approach to solving this task and that the problems combined in different ways to produce varied results . Some students (especially those who used personal heuristics) ap peared to use reasoning that they could apply fairly consistently (if not correctly), but s tudents who used a similar overarching heuristic often came to differ ent conclusions. It may be that a few of these students reasoning strategies might more closely . While their reasoning might be consistent , it was often based on a flawed model of phase change or an inability to decode the meaning of structural representations and technical terms. However , we belie ve that students often chang ed their responses dur ing the course of the interview in a process more reflective of the concep tual change theories of diSessa where students responses are constructed on the fly from a loosely woven tapestry of facts, skills , and concepts . 26 What seems apparent from our findings is that , even after two years of chemistry courses, we have failed to help students make the crucial link between mole cular - l evel structure and properties. Question s and implications for teaching This study seems to imply that students can take , and do well in, chemistry courses without a thorough und erstanding of a core chemistry concept . All of these students were students who made good grade s and yet many reported that they had never thought about the questions asked in our interviews . That is , students appear to be quite accomplished 84 and yet can h It should be reiterated here that all the students in this study had already been taught this information and passed examinations, including those developed by the chemistry c ommunity (ACS examinations) that are designed to encompass what students should know at the end of a given course. This proble m brings up two major questions: By necessity , the use of reasoning shortcuts increases as stu de nts move through the curriculum. Clearly students in organic chemistry cannot be expected to lab oriously draw out each molecule and go through the long , draw n - out process of determining the 3 - dimensional structure, polarity , and types of IMFs for each qu estion they are asked. overload, since they are also learning new material. However, if the answer to a question is the boiling point is higher because of hydrogen bonding , we must be certain that students mean hydro gen bonding between molecules, rather than within molecules. That is, reason ing shortcuts or heuristics must be based on a firm foundation, otherwise what appear to be reasonable answers to questions may hide fu ndamental problems. What can be done to improve student understanding in chemistry so that when students do use shortcuts, they are appropriate and useful ? Our findings make it clear that there is something wrong with our conventional approach to the deve lopment of these complex ideas. Each student constructed a unique approach to predicting structure - property relationships that emerged from the interaction of the factors discussed above . Although we found misconceptions that had previously been well - docum ented, the ways in which these ideas played out in the con text of the question prompts were different and it is clear that addressing each problematic area separately will not be helpful for students in developing coherent conceptual frameworks. Our findin gs 85 suggest that a scaffolded approach to the development of structure - property relationships, the development of progressions in which students are explicitly asked to connect their prior knowledge to new knowledge, and the explanation of how that knowledg e will be used may help. For example, we have shown that students in a course designed in this way have an improved understanding of structure - property relationships and further studies are being conducted to assess how long these improvements are retaine d. 3,20 What is also clear is that , as students go through organic chemistry, they tend to lose sight of the underlying principles that determine how substanc es interact. That is, while they have more content knowledge than students in general chemistry, they may be no more able to apply basic principles and , in some cases , the extra knowledge may actually impede their understanding. In addition to reconsiderin g how structure - property relationships are taught in general chemistry, we also recommend that the teaching of organic chemistry begin with a thorough, lengthy review of structure, interactions, and properties. While most organic texts begin this way, in o ur experience many instructors assume that an understanding of intermolecular forces and (for example) simple acid - base chemistry are prior knowledge and all that is needed is a brief reminder. Unfortunately, this is not the case. In general, students do n ot begin organic chemistry with a robu st understanding of these ideas - Some students never do catch up and though they may emerge from the course with a database of memorized reactions, they are telling the truth when they inform the next generation of students that organic chemistry is all memorization. For them it can be no other way, since they do not have the tools to understand in a more meaningful way. We also recommend that instructors return to these prin ciples early and often, rein forcing the underlying concepts rather than expecting students to memorize large databases of reactions. 86 Organic chemistry is a terminal course in chemistry for many students and is the last time many students will have the oppo rtunity to develop important and worthwhile skills. Students must be asked to construct and explain their answers, so that their thinking can me made clearer, both to themselves and to their instructors . It is well documented, for example, that socially me diated learning provides opportunities for students to explain and construct understanding . 4 Other approaches may involve explanatory writing 55,56 and the use of modeli ng and construction of models of appropriate systems. 34 I f students are never required to articulate their ideas, it is unlikely that they will have the opportunity to reconstruct them. Limitations of this study In this study , we interviewed 17 students and each student pro vided us with a different combination of models, heuristics, and understanding of the meaning of both words and structures to answer our questions and construct explanations . We do not believe that we have uncovered every potential problem for students or described every student model, but we do think that we have presented a rich picture of the nature of the problem that faces us. We contend that it is not possible, nor is it necessary , to predict all the ways that students use their understandings to cons truct explanations for these phenomena. What is important is that instructors are awar e of the extent of the problem and redesign both curricula and formative assessments to help students explicitly develop and connect these core ideas. 87 REFER ENCES 88 REFERENCES (1) Cooper, M. M.; Grove, N.; Underwood, S. M.; Klymkowsky, M. W. J. Chem. Educ. 2010 , 87 , 869 874. (2) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Chem. Educ. Res. Pract. 2012 , 13 , 195 200. (3) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. J. Chem. Educ. 2012 , 89 , 1351 1357. (4) National Research Council. Discipline - based education research: Understanding and improving learning in undergraduate science and e ngineering ; Singer, S. R, Nielson, N. R., Schweingruber, H. A., Eds.; National Academies Press: Washington, DC, 2012. (5) Barke, H.; Hazari, A.; Yitbarek, S. Misconceptions in chemistry: Addressing perceptions in chemical education ; Springer: Berlin, 200 8. (6) Kind, V. ed.) , 2nd ed.; Royal Society of Chemistry: London, 2004. (7) Bodner, G. M. J. Chem. Educ. 1991 , 68 (5), 385 388. (8) Duis, J. M. J. Chem. Educ. 2011 , 88 , 346 350. (9) Nicoll, G. Int. J. Sci. Educ. 2001 , 23 (7), 707 730. (10) Özmen, H. J. Sci. Educ. Technol. 2004 , 13 , 147 159. (11) Taber, K. S. Int. J. Sci. Educ. 2009 , 31 (10), 1333 1358. (12) Nakhleh, M. B. J. Chem. Educ. 1992 , 69 (3), 191. (13) Hort on, C. Calif. J. Sci. Educ. 2007 , 7 (2), 1 85. (14) Othman, J.; Treagust, D. F.; Chandrasegaran, A. L. Int. J. Sci. Educ. 2008 , 30 (11), 1531 1550. (15) Pierri, E.; Karatrantou, A.; Panagiotakopoulos, C. Chem. Educ. Res. Pract. 2008 , 9 (3), 234 239. ( 16) Smith, K. C.; Nakhleh, M. B. Chem. Educ. Res. Pract. 2011 , 12 (4), 398 408. 89 (17) Chi, M. T. H. In Handbook of research on conceptual change ; Vosniadou, S., Ed.; Erlbaum: Hillsdale, NJ, 2008. (18) Taber, K. S. Int. J. Sci. Educ. 2008 , 30 (8), 1027 1053. (19) Boo, H. K. J. Res. Sci. Teach. 1998 , 35 , 569 581. (20) Cooper, M. M.; Klymkowsky, M. W. CBE Life Sci. Educ. 2013 , 12 , 306 312. (21) Genter, D. In Similarity and analogical reasoning ; Vosniadou, S., Ortony, A., Eds.; Cambridge University Pr ess: Cambridge, 1989; pp 199 241. (22) Johnson - Laird, P. N. Mental models: Towards a cognitive science of language, interference, and consciousness ; Harvard University Press: Cambridge, MA, 1983. (23) Justi, R.; Gilbert, J. In Chemical education: Towar ds research - based practice ; Gilbert, J., De Jong, O., Justi, R., Treagust, D. F., Van Driel, J. H., Eds.; Kluwer Academic Publisher: Netherlands, 2003; pp 47 68. (24) diSessa, A. A. In The Cambridge Handbook of the Learning Sciences ; Sawyer, K. R., Ed.; Cambridge University Press: Cambridge, 2006; pp 265 282. (25) Vosniadou, S.; Vamvakoussi, X.; Skopeliti, I. In International handbook of research on conceptual change ; Vosniadou, S., Ed.; Routledge: New York, 2008; pp 3 34. (26) diSessa, A. A. In Inter national handbook of research on conceptual change ; Vosniadou, S., Ed.; Routledge: New York, 2008; pp 35 60. (27) Hammer, D. J. Learn. Sci. 1996 , 5 (2), 97 127. (28) Posner, G. J.; Strike, K. A.; Hewson, P. W.; Gertzog, W. A. Sci. Educ. 1982 , 66 (2), 2 11 227. (29) Linn, M. C. In The Cambridge Handbook of the Learning Sciences ; Sawyer, R. K., Ed.; Cambridge Handbooks in Psychology; Cambridge University Press, 2006; pp 243 264. (30) Corcoran, T.; Mosher, F. A.; Rogat, A. Learning progressions in scien ce: An evidence based approach to reform ; RR - 63; Consortium for Policy Research in Education: Teachers College - Columbia University, 2009. (31) Jin, H.; Anderson, C. W. J. Res. Sci. Teach. 2012 , 49 , 1149 1180. (32) Johnson, P.; Tymms, P. J. Res. Sci. Teach. 2011 , 48 , 849 877. (33) Neumann, K.; Viering, T.; Boone, W. J.; Fischer, H. E. J. Res. Sci. Teach. 2013 , 50 , 162 188. 90 (34) Schwarz, C. V.; Reiser, B. J.; Davis, E. A.; Kenyon, L.; Achér, A.; Fortus, D.; Shwartz, Y.; Hug, B.; Krajcik, J. S. J. Re s. Sci. Teach. 2009 , 46 , 632 654. (35) Johnstone, A. H. Sch. Sci. Rev. 1982 , 64 , 377 379. (36) Taber, K. S. In Multiple representations in chemistry education ; Treagust, D. F., Gilbert, J. K., Eds.; Springer: Dordrecht, 2009; pp 27 108. (37) Gilhooly , K. J. In The nature of reasoning ; Leighton, J., Sternberg, R., Eds.; Cambridge University Press: Cambridge UK, 2004; pp 49 77. (38) Roberts, M. J.; Gilmore, D. J.; Wood, D. J. Br. J. Psychol. 1997 , 88 (3), 473 492. (39) Maeyer, J.; Talanquer, V. Sci. Educ. 2010 , 94 , 963 984. (40) Evans, J. S. B. Trends Cogn. Sci. 2003 , 7 (10), 454 459. (41) Stanovich, K. E.; West, R. F. Behav. Brain Sci. 2000 , 23 (5), 645 665. (42) Gilovich, T.; Griffin, D.; Kahneman, D. Heuristics and biases: The psychology of intuitive judgement ; Cambridge University Press: Cambridge UK, 2002. (43) Creswell, J. W.; Plano Clark, V. L. Designing and conducting mixed methods research ; Sage Publications, Inc: Thousand Oaks, CA, 2007. (44) American Chemical Society Examinations Institute http://chemexams.chem.iastate.edu/ (accessed Jan 9, 2015). (45) Linenberger, K. J.; Bretz, S. L. J. Coll. Sci. Teach. 2012 , 42 , 45 49. (46) Corbin, J.; Strauss, A. Qual. Sociol. 1990 , 13 , 3 21. (47) Glaser, B. G.; Strauss, A. L. Awareness o f dying ; Aldine: Chicago, 1965. (48) Henderleiter, J.; Smart, R.; Anderson, J.; Elian, O. J. Chem. Educ. 2001 , 78 (8), 1126 1130. (49) Taber, K. S. Chemical misconceptions: Prevention, diagnosis and cure ; Royal Society of Chemistry: London, 2002; Vol. 1. (50) Johnstone, A. H.; Selepeng, D. Chem. Educ. Res. Pract. 2001 , 2 (1), 19 29. (51) diSessa, A. A. Cogn. Instr. 1993 , 10 (2 & 3), 105 225. (52) Hammer, D. Am. J. Phys. 2000 , 68 , S52 S59. 91 (53) diSessa, A. A.; Gillespie, N. M.; Esterly, J. B. Cog n. Sci. 2004 , 28 (6), 843 900. (54) Kahneman, D. Thinking, fast and slow ; Farrar, Straus and Giroux: New York, 2011. (55) Greenbowe, T. J.; Rudd, J. A.; Hand, B. M. J. Chem. Educ. 2007 , 84 (12). (56) Vázquez, A. V.; McLoughlin, K.; Sabbagh, M.; Runkl e, A. C.; Simon, J.; Coppola, B. P.; Pazicni, S. J. Chem. Educ. 2012 , 89 (8), 1025 1031. 92 CHAPTER V: DESIGN AND VALIDATION OF THE INTERMOLECULAR F ORC ES ASSESSMENT In the previous chapter, we outlined the various ways that students explain physical and chemical properties of molecular substances. Some students used hydrogen bonding to help explain trends in relative boiling and melting points (although often in a heuristic manner), but few included a discussion of dipole - dipole or LDFs. Students lack of discussion about IMFs led us to further explore their understanding of this topic, particularly because of its essential role in bridging the gap between molecular structure and the resulting physical and chemical properties. If students do not fully unde rstand what IMFs are and where they occur, then it should come as no surprise that these students do not use IMFs to help them reason about properties. 1 This chapter will outline the design, validat ion, administration, and coding of the Intermolecular through written descriptions and drawn representations. IMFs interviews: Protocol and findings Before we cou IMFs, we first conducted five short interviews to help develop and refine potential questions we could include in our IMFs assessment. While most students discussed IMFs to some degree during our structure - property interviews, few students went into detail about their understanding of the various types of IMFs. Since the main goal of these initial interviews was to investigate how students predict property information using a chemical st ructure and not necessarily how they viewed IMFs, we decided to conduct an additional five semi - structured interviews to 93 specifically probe students on their understanding of IMFs. These interviews were conducted with students enrolled in a second - semester organic chemistry course in the summer of 2012 and ranged from 10 - 20 minutes in length. We used a digital voice recorder as well as a Livescribe 2 These students received laboratory participation credit for the interviews and all students signed informed - consent forms. The interview protocol, shown in Figure 5.1, began by asking students to tell us what they understood about IMFs. We chose to keep this question intentionally broad so that studen ts could reveal their initial ideas about the topic before we inquired about specific IMFs and compounds. Additional follow - up questions were asked about the IMFs they may have mentioned so we could identify how students determined if a given compound was capable of - dipole interactions, and LDFs as these three IMFs are most commonly covered across general chemistry curricula. After a general discussion of the three m ain IMFs, we asked students to draw several different types of molecules and discuss any IMFs that may be present. We were specifically interested in how students depicted IMFs. With these interviews we did provide structural guidance to students upon requ est (with the exception of acetamide, most students did not require assistance) since previous research shows that students have difficulty constructing viable Lewis structures. 3 94 Figure 5.1: lar forces Our IMFs interviews, like our structure - property interviews, reinforced the benefit of asking students to provide representations of the topics they discussed. Four of the five students interviewed were able to recall all three types of IMFs an d provide a representation for at least two of the three IMFs discussed. Only one of those four students ( Rob ) required additional prompting about hydrogen bonding to invoke further discussion. We found that the other remaining student, Caitlyn , did not re call any IMFs, even when asked about specific types. Instead she described various reactions when asked about the possibility of interactions between molecules. This was consistent with our previous structure - property interview findings 1 where organic students were more likely to discuss additional (albeit not applicable) ideas from their organic course such as steric hindrance or reaction mechanisms. The interview protocol designed from intentional ly broad to asking about specific IMFs dipole - allowed us to better understand how th ey viewed IMFs (e.g. as an interaction between 1. What is your understanding of the term 2. What features and requirements do you look for to determine if a compound would exhibit a given intermolecular force? 3. Can you draw examples of the intermolecular force(s) that you mentioned, using a compound of your choice? 4. Can you describe your understanding of the term(s) hydrogen bonding, dipole - dipole, and/or LDFs? (Only if the student did not discuss these specific IMFs on their own earlier in the interview) 5. Please draw several water molecules. What IMFs would be present? a) If a student represented IMFs as occurring within a water molecule, we ask Question 5 was repeated for ammonia (NH 3 ), ethane (CH 3 CH 3 ), dimethyl ether (CH 3 OCH 3 ), and acetamide (CH 3 CONH 2 ). 95 molecules). For instance, Margaery described dipole - dipole as, partially negative and a partially positive charge, but the electron sharing will be a lot more While it was unclear what she meant by electron sharing in her response, her drawing, shown in Figure 5.2, confirmed that she was conflating the idea of dipole - dipole interactions with the bond dipole between two atoms (oxygen and nitrogen). A table containing eac h . Because the drawings proved beneficial during our interviews, we combined questions requiring students to draw with several short answer questions to design a preliminary IMFs assessment. Figure 5.2: - dipole between oxygen and nitrogen atoms Designing the initial version of the Intermolecular Forces Assessment (IMFA) of the IMFA. With this first administration of the IMFA at the beginning of the first - semester of general chemistry (GC1), we wanted to determine what information about IMFs that students brought with them from high school or previous courses about IMFs. L ike our interviews, we wanted students to both describe and construct representations of IMFs. In the first version of the IMFA, we provided students with the Lewis structures of four compounds (tetrafluoromethane (CF 4 ), acetic acid (CH 3 COOH), propane (CH 3 CH 2 CH 3 ), and trimethyl amine ((CH 3 ) 3 N)) since these compounds are capable of a variety of IMFs. For example, while propane only exhibits LDFs, acetic acid is capable of all three IMFs. We then asked students if two molecules of each compound would interact since we did not want to make the assumption that students understood that all molecules we provided interacted in some manner. We then asked them what type of 96 IMFs they would exhibit if they did interact to determine what IMFs they were aware of. Finally , we asked students to provide a molecular - level picture of the interaction they described using better determine their understanding about IMFs. A copy of the pr e - instruction assessment with a . This initial version of the IMFA was given to 66 students enrolled in first - semester general chemistry (GC1). It is important to note that some of these students did receive som e instruction regarding LDFs occurring between helium atoms prior to the administration of this pre - instruction assessment. They had not been told, however, that LDFs occur between molecules like the ones included in our assessment. Despite this, several s tudents did reference LDFs in their responses, as discussed below, but this information did not impact any representations of dipole - dipole or hydrogen bonding. The assessment was given on paper in the laboratory setting by teaching assistants and research ers to avoid instructor bias. Students were given participation credit towards their laboratory course for at least attempting to complete the assignment. We were aware that students may not have encountered IMFs before and therefore notified students that the collection of preliminary data signed informed - consent forms. Of the 66 students who completed the initial IMFA, approximately half (N=35, 53%) did not discuss any IMFs in t heir responses, shown below in Figure 5.3. Instead, common responses sections blank. The remaining students mentioned at least one IMF. The most commonly referenced IMF was hydrogen bonding, however only one student was able to provide an appropriate representation. The remaining students either represented hydrogen bonding as literal 97 bonds between hydrogen atoms of separate molecules, redrew the molecules provided i n an elaborate lattice structure, or offered no representation. LDFs (or Van der Waals forces) were the next most commonly listed IMF. All but one of the students had been exposed to the idea of LDFs during lecture prior to our assessment, so we were not s urprised to find discussions of LDFs. Three of the students who mentioned LDFs depicted them as occurring between molecules, however, again most students provided no representation or combined the molecules into a lattice - type structure. Only one student m entioned dipole - dipole, but they did not provide any representation. It is important to note that some students did mention more than one IMF; six listed both hydrogen bonding and LDFs, and one mentioned hydrogen bonding and dipole - dipole. Figure 5.3: F requencies of students who did and did not mention IMFs in their pre - instruction intermolecular forces assessment It is important to reiterate that this was an assessment given pre - instruction, and we did not expect students to show a strong understandin g of IMFs. Our main conclusion from this first iteration of the IMFA was that the majority of students either enter general chemistry remembering nothing about IMFs or only remembering specific terms like hydrogen bonding. For many of our students, complet ing this assessment was an overwhelming process. Asking students to elaborate on topics they do not remember, or were never taught, can be frustrating 0% 10% 20% 30% 40% 50% 60% 70% 80% No IMFs mentioned Listed at least one IMF Hydrogen bonding Dipole-dipole LDFs Listed IMFs? IMFs mentioned Responses to the pre - instruction intermolecular forces assessment GC1 students (N=66) 98 and cause students to doubt themselves. We found this glaringly apparent in some of the responses we rece placed on students who attempted to complete our pre - instruction assessment, we decid ed that it would be more beneficial to only collect post - instruction data on IMFs in the future. Redesigning the Intermolecular Forces Assessment (IMFA) We realized that the questions used in our pre - instruction assessment did not require students to di scuss their understanding of IMFs. In fact, we had only asked them to provide a list of IMFs that would be present for a series of compounds. We felt it was important to ask students to describe their understanding of IMFs, both generally and in terms of s pecific IMFs. Simply being able to identify or list the IMFs that would be present for a given compound might not necessarily reflect a solid understanding of IMFs. As a result, when revising the IMFA we continued to further draw upon our previous IMFs int erviews, specifically probing students understanding of each IMF and allowing them to describe compounds capable of interacting through those IMFs. With these questions as a guide, we redesigned the IMFA to first ask students about their general understan ding of IMFs and to then list all of the IMFs that they knew of. Like our earlier IMFs interviews, we then asked students to provide example compounds that would exhibit those IMFs. In case students did not discuss the three IMFs we were specifically inter ested in, in their previous responses, we asked about their understanding of hydrogen bonding, dipole - dipole, and LDFs. 99 Like both the interviews and the initial IMFA, we wanted students to construct representations of IMFs for a given compound. In our fir st IMFA attempt, we provided students with four different structures and asked them to construct representations of interactions. This proved to be rather time consuming and repetitive for students, thus we decided for our redesign to focus on a single com pound and ask students to draw out and clearly label each IMF. Water was initially discussed as a possible compound to use but ultimately we decided against it; most students are overly familiar with water (it is often the quintessential example used to ex plain hydrogen bonding). Instead, we chose to use ethanol because it is capable of all three IMFs and is less complex than structures we had used previously like acetic acid or trimethyl amine. We included the chemical formula (CH 3 CH 2 OH) to provide some gu idance for students drawing the ethanol structure. We did not want students to mistakenly draw dimethyl ether (a valid structure for C 2 H 6 O) or other non - ethanol structures. We did not provide students with the Lewis structure for ethanol in the IMFA becaus e we wanted students to have the freedom to use a representation of their choice (e.g. particulate, bond - line, etc.). An example of Item 7, asking students to construct representations of hydrogen bonding, is provided below in Figure 5.4. 100 Figure 5.4: An example of item 7 on the IMFA We learned from our pre - instruction IMFA administration that collecting student responses through a paper medium has its disadvantages, particularly when storing and reviewing student work. Therefore, we chose to complete the redesign of the IMFA and administer it using the online platform beSocratic . 4,5 beSocratic is an interactive program that allows for free - form student input through writing, drawing, and constructing graphs. It can also be used for more traditional methods of input, such as multiple - choice or fill in the blank questions. By using beSocratic a Additionally, beSocratic gave us the ability to prevent students from returning to previous questions in the IMFA and editing their responses as they progr essed through the assessment and were given additional pieces of information. 101 Pilot testing the redesigned IMFA Once we created our redesigned IMFA on beSocratic , we gave the assessment to three second - semester general chemistry (GC2) students in an int erview setting to finalize instruction wording and make sure that students interpreted each question as intended. We made minor wording changes to address feedback given by the students. We then pilot tested the redesigned IMFA with 68 GC2 students in the middle of the Spring 2012 semester. All of these students had been instructed, and tested, on IMFs so the information discussed in the IMFA should have been familiar to them. The redesigned IMFA pilot test was administered in the laboratory setting on beSo cratic provided stylus. By collecting responses outside of the lecture setting, we hoped to remove (or at least lessen) any instructor influences. The IMFA pilot test was administered by teaching data collection. Finalizing the IMFA design We did not encounter any major difficulties during the pilot - testing phase and thus bega n collecting from a larger population of students at the end of the Spring 2012 semester ( N =181). The IMFA was administered to students enrolled in either general chemistry or organic inistered the IMFA to students in the laboratory setting to remove instructor bias. Student responses were collected through beSocratic We made two additions to the IMFA after our Spring 2012 data collection. Some students exp ressed frustration in having to describe specific IMFs, like hydrogen bonding, if they had 102 already provided an explanation in a previous question. In response, we added an option to items 4 - ed a specific IMF. students in our Spring 2012 data collection, we chose to add a text box to items 7 - 9 to allow students to describe anything they were unable to r epresent in their drawings. Most students have not used this feature, but we have kept it in the IMFA for clarification during the coding process. The final version of the IMFA is included below in Figure 5.5. A timeline of the IMFA design and data collect ions is provided below in Figure 5.6, including collections from organic chemistry 1 (OC1) and 2 (OC2) students. Figure 5.5: Intermolecular Forces Assessment 6,7 2. List all types of intermolecular forces that you know of below a nd please define each. 3. Please give example(s) of a compound that would exhibit the intermolecular force(s) that you listed previously. Be sure to list the intermolecular force(s) that the compound is representing. 4 6. What is your current understandi ng of the terms hydrogen bonding, dipole - dipole interactions, and London dispersion forces? 7 9. Please draw and label a representation below that clearly indicates where the hydrogen bonding is present for three molecules of CH 3 CH 2 OH. In the box, please describe, in words, anything you were unable to adequately represent in your drawing. If you do not think this interaction is present, please write , Items 8 and 9 are similarly phrased and ask for representations of dipole - dipole and Londo n dispersion forces respectively . 103 Figure 5.6: IMFA data collections 104 IMFA coding and validation Student drawings (Items 7 - 9) At the end of our first large data collection in the Spring of 2012, we began to devise a coding scheme for the IMFA. The IMFA contains nine assessment items; in order to narrow down the amount and type of responses we would need to code, we chose to initially focus on - 9. These three items essenti ally form the heart of the through which we can view their understanding of IMFs. 8,9 Three researchers, m yself, a post - doctoral researcher, and a faculty member, used beSocratic and apply a code at a desired time point. For our p urposes, we were mainly interested in the final representation a student provided, rather than the process they used to achieve that drawings of IMFs were appl ied at the end of the replay rather than at a specific time point during the replay. Additionally, by using beSocratic to both collect and analyze student data, we were able to gather all responses from one administration in a single location. beSocratic a utomatically applies an anonymous identifier to each student to prevent bias on behalf of the researcher and allows for blind coding. It is only after coding is finished and exported that researchers can identify which students are enrolled in a particular course. We began our analysis of student drawings with hydrogen bonding (item 7) in beSocratic . From our interviews and IMFA pre - instruction data collection, we had some ion (i.e. that students might represent IMFs as within or between molecules). Because of this, parts of our 105 coding scheme were more deductive than inductive. That is, we used our prior work to inform the creation of our codes related to IMF location. 10 We did use an open - coding, inductive approach to create additional fine - grained codes unrelated to IMF location. 11 Our initial, fine - grained codes are shown below in Figure 5.7(a) and (b). It quickly became apparent, based on the codes created, that there were two initial key categories. Because of our previous research, approximately half of the codes created related specifically to the location of IMFs (as intended). The re maining codes developed from an open - coding approach mostly related to the structure used by students to represent ethanol. (a) (b) Figure 5.7: (IMFA item 7) Figure 5.8(a), shown below, outlines our initial reorganization of these codes into larger With this iteration of coding, we still believed it was important to determi ne where an IMF was specifically located, and thus we maintained the fine - grained location codes within the larger location categories. At this coding stage, we also reorganized our structural codes to change the 106 focus from structural features to the type of representation used, shown in Figure 5.8(b). We noticed students used a variety of representations and felt that this should be represented in our coding scheme. This involved the creation additional codes to encompass representations beyond Lewis struc tures such as particulate and condensed structures. We also created an additional code instructions in the IMFA explicitly asked students to include three molec ules of ethanol but we noticed students who drew only one (or none) for their representations of hydrogen bonding. By introducing this category, we were curious to see how many students drew one molecule or less (which would make it rather difficult to rep resent IMFs as between molecules). (a) (b) (c) Figure 5.8: (a) early reorganization of location codes into categories; (b) addition and organization of structure codes; (c) addition of the number of molecules codes After ap administration of the IMFA, we decided to refine and consolidate our coding to focus solely on 107 umber of between this category and the location category. For example, students who drew only one the coding for the type of representation used by students in their depictions of IMFs. While we did find some variation in how students represented ethanol, the type of representation used by students rarely affected how they depicted the location of the IMF. There was no correct representation that we expected students to use, and while most students favored Lewis structures, previous res constructing these structures. 3 We chose to focus primarily on the location code because we felt that the location category truly addressed the defining feature of IMFs: that they occur between small molecules. The distinction between bonds a nd IMFs is an important one and thus the location theme became subsume the fine - grained location codes (shown in Figure 5.8(a)) into the larger location codes, thus only coding for the five main location codes. Examples of student drawing s of hydrogen bonding that received these location codes can be found in Figure 5.9(a) (e). It should be noted may not necessarily be a correct representation. F or instance, a student may indicate that hydrogen bonding occurs between hydrogens on two ethanol molecules. 108 (a) (b) (c) (d) (e) Figure 5.9: Student representations of hydrogen bonding that received (a) between code, (b) with in code, (c) ambiguous code, (d) not present code, and (e) student DK code - d ipole - dipole representations, but an additional code was added for LDFs. When coding of exhibiting LDFs so the discussion of LDFs as occurring everywhere is somewhat understandable. Students awings of dipole - dipole and LDFs that received each location code can be found in Appendix F . In order to determine inter - rater reliability of the location coding scheme, another graduate student and I coded a random subset of 30 student drawings for hyd rogen bonding, 109 dipole - bonding and dipole - Student t ext (Items 2, 4 - 6) to their written responses in search of a discussion of location. Specifically, we wanted to explore how their discussion of location in their text responses compared to the locations we purposefully mirrored our location drawing scheme when developing codes for student writing. er). In order to receive a explicitly state the location of the IMF. By using similar coding schemes, we hoped to highlight similarities and differences between the two types of response modalities. We used b - 6, we exported their text and recombined them based on the IMF discussed. So, for example, a of hydrogen bonding from item 4 to create a complete description within a single response. We then imported these consolidated responses back into b eSocratic for text coding. Figure 5.10 110 shows an example of a consolidated student response as well as the coding scheme used for their writing. Figure 5.10: coding scheme To determine inter - responses, another graduate student and I coded a random subset of 40 student written responses 0.87 and 0.58. We identified discrepancies in how each coder was determining which LDFs text responses would responses if they believed LDFs act ually occurred within the molecule or were simply influenced by dipoles within the molecule. Unless a student explicitly stated that an IMF occurred within a 111 value o f 1 for all three codes after further negotiation. A faculty member and I coded a random - IMFA data collections To date, the IMFA has been administered at two univ ersities, Clemson University and Michigan State University, and a residential college within Michigan State University. We have collected responses from students enrolled in GC1, GC2, OC1, and OC2. We have also collected IMFA data longitudinally, following a small group of student from general chemistry through organic chemistry. All of our administrations of the IMFA are shown in Figure 5.6. Results from bolded collections in Figure 5.6 are discussed in greater detail in the next two chapters. The first ch while the next chapter will show the effect of a reformed general chemistry curriculum on onal course. It will 112 REFERENCES 113 REFERENCES (1) Cooper, M. M .; Corley, L. M.; Underwood, S. M. J. Res. Sci. Teach. 2013 , 50 , 699 721. (2) Linenberger, K. J.; Bretz, S. L. J. Coll. Sci. Teach. 2012 , 42 , 45 49. (3) Cooper, M. M.; Grove, N.; Underwood, S. M.; Klymkowsky, M. W. J. Chem. Educ. 2010 , 87 , 869 874. (4 ) Bryfczynski, S. P. BeSocratic: An intelligent tutoring system for the recognition, evaluation, and analysis of free - form student input. Doctoral Dissertation, Clemson University, 2012. (5) Cooper, M. M.; Underwood, S. M.; Bryfczynski, S. P.; Klymkowsk y, M., W. In Tools of Chemistry Education Research ; Cole, R., Bunce, D., Eds.; ACS Symposium Series; American Chemical Society, 2014; Vol. 1166, pp 219 239. (6) Cooper, M. M.; Williams, L. C.; Underwood, S. M. J. Chem. Educ. 2015 . (7) Cooper, M. M.; Wi lliams, L. C.; Underwood, S. M.; Klymkowsky, M. W. Submitted . (8) Ainsworth, S. E.; Prain, V.; Tytler, R. Science 2011 , 333 , 1096 1097. (9) Schwamborn, A.; Mayer, R. E.; Thillmann, H.; Leopold, C.; Leutner, D. J. Educ. Psychol. 2010 , 102 , 872 879. (10 ) Boyatzis, R. E. Transforming qualitative information: thematic analysis and code development ; SAGE Publications, Inc: Thousand Oaks, CA, 1998. (11) Corbin, J.; Strauss, A. Qual. Sociol. 1990 , 13 , 3 21. 114 CHAPTER VI: STUDENT UNDERSTANDIN G OF INTERMOLE CULAR FORCES: A MULTIMODAL STUDY Preface This chapter discusses findings from our research on how students write about and represent IMFs. This research has been previously published in the Journal of Chemical and is thus available for copying and redistribution or any adaptations for non - commercial purposes. Reprinted with permission from : Cooper, M. M.; Williams, L. C .; Underwood, S. M. Student understanding of inter molecular forces: A multimodal study. J. Chem . Educ. 201 5 , DOI : 10.1021/acs.jchemed.5b00169 . Copyright 2015 America n Chemical Society. on Theses and Dissertations is included in Appendix G. A copy of the American Chemical Appendix H. Words are useful for representing certain kinds of material perhaps representations that are more formal and require more effort to translate whereas pictures are more useful for pre senting other kinds of material perhaps more intuitive, more natural representations. In short, one picture is not necessarily equivalent to 1,000 w ords (or any number of words). - Richard Mayer 1 115 Introduction The importance of intermolecular forces The study reported here and property relationships. 2 6 Our goals for this research are to: (i) investigate the di fficulties that students have and (ii) develop assessment strategies for the steps involved in learning to use structures as models to predict and explain properties. Our ultimate goal is to use the data from these investigations to develop evidence - based approaches to teaching and learning that will improve understanding of this important construct. In this study, we focus on student understanding of intermolecular forces (IMFs), specifically hydrogen bonding, dipole - dipole interactions, and London dispers ion forces (LDFs). As we have previously noted, 3 the pathway that connects a molecular stru cture to the properties of a substance requires a long chain of inferences. Ideally, a student should be able to construct and then use a structure (by understanding that the shape and electron distribution in the molecule determine molecular polarity) to make deductions about interactions between molecules (intermolecular forces) that govern both physical and chemical properties. Each of these tasks is difficult in itself 3 and connecting them to make predictions about properties is highly demanding for students. In essence, we are asking students to move from using Lewis structures as representati ons to using them as models with which they can predict and explain properties. 7 If this shift in perspective is not made explicit to students, then even the simple task of constructing the representation may become difficult for many students because they may not see the purpose of drawing struct ures. property relationships is not conditionalize d and is therefore often inert; that is, their knowledge is not useful for this purpose. For example, we have shown that even organic students struggle to construct Lewis st ructures 1 and have proposed that, while a 116 rules - based approach to structure drawing provides a deceptively easy way to teach this skill, if students do not understand why they are learning to draw structures then the tenets for meaningful learning 8 ,9 are not met and students do not connect and reinforce skills that do not seem relevant to them. This is supported by our findings that, even after organic chemistry, many students do not understand how to use Lewis structures to predict anything other than surface - level features of a molecule. 2,3 Many students have not progressed from the idea of a structure as a representation to the idea of a str ucture as a model. In another study 4 , we interviewed students about how they used structural representations to predict properties. In this environment, where prompting and further elicitation of student ideas was possible, it became clear that, for many s tudents, structure and properties were not explicitly connected. Typically, students tended to rely on heuristics and surface - level features of molecules to make predictions rather than using the sequence of inferences that they had been taught. In this st udy 4 , students usually did not use IMFs as a construct to help them reason about properties such as relative boiling points, even when specifically asked probing questions designed to elicit such ideas. Although some students used terms such as hydrogen bo nding or London dispersion forces, few students used them in anything other than a rote fashion. Only two students invoked London dispersion forces to explain the difference in boiling point of methanol and ethanol, while most instead relied on rules such phase changes. 5 This finding supports the work of Talanquer and coworkers who have reported that heuristic reasoning is prevalent in a range of tasks such as identifying acids and bases 10 and predicting relative melting and boili ng points and solubility. 11 Since many students did not seem 117 to und erstand the need to incorporate IMFs into their reasoning about bulk properties of molecular species, we decided to investigate what students do understand about IMFs. Prior work on student understanding of I MFs The study described here is the first of a s eries of papers 12 understanding of intermolecular forces and how students write about and construct representations of IMFs; that is, the interactions between separate molecules that govern the properties of those molecules suc h as boiling point and acid - base reactivity. Prior research involving IMFs has focused specifically on hydrogen bonding 13 15 p erhaps because of its importance in the properties of water and in biological systems or more broadly on the general topic of IMFs. 16,17 Most of these studies have found that some students are (perhaps understandably) confused about the nature of the hydrogen bond. We should not be surprised when students have difficulties with the difference between covalent bonds and int ermolecular forces, especially when they are exacerbated by the fact that an intermolecular force is named of hydrogen bonds and found that, of the 22 organic st udents, four of them indicated that the hydrogen bond was the covalent bond between an O and H in the same molecule. 13 Similarly, Peterson and coworkers, using a two - tier m ultiple choice diagnos tic test, reported that 23% of g rade 12 high school students indicated that intermolecular forces were the forces within a molecule. 16 Using this same diagnostic test, other researchers found that 35% of students in Singapore also displayed this idea. 17 While there are other well - documented problems with student understanding of hydrogen bonds (for exa mple that any molecule containing hydrogen and oxygen can hydrogen bond 14 ), it is the notion that an intermolecular force such as hydrogen bonding is actually a covalent bond that is most problematic. 11 8 If students believe that IMFs are interactions within molecules, this idea must affect their models of phase change. For exampl e, if a student learns that water has a relatively high boiling point because the strong hydrogen bonds must be overcome, then we should not be too surprised 5,13,14,18 20 Barker and Millar found that while the students they studied from age 16 through 18 improved in their understanding of hydrogen bonding over time, none of the students in the study invoked any other IMFs to explain trends in boiling point. 18 As discussed earlier, our own work has supported this finding and extends it to students at the college level in general and organic chemistry. 5 Schmidt and coworkers reported that upper secondary school students in Germany had great difficulty predicting boiling points of organic compounds and very few (15%) used appropriate reasoning when asked to explain their answer to a multiple - choice question. 14 This paper describes an investigation into the external representations that students use to communicate about IMFs. Rather than hoping to prompt dis cussion of IMFs in a context with a phenomenon such as predicting relative boiling points (as we had done previously and which we now know is unlikely to happen 5 ), we wanted to ask students specific ally about their understanding of IMFs. As noted, earlier studies with IMFs have most often used forced - choice instruments. 14 17 important to use the best evidence available to draw conclusions. 21 For us, this means having students construct their own responses rather than restrict ing them to choosing an answer from a list. While multiple - choice items and diagnostic two - tier instruments are fast and reliable, it has been shown that students can answer these questions without recourse to appropriate scientific thinking. 22 For example, previous research has shown that students were able to rank the boiling 119 points of a r ange of compounds without thinking about intermolecular forces 4,18 and instead used heuristics that may have led them to the correct answer but were scientifically flawed. Theoretical p erspective The importance of multimodal learning, that is providing bot h visual (pictures) and verbal (words) support for student learning, has long been emphasized. It has been proposed that instructional materials providing both words and pictorial representations are more effective because student understanding can be enha nced by the addition of non - verbal knowledge representations. 23 However, there is less research on how students use multimodal (pictorial and verbal) representations to explain and represent their understanding. It has been proposed that e a number of studies 24 29 that have investig ated the effectiveness of drawing in support of learning in science. Drawing should be particularly helpful in identifying student ideas about spatial information; for example, understanding how they view the relative positions of molecules and the forces that act between them. Similarly, having students write about their understanding can also provide useful insights into student thinking. Certainly both modalities require students to construct answers and thus make their ideas explicit. There are several studies that compare two groups of student responses: those who draw and those who write. For example, Gobert and Clement compared responses for student who drew diagrams or produced text summaries about plate tectonics. 30 Similarly, Akayagun and Jones looked at the ways that students wrote about or constructed representations of equilibrium systems. 31 Both studi es found that the representations students constructed tended to emphasize different features than the written explanations. However, in 120 neither study were students asked to both write and draw so it was not possible to compare a particular student s textu al explanations and drawings. Our goals with this study were to investigate how both writing and drawing about IMFs can provide us with insight into how students understand this concept. Therefore, students were asked to both construct a representation of an IMF (i.e. draw three structures of ethanol to show how they interact) and discuss their understanding of that particular IMF in words. That is, we asked students to use more than one modality to answer questions about IMFs in hopes that it would provide us with a more nuanced picture of their understanding than either writing or drawing alone. The study was designed to address three research questions: RQ1: How do students represent IMFs in free - form drawings? RQ2: How do students discuss and describe IM Fs in open - ended written responses? RQ3: Methods Student p opulation The participants in this study consisted of a subsample of students from a larger population of 1600 students enrolled in general chemistry at a mid - sized public southeastern research university (Cohort 1, Fall 2011 Spring 2012, N = 94). An additional cohort from the following year was included in this study for replication purposes (Cohort 2, Fall 2012 Spring 2 013, N = 160). The freshman population at this university is approximately 48% female and 52% male with the majority of students, 84%, identifying as white. The average ACT score for incoming freshman ranges from 26 to 31 and the mean SAT score is 1246. Wh ile the general chemistry course at this university was traditional in content (i.e., taught using a commercially available text), the course had been revised to include a more conceptual approach and some sections of the courses 121 employed reformed pedagogi es such as the use of clickers and in - class group quizzes. Students also completed online homework assignments using a commercially available homework system (Mastering Chemistry 32 ). The common examinations for these courses were exclusively multiple - choice and, at the end of a full academic year, the American Ch emical Society (ACS) nationally normed general chemistry examination 33 was administered as the final exam. Students in this course typically score around the 75th percentile on the ACS general chemistry exam. All the students included in this study consente d to participate in this research by signing informed consent forms. Demographic information for each cohort is provided in Appendix I . Development of the Intermolecular Forces Assessment (IMFA) The Intermolecular Forces Assessment (IMFA) * was designed to understanding of IMFs by asking them a range of questions that probe the way students think about IMFs. It was developed based on responses from interviews with general chemistry and organic chemistry students where students discussed how they used structural representations to help them understand phase changes. 5 The representations that students constructed along with their verbal descriptions provided us with insight into how the s tudents were thinking about these processes. Interim versions of the IMFA were piloted in student interviews and revised for clarity where necessary. The final version ( Figure 6. 1) was administered to 68 students in a pilot study and was then used in the s tudies reported here. * The development and design of the IMFA is further addressed in Chapter 5. 122 Figure 6.1: Items included on the Intermolecular Forces Assessment The IMFA is designed to explore how students think about and represent IMFs. Items 1 3 ask students for general examples and explanation s of IMFs without any specific prompts. In this particular IMF. For example, students are asked to explain what they understand by the general term intermolecular fo rces, which IMFs they know about, and to provide an example of a substance that would exhibit those IMFs. In items 4 9, students are asked specifically about hydrogen bonding, dipole - dipole and London dispersion forces, both by explaining what they underst and by these terms (items 4 6) as well as constructing drawings or representations (items 7 9) that would show the presence of specific IMFs (if present). Intermo lecular Forces Assessment 1. 2. List all types of intermolecular forces that you know of below and please define each. 3. Please give example(s) of a compound that would exhibit the in termolecular force(s) that you listed previously. Be sure to list the intermolecular force(s) that the compound is representing. 4 6. What is your current understanding of the terms hydrogen bonding, dipole - dipole interactions, and London dispersion forces ? 7 9. Please draw and label a representation below that clearly indicates where the hydrogen bonding is present for three molecules of CH 3 CH 2 OH. In the box, please describe, in words, anything you were unable to adequately represent in your drawing. If yo Items 8 and 9 are similarly phrased and ask for representations of dipole dipole and London dispersion forces, respectively. An example showing the question layout is included in Suppo rting Information. 123 Note that students were explicitly asked to draw three molecules and the term three was bolded, since in early iterations of the IMFA many students drew only one molecule. Ethanol was selected as the target for these items because it is a relatively simple molecule that is capable of exhibiting hydrogen bonding, dipole - dipole, and London dispersion forces (LDFs). Students were asked to draw structures of ethanol, but were given structural cues (CH 3 CH 2 OH) so that most students in this study were able to construct a reasonable (recognizable) representation. The IMFA was administered to both cohorts of studen ts at the end of their second - semester of general chemistry (GC2) to ensure that all students had been exposed to, and tested on, the relevant material . The IMFA was administered outside of lecture in the laboratory setting (which students take concurrentl y with lecture). Students received participation points for at least attempting to complete the IMFA. None of the instructors for the course were involved in data collection or analysis process. Research and teaching assistants collected all student respon ses on beSocratic , which allows collection of both free - form student drawing s and text inputs. 34,35 That is, we asked students to both draw representations of IMFs and explain the IMFs in words. Using this system, we prevented students from returning to any prior items once they moved forward so that students were not able to alter their answers as they progressed through the assessment. In this study, we focus on the student responses to items 2 and 4 9. Drawings (items 7 9) from both Cohorts 1 and 2 were analyzed, and written responses (items 2, 4, 5, and 6) were analyzed for Cohort 1. Data a na lysis: Drawings (Items 6 9) - analysis tools in beSocratic . 34,35 The rawing input step - by - step, allowing the researcher to replay a beSocratic was used to code and store 124 important actions or features of the drawing. An open - coding, constant comparison approach was us 36,37 Three researchers (a graduate student, a post - doctoral researcher, and a faculty member) analyzed and discussed the set of codes created from the open - coding process and agreed that there were only a few, distinct ways that students represented IMFs. The major code categories that emerged for drawings of each type of IMF were : within, between, ambiguous, and not present. If a student clearly indicated that the IMF occurred within a molecule (i.e. circling or pointing to a particular a student made an indication that the IMF was located between two molecules, typically by marking the space between ethanol molecules (see Table 6. 1 for examples using drawings of hydrogen bonding). 125 Table 6.1 : Coding examples for student drawings demo nstrating understanding of selected types of intermolecular forces IMF Type Code for IMFA Response Drawings Characterizing IMF Locations Within the Molecule Between Molecules Ambiguous Hydrogen Bonding Dipole Dipole Interaction LDFs If the location of an IMF was not clearly specified (i.e. within or between), the response was a bond within a molecule as students, rather than providing a representa tion, described (in words) that LDFs are something that all substances are capable of or is always present for compounds. Some students indicated in - dipole and LDFs are provided in Table 6. 1. 126 representation of the IMF w as completely correct. For example, a student might indicate that the hydrogen bonded to carbon in the ethanol molecule would hydrogen bond with the oxygen of depiction, even though their representation of hydrogen bonding is incorrect. We did analyze the drawings for correctness of dipole dipole and LDFs was more challenging because students may represent charge distribution or fluctuating dipoles in many different ways, or not include indications of the role of charges at all. Even variations in structural representations, such as Lewis structures, condensed structures, or pa rticulate representations can blur the lines between To determine the inter - rater reliability of the analyses, one of the authors and another Kappa of 1.0 for hydrogen bonding and dipole - LDFs. Data a nalysis: Text r esponses (Items 2, 4 6) Student te xt responses collected from items 2, 4, 5, and 6 were analyzed using the coding (where students were asked in general to identify types of IMFs) were combined with th e specific items 4 6, since many students wrote more detailed responses in item 2 and simply referred to their prior responses in items 4 6. For this study, each text response was coded specifically for a discussion of the location of the IMFs. That is, th e text was analyzed to see 127 text responses corresponded to their drawn representations. Examples of text responses and the corresponding codes are shown in Table 6. 2. Table 6.2: sponses for each IMF IMF Pseudonym Quote Text c ode Hydrogen Bonding Tracey a large difference in electronegativity (i.e., hydrogen and oxygen) is attracted to a negative portion of Betwee n Lindsay hydrogen atom must be bonded to another hydrogen, nitrogen, oxygen or floride [ sic Ambiguous Dipole Dipole Adelaide between two polar mol Between Laura molecule are attracted to each other and hold the Within Marta sic ] polar, Its the Ambigu ous LDFs Ann Between Rebel Ambiguous Casey Within To determ ine the inter - rater reliability of the analyses, one of the authors and a graduate student coded a random sample of 40 student text responses for both hydrogen bonding and ers further 128 within by one coder and ambiguous by the other. After discussion, it was determined that, while the student did mention dipoles within the molecule, it was not apparent where the student considered the weak force to be located. If a response did not explicitly state that LDFs w clarify any discrepancies. Two of the authors coded a random sample of 40 dipole dipole re Results and d iscussion RQ1: How do students represent IMFs in free - form d rawings? dipole, and LDFs (items 7 9) are shown in Figure 6.2 . Of the 94 GC2 students who completed the IMFA, only 15% ( N = 14) of students correctly indicated that hydrogen bonding occurs between separate molecules. Of these students, only nine were completely correct in showing the hydrogen bonding interaction between an H (bonded to an O) in one molecule and an O in another molecule. Of the remaining students, 72% ( N = 68) clearly represented hydrogen bonding as an O H bond within a single molecule of ethanol. In fact, 54% ( N = 51) of students drew only one molecule or no ne at all (even though they were explicitly asked to draw three molecules). All but three of the drawings coded as within clearly depicted the IMF as the covalent O H bond to the covalent oxygen hydrogen bond within the molecule has been previously documented, the 129 extent of this error has not. As discussed earlier, previous studies have shown a much lower prevalence of this idea ranging from 23 35% of students. 13,16,17 Figure 6.2: Code frequencies for studen representations of b oth dipole dipole and LDFs as well. Again only a small number of students drew dipole dipole (11%, N = 10) and LDFs (12%, N = 11) as interactions between molecules. As we saw with hydrogen bonding, the majority of students (61%, N = 57) in Cohort 1 drew di pole dipole interactions as occurring within the ethanol molecule and as did 55% ( N = 52) for dipole and LDFs were somewhat more varied than those for hydrogen bonding. For example, in Table 6. 1 (within, dipole dipole) all of the C H bonds are depicted as dipole dipole interactions. Representations of LDFs include circling individual atoms (such as hydrogen in Table 6. 1 (within, LDFs) ) , lone pairs of electrons on oxygen atoms, or bonds. In fact, of the students in Cohort 1 who drew LDFs occurring within the molecule, 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Within Between Comparison of IMFs drawing code frequencies for Cohort 1 and Cohort 2 Cohort 1 (N=94) Cohort 2 (N=160) 130 seven students circled every bond in the ethanol molecule, which may be related to the idea that The IMFA was also administered to a second cohort of general chemistry students (Co hort 2) in the following year, and the results are also shown in Figure 6.2 . While there was a slight increase in the number of Cohort 2 students who drew representations of hydrogen bonding and LDFs occurring between molecules, again the majority of these hydrogen bonding (56%, N = 90), for dipole dipole (58%, N = 93), and for LDFs (56%, N = 89) clearly indicated that the IMFs were located within a single molecule. We attribute the slight improvement to the fact that the instr uctors in the courses were now aware of our results for Cohort 1 and had emphasized IMFs more than usual in the following year. RQ2: How do students discuss and describe IMFs in open - ended written r esponses? Written responses about hydrogen bonding, dipol e dipole, and LDFs (items 2, 4 6) were analyzed for the students in Cohort 1 ( N = 94), using a coding scheme similar to that for their drawings. Unlike the drawings where the location of IMFs, as either within a molecule or between molecules, was usually q explicit. Of the 94 students in Cohort 1, only 4% ( N = 4) of students explicitly stated that IMFs occur between molecules for all three types of IMFs. Similarly only a few students stated explicitl y that IMFs occur within a single molecule. Specifically, 5% ( N = 5) of students stated that dipole dipole occurred within a single molecule and only one student claimed that LDFs did the same. None of the students in Cohort 1 explicitly stated that hydrog en bonding occurs within a molecule. Indeed most students failed to make any reference to the location of IMFs at all, meaning that most responses received the ambiguous text code, as shown in Figure 6.3 . It should 131 be noted that an ambiguous code did not m that it was missing a discussion of the location of the IMF. Figure 6.3: n in Table 6. 2, is typical in that the elements involved and the strength of the IMF are discussed. Similarly, many students (for example, Rebel) provided ambiguous responses for LDFs. Not represented in the Figure 6.3 are the 26% ( N = 24) of students who explicitly stated in their written responses that LDFs are present for all molecules or always present. We suspect that this response may stem from students hearing their instructors talk about LDFs in a similar fashion. While it is true that all molecules are capable of interacting via LDFs, it is easy to imagine that students may understand this as a property of the molecule rather than of the interactions between molecules. Interestingly, the written responses about dipole dipole interactions did not fol low the same pattern as those for hydrogen bonding and LDFs. A much larger group of Cohort 1 students (49%, N = 46) stated explicitly that dipole dipole interactions occurred between molecules, shown in Figure 6.3 . As discussed below, this is in contrast t dipole dipole interactions where a majority of students drew structures showing dipole dipole 0% 20% 40% 60% 80% 100% Between molecules Within molecules Ambiguous Code frequencies for IMFs text responses Hydrogen bonding Dipole-dipole LDFs 132 interactions within a particular molecule (Figure 6.2 ). While we do not know the reason why more students wrote about dipole di pole interactions between molecules, like the responses for H - bonding and LDFs, the responses were somewhat superficial. It may be that the students had learned a definition of dipole dipole that specifically included the idea that the interaction was betw een molecules. One factor that made student written responses difficult to interpret was that students often appeared to use words without understanding their meaning. In addition to the term vice versa) and bond to mean IMF (or vice versa). For example, Rueben, as shown in Table 6. 3, refers to a molecule of fluorine when discussing hydrogen bonding but he probably meant to describe an atom. This interchanging of atom and molecule has been in r eported previously in work by Cokelez and Dumon. 38 The confusion between atom and molecule is fairly obvious here, but this mistake may not always be so clear. A student who confuses these terms might discuss an interaction between atoms but mean an interaction between molecules an important distinction. Additionally, some students used the terms bond and interaction interchangeably. While a bond is a type of interaction, we usually do not refer t o IMFs as bonds (except, of course, for hydrogen bonds!). Thomas, for example, discusses a bond between molecules but it is unclear whether he understands the difference between the terms bond and interaction and whether he is using the appropriate one in this scenario. Since student use of terminology can be imprecise, it can make it difficult to know what students mean from their writing alone, which often resulted in the where terminology becomes problematic are shown in Table 6. 3. 133 Table 6.3: Examples of terminology issues in student responses Terminology i ssue IMF Pseudonym Quote Text c ode Drawing c ode Atom vs. Molecule Hydrogen Bonding Georgia - betw een a hydrogen atom on one molecule and either an oxygen, nitrogen, or fluorine of Ambiguous Between Rueben molecule of fluorine [ sic ], oxygen, or nitrogen bonded with Ambiguous Within Regina ds - an intermolecular force between a hydrogen atom and either an O Ambiguous Within Bond vs. Interaction Dipole Dipole Thomas Dipole - pretty weak bond between polar Between Within Ray - Dipole - bond between two molecules that is Between Ambiguous LDFs Betty weak bond that binds any two molecules, or parts of any two Between Ambiguous ompare to their drawn r eprese ntations? Drawing and writing provide different approaches to eliciting student understanding and our use of similar codes for both modalities allowed us to compare the text and drawn responses. Most students constructed a drawing of an IMF showing its loc ation explicitly within a single molecule, coupled with an ambiguous text description (for hydrogen bonding: 70%, N = 66; and 134 LDFs: 48%, N answer for the location of hydrogen b onding by explicitly stating that hydrogen bonding occurred between molecules and drawing an interaction occurring between molecules. Comparisons of the major categories for drawing and written explanations are shown in Figure 6.4 . All combinations of text and drawing responses from students are provided in Appendix J. Figure 6.4: awings of IMFs As discussed earlier most students were not specific about the location of IMFs in their written explanations; in fact, 93% ( N received an ambiguous code (Figure 6.3 ). It is only whe n we look at the drawings of hydrogen bonding that we can see what students are trying to explain. For example, Tobias and Maeby have similar explanations of hydrogen bonding, as shown in Table 6. 4. It would be difficult to 0% 10% 20% 30% 40% 50% 60% 70% 80% Between text Ambiguous text Within text Between text Ambiguous text Within text Between text Ambiguous text Between drawing Within drawing Ambiguous drawing All other responses Comparison of students' text and drawing responses Hydrogen Bonding Dipole-dipole LDFs 135 distinguish between their respon ses (knowing as we do that the term intermolecular is often misunderstood by students), without looking at their drawings, which show that Tobias understands hydrogen bonding as interactions between molecules, while Maeby does not. Table 6.4: Drawing and t ext comparisons for Tobias and Maeby Pseudonym Drawing Quote Tobias Between Molecules Hydrogen bonding intermolecular force between hydrogen and N O F. It is the Maeby Within Molecules Hydrogen bonding een hydrogen and oxygen, nitrogen, Similarly for LDFs, the drawings provided more information than the writing. Consider, for example, Oscar and Rita in Table 6. Oscar has an understanding of LDFs as interactions between separate molecules, while R ita does not. 136 Table 6.5: Drawing and text comparisons for Oscar and Rita Pseudonym Drawing Quote Oscar Between Molecules LDFs Rita Within Molecules LDFs but the one force in po lar molecules. The weakest of the Just as with hydrogen bonding and LDFs, few students (9%, N = 8) correctly described dipole dipole as an interaction between molecules and provided an appropriate representation (Figure 6.4 ). Unlike hydrogen bondi ng and LDFs, however, 49% ( N = 46) of students in Cohort 1 provided text responses for dipole dipole that described the interaction as taking place between molecules (Figure 6.3 ). Despite this, 28% ( N = 26) of Cohort 1 drew an explicit representation of di pole dipole interactions as occurring within a molecule of ethanol while at the same time describing the interaction as occurring between molecules in their written response (Figure 6.4 ). For instance, Gene described dipole ce formed between two Table 6. 6, clearly shows dipole dipole interactions as C H bonds in one molecule of ethanol. Perhaps more important to note is that, while 137 students are ab le to show us where they believe these interactions occur 24 and, as a result, show aspects of their understanding that are not captured in their written descriptions. Table 6.6: Drawing and text comparisons for Trisha and Gene Pseudonym Drawing Quote Trisha Between Molecules Dipole dipole molecules that have positive and negatively charged ends. The different ends are attracted to each Gene Within Molecules Dipole - dipole force formed between two dipole molecules that comes from Summary The three mai n findings of this st udy follow: First, drawings of IMFs collected in this study indicate that a majority of students draw representations showing that IMFs are located within a single molecule rather than between separate molecules (Figure 6.2 ). Although this finding is simil ar to previous studies in which students confuse IMFs and covalent bonds within a molecule 13,16,17 , the results presented here contrast with prior studies where a much smaller percentage of the students (certainly less than a majority) exhibited this misunderstanding. There are several possib le explanations for this finding; perhaps the most obvious is that these students have not been taught appropriately. However, these students are representative of a cohort who averaged around the 75th percentile 138 on the ACS general chemistry examination. W e also have some evidence 12 that while the student response is highly dependent on the learning environment, the finding that many students depict intermolecular forces as interactions within a molecule is not unusual for a traditionally sequenced general chemistry course. Most of the previous reports on student understanding of IMFs rely on forced - choice assessments, in which some of the responses may not even address the particular problem of inter - versus intra - molecular forces. For example, in Schmidt a understanding of IMFs, 14 several items w the idea that hydrogen bonding occurs within a molecule. Villafañe and colleagues developed a multiple - choic e instrument using clusters of questions designed to probe topics important to the 15 Thei r assessment included three items designed to address how students understand hydrogen bonding. It is significant that this cluster e three items. Only 12% of the students in their study gave completely correct responses to all three items in the hydrogen bonding cluster. 21 (Figure 6.3) . While most s provided in the prompt, many were paraphrases of textbook definitions: for example, listing the elements involved in hydrogen bonding, or indicating that all molecules are capable of London dispersion forces. Very few students specifically indicated that intermolecular forces occur between molecules or discussed the origin of intermolecular forces as electrostatic interactions. While this may have been a consequence of the lack of specificit y of the prompt, in this study it 139 was not possible to determine from student writing whether students have an appropriate conception of IMFs. Third, comparison of student writing and drawing indicates that drawings are much easier to categorize. For the mo drawings were not (Figure 6.4 ). In fact, we provided some examples within this paper that show 6. 6. 6. We believe that student - constructed representations can provide more insight into student understanding, particularly with respect to spatial information such as t he location of IMFs. Conclusions It is clear from inspection of student drawings that many students have problematic ideas about intermolecular forces. For each IMF, more than half of the students in both cohorts drew representations that explicitly showed an interaction within a molecule, yet student written descriptions were often much more ambiguous or, in the case of dipole dipole, contradictory. The fact that the majority of students drew pictures indicating each IMF as interactions within a molecule l eads us to believe that, like many other concepts, student understanding of IMFs is highly problematic, fractured, and unstable. 5,39 41 Depending on the nature of the prompt, w e may elicit differing and often contradictory ideas. Because of this, we must be particularly careful not to draw conclusions from single assessment items. Clearly it is important to provide students with opportunities to construct responses in multiple f ormats and to help them reconcile differences between their responses. 140 Implications for teaching and future w ork The fact that a majority of students can emerge from a general chemistry course without a consistent understanding that intermolecular forces o perate between molecules is highly problematic. Intermolecular forces mediate much of chemistry, from the temperature at which covalent bonds break when a phase cha nge occurs 5,13,14,18 20 becomes more understandable in light of this finding. As we have previously noted, determining which IMFs are present within the bulk substance are part of a long sequence of ideas and skills that students must construct for themselves before they can understand structure property relationships. Clearly one a pproach to helping students might be to emphasize the teaching of IMFs more in general chemistry courses. Inspection of a number of popular texts, including the one used by students in this study ( General Chemistry: Atoms First by McMurray and Fay 42 ) shows that the topic of IMFs is clearly explicate d with well - designed and clear representations. However, knowledge is not transferred intact (either from a text or in a lecture), but is instead constructed by the student. 43,44 Clear exposition and repetition of important ideas are not sufficient to produce a robust and useful un derstanding. In fact, in many general chemistry courses the topic of IMFs is often separated both from the prior knowledge that is needed to understand it (i.e., molecular structure, shape, and polarity) and from the material for which IMFs are needed to u nderstand a concept (e.g., solubility and phase changes). That is, the teaching of IMFs does not meet the tenets for meaningful learning in that prior knowledge, instruction, and the purpose of that knowledge must be explicitly connected. 8,43 Knowledge must be contextualized before it becomes useful. 45 While this study looked at IMFs using small molecules as our substrate, we must also bear in mind that 141 non - covalent interactions mediat e much of biological chemistry from protein folding to enzyme substrate interactions. It is our contention that, to develop a robust understanding, the curriculum must be restructured to emphasize the connections between important ideas and that students m ust be given opportunities to reflect on and make their thinking visible. 46 That is, students must have the opportunity to construct and revise representations, models, and explanati ons that allow them to predict and explain phenomena. Otherwise, it becomes too easy to assume that students have learned important concepts because they can choose the correct answer on an examination. Indeed one might wonder why the extent of this proble m has gone unrecognized for so long. It may well be that our increasing reliance on homework using online course management systems and multiple - choice tests has contributed to the problem. If students are not ever asked to write and draw, to reflect, to e xplain, and to revise their ideas, but instead are only assessed by which item they choose on a test or randomly generated homework, it is unlikely that they will develop a robust and coherent understanding of core concepts. This is not to say that multipl e - choice items are never useful (indeed they are almost unavoidable in large enrollment courses), but that students must also be given many opportunities to construct answers for themselves as they learn. One further note, some authors have recommended tha t intermolecular forces such as hydrogen bonding or London dispersion forces be considered as bonds 47 and there is a compelling argument that bonds and intermolecular forces be considered part of a continuum of interactions between atoms. However, it is crucial that students also under stand the differences. That is, when an IMF is overcome, the result is typically not a new chemical substance but rather a phase change (or in the case of large molecules a change in conformation or shape). When 142 bonds are broken, new chemical substances ar e produced with different properties and arrangements of atoms. These differences, while quite apparent to the expert, are clearly not so obvious to students. It is our contention that students who do not have a firm grasp on the forces that act both withi n and between molecules will be unable to make sense of phase changes, solution formations, and chemical reactions. While experts may point out that the word study and in other reports, that students have difficulty using terminology appropriately. 38,48 Our future work on helping students understand and use IMFs focuses on two areas. The first is investiga ting how changes in learning environment affect the ways that students represent and understand intermolecular forces. In a follow - up paper in this series, 12 we present a comparison between matched cohorts of students from traditional and transformed cours es. We also are collecting data from a wider range of institutions and instructional settings. The second area of research is to develop more nuanced approaches to eliciting the ways in which students think about IMFs and their role in bulk properties of m atter, including designing scaffolded prompts to elicit student beliefs about how IMFs are formed and how they are linked to properties. We believe that a major goal of chemistry education is to help students develop causal, mechanistic explanations of phe nomena, and understanding IMFs is crucial to this goal. Limitat ions of the s tudy The IMFA was designed to require students to first write about their understanding of IMFs and then provide constructed representations of those IMFs. We did not ask students to justify or intentions from their drawings alone. 143 The study reported here was performed at one institution and it might be argued that the institutional se tting was such that the results are not applicable to other institutions. However, in future papers we will be reporting similar studies from multiple institutions and with multiple types of courses, and we have reason to believe that the data presented he 144 REFERENCES 145 REFERENCES (1) Mayer, R. E. Multimedia Learning New York, 2001. (2) Cooper, M. M.; Grove, N.; Underwood, S. M.; Klymkowsky, M. W. J. Chem. Educ. 2010 , 87 , 869 874. (3) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Chem. Educ. Res. Pract. 2012 , 1 3 , 195 200. (4) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. J. Chem. Educ. 2012 , 89 , 1351 1357. (5) Cooper, M. M.; Corley, L. M.; Underwood, S. M. J. Res. Sci. Teach. 2013 , 50 , 699 721. (6) Underwood, S. M.; Reyes - Gastelum, D.; Cooper, M. M. Sci. Educ. in press . DOI: 10.1002/sce.21183. (7) National Research Council. A framework for K - 12 science education: Practices, crosscutting concepts, and core ideas ; National Academies Press: Washington, DC, 2012. (8) Novak, J. D. Learnin g, Creating, and Using Knowledge: Concept Maps As Facilitative Tools in Schools and Corporations ; Lawrence Erlbaum Associates: Mahwah, NJ, 1998. (9) Bretz, S. L. J. Chem. Educ. 2001 , 78 , 1107 1117. (10) McClary, L.; Talanquer, V. Int. J. Sci. Educ. 201 1 , 33 (10), 1433 1454. (11) Maeyer, J.; Talanquer, V. Sci. Educ. 2010 , 94 , 963 984. (12) Cooper, M. M.; Williams, L. C.; Underwood, S. M.; Klymkowsky, M. W. Submitted . (13) Henderleiter, J.; Smart, R.; Anderson, J.; Elian, O. J. Chem. Educ. 2001 , 78 (8), 1126 1130. (14) Schmidt, H. - J.; Kaufmann, B.; Treagust, D. F. Chem. Educ. Res. Pract. 2009 , 10 , 265 272. (15) Villafañe, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Biochem. Mol. Biol. Educ. 2011 , 39 , 102 109. (16) Peterso n, R. F.; Treagust, D. F.; Garnett, P. J. Res. Sci. Teach. 1989 , 26 , 301 314. (17) Goh, N. K.; Khoo, L. E.; Chia, L. S. Aust. Sci. Teach. J. 1993 , 39 (3), 65 68. 146 (18) Barker, V.; Millar, R. Int. J. Sci. Educ. 2000 , 22 , 1171 1200. (19) Bodner, G. M. J . Chem. Educ. 1991 , 68 (5), 385 388. (20) Osborne, R. J.; Cosgrove, M. M. J. Res. Sci. Teach. 1983 , 20 (9), 825 838. (21) Mislevy, R. J.; Almond, R. G.; Lukas, J. F. A brief introduction to evidence - centered design ; The National Center for Research on Evaluations, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA: Los Angeles, CA, 2003. (22) Lee, H. - S.; Liu, O. L.; Linn, M. C. Appl. Meas. Educ. 2011 , 24 (2), 115 136. (23) Moreno, R.; Mayer, R. Educ. Psychol. Rev. 2007 , 19 (3 ), 309 326. (24) Ainsworth, S. E.; Prain, V.; Tytler, R. Science 2011 , 333 , 1096 1097. (25) Leenaars, F. A. J.; van Joolingen, W. R.; Bollen, L. Br. J. Educ. Technol. 2013 , 44 , 82 94. (26) Schwamborn, A.; Mayer, R. E.; Thillmann, H.; Leopold, C.; Leu tner, D. J. Educ. Psychol. 2010 , 102 , 872 879. (27) Van Meter, P.; Aleksic, M.; Schwartz, A.; Garner, J. Contemp. Educ. Psychol. 2006 , 31 , 142 166. (28) Zhang, Z. H.; Linn, M. C. J. Res. Sci. Teach. 2011 , 48 (10), 1177 1198. (29) Stieff, M. Sci. Educ . 2011 , 95 , 310 336. (30) Gobert, J. D.; Clement, J. J. J. Res. Sci. Teach. 1999 , 36 , 39 53. (31) Akaygun, S.; Jones, L. L. Int. J. Sci. Educ. 2014 , 36 (5), 783 807. (32) MasteringChemistry http://masteringchemistry.com/site/index.html (accessed Sep 1, 2013). (33) American Chemical Society Examinations Institute http://chemexams.chem.iastate.edu/ (accessed Jan 9, 2015). (34) Bryfczynski, S. P. BeSocratic: An intelligent tutoring system for the recognition, evaluation, and analysis of free - form stu dent input. Doctoral Dissertation, Clemson University, 2012. (35) Cooper, M. M.; Underwood, S. M.; Bryfczynski, S. P.; Klymkowsky, M., W. In Tools of Chemistry Education Research ; Cole, R., Bunce, D., Eds.; ACS Symposium Series; American Chemical Society , 2014; Vol. 1166, pp 219 239. 147 (36) Corbin, J.; Strauss, A. Basics of qualitative research: Techniques and procedures for developing grounded theory , 3rd ed.; Sage Publications: Thousand Oaks, CA, 2008. (37) Glaser, B. G.; Strauss, A. L. Awareness of d ying ; Aldine: Chicago, 1965. (38) Cokelez, A.; Dumon, A. Chem. Educ. Res. Pract. 2005 , 6 (3), 119 135. (39) Hammer, D. J. Learn. Sci. 1996 , 5 (2), 97 127. (40) diSessa, A. A. Cogn. Instr. 1993 , 10 (2 & 3), 105 225. (41) diSessa, A. A. In Internatio nal handbook of research on conceptual change ; Vosniadou, S., Ed.; Routledge: New York, 2008; pp 35 60. (42) McMurry, J. E.; Fay, R. C. General chemistry: Atoms first ; Pearson Education, Inc: Upper Saddle River, NJ, 2010. (43) Novak, J. D. Sci. Educ. 2 002 , 86 , 548 571. (44) Ausubel, D. P. Educational Psychology: A Cognitive View ; Holt, Rinehart and Winston: New York, 1968. (45) National Research Council. How people learn: Brain, mind, experience, and school. ; National Academies Press: Washington, DC , 1999. (46) Cooper, M. M.; Klymkowsky, M. W. J. Chem. Educ. 2013 , 90 , 1116 1122. (47) Nahum, T. L.; Mamlok - Naaman, R.; Hofstein, A.; Krajcik, J. S. Sci. Educ. 2007 , 91 , 579 603. (48) Johnstone, A. H.; Selepeng, D. Chem. Educ. Res. Pract. 2001 , 2 (1) , 19 29. 148 CHAPTER VII: ARE NON - COVALENT INTERACTIONS AN ACHILLES HEEL IN CHEMISTRY? A COMPARISON OF INSTRUCTIONAL APPROACHES Introduction The question of how to improve student learning in chemistry has long been debated . Since the earliest , au thors have proposed improved ways to teach particular topics or to restructure courses. 1 5 However, t hese earlier efforts were hampered by several factors, including a lack of appropriate assessments and a dearth of research on teaching and learning guided by theories of learning. Therefore, although it was not for lack of effort, it was difficult to provide strong evidence about effective strategies, interventions , or curricular innovations designed to improve specific aspects of student learning . Over the past twenty years or so much more evidence has been gathered about how people learn 6 and , more specifically , what p roblems students encounter when they learn chemistry . 7,8 Despite this, there is still sparse evidence about strategies that have actually improve d student understanding and use of specific concepts. M ost of the current approaches to improve teaching and learning focus on ma king the classroo m more student - centered, 9,10 and there is now a good deal of evidence that these techniques can improve student success and persistence in a particular course. 11 T he measures of such success , however, are typically grades in the course, per formance on nationally - normed examinations, or multiple - choice concept inventories. 11,12 Much of the chemistry education research literature describ es student difficulties and misconceptions 7,13 and , while many authors also provide a section on implications for teaching and learning , there i s scant evidence that these findings have made th eir way into mainstream courses in a way that shows they have improved student understanding. At the same time, there is a great deal of evidence that even students who perform well in our 149 courses and abo ve average on nationally - normed tests may develop profound misunderstandings of core concepts in chemistry during instruction and are often unable to explain these concepts or transfer their ideas to new situations. 7,14,15 In our current work 14 18 , we are using both research on student difficulties and theories of learning to design, implement and asse ss evidence - based approaches to improving student learning. In this chapter, we describe the effects of a curriculum transformation designed to help students develop a robust understanding of structure - property relationships and present evidence of student - property relationships: intermolecular forces. Intermolecular forces We have previously shown that many students, who perform well on traditional assessments , have p rofound misunder standings about intermolecular forces ( IMFs ) . 14,19 In particular, we found that a majority of students in our studies constructed representations of IMFs showing them as interactions within (rather than betwe en) molecules. We believe that one reason this problem has been under - reported for so long is because of the nature of the assessments used to probe student understanding of IMFs . For example , one study on student ideas about hydrogen bonding uses items su ch as In which of the following compound(s) is hydrogen bonding likely to occur between the molecules? 20 That is, there is an assumption that understanding which substances can exhibit hydrogen bonding is evidence that the student understands what hydrogen bonding is and where it occurs. This is almost certainly an invalid assumption for many students. Most prior studies on how students understand IMFs use either multiple - choice instruments or short answer assessment items. For example, studies employing a 150 two - tier diagnostic test found that around 30% of students tend to ha ve a problematic understanding of IMFs as forces within a molecule, rather than between. 21,22 Others have focused on hydr ogen bonding and have also reported problematic ideas, particularly around the notion that the O H covalent bond is the hydrogen bond. 20,23 While confusions about the nature of hydrogen bonding are quite understandable (the name itself is mislea ding), there are fewer studies that include other types of intermolecular forces , such as London dispersion forces or dipole - dipole interactions . 24,25 W e developed the Intermolecular Forces Assessment (IMFA), discussed in Chapter 5, to investigate student understanding of IMFs by requiring students to construct representations and t o explain in words what they understand about each IMF . 19 Th e IMFA was previously administer ed to students using our online platform beSocratic , which allows us to collect, record and analyze both student writing and drawings . 26,27 We found t hat only one student out of 94 in our previous study was consistently able to represent all three intermolecular forces (hydrogen bond ing, dipole - dipole and LDFs) as interacti ons between separate molecules. 19 In general, about 55% of each student cohort tended to represent IMFs as interactions within a single molecule by, for example, drawing a molecule and circling one or more bonds. We also found that much of what students wrote about IMF s was ambiguous ; indeed without the student - generated drawing s as further evidence it would not have be en possible to determine whether students underst oo d that intermolecular forces are forces between molecules rather than within . This result was surpris ing and perturbing. The students involved were successful students in a well - designed traditional general chemistry course where students average around the 75 th percentile on the ACS general chemistry examination. 28 If a majority of students do not have a working understanding of the difference between IMFs and chemical bonds , it should not be 151 surpris ing when we find that students believe that bonds break during phase changes 14,20,23,29,30 or that they have trouble developing reaction mechanisms. 31 34 Developing improved understanding in a reformed curriculum Cooper and Klymkowsky have previously reported on the development of a new general chemistry curriculum, Chemistry, Life, the Universe and Everything (CLUE) . 35 This curriculum is based on a carefully scaffolded progression of th re e core ideas: structur e, properties and energy. Each core idea is developed over the year long sequence and is connected simultaneously and explicitly to the other core ideas. Scientific practices 36 such as constructing explanations , arguments and models are emphasized. At the same time , we developed an online syste m ( beSocratic ) that allows us to ask students to explain phenomena and construct diagrams and models by drawing and writing in response to prompts . 26 ,27 Our current work focuses on assessing how the CLUE curriculum affects student performance and understanding. We have previously reported 16 on a comparison of student ability to draw and u s e Lewis structures for matched cohorts of CLUE and traditional students. We found that CLUE students were significantly bet ter at drawing Lewis structures and were also more likely to self - report that these structures could be used to predict both chemica l and physical p roperties 16 using the Implicit Information from Lewis Structure Instrument (IILSI), a self - report instrument. 17 That is , CLUE students were more likely to recognize that structural representations could be used to predict and explain properties than traditional students. In this chapter, we continue this study with an investigation of how CLUE and traditional students compare in their representations of intermolecular forces. In our studies using the IILSI, we found that by the end of general chemistry both CLUE and tradi tional students were equally 152 likely to indicate that they could predict the types of intermolecular forces a substance would exhib it. 16,37 However, the IILSI is a self - report instrument and , although recognizing that IMFs can be predicted from structures is an important first step, it does not mean that students ac tually understand what IMFs are . We , therefore , turned to the newer assessment instrument, the IMFA 19 , in which students are asked to draw three molecules of ethanol and show the loca tion of each intermolecular force (hydrogen bonding, di pole - dipole and LDFs). They are also asked to explain what they understand about each IMF and to explain anything they cannot portray in their drawings . The details and design of the in strument are reported in Chapter 5 . Research Questions 1. How do CLUE students representations of IMFs compare to students enrolled in a traditional general chemistry course? 2. How do students at different institutions compare in their representations of IMFs? 3. of the subseque nt organic chemistry course ? Methods Student p opulation s and study d esign The Intermolecular F orces A ssessment (IMFA) was administered at two universities. Clemson University is a medium - sized, southeastern public research university with an average fall enrollment of 1600 students in first - semester general chemistry (GC1). Subsequently, approximately 600 students enroll each fall in organic chemistry. In the fall of 2012, the composition the 3400 - student freshman class of Clemson was approximately 52% ma le and 48% 153 female with the freshman population being predominately white (84%). The mean SAT score for incoming freshman was 1246 with an average ACT range of 26 - 31. Michigan State University (MSU) is a large, midwestern public research university with a f all enrollment of approximately 2500 students in first - semester general chemistry. The freshman class at MSU in 2013, consisting of over 7000 students, was approximately 48% male and 52% female. As with Clemson, the freshman population at MSU was predomina tely white, at 76%, with an average ACT range of 23 - 28. While both universities are research intensive with comparable student populations and demographics, there are some differences in course requirements and learning environments between Clemson and MS U. First, about 65% of students at Clemson take both semesters of general chemistry (GC) sequentially from fall to spring semester, unlike MSU where this percentage drops to around 30%. Second, the typical size of the lecture course ranges from 100 - 170 at Clemson and from 350 - 430 for MSU. The general chemistry course at MSU also has a recitation component where students are encouraged to work additional problem sets and seek assistance from graduate teaching assistants. Lastly, at Clemson the general chemis try lecture and laboratory sections are listed as a single course, while at MSU, the general chemistry lecture and laboratory are separate courses and, in fact, most students do not take the GC lab course concurrently with lecture. The IMFA was administer ed to two cohorts of students (Cohort 1 and 2) enrolled in introductory chemistry courses at Clemson and an additional cohort of students (Cohort 3) enrolled at MSU. These cohorts consist of students who have completed the first semester of general chemist ry and are enrolled in the second - semester of general chemistry (GC2). Additionally, students from Cohort 1 were followed through two years, from first - semester 154 general chemistry through the second semester of organic chemistry (OC2). Student populations f or each cohort can be seen in Table 7.1. Table 7.1: Student populations for each cohort at Clemson University and Michigan State University Clemson MSU Cohort 1 ( Fall 2011 Spring 2013) Cohort 2 ( Fall 2012 Spring 2013) Cohort 3 ( Fall 2013 Spring 201 4) Traditional GC: N=94 CLUE GC: N=87 Longitudinal Study (through OC2) : Traditional N =25 CLUE N=30 Traditional GC: N=160 CLUE GC: N=117 Traditional GC: N=239 CLUE GC: N=187 Study 1: A comparison of CLUE and traditional students at Clemson University Coh orts 1 and 2 from Clemson consisted of two groups of students: those enrolled in both the first (GC1) and second semester (GC2) of the CLUE general chemistry course or the traditional general chemistry course. The traditional general chemistry course at Cl emson uses a widely available general chemistry textbook. 38 Learning objectives are provided for traditional students and course examinations are designed to address multiple representations and concepts as well as testing facility with calculations and other skill - based questions. Many of the tradit ional lecture sections are designed to be interactive with students answering clicker questions, taking group quizzes, and participating in in - class activities. Students complete the the end of GC2 and typically score around the 75 th percentile. 28 We used a quasi - experimental cont rol - treatment design. 39 CLUE students in both Cohort 1 and Cohort 2 were directed towards the CLUE lecture sections of the course because of their intended majors in biological sciences or pre - professional health studies, but a number of other majors, includ ing engineering, were also enrolled. The traditional comparison groups for both 155 Cohorts 1 and 2 were selected from the larger general chemistry population of students enrolled in the traditional course. For Cohort 1, traditional students were selected base d on their similarity to the CLUE student population in major, sex, SAT composite score, Metacognitive Activities Inventory (MCAI) scores 40 , Student Understanding of Models in Science (SUMS) scores 41 and Implicit Information from Lewis Structures Instrument (IILSI) scores 17 . Traditional students in Cohort 2 were selected based on their similarity to the CLUE population in major, sex, SAT composite score, MCAI scores, and responses to the IILSI (further details are listed in Appendix K). CLUE students in Coh orts 1 and 2 had different lecture instructors for the course, both of whom were familiar with the design and implementation of the CLUE curriculum. IMFA responses were collected from Cohort 1 and Cohort 2 at the end of GC2 in order to ensure that all CLU E and traditional GC students had been exposed to, and been assessed on, the topic of intermolecular forces. Student responses were collected outside of the lecture setting in the laboratory course, which runs concurrently with the lecture. Since there is an inherent conflict of interest when assessing the effects of a reform designed by the research team, it was important to separate the data collection from the lecture as much as possible. Neither of the CLUE lecture instructors was involved with data col lection. Students received credit (laboratory participation points) for at least attempting to complete the assessment. The IMFA was administered by research assistants not involved with the lecture course using beSocratic , a web - based system that allows f or free - form student input such as written text responses, drawings, and constructed graphs. 26,27 allow for drawing with a stylus and typing written responses. 156 Study 2: A comparison of CLUE and traditional students from different universities Cohort 3 consists of CLUE and traditional GC students enrolled at MSU. It is important to not e that, for Cohort 3, we do not have a matched control group to make statistical comparisons between instructional approaches. In this chapter, we are presenting the analysis of CLUE student responses to provide preliminary evidence of how the curriculum t ransfers from one institution to another. We also include the traditional student responses in a comparison to those from Clemson to address additional difficulties experienced by students. As with Cohorts 1 and 2, student responses for Cohort 3 were colle cted at the end of GC2 after all students had been taught, and tested, on intermolecular forces. Since students are not required to take the laboratory course concurrently with lecture at MSU, student responses were instead collected by research and teachi ng assistants during recitations using the beSocratic recitation work closely with the lecturer for the course. Therefore, the data co llection at MSU is more closely tied to the lecture section and the instructor of the course. The CLUE instructor for Cohort 3 at MSU is the same instructor who taught CLUE Cohort 1 at Clemson. IMFs While the majority of our data from Clemson was collected from students enrolled in general chemistry, we also followed a subset of general chemistry students in Cohort 1 who continued on through both sequential semesters of organic chemistry (OC1 an d OC2) to investigate how student representations of IMFs changed in subsequent courses. In this study, the IMFA was administered again at the end of OC2. Since most majors at Clemson do not require students to take more than one year of chemistry, many st udents do not go on to enroll in organic 157 chemistry, therefore our CLUE and traditional groups by the end of OC2 were significantly smaller (N = 30 and N = 25, respectively). For this study, we compared the results for only this smaller subset of students w ho had completed all administrations of the IMFA. Data coding and analysis The studies discussed here focus on student drawings; as we have previously reported to the location of IMFs. 19 Therefore, we chose to code the drawings only, which provided us with a much less ambiguous representation of students understanding about IMFs. As in our prior study, the three major codes for each IMF are between (molecules), within (molec ules) and ambiguous as shown in Table 7.2. Table 7.2: Coding examples for student drawings of selected types of intermolecular forces IMF t ype Code for IMFA response drawings characterizing IMF l ocations Within the m olecule Betwe en m olecules Ambiguous Hydrogen Bonding Dipole Dipole Interactions LDFs 158 applied to representations that clearly indicated an IMF as occurring both within a molecule as well because some students, rather than providing a representation indicating the location, wrote that LDFs are always present, or occur everywhere. For the purpose of clarity, we will present the majority of responses in our study. The full data sets are available in Appendix L. Results an d discussion Study 1: R esults and d iscussion Clemson University, C ohorts 1 and 2, GC2 As shown below in Figure 7.1 and Table 7.3, there is a significant difference between the representations of IMFs locations from CLUE students and those in the traditional general chemistry c lass for both Cohorts 1 and 2. In general, the majority of CLUE students draw all types of IMFs as interactions between molecules, while the majority of traditional students draw them as within individual molecules. Specifically, 83 % (N=72) of CLUE student s in Cohort 1 and 84% (N=98) from Cohort 2 drew hydrogen bonds between molecules. Of the 72 Cohort 1 CLUE students who drew hydrogen bonding between molecules of ethanol, 96% (N=69) correctly indicated the location between the oxygen of one molecule and th e hydrogen covalently bonded to oxygen on another molecule. That is, not only did the majority of CLUE students 159 correctly depict hydrogen bonding as occurring between molecules, but almost all of them provided what would be considered a correct representat ion of hydrogen bonding between appropriate elements on each molecule. O nly 10% of CLUE students in both Cohort 1 (N=9) and Cohort 2 (N=12) provided a n incorrect representation of hydrogen bonding as a covalent bond within a molecule of ethanol. These data are shown in Figure 7.1. Figure 7.1: dipole - dipole, and LDFs from Cohort 1 and Cohort 2 As we have previously reported 19 , however, 72 % (N=68) of traditional GC2 students in Cohort 1 and 56% (N=90) of traditional students in Cohort 2 indicated that hydrogen bonding occurred within the molecul e as seen in Figure 7.1 . This pattern, where CLUE students represent IMFs between molecules and traditional students indicate they are bonds withi n a molecule, was not limited to hydrogen bonding. T representations of both dipole - dipole (63%, N=55 for Cohort 1 and 72%, N=84 for Cohort 2) and LDFs (62%, N=54 for Cohort 1 and 68%, N=80 for Cohort 2) also showed IMFs as int eractions between molecules . Similarly, 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Within Between Comparing IMFs drawing code frequencies for Cohort 1 and Cohort 2 Traditional Sp12 (N=94) Traditional Sp13 (N=160) CLUE Sp12 (N=87) CLUE Sp13 (N=117) 160 t - dipole (61%, N=57 for Cohort 1 and 58%, N=93 for Cohort 2) and LDFs (55 % , N=52 for Cohort 1 and 56 % , N=89 for Cohort 2) as occurring within the et hanol molecule. F or each of the three IMFs, at least 55% of the traditional student representations from Cohort 1 and 2 depicted all IMFs as a bond within a single ethanol molecule. At most only 31 % (N=50, Cohort 2) of traditional students ever provided a depiction of hydrogen bonding as located between molecules , and even f ewer represented dipole - dipole and LDFs as occurring between molecules, as shown below . While Figure 7.1 shows the most common codes, within and between, applied to student drawings, a ll code frequencies for all Cohort 1 and 2 responses are included in Appendix L. As shown in Table 7.3, significant differences were identified between the CLUE and traditional groups code frequencies for both withi n and between molecules . These differences were found for all three IMFs in both Cohorts 1 and 2. Code frequencies were analyzed using chi - square statistics and effect sizes ( ) are included. Effect size values greater tha n 0.3 are considered a medium effect size and those greater than 0.5 are considered large. 42 Major codes that resulted in statistically significant differences between the two groups are included in Table 7.3. Additional codes, including ambiguous, not present, and always present, are included in Appendix L. 161 Table 7.3: IMF drawings at the end of GC2 in Cohort 1 and Cohort 2. p - values from chi - square analysis are reported along with calculated effect sizes ( , phi coefficient) Cohort IMF Code Traditional p ercentage CLUE p ercentage p - value Cohort 1 Hydrogen bonding Within 72.3 10.3 < .001 0.62 Between 14.9 82 . 8 < .001 0.67 Dipole - dipole Within 60 . 6 13 . 8 < .001 0.47 Between 10 . 6 63 . 2 < .001 0.54 LDFs Within 55 . 3 14 . 9 < .001 0.41 Between 11.7 62 . 1 < .001 0.51 Cohort 2 Hydrogen bonding Within 56.2 10.2 < .001 0.46 Between 31.3 83.8 < .001 0.51 Dipole - dipole Within 58.1 16.2 < .001 0.41 Between 15.6 71.8 < .001 0.56 LDFs Within 55.6 12.8 < .001 0.43 Between 20 .0 68.4 < .001 0.48 The findings from Cohort 1 and 2 for both CLUE and traditional students are very similar despite the fact that diff erent instructors taught CLUE each year and that the traditional students were chosen from sections taught by at least five different instructors. We believe that the persistent differences we see here are a result of the curriculum, not a function of stud ent ability or the instructor. Consistency of responses for Cohort 1 While Figure 7.1 depicts the percentage of students who received within or between codes for each IMF, it does not show whether these students are consistent in the way they represent I MFs. That is, a single student could have drawn hydrogen bonding as occurring within a molecule, but provided ambiguous drawings for dipole - dipole and LDFs. We used Sankey diagrams 43,44 were consistent. Figure 7.2 shows two Sankey diagrams, the first for traditional students in Cohort 1 and the second for CLUE students in Cohort 1, to show how the representations 162 provided by each student change (or do not change) between all three IMFs. The width of the pathways represents the number of students who took that path. Although the d iagram can only show how students change between two consecutive IMFs, it does indicate that there is a lack of consistency for many students. For example, 10 traditional students who received a within code for hydrogen bonding then stated that dipole - dipo le was not present for ethanol. However, the diagram does not show what codes those 10 students received next for LDFs. But we can see, for all - dipole, what codes they received for LDFs in their subsequent draw ing. Group Hydrogen Bonding Dipole - dipole LDFs Figure 7.2: Flow chart representing the consistency of code frequencies applied to student drawings in Cohort 1, both traditional and CLUE, across all three IMFs Ideally we would like to see that st udents have a consistent understanding of all IMFs as interactions between molecules. While at least 60% of CLUE students indicate each IMF operates between molecules, when we look at consistency we see that a somewhat smaller group, 163 traditional student population only one student out of the entire sample consistently represented all three IMFs as occurring between molecules. Only 6% (N=5) of CLUE students consistently received within codes for all three IMFs representations, while a significant subset of traditional students consistently represent IMFs as interactions within a single molecule (38%, N=36) or provided an ambiguous representation (2%, N=2). This leaves a m ajority of traditional students (59%, N=55) who were inconsistent in their depictions of intermolecular forces as examples of the same phenomenon (be it within or between). Similarly, even though a majority of CLUE students provided accurate representation s for the location of each IMF as between molecules, in fact many CLUE students (47%, N=41) were inconsistent from one IMF to another. All drawing code are provided in Appendix M. Study 2: Results and d iscussion Michigan State University, Cohort 3, e nd of GC2 While we were able to successfully reproduce our initial findings of the impact of the t of students taught by a different instructor, it might be argued that the results are not generalizable since all of the responses collected from both CLUE and traditional students for the first study came from a single university. Therefore, we administ ered the IMFA to both CLUE and traditional students enrolled in a second - semester general chemistry course (GC2) at MSU. Figure 7.3 shows the third cohort from MSU (Univ. 2). 164 Figure 7.3: cohorts collected at two universities As shown in Figure 7.3, the CLUE students responses are quite consistent among the three cohorts . However, we c annot make statistical comparisons between them because we do not have measures for student performance or prior achievement that are consistent across all three cohorts. For hydrogen bonding, slightly more than 8 0 % of students in all three cohorts receive d between codes for their representations. Between 58% and 72 % of students in each cohort received between codes for their drawings of dipole - dipole and LDFs. It is notable that these responses are similar despite the differences in classroom environments between Clemson and MSU. We also collected IMFA responses from students enrolled in the traditional course at MSU. The data were collected in the same manner as the CLUE student responses, and the students were given the same amount of credit for completi on of the activity. While we cannot make claims about any comparison between CLUE and traditional student achievement since the 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Within Between Within Between Within Between Hydrogen bonding Dipole-dipole LDFs Drawing code frequencies for CLUE students across all three cohorts CLUE Univ. 1, Cohort 1 (N=87) CLUE Univ. 1, Cohort 2 (N=117) CLUE Univ. 2, Cohort 3 (N=187) 165 students were not a matched cohort, we did find that traditional students at MSU had great difficulty with this task as seen bel ow. Figure 7.4 shows code frequencies for traditional students at MSU. The percentage of students from MSU who drew IMFs between molecules was similar to tradition al students from Clemson ( 23 % (N=56)). However, when compared to Clemson, far fewer students from MSU provided representations of IMFs within molecules. Instead, as can been seen from Figure 7.4, larger percentages of students at MSU (Univ. 2) received oth larger percentage of students at MSU (24%, N=57) had difficulty drawing the structure of ethanol for the IMFA than at Clemson (13%, N=12), which also made it difficult to interpre t some of their representations and determine the intended location of each IMF. Additional graphs showing the frequency of all other codes can be found Appendix N. Figure 7.4: ll three cohorts collected at two universities 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Within Between All other codes Within Between All other codes Within Between All other codes Hydrogen bonding Dipole-dipole LDFs Drawing code frequencies for traditional students across all three cohorts Trad Univ. 1, Cohort 1 (N=94) Trad Univ. 1, Cohort 2 (N=160) Trad Univ. 2, Cohort 3 (N=239) 166 Study 3 : Longitudinal s tudy results and d iscussion Clemson , Cohort 1, GC2 through OC2 As noted in the DBER report 7 , longitudinal studies of student lea rning are rare for many reasons. They are often difficult, expensive and time - consuming, and it is frequently impossible to track students over time because of the varying paths they take through their studies. We were able to follow a group of students fr om Cohort 1, both CLUE and traditional, through a full year of organic chemistry . O rganic chemistry , however, is not required for all majors and, as might be expected, there was a significant reduction in our sample size. By the end of OC2, 25 traditional students and 30 CLUE students remained from the original Cohort 1 who had completed all administrations of the IMFA over the course of two years. A comparison of the two groups showed that, even after a full year of organic chemistry, the majority of CLUE students continued to show IMFs between molecules and the traditional students still represented IMFs as occurring within molecules, as seen in Figure 7.5. That is, tudents left GC2 . Statistical comparisons between CLUE and traditional groups at each time point (the end of GC2 and again at the end of OC2) are shown in Table 7.4. The significant differences that were present at the end of GC2 were still significant aft er a full year of organic chemistry instruction with medium to large effect sizes. 167 Figure 7.5: representations of hydrogen bonding, dipole - dipole, and LDFs from GC2 to OC2 (Co hort 1) Table 7.4: Statistical results for longitudinal comparison of code frequencies for CLUE and p - values from chi - square analysis are provided with calculated effect sizes ( , phi coefficient) Drawing Semester Code Traditional p ercentage CLUE p ercentage p - value Hydrogen bonding GC2 Within 72 10 < .001 0.60 Between 24 90 < .001 0.63 OC2 Within 64 3 < .001 0.61 Between 36 90 < .001 0.53 Dipole - dipole GC2 Within 56 7 < .001 0.50 Between 20 87 < .001 0.63 OC2 Within 60 0 < .001 0.63 Between 8 70 .001 0.45 LDFs GC2 Within 60 10 < .001 0.49 Between 8 70 < .001 0.59 OC2 Within 60 3 < .001 0.58 Between 20 77 .001 0.53 Figure 7.6 shows a Sankey diagram o over time, and again it is clear that there is very little change in student responses after they leave GC2. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Hydrogen Bonding Dipole-dipole LDFs Hydrogen Bonding Dipole-dipole LDFs Within Between Longitudinal comparison of IMFs drawing code frequencies for CLUE and Control Traditional GC2 (N=25) Traditional OC2 (N=25) CLUE GC2 (N=30) CLUE OC2 (N=30) 168 Group End of GC2 End of OC2 Figure 7.6: Code frequencies applied to Cohort 1 CLUE and tradi representations of hydrogen bonding at the end of GC2 and again at the end of OC2 While the sample sizes are small, it seems clear that neither group changes much over an entire year of organic chemistry. While this is evidence that the e ffects of the CLUE curriculum are not temporary, it does show that organic chemistry does not seem to improve traditional students understanding of IMFs. This is not particularly surprising since most organic faculty presumably believe that students have a lready learned this material in general chemistry (de spite the fact that a robust understanding of organic chemistry requires students to understand how molecules interact as a prelude to reactions) . What our study implies is that the understanding of IMFs that students develop in general chemistry is crucial. 169 Conclusions As discussed earlier, the CLUE curriculum is explicitly designed to help students build a strong foundation of core ideas using a scaffolded progression of concepts. The topic of IMFs i s central to a robust understanding of structure - property relationships. Not only do students need to understand the linked set of ideas that support understanding of IMFs, but they also must understand the subsequent role of IMFs in a wide range of chemic al phenomena including phase changes and chemical reactions. In the CLUE curriculum, students are required to construct and revise answers to questions on a daily basis, whereas t he tradi tional general chemistry course, like most general chemistry courses, covers the material following the order of topics in the textbook . However, as with most general chemistry curricula, the design of the traditional course does not explicitly connect the numerous steps required to connect structure and properties, in whic h an understanding of IMFs plays a significant part, and little focus is placed on scaffolding these topics and building upon earlier foundational ideas. To understand IMFs, students must be able to use the molecular structure to predict the molecular pola rity and must also understand how the interactions between molecules determine properties such as melting or boiling points and acid - base behavior. If these ideas are not explicitly connected and reinforced at each step, and if students do not understand h ow each idea relates to the others and is used, then we believe students may not learn the material meaningfully and will instead rely on heuristics and rules. 14,45 47 170 Limitations The limitations of this study are twofold. First, we only have data from two universities. It will be important to determine if a wider range of students have the same kinds of trouble constructing representations of intermolecular forces and whether the CLUE curriculum is as effective when more broadly disseminated. We also do not discuss here whether students can actually put their understanding of IMFs to use; that is, can they use IMFs to predict relative melting and boiling points and to explain how molecules interact? These studies are ongoing and will be reported elsewhere. Implications for teaching We believe that the results shown here stem direct ly from the carefully designed CLUE curriculum, where the important concepts are connected, and students are made explicitly aware of the purpose of each of the concepts and skills being learned. That is, students learn meaningfully in ways that allow them to put knowledge to use. In addition, students regularly construct and use chemical representations to explain and predict phenomena. We believe that the inability of many students at MSU to construct reasonable drawings of individual molecules is a conse quence of the fact that they were never asked to construct answers to prompts, but rather they practiced multiple - choice questions that test fragments of knowledge and recognition. In this study, students do not appear to change they way they think about intermolecular forces once they leave general chemistry. Clearly this is problematic, both for future studies in chemistry, but also in biology where an understanding of IMFs (and more broadly non - covalent interactions) is assumed as prior knowledge. If th e majority of students have (at best) an inconsistent notion of this concept, it is unlikely that they will be able to reason about molecular interactions appropriately. 171 While there are certainly many ways to improve student understanding (besides wholesa le adoption of a new curriculum), we believe that, at the very least, students must be asked to construct representations and to draw and write about their understanding of chemical principles. It is also important that students be made explicitly aware of the purpose of the fragments of knowledge and skills that they are learning and that instructors help them to construct a more coherent base of knowledge on which to build their future understanding. 172 REFERENCES 173 REFERENCES (1) Havighurst, R. J. J. Chem. Educ. 1929 , 6 (6), 1126 1129. (2) Walker, N. Sch. Sci. Math. 1967 , 67 (7), 603 609. (3) Pauling, L. College chemistry: An introductory textbook of general chemistry ; WH Freeman: New York , 1964. (4) Merrill, R. J. Acad. Med. 1963 , 38 (1), 34 37. (5) Sienko, M. J.; Plane, R. A. Chemistry: Principles and Properties ; McGraw - Hill: New Yotk, 1966. (6) National Research Council. How people learn: Brain, mind, experience, and school. ; Natio nal Academies Press: Washington, DC, 1999. (7) National Research Council. Discipline - based education research: Understanding and improving learning in undergraduate science and engineering ; Singer, S. R, Nielson, N. R., Schweingruber, H. A., Eds.; Nation al Academies Press: Washington, DC, 2012. (8) Taber, K. S. Chemical misconceptions: Prevention, diagnosis and cure ; Royal Society of Chemistry: London, 2002; Vol. 1. (9) Tien, L. T.; Roth, V.; Kampmeier, J. a. J. Res. Sci. Teach. 2002 , 39 , 606 632. (1 0) Lewis, S. E.; Lewis, J. E. J. Chem. Educ. 2005 , 82 (1), 135. (11) Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Proc. Natl. Acad. Sci. 2014 , 111 , 8410 8415. (12) Hake, R. R. Am. J. Phys. 1998 , 66 (1), 64 74. (13) Özmen, H. J. Sci. Educ. Technol. 2004 , 13 , 147 159. (14) Cooper, M. M.; Corley, L. M.; Underwood, S. M. J. Res. Sci. Teach. 2013 , 50 , 699 721. (15) Cooper, M. M.; Grove, N.; Underwood, S. M.; Klymkowsky, M. W. J. Chem. Educ. 2010 , 87 , 869 874. (16) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. J. Chem. Educ. 2012 , 89 , 1351 1357. 174 (17) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Chem. Educ. Res. Pract. 2012 , 13 , 195 200. (18) Underwood, S. M.; Reyes - Gastelum , D.; Cooper, M. M. Sci. Educ. in press . DOI: 10.1002/sce.21183. (19) Cooper, M. M.; Williams, L. C.; Underwood, S. M. J. Chem. Educ. 2015 . (20) Schmidt, H. - J.; Kaufmann, B.; Treagust, D. F. Chem. Educ. Res. Pract. 2009 , 10 , 265 272. (21) Peterson, R . F.; Treagust, D. F.; Garnett, P. J. Res. Sci. Teach. 1989 , 26 , 301 314. (22) Goh, N. K.; Khoo, L. E.; Chia, L. S. Aust. Sci. Teach. J. 1993 , 39 (3), 65 68. (23) Henderleiter, J.; Smart, R.; Anderson, J.; Elian, O. J. Chem. Educ. 2001 , 78 (8), 1126 11 30. (24) Villafañe, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Biochem. Mol. Biol. Educ. 2011 , 39 , 102 109. (25) Tarhan, L.; Ayar - Kayali, H.; Urek, R. O.; Acar, B. Res. Sci. Educ. 2008 , 38 , 285 300. (26) Bryfczynski, S. P. BeS ocratic: An intelligent tutoring system for the recognition, evaluation, and analysis of free - form student input. Doctoral Dissertation, Clemson University, 2012. (27) Cooper, M. M.; Underwood, S. M.; Bryfczynski, S. P.; Klymkowsky, M., W. In Tools of Ch emistry Education Research ; Cole, R., Bunce, D., Eds.; ACS Symposium Series; American Chemical Society, 2014; Vol. 1166, pp 219 239. (28) American Chemical Society Examinations Institute http://chemexams.chem.iastate.edu/ (accessed Jan 9, 2015). (29) B arker, V.; Millar, R. Int. J. Sci. Educ. 2000 , 22 , 1171 1200. (30) Bodner, G. M. J. Chem. Educ. 1991 , 68 (5), 385 388. (31) Grove, N. P.; Cooper, M. M.; Rush, K. M. J. Chem. Educ. 2012 , 89 , 844 849. (32) Bhattacharyya, G.; Bodner, G. M. J. Chem. Educ . 2005 , 82 (9), 1402. (33) Bhattacharyya, G. Chem. Educ. Res. Pract. 2014 . (34) Ferguson, F.; Bodner, G. M. Chem. Educ. Res. Pract. 2008 , 9 (2), 102 113. (35) Cooper, M. M.; Klymkowsky, M. W. J. Chem. Educ. 2013 , 90 , 1116 1122. 175 (36) National Resear ch Council. A framework for K - 12 science education: Practices, crosscutting concepts, and core ideas ; National Academies Press: Washington, DC, 2012. (37) Underwood, S. M.; Re yes - Gastelum, D.; Cooper, M. M . Sci . Educ. in press . DOI: 10.1002/sce.21183. ( 38) McMurry, J. E.; Fay, R. C. General chemistry: Atoms first ; Pearson Education, Inc: Upper Saddle River, NJ, 2010. (39) Campbell, D. T.; Stanley, J. C.; Gage, N. L. Experimental and quasi - experimental designs for research ; Houghton Mifflin: Boston, 19 63. (40) Cooper, M. M.; Sandi - Urena, S. J. Chem. Educ. 2009 , 86 , 240 245. (41) Treagust, D. F.; Chittleborough, G.; Mamiala, T. L. Int. J. Sci. Educ. 2002 , 24 (4), 357 368. (42) Cohen, J. Statistical power analysis for the behavioral sciences , Second .; Lawrence Erlbaum Associates: Hillsdale, New Jersey, 1988. (43) Schmidt, M. J. Ind. Ecol. 2008 , 12 (1), 82 94. (44) Schmidt, M. J. Ind. Ecol. 2008 , 12 (2), 173 185. (45) Maeyer, J.; Talanquer, V. Sci. Educ. 2010 , 94 , 963 984. (46) McClary, L.; Ta lanquer, V. Int. J. Sci. Educ. 2011 , 33 (10), 1433 1454. (47) Taber, K. S. Int. J. Sci. Educ. 2009 , 31 (10), 1333 1358. 176 OF HYDROGEN BONDING The key idea to keep in mind is that the true power of educational technology comes not from replicating things that can be done in other ways, but when it is D. Thornburg 1 Introduction he need to incorporate scientific practices, disciplinary core ideas, and cross - cutting concepts into STEM courses. 2 While these changes are crucial to improving the way STEM content is taught, in order to successfully incorporate these changes we have to reconsider the ways in which we assess st udent knowledge. Specifically, we need to ensure that we are creating meaningful assessments that target key ideas and practices. Today, multiple - choice (or forced - choice) assessments are an extremely common form of testing in post - secondary education. Mul tiple - choice (MC) assessments certainly have their benefits, which undoubtedly have contributed to their rising popularity over time. They are easy to administer to large groups of students and typically result in quick and consistent scoring. 3 In fact, Zeidner found that most students prefer multiple - choice over essay exams because they often view them as easier, less complicated, and less stressful. Oddly enough, this is despite the finding that the majority of students perceive essay exams as better r eflecting their content knowledge. He even noted that students found it easier to prepare for MC exams, spending less time and effort studying. 4 177 There are certainly disadvantages to using MC assessments to elicit student understanding. By design, multiple - choice questions limit the number of possible a nswer choices a student can make; they can also provide unintended benefits for students like allowing students to work backwards from provided answers and increasing the odds of students guessing the correct answer. 5 Effective MC questions can be difficult to design and poor distractor choices can inadvertently lead a student to the correct answer. Additionally, Birenbaum and Tatsuo ka compared the effect of multiple - choice and open - fractional addition problems and found it more difficult to identify s misconceptions using multiple - choice items when compared to an open - ended format. 6 As shown from our own interviews, students are certainly able to circumvent the intent of questions like boiling point ranking task s by memorizing general trends or rules rather than using a deeper understanding of the relationship between structure and properties. 7 Fisher and Lipson ng tendency to avoid extra mental effort, so as to minimize their information processing load and to conserve their attentional resources. This tendency often results in attention to, and use of, superficial rather than deep - structure aspects of a situatio 8 The propensity to use heuristics, memorization, and trends is well documented in the chemistry education literature and is not particularly surprising. 7,9 12 Heuristics and trends allow us to bypass our deeper System 2 thinking processes in favor of the faster and more efficient System 1 processes. 13 15 While heuristics and shortcuts certainly have their benefits when used appropriately, much of chemistry (and STEM disciplines in general), require the conscious use of System 2 processes to truly develop understanding of the subject matter. Assessment questions that promote deeper thinking are certainly more favorable for probing student understanding of scienti fic concepts. While some MC questions could be constructed in 178 such a way as to get at this understanding, there are other question formats that may be better understanding. Constructed response questions include a wide rang e of question formats ranging from free - response and essays to drawings and graphs. 16 Unlike their multiple - choice counterparts, they often cover greater cognitive range and can elicit complex performance and divergent student ideas. 17 Drawings in particular can provide a unique view into student understanding and can help foster a deeper learning of science. 18 22 Several studies have compared responses from students who wrote explanations to those who provided drawings, and they found differences in the features emphasized by each group. 23,24 Indeed in our own work described in Chapter 6, we were better able to determin 25 By requiring students to draw representations, we can get a glimpse into the ways in which they think about chemistry on the microscopic and particulate levels. 18,20,26 While requiring students to construct representations can certainly be illu minating, it is not always practical. Viewing and scoring large numbers of student drawings can be time consuming and requires a well - defined and refined scoring rubric. In order to use these rubrics graders require training and calibration; even so, the s coring of responses is not always consistent. Open - ended written explanation questions provide a middle ground between MC questions and those requiring students to construct representations or graphs. Like drawings, eater variety. As Ha and colleagues outlined in their own studies in the field of biology, most MC assessments focus on identifying either novice or expert ideas. Many do not allow for the wide range of interconnected ideas that students often possess. 27 179 possess a large and varied set of fragmented ideas that can be combined in different ways to inform their understanding of a phenomenon. 28 Open - ended writing formats are better able than MC questions to reve al the assortment of ideas that students possess and how they are interwoven. 27 Sadly, these questions possess similar drawbacks to drawing and constructed responses. Specifically, they can be difficult to interpret, require more time to score, involve extensive grader training, and have greater variability in rater scoring. 29 While constructed representations can provide some additional benefits over open - ended writing questions, like eliciting structural information 23 25 technological advancements in recent years. Researchers are now training computer programs to code text responses like discipline experts. A utomated computer scoring Automated computer scoring allows researchers to train computers to predict human scoring of written responses using a series of algorithmic models. This is often achieved through a stan dard procedure. Human researchers code (or score) a large set of written responses. Typically, this larger set of responses is then divided into two smaller sets: a training set and a validation set. Computers use the training set to create algorithms and models that best fit the data provided. These algorithms can then be applied to the validation set and the resulting scoring predicted by the computer is compared to the original human scoring. If the models and algorithms are robust, there should be high agreement between the computer scoring and human scoring. 30 Many studies have successfully used automated text analysis, or lexical analysis, to achieve high scoring agreement for written essays and short answer respo nses. 27,29 33 Several stu dies in the field of biology education specifically, ranging from evolution to acid - base 180 reactions, have found computer scoring models to accurately reflect human scoring. 27,29,32 A well - designed and tested automated scoring model should ultimately provide quick, uniform include the fact that computers lack bias, do not grow tired of coding, and are consistent in their scoring (unlike humans at times). 30 While there are numerous programs 34 36 available for text analysis, we used SPSS Modeler, a commercial software program provided by IBM, to conduct our text anal ysis. 37 SPSS Modeler includes a wide range of predictive analytic tools, including text analytics, to inform and guide research . After importing large sets of text data, Modeler identifies, as best it can, every term used in the provided responses. The program comes with a library of preset dictionaries that include common terms the system readily recognizes. The user can add new terms to an existing dictionary or create new dictionaries to expand the terms the software can recognize, like creating a dictionary for terms related specifically to chemistry. Once the software recognizes the relevant terms, it is the job of the researc her (or user) to place the important and relevant terms into appropriate bins, or categories, for later analysis. For instance, Hydrogen Ions . Once categories have been est ablished, Modeler offers an array of statistical tests that use these categories as variables in exploratory or predictive statistical models. Exploratory statistical tools allow researchers to visualize connections between categories or cluster similar re sponses 181 Text analysis, with programs li ke Modeler, has some disadvantages. For instance, in order to build a robust model, the system typically requires a large number of text responses to analyze (on the order of hundreds). An additional set of responses, preferably large in size, is required to test the created model for validity. However, large data sets are not the only downside. Certain question formats, such as compare and contrast questions, often do not perform well in Modeler, at least not without extra work on behalf of the user. So, f or example, when students compare and contrast an acid and a base, they often (and should) discuss both types of compounds. However, because the system does not have a built - in proximity function, it cannot determine if words such as acid or base go with s odium hydroxide or with hydrochloric acid. The system can identify that a student used both compound names but cannot determine which words in their response relate to the acid and which relate to the base. While the use of SPSS Modeler does require a lar ge amount of work in the initial phases, the rewards of creating a robust set of categories and predictive models can be numerous. For new sets of text response s can be inserted into the model to be automatically scored. This is particularly useful when analyzing student responses across multiple courses or years. For example, if an instructor were interested in giving the same question each year to monitor the p rogress of a specific group of students, they could potentially use a single model to continually and consistently analyze new responses. Purpose of study With our previous studies, we attempted to explore the connection between what students say and wha t they draw. That is, do the constructed representations they provide for IMFs align 182 with the ways in which they describe IMFs? We found that students were more likely to provide structural information, like the location of IMFs, in their drawings than in their written descriptions of IMFs could predict the code assigned t o their drawings of IMFs. That is, since know which words or combinations of words were more strongly associated with students who represented IMFs as between mol ecules rather than within. Coding drawing responses is certainly time - their text responses could provide quick feedback as to their level of understanding. For this analysis compared to their drawings of hydrogen bonding as elicited on the IMFA. By doing so, the content of both questions is the same with the difference lying in modality. Specifically we were int erested in the following research questions: RQ1 bonding predict the location of hydrogen bonding in their constructed representation? RQ2. What impact does an alternative gener al chemistry curriculum have on written IMFA responses? That is, can lexical analysis of written responses differentiate between students enrolled in different curricula? 183 Methods Data for this study was collected from students enrolled in the second - semester of general chemistry at Michigan State University (MSU, Cohort 3). These are the same CLUE and traditional student responses analyzed in Chapter 7 to compare the performances of students on the IMFA across different universities and curricu la. For this particular study, we chose to use data from MSU because of the need for a large number of responses ( N =320). While we could have combined responses from both Clemson and MSU to create a larger student sample, there is some evidence that the un iversity at which students are enrolled can affect the language of their responses and thus the outcome of text analysis. 27 In order to reduce the number of possible confounding variables, we limited responses in our study to those from a single university, collected at a single time point. We did, however, include both CLUE and traditional general nses in this study. We did not have enough responses from the traditional group ( N =144) or CLUE group ( N =176) alone to create reliable statistical models. previously in Chapter 5. Text responses from items 2 and 4 were exported from beSocratic, combined, and imported into SPSS Modeler. To produce reliable results, automated computer scoring requires a sizeable sample of responses for each drawing code from item 7 from whi ch a model is built. Therefore, the only drawing codes models we report in this analysis are 40 responses). With such a small number of responses, we could make s ome claims based on the results, but more responses would certainly be needed to verify the validity of any findings. For 184 Lexical category creatio n After we uploaded our entire data set of 320 responses into Modeler, the text analytics package extracted 263 Figure 8.1(a). From created 59 fine - grained lexical categories (LC). That is, most catego ries we created were for a single word /term , like nitrogen , and were pertinent to the topic of hydrogen bonding. We did not applicable our data but alone, held little meaning o r more than one meaning, making them difficult to correctly and reliably categorize. Examples of created categories can be seen in Figure 8.1(b). The F luorine lexical category ncluded and others like it were also included in both the O xygen and N itrogen for consistency. An example of LC application ca Hydrogen bonds are weak attraction forces between hydrogen and another atom because of hydrogens partial positive charge hydrogen bond, strength, attraction (as a subcategory of interaction), force, between, hydrogen, another, atom, and polarity (specifically the subcategories partial, positive, and charge). It was extremely common for an individual response to be classified in several different categories. These connections betwe en multiple categories can distinguish between groups of responses during later statistical analysis. 185 (a) (b) Figure 8.1 : a) Examples of terms (concepts) extracted by Modeler using the Text analysis node; b) Examples of lexical categories created for terms extracted by Modeler Some of our more complex lexical categories arose from a need to address differentiation have the same mean ing ond. Unfortunately for us, Modeler recognized Unlike some text analysis software packages, Modeler does not have a buil t - in word proximity function that would allow 186 hydrogen bond were unsuccessful. Our solution involved creating an entirely new term in the chemistry library within Modeler hydrogen and bonding in student responses. This allowed the software to differentiate between drogen bond = We used a similar solution to address the difference between t London dispersion force s . We were able to use rules functions le, we created the Between Molecules LC to attempt to identify the location of the IMF that students were describing. The goal of this rules - refer to between atoms or specific element s instead of between molecules. We also created a few larger categories by combining related subcategori es. For instance, we created a Correct EN A toms LC that included oxygen , nitrogen , and fluorine . We decided to combine these categor ies after a cursory cluster analysis revealed that students most often used these three terms simultaneously. Conceptually, we were not particularly concerned if students could remember all three electronegative atoms involved in hydrogen bonding. In t he s ame manner, we created the Polarity LC , which contained terms like dipole (not to be confused with dipole - dipole) and charges, and the Incorrect EN atoms LC with terms like carbon and sulfur . For analysis purposes, we could choose to treat these eith er as large categories or split them out in to their individual subcategories if we wanted a more detailed analysis . It is important to note that, while Modeler is particularly good at extracting terms from a given set of responses, it cannot assign mean ing to these terms. That is, Modeler can identify the 187 creating an attractive force between the two but a hydrogen bond is not actual ly a real bond. Force LC and Bond LC to not a bond. Unfortunately, there are not many simple ways to correct for this err or. The most obvious solution is the inclusion of a proximity function, which Modeler does not have. For our B ond LC as is, even though a few responses would get incorrectly categorize d. Ideally, the categorization of the rest classification via discriminant analysis than a single misapplied category. Applying discriminant analysis We used the responses about hydrogen bonding could be used to predict the location code they would receive for their subsequent drawing of hydrogen bonding. Discriminant analysis works by creatin g a discriminant score for each text response from a number of independent variables . In this c ase, . These discriminant scores can often be combined into groups based on similarity of scores, creating a mean discriminant score called the group centroid . Discriminant analysis attempts to maximize the distance between group centroids to create higher performing models. The s e discriminant score s can then be used to predict the probability that a specific response falls into a given dependent variable category (in our case whether or not a student draws hydrogen bonding as occurring between molecules) . The larger 188 the difference between the centroids, the more distinctive the groups are and the better the mo del will perform in predicting which responses fall into which groups. 38 We chose discriminant analysis over linear regression since discriminant analysis is better suited for categorical data, such as our mutually exclusive drawing codes. Additionally, discriminant analysis measures how the independent variables change together (the covariance). 38 Because of this, the in analysis. We chose to use a step - wise discriminant analysis in Modeler. By doing so, Modeler works to build a model by adding categories one at a time until the model no longer shows improvement. Th is helps to prevent over - fitting the model, which is especially important when working with smaller sets of data. Categorizing responses using web diagrams While discriminant analysis can be useful in predicting the coding category a response would rece ive, it can be a complicated process if the goal is to generally characterize a group of responses. Since our data set consisted of responses from both CLUE and traditional students, we chose to use web diagrams to explore similarities and differences in s from these two groups. While we knew from previous studies that these two groups differ in both how they represented IMFs and whether they discussed the location of IMFs in the written descriptions, we were also interested in explor ing what other differences might exist in the language they used to describe IMFs. Modeler provides the option of creating web diagrams to into connections st udents think are important in answering the question and characterize similarities and differences in the ways each group describes IMFs. 189 Th number of responses. The thickness of the line relates to the prominence of the link in terms of overall percentage. For example, a link between H ydrogen and A cid wit h medium thickness could represent 10% of the number of overall links made. Differences in the type and thickness of various links can help characterize differences in how our two groups discuss hydrogen bonding. Results and Discussion Using discriminant analysis with lexical categories to predict the hydrogen bonding drawing code H ydrogen followed by Hydrogen B onding and B ond , as seen in Table 8.1. This is probably beca use students often repeat words given in the prompt (like hydrogen bonding) in their like molecule, interaction, and another, were used by less than half of the sample. Additionally, there is an interesting difference in the percentage of responses that fall in the B etween LC as opposed to the Between Molecules LC alone is not always indicative of between molecules . 190 students use the word to describe the relationship between molecules. Table 8.1: Most commonly assigned lexica l categories (those applied to greater than 20% of the total sample) Lexical category Percent of responses with the lexical category Hydrogen 91.9% Hydrogen bond(ing) 73.1% Bond 56.3% Between 50.9% Correct electronegative atoms (includes O, N, and/or F) 49.7% Atom 40.0% Molecule 39.1% Interaction 29.3% Strong 26.9% Another 25.3% Electronegativity 23.1% Polarity (includes dipole, charges, negative, and positive) 22.2% Between molecules (category using rules to collect responses using between and molecules) 21.3% In order to address our first research question, we used discriminant analysis to develop models that could predict the drawing code a student would receive for their constructed representation using their written description of hydroge n bonding. We generated two independent models, one for each drawing code (between and within) using the entire data set ( N =320). Table 8.2 , shown below, lists the lexical categories chosen by Modeler as predictors for each model. For the model that predi - weights in a reg ression analysis). 38 The variety shows that , rather than a single term that easily discriminates, it is a combination of all of these terms that distinguishes between 191 lambda = 0.675, chi - square = 122.082, degrees of freedom = 14, p - value = <0.001) and the significant p - ues listed in Table 8.2 is dependent on where Modeler places the group centroids. In each model, centroids as being placed on a linear axis, ranging from negative to positive values. 38 For example, for the between model, th e system placed the group centroid for responses that received a between code (present) at 0.550, while placing the centroid for those that did not receive a between code (absent) at - 0.869. For this particular model, coefficients with a positive value are more likely to cause a response to be predicted to have a present between code rather than the absence of the code. Of the fourteen variables selected by Modeler, nine had positive coefficient values indicating they were more strongly associated with rece iving a between drawing code; the largest coefficient was associated with the LC Oxygen . When considering the context of the M olecule and Another with the understanding that h ydrogen bonding is an interaction between molecules. E lectronegativity and P olarity , which are more closely associated with understanding of scientific principles, also lend to a more developed understanding of IMFs as interactions. This means that students who included ideas like electronegativity, polarity, and interaction in their writing were more likely to receive a between molecules code for their drawing. 192 Table 8.2: Lexical categories used as standardized cano nical discriminant function coefficients for each drawing code model Model LC name Canonical coefficients Between Oxygen 0.549 Electronegativity 0.431 Molecule 0.363 Polarity 0.337 Another 0.323 Different 0.228 Interaction 0.175 IMFs 0.103 Electrons 0.091 Attached - 0.030 Occur - 0.036 Atom - 0.114 Bond - 0.156 Carbon - 0.262 Within Molecule 0.659 Electronegativity 0.403 Negative 0.383 Bond - 0.396 molecules model, and the larger negative values make sense as predictors for the lack of a between molecules code in the context of the assessment. We would hope that students who possess a strong understanding of hydrogen bonding as an interaction between molecules are less likely to and fluorine, carbon does not participate in hydrogen bonding because the electronegativity difference between carbon and hydrogen is relatively small (or practically negligible), so students should not be discussing carbon as part of their explanation of hydrogen bonding. receive a between mo lecules drawing code (absent). 193 It is important to note that the Bond LC few scenarios in which a student may appr opriately include the term bond while discussing when a hydrogen that is bonded to a n atom like oxygen or nitrogen is attracted to another atom with a hi Conversely, only four lexical categories were used by Modeler to create a model for - square = 35.758, degrees of freedom = 4, p - value = <0.001). Unlike the between model, the group centroid for the presence of a within code was negative while the absence of the code was positive. This means that the lexical category Bond was the only predictor used in the computer mo del for those who received a within drawing code. E lectronegativity , M olecule , and N egative were used as predictors for absence of the within drawing code . For students who represent hydrogen bonding as a covalent bond within the molecule, it makes sense t hat their discussion of the IMF would also include the of terms related to scientific principles. Both models created were moderately successful in predicting Specifically, t he between he within model correctly classified 79.1%. Modeler provides a break down of responses into correct classifications, false positive, and false negatives. Correctly classified for our study meant that the predicted presence or absence of a drawing code from our model matched the human scoring Table 8.3 shows the classification break down both drawing code models. With this information we can see where our models may be lacking and attempt to refine them. 194 Table 8.3: Agreement and classifications of the between and within drawing code models Lexical category Correctly classified False negative False positive Bet ween 250 (78.1%) 31 (9.7%) 39 (12.2%) 0.54 Within 253 (79.1%) 67 (20.9%) 0 (0%) N/A Using the information in Table 8.3, we within drawing codes. Even though the percent agreement between the human coder an d Modeler is fairly high, calculating inter - rater reliability allows us to factor in the effect of chance agreement between raters. We could not calculate kappa for the within code because Modeler did not predict any student responses as having a within co de, resulting in no false positives. As a result, it is difficult to make any inferences about reliability for the within code. For the between - 0.6 are considered moderat e and 0.2 - 0.4 fair. 39 These kappa values, however, are lo wer than we would normally expect for inter - rater agreement. Often, we report values at 0.8 or higher when comparing the scoring of two human coders. While the percentage of correctly classified responses for the between model (78.1%) and the within model (79.1%) predict their performance on a drawing task. Typically, discriminant analysis is used to predict human scoring of written response s. Considering the fact that we are attempting to use student writing to predict student drawings, we consider the percentage of students correctly classified Exploring differences in CLUE and tradition al student responses using web diagrams In order to address our second research question, we chose an exploratory approach to identifying the differences between students enrolled in the CLUE and traditional curricula. 195 Rather than starting with discrimina each grou p (CLUE N =176, Traditional N =144) using web diagrams. It is important to note that does not ne correlations to our CLUE/traditional course variable to observe how students w ere connecting these terms and with what prevalence. For instance, do some students more readily use both bond and oxygen in their response, and what inferences can we make from that information? For creating web diagrams of CLUE and traditional student re sponses from MSU, we included the following lexical categories: Another, Bond, Carbon, Correct E lectronegative A tom , Dipole - Dipole, Element, Hydrogen Bond(ing), Highly, Hydrogen, Intermolecular Forces, L one (specifically in reference to lone pair ), Molecul e, Not, Polarity, Weak, Interaction, O ccur, and P resent Interaction and Polarity were subsumed under the main category for simplicity as fewer categories typically lend to web diagrams with greater readability. The only exceptio n was for the Correct EN atoms LC where the subcategories O xygen , N itrogen , and F luorine were individually included. For the web diagrams included here, we chose to have the lines between categories represent overall percentages. By using overall percenta ges, we can show values as percentages of the total number of links in the web diagram. For instance, 2.6% of links made by CLUE students were between hydrogen and oxygen. It was the fourth highest percentage out of all links made by CLUE students. While 2 .6% does not appear to be very large, we have to consider that, in total, CLUE students made approximately 3360 links. In absolute numbers, that means 88 CLUE students (or 50% of our sample) included both of these terms in their responses. By using 196 overall percentage, we can help reduce the impact of the fact that one group of students may also wrote more words in general, the overall percentages between the two g roups would be comparable and not skewed in favor of the group who simply wrote more words. The web diagrams provided below only show percentages greater than 1% of the overall number of links for each group. Link percentages below 1% tend to apply to less than 10% of students in the sample and are often not as useful. Thicker lines denote stronger connections with link percentages greater than or equal to 2%, while thin lines show weaker connections with link lso important to note that the web diagram for traditional students contains two additional categories: C arbon and E lement . These two lexical categories fell over the 1% threshold for traditional responses but not for CLUE responses. We will explore this m ore later, particularly in reference to carbon. We can make some initial inferences from the web diagrams shown in Figures 8.2 and 8.3. While traditional students have more strong connections in general than CLUE students (16 compared to 13), CLUE studen Polarity , Interaction , and Another , which relate to an understanding of the correct underlying chemical concept s . The presence of the Polarity LC and Interaction LC again implies an understanding of the underlying scie ntific principles involved in interactions between molecules. The Another LC may be bonding between molecules. While we did not require students to explicitly d iscuss the underlying causes of hydrogen bonding (i.e. columbic attractions between molecules arising from differences in electronegativity) it is interesting that more CLUE students are using these specific terms in conjunction with those common to all re sponses, such as hydrogen or molecule. 197 Figure 8.2: Figure 8.3 : 198 As seen in Figure 8.3, traditional Carbon and Element . While the E lement LC itself may not be particularly insightful, the addition of the Carbon LC is an interesting finding and is consistent with reports by Henderleiter and colleagues of students identifying carbon as capable of hydrogen bonding. 40 Generally, we do not expect students to discuss carbon when describing hydrogen bonding. Instead, both groups of students refer to specific elements like hydrogen, oxygen, nitrogen, and fluorine as is evident in both web diagrams above. It is important to note that the overall percentage is small for links to C arbon ; the largest link is between C arbon and H ydrogen at 1.22%. Only about 10% of traditional (compared to 2% of CLUE students). One way to correctly include carbon in a response would be to say that hydrogen bonding could not happen between hydrogen and carbon. However, we know that most, if not all, traditional students are not using allows us to infer that students are indeed indicating that carbon is capable of hydrogen bonding, udents used the term traditional students is troubling. We can glean other pieces of information from the data used to create the web diagrams. For instance, w e know that CLUE students, in total, made an average of 3360 links while traditional students made about 1060 links. Taking into account their different population sizes, CLUE students made approximately twice as many links as traditional students (14 link s per CLUE student and 7 links per traditional student). This indicates that CLUE students have longer and richer explanations. This is not entirely surprising considering that students enrolled in the traditional GC course at MSU are rarely asked to expla in their understanding, unlike CLUE 199 students. Based on the web diagrams, the responses are not just longer, but contain more apply only to the lexical categories we included for the creation of our web diagrams, these categories were specifically chosen because of their strong correlation to the CLUE/traditional variable. Using discriminant analysis of student responses to predict GC course enrollment After anal ysis of CLUE and traditional web diagrams, we used discriminant analysis to try and determine if Modeler could identify a student as CLUE or traditional based on their written response about hydrogen bonding. Using the lexical categories created from stude responses, we were able to successfully build a model to predict whether students from Cohort 3 were enrolled in the CLUE or traditional general chemistry course. In this case, only a single model (course enrollment) was necessary to determine if a st udent was in the CLUE or traditional group because these two groups were the only options for students in our data set. Our - value of <.001 (chi - square = 148.852, degrees of freedom = 10), meaning t he variables selected explained approximately 38% drawing codes, our model achieved a fairly large separation between group centroids with CLUE at 0.704 and traditio nal at - significant in discriminating between the two groups. 200 Table 8.4: GC Model LC names Canonical c oef ficients CLUE/traditional course Electronegativity 0.541 Interaction 0.475 Fluorine 0.389 Bond 0.319 Another 0.288 Element 0.271 IMFs 0.242 Oxygen 0.241 Strength 0.207 Carbon - 0.219 Electronegativity , I nteraction , and F luorine had the highest positive function value indicating that they were stronger predictors for CLUE students. The LC I nteraction is particularly encouraging to see listed as such an influential coefficient. As discussed previously, it is certainly preferable for studen ts to refer to IMFs as interactions or attractions rather than bonds. The confusion of bonds and IMFs can result in other ideas, such as the notion that boiling breaks covalent bonds rather than overcoming IMFs. 7,41,42 The inclusion of the LC E lectronegative as a strong predictor of CLUE enrollment is also promising. While we did not specifically ask students why IMFs occur, the inclusion of a discussion of polarity and electronegativity alludes to a deeper understanding of the relev ant scientific principles. Like Polarity LC (as seen in the web diagrams), the inclusion of E lectronegativity as a strong predictor is encouraging. The Fluorine LC as the third highest predictor for C LUE enrollment was surprising. To the best of our knowledge, neither curriculum placed additional emphasis on fluorine as a requirement for hydrogen bonding (although it is one of the three elements most often cited). One possible reason for fluorine as a predictor of enrollment in the CLUE course is that the 201 electronegat ive element, located at the top right of the periodic table. Additionally, we are unsure as to why the Bond LC was included as a predictor for CLUE enrollment. This category does not ay be that CLUE students were referring to the need for hydrogen to be covalently bonded to oxygen, nitrogen, or fluorine in order to participate in hydrogen bonding. Our predictive model for GC course enrollment performed slightly better than our previou are provided below in Table 8.5. The kappa value for inter - rater reliabil ity is just above the 39 While still not in our ideal range of higher than 0.8, again we want to emphasize the effect of the nature of the study on measurements of inter - rater reliability. As with our previous discriminant analysis models, we are at tempting to use text responses to predict variables other than text coding or scoring rubrics. Additionally, we have only used one data collection to attempt to predict course enrollment. More responses from additional cohorts would most likely help highli ght the differences between the courses. Like the drawing codes, when attempting to predict which course a student is enrolled in, we would not expect perfect agreement. Table 8.5: Classification and agreement for the GC course model Lexical category Corr ectly classified False negative False positive CLUE/Traditional 260 (81.3%) 32 (10.0%) 28 (8.8%) 0.622 202 Conclusions location of IMFs in their constru cted representations do seem promising, currently the models created are not robust enough to consistently and accurately predict drawing codes. While we did manage to achieve relatively high percentages of co rrectly classified responses, our models did no t achieve large kappa values indicating less than ideal agreement between human coders and the predicted scoring. There are certainly several reasons as to why this may be the case. The model created for the within drawing code was heavily biased towards t he presence of a within code, resulting in no responses being categorized as lacking a within code by Modeler. Some of these issues could possibly be mediated by using larger sample sizes. In fact, Ha and colleagues tested the effects of using a larger tra ining set on automated computer scoring models with the machine - learning program LightSIDE 43 , designed by LightSide Labs . They found that, by doubling the number of responses used to create their models, they did increase performance in almost all of their models (although not always substantially). They noted that the frequencies of certain concepts or terms in the training set also impacted the resulting model performance and should be ta ken int o consideration . 27 As stated previously, we should not be expecting ideal agreement between human coding predicted coding when we are attempting to use written responses to predict unexpected to find robust (but moderate) correlation magnitudes on test s of the s ame domain but research, they found moderate correlation coefficients when comparing oral interview responses to written responses. They argue that, while the ir values are not traditionally high, the correlation 203 between human - scored responses and interviews , as well as computer - scored responses and interviews , were considerably higher than the correlations between the forced - choice assessment and interviews. 31 While they used different statistical analyses in their study, the same general idea can apply here. The between drawing model effectively discriminated between student drawing responses that would receive a within or between drawing code based on their written descriptions of hydrogen bonding. These results d location. In Chapter X, we found that many students failed to discuss the location of IMFs in their written responses , making it difficult to determine (based on that information alone) if they understood IMFs to be interactions between molecules or bonds within a single molecule. Modeler, on the other hand, takes into account all terms used by students in order to build a predictive model. Our predictive models were better able to predict the drawing code a student would receive based on their written response, essentially highlighting if the student understood hydrogen bonding to occur between molecules rather than with in. It stands to reason that students with a stronger understanding of the forces and interactions taking place would be more likely to include scientific principles in their writing as well as draw IMFs as an interaction between molecules. While our draw ing code models did not achieve 100% agreement, they may still be applicable, with further revision, for quickly representation. That is, while we certainly encourage instr uctors to have their students draw and construct representations, applying our model for text analysis could give instructors a quick 204 student drawings. As for c omparing two different introductory general chemistry courses, it appears that the CLUE and traditional students differ in the words they use to describe hydrogen bonding. While both groups use several common terms (i.e. hydrogen, nitrogen, and oxygen) fre quently, there are a few key terms that appear to be used predominantly by one group over the other, to use the term carbon. While these differences may seem small, the fact that Modeler can, based on words alone, differentiate between two different GC courses certainly indicates that there must be differences in the way they discuss IMFs and the understa emphasis on columbic forces, energy, and their impact on molecular interactions. Ideally, we would like stud ents to discuss concepts like molecular shape, electronegativity, and the resulting bond and molecular polarity as contributing factors to the type and strength of IMFs a molecular compound is likely to exhibit. The inclusion of these terms in CLUE student certainly promising data indicating that the curriculum is succeeding in building these connections. Responses that only mention the elements involved or the strength of IMFs fail to show a solid understanding of the underlying principles t hat lead to IMFs and are a reflection of the types of information accepted as evidence of understanding the traditional course. 205 Limitations One limitation to using discriminant analysis is the need for a large amount of student responses. We were able occurring within or between molecules, but unfortunately we could not produce successful t o the low number of responses. More submissions from students who express these ideas in their drawings would be required to pursue building robust models for these codes. Additionally, in this study we only explored one IMF (hydrogen bonding) and thus us ed only one written response and one drawing from each student. By focusing on hydrogen bonding , we were less likely to a thorough discussion of scientific concepts like electronegativity or polarity because many students do not need to use these ideas to predict the IMF present ; they can often successfully rely the presence of hydrogen directly bonded to nitrogen, oxygen, or fluorine. Future work would benefit from exploring multiple explanations from students about the same topic to create a more rounded picture of their understanding and improve our models. - dipole and LDFs that have yet to be explored using text analysis. It is important to note that, while we did have enough written responses to cre ate predictive models, we did not have enough to create a second data set to test our models. Ideally, we would want to test the reliability of our models on a new group of student responses to determine if the models consistently predicted human scoring. Because we were unable to do so, 206 REFERENCES 207 REFERENCES (1) Thornburg, D. D. Technology in K - 12 educ ation: Envisioning a new future. Forum on Technology in Education: Envisioning the Future , 1999. (2) National Research Council. A framework for K - 12 science education: Practices, crosscutting concepts, and core ideas ; National Academies Press: Washington , DC, 2012. (3) Simkin, M. G.; Kuechler, W. L. Decis. Sci. J. Innov. Educ. 2005 , 3 (1), 73 97. (4) Zeidner, M. J. Educ. Res. 1987 , 80 (6), 352 358. (5) Bridgeman, B. J. Educ. Meas. 1992 , 29 (3), 253 271. (6) Birenbaum, M.; Tatsuoka, K. K. Appl. Psy chol. Meas. 1987 , 11 (4), 385 395. (7) Cooper, M. M.; Corley, L. M.; Underwood, S. M. J. Res. Sci. Teach. 2013 , 50 , 699 721. (8) Fisher, K. M.; Lipson, J. I. Instr. Sci. 1985 , 14 (1), 49 74. (9) Bodner, G. M. J. Chem. Educ. 1991 , 68 (5), 385 388. (1 0) Maeyer, J.; Talanquer, V. Sci. Educ. 2010 , 94 , 963 984. (11) McClary, L.; Talanquer, V. Int. J. Sci. Educ. 2011 , 33 (10), 1433 1454. (12) Taber, K. S. Int. J. Sci. Educ. 2009 , 31 (10), 1333 1358. (13) Stanovich, K. E.; West, R. F. Behav. Brain Sc i. 2000 , 23 (5), 645 665. (14) Evans, J. S. B. Trends Cogn. Sci. 2003 , 7 (10), 454 459. (15) Gilovich, T.; Griffin, D.; Kahneman, D. Heuristics and biases: The psychology of intuitive judgement ; Cambridge University Press: Cambridge UK, 2002. (16) Be nnett, R. E. In Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment ; Ward, W. C., Bennett, R. E., Eds.; Lawrence Erlbaum Associates, Inc. Publishers: Hillsdale, NJ, 1993; pp 1 2 8. (17) Martinez, M. E. Educ. Psychol. 1999 , 34 (4), 207 218. (18) Ainsworth, S. E.; Prain, V.; Tytler, R. Science 2011 , 333 , 1096 1097. 208 (19) Leenaars, F. A. J.; van Joolingen, W. R.; Bollen, L. Br. J. Educ. Technol. 2013 , 44 , 82 94. (20) Schwambor n, A.; Mayer, R. E.; Thillmann, H.; Leopold, C.; Leutner, D. J. Educ. Psychol. 2010 , 102 , 872 879. (21) Van Meter, P.; Aleksic, M.; Schwartz, A.; Garner, J. Contemp. Educ. Psychol. 2006 , 31 , 142 166. (22) Zhang, Z. H.; Linn, M. C. J. Res. Sci. Teach. 2 011 , 48 (10), 1177 1198. (23) Gobert, J. D.; Clement, J. J. J. Res. Sci. Teach. 1999 , 36 , 39 53. (24) Akaygun, S.; Jones, L. L. Int. J. Sci. Educ. 2014 , 36 (5), 783 807. (25) Cooper, M. M.; Williams, L. C.; Underwood, S. M. J. Chem. Educ. 2015 . (26) Stieff, M. Sci. Educ. 2011 , 95 , 310 336. (27) Ha, M.; Nehm, R. H.; Urban - Lurain, M.; Merrill, J. E. CBE - Life Sci. Educ. 2011 , 10 , 379 393. (28) diSessa, A. A. In International handbook of research on conceptual change ; Vosniadou, S., Ed.; Routledge: New York, 2008; pp 35 60. (29) Nehm, R. H.; Ha, M.; Mayfield, E. J. Sci. Educ. Technol. 2012 , 21 (1), 183 196. (30) Graesser, A. C.; McNamara, D. S. In APA handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and psychome trics ; Cooper, H., Camic, P. M., Long, D. L., Panter, A. T., Eds.; American Psychological Assocation: Washington, DC, 2012; pp 307 325. (31) Beggrow, E. P.; Ha, M.; Nehm, R. H.; Pearl, D.; Boone, W. J. J. Sci. Educ. Technol. 2014 , 23 (1), 160 182. (32) Haudek, K. C.; Prevost, L. B.; Moscarella, R. A.; Merrill, J.; Urban - Lurain, M. CBE - Life Sci. Educ. 2012 , 11 , 283 293. (33) Kaplan, J. J.; Haudek, K. C.; Ha, M.; Rogness, N.; Fisher, D. G. Technol. Innov. Stat. Educ. 2014 , 8 (1), 1 26. (34) Landauer, T. K. Assess. Educ. Princ. Policy Pract. 2003 , 10 (3), 295 308. (35) Sukkarieh, J.; Bolge, E. In Intelligent Tutoring Systems ; Woolf, B. P., Aimeur, E., Nkambou, R., Lajoie, S., Eds.; Lecture Notes in Computer Science; Springer Berlin Heidelberg: Berlin Germany, 2008; Vol. 5091, pp 779 783. 209 (36) Burstein, J. In Automated essay scoring: A cross - disciplinary perspective ; Shermis, M. D., Burstein, J., Eds.; Lawrence Erlbaum Associates Publishers: Mahwah, N.J, 2003; Vol. 16, p 238. (37) SPSS Modeler http: //www - 01.ibm.com/software/analytics/spss/products/modeler/ (accessed Mar 13, 2015). (38) Spicer, J. Making sense of multivariate data analysis ; SAGE Publications, Inc: Thousand Oaks, CA, 2005. (39) Landis, J. R.; Koch, G. G. Biometrics 1977 , 33 (1), 15 9 174. (40) Henderleiter, J.; Smart, R.; Anderson, J.; Elian, O. J. Chem. Educ. 2001 , 78 (8), 1126 1130. (41) Othman, J.; Treagust, D. F.; Chandrasegaran, A. L. Int. J. Sci. Educ. 2008 , 30 (11), 1531 1550. (42) Smith, K. C.; Nakhleh, M. B. Chem. Educ . Res. Pract. 2011 , 12 (4), 398 408. (43) LightSide Labs. LightSide http://lightsidelabs.com (accessed Jun 24, 2015). 210 CHAPTER IX : CONCLUSIONS, IMPLICATIONS, AND FUTURE WORK relationship between structure and properties and the role of intermolecular forces. Specifically, I completed four main studies to identify how students connect structure and properties, how they write about and represent intermolecular forces (IMFs), ho w a reformed curriculum (Chemistry, Life, the Universe, and Everything - expedite the analysis of written and drawn responses on the Intermolecular Forces Assessment (IMFA). Conclusions Main - property relationships As discussed in Chapter 4 1 , we interviewed students enrolled in both general and organic chemistry to explore how they understood the connection between structure and properties. We found that students possessed a wide range of diverse ideas; in fact, no two students answered our interview questions in the same manner and, as a result, we uncovered several different approaches that were used to discuss properties such as boiling and melting. Many students relied on heuristics, both personal and taught, to explain relative trends in boiling points. These heuristics were often consistently used, although they did not always lead to the correct answer. Others explained these physical processes using a collection of fragmented ideas, akin to - prims 2 , woven together to form an explanation. Typically, th ese ideas were not consistent since they were often dependent on the prompt. Some students struggled with the terminology used in chemistry courses, like the differences between bonds and interactions, 211 while others found it difficult to construct appropria te representations of phases, drawing lattice structures or breaking apart molecules. simplistic an approach. Many of the explanations voiced by students contained a n assortment of ideas, pieced together to answer the question at hand. This was particularly apparent with organic students who included irrelevant concepts such as steric hindrance or the inductive effect to explain boiling and melting processes. Dual pro cess theory can serve as a lens through which we can better understand these responses. 3 Several students often altered their answer choice after spending additional time thinking about the question. For instance, when Joy was asked if wanna say this one has an O, a hydroxyl group on it, so I feel like it would boil quicker than the Joy : Wait, actually I lie. I think this [ethanol] has the higher boiling point. Interviewer : Ok so what would you think, that the bonding was there before, so Joy With Joy, it is rather obvious that her initial reaction to ranking boiling points only invoked a System 1 response. It was only after she had taken the time to think through the question that she realized ethanol was capable of hydrogen bonding and thus had the higher boiling point of the two. 212 Main writing and drawing responses on the IMFA was described in Chapter 6 4 . While we would hope students understand IMFs as interactions that occur between small molecules, the majority of students in this study represented IMFs as bonds within a single molecule of ethanol. The numbers we have reported are certainly larger than what has been previously discussed in the lite rature. 5 7 Previous studies, however, often used multiple - choice assessments to determine where students believed IMFs to be located. I would argue that the act of selecting a correct answer from a set of four possible choices is inherently different than cons tructing a representation and drawing IMFs, which is more likely to make key structural information apparent. the interactions as within or between molecules, their written r esponses were far more ambiguous. Most students did not discuss the location of IMFs, instead citing surface - level definitions such as the strength of a given IMF or the elements involved. Few students discussed IMFs as electrostatic interactions, but this is not entirely surprising considering we did not ask for these details in our question prompts. In hindsight, the format of our short answer questions was less likely to invoke a System 2 response from students than questions probing for deeper understan ding. For instance, describing hydrogen bonding as the strongest intermolecular force, requiring the presence of hydrogen and nitrogen, oxygen, or fluorine is a completely legitimate response to the 213 question we asked. A discussion of future work below will outline some ideas to improve the design of the IMFA or create a new, sister assessment. Perhaps the most important finding from this study was that we would have been interactions between molecules rather than bonds within a molecule. It was only when we draw a representation of IMFs, we were essentially asking them to articulate their ideas and communicate them in a manner that is more likely to reveal spatial information, were completely contradictory. In the case of dipole - dipole, several stu dents described the interaction as between molecules but provided a representation as a bond within ethanol. We are not entirely sure as to the cause of this contradiction, but it is clearly evidence of a disconnect in student understanding. Main study 3: understanding of intermolecular forces understanding of IMFs through analysis of their drawings. We found that the majo rity of CLUE students from Cohort 1 consistently represented all three IMFs as interactions between molecules, unlike the majority of traditional students who represented IMFs as within molecules. We attribute this difference to the design of the CLUE curr iculum, which emphasizes the interconnectivity of structure, properties, and energy throughout 214 the year. The replication of these results in the data collected from Cohort 2 further supports the positive impact of the CLUE curriculum on student understandi ng of IMFs. In addition to collecting IMFA data from Cohort 1 and Cohort 2 at Clemson University, we also administered the IMFA to students at Michigan State University (MSU) to determine if CLUE had the same impact on student understanding when introduce d at a new university. While we could not make statistical comparisons between MSU and Clemson students, it does seem that CLUE students at MSU are just as likely as those at Clemson to represent IMFs as interactions between molecules. Traditional students at MSU received fewer within molecules codes for their representations of IMFs difficul ties constructing appropriate structures of ethanol, which made coding their responses more difficult. This is not entirely surprising, however, as these students were never asked on examinations or homework to construct representations of molecules. Fina lly, we followed a subset of students from general chemistry through organic understanding of IMFs. While our sample sizes were much smaller (it is difficult to follow students longitudinally through organic chemistry), we found that CLUE students still represented IMFs as occurring between molecules after a full year of organic chemistry and traditional students still represented them as within molecules. These results indicate that the effects of the CLUE curriculum are not temporary, rather the ideas that students form during their time with the CLUE course stick with them through subsequent chemistry courses. On the other hand, this also means that a full year of 215 organic chemi only require students to take two years of chemistry. The ideas about IMFs that traditional students leave general chemistry with are most likely the ideas that they will retain. Main study 4: Automated text analysis of IMFA responses In the previous chapter, we outlined the possibility of using automated text explored two different aspects of analysis: responses to predict their location drawing codes and 2) using web diagrams and We were able to build moderately su ccessful models to predict the location drawing code (between or within) a student would receive for hydrogen bonding from their written like Electronegativity and Polari ty used as strong indicators by our model to predict the these concepts would be linked to an understanding of IMFs as interactions between molecules. Similarly Bond and Carbon make sense as strong predictors of the presence of hydrogen bonding. While these models were able to determine the presence or absence of a location code (be it betw een or within), they were not robust enough use on new sets of data. Our 216 kappa values. This was not entirely surprising, however, considering we were using responses in one modality to attempt to predict the code a student would receive in another modality. We could possibly improve these models by using larger sets of data. By creating robust models, instructors would be able to give the IMFA to their students at several tim e points and quickly determine how their students are representing IMFs to gauge general understanding. We were also able to construct web diagrams of CLUE and traditional students descriptions of hydrogen bonding and identify key differences. For instan ce, CLUE Polarity and Interaction while traditional students possessed more links to the LC Carbon . Similarly, we were able to use discriminant analysis to build a model that could effectively differentiate between CLUE Electronegativity and Interaction highlights the effects of the CLUE curriculum on intermolecular for ces back to shape, polarity, and electronegativity, promoting meaningful learning, and this is reflected in the our discriminant model. 8,9 Implications The positive impact of the CLUE curriculum: Scaffolding structure - property concepts We believe there are several factors that explain why the CLUE curriculum appears to have suc CLUE curriculum is grounded in theories of how students learn and is intentionally designed to reflect the tenets of meaningful learning. It provides students with a solid f oundational 217 knowledge of the core concepts of structure, properties, and energy early in the first semester so that new ideas and topics can then be readily connected back to these foundational concepts. The curriculum emphasizes the interconnectedness of these three core ideas throughout the entire one - year course. Table 9.1 and 9.2 shows the order of topics that are addressed both in the CLUE course as well as the traditional textbooks used by GC courses at Clemson University (CU) and Michigan State Unive rsity (MSU). 10 13 218 Table 9.1: Table of contents for the material covered in the first semester of each general chemistry course Chemistry, Life, the Universe, and Everything (CLUE) CU and MSU General Chemistry: Atoms First Traditional GC, CU Chemistry Volume two Traditional GC, MSU 1) Atoms - Scientific theories - Atomic theory and evidence - Atomic structure - Interactions between atoms and molecules 2) Electrons and Orbitals - Light and quan tum mechanics - Spectroscopy - The periodic table 3) Elements, Bonding, and Physical Properties - Elements and their interactions - Discrete vs. continuous molecules - Molecular orbital theory - Metals 4) Heterogeneous Compounds - 3D and 2D representations - Lewis structures and shape - Shape, polarity, and interactions - Ionic bonding 5) Systems Thinking - Kinetic energy and temperature - Energy and gases - Thermodynamics and systems - Phase changes 1) Chemistry: Matter and Measurement - Elements and the P eriodic Table - Units and stoichiometry 2) The Structure and Stability of Atoms - Atomic theory and nuclear chemistry 3) Periodicity and the Electronic Structure of Atoms - Light and wave - particle duality - Quantum mechanics 4) Ionic Bonds and Some Main - Gr oup Chemistry - Molecules, ions, and bonds 5) Covalent Bonds and Molecular Structure - Electronegativity - Lewis structures and shape - Molecular orbital theory 6) Mass Relationships in Chemical Reactions - Stoichiometry and molarity 7) Reactions in Aqueou s Solutions - Types of reactions 8) Thermochemistry: Chemical Energy - Energy and enthalpy - Entropy and free energy 9) Gases: Their Properties and Behavior - Gas laws - Kinetic molecular theory 10) Liquids, Solids, and Phase Changes 1) Chemistry - Measure ment - Atoms and molecules 2) Stoichiometry 3) Reactions in Solution - Oxidation Reduction - Reactions in aqueous solutions 4) Energy - First law of thermodynamics - Changes of state - Enthalpies of reactions 5) Atomic Structure - Periodic properties 6) Bo nding and Molecular Structure - VSEPR - Molecular orbital theory 7) States of Matter - Gases and gas laws - Kinetic molecular theory - Intermolecular forces 8) Thermodynamics, Phase Diagrams and Solutions - Changes of state 9) Chemical Equilibria 219 Table 9.2: Table of contents for the material covered in the second semester of each general chemistry course Chemistry, Life, the Universe, and Everything (CLUE) CU and MSU General Chemistry: Atoms First Traditional GC, CU Chemistry Volume two Traditional GC, MSU 6) Solutions - Solubility and Gibbs energy - Polarity and solu tions - Temperature and solubility 7) A Field Guide to Chemical Reactions - Collisions and reactions - Acid - base - Nucleophiles and Electrophiles - Oxidation Reduction 8) How Far? How Fast? - Factors that control reactions - Reaction rate - Kinetics and activation energy - Equilibrium 9) Reaction Systems - Buffered reactions - Coupled non - equilibrium reaction systems 11) Solutions and Their Properties - Energy changes - Colligative properties 12) Chemical Kinetics - Reaction rates and rate laws 13) Chemical Equilibrium - Factors that a ffect equilibrium - Equilibrium and kinetics 14) Aqueous Equilibria: Acids and Bases - pH and acid - base theories - Equilibrium and strength 15) Applic ations of Aqueous Equilibria - Buffers and titrations - Factors that a ffect solubility 16) Thermodynamics: Entropy, Free Energy, and Equilibrium 17) Electrochemistry 18) Hydrogen, Oxygen, and Water 19) The Main - Group Elements 20) Transition Elements and C oordination Chemistry 21) Metals and Solid - State Materials 22) Organic Chemistry 1) Chemical Kinetics - Mechanisms - Rates of reactions 2) Aqueous Equilibria - Acid and Bases - Solubility equilibria 3) Thermodynamics - First and second law of thermodyna mics 4) Electrochemistry 5) Chemistry of the Main Group Elements 6) Chemistry of the Transition Elements - Coordination compounds - Ligand field theory and MO theory 7) Nuclear Chemistry - Radioactivity - Fission and Fusion 8) Organic Chemistry and Bioche mistry 220 The CLUE curriculum, as shown in Table 9.1, begins with a discussion of atomic theory, energy, and how atoms interact. This initial conversation introduces students to the idea that atomic interactions are governed by differences in charge, which provides the foundation for future discussions of how and why molecules interact. After examining current atomic theory using basic quantum mechanics, the CLUE curriculum begins to introduce the relationship between structure and properties for molecular c ompounds. Students are encouraged to think about properties that they know of for diamond and graphite, two substances that are composed entirely of carbon atoms, before entering a discussion of how these properties relate back to differences in structure on the molecular level. Here, the connection between structure and properties is made explicit for students and is discussed in terms of familiar macroscopic phenomena. Figure 9.1 shows the progression of topics discussed in the CLUE curriculum over the c ourse of Chapters 3, 4, and 5 to link molecular structure and properties. Each of the topics introduced in Figure 9.1 is clearly connected to the previous concept(s) that came before it. By discussing the relationship between these topics, we can help stud ents better understand the intermolecular forces encourages students to recall previous conversations on electrostatic interactions between atoms as well as the topics of electronegativity and polarity. The topic of IMFs is ultimately connected to larger concepts such as what happens at the molecular level during a phase change and solute - solvent interactions when discussing solubility. 221 Figure 9.1: The progression o f topics discussed in CLUE to connect molecular structure and properties . Reprinted with permission from Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. J. Chem. Educ. 2012 , 89 , 1351 1357. 14 Copyright 2015 American Chemical Society. Unlike the CLUE curriculum, the traditional textbooks used by Clemson and MSU fail to meet the ten et s of meaningful learning 8,9 ; IMFs are typically separated, both from the foundational material needed to understand them (Lewis structures, geometry and electronegativity ) , a nd from the physical and chemical properties that they help explain (solubility, phase changes). For instance, the traditional GC text used at Clemson University, General Chemistry: Atoms First, discusses Lewis structures, geometry, and polarity in Chapter 5, seen in Table 9.1, but does not begin to discuss IMFs or properties until Chapter 10. 11 Both curricula at Clemson and MSU include an extensive discussion of gas laws and kinetic molecular theory after introducing molecular structure and polarity but before a discussion of IMFs and properties of molecular substances. The CLUE curriculum explicitly connects these ideas and emphasizes the three core concepts of structure, properties, and energy throughout the course. 222 This emphasis is reflected in our IMFA results and it is clear that making these co nnections The positive impact of the CLUE curriculum: Engaging in scientific practices In addition to focusing on core chemistry concepts and using a scaffolded approach to introduce and teach these co ncepts, the CLUE curriculum engages students in scientific practices and encourages group work and discussion. Students are asked to build scientific explanations and arguments as well as construct representations in in - class assignments, homework, and eve n on exams. For instance, when learning how to draw Lewis structures, CLUE students are provided molecular modeling kits in class and encouraged to work together in groups to build three - dimensional representations of assigned molecules and translate those representations to two - dimensional Lewis structures. Students are also asked to practice constructing appropriate Lewis structures for homework through beSocratic and later on asked to use these structures to represent molecular interactions in a variety of ways (i.e. solute - solvent interactions, acid - base reactions, and IMFs). CLUE exams consist of a combination of multiple - choice and constructed - response assessment items. These constructed - response items often require students to write explanations, draw graphs, and depict representations of various chemical and physical phenomena. While the traditional course at Clemson could be considered reformed in that students used clickers and completed group assignments, neither traditional course at Clemson or MS U emphasized the need to construct representations, scientific explanations, or arguments. In fact, students at MSU are never given the opportunity in lecture or for homework to draw Lewis structures. Both universities use a standard online 223 homework manage ment system ( like Mastering Chemistry 15 ) that does not require students to construct representations. Additionally, traditional exams at both universities consist entirely of multiple - choice questions. As we have seen f rom our work with the IMFA, asking students to construct representations can reveal student understanding that would be otherwise unapparent in written responses or in answers to multiple - choice questions. Constructivism tells us that knowledge is construc ted in the mind of the learner. 16 How can we expect students to effectively develop a nd communicate ideas if we have never asked them to express their understanding and reconstruct it? As we have seen with the CLUE curriculum, giving students the opportunity to engage in these scientific practices can enhance their understanding of core id eas and provide them with a more realistic understanding of how science happens. 17 concepts (structure, properties, and energy), its scaffolded approach to teaching these concepts, and its emphasis on scientific practices can help explain why CLUE appears to explicitly ties the topic of IMFs to each core concept and students are asked to develop and demonstrate understanding of IMFs through the use of scientific explanations and constructed representations. Implications for assessment Our structure - property interviews have shown that students can effectively answer boiling point ranking tasks with little understanding of the underlying scie ntific concepts and ideas. Often students used heuristics to explain these trends. That is not to say that 224 heuristics are necessarily bad; experts use them, too. And it can certainly be argued that developing schema to expedite knowledge retrieval is neces sary to make it through any chemistry course. These shortcuts are essential to reduce cognitive load and prevent overwhelming working memory. 18,19 solid, foundational understanding of the relevant chemistry topics, then heuristics only succeed in masking an underlying lack of understa nding. Assessment items that can be easily answered with heuristics are doing a disservice to students. They often fail to highlight deep understanding and instead reward memorization and rote learning. Our reliance on multiple - choice assessments has cont ributed to the problem. Multiple - choice questions are useful and sometimes necessary ( especially for large class sizes ) but there are additional, and often times better, ways uncover to student understanding. We have also shown in our work with the IMFA th at question format is important; if the intent of a question is to uncover student understanding related to spatial information, then questions requiring students to draw may be more beneficial. We should be encouraging our students to build scientific exp lanations, use models, and construct representations. These practices should be reflected both in summative assessments, like exams, as well as formative assessments like homework and in - class assignments, as discussed earlier. With modern improvements in text analysis, it is already possible to code large data sets of text responses for the presence or absence of themes and ideas. 20,21 Hopefully, these advances will encourage instructors to expand beyond multiple - choice tests as th is technology becomes more available. In an effort to make text analysis more accessible, the Automated Analysis of Constructed Response group (AACR) at MSU has put their 225 questions and models online for instructors who would like to use them in their own c ourses and look for the presence of alternative biology ideas. Examples include questions pertaining to ecosystems, evolution, cellular respiration, and strong versus weak acids and bases 22 They are currently working to expand their studies to include questions in chemistry and mathematics fields. In the same vein, it may be possible, with onstructed representations (like our work with the IMFA), which could allow instructors to give a variety of assessments while still quickly and consistently coding/scoring responses for relevant information and ideas. Future work We have two future pro jects planned to extend the work described here. The first is to administer the IMFA at several universities of varying types and sizes across the country. While we have data from Clemson University and Michigan State University, as well as a small collect ion for a residential college within MSU, we would like to be able to determine if our findings from the IMFA are reflected in other educational settings. It may be that traditional students at Clemson and MSU experience more difficulties representing and describing IMFs than other universities, but we do not believe this is the case. By collecting similar responses from other institutions, we can highlight the extent Our second project would require ei ther revision of the IMFA or the creation of a new, sister assessment(s). We now understand that some of the short answer questions included in the IMFA do not appear to uncover deep student understanding of the 226 scientific concepts behind IMFs and why they to the IMFA were more surface - level and seemed to invoke a System 1 response. 3 We also never asked students in our assessment to use their understanding of IMFs to predict and explain various properties like boiling and melting points. A second (or even third) assessment would most likely inco rporate the drawings slides from the IMFA and focus on these two additional aspects of IMFs. Questions probing polarity and electronegativity could help identify if students understand their role in determining the type and strength of IMFs that a molecul e would exhibit. Possible items could be providing students with a clear depiction of hydrogen Why does this interaction occur between the oxygen in water and the hydrogen covalently bonded to oxygen in met hanol? What do you think is causing this interaction? oxygen in water does not interact with the hydrogens covalently bonded to carbon and ask them to explain why that is. Perhaps more difficult will be designing questi ons that effectively require students to connect their knowledge of IMFs to physical properties. We have made past attempts to uncover these ideas using assessment items and have not had much success. It may be that our questions were not directed enough t hat the information we were hoping to assess. For instance, we have given students the structure of an amino acid and ask them to predict and explain any properties it might have. Our intention was to see what properties students would identify and if they would relate them to the structure. Instead, however, responses varied widely in the type of properties predicted and few were ever explained or explicitly connected to the Lewis structure. Like some of our IMFA short 227 answer questions, the amino acid ques tion may have been too broad. Questions asking students to represent interactions between molecules and explain how these interactions affect the boiling point could be one possible direction. 228 REFERENCES 229 REFERENCES (1) Cooper, M. M.; Corley, L. M.; Underwood, S. M. J. Res. Sci. Teach. 2013 , 50 , 699 721. (2) diSessa, A. A. In International handbook of research on conceptual change ; Vosniadou, S., Ed.; Routledge: New York, 2008; pp 35 60. ( 3) Evans, J. S. B. Trends Cogn. Sci. 2003 , 7 (10), 454 459. (4) Cooper, M. M.; Williams, L. C.; Underwood, S. M. J. Chem. Educ. 2015 . (5) Peterson, R. F.; Treagust, D. F.; Garnett, P. J. Res. Sci. Teach. 1989 , 26 , 301 314. (6) Goh, N. K.; Khoo, L. E .; Chia, L. S. Aust. Sci. Teach. J. 1993 , 39 (3), 65 68. (7) Villafañe, S. M.; Bailey, C. P.; Loertscher, J.; Minderhout, V.; Lewis, J. E. Biochem. Mol. Biol. Educ. 2011 , 39 , 102 109. (8) Bretz, S. L. J. Chem. Educ. 2001 , 78 , 1107 1117. (9) Ausubel, D. P. The psychology of meaningful verbal learning ; Grune & Stratton: New York, 1963. (10) Cooper, M. M.; Klymkowsky, M. W. CLUE: Chemistry, Life, the Universe & Everything http://besocratic.colorado.edu/CLUE - Chemistry/ (accessed Apr 28, 2015). (11) Mc Murry, J. E.; Fay, R. C. General chemistry: Atoms first ; Pearson Education, Inc: Upper Saddle River, NJ, 2010. (12) Hunter, P. W. W.; Pollock, A. M. Chemistry Volume I , Second.; McGraw - Hill Learning Solutions: Boston, MA, 2011; Vol. 1. (13) Hunter, P. W. W.; Pollock, A. M. Chemistry Volume II , first.; McGraw - Hill Learning Solutions: Boston, MA, 2010; Vol. 2. (14) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. J. Chem. Educ. 2012 , 89 , 1351 1357. (15) MasteringChemistry http://maste ringchemistry.com/site/index.html (accessed Sep 1, 2013). (16) Bodner, G.; Klobuchar, M.; Geelan, D. J. Chem. Educ. 78 , 1107. (17) National Research Council. A framework for K - 12 science education: Practices, crosscutting concepts, and core ideas ; Nati onal Academies Press: Washington, DC, 2012. 230 (18) Paas, F.; Renkl, A.; Sweller, J. Educ. Psychol. 2003 , 38 (1), 1 4. (19) Sweller, J. Learn. Instr. 1994 , 4 , 295 312. (20) Haudek, K. C.; Prevost, L. B.; Moscarella, R. A.; Merrill, J.; Urban - Lurain, M. CBE - Life Sci. Educ. 2012 , 11 , 283 293. (21) Nehm, R. H.; Ha, M.; Mayfield, E. J. Sci. Educ. Technol. 2012 , 21 (1), 183 196. (22) AACR Questions http://create4stem.msu.edu/project/aacr/questions (accessed Jun 25, 2015). 231 APPENDICES 232 Appendi x A: Copy of permissions from the Journal of Research in Science Teaching http://onlinelibrary.wiley.com/doi/10.1002/tea.21093/full 233 234 235 236 237 238 239 Appendix B: Full structure - property interview protocol Here we provide the full interview protocol: Part 1: Structure and Properties of water, ammonia, and eth ane 1. If you had an unknown compound in lab, what kinds of tests might you run to figure out what it is? 2. What kind of properties does water have? 3. Does the molecular - level structure affect these properties (mentioned in #2)? 4. What types of representations are used to show the molecular - level structure? 5. Which do you use? 6. Can you draw the structure of water and explain how you would use it to determine the properties you have talked about? 7. What is the shape of the water molecule? Does this affect the properties? *Repeat questions #2 - 7 for ammonia and ethane Part 2: Comparing properties 1. Compare CH 3 CH 2 OH and CH 3 CH 3 a. Do they have different boiling points? If so, which is higher? b. Why 2. Compare CH 3 CH 2 OH and CH 3 OH a. Do they have different boiling points? If so, which is hig her? b. Why 3. Compare CH 3 CH 2 OH and CH 3 OCH 3 a. Do they have different boiling points? If so, which is higher? b. Why Part 3: Deducing properties of an unfamiliar structure *The interviewer provides the interviewee with the Lewis structure of acetamide: 1. What properti es do you think this substance would possess based on the structure? 2. Why? 240 Appendix C: Additional examples of structure - property interview themes Here we provide additional examples of each overarching theme Inappropriate Models of Phases or Phase Change : Like Joe and Jill difficulties with depicting ethane as a solid, Joy (OC2) struggled to visualize ethane in liquid form. Her first instinct was to bond them together into a structure that resembled butane, drawing a long connecting line between two ethan e molecules. After further thought, however, she stated that she could not depict the interaction on paper: Interviewer : So two ethane molecules would interact to form butane? Joy Jane (OC2) also struggled with her understanding of phase and phase changes. Upon first impression, it appeared that Jane had a coherent understanding of the phase change fro m liquid to gas: Jane uhh I mean water molecule become, becomes gas phase. Interviewer happens when water mel ts? So from solid to liquid? Jane very, very close to each other. And umm when they transfer to liquid, the distance the molecular, the intermolecular uhh force should still exist. It only became apparent after further questioning, however, that for Jane intermolecular forces were only present in the liquid phase. Jane : I think the intermolecular force is talking about, is talking only in liquid phase. Interviewer : How is ice structured then? Like what holds it together? Jane : Probably the, the, I mean the, since the temperature is very low, umm the stable at where they are. Her assertion that in solids there are no intermolecular forces, seemed to imply that at lower them there (except perhaps for gravit y). This could be another manifestation of a p - prim invoking a macroscopic phenomenon at the molecular level. Representational difficulties: presented similar issues wit h translating between 2 - dimensional structures on paper and their actual 3 - dimensional shape, particularly with the interaction of water molecules. negative positive, negative po 241 the negative is right up there with the po sitive. Rather than the 3 - D structure of ice that would result from the tetrahedral shape of the electron pairs in water, she envisages a linear chain of water molecules in the solid state (ice). Language and Terminology issues: In the paper, we provided intermolecular, and hydrogen bonding. Marshall (OC2) also had terminology issues with the term hydrogen bonding. He, like Ted, believed that hydrogen bonding was a bond within the molecule, which created problems when he started to talk about its strength in comparison to ionic and covalent bonding: Marshall either an oxygen atom, a nitrogen atom, or a fluorine atom. Interview er : Is it a real one or is it just like, like ok so you have your different strength of bonds right? Marshall : Yeah. Interviewer : So you have ionic you were talking about and covalent. Marshall : Yeah. Interviewer : Umm so where would like a hydrogen bond fi t? Marshall : Umm I would say in between ionic and covalent. After further questioning it became apparent that he had the definitions of ionic and covalent two nonm forces can exacerbate misunderstandings if not properly incorporated. Other st of molecules, sometimes providing a reaction when asked if the molecules interacted. In this example, Joy (OC2) was asked if two molecules of ethanol would interact. She re sponded, intent of the interviewer was that she would di scuss intermolecular forces between the two molecules using terminology commonly found in chemistry courses. Subsequently, Joy When I think react I just I think Use of Heuristics in Student Reasoning: Heuristics Instructionally derived: Instructionally derived heuristics hav e been devised to provide students with tools that lower the cognitive load of the task and allow them to make students in constructing appropriate Lewis structures for simple compounds; however, what students often do not understand is that it only works for predicting four (C, N, O, F) out of the current 112 elements. Daisy (OC2) is one example of a student who invoked the octet rule for ethane to argue that octets - well all 242 didaskalogenic problems if it becomes the only reasoning to explain solubility. This heuristic only serves as a rule and fails to provide the explanation as to why the solute is soluble in a given differences for ethane in water to predict its insolubility, Noah (GC2) uses it more broadly to explain the interaction of organic and inorganic c usually water - - soluble...Because Noah proceeds to correct himself and discuss instead the at traction between molecules. This is an example of how easy it is for students to spontaneously use heuristics without considering their underlying conceptual basis. If care is not taken to explain the origins of these heuristics, students can even misapply them to inappropriate situations. Heuristics Marshall (OC1), similar to Robin (OC1) in the paper, uses this reasoning when comparing ethanol and methanol. He argues that ethanol is a heavier molecule, which results in a higher boilin guess it w ould be in order to move from a liquid phase to a gaseous phase it would need more 243 Appendix D: Table A.1: Written descriptions and drawn representations of IMFs from our interviews Student pseudo - nym IMFs initially identified IMF Description IMF Representation Jamie Hydrogen bonding you get an interaction between Dipole - dipole Basically it has positive and have a weak bond sitting right here Van der Waals Electrons in the orbital they move around so have more electronegative molecule, the electrons will side Margery Hydrogen bonding etty sure. All of experience hydrogen Initially within molecules: Later modified to between: Dipole - dipole going to have a partially negative and a partially positive charge, Van der Waals experiences Van der Waals forces. momentary dipoles, like everything or another experiences a partial 244 Rob ert Hydrogen bonding are in hydrogen bonding, so I guess any sort of electronegative atom with Within and between molecules: Dipole moment Any bond has, between two atoms is going to have some sort Van der Waals Could not describe N/A Caitlyn None Did not remember any IMFs, even after prompting N/A John Dipole - dipole Could not describe N/A Van der Waals attraction between electrons holding it together. *Hydrogen bonding after prompting electronegative molecule is attracted to the hydrogen, I mean the hydrogens can like 245 Appendix E: Copy of the initial Intermolecular Forces A Figure A.1: 246 247 Appendix F: - dipole interactions an d London dispersion forces (a) (b) (c) (d) (e) Figure A.2: Student representations of dipole - dipole that received (a) between code, (b) within code, (c) ambiguous code, (d) not present code, and (e) student DK code 248 (a) (b) (c) (d) (e) (f) Figure A.3: Student representations of LDFs that received (a) between code, (b) within code, (c) ambiguous code, (d) not present code, (e) student DK code, and (f) alway s present code 249 Appendix G: dissertations 250 Appendix H: C Standard Usage Agreement Th is ACS article is provided to You under the terms of this Standard Choice - chartered nonprofit located at 1155 16th Street NW, Washington DC 20036. Your access and use of this ACS article means that you have accepted and agreed to the Terms and Conditions of this 1. SCOPE OF GRANT ACS grants You non - exclusive and nontransferable permission to access and use this ACS article subject to the terms and conditions set forth in this Agreement. 2. PERMITTED USES a. For non - commercial research and education purposes only, You may access, download, copy, display and redis tribute articles as well as adapt, translate, text and data mine content contained in articles, subject to the following conditions: i. The authors' moral right to the integrity of their work under the Berne Convention (Article 6bis) is not compromised. ii. Where content in the article is identified as belonging to a third party, it is your responsibility to ensure that any reuse complies with copyright policies of the owner. iii. Copyright notices or the display of unique Digital Object Identifiers ( logos, bibliographic (e.g. authors, journal, article title, volume, issue, page numbers) or other references to ACS journal titles, web links, and any other journal - that are included in the article or t hat are provided by the ACS with instructions that such should accompany its display, should not be removed or tampered with in any way. The display of ACS AuthorChoice or articles on non - ACS websites must be accompanied by prominently displayed links to the definitive published versions of those articles on the ACS website. iv. Any adaptations for non - commercial purposes must prominently link to the definitive published version on the ACS website and prominently display the statement: unofficial adaptation of an article that appeared in an ACS publication. ACS has not endorsed the v. Any translations for non - commercial purposes, for which a prior translation agreement with ACS has not been established, must prominently link to the definitive published version on the article that appeared in an ACS publication. ACS has not endors ed the content of this translation b. Each time You distribute this ACS article or an adaptation, ACS offers to the recipient a 251 license to this ACS article on the same terms and conditions as the license granted to You under t his License. c. For permission to use ACS copyrighted articles beyond that permitted here, visit: http://pubs.acs.org/copyright/permissions.html 3. PROHIBITED USES a. Use of this ACS article for commercial purposes is prohibited. Examples of such prohibited commercial purposes include but are not limited to: i. Copying or downloading of articles, or linking to such postings, for further distribution, sale or licensing, for a fee; ii. Copy ing, downloading or posting by a site or service that incorporates advertising with such content; iii. The inclusion or incorporation of article content in other works or services (other than normal quotations with an appropriate citation) that is then a vailable for sale or licensing, for a fee; iv. Use of articles or article content (other than normal quotations with appropriate citation) by a for - profit organizations for promotional purposes, whether for a fee or otherwise; v. Sale of translated ver sions of the article that have not been authorized by license or other permission from the ACS 4. TERMINATION ACS reserves the right to limit, suspend, or terminate your access to and use of the ACS Publications Division website and/or all ACS articles im mediately upon detecting a breach of this License. 5. COPYRIGHTS; OTHER INTELLECTUAL PROPERTY RIGHTS Except as otherwise specifically noted, ACS is the owner of all right, title and interest in the content of this ACS article, including, without limitatio ns, graphs, charts, tables illustrations, and copyrightable supporting information. This ACS article is protected under the Copyright Laws of the United States Codified in Title 17 of the U.S. Code and subject to the Universal Copyright Convention and the Berne Copyright Convention. You agree not to remove or obscure copyright notices. You acknowledge that You have no claim to ownership of any part of this ACS article or other proprietary information accessed under this Agreement. products are trademarks of ACS. 6. DISCLAIMER OF WARRANTIES; LIMITATION OF LIABILITY ACS warrants that it is entitled to grant this Agreement. EXCEPT AS SET FORTH IN THE PRECEDING SENTENCE, A CS MAKES NO WARRANTY OR REPRESENTATION OF ANY KIND, EXPRESS OR IMPLIED, WITH RESPECT TO THIS ACS ARTICLE INCLUDING, BUT NOT LIMITED TO WARRANTIES AS TO THE ACCURACY OR COMPLETENESS OF THE ACS ARTICLE, ITS QUALITY, 252 ORIGINALITY, SUITABILITY, SEARCHABILITY, O PERATION, PERFORMANCE, COMPLIANCE WITH ANY COMPUTATIONAL PROCESS, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. ACS SHALL NOT BE LIABLE FOR: EXEMPLARY, SPECIAL, INDIRECT, INCIDENTAL, CONSEQUENTIAL OR OTHER DAMAGES ARISING OUT OF OR IN CONNECTION WIT H THE AGREEMENT GRANTED HEREUNDER, THE USE OR INABILITY TO USE TERMINATION OF THIS AGREEMENT BY ACS OR THE LOSS OF DATA, BUSINESS OR GOODWILL EVEN IF ACS IS ADVISED OR AWARE OF THE POSSIBILITY OF SUC H DAMAGES. IN NO EVENT SHALL THE TOTAL AGGREGATE LIABILITY OF ACS OUT OF ANY BREACH OR TERMINATION OF THIS AGREEMENT EXCEED THE TOTAL AMOUNT PAID BY YOU TO ACS FOR ACCESS TO THIS ACS ARTICLE FOR THE CURRENT YEAR IN WHICH SUCH CLAIM, LOSS OR DAMAGE OCCURRED , WHETHER IN CONTRACT, TORT OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, DUE TO NEGLIGENCE. The foregoing limitations and exclusions of certain damages shall apply regardless of the success or effectiveness of other remedies. No claim may be made against A CS unless suit is filed within one (1) year after the event giving rise to the claim. 7. GENERAL This Agreement sets forth the entire understanding of the Parties. The validity, construction and performance of this Agreement shall be governed by and const rued in accordance with the laws of the District of Columbia, USA without reference to its conflicts of laws principles. You acknowledge that the delivery of the ACS article will occur in the District of Columbia, USA. You shall pay any taxes lawfully due from it, other than taxes on ACS's net income, arising out of your use of this ACS article and/or other rights granted under this Agreement. You may not assign or transfer its rights under this Agreement without the express written consent of ACS. 8. ACCE PTANCE You warrant that You have read, understand, and accept the terms and conditions of this Agreement. ACS reserves the right to modify this Agreement at any time by posting the modified terms and conditions on the ACS Publications Web site. Any use of this ACS article after such posting shall constitute acceptance of the terms and conditions as modified. 253 Appendix I: Demographic and pre - instruction assessment data for Cohorts 1 and 2 Here we provide demographic and pre - instruction assessment informat ion for Cohorts 1 and 2. Data shown below includes sex, common majors, SAT composite scores, Metacognitive Activities Inventory (MCAI) scores 1 scores 2 , and Implicit Information from Lewis Structures Inst rument (IILSI) scores 3 . These (GC1). The MCAI, which contains 27 items using a 5 - point Likert scale, was designed to assess n problem - solving skills in the context of chemistry. The SUMS instrument consists of 27 items, each with a 5 - point Likert - type scale ranging from ng of models and use of models through five different scales: Models as multiple representations (MR), Models as exact replicas (ER), Models as explanatory tools (ET), Uses of scientific models (USM), and The changing nature of models (CNM). The IILSI is a one - question instrument, requiring students to select all items that apply out of 17 possible items. structure of a molecule. Table A.2: Sex and most common ma jors for Cohort 1 and Cohort 2 Demographics Cohort 1 (N=94) Cohort 2 (N=160) Sex 67% Female, 33% Male 73% Female, 27% Male Majors 30% Biological Sciences, 18% General Engineering 32% Biological Sciences, 16% Animal and Veterinary Sciences Table A.3: Pre - Instruction scores on SAT composite, MCAI, and SUMS for Cohorts 1 and 2 Pre - Instruction Assessments Cohort 1 (N=94) Mean Cohort 2 (N=160) Mean SAT Composite (out of 1600) 1248 1227 MCAI (out of 100) 78.5 76.6 ET SUMS 4.32 N/A ER SUMS 4.32 N/A USM SUMS 3.78 N/A CNM SUMS 3.43 N/A MR SUMS 4.32 N/A 254 Tab le A.4: Pre - Instruction performance on all 17 IILSI items for Cohorts 1 and 2 Pre - Instruction IILSI Cohort 1 (N=94) Mean Cohort 2 (N=160) Mean No information 1% 2% Element(s) present 80 % 79% Number of valence electrons 86% 83% Number of bonds between particular atoms 89% 71% Type of bond(s) 93% 79% Formal charges 65% 22% Bond angle 49% 36% Geometry/shape 58% 53% Potential for resonance 48% 8% Hybridization 25% 17% Polarity 58% 5 1% Intermolecular forces 30% 14% Acidity/basicity 13% 18% Reactivity 24% 28% Relative boiling points 10% 8% Relative melting points 8% 6% Physical properties 17% 22% 255 Appendix J: A comparison of all drawing and text code frequencies for Cohorts 1 a nd 2 several major code categories: within, between, ambiguous, within and between, student DK and as shown in Figure 2 for Cohorts 1 and 2. Figu re A.4: Comparison of all drawing code frequencies for Cohort 1 and 2 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Hydrogen bonding Dipole-dipole LDFs Within Between Ambiguous Not Present Within and Between Always Present Student DK Comparing all drawing codes for Cohorts 1 and 2 Cohort 1 (N=94) Cohort 2 (N=160) 256 Appendix K: Demographic and pre - instruction assessment data for CLUE and traditional students in all three cohorts as well as statistical comparisons for Cohorts 1 and 2 Here we provi de basic demographic information, like sex and common majors, as well as pre - instruction assessment scores for all three cohorts included in our study. For Cohort 1, pre - instruction measures included SAT composite scores, Metacognitive Activities Inventory (MCAI) scores 1 2 , and Implicit Information from Lewis Structures Instrument (IILSI) scores 3 . For Cohort 2, we provide SAT composite scores, MCAI scores, and IILSI scores. We do not have pre - instru ction assessment data, aside from ACT composite scores, from Cohort 3 at Michigan State University (MSU) and therefore do not include chi - square analysis for significant differences between the CLUE and traditional group. All pre - instruction assessments we re administered for each cohort early in their first fall semester for the purpose of collecting baseline data. The MCAI is designed to explore what students think about their problem - solving skills in their chemistry courses and consists of 27 items on a 5 - point Likert scale. The SUMS 5 - point Likert scale. These items on the SUMS cover five different ideas: models as multiple representations (MR), models as exa ct replicas (ER), models as explanatory tools (ET), uses of scientific models (USM), and the changing nature of models (CNM). Lastly, the IILSI was exploring th structure. The IILSI is a single question assessment with 17 possible item choices, requiring students to select all that may apply. For both Cohorts 1 and 2, we used pre - instruc tion assessments to determine if the CLUE and traditional groups were similar within each cohort using a chi - square analysis. We have included group means as well as p - values when applicable. Any p - values showing a significant difference have been bolded. It should be noted that the IILSI for Cohort 1 students was administered after they received instruction on the topic of Lewis structures, which is evident by the student responses. These students had not, however, been instructed on the true purpose for t hese structures (i.e. predicting chemical and physical properties). When considering the items that both groups had not been instructed on for Cohort 1, we believe that in fact the students within this cohort are equivalent for this measure. 257 Cohort 1: Clem son University , Fall 2011 Spring 2013 Table A.5: Sex and most common majors for Cohort 1 traditional and CLUE groups Demographics Traditional CLUE p - value Sex 67% Female, 33% Male 59% Female, 41% Male 0.310 Majors 30% Biological Sciences, 18% General Engineering 47% Biological Sciences, 10% General Engineering N/A Table A.6: Pre - instruction scores on SAT composite, MCAI, and SUMS for Cohort 1 traditional and CLUE groups Pre - Instruction Assessments Traditional Mean CLUE Mean p - value SAT Composite (o ut of 1600) 1248 1260 0.236 MCAI (out of 100) 78.5 78.4 0.806 ET SUMS 4.32 4.41 0.198 ER SUMS 4.32 4.41 0.198 USM SUMS 3.78 3.71 0.454 CNM SUMS 3.43 3.56 0.371 MR SUMS 4.32 4.43 0.122 Table A.7: Pre - Instruction performance on all 17 IILSI items for Cohort 1 traditional and CLUE groups Pre - Instruction IILSI Traditional Mean CLUE Mean p - value No information 1% 0% 1.000 Element(s) present 80% 85% 0.526 Number of valence electrons 86% 63% 0.003 Number of bonds between particular atoms 89% 78% 0.160 Type of bond(s) 93% 67% < .001 Formal charges 65% 14% < .001 Bond angle 49% 44% 0.669 Geometry/shape 58% 49% 0.356 Potential for resonance 48% 9% < .001 Hybridization 25% 21% 0.725 Polarity 58% 71% 0.128 Intermolecular forces 30% 27% 0.8 94 Acidity/basicity 13% 13% 1.000 Reactivity 24% 22% 1.000 Relative boiling points 10% 4% 0.337 Relative melting points 8% 4% 0.505 Physical properties 17% 20% 0.797 258 Cohort 2: Clemson University, Fall 2012 Spring 2013 Table A.8: Sex and most com mon majors for Cohort 2 traditional and CLUE groups Demographics Traditional CLUE p - value Sex 73% Female, 27% Male 64% Female, 36% Male 0.174 Majors 32 % Biological Sciences, 16% Animal and Veterinary Sciences 74% Biological Sciences, 13% Microbiology N /A Table A.9: Pre - instruction scores on SAT composite and MCAI scores for Cohort 2 traditional and CLUE groups Pre - Instruction Assessments Traditional Mean CLUE Mean p - value SAT Composite (out of 1600) 1227 1243 0.264 MCAI (out of 100) 76.6 76.2 0.567 Table A.10: Pre - instruction performance on all 17 IILSI items for Cohort 2 traditional and CLUE groups Pre - Instruction IILSI Traditional Mean CLUE Mean p - value No information 2% 1% 0.851 Element(s) present 79% 78% 1.000 Number of valence electrons 83% 76% 0.295 Number of bonds between particular atoms 71% 64% 0.279 Type of bond(s) 79% 73% 0.383 Formal charges 22% 22% 1.000 Bond angle 36% 27% 0.182 Geometry/shape 53% 42% 0.118 Potential for resonance 8% 4% 0.106 Hybridization 17% 17% 1.000 Polar ity 51% 47% 0.656 Intermolecular forces 14% 22% 0.146 Acidity/basicity 18% 28% 0.436 Reactivity 28% 23% 0.473 Relative boiling points 8% 10% 0.760 Relative melting points 6% 9% 0.493 Physical properties 22% 20% 0.815 259 Cohort 3: Michigan State Univ ersity, Fall 2013 Spring 2014 Table A.11: Sex, most common majors, and ACT composite scores for Cohort 3 traditional and CLUE groups Demographics Traditional CLUE Sex 49% Female, 51% Male 55% Female, 45% Male Majors 17% Human Biology, 12% Kinesiology 2 7% Human Biology, 24% Pre - medical ACT Composite (out of 35) 26 26 260 Appendix L: Chi - square analyses of all drawing code frequencies for CLUE and traditional students in Cohorts 1 and 2 In Table A.12 , we have included means, p - values, and effect sizes wh en appropriate for all drawing c . Significant p - values have been bolded . Table A.12: Chi - square statistical analysis results for comparing all drawing code frequencies for CLUE and Traditional students at the end of GC2 i n Cohort 1 and Cohort 2 Cohort IMF Code Traditional p ercentage CLUE p ercentage p - value Cohort 1 Hydrogen bonding Within 72.3 10.3 < .001 0.62 Between 14.9 82.8 < .001 0.67 Not Present 4.3 0.0 0.15 Ambiguous 3.2 4.6 0.92 Within and Between 5.3 2.3 0.51 Dipole - dipole Within 60.6 13.8 < .001 0.47 Between 10.6 63.2 < .001 0.54 Not Present 14.9 9.2 0.35 Ambiguous 12.8 12.6 1.0 Within and Between 1.1 1.2 1.0 LDFs Within 55.3 14.9 < .001 0.41 Between 11.7 62.1 < .001 0.51 Not Present 6.4 1.2 0.15 Ambiguous 12.8 14.9 0.84 Within and Between 0 2.3 0.4 4 Always Present 11.7 4.6 0.14 Student DK 2.1 0.0 0.51 Cohort 2 Hydrogen bonding Within 56.2 10.2 < .001 0.46 Between 31.3 83.8 < .001 0.51 Not Present 1.9 0.0 0.37 Ambiguous 7.5 3.4 0.24 Within and Between 3.1 2.6 1.0 Dipole - dip ole Within 58.1 16.2 < .001 0.41 Between 15.6 71.8 < .001 0.56 Not Present 13.8 6.0 0.06 Ambiguous 3.7 6.0 0.56 Within and Between 1.9 0.0 0.37 Student DK 6.9 0.0 0.006 0.16 LDFs Within 55.6 12.8 < .001 0.43 Between 20.0 68.4 < .001 0.48 Not Present 9.4 4.3 0.17 Ambiguous 4.4 9.4 0.15 Within and Between 5.0 3.4 0.73 Always Present 5.0 0.9 0.11 Student DK 0.6 0.9 1.0 261 Appendix M: While comparisons can pro vide a snapshot of the differences for a given IMF, we also diagrams provide a visual aid that allows the reader to quickly see where most students travel between co des. In Table A.13 , we have provided the specific percentages of students who consistently received a particular code for all three of their IMFs representations. These values are only included for students in Cohort 1 since Sankey diagrams were generated for that particular cohort. Table A.13: Consistency of students receiving a particular drawing code across all three IMFs Consistency of responses Traditional (N=94) CLUE (N=87) Within 38% 6% Between 1% 46% Ambiguous 2% 0% Within and Between 0% 1% In consistent 59% 47% 262 Appendix N: In our chapter , we compared the traditional students among all three cohorts. While we could not make any statistical comparisons due to a lack of pre - instruction assessments for Cohort 3, we were able to see some general differences betwe en the universities. In Figure A.5 , Figure A.5: Compariso cohorts 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Ambiguous Not Present Student DK Ambiguous Not Present Student DK Ambiguous Not Present Student DK Hydrogen bonding Dipole-dipole LDFs Drawing code frequencies for "other" categories across all three cohorts Trad Univ. 1, Cohort 1 (N=94) Trad Univ. 1, Cohort 2 (N=160) Trad Univ. 2, Cohort 3 (N=239) 263 REFERENCES 264 REFERENCES (1) Cooper, M. M.; Sandi - Urena, S. J. Chem. Educ. 2009 , 86 , 240 - 245. (2) Treagust, D. F.; Chittleborough, G.; Mamiala, T. L. Int. J. Sci. E duc. 2002 , 24 , 357 - 368. (3) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Chem. Educ. Res. Pr. 2012 , 13 , 195 - 200.