CASE STUDIES OF UNDERGRADUATE STUDENT INTERACTIONS WITH AN ONLINE COMPUTER ADAPTIVE INSTRUCTION INTERMEDIATE ALGEBRA COURSE By Jennifer L. Nimtz A DISSERTATION Michigan State University in partial fulfillment of the requirements Submitted to for the degree of Mathematics Education—Doctor of Philosophy 2018 ABSTRACT CASE STUDIES OF UNDERGRADUATE STUDENT INTERACTIONS WITH AN ONLINE COMPUTER ADAPTIVE INSTRUCTION INTERMEDIATE ALGEBRA COURSE By Jennifer L. Nimtz Remedial/developmental and introductory university mathematics courses have a long history of high attrition rates. Recently, university administration and mathematics departments have been considering technological solutions, and one such solution is computer-adaptive- instruction (CAI). In fact, CAI has been touted as a “silver bullet” to the dilemma of undergraduate mathematics attrition and failure rates (Twigg, 2011), yet little research has documented the nature of student engagement in these courses and what they actually learn. Although the use of CAI in college introductory mathematics has been increasing, research about student engagement in CAI mathematics is scarce. The goal of this dissertation was to illustrate and understand that nature of student engagement in an online CAI intermediate algebra course. Drawing on qualitative case-study methods, I investigated the overarching question: What is the nature of student engagement in an online intermediate CAI intermediate algebra course? Specifically, I investigated the nature of students’ cognitive, academic, and affective interactions. The primary data collection method included the combined use of screencast and pen-cast video technology to produce weekly think-aloud recordings. These recordings were independently conducted by each student as they worked on assignments in an online CAI intermediate algebra course. Secondary sources of data included responses to pre-and post- questionnaires followed by interviews. After processing and transcribing the data for each case, a comparative analysis and relational analysis across the three cases were conducted and described in the results section. This dissertation study presents an original framework with which to analyze the nature of an individual’s mathematical work. The foundation of this framework was synthesized from seminal work, such as Polya (1985) and Schoenfeld (1985), concerning the solving of non- routine mathematics tasks. I posited the three phases of this new framework (orientation, generation, and conclusion) would be applicable to any type of mathematical task, even routine exercises common in CAI, but that the activities within each of the three phases would differ. The results of this study suggest this is true, yet further research is required. I also examined the cognitive demand of the mathematics tasks presented in the CAI environment, and all 57 of the tasks recorded in the data were low cognitive demand. Because of this finding regarding low cognitive demand, it was surprising to discover that for a few of the CAI tasks, students engaged with the mathematics at a deeper level than expected. Again, further research is warranted to determine what may be contributing factors for these deeper interactions in a CAI environment even when the tasks are low cognitive demand. Implications of this research suggest cautions, actions, and future research for various groups within the undergraduate mathematics education community: mathematics department chairs, course instructors, mathematics education researchers, and curriculum and CAI developers. Copyright by JENNIFER L. NIMTZ 2018 I dedicate this dissertation to my spouse, Miriam Danu, my steadfast partner in this life journey—sharing in the difficult to delightful with integrity, passion and strength. v ACKNOWLEDGEMENTS This dissertation and my broader doctoral experience would not have been possible without the inspiration and support of faculty, administrative staff, doctoral student colleagues, friends, and family. I have been blessed with many people who have contributed to this journey, so many that it is not possible to name all of them individually. Here I will attempt to express the depth of my gratitude to those who have been instrumental in my development as a researcher. First, I am grateful to the students who participated in this study. Their dedication and follow through for the duration of the semester, combined with their willingness to openly share their mathematical experiences, thinking and study habits, made this work possible. Young people like them continue to inspire me and give me hope. I am thankful to my dissertation advisers, Ralph Putnam and Sandra Crespo. I am immensely grateful for their advice, feedback, encouragement, and flexibility with their time. I am also grateful for those professors who served on my committee, Kristen Bieda and John (Jack) P. Smith III, whose balance of complements and critical feedback boosted my confidence as an emerging scholar and have pushed me to communicate precisely and think in new ways. I want to express my appreciation for financial support from the Michigan State University Graduate School and the College of Natural Science. The Graduate School’s Research Enhancement Award funded the purchase of the Live Scribe pens and paper utilized in this study. In addition, the College of Natural Science provided a Dissertation Completion Fellowship, which allowed me to focus my time and energy on writing. I am grateful to all of the many faculty members, graduate students, and staff with whom I have had the pleasure to work and learn from these past years. I was fortunate to have worked vi on the Connected Mathematics Project (CMP) for several years with Betty Phillips, Glenda Lappan, Yvonne Grant, Jaqueline Stewart, and A.J. Edson. I especially would like to thank Betty Phillips and Yvonne Grant for their feedback on presentations related to this work, Amy Lucas for sharing her office, Billie Lozen for providing space to conduct interviews, and Judith Miller for her open door and friendship. In addition, I would like to express my gratitude for the opportunities to have worked in CREATE for STEM Institute with Raven McCrory and Kristen Bieda. Raven was my advisor in my first years at MSU, and I appreciated her guidance and missed her when she retired. I have been very fortunate to have worked with Kristen Bieda on multiple research projects and learned a great deal from her mentoring. In the Mathematics Department, I am grateful for the opportunity to have worked with Jane Zimmerman, whose vision and passion about college algebra curriculum, assessment, teaching and learning is inspiring. I also am appreciative of the opportunity to have learned more about Quantitative Literacy curriculum through work with Vince Melfi, Bronlyn Wassink and Rachael Lund. I also want to express appreciation for my family and friends. I am thankful for my parents, Gloria and Michael Nimtz, who have provided steady, strong, supportive love. In addition, I am grateful for my grandmother, Elizabeth Nimtz, who had always encouraged me to continue with my education. I also wish to express my gratitude to the many friends whose care and love have helped to sustain Miriam and I both as individuals and as a couple. Finally, I wish to express deep appreciation and love for my spouse, Miriam Danu. We have participated in this academic marathon together, side by side, experiencing both devastations and elations along the way. Miriam’s encouragement, support, and love has been and continues to be unwavering. I am forever grateful that we share this life journey. vii TABLE OF CONTENTS LIST OF TABLES ........................................................................................................................ xii LIST OF FIGURES ..................................................................................................................... xiv KEY TO ABBREVIATIONS ...................................................................................................... xvii CHAPTER 1 ....................................................................................................................................1 INTRODUCTION ...........................................................................................................................1 Research Questions ................................................................................................................... 4 Overview of Research Methodology and Goals ....................................................................... 5 Study Rational and Significance ............................................................................................... 6 Overview of the Forthcoming Chapters .................................................................................... 7 CHAPTER 2 ....................................................................................................................................8 BACKGROUND LITERATURE ....................................................................................................8 Why Non-Credit-Bearing Mathematics Courses? .................................................................... 8 Enrollment in NCBMC and Online Courses .......................................................................... 11 Interactions in NCBMC Teaching and Learning .................................................................... 12 NCBMC that use Web-Based CAI Technology ...................................................................... 13 Programmed Instruction and Assessment of LEarning in Knowledge Spaces ....................... 15 Programmed Instruction ...............................................................................................16 ALEKS and Knowledge States ....................................................................................18 Defining a Knowledge State ........................................................................................20 How ALEKS Works .....................................................................................................21 Comparing Programmed Instruction and ALEKS .......................................................21 Students’ Mathematical Thinking in NCBMC ........................................................................ 23 Student Experiences, Identity and Mathematical Socialization in NCBMC .......................... 25 Background Literature Summary ............................................................................................ 27 CHAPTER 3 ..................................................................................................................................29 THEORETICAL ORIENTATION AND FRAMING ....................................................................29 Overview of Student Engagement Research ........................................................................... 29 A Quantitative Approach to Student Engagement .......................................................29 Widening the Lens on Student Engagement ................................................................32 Theoretical Orientation: The Sociocultural Nature of Student Engagement .......................... 34 Student Engagement ....................................................................................................36 Structural Influences ....................................................................................................37 Psychosocial Influences ...............................................................................................38 Proximal and Distal Consequences ..............................................................................38 Theoretical Orientation Summary: Revisiting the Background Literature ..................39 viii Conceptual Framework: Student Engagement in a CAI NCBMC ......................................... 43 Cognitive Interactions ..................................................................................................45 Academic Interactions .................................................................................................52 Affective Interactions ...................................................................................................53 Social Interactions ........................................................................................................54 Summary: Conceptual Framework of SE and Revisiting Research Questions ...........54 CHAPTER 4 ..................................................................................................................................58 RESEARCH METHOD ................................................................................................................58 Researcher Positioning and Assumptions ............................................................................... 59 Research Context .................................................................................................................... 60 Brief History of NCBMC at MSU ...............................................................................60 Intermediate Algebra Course Context for this Study ...................................................62 Continued Efforts to Improve Introductory Mathematics ...........................................63 Research Participants .............................................................................................................. 64 Data Collection ....................................................................................................................... 65 Data Analysis .......................................................................................................................... 70 Individual Case Analysis ..............................................................................................70 Cross-Case Analysis.....................................................................................................74 Trustworthiness and Validity .................................................................................................. 74 Strengths and Limitations ....................................................................................................... 75 Analysis Summary .................................................................................................................. 75 CHAPTER 5 ..................................................................................................................................77 Jade “Science and math overlap …” ..............................................................................................77 Jade’s Academic Interactions .................................................................................................. 79 Jade’s Notetaking .........................................................................................................79 Jade’s Memory Strategy ...............................................................................................82 Jade’s Planning .............................................................................................................83 Jade: Academic Interactions Summary ........................................................................83 Jade’s Cognitive Interactions .................................................................................................. 83 Jade: Think Aloud Recording 1 (Figures 12, 13, & 14) ...............................................84 Jade: Think Aloud Recording 4 (Figures 15 & 16) ......................................................86 Jade’s Affective Interactions ................................................................................................... 89 Jade’s Stated Confidence .............................................................................................90 Jade’s Value of Mathematics and Course .....................................................................90 Summarizing Jade’s Engagement ........................................................................................... 91 CHAPTER 6 ..................................................................................................................................93 Chad “When you fail, you pick back up and do it again.” .............................................................93 A Series Unfortunate Events (FS16) ....................................................................................... 94 Chad’s Cognitive Interactions (SS17) ..................................................................................... 97 Chad: Think Aloud Recording 2 (Figures 17-21) ........................................................99 Chad: Think Aloud Recording 6 (Figures 22-25) ......................................................105 Chad’s Affective Interactions (SS17) .................................................................................... 109 ix Chad’s Stated Confidence ..........................................................................................109 Chad’s Value of Mathematics and Course .................................................................109 Chad’s Other Affect Related Statements .................................................................... 111 Chad’s Academic Interactions (SS17) .................................................................................. 112 Summarizing Chad’s Engagement ........................................................................................ 112 CHAPTER 7 ................................................................................................................................ 115 Tia “I like to get things done.” ..................................................................................................... 115 Tia’s Cognitive Interactions: Mixed Messages ..................................................................... 117 Tia’s Think Aloud Recording 3 (Figures 26 & 27): Miscopied ................................. 118 Tia’s Think Aloud Recording 5 (Figures 28 – 30): I did it all wrong! .......................120 Tia’s Think Aloud Recording 7 (Figure 31): Misconception .....................................124 Tia’s Cognitive Interactions Summary ......................................................................125 Tia’s Academic Interactions .................................................................................................. 126 Tia’s Affective Interactions ................................................................................................... 127 Summarizing Tia’s Engagement ........................................................................................... 128 CHAPTER 8 ................................................................................................................................130 CROSS-CASE ANALYSIS .........................................................................................................130 Comparative Analysis ........................................................................................................... 130 Cognitive Interactions ................................................................................................131 Academic Interactions ...............................................................................................135 Affective Interactions .................................................................................................137 Summary of Comparative Analysis ...........................................................................141 Relational Analysis ............................................................................................................... 142 Confidence and Cognitive Interactions ......................................................................142 Research Questions and Findings Summary ......................................................................... 146 CHAPTER 9 ................................................................................................................................148 DISCUSSION—TYING IT ALL TOGETHER ..........................................................................148 Sociocultural Influences ........................................................................................................ 149 Power and Privilege Associated with Mathematical Success ....................................149 Cultural Assumptions About the Nature of Mathematical Activity ...........................151 Antecedents to Student Engagement .................................................................................... 152 Structural Influence: The ALEKS “Bite-Sized Pieces” Curriculum ..........................152 Psychosocial Influence: ALEKS Instruction and Participation Structure ..................153 The Nature of Student Engagement in this Study ................................................................. 154 Consequences of Student Engagement ................................................................................. 155 Affective Consequences: Beliefs about Mathematics ................................................155 Academic Consequences: Learning Goals for Collegiate Mathematics ....................157 Promises and Pitfalls of CAI ................................................................................................. 158 Implications ........................................................................................................................... 160 Implications for Policymakers and Mathematics Departments .................................160 Implications for Curriculum and CAI Development .................................................161 Implications for Mathematics Instructors ..................................................................161 x Implications for Mathematics Education Research ...................................................162 Future Research and Conclusion .......................................................................................... 163 APPENDICES .............................................................................................................................171 Appendix A: Research Participation Consent Form ............................................................. 172 Appendix B: Research Design Summary Table .................................................................... 175 Appendix C: Research Design by Student Engagement Indicator ....................................... 176 Appendix D: Mathematics History Questionnaire ................................................................ 179 Appendix E: End of Course Questionnaire ........................................................................... 186 Appendix F: Interview Protocols .......................................................................................... 188 Math History Questionnaire Follow-Up Potential Questions ....................................188 Observed and Extended Think Aloud Potential Questions ........................................188 End of Course Interview ............................................................................................189 Appendix G: IQA Task Potential .......................................................................................... 190 Appendix H: ALEKS Learning Mode .................................................................................. 191 Appendix I: Think Aloud Recording Data Table Headings .................................................. 193 Appendix J: Sample Pages of Each Participant’s Written Work ........................................... 194 REFERENCES ............................................................................................................................201 xi LIST OF TABLES Table 1. Mathematics instructional models that use internet-based technology ........................... 13 Table 2. Six types of problems in Elementary Algebra (Falmagne et al., 2006, p. 5) .................. 19 Table 3. NSSE themes and engagement indicators ....................................................................... 30 Table 4. Example of NSSE survey questions ................................................................................ 31 Table 5. Problem solving frameworks comparison and synthesis ................................................ 49 Table 6. Problem solving phases and activities when solving non-routine mathematics tasks .... 50 Table 7. Surface and deep learning approaches (adapted from Beattie et al, 1997) ..................... 51 Table 8. Participant demographics ................................................................................................ 65 Table 9. Summary of data collected in the beginning and throughout the course. ....................... 69 Table 10. Summary of data collected at the end of the course. ..................................................... 70 Table 11. Example of ALEKS Topic and Routinized ALEKS Sequence ...................................... 71 Table 12. Example of ALEKS Topic and Critical Incident ALEKS Sequence (Interrupted) ....... 72 Table 13. Patterns of Cognitive Interactions of Activities in Problem Solving Phases. ............... 72 Table 14. Patterns of Cognitive Interactions in Jade's Think Aloud 1 (Figures 12, 13, & 14) ..... 86 Table 15. Patterns of Cognitive Interactions in Jade's Think Aloud 4 (Figures 15 & 16) ............ 89 Table 16. Patterns of Cognitive Interactions in Chad's Think Aloud 2 (Figures 17-21) ............. 104 Table 17. Patterns of Cognitive Interactions in Chad's Think Aloud 6 (Figures 22-25) ............. 108 Table 18. Unproductive and Productive Beliefs about Learning Mathematics .......................... 123 Table 19. Patterns of Cognitive Interactions in Tia's Think Aloud 3, 5, & 7 (Figures 26-31) .... 125 Table 20. Observed Patterns of Cognitive Interactions with ALEKS problems. ........................ 133 Table 21. Quantification of Confidence Statements per ALEKS TOPIC in Critical Incidents .. 139 xii Table 22. Relationship between Confidence and Patterns of Cognitive Interaction ................... 143 Table 23. Unproductive Beliefs about Mathematics and Evidence from this Research ............. 156 Table 24. Summary of Promises and Pitfalls currently apparent in CAI .................................... 158 Table 25. Patterns of Cognitive Interactions: Observed and Thought Experiment ..................... 165 Table 26. Research Questions, Student Engagement Indicators, Definitions, and Evidence ..... 175 Table 27. SE Indicator 1: Cognitive Interactions ........................................................................ 176 Table 28. SE Indicator 2: Academic Interactions ........................................................................ 177 Table 29. SE Indicator 2: Academic Interactions ........................................................................ 177 Table 30. SE Indicator 3: Affective Interactions ......................................................................... 178 Table 31. SE Indicator 3: Affective Interactions ......................................................................... 178 xiii LIST OF FIGURES Figure 1. A Learning Hierarchy for a Task in Elementary Mathematics ...................................... 17 Figure 2. Precedence diagram for problem types described in Table 2 ........................................ 19 Figure 3. The outer fringe and inner fringe of Knowledge State .................................................. 21 Figure 4. Kahu's (2013, p. 766) student engagement framework ................................................. 36 Figure 5. Theoretical framework and background literature (in red) synthesis ........................... 40 Figure 6. Overarching theoretical framework: Sociocultural nature of student engagement ....... 45 Figure 7. Conceptual framework, student engagement indicators ............................................... 55 Figure 8. Overview of data collection phases ............................................................................... 65 Figure 9. Screencast frame ........................................................................................................... 67 Figure 10. Pen-cast frame ............................................................................................................. 68 Figure 11. Overview of the data analysis by SE Indicator Categories ......................................... 71 Figure 12. Jade Think Aloud 1, Example ..................................................................................... 80 Figure 13. Jade Think Aloud 1, Problem 1 ................................................................................... 82 Figure 14. Jade Think Aloud 1, Problem 2 ................................................................................... 85 Figure 15. Jade Think Aloud 4, Example ..................................................................................... 87 Figure 16. Jade Think Aloud 4, Problem 1 ................................................................................... 88 Figure 17. Chad Think Aloud 2, Example .................................................................................... 99 Figure 18. Chad Think Aloud 2, Problem 1 Solution ................................................................. 100 Figure 19. Chad Think Aloud 2, Problem 1 ALEKS Explanation .............................................. 101 Figure 20. Chad Think Aloud 2, Problem 2 ................................................................................ 102 Figure 21. Chad Think Aloud 2, Problem 3 ................................................................................ 103 xiv Figure 22. Chad Think Aloud 6, First Topic, Problem 1 ............................................................ 105 Figure 23. Chad Think Aloud 6, First Topic, Problem 4 ............................................................ 106 Figure 24. Chad Think Aloud 6, Second Topic, ALEKS Example ............................................ 107 Figure 25. Chad Think Aloud 6, Second Topic, Problem 2 ........................................................ 107 Figure 26. Tia Think Aloud 3, Problem 2 ................................................................................... 118 Figure 27. Tia Think Aloud 3, Problem 3 ................................................................................... 119 Figure 28. Tia Think Aloud 5, ALEKS Example ....................................................................... 121 Figure 29. Tia Think Aloud 5, Problem 1 ................................................................................... 122 Figure 30. Tia Think Aloud 5, Problem 1 Explanation............................................................... 123 Figure 31. Tia Think Aloud 7, Problem 3 ................................................................................... 124 Figure 32. Tia Note Taking Example (Think Aloud Recording 3) ............................................. 126 Figure 33. Representation of Relationship: Confidence and Patterns of Cognitive Interaction 144 Figure 34. Conceptual framework, student engagement indicators ........................................... 148 Figure 35. Sociocultural Antecedents, and Consequences of Student Engagement ................... 149 Figure 36. Compare and Evaluate Two Correct Solutions ......................................................... 166 Figure 37. Verifying and Evaluating Two Solutions, One Correct, and One Incorrect .............. 167 Figure 38. Creating and Assembling Examples .......................................................................... 168 Figure 39. ALEKS Learning Mode Reference ........................................................................... 191 Figure 40. ALEKS Learning Mode Reference ........................................................................... 192 Figure 41. Sample Page of Jade's Written Work (Think Aloud 1, p. 1) ...................................... 194 Figure 42. Sample Page of Jade's Written Work (Think Aloud 1, p. 2) ...................................... 195 Figure 43.Sample Page of Chad’s Written Work (Think Alouds 1 & 2, p. 1) ............................ 196 Figure 44. Sample Page of Chad’s Written Work (Think Aloud 2, p. 2) .................................... 197 xv Figure 45. Sample Page of Tia’s Written Work (Think Aloud 3, p. 1) ....................................... 198 Figure 46. Sample Page of Tia’s Written Work (Think Aloud 3, p. 2) ....................................... 199 xvi KEY TO ABBREVIATIONS ALEKS Assessment and LEarning in Knowledge Spaces CAI ECQ FS16 IQA MHQ MLC MSU NCB Computer Adaptive Instruction End of Course Questionnaire Fall Semester 2016 Instructional Quality Assessment Mathematics History Questionnaire Mathematics Learning Center (mathematics department tutoring) Michigan State University Non-Credit Bearing NCBMC Non-Credit Bearing Mathematics Course(s) NCES NSSE OETA SE SS17 UA National Center for Education Statistics National Survey of Student Engagement Observed & Extended Think Aloud Student Engagement Spring Semester 2017 Unit of Analysis xvii CHAPTER 1 INTRODUCTION The challenge that has always faced American education . . . is how to create both the social and cognitive means to enable a diverse citizenry to develop their ability. It is an astounding challenge: the complex and wrenching struggle to actualize the potential not only of the privileged. (Rose, 1989, p. 225) In the above quote, Rose summarized the struggle of well-meaning educators—and a struggle that has existed throughout the 20th and into the 21st Century. Mathematics, algebra in particular, serves as an excellent example of this challenge. For instance, for at least the past 30 years, algebra has been a gatekeeper to college as well as the social and economic status associated with a college degree. More recently, algebra has been deemed a gatekeeper to high school graduation. As Moses and Cobb (2001) aptly summarized: Algebra was assigned a certain role, a certain place in the education system. Students learned how to manipulate abstract symbolic representations for underlying mathematical concepts. Now here comes history, which brings in a technology that places symbolic representations front and center… So, now algebra becomes an enormous barrier…there’s nothing that says it has to be algebra. It could be a mix of a number of things—and some people would argue that it should be…For the time being, it’s going to be algebra. (pp. 13-14) Thus, social and economic status is afforded college degrees with algebra playing a key role in whether a student attains a college degree. To increase the number and diversity of students earning college degrees, “Algebra for All” policies and related efforts to improve K-12 mathematics education have been in place for some time. Although some progress has been made, the hard work of shifting systemic social systems such as the education system has been frustratingly slow, and algebra remains a barrier for many people. Roughly 20% to 30% of entering college freshman throughout the United States place into a developmental/remedial mathematics course that largely consists of high school algebra content (Hill, 2006; Parsad & 1 Lewis, 2003). Despite the fact that the validity of mathematics placement tests have been called into question by recent research, the use of such tests remains a prevalent practice that dictates students’ first undergraduate mathematics course (Drake, 2010; Fain, 2012; Latterell & Regal, 2003). Students enrolled in these developmental/remedial mathematics courses pay tuition, but the courses do not count toward the completion of a degree. Throughout this paper, I will refer to these courses as either intermediate algebra, which is the course specific to this study, or non- credit-bearing mathematics courses (NCBMC). I emphasize the latter to acknowledge the money, time, and effort the students enrolled in these courses put forth to attain a college degree. I will discuss the issues surrounding the language used to define these courses and the stigma associated with that language in Chapter 2. The intent for NCBMC is to prepare students to succeed in college-level mathematics coursework and increase their chances of completing a degree program. However, when compared with students who take credit-bearing courses, students who take NCBMC often do not complete their education within six years, are less likely to take advanced courses, and are more likely to drop out of college altogether (Bahr, 2010b; Bailey, 2009; Bailey, Jeong, & Cho, 2009; Goldrick-Rab, 2010). In addition, university enrollment statistics indicate that NCBMC are disproportionately populated by African American, Latina/o students, as well as poor and working-class students (Bahr, 2010a; Larnell, 2011; Meza, 2015). Thus, the university education system continues to perpetuate the prevalent social and economic inequities for poor people and people of color. Because students who begin their college coursework in NCBMC are less likely to finish their programs of study, university mathematics departments have turned to an assortment of strategies to improve student completion rates, such as curricular strategies, student academic 2 support strategies, and evaluating the effectiveness of placement tests. Curricular strategies include acceleration through NCBMC, modularization of the courses, and alternative mathematics curricular tracks such as contextualized co-requisite courses and quantitative reasoning courses. Student academic support strategies include learning communities, tutoring services, and supplemental instruction. In addition, colleges are examining their assessment and placement practices to accurately place more students into credit bearing courses. (Meza, 2015) Technological advancements are also being used to meet the diverse needs of students who place into NCBMC. For example, community college and university mathematics instructors rely on online homework systems to ensure that students get adequate practice and immediate feedback on their homework assignments. Also, colleges and universities increasingly utilize various methods of delivering mathematics course content online (hybrid, synchronous, and asynchronous) to provide flexibility for student schedules. Most relevant to this study, computer adaptive instruction (CAI) mathematics software (e.g. ALEKS, and My Math Lab), provides technologically mediated individualized learning environments for students. CAI mathematics software generally begins with an assessment of an individual student’s mathematics skills, and then uses a combination of ongoing assessment and the prescription of mathematics exercises that the software algorithm assigns for the student to advance student learning. Current reports about the efficacy of CAI vary from glowing reports to unfavorable reports. On the one hand, Twigg (2011) of the National Center for Academic Transformation, has touted CAI mathematics software and associated instructional models (e.g., the math emporium model) as a “silver bullet for higher education mathematics courses” (p.1). On the other hand, researchers have argued that mathematics CAI software does not foster the mathematical 3 thinking that prepares students to use mathematics to solve problems other than those that are presented in the software (Webel, Krupa, & McManus, 2015). In the midst of this flurry of reforms and related data gathering efforts regarding NCBMC, most studies have examined end of course outcomes, few have documented students’ learning experiences or mathematical thinking. Consequently, there is a dearth of knowledge about student experiences in NCBM courses that has just begun to be addressed by recent research (Larnell, 2011, 2016). Even less research has documented students’ learning experiences in NCBM CAI mathematics environments. Research Questions Because we know little about student experiences in NCBMC in general, and even less about the mathematical interactions that occur between students and CAI mathematics environments, the purpose of this study was to address that gap in the existing research literature. The study examined the nature of students’ engagement with an online intermediate algebra CAI course to describe and understand that phenomena. To examine the nature of students’ engagement, I adapted Finn and Zimmer’s (2012) framework for indicators of student engagement, which include cognitive interactions, academic interactions, affective interactions, and social interactions, as indicated in the subsequent overarching research question and related sub-questions: 1. What is the nature of students’ engagement with an online CAI intermediate algebra course? a. What is the nature of students’ cognitive interactions within an online CAI intermediate algebra course? b. What is the nature of students’ academic interactions within an online CAI intermediate algebra course? 4 c. What is the nature of students’ affective interactions within an online CAI intermediate algebra course? Note that in these research questions I use “the nature of” in its plural sense, not its singular sense. In other words, my assumption is that many patterns of interaction exist, and that multiple descriptions of various patterns of interactions are what comprises the nature of student engagement for this study. Students’ social interactions were not observed in this study because the research context was individual student’s learning interactions in the online course. These phenomena did not necessarily include interactions with others in the students’ quest to learn in this environment. That said, my theoretical research perspective on learning is that learning in any environment, whether it be face-to-face classroom instruction or virtually mediated instruction, occurs within the larger, pervading sociocultural environment. The sociocultural environment includes issues of culture, power, politics, and economics. My sociocultural approach to this case study is clarified in the theoretical and conceptual framework of Chapter 3. Overview of Research Methodology and Goals This research is a multi-case study of three university students as they interact with an online intermediate algebra CAI environment, a NCBMC at Michigan State university. The data gathered included responses to a math history questionnaire, weekly independent screencast recordings conducted by each student as they worked in the online CAI intermediate algebra course (approximately 15 minutes each week), a longer mid-semester recorded session of students working in the online CAI intermediate algebra course that I observed and recorded (approximately 45 minutes for each student), and an end of course questionnaire and interview. The goal of this study is to contribute to the existing research base about students’ experiences in 5 NCBMC, and more specifically, to document students’ learning interactions within an online CAI intermediate algebra course. Study Rational and Significance I conducted this study because online instructional software use is prevalent and is predicted to increase. Yet, I have reservations about how some of the current software on the market has been implemented, as well as how some implementations have been touted as a “silver bullet” for higher education mathematics (Twigg, 2011). If students need to engage in activities such as mathematical problem solving and modeling to be successful in subsequent STEM courses and careers, then it is important that students have opportunities to engage in these practices and develop the associated necessary mathematical habits of mind. This study has the potential to influence the development of undergraduate mathematics education courses that utilize computer adaptive instruction. This research has the potential to influence the design of CAI software such as ALEKS and My Math Lab. It is my hope that the results of this and similar research studies will inform future software design to foster productive mathematical interactions and deeper student thinking. Lastly, as colleges and universities continue to turn to various instructional models that incorporate CAI (See Chapter 2, Table 1, p. 13), the need for independent scholarly research in this area will persist. A wider range of quantitative and qualitative research methods for evaluating these programs will continue to be necessary to provide a more detailed picture of the nature of student experiences and learning in these environments. Furthermore, the area of CAI in mathematics will also continue to provide rich research opportunities with the ultimate goal of improving the mathematical opportunities that students interact with and learn from. 6 Overview of the Forthcoming Chapters In the next chapter, I present background literature on remedial/developmental mathematics in higher education, beginning with an analysis of the terms remedial and developmental and why I have chosen to use non-credit-bearing mathematics courses instead. This is followed by an overview of the related topics relevant to this research, enrollment in these courses, teacher-student interactions, instructional models that utilize various forms of online technology, research about student thinking, and research about student experiences, identity and mathematical socialization. In the first part of Chapter 3, I describe the overarching theoretical framework of this study, the sociocultural aspects of student engagement in higher education, followed by how I connect this framework with the background literature in Chapter 2. In the second part of Chapter 3, I illustrate the more specific conceptual framework utilized in this study, and provide details of student engagement indicators, cognitive interactions, academic interactions, affective interactions, and social interactions. In Chapter 4, I describe the study methods and how the conceptual framework was operationalized. In Chapters 5, 6, and 7, I present three descriptive case studies of the participants—Jade, Chad, and Tia. In Chapter 8, I present a cross-case analysis and findings from the three cases. In Chapter 9, I conclude with further interpretation and synthesis of the research, revisit the strengths and limitations of the study, discuss implications of this research and make recommendations for future research. 7 CHAPTER 2 BACKGROUND LITERATURE This chapter begins with my rationale for using non-credit-bearing mathematics course rather than the more common labels remedial mathematics or developmental mathematics courses. Then, I briefly review the literature about higher education enrollment in general, followed by enrollment in NCBMC and enrollment in online courses to further bolster the rational for this study. Third, because this study examines students learning interactions within a CAI NCBMC, I examine the existing literature regarding teacher-student interactions in NCBMC. In the fourth subsection about NCBMC, I describe the various mathematics instructional models that use web-based CAI technology. In the fifth subsection, I outline some existing literature that examines student thinking in NCBMC. Lastly, I summarize a study that describes student experiences in a NCBMC, and then conclude this section with a summary/synthesis of this background literature. Why Non-Credit-Bearing Mathematics Courses? As stated in Chapter 1, when students transition from high school to college, most are required to take a mathematics placement exam, the results of which imply that 20 to 30 percent of students are not ready for college-level mathematics coursework despite the fact that the most have taken 3 or more years of high school mathematics (Hill, 2006; Parsad & Lewis, 2003). Students who perform poorly on the placement exam typically are required to take courses offered by the college or university, but that are not considered college-level mathematics. Many names have been used to describe these courses—and often (inappropriately) students as well: “developmental, remedial, compensatory, intermediate, college preparatory, refresher, basic skills” (Larnell, 2016, p. 237). The terms, remedial and developmental mathematics courses are 8 most often used in research literature, and sometimes interchangeably, but there are subtle differences between the the philosophy of education associated with each. In this section I briefly outline the use of remedial education and developmental education in the context of the history of higher education, and then deconstruct the definitions of these words in a manner similar to Clowes (1980), Higbee (1996), and Arendale (2005) to examine the source of meaning associated with these terms. Remedial education was most commonly used to describe this field from the 1860s through the early 1960s, a time period when higher education was reserved for the elite, upper class (Arendale, 2005). Remedial is defined as “giving or intended as a remedy or cure” with the secondary definition, “provided or intended for students who are experiencing learning difficulties” ("New Oxford American Dictionary," 2005). These definitions present students as requiring a remedy or cure for their learning difficulties. Remedial education typically uses diagnostic testing to identify and remediate specific skill deficits. Often, students enrolled in remedial courses are required to do so as a condition of admission to the college. Whereas remedial education comes from a deficit perspective, developmental education takes a more holistic view of student growth and recognizes the value of diverse student experiences (Clowes, 1980). This shift was due, in part, to the social revolution occurring during the 1960s and 1970s, when the use of developmental education emerged. Furthermore, all college students were thought to be growing in their overall development. The root of developmental is develop, which is defined as “grow or cause to grow and become more mature, advanced, or elaborate” ("New Oxford American Dictionary," 2005). Even if we consider developmental to apply to every living being, a question remains. Why is the label developmental education only used in higher education circles to describe courses that are not considered college-level content? Maxwell 9 stated, “…developmental education has become a euphemism for remedial with all the negative connotations that word implies…students taking developmental courses are stigmatized” (Piper, 1998, p. 35). For instance, in K-12 education, developmental is often associated with terms such as developmental delays or developmental disabilities both having the social connotations of somehow less able or less intelligent. Despite the efforts to adopt language that is holistic and that promotes the diversity of learners, the terms remedial and developmental both stigmatize the students enrolled in those courses as somehow less able or less intelligent. For example, Larnell (2016) transcribed and presented a student’s definition of remedial: The word means like simple, or like, dumb. Well, I don't want to say dumb, but . . . And how it applies to the course that I just took? I think that most of the material was simple. But . . . and I feel like . . . Okay. Like when people ask you, like, what course are you in, [the NCBR mathematics course], it’s kind of like, “Well, gee, you’re really bad at math!” (p. 258) This student’s definition and experience clearly illustrated the stigma associated with these words. Thus I have rejected both of these terms in this study. I use non-credit-bearing mathematics course (NCBMC) to describe courses that are not considered college-level content. This term is similar to Larnell’s (2011, 2016) description non- credit-bearing remedial mathematics courses, but without the word remedial. Remedial is redundant to non-credit-bearing because if a course is designated as developmental or remedial at the university level, the content is not considered college-level content and the accumulated credits earned by the students enrolled in NCBMC do not count toward a degree. Lastly, I use non-credit-bearing rather than remedial to honor the fact that students dedicate their effort, time, and money to pay for the credits toward achieving their academic goals, but the ownership of the fact that the course is non-credit-bearing remains with the educational system as a whole, and specifically with the college or university department. 10 Enrollment in NCBMC and Online Courses According to the National Center for Education Statistics (NCES) (Snyder, Brey, & Dillow, 2016), general enrollment in higher education has increased about 22% between 2003 and 2013. In addition, the NCES has projected that college enrollment will “set new records from 2018 through 2024” (Snyder et al., 2016, p. 8). Roughly 20% to 30% of entering college freshman place into a NCBMC that largely consists of high school algebra content (Engstrom & Tinto, 2008; Hill, 2006; Parsad & Lewis, 2003), so as enrollment in colleges and universities increases, so does enrollment in NCBMC. One strategy that higher education has used to meet the demands of increased enrollment has been various forms of online instruction, particularly for graduate and undergraduate entry-level courses. The latter has also been true in undergraduate mathematics courses, as well as NCBMC (Ashby, Sadera, & McNary, 2011; Meza, 2015; Twigg, 2011). Because approximately 30% of undergraduate students take at least one online course, and 8% are enrolled in programs that are delivered entirely online, enrollment in online courses has also continued to increase. For instance, Allen and Seaman (2011) reported that “Over 6.1 million students were taking at least one online course during the fall 2010 term; an increase of 560,000 students over the number reported the previous year” (p. 4). The use of technology in NCBMC has been growing. This study documents students’ learning experiences and students’ interactions with and ways of thinking about mathematics in an online CAI NCB, intermediate algebra course. In line with this purpose, I have organized the background research literature into these sections: (a) research on classroom interactions in lecture based NCBMC, (b) research on web-based technology mathematics instructional models, (c) a summary of ALEKS, the CAI software specific to this study, (d) research on student 11 experiences in NCBMC, and (e) research on the mathematical thinking of students enrolled in NCBMC. Interactions in NCBMC Teaching and Learning Because this study focusses on student interactions in learning mathematics (albeit in an online course) here I briefly discuss a few studies that touch on teacher-student interactions in a lecture-based NCBMC. Although many articles on teaching lecture-based NCBMC have been published, only a few are empirical studies that report on interactions occurring in those classes and include research methods and supporting data for the claims that are made. My review of existing literature revealed two such studies. Both Mesa (2010) and Kanter (2009) conducted empirical studies which found that college lecture-based NCBMC interactions were shaped and often constrained by instructors’ ingrained beliefs about the supposed simplicity of the course content, and purported students’ high anxiety and limited mathematical abilities. As a result of these beliefs, NCBMC instructors typically lectured about mathematical procedures and questions posed to students generally were low cognitive demand questions. NCBMC with a face-to-face lecture format often incorporate technology. For instance, handheld or online graphing calculators are often used in class to demonstrate the relationships between a functions table, graph and equation representations. In addition, other uses of technology may be incorporated, but the lecture is essentially a face-to-face course that typically uses web-based technology such as a course management system (e.g. Desire to Learn) and/or web pages to post the syllabus, assignments, and for other communications. In this technological age, the majority of college lecture courses are “web-facilitated” in this way (Allen & Seaman, 2011). 12 NCBMC that use Web-Based CAI Technology Because the use of online technology in mathematics instruction varies, descriptions of web-based CAI technology instructional models is required. Thus, the various instructional models use web-based CAI technology are defined in Table 1 below. Table 1. Mathematics instructional models that use internet-based technology Course Type Lecture With Online Homework Hybrid Personalized System of Instruction Math Emporium Distance Learning Description A lecture mathematics course that is essentially a face-to-face course enhanced with an online homework system (e.g. Web Assign, or WeBWorK) or computer assisted instruction (CAI) or intelligent tutoring software (e.g. ALEKS, or Cognitive Tutor). A hybrid mathematics course blends face-to-face and online instruction. There is a reduced number of face-to-face meetings and a substantial portion of the content is delivered online. A hybrid course may include online discussions and/or online homework or CAI. The PSI instructional method uses an interactive computer program and typically includes these components: pretesting with a customized study plan, mastery-based progression, post testing mastery before written tests; intervention by instructor, counselors, and disability services; frequent communication between instructor and student; and mini-lectures focused on critical thinking, study skills, and common areas of difficulty. (Keller, Bower, & Chen, 2015, p. 5) A math emporium is a course in which a large number of students are enrolled and CAI provides individualized instruction. It includes a number of additional resources, such as electronic text and video lectures. However, the math emporium model differs from the fully online course in that students may choose or be required to use a large designated computer lab that is staffed by tutors and instructional staff. Emporiums typically include a proctored testing area. Often a series of introductory courses are offered in the math emporium model, and in theory students may complete more than one course per semester to accelerate their progression through the NCBMC and/or introductory mathematics sequence of courses. In an online mathematics course, the vast majority of the content is delivered online, often utilizing CAI, and typically includes no face-to-face meetings. Tests may be proctored face-to-face or in testing centers. Online courses may be designed to be synchronous or asynchronous. 13 Studies comparing the technological instructional models in Table 1 have produced inconsistent results. Zavarella and Ingnash (2009) compared the pass rates of basic algebra students enrolled in lecture classes, hybrid classes, and distance learning. Students who enrolled in lecture courses had the highest pass rates (80%) compared to distance learning (61%) and hybrid classes (58%). Keller, Bower, and Chen (2015) conducted a large study documenting the pass rates of over 9,000 community college students enrolled in NCBMC (pre-algebra, elementary algebra, and intermediate algebra) and compared various modes of instruction along with demographic data. They too found that lectures had the highest pass rates for students, but at a much lower pass rate of 43.3%. Next the hybrid pass rate was 36.4%, followed by distance learning with a pass rate of 22.4% and personalized system of instruction at 22.1%. Although all these pass rates were low, they varied among the different instructional models. Keller and colleagues suggested that community colleges may want to consider providing more lecture- based modes of instruction because the online modes may be less effective at this level of mathematics and for these populations of students. On the other hand, an alternative explanation for differing outcomes based on different delivery methods could be the impact of selection bias. For instance, students who enroll in lecture-based sections may be more motivated or have more time to study than students who choose the more flexible options, and students who are more motivated and who spend more time studying typically perform better. Weller, Trouba, and Wood (2015) compared a traditional lecture, lecture with online homework, and a math emporium model of instruction for intermediate algebra. In contrast to Zavarella and Ingnash (2009) and Keller et al. (2015), students in the math emporium had higher grades and retention rates than those in the lecture related courses, and that the lecture with online homework had the lowest retention rate. Furthermore, Weller and colleagues used pre-post 14 department-created content tests to determine growth, and found similar results from all three instructional models. These results are consistent with the meta-analysis findings published by the United Stated Department of Education (USDE), Evaluation of Evidence-Based Practices in Online Learning (Means, Toyama, Murphy, Bakia, & Jomes, 2009). Jaggars and Baily (2010) have offered a plausible explanation for the discrepancy in the research studies thus far. They noted that of the studies analyzed in the USDE meta-analysis, over half consisted of undergraduates and graduate students enrolled in university or professional courses in health and medicine and teacher education content areas. Jaggars and Baily argued that different student populations, different contexts, as well as different content areas are likely to produce different results. The contradictions in the studies mentioned here appear to confirm Jaggers and Baily’s argument because Keller et al. (2015) and Zavarella and Ignash (2009) were based in a community college context, whereas Weller et al. (2015) was based in a university context. Nonetheless, these contradictions warrant further research. Moreover, the existing research does not clearly point to a “silver bullet” solution to the challenges of mathematics instruction and learning in higher education. Programmed Instruction and Assessment of LEarning in Knowledge Spaces Some online mathematics courses utilize a CAI software package as the primary mode of instructional delivery. The CAI Intermediate Algebra course that was the context of this study utilized the CAI package known as Assessment of LEarning in Knowledge Spaces, or ALEKS, which is a widely used Web-based assessment and CAI system. CAI programs such as ALEKS were heavily influenced by the instructional design movement, and in particular, the ideas of programmed instruction (Lockee, Moore, & Burton, 2004; Park & Lee, 2004; Reiser, 2001; Schrock, 1995). For this reason, here I provide an overview of the learning theories underpinning 15 programmed instruction as well as the mathematical psychology theory of knowledge spaces that serve as the foundation for ALEKS. Programmed Instruction Lockee, Moore, and Burton (2004) contended that term programmed instruction was derived from B.F. Skinner’s work (1954, 1958, 1968) and what he referred to as “teaching machines.” Skinner stated that programmed instructional materials should present short sections of content, with frequent questions, require responses to those questions, provide immediate feedback as to whether the response was correct, and permit the student to set their own learning pace (Lockee et al., 2004; Park & Lee, 2004; Reiser, 2001; Schrock, 1995; Skinner, 1968). It is widely known that Skinner was a behaviorist, and the steps listed above are indicative of the typical behaviorist stimulus—response—feedback cycle designed to reinforce desired behavioral responses. Gagne, also a behaviorist and prolific author, made contributions to instructional design with his writings about the conditions of learning, events of instruction, hierarchies of learning, and theory of cumulative learning (Gagne, 1965, 1985; Gagne & Briggs, 1974; Gagne, Briggs, & Wager, 1992; Gange, 1968). Gagne’s hierarchies of learning and theory of cumulative learning were particularly influential in the development of programmed instruction (Reiser, 2001; Schrock, 1995). Although Gagne (1968) acknowledged the appeal of Piaget’s and Bruner’s constructivist theories cognitive development, he expressed concerns about their methods and proceeded to present his theory of cumulative learning as a scientifically-verifiable explanation. Gagne’s (1968) theory of cumulative learning merged his conceptions about hierarchies of learning with the accumulation of knowledge. Learning contributes to the intellectual development of the human being because it is cumulative in its effects. The child progresses from one point to the next in his 16 development, not because he acquires one or a dozen new associations, but because he learns an ordered set of capabilities which build upon each other in progressive fashion through the processes of differentiation, recall, and transfer of learning. (p.181) The statement from the above quote, “an order set of capabilities,” indicated Gagne’s hierarchy of learning. The statement, “which build upon each other in a progressive fashion,” indicated his belief in the accumulative effect of learning and development. Gagne believed that the analysis of academic topics, which included subdividing the intellectual skills into small increments to be learned in a prescribed order, was the best method of instruction. An example of a learning hierarchy is provided in Figure 1 below. Demonstrate “halves” of a set of objects as 2 equal subsets; “thirds” as 3 equal subsets. Identify 2 apparently equal subsets as “halves” of total; 3 subsets as “thirds” of total Construct subsets which are equal by pairing objects, one to each subset. Identify the number of subsets of a group of objects as 2 or 3. Deal out objects, one to each subset. Apportion leftover objects, one to each subset. Identify one object of a set. Figure 1. A Learning Hierarchy for a Task in Elementary Mathematics (Gagne & Briggs, 1979, p. 148) Current researchers, Lockee, Moore, and Burton have contended that programmed instruction “has never really ceased to exist. Its influence is apparent in the instructional design processes that have continued to serve as the standards for our field” (2004, p. 563). Park and Lee (2004, p. 662) stated that programmed instruction was the precursor to CAI, but highlighted 17 that the differences between the two was primarily due to the sophistication of CAI to adapt the learning environment to meet individual student needs. The ability of CAI, such as ALEKS, to provide individualized instruction is the primary draw of interest in these programs. The next section discusses how ALEKS accomplishes this individualization. ALEKS and Knowledge States The ALEKS software originated from Falmagne and Doignon’s applied mathematical theory of knowledge spaces (1985) which in turn emerged from the field of mathematical psychology. In short, mathematical psychology, is the use of mathematics to model, measure, and study various psychological phenomena, such as “development, perception, learning, cognition, information processing, psychophysiology, and measurement” (Grossberg, 1980, p. vii), and has its roots in behaviorist and cognitive theories. In 1994, Falmagne and colleagues received funding from the National Science Foundation to program knowledge space theory into the computer adaptive instruction program now known as ALEKS (ALEKS, n.d.; Ashback, 2013). ALEKS uses artificial intelligence (AI) and operates on an algorithm that creates a model of a student’s knowledge state, which consists of two lists: “what the student can do” and “what the student is ready to learn” (Falmagne, Doigon, Cosyn, & Thiery, 2006). The ALEKS mathematical algorithm combines the student’s knowledge state with a directed combinatorial graph model of a complex network of mathematical problem types to make decisions about what problems the student is ready to solve. The combinatorial graph model is a directed graph because ALEKS assumes a “precedence relation,” implying that mastery of some problem types must precede others in learning. Each type of problem is represented by a node, or vertex, of the graph, and connections between the types of problems are represented by an arrow, or edge, of the graph. Figure 2 18 depicts a simple precedence relation for the problem types outlined in Table 2. The precedence relation is noted by the direction of the arrow. For instance, in this diagram, the mastery of problem type (d) would imply mastery of problem types (a) and (c), or type (b). Similarly, the mastery of problem type (f) would imply the mastery of 4 potential problem sequences: a-c-d, a- c-e, b-d, and/or b-e. Problem Type yet to be Mastered Mastered Problem Type Outer Fringe Problem Type Inner Fringe Problem Type a c d f b e State K Figure 2. Precedence diagram for problem types described in Table 2 Table 2. Six types of problems in Elementary Algebra (Falmagne et al., 2006, p. 5) (Falmagne et al., 2006, p. 4) Problem Type (a) Word problem on proportions (b) Plotting a point in the coordinate plane (c) Multiplication of monomials (d) Greatest common factor of two monomials (e) Graphing the line through a given point with a given slope (f) Writing the equation of the line through a given point and perpendicular to a given line Example of Instances A car travels on the freeway at an average speed of 52 miles per hour. How many miles does it travel in 5 hours and 30 minutes? Using the pencil, mark the point at the coordinates (1,3). Perform the following multiplication: 4"#$#∙2"∙5$(. expressions 14*+$ and 4*,-$.. Simplify your answer as much as possible. Find the greatest common factor of the Simplify your answer as much as possible. Graph the line with slope -7 passing through the point (-3, -2). Write an equation for the line passes through the point (-5, 3) and is perpendicular to the line 8x + 5y = 11. 19 In ALEKS, precedence relations have been determined by interviewing several experts and combining their answers into a precedence relation. In addition, these precedence relations have been verified by data from thousands of students who have used the software. However, Falmagne and colleagues also state: Some algebra problems may be solvable by a student only if some other problems have already been mastered by that student. This may be because some prerequisites are required to master a problem, but may also be due to historical or other circumstances. For example, in a given environment, some concepts are always taught in a particular order, even though there may be no logical or pedagogical reason to do so (p. 4). The authors seem to have chosen a set of problem types to exemplify this point. In my humble opinion, problem types (a) through (d) of the set of problem types listed in Table 2 would not be necessary to solve a problem of type (f). However, it is important to note that precedence relation diagram depicted in Figure 2 is a subset of vertices from a larger precedence relation diagram for Beginning Algebra (see Falmagne et al, 2006, p. 6, Figure 2). As a result, there are even more problem types and paths between each of the nodes (a) through (f) listed in Table 2. Defining a Knowledge State Once a student has been assessed, ALEKS has determined a knowledge state for that student. A knowledge state is a dynamic interpretation of what problem types the student solves correctly and those they still need to learn. A student’s knowledge state and is updated as the student solves problems assigned by the ALEKS algorithm. Figure 3 displays some essential characteristics of a knowledge state, the outer fringe and inner fringe. When a student has mastered the elements of a knowledge state, then ALEKS assigns the outer fringe for the student to learn. The outer fringe consists of the problem type vertices that immediately follow a mastered problem type. Conversely, if a student incorrectly solves the problem types from the outer fringe, then problems from the inner fringe of the knowledge state are assigned. The inner 20 d e fringe consists of the vertices that immediately precede a problem type that has been attempted but not mastered. f Knowledge State Legend Problem Type yet to be Mastered Mastered Problem Type Outer Fringe Problem Type Inner Fringe Problem Type Figure 3. The outer fringe and inner fringe of Knowledge State (Falmagne et al., 2006, p. 9). How ALEKS Works When a student first signs into their personal ALEKS account, the software uses a pre- assessment, or knowledge check, to create a personal knowledge state (Figure 3) representing what the student knows about the course content. After the initial knowledge check, the ALEKS software utilizes a directed graph theory algorithm to determine what problem types are proximal for the student to attempt next, as determined by the inner and outer fringe of the student’s knowledge state (Falmagne et al., 2006). As the student solves problem types, their knowledge state is updated, and the ALEKS algorithm updates what problem types are proximal for the student to attempt next. Comparing Programmed Instruction and ALEKS Taking into consideration current researchers’ claims that programmed instruction “has never really ceased to exist” (Lockee et al., 2004) and that its practices are still prevalent current instructional design practices (Park & Lee, 2004), here I compare the tenets of programmed instruction and knowledge space theory. The design of ALEKS curriculum can be compared to Skinner and Gagne’s behaviorist conceptions of the design of programmed instruction. First, the 21 developers of ALEKS assembled a group of experts to delineate the content into specific ALEKS Topics. Next, the experts were asked to arrange these ALEKS Topics into ordered “precedence relations” represented by a large, complex directed graph (Falmagne et al., 2006, p. 4). These two steps could be considered a version of Gagne’s content analysis methods to create a hierarchy of learning. Students are provided with explicit instruction and opportunities to practice each of ALEKS Topics and receive immediate feedback, just like the immediate reinforcement both Skinner and Gagne required. Lastly, a student’s mathematical understanding is measured by the number of ALEKS Topics the student has mastered, which parallels Gagne’s theory of cumulative learning. Despite these similarities, ALEKS is more advanced than the past, paper and pencil, manual iterations of programmed instruction because of its technological advancements, computing power combined with the mathematical psychological model of knowledge spaces. ALEKS utilizes Knowledge Space Theory, which represents the curriculum as a directed combinatorics graph. In this graph, the vertices represent each bite-sized mathematics topic which are connected by arrows and arranged in a predetermined precedence relation (Falmagne, Albert, Doble, Eppstein, & Hu, 2013; Falmagne et al., 2006). Student’s progress through the curriculum is measured by their progress through this directed graph. Despite these theoretical and technological advancements, the question remains as to whether the various modes of instruction that utilize CAI in NCBMC encourage the types of interactions that foster productive ways of mathematical thinking and essential algebraic reasoning necessary for success in requisite STEM coursework. This question cannot be answered by reports of average exam scores, end of course grades, and course completion rates. It requires different types of research that reveal more details about student thinking. A few studies that examine student thinking in NCBMC are summarized in the next section. 22 Students’ Mathematical Thinking in NCBMC Little research has examined what students enrolled in NCBMC know about mathematics beyond course assessments and grades. A search of several educational indices revealed only a few empirical studies that examine how students enrolled NCBMC think about mathematics (Givvin, Stigler, & Thompson, 2011; Stigler, Givvin, & Thompson, 2010; Webel et al., 2015). These studies are summarized here. Stigler, Givvin, and Thompson (2010) analyzed data from three sources: student responses to the Mathematics Diagnostic Testing Project (MDTP) placement test items (n=5830 from students taking the placement test during the 2008-09 school year), student responses to a researcher authored survey (n=748 from a convenience sample of students), and follow-up interviews with students (n=30). They summarized three findings from the MDTP placement test and survey data. First, students routinely called upon procedures to solve mathematics problems, even when they can be solved more easily by reasoning. Second, students used reasoning under certain conditions (i.e. the survey), but they rarely used reasoning otherwise (i.e. many procedural errors on the MDTP might have been corrected if students had used reasoning). Third, when students were able to provide conceptual explanations, they also tended to provide correct answers. The researchers noted that the latter may be a causal relationship, but that hypothesis warrants further research. Givvin, Stigler, and Thompson (2011) followed their analysis of the MDTP placement test and survey data with interviews of 30 students. Most students believed mathematics to be a collection of procedures to be memorized and applied. These students approached mathematical problems by selecting a procedure they recalled, whether the problem required a procedure or 23 not. The researchers also noted that when students were encouraged to use their intuitive knowledge of mathematics, they performed better on mathematical tasks. In another study, Webel, Krupa, and McManus (2015) interviewed 10 randomly selected students who had successfully completed a math emporium model of intermediate algebra — nine out of the 10 had earned grades of A or B, and one had earned a grade of C. The interviews consisted of applications of algebra involving a system of equations, the maximum of a quadratic equation, and a rational equation. Although students may have solved the problems a number of ways (e.g. using tables, graphs, equations, or guess and check), they relied on remembering how to translate the problems to equations and then on remembering the procedures for how to manipulate those equations. Across the interviews, students’ approaches to solving the interview tasks involved quickly translating the problem into a symbolic form, and then trying to remember the correct rule to solve that type of problem. To remember the rule, students did not appear to think about the equations as representing relationships between the quantities given in the problem, but instead paid attention to the appearance of the equation. They acted on the symbols according to what they could remember [emphasis added] doing in problems that looked similar. And when students misremembered [emphasis added] the correct procedure, they often failed to recognize their errors or became confused. (Webel et al., 2015, p. 9) In other words, all 10 students exhibited some sort of “conceptual atrophy” (Stigler et al., 2010). The students relied on memorized procedures and were not inclined to reason about quantities to represent the problem, to reason through the algebraic procedures to solve the problem, or check that the solution made sense. Although a small qualitative study of 10 students cannot be generalized to the larger sample, the results of this research do lead one to question some of the technology-based modes of instruction outlined earlier in Table 1, such as personalized instruction and the math emporium models. These results point to the need for more research in this area. 24 Student Experiences, Identity and Mathematical Socialization in NCBMC Using sociocultural and sociopolitical theories and related research methods, phenomenology and ethnography, Martin (2000, 2009a) introduced the ideas of mathematics identity and mathematics socialization to represent African American students’ achievement in mathematics. Martin pointed out that the current emphasis on achievement gaps between White and African American students, discussed from the dominant paradigm of research and typically taken out of context, did more harm than good by normalizing White behavior and reinforcing existing racialized stereotypes. Building upon Martin’s prior research method, Larnell’s (2011, 2016) phenomenological case study research sought to understand and describe African American and Black students experiences in 4-year university non-credit-bearing remedial mathematics courses. In his phenomenological case study, Larnell defined “mathematics identity as a narrative construct” (p. 238) as an alternative to the dominant cognitive methods used to document beliefs, attitudes, or other cognitive concepts to measure and describe students’ orientation and mathematical abilities. In contrast, mathematics identity as a narrative construct attends to “how learners make sense of their learning experiences and the contexts in which they are positioned” (p. 238). In other words, Martin, Larnell, and others have woven together ongoing observational data with multiple interviews to co-construct with the participants a narrative that illustrates their learning experiences within the context in which that learning occurred. Larnell (2011, 2016) described the fluidity of academic mathematics identity through the experiences of two students, Vanessa and Cedric, in a NCBMC at a large university. These two students both had a strong high school academic background in which they reported maximizing their effort and resources on almost every academic opportunity (maximizing identity). Yet, 25 when the challenge of the university placement test presented itself, each reported that they did not put much effort into taking the test. Larnell described this action as satisficing (Simon, 1955 as cited in Larnell, 2016)—choosing an option that is merely adequate as opposed to the best option. Thus the students’ identity was demonstrated as fluid and depending upon the context. In their high school mathematics courses, these students identified as maximizing, but in their transition to the context of university mathematics, they seemed to identify as satisficing despite that they both had verbalized a preference for academic success. Larnell also described situations in which both students experienced social signals that indicated their marginal status as a source of racialized identity threat. For example, Cedric noticed the overrepresentation of Black students in the NCBMC in which he was enrolled, as well as the underrepresentation of Black students in courses such as calculus, and stated, “It kinda hurts me to see so many Black people, like me, in the classroom [NCBMC]…because it, kinda like, says to me, ‘Okay, African American students can’t succeed in this class,’ you know” (p. 257). Cedric discussed the personal pain, or racialized identity threat, that the disproportionate number of Black students in the NCBMC presented. According to Steele (2010), a racialized identity threat (also called stereotype threat) adds an additional task and pressure for the students who experience it. Not only do these students have the typical adjustments of transitioning to a large university and learning in that environment, but they also need to disprove “the negative stereotype and its allegations about you and your group. . . Disproving a stereotype is a Sisyphean task; something that you have to do over and over again as long as you are in the domain where the stereotype applies” (Larnell, 2011, pp. 110-111). In other words, larger contextual and social factors of NCBMC adds to the challenges of a racialized identity threat and adds stress for students of color, which is not a hurdle that can be 26 overcome one time, but a challenge that is repeatedly faced. This summary does not do justice to the research and detailed analyses that Martin (2000, 2009) and Larnell (2011, 2018) conducted, but it serves as an attempt to convey the spectrum of mathematical socialization pressures faced by students of color enrolled in a NCBMC at a predominately White institution. Background Literature Summary This review of the literature has revealed many challenges of teaching and learning in NCBMC. Because a college education has become viewed as a path to economic stability, more students are enrolling in community colleges and universities. With expanded enrollments in postsecondary education, students with more diverse mathematical backgrounds and varied educational needs have increased the demand for NCBMC. Although the purpose of these courses is to prepare students to be successful in subsequent courses, the students who enroll in NCBMC are less likely to complete their degrees. Mesa’s (2010) classroom research and Kanter’s (2009) instructor interviews revealed that instructor’s beliefs shape, and perhaps limit, the type of mathematics content that students enrolled in NCBMC have the opportunity to engage in to prepare for college-level mathematics course work. Community colleges and universities have begun to turn to technology to fulfill enrollment demands and to meet the personal instructional needs of students. Yet, recent research (Webel et al., 2015) has raised the concern about whether these technologically based instructional models are truly preparing students to succeed in the requisite mathematics coursework. Webel and colleagues interviewed 10 students who succeeded in math emporium intermediate algebra course, and noted that all of the students struggled to solve various application problems due to the predominant use of recall and memorization rather than reasoning about the quantities and making connections with algebraic representations and 27 solutions. Based on this research, one might make assumptions about the nature of students’ interactions in a math emporium or other web-based CAI environment, but the solution to these challenges require evidence supported theories, not assumptions. In addition, although the data from a case study involving 10 students cannot be generalized to the population, the Webel study contributes to the overall knowledge base and warrants further research. From a sociocultural perspective on learning, the environment in which student learning occurs has an influence on student learning experiences. Existing research about mathematics identity and mathematics socialization argues that understanding students learning experiences requires framing those experiences by students’ psychosocial experiences that occur within the larger sociopolitical and sociohistorical contexts. “Aside from the contribution to identity- oriented research in mathematics education, there is still much need to study the experiences of learning in NCBR mathematics courses” (Larnell, 2016, p. 261). Furthermore, there is also a need to study student learning experiences in CAI NCBMC, and to situate this work within the larger socio-cultural, -political, and -historical contexts. Although this study does not specifically examine the socio-cultural, -political, and -historical contexts, it draws upon Kahu’s (2013) student engagement framework as the overarching theoretical framework because Kahu situates student engagement within these broader contexts. This is an important framing because Larnell’s research illustrates how these broader socio-cultural, -political, and -historical contexts may influence students’ interactions with their learning environments. 28 THEORETICAL ORIENTATION AND FRAMING CHAPTER 3 This chapter begins with a brief overview of research literature concerning student engagement (SE) and describes the prevalent ways that SE has been operationalized in existing research. Next, I provide an overview of the overarching theoretical perspective, a sociocultural theory of SE that describes the influence of the sociocultural environment, the educational institution and psychosocial influences on SE as well as the immediate outcomes and distal outcomes of SE. In the last section of this chapter, the conceptual framework of SE utilized in this study is described. Overview of Student Engagement Research This overview of the research literature concerning SE begins with a brief history of the National Survey of Student Engagement (NSSE) because the NSSE has been a dominant force in the research concerning SE in higher education. Next, I discuss broader views of SE commonly found in the existing research literature. A Quantitative Approach to Student Engagement The NSSE conception of student engagement in higher education emerged from dissatisfaction with the growing influence of the U.S. News & World Report ranking of colleges and universities. One critical contention was that this ranking was based in part on entering student characteristics, such as SAT scores, and therefore served to perpetuate the inequities in higher education admissions. Also, the ranking did not take into consideration student growth as a result of their university learning experiences, so it did little to compare the quality of university educational practices. In an effort to remedy this situation, the Pew Foundation gathered a panel of experts to develop a survey instrument to measure SE in high quality learning 29 experiences. This group developed the National Survey of Student Engagement (NSSE), which was largely based on Chickering and Gamson’s (1987) Seven Principles for Good Practice in Undergraduate Education and Pace’s (1984) College Students’ Experiences Questionnaire (Kuh, 2009). Kuh, Cruce, Shoup, Kinzie, and Emerson (2008) defined engagement as “both the time and energy students invest in educationally purposeful activities and the effort institutions devote to effective educational practices” (p. 542). In other words, the NSSE was designed to measure the degree that undergraduate students engage in activities that existing research considered sound educational practices (Table 3). Table 3. NSSE themes and engagement indicators (Center for Postsecondary Research, 2017b) Theme Engagement Indicators Higher-Order Learning (HO) Reflective & Integrative Learning (RI) Learning Strategies (LS) Quantitative Reasoning (QR) Collaborative Learning (CL) Discussions with Diverse Others (DD) Student-Faculty Interactions (SF) Effective Teaching Practices (ET) Quality of Interactions (QI) Supportive Environment (SE) Academic Challenge Learning with Peers Experiences with Faculty Campus Environment The underlying assumption of the NSSE is that by measuring the existence and frequency of sound educational practices in undergraduate institutions, then SE in experiences that positively influence learning and personal development is also being measured. Primarily Likert- style survey questions ask students to indicate how much they have engaged in various activities. 30 For example, see Table 4. As of 2017, 708 higher education institutions were actively using the NSSE (Center for Postsecondary Research, 2017b, p. 1). Table 4. Example of NSSE survey questions (Center for Postsecondary Research, 2017a, p. 3) Item Item # Values and Labels 1 = Never 2 = Sometimes 3 = Often 4 = Very Often 1 2 3 4 5 During the current school year, about how often have you done the following? Asked questions or contributed to course discussion in other ways Prepared two or more drafts of a paper or assignment before turning it in 1 2 3 4 5 Come to class without completing or reading assignments 1 2 3 4 5 Attended an art exhibit, play, or other arts performance (dance, music, etc.) 1 2 3 4 5 Asked another student to help you understand course material 1 2 3 4 5 Explained course material to one or more students 1 2 3 4 5 Prepared for exams by discussing or working through course material with other students 1 2 3 4 5 Worked with other students on course projects or assignments 1 2 3 4 5 Given a course presentation 1 2 3 4 5 1. 1.a. 1.b. 1.c. 1.d. 1.e. 1.e. 1.f. 1.g. 1.h. Because of its widespread use, the NSSE has dominated research literature about SE in higher education in the United States; however, there have been many criticisms of the NSSE student engagement framework and survey items. For instance, Kahu (2013) argued that because 31 of the NSSE history as a tool for comparing colleges and universities and institutional improvement, the framework’s definition of SE in unclear because it commingles institutional practices with student experiences and behaviors. Also, although the NSSE authors claim the survey satisfies research criteria for the research validity of self-report survey data (Kuh, 2001), the validity of the survey results has been challenged. For instance, the survey relies heavily on students’ memory of the frequency of events throughout the year, which is a common limitation in data validity. Additionally, the context of questions and potential social bias also serve as validity limitations (Campbell & Cabrera, 2011; Porter, 2011). Regardless of these debates, the NSSE has made an important contribution by initiating widespread research regarding the multiple factors that may influence undergraduate SE and learning in higher education. The NSSE framework is “bounded within the learning institution” (Zepke, 2017, p. 7) and does not examine influences outside of the institution that may also influence SE. Reviews of SE literature have classified the NSSE as a behavioral model of SE (Fredricks, Blumenfeld, & Paris, 2004; Kahu, 2013; Zepke, 2017) because of its focus on institutional behaviors (professors, instructors, and staff) and student behaviors. A common criticism of the NSSE framework is that it is too narrow, and does not consider other influences on SE. Other research perspectives regarding SE in productive learning activities stretch the view beyond behaviors and the boundaries of the learning institution. Widening the Lens on Student Engagement Many studies, particularly on K-12 education (Fredricks et al., 2004), include emotion and cognition, in addition to behavior, as contributing components of SE (Fredricks et al., 2004; Kahu, 2013; Lawson & Lawson, 2013; Zepke, 2017). Fredericks and colleagues (2004) defined and differentiated three components of engagement: 32 • Behavioral engagement draws on the idea of participation: it includes involvement in academic and social or extracurricular activities and is considered crucial for achieving positive academic outcomes and preventing dropping out (p. 60). • Emotional engagement encompasses positive and negative reactions to teachers, classmates, academics, and school and is presumed to create ties to an institution and influence willingness to do the work (p. 60). • Cognitive engagement draws on the idea of investment: it incorporates thoughtfulness and willingness to exert the effort necessary to comprehend complex ideas and master difficult skills (p. 60). Finn and Zimmer’s (2012) four components of SE (academic, social, cognitive, and affective) are similar to Frederick and colleagues’ framework. In Finn and Zimmer’s framework, cognitive engagement is similar, affective engagement parallels their definition of emotional engagement, and academic engagement and social engagement behaviors provided more detail for the definition of behavioral engagement. • Academic engagement refers to behaviors related directly to the learning process, for example, attentiveness and completing assignments in class and at home or augmenting learning through academic extracurricular activities (Finn & Zimmer, 2012, p. 102). • Social engagement refers to the extent to which a student follows written and unwritten classroom rules of behavior, for example, coming to school and class on time, interacting appropriately with teachers and peers, and not exhibiting antisocial behaviors such as withdrawing from participation in learning activities or disrupting the work of others (Finn & Zimmer, 2012, p. 102). 33 In this way, the idea of engagement moved beyond the behaviorist tradition to a broader psychological tradition and added the potential to provide more nuanced descriptions of the nature of SE. Unfortunately, students have sometimes been viewed from a deficit perspective in research that utilizes the engagement frameworks that focus only on the student. For example, Fredericks and colleagues (2004, p. 60) state, “The term [engagement], in both popular and research definitions, encapsulates the qualities that are seen as lacking in many of today’s students [emphasis added].” Mann (2001) takes a different perspective on SE, and argues that student identity and sense of agency may be the cause of student alienation from the higher education system due to the lack of creativity in the opportunities to learn, ownership of disciplinary knowledge and the related issues of power, and an emphasis on performance rather than learning (Mann, 2001, p. 17). Mann also suggests that prevalent definitions of SE and associated research methods may actually measure student compliance with the education institution rather than engagement with intellectual processes. Broadening the view of SE even further to include the sociocultural context has the potential to shift the discussion from a deficit model to examining the potential social and cultural influences on SE (Kahu, 2013; Lawson & Lawson, 2013; Zepke, 2017) as well as taking into consideration student experiences and perceptions (Mann, 2001). Theoretical Orientation: The Sociocultural Nature of Student Engagement The overarching theoretical orientation of this research is based upon Kahu’s (2013) multi-dimensional conceptual framework of undergraduate student engagement, antecedents, and consequences. The purpose of this framework is to describe the wide range of factors that have the potential to influence a student’s engagement. In addition, Kahu situates this array of factors 34 within the broader sociocultural context (social, cultural, and political discourses) to more clearly illustrate how the larger sociocultural climate permeates every element of SE, including the student’s lived experiences before, during, and after education. For example, consider the current nationalistic political environment in the United States since the 2016 presidential election. Research has indicated that racialized hostilities, such as those that have increased in number since the election, have had a negative impact on the mental and physical health of people who are members of the groups that have been targeted by hate (Venkataramani & Tsai, 2017; Williams & Medlock, 2017). It stands to reason that these larger social, cultural, and political discourses and the related rise in hate incidents also negatively impact how students who are members of those targeted groups engage in academic environments. In this way, highlighting the influences of the social, cultural, and political discourses begins to address the critique that prevalent definitions and research about SE have been too narrowly focused and have not taken into consideration “substantial ethical and political issues” (McMahon & Portelli, 2004, p. 60). Kahu’s framework (2013) depicts five elements related to SE (Figure 4): the sociocultural context, structural and psychosocial influences on engagement, and the proximal and distal consequences of engagement. “A key strength of envisioning engagement in this way is that it acknowledges the lived reality of the individual, while not reducing engagement to just that” (Kahu, 2013, p. 766). In making this comment, Kahu argues that individual experiences are unique and yet highlights the necessity to better understand particular student populations. With this SE framework, Kahu has taken a stance that encourages researchers to embrace the complexity of SE and the individual students’ lived experiences, and yet to continue to strive to make sense of the general relationships of the sociocultural dimensions of SE. In addition, Kahu argued that existing research does not make a clear distinction between antecedents, engagement, 35 and consequences, and has presented this framework as an avenue for clarification of these dimensions and directions of influences, to facilitate a shared understanding of the complex nature of SE (Kahu, 2013, p. 768). Sociocultural Influences Political and Social Environment: Culture, power, policy, economics Structural Influences University Culture Policies Curriculum Assessment Discipline Student Background Support Family Lifeload Psychosocial Influences University Teaching Staff Support Workload Relationships Student Motivation Skills Identity Self efficacy STUDENT ENGAGEMENT Affect Enthusiasm Interest Belonging Cognition Deep Learning Self Regulation Behaviour Time and Effort Interaction Participation Proximal Consequences Distal Consequences Academic Learning Achievement Affective Satisfaction Well-being Academic Retention Work success Lifelong learning Social Citizenship Personal growth Figure 4. Kahu's (2013, p. 766) student engagement framework (Kahu, 2013, p. 766) Student Engagement SE is at the center, or heart, of Kahu’s framework. The SE heart of the framework illustrates the psychological dimension by incorporating the three aspects recommended by Fredericks, Blumenfeld, and Paris (2004): affect, cognition, and behavior. The affect dimension included enthusiasm for learning, interest in content, and a sense of belonging. The cognition dimension included deep learning (learning with understanding as opposed to learning by rote), and self-regulated learning (learning strategies and monitoring understanding as well as skill development). The behavior dimension included time and effort devoted to studying, interaction with instructors and other students, and participation in the larger academic community. Although Kahu’s central focus on SE was based on a psychological dimension, engagement was 36 also portrayed as a dynamic construct that is affected by several factors within structural and psychosocial dimension influences. Structural Influences In Kahu’s framework, structural factors associated with the university and with the student have an influence on the psychosocial factors. University structural factors include culture, policies, curriculum, assessment, which all differ by discipline. For example, Brint, Cantwell, and Hanneman (2008) found that the culture of engagement differed significantly between the humanities/social sciences and the natural sciences/engineering majors. The humanities and social sciences culture valued class discussion, asking questions, and making connections between courses. The natural sciences and engineering culture valued mathematical and computer skills, and solving problems. In addition, the natural sciences and engineering culture valued working toward these competencies through individual study as well as studying with other students outside of class, but the student-faculty interaction or participation in class were not valued as much as they were in the humanities and social sciences. These different cultures in the various departments across the university also influence the curriculum that is offered, which in turn influences psychosocial factors such as instructors’ teaching and assessment practices, and the ways that students engage with the content of the curriculum. Structural factors related to students include student background, family, support and life load. It stands to reason that a student’s academic background influences how they engage with their college coursework. Research has also documented that first generation college students tend to have a more difficult transition to college due to a lack of cultural capital. Lastly, life load, the amount of pressures outside of school, such as work, family, health issues, etcetera influence the amount of time that students have to engage with coursework and extracurricular 37 activities. These structural student factors in turn influence the psychosocial student factors of motivation, skills, identity, and self-efficacy. Psychosocial Influences Examining the symbolism of Kahu’s framework illustration reveals the bi-directional relationship between psychosocial influences and SE by the connection of the two with two-way arrows. Within the psychosocial influences are the university factors of teaching, staff, support, and workload. Student relationships with these university factors influence a sense of student belonging at the university, as well as student factors of motivation, skills, identity, and self- efficacy—factors that have been shown in research to influence SE. Kahu states, “It is important to recognize that engagement is not an outcome of any one [original emphasis] of these influences, but rather a complex interplay between them, as suggested by the arrows within this section of the framework” (2013, p. 767). In short, the interplay both within psychosocial factors and between psychosocial factors and SE are complex and multifaceted. Researchers may need to zoom in on a few of these psychosocial factors to understand their potential influence on student engagement, but also must keep in mind the complexity of these relationships and the potential to make incorrect claims of correlation because other factors may also be at play. Just as the influences between SE and its immediate antecedent, psychosocial factors, are bi- directional, the influences between SE and its proximal consequences are also bi-directional. Proximal and Distal Consequences The proximal consequences are categorized into academic and affective consequences of engagement. Academic consequences consist of learning and achievement, and affective consequences consist of satisfaction and well-being. The influence of these proximal consequences are bi-directional with student engagement, because it has been generally accepted 38 that results of student engagement generates more engagement. In Kahu’s framework, proximal consequences have been assumed to lead to distal consequences, which are categorized into the long term academic and social effects of engagement. Academic effects are retention and completion of the university degree, work success after graduation, and an orientation toward lifelong learning. Social effects are informed and engaged citizenship and ongoing personal growth. Theoretical Orientation Summary: Revisiting the Background Literature In this summary of the overarching theoretical framework on the sociocultural nature of SE, each section of the background literature has been revisited and connected with components of the framework (Figure 5). The background literature sections (denoted with red font) have been assigned to one or more components of the theoretical framework. The background literature section, Why “Non-Credit Bearing Mathematics Courses?”, discussed the structural influence of the university culture regarding remedial/developmental mathematics courses. The deconstruction of the words remedial and developmental (Arendale, 2005; Clowes, 1980; Higbee, 1996), plus student and researcher statements from existing research (Larnell, 2016; Piper, 1998), revealed the stigmatism associated with these terms and enrollment in these courses. The use of NCBMC was introduced as a rejection of the derogatory nature of the words remedial and developmental. The section, Enrollment in NCBMC, discussed the increasing enrollment trends in universities (Snyder et al., 2016), and subsequently enrollment in online courses (Allen & Seaman, 2011) and NCBMC (Engstrom & Tinto, 2008; Hill, 2006; Parsad & Lewis, 2003). In addition to enrollment, this section briefly discussed university admissions and mathematics 39 Sociocultural Influences STUDENT ENGAGEMENT Proximal Consequences Distal Consequences Cognition Students' Mathematical Thinking in NCBMC Academic Academic Behaviour Affect Affective Social Psychosocial Influences University Teaching Relationships Interactions in NCBMC Teaching and Learning Student Structural Influences University Why "Non- Credit Bearing Mathematics Courses"? Enrollment in NCBMC NCBMC that use Web-Based Technologies Student placement test policies, so this section of background literature corresponded with the structural influence of university policy on student engagement. Student Experiences, Identity, and Mathematical Socialization in NCBMC Figure 5. Theoretical framework and background literature (in red) synthesis The background literature section, NCBMC that use Web-Based Technologies, provided an overview of how community college and university mathematics departments have created various instructional models that incorporate different uses of web-based technologies. These different instructional models include: lecture courses with online homework, hybrid courses, personalized system of instruction courses, mathematics emporium courses, and distance learning. This section of the background literature falls under the theoretical framework category of structural influences and curriculum. The background literature section, Interactions in NCBMC Teaching and Learning, provided insight into the psychosocial influence of relationships between teachers and students through the lens classroom observations of interactions in NCBMC and interviews with 40 instructors. Classroom observations revealed that the instructors presented procedures and asked low cognitive demand questions (Mesa, 2010). Interviews suggested that this was because instructors viewed the students as anxious and less able to deal with complex problems and viewed the mathematics of NCBMC as simple and not worthy of exploration (Kanter, 2009). The background literature section, Student Thinking in NCBMC, spanned two categories, student engagement and cognition, and proximal consequences that are academic. One study was aligned with student engagement and cognition because it examined how students enrolled in NCBMC thought about and solved procedural problems, and found that when students were encouraged to reason about the problems they were more likely to solve them correctly(Stigler et al., 2010). The other study was aligned with proximal consequences that are academic because it examined how students who had successfully completed a NCBM emporium course would approach and solve algebra applications. Although the problems might have been solved a number of ways, the students relied on recall of procedures and did not verify their answers, oftentimes resulting in incorrect solutions (Webel et al., 2015). Although these students had successfully completed the course, these research results did not bode well for the students’ academic proximal consequences for requisite courses. Lastly, the content of the background literature section, Student Experiences, Identity, and Mathematical Socialization in NCBMC, spans several theoretical framework components: sociocultural influences, structural influences, psychosocial influences, student engagement, and proximal consequences. This is due to the phenomenology and case study research methods, in which the researcher sought to understand the nature of Black students’ lived experiences in relation to their enrollment in a NCBMC at a large Michigan State university. In this study, the researcher explicitly sought to understand and described students’ racialized experiences in 41 NCBMC at the university (structural influences and psychosocial influences), observed and described students’ behaviors in the NCBMC (student engagement), and how these experience potential influenced students’ future engagement (proximal consequences) and sense of belonging at the university (identity), all situated within the larger sociocultural context including culture, power, policy and economics (Larnell, 2011, 2016). In this summary regarding Kahu’s framework of the sociocultural nature of student engagement, I have demonstrated the connections between the background literature in Chapter 2 and the overarching theoretical framework for this study. This process has illustrated what some researchers may consider an advantage of this broad sociocultural approach to the SE framework, which is the flexibility with which the framework can be applied. On the other hand, this process has also illustrated what other researchers may consider a disadvantage of this broad sociocultural approach to the SE framework, which is its constructs overlap many areas of existing studies that were not originally designed as SE research. In addition, it has also been revealed that this broad sociocultural approach to the SE framework is a complex and at times unwieldy theoretical construct. Also, it is important to note that the boundaries of the theoretical framework on the sociocultural nature of SE are not indisputably distinct, yet may still provide a general guide and more specificity than prior SE theoretical frameworks. Although I grant that this broad sociocultural approach to the SE framework is a complex and at times unwieldy theoretical construct, I maintain that there is value in the complexity of the framework, particularly for qualitative studies as opposed to quantitative studies. Quantitative studies are conducted because of their predictive value, and the complexity of this sociocultural SE framework would be an unwieldy disadvantage. However, for qualitative studies, which seek to understand the nature of phenomena, the complexity of this sociocultural SE framework may 42 provide an explanatory model. For example, in this study, situating the context of SE within the larger sociocultural context served the purpose of taking an approach that facilitated a richer description of the case studies, and yet provided a general structure that had emerged from the data (for more detail, see the Data Analysis section, Chapter 3). In the next section, I describe and illustrate the conceptual framework that I have used to document and understand the nature of SE in a CAI NCBMC. Conceptual Framework: Student Engagement in a CAI NCBMC The conceptual framework for this study is set within Kahu’s larger sociocultural theory of SE (2013). The purpose of this theoretical framing of SE is to acknowledge the various influences on students’ implicit and explicit choices of how to engage with learning. In taking a sociocultural perspective on learning, it is important to situate the CAI environment within the larger sociocultural context of the students’ learning experience to acknowledge those potential influences, as in this study, or to explicitly examine the spectrum of influences, as in the phenomenological case study conducted by Larnell. In this study, the primary focus is the nature of student engagement itself, and I draw heavily upon Finn and Zimmer’s (2012) psychological perspective of student engagement for this purpose. Unfortunately, reviews of the literature point to a lack of clear definitions of SE (Fredericks, et al, 2004, Finn and Zimmer, 2012, Kahu, 2013). “The key limitations of the psychological perspective center on a lack of definition and differentiation between the dimensions” (Kahu, 2013, p. 762). For example, Jimmerson, Campos, and Grief (2003) reviewed 45 articles about student engagement, and found that 31 lacked clear, explicit definitions concerning SE and aspects of SE. For this reason, I meticulously define the psychological dimension of SE and how it will be operationalized in the context of this study. I begin by 43 describing how I have modified Finn and Zimmer’s SE framework, followed by how each SE dimension is defined in the context of this study, and lastly, summarizing my definition of student engagement in a CAI NCBMC. To begin, I use subtly different language in my adapted form of Finn and Zimmer’s SE framework. Instead of naming each subcomponent of SE a form of ‘engagement,’ I call them ‘interactions’ that are ‘indicators’ of student engagement. Thus, my framework of student engagement is composed of these indicators of student engagement: cognitive interactions, academic interactions, social interactions, and affective interactions. The reason for this cautious use of language is that ‘engagement’ is an ambiguous term, in a manner similar to the ambiguity of ‘understanding.’ Researchers can no more know the level of student engagement than they can know the depth of a students’ understanding. However, researchers can assemble and describe evidence that indicates a level of engagement or a depth of understanding. My modified version of Kahu’s (2013) framework of engagement antecedents and consequences, combined with this modified version of Finn and Zimmer’s (2012) framework for student engagement is illustrated in Figure 6. As stated in the prior paragraph, each dimension of the SE indicators requires a clear definition because of inconsistent definitions in existing research. I begin with cognitive interactions because this indicator was a primary focus of this study, and also because my construction of the cognitive interactions dimension differs the most from the typical manner in which the cognitive component of student engagement is described. 44 Sociocultural Influences Political and Social Environment: Culture, power, policy, economics Structural Influences University Culture Policies Curriculum Assessment Discipline Student Background Support Family Lifeload Psychosocial Influences University Teaching Staff Support Workload Relationships Student Motivation Skills Identity Self efficacy STUDENT ENGAGEMENT INDICATORS Cognitive Interactions Academic Interactions Affective Interactions Proximal Consequences Distal Consequences Academic Learning Achievement Affective Satisfaction Well-being Academic Retention Work success Lifelong learning Social Citizenship Personal growth Social Interactions (Adapted from Kahu, 2013, p. 766; Finn & Zimmer, 2012, p. 104) Figure 6. Overarching theoretical framework: Sociocultural nature of student engagement Cognitive Interactions Because the culture of engagement differs significantly between the humanities/social sciences and the natural sciences/engineering majors (Brint et al., 2008), I drew upon existing mathematics education literature to construct the dimension of cognitive interactions within the CAI NCBMC that is the context of this study. In both the SE and mathematics education research literature, it is generally accepted that the curriculum and mathematical tasks that students interact with influences how the students engage with academic content (Boston & Smith, 2009; Fredricks et al., 2004; Schoenfeld, 1992; Stein, Smith, Henningsen, & Silver, 2000). Thus, in this section, the research literature on the nature of mathematical problems and tasks that students encounter is summarized first and followed by a summary of the research literature of how students solve mathematical problems. The latter is based on research about mathematical habits of mind that are viewed as important for success in solving mathematical problems encountered in STEM courses and careers. 45 Structural Influence: The Nature of Mathematical Problems and Tasks. Before delving into the mathematical habits of mind that are essential to solve mathematical problems, it is important to provide definitions for the various types of mathematics problems that people encounter. Schoenfeld (1992) characterized mathematical problems in three ways: problems as routine exercises, problems as a means to a focused end, and problems that are problematic. He described “problems as routine exercises” as sets of mathematical problems that have traditionally been present in mathematics texts and used as practice to acquire mathematical skills. He characterized “problem solving as a means to a focused end” as problems used by text authors and teachers to provide a justification for the usefulness of mathematics, to motivate mathematical topics, and as recreation. Lastly, Schoenfeld discussed “problems that are problematic”—in other words, problems that are perplexing and difficult and represent the nature of mathematics from a mathematician’s perspective. The mathematics education researchers, Silver, Smith, and Nelson, (1995) were the first to distinguish between mathematical problems and mathematical tasks. Later, Smith and Boston (2009) define a mathematical task as “a set of mathematical problems or a single complex mathematical problem that focusses students’ attention on a particular mathematical idea” (Boston & Smith, 2009, p. 121). They also characterized tasks as low cognitive demand and high cognitive demand tasks. Low cognitive demand tasks are memorization tasks or procedural tasks in which an algorithm is provided for students to follow. These tasks are similar to Schoenfeld’s “problems as routine exercises.” They do not require students to make connections to the underlying mathematical concepts for performing the task. Lastly, low cognitive demand tasks do not require an explanation, or if there is an explanation, the focus is on the procedures that are used to complete the task, not why those procedures work. On the other hand, Boston and Smith 46 provided two categories of high cognitive demand tasks “procedures with connections” and “doing mathematics.” Procedures with connections focus student attention on underlying mathematical concepts, ideas, or structure, and require students to explain the relevant concepts, ideas, or structure involved. Boston and Smith characterized tasks with the highest cognitive demand as “doing mathematics.” These tasks require complex, non-algorithmic thinking, similar to Schoenfeld’s “problems that are problematic.” Mathematical problems in the real world do not have set algorithmic solutions, so students need to encounter high cognitive demand problems so that they can build the mathematical habits of mind necessary to see mathematics as valuable, useful, and develop their abilities to solve complex problems. The types of problems in a curriculum influence the mathematics that students encounter, interact with, and actually do during the majority of their time in a mathematics course. The types of problems in a mathematics curriculum also shape the form of mathematics that students have the opportunity to learn. Boston and Smith (2009) argue that low level cognitive demand tasks have their place in the curriculum, but that low level cognitive demand tasks encompass far too high a proportion of students’ overall mathematical experiences. In other words, students spend too much time merely memorizing the procedures of how to solve specific types of mathematics problems, without understanding why the procedures work and without making connections between the procedures used in one problem type with how the same or similar procedures might or might now apply in the next problem type. In contrast, working on complex mathematical tasks requires that students engage in mathematical thinking, make mathematical connections, and utilize important mathematical habits of mind to solve those tasks. 47 Cognitive Interactions: Problem Solving Activities. Polya’s (1945/1985) work has served as a foundation of the research in mathematical problem solving, heuristics, and metacognition. Polya characterized problem-solving as four rather linear phases: a) understanding the problem, b) devising a plan, c) carrying out the plan, and d) looking back (1985, p. 5-6). Garfalo and Lester (1985) provided what they called a “cognitive-metacognitive framework” that also consisted of four rather linear phases: a) orientation, b) organization, c) execution, and d) verification. In 2001, Pugalee adapted their framework, but the overall structure remained the same. Schoenfeld’s extensive research of mathematics problem solving, published in 1985, is also considered a seminal work. Shoenfeld illustrated the complex nature of problem solving through five mathematical problem solving strategies: a) analysis, b) exploration, c) design, d) implementation, and e) verification. Furthermore, Schoenfeld showed that the process of problem solving was not linear and smooth, but that problem solvers moved back and forth between the various problem solving strategies throughout the problem solving process. The last framework referenced was the National Council of Teachers of Mathematics (2009), which outlined four overarching, essential reasoning habits: a) analyzing a problem, b) implementing a strategy, c) seeking and using connections, and d) reflecting on a solution. The conceptual framework for this study draws upon and synthesizes these five frameworks into three non-linear phases: a) orientation, b) generation and production, and c) conclusion, with subcategories of problem solving activities that occur within each phase. A comparison of these referenced frameworks and their relationship to the new conceptual framework used in this study is provided in Table 5 below. 48 Table 5. Problem solving frameworks comparison and synthesis Problem Solving Phases Problem Solving Strategy (Polya, 1985, pp. 5-6) Understanding the problem (Schoenfeld, 1985, p. 110) Analysis Cognitive- Metacognitive Framework (Garofalo & Lester, 1985, p. 171) Orientation Devising a plan Carrying out the plan Exploration Design Implementation Organization Execution Looking back Verification Verification Reasoning Habits (NCTM, 2009, pp. 9-10) Analyzing a problem Implementing a strategy Seeking & using connections Reflecting on a solution Problem Solving Phases and Activities Nimtz (2016) Orientation Understand Analyze Generation Explore Plan Execute Conclusion Interpret Verify Reflect This synthesized conceptual framework for problem solving phases and activities for cognitive interactions has been summarized below in Table 6. Although these processes may appear linear, when observed in action, they may not be enacted in a linear fashion. The nonlinear manner of problem solving behavior has been demonstrated to be the case, particularly in cases in which the mathematical problem or task is complex, or in other words, a high cognitive demand task (Schoenfeld, 1985). The problem solving phases and activities outlined in Table 6 may be applied to any area of mathematics (e.g. number and operations, geometry, statistics, algebra, calculus). A challenge regarding the application of the problem solving phases and activities framework as an indicator of cognitive engagement in mathematics is that the research informing the foundational frameworks about problem solving heuristics from which this was synthesized was focused on the solving of non-routine mathematics tasks. In this study, the focus is on the nature of students’ 49 interactions with an online CAI intermediate algebra course, and the problems are prescribed exercises for which an example has been provided. Therefore, a predictable question is whether or not the problem solving phases and activities will be an appropriate framework for prescribed mathematics exercises. My hypothesis was that as students work on these problems, the three phases of mathematical problem solving (Orientation, Generation, and Conclusion) will be similar, but that the mathematical activities that occur within these phases will differ based on the types of mathematical problems students encounter (e.g. the level of cognitive demand). Table 6. Problem solving phases and activities when solving non-routine mathematics tasks Problem Solving Activities Problem Solving Phases • Orientation: Reading the problem to become familiar with the problem context and to determine essential aspects of the problem. • Generation and Production: Includes exploration of the mathematics in the problem, planning, and execution of the plan to solve the problem. • Conclusion: Includes interpretation and verification of the solution, and reflection on the solution process and how the problem might fit in the larger context of mathematics. o Understand: Reading the problem to understand what the problem is asking in the particular problem context. o Analyze: Re-reading the problem to determine essential aspects of the problem, such as concepts, procedures, variables, representations, and/or conditions. o Explore: Looking for patterns, relationships, representations, or algebraic structures that help in solving the problem. Considering if solutions to similar problems might be helpful. o Plan: Planning the solution steps. o Execute: Carrying out the plan. o Verify: Determining if the solution is correct and o Interpret: Checking to see if the solution makes o Reflect: Examining the solution process, comparing sense in the context of the problem. checking for extraneous solutions. different solutions to the same problem, and considering similarities and differences between this problem, its solution, and other mathematical problems and their solutions. Cognitive Interactions: Surface-to-Deep Continuum. Finn and Zimmer defined cognitive engagement as “the expenditure of thoughtful energy needed to comprehend complex 50 ideas to go beyond the minimal requirements” (2012, p.102). This statement could be interpreted to say that a minimal expenditure of effort and thought does not constitute cognitive engagement. I take a stance more in line with Mann (2001), that students may be characterized as somewhere along a continuum between alienated or engaged. Furthermore, in the same vein as several studies that have characterized students’ approach to academic work by describing their orientation to learning (Biggs, 1979; Case & Marshall, 2004; Vivien Beattie IV, Collins, & McInnes, 1997), I draw upon Biggs initial portrayal of of three student orientations toward learning, utilizing, achieving, and internalizing. A summary relevant to this study outlined in Table 7 below. Table 7. Surface and deep learning approaches (adapted from Beattie et al, 1997) Approach to Categorization Learning Utilizing Surface learning Strategies Limits learning to the bare essentials to reproduce procedures through memorization. Studies content to understand the concepts underlying procedures and to make connections between mathematical ideas. Uses performative study strategies of the “model student” by organizing time and working space. Deep learning Strategic learning Internalizing Achieving The utilizing and internalizing approaches to learning represent the opposite ends of a spectrum of potential approaches to learn content. My hypothesis is that there may be learning approaches that do not clearly align with the utilizing and internalizing approaches, but lie somewhere on the spectrum between the two. The achieving approach to learning, sometimes called the strategic learning (Mann, 2001), belongs more with the SE indicator of academic 51 interactions than cognitive interactions because study strategies may be used to accomplish either surface learning, deep learning, or some type of learning within that continuum. Definition of Cognitive Interactions. For the purposes of this study, mathematical cognitive interactions are defined as the thinking activities that are undertaken as one solves mathematics problems, perhaps within, but not limited to, a school mathematics environment. Specifically, mathematical cognitive interactions are comprised of the thinking activities that are carried out during the orientation, generation, and conclusion phases of mathematical problem solving. Cognitive interactions may be characterized along a continuum of surface to deep, which is contingent upon the nature of the aforementioned thinking activities. At one end of the continuum, surface cognitive interactions are described as the thinking activities that are limited to the bare essentials to imitate or reproduce procedures through memorization. At the other end of the continuum, deep cognitive interactions are described as the intensive effort or study of mathematics to understand the concepts underlying mathematical procedures and to make connections between mathematical contexts, symbols, ideas and representations. In the context of solving a high cognitive demand mathematical problem, deep cognitive interactions would include all of the activities summarized in Table 6: Problem solving phases and activities when solving non-routine mathematics tasks. It is important to note that the nature of cognitive interactions is influenced by the cognitive demand of mathematical task being solved. Academic Interactions There are subtle differences between academic and cognitive interactions. Academic interactions are the observable behaviors demonstrated when a student participates in class or completes course work. Cognitive interactions are the internal thought processes that occur when a student participates in class or completes course work, and these thought processes are not 52 readily observable. The definition of academic interactions used in this study (see below) is quite similar to that of Finn and Zimmer’s definition of academic engagement, which “refers to behaviors related directly to the learning process, for example, attentiveness and completing assignments in class and at home or augmenting learning through academic extracurricular activities” (2012, p.102). Definition of Academic Interactions. For the purposes of this study, mathematical academic interactions are defined as researcher observed and student stated behaviors related to participating in class and learning course material, such as the study strategies used to learn mathematics content, the use of various learning resources and tools, and time management techniques used to complete course work. For example, study strategies may include, but are not limited to, memory techniques, scheduling time to study or to complete a certain proportion of assignments each day, taking notes and reviewing material. In addition, academic interactions consist of the utilization of academic resources to learn the material, which may include, but is not limited to, course texts, online resources, or university support services such as tutoring. Affective Interactions The definition of mathematical affective interactions used in this study (see below) is quite similar to that of Finn and Zimmer’s, “Affective engagement is a level of emotional response characterized by feelings of involvement in school as a place and a set of activities worth pursuing” (2012, p.103). Definition of Affective Interactions. For the purposes of this study, mathematical affective interactions are defined as emotional response to course content and class participation, which includes feelings of confidence (or feelings of a lack of confidence and/or anxiousness) in tackling various assigned mathematical problems, as well as a sense of mathematics course 53 content value and usefulness. The sense of mathematics course content value and usefulness may include observations of unprompted student statements (e.g. “When will I ever use this.”) or student statements in response to the end of course interview prompts (e.g. “Please elaborate on how math will play a role in your career, or not.”). Social Interactions In this study, social interactions are not a data source because the NCBMC that is the context of interest is an individualized online, intermediate algebra course that utilizes CAI technology as its primary mode of instruction. The CAI program also includes an online textbook and online short instructional videos on the algebra topics of the course. However, I still provide a definition of social interactions as an indicator of SE, which is very similar to what Finn and Zimmer provided (see quote below). My definition of social interactions in SE are observed student behaviors that are in accordance with explicit (written) and implicit (unwritten but generally accepted) classroom norms (rules of behavior) similar to Finn and Zimmer’s as stated below. Social engagement refers to the extent to which a student follows written and unwritten classroom rules of behavior, for example, coming to school and class on time, interacting appropriately with teachers and peers, and not exhibiting antisocial behaviors such as withdrawing from participation in learning activities or disrupting the work of others (Finn & Zimmer, 2012, p. 102). It is important to note that social interactions are influenced by the pervading socio- cultural, socio-political, and socio-historical environment, but these are two different constructs. Summary: Conceptual Framework of SE and Revisiting Research Questions In summary, the SE indicators, which are cognitive interactions, academic interactions, affective interactions, and social interactions, are illustrated in Figure 7 below. Note that the SE indicators are centrally located in the overarching theoretical framework in a similar manner to 54 Kahu’s original framework (2013). This is to highlight the sociocultural nature of SE; however, the SE indicators are foregrounded in this diagram to illustrate that they are also foregrounded in this study. In addition, observe that the SE indicator, social interactions, is faded to indicate that it exists, but was not examined because students’ interactions were with an individualized CAI environment. Also, the categories of relevant evidence for each SE indicator are listed. For example, under the SE indicator, affective interactions, the categories of relevant evidence are level of confidence and value of math. Furthermore, I conjecture that the influence between each of the SE indicators is bi-directional, as illustrated by the two-way arrows in the diagram, which suggests even more complexity in an already complex theory. Sociocultural Influences Political and Social Environment: Culture, power, policy, economics STUDENT ENGAGEMENT INDICATORS Cognitive Interactions Problem Solving (Phases & Activities) Surface Deep Affective Interactions Level of Confidence Value of Math Social Interactions Participation in Academic Norms Academic Interactions Study Strategies Resources Structural Influences University Culture Policies Curriculum Assessment Discipline Student Background Support Family Lifeload Psychosocial Influences University Teaching Staff Support Workload Relationships Student Motivation Skills Identity Self efficacy Proximal Consequences Distal Consequences Academic Learning Achievement Affective Satisfaction Well-being Academic Retention Work success Lifelong learning Social Citizenship Personal growth Figure 7. Conceptual framework, student engagement indicators (adapted from Kahu, 2013; Finn & Zimmer, 2012) The complexity of this framework is an important contribution to this qualitative study. I take the perspective emphasized by Barbara Rogoff (2008) noted in the quote below. In research, when we examine a phenomenon, we often isolate that phenomenon and use a simplified 55 framework, in other words a foregrounded aspect of a complex framework, to analyze it. Yet, that event does not occur in isolation, and the simplified or foregrounded aspect of the framework, although allowing us to focus details, is not separate from the whole or the backgrounded aspects of the framework. The use of "activity" or "event" as the unit of analysis - with active and dynamic contributions from individuals, their social partners, and historical traditions and materials and their transformations - allows a reformulation of the relation between the individual and the social and cultural environments in which each is inherently involved in the others' definition. None exists separately. Nonetheless, the parts making up a whole activity or event can be considered separately as foreground without losing track of their inherent interdependence in the whole. Their structure can be described without assuming that the structure of each is independent of that of the others. Foregrounding one plane of focus still involves the participation of the backgrounded planes of focus. (pp. 58-59) I conclude this section by revisiting and elaborating upon the research questions that were presented in the introduction of this study. The research questions listed below are tightly aligned with the foregrounded student engagement indicators conceptual framework (Figure 7 above) and clarify the indicators of the student engagement as framed and defined in this study. Overarching Research Question: What is the nature of students’ mathematical engagement in an online, CAI intermediate algebra course? 1. What is the nature of students’ cognitive interactions in mathematics in an online, CAI intermediate algebra course? a. What is the potential cognitive demand of the way CAI presents the course content to students? b. What are the activities within the problem solving phases (orientation, implementation, verification) that students use to solve the CAI problems? 2. What is the nature of students’ academic interactions in an online, CAI intermediate algebra 56 course? a. What academic study strategies do students use to learn the course content? b. What academic resources do students draw upon to learn the course content? 3. What is the nature of students’ affective interactions in an online, CAI intermediate algebra course? a. In what ways do students affectively respond to mathematical tasks of an online, CAI intermediate algebra course? b. In what ways do students affectively respond to the course participation structure of an online, CAI intermediate algebra course? c. What is the nature of students’ value of mathematics in their lives and future careers? 57 CHAPTER 4 RESEARCH METHOD There exist only a few studies that examine student experiences in NCBMC, and, more specifically, no studies about the nature of student experiences and mathematical interactions within CAI mathematics environments in NCBMC. Therefore, the purpose of this study was to begin to fill that gap in the research—to describe and understand the nature of students’ mathematical interactions within CAI mathematics environments in a NCBMC. To examine the nature of students’ engagement, I adapted Finn and Zimmer’s SE conceptual framework to be indicators of SE, which included cognitive interactions, academic interactions, affective interactions, and social interactions. The latter, social interactions, was not studied because students worked individually on a computer throughout the intermediate algebra course. The SE conceptual framework in the prior chapter provided a detailed definition for each of the indicators of SE. In this chapter, I operationalize how evidence of these indicators was gathered and analyzed (see Appendix B: Research Design Summary Table). This chapter contains a detailed account of the case study research method used to examine the nature of SE with an online CAI intermediate algebra course. This includes how the research was conducted, beginning with a clarification of my perspective as a researcher as I undertook this study. Next, I describe the research context and participants, followed by a description of how the data was collected and analyzed. This is followed with a discussion of the limitations and strengths of this study design. Lastly, the chapter is concluded with a summary of the methods and revisiting the research questions. 58 Researcher Positioning and Assumptions My experiences teaching NCBMC at a community college have forged my interest in this topic. In addition, my personal experience as a first generation college student from a working- class background has contributed to my interest in this research because a large proportion of students in NCBMC are first generation college students from working-class backgrounds. My perspective of knowledge is from a sociocultural viewpoint in which “Humans engage with their world and make sense of it based on their historical and social perspective” (Creswell, 2003, p. 9). As such, I believe researchers must reflect upon and make clear their related experiences to remain as objective as possible, to avoid blatant subjective judgements as they conduct a study, and to place the study within the context of the researcher’s background. I enter into this research study with the assumption that all students enrolled in the NCBMC enter the course motivated to succeed and that each student has the intellectual capacity to do so. Furthermore, I counter the commonly accepted belief that verbal and mathematical ability constitute evidence of innate intelligence with the argument that all abilities to function in the world are learned in a social and cultural context, are related to opportunity to learn, and are strongly influenced by experientially and culturally formed beliefs about what is important and necessary to know and learn. For instance, an academic’s perspective may be that skilled labor and everyday labor is simple, overlooking the cognitive complexity of the tasks performed (Rose, 2009). Similarly, skilled and everyday laborers may view academic work as pretentious, detached from common sense, and irrelevant. Of course, these are overly broad and extreme generalizations of potential beliefs of these two groups, but I use them to illustrate contrasting culturally formed views about intelligence and work. 59 In summary, my perspective is that one’s personal interests and ability to function in a culture and society are formed by our experiences and do not serve as indicators of innate intelligence. In short, I assume a “growth mindset” (Dweck, 2009)—that intelligence, in other words the knowledge, actions, and interactions occurring within various cultures and situations, can be learned and that it is important to be open to new experiences of learning. I also assume the contrapositive, that intelligence is not an inherent characteristic and failure on a task or a test does not indicate a lack of intelligence. Similarly, my assumptions about students who place into NCBMC are that they are motivated, intelligent students who are more than capable of learning mathematics. Research Context This study took place at Michigan State University (MSU), a large Mid-Western university. In the school year, 2016-17, this university enrolled more than 39,000 undergraduate students, which included over 7,950 freshmen. All entering freshman are required to take a mathematics placement exam at least 4 weeks prior to freshman orientation. This requirement is waived if the student has taken and passed the AP Calculus exam, or earned a score of 28 on the ACT, or scored 640 on the SAT. Based on data from the registrar, over the last 10 years, approximately 12% of entering college freshman have been enrolled in an Intermediate Algebra course, the NCBMC that was the specific context of this study. Brief History of NCBMC at MSU In 1972, MSU was concerned about the high failure rate in College Algebra and Precalculus courses. In an attempt to remedy this problem, the MSU mathematics department was charged with developing a placement test and two new NCBMC, Beginning Algebra and Intermediate Algebra. In addition, the mathematics department hired a Mathematics Academic 60 Specialist to supervise course instruction and develop instructional materials and curriculum. In 1992, when MSU switched from the quarter system to the semester system, the content of the two NCBMCs was reviewed and merged into one new Intermediate Algebra course. In 2000, leadership for Intermediate Algebra changed, and a new Mathematics Academic Specialist was hired. With new leadership, the course structure also began to shift to larger lectures. For instance, an examination of MSU’s online schedule of courses ( www.schedule.msu.edu ) revealed a shift from 13 lectures averaging 63 students each in Fall Semester 2003 to 4 large lectures averaging about 237 students each in Fall Semester 2004. In an effort to counter the increase in lecture size and to provide more individualized instruction, the mathematics department also incorporated the use of ALEKS, an online CAI program. In 2011, the mathematics department chose to drop the large Intermediate Algebra lectures all together and relied solely on the online ALEKS software to provide instruction. This online Intermediate Algebra course was offered in 6 sections of approximately 150 students. Two MSU instructors oversaw 6 undergraduate lab aids who were tasked with monitoring student progress and answering student questions via the ALEKS interface. I am not certain why this course structure change was made, but informal discussions with people in the mathematics department suggested that the change was motivated by low student attendance at the large lectures combined with the decrease in costs of running the course entirely online, which are plausible explanations. This paragraph has described the course options that were available for the vast majority of students enrolled in Intermediate Algebra at MSU. The last course structure iteration for Intermediate Algebra, the sole reliance on the online ALEKS software to provide instruction, provided the course context of this study. 61 In addition to the Intermediate Algebra course options described above, MSU had also designed two types of course experiences with the intention of providing more support for students considered “at risk.” These particular sections of Intermediate Algebra have a long history at MSU, and data from MSU’s online enrollment system revealed they have existed for at least the past 15 years. First, MSU’s Office of Student Support has offered programs, such as TRIO or College Achievement Admissions Program, for student populations typically considered to be more “at risk” of failure or dropping out of school altogether (e.g. first-generation college students and/or low-income students). For these students, the mathematics department reserved 7 Intermediate Algebra sections for face-to-face instruction and limited the class size to approximately 20 to 25 students. Second, the mathematics department offered 6 sections of a Mathematics Enrichment course to be taken concurrently with the online Intermediate Algebra course. Mathematics Enrichment was created to provide additional support for students who were deemed “at risk” of failure by their advisors. These face-to-face Mathematics Enrichment co-enrollment courses were also limited to approximately 20 to 25 students and were facilitated by undergraduate lab assistants. The number of sections of these two types of Intermediate Algebra support courses have remained fairly constant over the years. However, this study did not examine the experiences of students enrolled in either type of support courses for Intermediate Algebra. For more information about the experiences of students required to enroll in these support sections, see Larnell (2011, 2016). Intermediate Algebra Course Context for this Study In Fall Semester 2016, MSU offered 10 sections of online Intermediate Algebra with an average of 74 students per section. In Spring Semester 2017, MSU offered three sections of online Intermediate Algebra with an average of 53 students per section. These online 62 Intermediate Algebra sections relied on the online ALEKS software to provide instruction, and the participants in this study were enrolled solely in the online course sections and not the support course sections. ALEKS, otherwise known as Assessment and LEarning in Knowledge Spaces, is a Web- based assessment and CAI software system. When a student first signs into their personal ALEKS account, the software uses a pre-assessment, or “knowledge check,” to create a personal “knowledge state” of what the student knows about the course content. After the initial knowledge check, the ALEKS software utilizes a directed graph theory algorithm to determine what problem types are proximal for the student to attempt next (Falmagne et al., 2006, p. vii). The ALEKS package provides a written example of how to solve the problem, plus additional resources, such as an electronic version of the course text and videos that explain how to solve the problems. Students enrolled in this course received the course instruction and completed their assignments through ALEKS. Also, students were able to get help from undergraduate and graduate tutors in a Mathematics Learning Center, which is open 40 hours per week in the hall that hosts the mathematics department, and 10 evening hours per week at 5 other locations around campus. Students’ grades were determined by 4 proctored tests (44.4% of grade) and a proctored final exam (22.2 % of grade), online ALEKS quizzes (11.1% of grade), and ALEKS homework assignments (22.2% of grade). Continued Efforts to Improve Introductory Mathematics Collaborative efforts between the MSU Department of Teacher Education and the Mathematics Department have provided high quality instruction and learning experiences for students enrolled in this NCBMC, specifically for a subset of students who had been identified as 63 “at risk” and who were enrolled in an optional co-requisite mathematics lab to support their learning. These efforts consisted of providing a course section of a co-requisite lab course for the CAI online intermediate algebra course. This co-requisite lab included high cognitive demand, collaborative activities that were taught by prospective secondary mathematics education teachers. These prospective teachers were provided mentoring for lesson planning as well as in the moment coaching as they taught the intermediate algebra students enrolled in the co-requisite lab. The results of this collaboration have been promising and it continues even though the NSF funding has ended (Bieda, Herbel-Eisenmann, McCrory, & Sikorski, 2013). In addition, for more than 10 years MSU has engaged in various efforts to improve students’ learning experiences and outcomes in MTH 1825 and College Algebra. One recent result of these efforts has been major changes in the mathematics course options for introductory mathematics. For example, shortly after this study was conducted, the MSU administration decided to phase out MTH 1825, the CAI intermediate algebra course that utilized ALEKS as its primary mode of instruction. Instead, students may now elect to take Quantitative Literacy mathematics courses. Also, students who have STEM aspirations, but whose mathematics placement exam score was low, have the option of taking College Algebra at a slower pace over 2 semesters, and with the optional addition of a support lab. Research Participants The research participants consisted of a subset of volunteers recruited from intermediate algebra students enrolled in the course of interest in this study. Four students who voluntarily completed the online mathematics survey responded that they were interested in learning more about participating in the case study (Table 8). After the initial meeting during which the 64 expectations and risks of participating were discussed (see consent form, Appendix A), these four students volunteered to participate. In Fall semester of 2016, two students participated. In Spring semester 2017, three students participated. One of these students, Chad, failed the course in the fall and agreed to participate again in the spring when he retook the course. Another student had agreed to participate in the study, but then withdrew, so her data was not included. Table 8. Participant demographics Pseudonym Semester Status Jade Chad Tia FS16 FS16 & SS17 SS17 Gender Race Prior HS Course Freshman Female Black Calculus Freshman Male White Pre-Calculus Freshman Female Black Calculus Data Collection Major Molecular Genetics Journalism Apparel & Textile Design This study consisted of 4 phases of data collection: participant recruitment, initial meeting, primary data collection, and end of course interview. These phases are outlined in Figure 8 below and described in detail in the subsequent paragraphs. Data Colletion Phase 1: Participant Recruitment DATA: Math History Questionnaire PROCEDURES: Included Consent to Participate in Survey Also Surveyed for Interest in Case Study Participation Data Colletion Initial Meeting Phase 2: DATA: Math History Questionnaire Follow-Up PROCEDURES: Consent to Participate Think Aloud Rehearsal Software & File Management Introduction Data Colletion Phase 3: Primary Data Collection DATA: Multiple Short Think Aloud Screencasts & Pencasts: Independently Conducted by Student One Extended Think Aloud Screencast & Pencast: Observed by Researcher Data Colletion End of Course Phase 4: DATA: End of Course Survey End of Course Interview PROCEDURES: Administer Survey Follow-up with Interview Figure 8. Overview of data collection phases 65 Phase 1: Participant Recruitment. I used the course email system to send a confidential, online mathematics history questionnaire (Appendix D). In Fall semester 2016, 46 students participated in the online mathematics history questionnaire, and 2 agreed to participate in the case study. In Spring semester 2017, 24 students participated in the online mathematics history questionnaire, and 3 agreed to participate in the case study. Later, one of these participants withdrew from the study. Phase 2: Initial Meeting. I met with the participants of the study to go over consent forms (Appendix A). During this meeting, I also introduced them to the website that I had created to serve as a resource regarding the technology included in the data collection process ( https://sites.google.com/view/mth1825researchproject ). This website included both written and video recordings of how to set up and use the Screencast-O-Matic and LiveScribe pen software. Next, we set up software that recorded screencasts of their online work, Screencast-O-Matic, and the software for using the LiveScribe Pens. After the software was set up, we practiced the process of recording and saving their think aloud screencasts and associated written work with pen-cast recordings. The participant and I discussed what a “think aloud” is and why it was important for this study, and I asked the participant to watch a short video of me conducting a think aloud using the LiveScribe pen. Then I left the room and the participant used the screencast software and LiveScribe pen as they solved a problem in ALEKS while thinking aloud. About 15 minutes later, when they were done, we rehearsed saving these files to their private, secure Google drive that was set up for this project. Lastly, we reviewed the think aloud they just conducted to co-create a shared meaning of the definition of think aloud. Phase 3: Independent Weekly Think Aloud. Each of the participants was asked to record at least one 15 minute think aloud screen cast per week. I created a separate, private, 66 password protected, university supported Google Drive folder for each participant so they could confidentially upload the files of their screen cast (Figure 9) and pen cast (Figure 10) recordings to save the files and share them with me. Three of the four participants preferred this independent method. The fourth preferred to meet with me weekly to use my computer for the recordings, but he independently worked in ALEKS in a separate office. In other words, outside of the Observed Extended Think Aloud, we did not interact while he worked in ALEKS and conducted his think aloud recordings. Figure 9. Screencast frame 67 Figure 10. Pen-cast frame 68 Phase 3: Mid-Semester Observed and Extended Think Aloud (OETA). Around the middle of the semester, I scheduled a time to meet with the students individually and to observe them working in the CAI environment, which also allowed me to observe as they solved problems related to more than one ALEKS topic. Up until this point, the shorter screencasts that participants had recorded on their own had primarily included only one topic. Based on what I understood about ALEKS, I wondered if the student participants made mathematical connections between the topics. As I observed students, I occasionally would ask questions about their choice of resource use in ALEKS and their mathematical thinking. These OETA sessions were recorded using the screen cast and pen cast. The OETA session with each participant lasted approximately 45 minutes. Phase 4: End of Course Survey and Interview. The data collection concluded with an online survey and related end of course interview questions in which students were asked to describe their overall experiences of the online CAI intermediate algebra coursework. Follow-up questions asked what were positive and negative aspects of the online algebra course and how might the course be improved. These questions were intentionally open-ended to prompt the participants to freely share their experiences (Appendices E and F). Table 9. Summary of data collected in the beginning and throughout the course. Pseudonym Semester Math History Survey Weeks of Semester Think Aloud Recordings Jade Chad Chad Tia FS16 FS16 SS17 SS17 Yes Yes -- Yes 8-12 8 2-11 3-11 4 1 10 11 69 Observed & Extended Think Aloud No No Yes Yes Table 10. Summary of data collected at the end of the course. Pseudonym Jade Chad Chad Tia Survey Semester End of Course FS16 FS16 SS17 SS17 Yes No Yes Yes End of Course Interview End of Course Grade No Yes Yes Yes 3.5 0.0 2.5 2.0 Data Analysis To analyze the data of this case study, I began by reviewing the sets of data about student engagement for themes. The themes and codes that emerged from the data were cognitive, academic, and affective interactions (Figure 11). These three types of student interactions with the ALEKS tasks were aligned with research literature on student engagement and served as the theoretical framework for this study (Finn & Zimmer, 2012; Kahu, 2013). Every think aloud recording was revisited multiple times and the data was recorded and coded in a spreadsheet consisting of the heading categories: logistical data, ALEKS Topic data, cognitive interaction data, academic interaction data, affective interaction data (see Appendix I for more details). After coding the data of each individual case, I described each case by summarizing the data according to the three student engagement themes and indicators (Chapters 5, 6, and 7), which was followed by the cross-case analysis (Chapter 8). The cross-case analysis included comparing the student engagement indicators across the three cases, followed by examining potential relationships between the student engagement indicators. Individual Case Analysis SE Indicator 1: Cognitive Interactions. To analyze the think aloud screencast and pen- cast recording data, the recordings were viewed multiple times for multiple purposes. First I watched each video and identified the ALEKS Topics. Each ALEKS Topic was the unit of 70 analysis, and I used the IQA rubric (Appendix G) to code for the cognitive demand of the mathematical task. In this study, each ALEKS Topic comprised the mathematical task. Also, for each ALEKS Topic I outlined the sequence of ALEKS pages that the student interacted with as well as whether the students answered the ALEKS problems correctly or not. Two general ALEKS sequence types emerged, a routinized sequence, which occurred when the all of the student’s answers were correct, and what was noted as a critical incident, in which the ALEKS Sequence was interrupted. Specifically, interrupted ALEKS Sequences included detours, unplanned events, application problems, or some other observer noted difference. (Tables 11 & 12). Data Analysis SE Indicator 1: Cognitive Interactions DATA SOURCE A: Think Aloud (TA) Recordings Screencast & Pencast Identify ALEKS topics Code for Cognitive Demand of the Task Identify and Transcribe Critical Incidents Identify Phases of Problem Solving (PPS) Identify and Code PPS Activities Data Analysis SE Indicator 2: Academic Interactions DATA SOURCE A: Think Aloud (TA) Recordings Screencast & Pencast Identify and Code Resources Used Identify and Code Study Strategies DATA SOURCE B: End of Course Survey & Interview Student reported Resources Used Student reported Study Strategies Data Analysis SE Indicator 3: Affective Interactions DATA SOURCE A: Think Aloud (TA) Recordings Screencast & Pencast Identify and Transcribe Confidence Statements DATA SOURCE B: End of Course Survey & Interview Student reported Confidence about Mathematics Performance Student reported Usefulness and Value of Mathematics Figure 11. Overview of the data analysis by SE Indicator Categories Table 11. Example of ALEKS Topic and Routinized ALEKS Sequence ALEKS Topic Introduction to simplifying a radical expression with an even exponent. Routinized ALEKS Example Problem 1: Correct Problem 2: Correct Problem 3: Correct New Topic 71 Table 12. Example of ALEKS Topic and Critical Incident ALEKS Sequence (Interrupted) ALEKS Topic Converting between scientific notation and standard form in a real world application. Critical Incident ALEKS Example Problem 1: Incorrect Problem 1 Tried Again: Correct Problem 2: Correct New Topic In general, student thinking processes were more evident during the interrupted ALEKS sequence because students were figuring out and correcting their error(s). For this reason, interrupted ALEKS sequences were identified as critical incidents and transcribed. Another criterion for a critical incident were those times that student solution methods did not directly correspond with the methods presented by ALEKS. In this case, the student brought some personal knowledge to the problem solution method. Lastly, critical incidents were identified as those instances when students made statements related to their confidence about an ALEKS Topic, their work, and/or answer to a problem. Once the critical incidents were identified and transcribed, the problem solving phases (orientation, generation, conclusion) were identified in the lines of the transcript. Next, the student’s mathematical activities that occurred within each of the problem solving phases were identified, described, and later classified as patterns of cognitive interactions in a table (Table 13) with references to the lines of the transcript (see Chapters 5-7, Tables 14-17, & 19). Table 13. Patterns of Cognitive Interactions of Activities in Problem Solving Phases. Patterns of Cognitive Interactions Problem Solving Phases Orientation Activities Generation Activities Conclusion Activities 72 SE Indicator 2: Academic Interactions (Resources & Study Strategies). Academic interactions included study strategies and resources used (SE Indicators 2, Figure 11). There were three sources of data for academic interactions: the think aloud recordings, the end of course questionnaire and interview. In the think aloud recordings, students sometimes explicitly stated their study strategies (see Jade), and other times the strategies were observed. For instance, a student may have been observed taking notes on the ALEKS example and referring back to that example, but the student did not explicitly state that strategy. (See Appendix C). In addition, the end of course questionnaire and interview included questions about student study strategies and resources used to be successful in the course. SE Indicator 3: Affective Interactions (Confidence & Value of Math). Affective interactions included student statements of confidence (or lack of confidence) about the mathematics tasks they were solving, and students’ value of mathematics (SE Indicators 3, Figure 11). There were three sources of data for academic interactions: the think aloud recordings, the end of course questionnaire and interview. The think aloud recordings sometimes contained student statements of confidence before before beginning work on a problem. At times these statements were positive (e.g. “I get this.”) indicating confidence, and at times these statements were negative (e.g. “I have never been good at fractions.”), indicating a lack of confidence. Similarly, after the student had worked out a solution to one of the problems, at times their statements indicated confidence about their solution (e.g. “That seems right.”), and at times students’ statements indicated a lack of confidence about their solution (e.g. “I think that’s wrong.”). Additional information about affective interactions was gathered in the beginning and end of course questionnaires and the associated end of course interviews. These questionnaires and 73 interviews contained questions about the students’ perspectives of their math ability, attitudes toward math, and usefulness of mathematics in their everyday life and future career (Appendices D, E, & F). Cross-Case Analysis The cross-case analysis consisted of two types of analysis. First, a comparative analysis was conducted to better understand and describe the indicators of student engagement and to identify those indicators that varied and those that remained fairly constant. This comparative analysis was followed by a relational analysis. The goal of the relational analysis was “to establish if there are patterns of association within cases that hold true across cases, without losing sight of the particularities of each case” (Bazeley, 2013, p. 285). In other words, I examined SE indicators that varied to see if some type of relationship between those indicators might exist. Trustworthiness and Validity Glesne has written that, “Most [qualitative research scholars] agree that we cannot create criteria to ensure that something is ‘true’ or ‘accurate’ if we believe concepts are socially constructed” (2011, p. 49). In other words, my interpretation of the data has been filtered through my own historical and social lens; however, existing accepted methods can help make the case that my interpretation has been trustworthy. The methods that have been included in this study are prolonged engagement, triangulation of the data, and researcher reflexivity. Prolonged engagement was a part of this study because the participants submitted weekly think aloud recordings throughout the semester they were enrolled in the course. The participants chose when, where, and how long they would conduct their think aloud recordings, so as a researcher, I had no control over what topics they submitted. The data was triangulated between the 74 participant think aloud recordings, extended observed think aloud recording, questionnaires, and interviews. Lastly, I have reflected on and included how my own life experiences have contributed to my interest in this research topic (Creswell, 2003; Glesne, 2011; Maxwell, 1996) Strengths and Limitations The strengths of this qualitative case study is the potential to document and describe the complexity of the phenomena and context under study—in particular to describe and understand the context and interactions between students and CAI deeply and in detail. A generally accepted limitation of qualitative case studies such as this is that these findings cannot be generalized to the wider population. Specific to this study, a limitation was that volunteer participants may not represent the norm. Also conducting a think aloud and screencast recordings may have influenced how the participants engaged with the software and the mathematics. (Creswell, 2003; Glesne, 2011; Maxwell, 1996) Analysis Summary In the next three chapters I use a student engagement conceptual framework (see Chapter 3) and the associated research questions as a guide to present descriptions of the nature of participants’ mathematical engagement in an online CAI Intermediate Algebra course. Drawing on multiple sources of data and using narrative inquiry methods and qualitative coding methods (Appendix B), I present a descriptive case study for each of the three participants to illustrate the nature of student engagement in this online CAI course. These three descriptive case study chapters are followed with a cross-case analysis chapter. The cross-case analysis includes a comparative analysis and a relational analysis. For each case study report, I first provide a brief overview of each participants’ demographic and mathematical history based upon their Math History Questionnaire responses 75 and the subsequent interview. Then I provide an illustration of the nature of each participant’s engagement with ALEKS, the CAI software of the Intermediate Algebra course, based upon data from multiple screencast and pen-cast think aloud recordings, and an end of course questionnaire and interview. The focus of these case study chapters is to provide rich descriptions of the nature of students’ mathematical engagement in this online CAI Intermediate Algebra course, using the following research questions as a guide: a) What is the nature of students’ cognitive interactions within an online CAI intermediate algebra course? b) What is the nature of students’ academic interactions within an online CAI intermediate algebra course? c) What is the nature of students’ affective interactions within an online CAI intermediate algebra course? 76 CHAPTER 5 Jade “Science and math overlap …” When I first met Jade, my impression was that she was a confident, outgoing student. She caught on quickly to all of the technology requirements of participating in the study and seemed pleased that she could make the think aloud recordings independently on her own time schedule. Jade was a consistent participant who submitted her think aloud recordings every Saturday morning. This case study begins with an overview of Jade’s demographic and mathematical history based upon her Math History Questionnaire (MHQ) responses and the subsequent interview. Then I provide an illustration of the nature of Jade’s engagement with ALEKS, the CAI software of the Intermediate Algebra course, based upon data from multiple screencast and pen-cast think aloud recordings, and an end of course questionnaire and interview. The nature of Jade’s engagement is organized around themes of her academic, cognitive and affective interactions with ALEKS. Jade is an African American female from an urban, in-state city. During the study, she resided on campus in one of the dorms. When Jade completed the Mathematics History Questionnaire (MHQ) in the Fall of 2016, it was her first semester as a freshman undergraduate at Michigan State University, and she listed her major as Molecular Genetics. In the MHQ, Jade reported that her high school mathematics experience was a positive one, stating that she “had a great math teacher Sophomore through Senior year. He had an amazing teaching style” (MHQ, #39). When asked to identify a statement that best described her high school mathematics experiences (MHQ, #18, Appendix D), she selected the statement: “My high school teacher primarily provided activities in which students worked together in small groups to learn mathematics, followed by whole class discussion.” Prior to her enrollment at 77 MSU, Jade had excelled academically in mathematics. She took algebra in eighth grade and four years of mathematics in high school, culminating with calculus her senior year. Despite this success, Jade stated that taking Algebra in eighth grade was detrimental to her high school mathematics learning. Jade: What messed me up was that I had Algebra as an 8th grader and my Algebra teacher skipped over a lot of the basic stuff that I needed for the rest of my high school career. Instead of learning in 9-12th grade, I was playing catch up because I missed a lot of the basic stuff. (MHQ, #39) In addition, although Jade had completed through calculus in high school, she rated her own mathematics ability as “average” on the MHQ. Her questionnaire responses also indicated that she felt the MSU mathematics placement test she had taken online was “probably” a good indication of her knowledge of mathematics. In addition, Jade responded that although she had “previous exposure to the topics in this course,” she believed she “needed a refresher experience before proceeding to college-level mathematics.” In the quote below, Jade also expressed indifference to mathematics as a subject and seemed to believe she did not retain mathematical learning because she does not find mathematics interesting. Jade: Math has never been a subject that I’ve liked or disliked. It’s just always been a course I am required to take. I do okay in math. I just am not very good at retaining what I’ve learned because the subject isn’t really interesting to me. I’ve taken all the way up to high school calculus, but I still have no interest in the subject. (MHQ, #1) However, in contrast to her indifference about mathematics, Jade became quite animated when discussing her major, Molecular Genetics, her job shadowing experiences, the competitive nature of the program and graduate school requirement, and her future career as a genetics counselor. Thus, it seemed as if Jade views mathematics merely as a “requirement,” or at most a required tool, for achieving her larger educational and career goals. 78 When I first met with the Jade, we reviewed the research consent form and process, and discussed her school experiences while we loaded the software on her computer. In addition, we discussed what it means to perform a think aloud, as well as the purpose of the think aloud, which was that I might have an idea of what she was thinking as she solved mathematics problems. Next, I left the room as Jade practiced using the screencast and pen-cast software and rehearsed her first think aloud. After about 15 minutes, I re-entered the room, I watched as Jade uploaded the videos, and then she and I reviewed her think aloud. During this first think aloud, Jade focused on her academic interactions, in other words, her study habits, in regards to the CAI intermediate algebra course. This was a key aspect of Jade’s engagement with ALEKS, so I begin by foregrounding Jade’s academic interactions. Jade’s Academic Interactions During her first think aloud rehearsal, Jade explicitly discussed her study routines as she studied the examples and solved problems presented in ALEKS. Jade’s stated study routines included note-taking conventions and utilizing a unique memory practice. Her academic interactions are illustrated below in Figures 12 and 13. In these figures, a screen shot of the recorded pen-cast is central to the figure and surrounded by transcription of Jade’s utterances as she wrote. Jade’s Notetaking Jade kept organized notes as she worked in ALEKS, beginning with the section title and example (Figure 12) and followed by numbered problems that she solved (Figure 13). As Jade began a new ALEKS topic, she wrote notes about the ALEKS example. In fact, her notes and utterances clearly indicated the example, and although Jade had not clicked the record button on the screencast, her pen-cast recording and notes were sufficiently clear and easy to follow. 79 Figure 12. Jade Think Aloud 1, Example First, Jade wrote the title of the section in her notes so she could refer back to the section at a later time if needed. Jade: The problem I am doing is Complex Fractions Involving Quadratics with Leading Coefficients less than 1. So I always write the title of the section so if I go back to look at my notes, I know what I was doing (Figure 12, transcript lines 1-8 and top two lines of inscriptions). Next, Jade also wrote the ALEKS Example problem as it was presented in ALEKS and included the direction to “simplify” (Figure 12, third line of inscription). Jade: First, I write the problem as is, without changing anything (Figure 12, transcript lines 9-10) 80 In Figure 12, transcript lines 18-22, I assumed that Jade was reading from the ALEKS screen due to her change to a more rigid tone intonation, which switched to a more informal tone in lines 23-24 when she stated she would write the steps as shown. Jade: We multiply the numerator and denominator by 2u to cancel the 2u in the denominator of the complex fraction. I’m gonna write that step like they show it (Figure 12, transcript lines 18-22). Jade appeared to rely on the solution path that was presented in ALEKS. In fact, Jade had begun to take a different solution path that may have been correct, but noticed the difference from ALEKS example and appeared to modify her approach to match the solution shown in the ALEKS. Jade: So then you distribute the 2u to both the numerator and denominator so you get 6u minus… Wait. Didn’t they multiply? Oh. No. Never mind. They don’t want you to distribute just yet (Figure 12, transcript lines 25-31). Jade paid close attention to the ALEKS examples. For instance, Jade spent a little over four minutes reading, studying, taking notes, and describing her thinking about this ALEKS example before proceeding to the first problem. In addition, Jade made a point to keep “all of the rules and properties” that were presented in ALEKS organized in her notes as well (Figure 15, transcript lines 23-27), indicating that Examples and “rules and properties” were important content to document in her notes. 81 Figure 13. Jade Think Aloud 1, Problem 1 As Jade completed the problems for each ALEKS topic, she wrote the problem number (Figure 13, transcript lines 42-43 and first line of inscriptions). Second, Jade copied the directions and the problem as they appeared on the screen, followed by each step of her solution. She followed this process of documenting her work for the second problem, and in her other think aloud recordings of ALEKS work as well. In summary, Jade used this systematic method of titling sections, carefully copying examples, and numbering the problems she solved to organize her notes for future reference, “so if I go back to look at my notes, I know what I was doing” (Figure 12, transcript lines 6-8). A sample page of Jade’s systematic organization of her notes has been included in Appendix J. Jade’s Memory Strategy After solving the first problem correctly, Jade mentioned the use of a unique memory strategy (see quote below), which include writing out the solutions to the first two problems, and then attempting to do the third problem mentally, in other words without writing anything down. 82 Jade’s use of this memory strategy was demonstrated in all of her recorded sessions that involved solving ALEKS homework problems. Jade: I only write down—work out—two practice problems, and the third one I try to do in my head. But that doesn’t always work out (Think Aloud 1). Jade’s Planning In her second think aloud recording and in the end of course interview, Jade mentioned another study strategy, a planning strategy in which she described dividing up the number of ALEKS topics assigned for the week into chunks of work to be completed each day. Jade: I divide the number of [ALEKS] topics I need to do up, so I'm going to do 10 today, 10 Monday, 10 Tuesday, 10 Wednesday, 10 Thursday, and 6 Friday. (Think Aloud 2) Jade: Academic Interactions Summary In summary, Jade used at least three academic strategies to learn in the online CAI environment. She systematically organized her notes and homework, utilized a unique memory technique, and divided up the weekly number of assigned ALEKS Topics into daily chunks. Jade appeared to rely solely on the mathematics resources provided by ALEKS for the course. Jade’s Cognitive Interactions To explicate Jade’s cognitive interactions, I present and analyze excerpts from the transcripts of two think aloud recordings on two separate ALEKS Topics. I parsed the transcripts into Problem Solving Phases (orientation, generation, and conclusion) as discussed in the conceptual framework in Chapter 3 and operationalized in the methods in Chapter 4. 83 Jade: Think Aloud Recording 1 (Figures 12, 13, & 14) In this section, I revisit Jade’s first think aloud recording to examine and understand her cognitive interactions. Jade began the problem solving process by reading the example problem and taking notes (see Figure 12)—the orientation phase of problem solving. Next she read and copied the exercise problem to be solved, which is a continuation of the orientation phase (Figure 13, transcript lines 44-45 and inscription). The generation phase of problem solving, which was comprised of imitating the solution to the problem based on the process of the example problem (Figure 13, transcript lines 46-64 and accompanying inscriptions). The problem solving activities in this generation phase were designated as reproduction due to the evidence in the existing think aloud itself, plus the following excerpt from the video recording as we watched and discussed this first think aloud pen-cast. Jen: You are doing a really great job of describing what you are doing. I am curious about why you can cross out those two [pointing to the crossed out 3w] …like why you do that. Jade: For the example is, said that you have to multiply the numerator and denominator by the complex fraction denominator. So I just went in and crossed them out because they have common numbers, I mean factors, I guess. So I can cross them out. I’m not really good at explaining it. I just know how to do it. So you just cross off the common numbers. I know you can’t cross off those [pointing to the screen], but I am not sure how to say it. Jen: Which ones? Jade: [pointing to the screen] The 2 times the 3w and the 5 times the 3w. Jen: OK. Thanks. Well, I really appreciate you doing these recordings, and you’re doing a great job of explaining what you are doing. If you would add the “why” you do something, or even if you’re not sure “why” that would be helpful. Jade: O.K. In addition, in the recording of Problem 2, Jade was not confident about her answer, and questioned whether it was correct (Figure 14 below, transcript line numbers 88-99). Both of these 84 interactions suggested that Jade was reproducing procedures without understanding why those procedures were correct, or not correct. Jade’s cognitive interactions in the form of Problem Solving Phases and associated activities are summarized in Table 14 below. Figure 14. Jade Think Aloud 1, Problem 2 Although Jade’s cognitive interactions with this particular ALEKS Topic provided a typical example of what is characterized as surface level knowledge, it is important to note that some recordings of her cognitive interactions with other ALEKS topics were different. In the next section, I illustrate how Jade’s cognitive interactions with a different ALEKS Topic varies from this one. 85 Table 14. Patterns of Cognitive Interactions in Jade's Think Aloud 1 (Figures 12, 13, & 14) Patterns of Cognitive Interactions ALEKS Example(s) &/or Routine Exercises (Figure #) Imitate the Steps ALEKS Example (Figure 12) • Read & Copy Example (1-41) o Backtracking (28-31) Problem Solving Phases Orientation Activities (Transcript Lines) (Transcript Lines) (Transcript Lines) Generation Activities Conclusion Activities Imitate the Steps Problem 1 [Correct] (Figure 13) Imitate the Steps Problem 2 [Correct] (Figure 14) • • • Read & Copy Problem (42-45) • • • Read & Copy Problem (75-77) • • Jade: Think Aloud Recording 4 (Figures 15 & 16) • • • Imitate the Steps (46-62) • • • Imitate the Steps (78-89) • Verify o External Authority (63-69) o • • Verify • o External Authority (63-69) o The analysis of the evidence of Jade’s cognitive interactions with the ALEKS Topic, Solving an equation of the form x2=a using the square root property, can be characterized as further along the surface-to-deep continuum. I illustrate this by presenting and describing the evidence of Jade’ cognitive interactions in Think Aloud 4. 86 Figure 15. Jade Think Aloud 4, Example Jade took an active role as she read the example for this ALEKS Topic by making a prediction about the answer before reading the example solution method (Figure 15, transcript lines 14-17). After making the prediction, she continued to read the example, and again made a prediction about the solution based on what she had read (Figure 15, transcript lines 28-29). Jade continued reading and taking notes from the example and the answer that was provided. Notably, before proceeding to work on the first exercise, Jade also exhibited an active role as she verified 87 the example solution mathematically, even though this step was not modeled as a step in ALEKS. Jade: And I know this is true because -5 times -5 equals positive 25, and 5 times 5 is a positive 25 (Figure 15, transcript lines 34-35). So Jade autonomously brought this activity to her conclusion problem solving phase for this ALEKS topic. In a manner similar to the example, when solving the first problem and during the orientation phase of problem solving, Jade made a prediction about the solution (Figure 16, transcript lines 41-45). Next, she recreated the solution process during the generation phase of problem solving. During the conclusion problem solving phase, she verified her solution mathematically before entering it into ALEKS (Figure 16, transcript lines 51-55). Figure 16. Jade Think Aloud 4, Problem 1 88 Comparing the summaries of Problem Solving Phases and activities in the two think aloud recordings (Table 14 above & Table 15 below) reveals that Jade engaged in different activities during Think Aloud 4. These activities were predicting the solution and verification of the solution using substitution, which indicated Jade’s internal authority for understanding and the solution procedure because neither of these activities were presented in the ALEKS Example. Problem Solving Phases Table 15. Patterns of Cognitive Interactions in Jade's Think Aloud 4 (Figures 15 & 16) Patterns of Cognitive Interactions ALEKS Example(s) &/or Routine Exercises (Figure #) Transcend the Procedure ALEKS Example (Figure 15) Generation Activities (Transcript • Verify Orientation Activities (Transcript Lines) Orientation Activities (Transcript Lines) o External Authority (33) o Internal Authority *Substitution (34-35) Lines) • Read & Copy Example (1-33) • Understanding *Predicting (14-17) • • Read & Copy Problem (39-43) • Understanding *Predicting (41-45) • Transcend the Procedure Problem 1 [Correct] (Figure 16) • • • Recreate the Procedure (45-51) • • Verify • o Internal Authority *Substitution (51-55) o External Authority (56-57) * Note: ALEKS did not present predicting a solution or a verification procedure. Jade’s Affective Interactions Here I discuss the Jades affective interactions with the online ALEKS as evidenced by her statements related to confidence, or conversely, a lack of confidence during think aloud recordings and interviews. In addition, I examined the data for evidence regarding how Jade’s valued mathematics. 89 Jade’s Stated Confidence Jade expressed confidence about the mathematics she was working only a few times. One time, Jade made a statement that indicated a lack of confidence in her solution after she solved an ALEKS problem about simplifying complex rational expressions. Jade: I know I can’t cross cancel anything, but I’m not sure if I can add the one or subtract the one or should I just leave it as 7v +1 over 3v minus 1. I think I’m gonna leave it like that, but it could be wrong. I think it’s wrong. [enters answer] No, it’s right! (Think Aloud 1, Figure 14, transcript lines 89-98) Jade expressed confidence before beginning to work on a set of problems in a different ALEKS Topic, solving an equation of the form x2=a using the square root property. Jade: My thoughts as I go forward are that this one section right here will be pretty easy. Umm. Because I feel like it’s just basic square root stuff. (Think Aloud 4, Figure 16, transcript lines 36-38) Jade also expressed confidence in her solution to these problems when she mathematically verified her answers before entering the answer into the CAI program. Jade: And I know this is true because -5 times -5 equals positive 25, and 5 times 5 is a positive 25 (Think Aloud 4, Figure 15, transcript lines 34-35). Thus, Jade’s statements indicating confidence outnumbered her statements that indicated a lack of confidence. Jade’s Value of Mathematics and Course Jade offered an interesting view regarding the value of mathematics. She saw the value in everyday mathematics and believed that formal mathematics may likely play a role in her future career in Molecular Genetics, but she remained indifferent to school mathematics. She neither enjoyed or disliked math. It just seemed to be necessary. Jade: Mathematics is in everything we do such as how many calories we eat, how long we sleep, how many hours we study for an exam, and so on. (ECQ, #12) 90 Jade: Genetics is a science. Science and math overlap in many areas and I am sure Genetics and Genomics are no different than any other science major when it comes to math. (ECQ, #10) Jade: I don’t hate math because it is necessary for day to day life, but I don’t enjoy it beyond class. (ECQ, #8) Jade explicitly mentioned that she liked working in the online CAI Intermediate Algebra course, but that she would have preferred the option of a hybrid course. Jade: Math 1825 was a great experience. It helped me identify my strong suits and weak points when it comes to me learning math. I found that I benefited a lot from the online portion of the class. I liked that I could set my own pace for learning the material. I also found the different tools the online portion offered…The one thing I didn’t like was that the class was 100% online. I wish it took the hybrid form, a combination of online and in-class instruction. There were times that I needed that face to face interaction with a professor or teaching assistant. (ECQ, #3) In fact, in the end of course interview, Jade mentioned that she liked the CAI instructional model better than the large lecture College Algebra course she was taking Spring semester. She felt that College Algebra moved too fast and she did not like how the online homework functioned (Web Work) in comparison to ALEKS. Summarizing Jade’s Engagement Based on what I observed, Jade put a lot of time and effort into learning the content of the CAI intermediate algebra course, and her efforts in the class paid off. She earned a 3.5 grade; however, I note Jade’s final grade with the caveat that there is not sufficient evidence to make the claim that her effort and strategies are directly related to her grade in the course. Jade consistently demonstrated academic interactions that were comprised of three study strategies: note-taking, a unique memory strategy, and a planning strategy. Jade used a systematic, organized method of note-taking: titling sections, carefully copying examples, and numbering the problems she solved to organize her notes for future reference, “so if I go back to 91 look at my notes, I know what I was doing” (Figure 12, transcript lines 6-8). Jade’s memory strategy consisted of solving the first two ALEKS problems in writing and solving the third problem mentally. Her use of systemic, organized note-taking and the aforementioned memory strategy was consistent in all of her think aloud recordings. Lastly, Jade planned her study time for the week by dividing up the number of assigned ALEKS topics into manageable amounts to be completed each day. The primary resources she used to learn the content were those available within the ALEKS environment, such as the ALEKS examples, written explanations, and videos. She did not report using the university Mathematics Learning Center. Jade’s cognitive interactions varied more than her academic interactions. Her cognitive interactions with the mathematics of different ALEKS topics fluctuated along the surface-to-deep continuum. This variation may have been related to her feelings of confidence about the mathematical topic. For instance, in her first think aloud recording, Jade was not confident about her work, and she closely followed the ALEKS example procedures and demonstrated surface learning characteristics (Think Aloud 1, Table 14, Figures 12-14). In contrast, in her fourth think aloud recording, Jade expressed confidence about a topic, and she went beyond the information and procedures presented by ALEKS. Jade made predictions about problem solutions and mathematically verified her solutions before entering them into the ALEKS software (Think aloud 4, Table 15, Figures 15 & 16), thus demonstrating a deeper cognitive interaction with that ALEKS topic. Despite her reported beliefs about the value and usefulness of mathematics in everyday life and her future career, Jade was indifferent about the study of mathematics. She seemed to view her study of mathematics as a requirement and a tool related to her larger academic and career goals related to molecular genetics. 92 Chad “When you fail, you pick back up and do it again.” CHAPTER 6 When I first met Chad, my impression was that he is an outgoing person, and my observation was that he talked expressively about his mathematical experiences. In addition, Chad chose to meet with me on a weekly basis to conduct his think aloud recordings. Although he still would perform those think aloud recordings alone and in a separate space, Chad felt he would be more likely to follow through if he had a scheduled meeting time. Each week, we would chat briefly, then I would leave, and he would begin his recording. This case study begins with an overview of Chad’s demographic and mathematical history based upon his Math History Questionnaire (MHQ) responses and the subsequent interview. Chad was a study participant for two semesters, and because he did not pass the course his first semester, I provide a brief overview of his first semester in the course according to data from his End of Course Interview. Next I provide by more detailed exposition of his second semester in the course. In the latter, I provide an illustration of the nature of Chad’s engagement with ALEKS, the CAI software of the Intermediate Algebra course, based upon data from multiple screencast and pen-cast think aloud recordings, and an end of course questionnaire and interview. The nature of Chad’s engagement is organized around themes of his academic, cognitive and affective interactions with ALEKS. Chad is a White male from a suburban, out-of-state city. During the study, he resided on campus in one of the dorms. When Chad completed the Mathematics History Questionnaire the 2016 fall semester, it was his first semester as a freshman undergraduate at Michigan State University, and he listed his major as Journalism. In the MHQ, Chad reported that he disliked 93 math and believed he was a weak math student. In the quote below, Chad expressed that he may not like math because he had never been good at it. Chad: I have never been very good at math, and it has colored my perception of it. I think perhaps if I had always been good [at math], I would like it a lot more, and there would be a positive feedback cycle. (MHQ, Question, #8) When asked to identify a statement that best described his high school mathematics experiences (MHQ, #18, Appendix D), he selected the statement: “My high school teacher both lectured to the class and provided activities in which students worked together in small groups to learn mathematics, followed by whole class discussion.” Prior to his enrollment at MSU, Chad had taken four years of high school mathematics including Pre-Calculus his senior year. In addition, Chad’s MHQ responses also indicated that he felt the MSU mathematics placement test he had taken online was “definitely” a good indication of his knowledge of mathematics and had he reviewed for the placement test or taken it a second time his score would “probably not” have improved. Chad responded that although he had “previous exposure to the topics in this course,” he believed he “needed a refresher experience before proceeding to college-level mathematics.” In the sections that follow, I use quotes from Chad to illustrate his experiences in the course Fall Semester 2016 (FS16). Then I follow this with a more in-depth discussion of his experiences Spring Semester 2017 (SS17) using the student engagement framework ideas of cognitive, academic and affective interactions. Lastly, I provide a summary of this case study. A Series Unfortunate Events (FS16) I first met Chad on October 26, 2016. We met to discuss the possibility of his participation in the study. Chad agreed to participate in the study. We proceeded with the math history questionnaire review and interview, set up the screencast and pen-cast software on his 94 computer, and Chad did a test run in ALEKS with both software. The meeting seemed to have gone well, but Chad never submitted any recordings in the fall semester of 2016. At the end of that semester, when Chad returned the LiveScribe pen, he was very talkative about his experiences and challenges. I asked if I could record the discussion, and Chad agreed. I learned that Chad had not realized the importance of the first ALEKS knowledge check, or the pre-assessment. He said that he did not take the ALEKS knowledge check seriously and responded with “I don’t know” to the questions that he was unsure about but perhaps could have answered correctly if he had tried. As a result, ALEKS assigned him a large number of topics to learn, and Chad reported that he began the course feeling overwhelmed by the number of ALEKS topics he needed to complete. In addition, I learned that Chad began having difficulty in the course shortly after our first meeting because the course content had become more challenging for him. This was compounded by the fact that the university had a data breach early in November of 2016 (Michigan State University, 2016), and Chad was one of the victims. As a result, Chad reported that he was unable to access his MSU accounts. Chad: So my technology problems coincided with the fact that the math problems got harder. But my technology problem was not the program thing [ALEKS]. There was a data breach at MSU and the whole thing got weird. I couldn't access my email or accounts for over a week. (End of Course Interview, FS16) Chad had basically given up and quit doing any work in his intermediate algebra course because “it felt like an un-scalable mountain” (Chad, Personal Communication, June 2, 2017). Also, another of Chad’s quotes indicated a feeling of being overwhelmed by how the intermediate algebra course utilized ALEKS. Chad: (FS16) The ALEKS program is weird to me because it really makes it daunting. . .. The number of topics is overwhelming. There are a lot. I wish there were less problems and maybe the problems were more complicated. Does that make sense? . . . So 75 problems, even if they are sort of simple, is a lot, to the point that you start evading it. (End of Course Interview, Fall Semester 2016) 95 In addition, as noted in Chad’s first statement in the communication excerpts below, it seemed that in this situation, the online feature of the course and the perceived remoteness of the course coordinating professor posed a barrier. Here Chad said that there was no easy way to get help from his professor, and because he had never met his professor, it was also “a tough and awkward situation.” Chad: There was no immediate or easy way to get help from my professor, since again, I never met him. So it was a tough, awkward situation. But yeah, I basically just stopped. I didn't know how to do it, and I didn't feel like I could. Obviously looking back, I could have . . . but in the moment, I felt like I couldn't. I’m not exactly proud of it, but whatever. I think I get it. It sounds like a set of circumstances made it feel impossible . . . So, to follow up—how did you decide to take it again right away spring semester? Jen: Chad: To be honest, I didn't really consider any other option. First of all, you need to take a couple of math classes to even be able to take certain science classes, and I did not want to—or even consider the possibility of—having a math and science- based senior year. To say that would not go well is an understatement. Look, I knew I could do it and I was just being lazy, so it never crossed my mind to not take it immediately. Also, shame and my ex-girlfriend's 4.0 at [another university] I'm sure had something to do with it [laughs]. In my background, when you fail, at anything, you immediately pick back up and do it again. When I was learning to drive, I wasn't good at first—obviously. But my dad took me out till midnight or later making sure I understood how to do it. When you fail, you pick back up and do it again. In the above communication, Chad responded that he immediately registered again for the math class because he did not want to be taking math and science classes his senior year. This statement could be interpreted as a type of strategic academic planning and forethought. He also expressed confidence, a positive emotion, that he could take and pass the course, “I knew I could do it.” Yet he also expressed “shame,” a negative emotion, about the fact that he had failed the course. Lastly, he elaborated on how his “background” influenced his decision to immediately re-enroll in the course spring semester 2017. Because the example Chad used to describe how his background influenced this decision included an experience with his father, I interpreted his use 96 of the word “background” to pertain to his family and family support during past instances of failure. This reference to background and family serves as an example related to the structural influences on student engagement from Kahu’s socio-cultural framework (see Chapter 3, Figure 7). I interpreted Chad’s reference to this past experience and his statement, “When you fail, you pick up and try again.” to indicate a belief that failure was not an indication of a lack of ability to succeed and was, in the words of Dweck (2009), a “growth mindset.” In summary, Chad did not pass the intermediate algebra course FS16 for several interrelated reasons. First, the mathematics content became more difficult and at about the same time he had technological problems due to the MSU data breach. He got behind in the coursework and did not feel he could catch up. Chad mentioned that he did not feel comfortable contacting the professor about his learning challenges and technology issues, perhaps due in part to the intangible virtual presence of the professor of the online course. So Chad simply gave up on math and focused on his other courses. However, it seemed that Chad’s family background of persistence provided the impetus for him to immediately re-enroll in the course. Chad’s Cognitive Interactions (SS17) Not only did Chad immediately enroll in the course again, but he changed his approach to the assignments in ALEKS, as noted in the quote below. Chad: [In FS16,] I would like just copy the example problem and then look at the example problem three times so I could do it three times to get through the topic and, OK move on to the next topic. But then as soon as I was done with that topic, it was gone. Does that make sense? Jen: Uhm, yeah. Chad: So I wasn't cheating or anything, but it was like I wasn't getting very much out of it. I was just like, ‘OK same type of problem, same type of problem, same type of problem.’ Whereas now [SS17] I'll look at the example problem, but I'll try to do it without the example problem and I'll refer back to the example problem rather than just trying to just plug in numbers and not learn very much from it like I did last semester. (Chad, End of Course Interview, SS17) 97 In addition, Chad claimed this change in his approach to ALEKS homework was a major contributing factor for why he passed the second time he took the course. He also said, “I got less technically done, but it was more retained. And I could see that in my test scores” (Chad, End of Course Interview, SS17). Here I summarize and analyze what Chad said about his different approaches to his work in ALEKS FS16 and SS17. In FS16, Chad had copied the ALEKS Example problem and then referred back to the example to imitate each step as he worked on each of the three following problems. His goal was to get the work done as quickly as possible. In contrast, in SS17, Chad would still read and write the ALEKS Example problem, but then would duplicate the procedure, only referring back to the example if needed. This description illustrates a subtle difference in Chad’s approach to learning mathematics procedures, and I argue that imitating each step and duplicating the procedure are at different places along the surface-to-deep continuum of cognitive interactions. Also, I argue that duplicating is somewhat deeper than imitating. Imitating, copying an example and then plugging in numbers from the problems, might require some recognition and matching of symbolic patterns, but other than that, does not require much cognitive effort. Duplicating the procedure also includes copying an example, but differs in that duplicating requires a memory retrieval of the example symbolic pattern, thus requiring more cognitive effort. Although duplicating may be somewhat deeper than imitating, it remains near the surface in the surface-to-deep continuum of learning. In the next two sections, I provide excerpts of data from two different think aloud recordings of Chad (Think Aloud 2 and 6). Also, I provide an analysis of these two think aloud recordings to illustrate how Chad’s cognitive interactions varied along the surface-to-deep continuum based on the ALEKS topic and his confidence, or lack of confidence, about the topic. 98 Chad: Think Aloud Recording 2 (Figures 17-21) In Chad’s second think aloud recording he solved problems about the ALEKS topic, Finding the original price given the sale price and percent discount. This recording began with an ALEKS example solution for the problem: ALEKS: Amy bought a suit on sale for $589. This price was 62% of [emphasis added] the original price. What was the original price? (Figure 17) As noted in Figure 17, Chad copied the example problem and solution, and appeared to feel confident about solving these types of problems. “That makes sense, I think we can apply that …” (Figure 17, Transcript line 6). Figure 17. Chad Think Aloud 2, Example 99 However, the first problem Chad encountered was worded differently than the example. ALEKS: At a sale this week, a desk is being sold for $133.40. This price is a 71% discount from [emphasis added] the original price. What was the original price? (Figure 18) Chad followed the same procedure as the example problem without noticing the change in wording of the problem. But he did use proportional reasoning to check the reasonableness of his answer and determined that his answer did not make sense (Figure 18, Transcript lines 17-24). Despite his sense the answer was incorrect, he entered the answer into ALEKS anyway “to see what it says”. Figure 18. Chad Think Aloud 2, Problem 1 Solution ALEKS provided the feedback that his answer was incorrect, and Chad chose to examine the ALEKS explanation for the problem (Figure 19). At this point, he noticed that the procedures ALEKS presented in this problem explanation were different from the prior example procedures. 100 A key here was Chad said, “I wonder how to differentiate that” (Figure 19, Transcript line 35). This statement suggested that Chad may have been trying to make a connection between the solution procedures and the context and/or words of the problem. Figure 19. Chad Think Aloud 2, Problem 1 ALEKS Explanation The next ALEKS problem was again worded differently, perhaps with the goal of encouraging the students to understand what the problem asks rather than just pull out the numbers and plug them into a predetermined formula. ALEKS: A sofa is on sale for $213.20, which is 74% less than [emphasis added] the regular price. What is the regular price? (Figure 20) Chad solved this problem correctly, but it was not clear from what he said whether he was making sense of the problem or imitating the prior ALEKS explanation. In addition, it was not clear whether Chad used proportional reasoning to check his solution because he only said, “820 dollars. That seems plausible” (Figure 20, Transcript line 48) and did not include any statement that indicated associated proportional reasoning. Chad may have used incidental knowledge 101 based on his past experience of what mathematics answers have typically looked like when he said, “Plus, it’s a round number” (Figure 20, Transcript line 49). Figure 20. Chad Think Aloud 2, Problem 2 Chad appeared to achieve greater understanding during his solution of the third problem. In this problem, Chad made a connection between the language of the problems and the calculation of the solution. Chad: If it’s [writing] 76% of [original verbal emphasis] then it’s 24% off [original verbal emphasis] (Figure 21, Transcript lines 53-58). In addition to this connection, Chad used proportional reasoning to verify his solution with the statement, “Because it’s only going to be about one fourth off” (Figure 21, Transcript line 63). 102 Figure 21. Chad Think Aloud 2, Problem 3 Chad’s cognitive interactions with this ALEKS topic are notable because the problem solving activities he engaged in appeared to fluctuate along the surface-to-deep continuum as he made sense of how to solve the problems. A summary of Chad’s problem solving activities has been provided in Table 16 below, and bolded italicized text indicates deeper problem solving activities. As discussed in the prior paragraphs, Chad often used proportional reasoning to estimate whether his solution to the problem made sense. As Chad read the ALEKS explanation for Problem 1, he wondered how to differentiate the two problems and how the problems were 103 solved. As he worked on Problem 3, Chad made connections between the wording of the problems and the mathematical concepts they represented when he said, “If it’s 76% of then it’s 24% off.” Chad’s cognitive interactions stabilized (as opposed to fluctuate) at a deeper level as he then used this connection to reason through the solution to Problems 3 and 4. Orientation Activities (Transcript Lines) Problem Solving Phases Generation Activities (Transcript Lines) Table 16. Patterns of Cognitive Interactions in Chad's Think Aloud 2 (Figures 17-21) Patterns of Cognitive Interactions ALEKS Example(s) &/or Routine Exercises (Figure #) Imitate the Steps ALEKS Example (Figure 17) Imitate the Steps AND Decipher the Procedure Problem 1 [Incorrect] (Figures 18 & 19) • Read & Copy Problem (8-11) • • • Read & Copy Example (1-7) • • • • • Imitate the Steps (12-16) Orientation Activities (Transcript Lines) * Note: ALEKS did not present estimation using proportional reasoning or connections between the language in the problem and the procedures used to solve the problem. 104 UNCLEAR Imitate the Steps OR Decipher the Procedure? Problem 2 [Correct] (Figure 20) Transcend the Procedure Problem 3 [Correct] (Figure 21) Transcend the Procedure Problem 4 [Correct] • Read & Copy Problem (39) • • • Read & Copy Problem (52) • Understand • Analyze *Connection (53- 58) • Read & Copy Problem • Understand • Analyze *Connection • • • Imitate OR Reproduce? (40-46) • • • Recreate (59-62) • • • Recreate • Verify o Internal Authority *Estimation (17-24) o External Authority (25-27) o Read Explanation (28-38) • Reflection (Figure 16) *Wondering (35) • Verify o Internal Authority? (47-49) o External Authority (50-51) • Verify • • Verify • o Internal Authority *Estimation (63-64) o External Authority (65-68) o Internal Authority *Estimation o External Authority Chad: Think Aloud Recording 6 (Figures 22-25) Chad began his sixth think aloud recording on the ALEKS topic, Power and quotient rules with negative exponents: Problem type A, by stating a disclosure, “I’m not good at exponents.” He then pointed to the orange bars in the upper right hand corner of the ALEKS window to indicate that he had been having difficulty with this topic (Figure 22). Note that these bars turn yellow, orange, and then red as a student enters incorrect answers. Chad’s statements as he works on this problem indicate that he does not know the property or understand the meaning of a negative exponent (Figure 22, Transcript lines 16-21). He ends his solution with the statement, “If it’s wrong, which I expect it to be, I’ll just go to the explanation.” Chad's written work. LiveScribe Pen did not capture this step. Figure 22. Chad Think Aloud 6, First Topic, Problem 1 105 Chad’s answer was incorrect, so he read the explanation, took notes, and solved the next three problems correctly. However, he continued with another disclaimer, this time for his success, “Alright! Well, I don’t know what happened, but I got it right” (Figure 23, Transcript lines 72-73). Figure 23. Chad Think Aloud 6, First Topic, Problem 4 Chad’s three consecutive correct answers moved him forward to the next ALEKS topic, Power and quotient rules with negative exponents: Problem type B, which began with an explanation of how to solve this new problem type, a more complex problem. Upon seeing the example, Chad exclaimed, “Oh, my God.” He sat back in his chair, laughed, waved his hands back and forth, and said, “It’s a wrap” (Figure 24, Transcript lines 1-3). But he soon got more serious and read and copied the example problem, stating at the end, “It looks semi-doable” (Figure 24, Transcript line 32). Chad proceeded to work through the first problem, but his solution was incorrect. He chose to read the explanation, but did not copy the explanation down as he had for earlier problem explanations. Next, he read the second problem, but it was a 106 slightly different exponent problem than the original example and first problem, so Chad did not know how begin. He decided to stop the recording and to ask for help (Figure 25). LiveScribe Pen did not capture these steps. Figure 24. Chad Think Aloud 6, Second Topic, ALEKS Example Figure 25. Chad Think Aloud 6, Second Topic, Problem 2 107 Chad’s cognitive interactions, Problem Solving Phases and the mathematical activities occurring within those phases, are summarized in Table 17 below. Chad unsuccessfully attempted to reproduce the solution to the Power and quotient rules with negative exponents in Problem 1, but after reading and copying the ALEKS explanation he was able to solve the next three similarly structured problems (Figures 22 & 23, & Table 17, Topic Type A, Problems 2-4). However, once the problem structure changed slightly, Chad was unable to transfer the skills from the prior problem type to a more complex problem (Figure 24 & Table 17 Topic Type B) or a new problem structure (Figure 25). This indicated surface level cognitive interactions. Table 17. Patterns of Cognitive Interactions in Chad's Think Aloud 6 (Figures 22-25) Patterns of Cognitive Interactions ALEKS Example(s) &/or Routine Exercises (Figure #) Problem Solving Phases Orientation Activities Generation Activities (Transcript Lines) (Transcript Lines) Orientation Activities (Transcript Lines) • Verify o External Authority (24-27) o • Reflect o Read & Copy Explanation (28-32) • Verify o External Authority (42-44) o • • Verify o External Authority (57-62) o • • Verify o External Authority (70-74) o • • • • Imitate Steps (12-24) • • • Imitate Steps (36-42) • • • Imitate Steps (47-56) • • • Imitate Steps (65-69) Imitate the Steps Topic Type A, Problem 1 [Incorrect] (Figure 22) Imitate the Steps Topic Type A, Problem 2 [Correct] Imitate the Steps Topic Type A, Problem 3 [Correct] Imitate the Steps Topic Type A, Problem 4 [Correct] (Figure 23) Imitate the Steps Topic Type B, ALEKS Example (Figure 24) • Read & Copy Problem (10-11) • • • Read & Copy Problem (33-36) • • • Read & Copy Problem (45-46) • • • Read & Copy Problem (63-64) • • • Read & Copy Example (1-32) • • 108 Table 17. (Cont’d) Imitate the Steps Topic Type B, Problem 1 [Incorrect] Stuck àAsk for Help Topic Type B, Problem 2 [No Attempt] (Figure 25) • Read & Copy Problem (33-39) • • • Read Problem (74-80) • • • Imitate Steps (40-56) • Verify o External Authority (56-59) o • Reflect o Read Explanation (60-73) Chad’s Affective Interactions (SS17) Chad displayed a tendency to freely express his feelings as well as his thoughts as he worked in ALEKS, noticeably celebrating when he got a problem correct (e.g. loudly saying “Alright!” and smiling and/or fist raised). In addition, Chad may have used humor as a coping mechanism when he was having difficulty with an ALEKS topic (e.g. see Figure 24, Transcript lines 1-5, & 18-20). He also freely expressed his feelings and opinions in the interviews. Chad’s Stated Confidence Chad readily revealed his belief that he was generally “not good at math.” This theme came up in his Math History Questionnaire and FS16 End of Course Interview. In this quote, Chad: I know I don’t have a math brain. I’ve always been good at the humanities, but I’m not good at math. (End of Course Interview, FS16) Chad also expressed a belief in innate mathematical ability, a “math brain,” and his lack in this area, but strength in the humanities. It seemed that Chad deeply held this belief that he was not good as mathematics, because it also was a theme in his SS17 think aloud recordings (e.g. see Figure 22, Transcript lines 1-4), as well as his SS17 End of Course Questionnaire and Interview. Chad’s Value of Mathematics and Course Chad did not seem to value mathematics, or at least school mathematics, because he viewed it as lacking any opportunity to be creative or to make mathematics personal. 109 Chad: Journalism and math have very little in common. Math isn’t personal or creative. It just is. (ECQ, #10) Chad also viewed mathematics as a subject in which there was only one way to find the correct answer. This was a sharp contrast to his view of journalism (his major), which encouraged different approaches and viewpoints. Chad: With math, well there’s just not all that much creativity in it. [laughs] It's like you've gotta do what you gotta do. Ummm. . .. Journalism is more like, ‘How can this be a different way?’ Whereas math is like, math is like, [deep Darth Vader like voice] ‘This is the way, so learn it or die.’ (End of Course Interview Follow-up) In addition to his general lack of interest in mathematics, Chad could not envision how “college-level” mathematics might play a role in his day-to-day life or career as a journalist. Chad: Other than bill paying and investments, I can’t see a time where college-level math is going to affect me. (ECQ, #12) When I followed up with a question about this statement, in part due to my own belief about the importance of statistics in journalism, Chad responded that he did not consider statistics to be college-level mathematics, but more like middle school mathematics. Also, Chad’s response indicated that because statistics are typically calculated “for you,” he believed he only needed to know how to interpret statistical results. Jen: So, the statistics part of that, do you consider that math? Chad: Yeah, well but I mean, sure. But that is like 8th grade, or even lower than that, like addition and subtraction and multiplication and division. It's not, well there's no radicals. You know. And there are more complicated statistics, but basically, the way it works, those will be pretty much calculated for you. So, I read advanced statistics now because I am a nerd when it comes to sports, but I don't have to process them and do the math myself. I just know that if they have this point one percentage points, or whatever, then it’s better than if it’s this much. That's all I know. And I don't know how it's calculated but I know that it’s true. In sum, Chad seemed to possess strong beliefs about mathematics, what constitutes mathematics, and that some people have a “math brain” and others do not, and classified himself in the latter group. 110 Chad’s Other Affect Related Statements Chad made two additional statements about mathematics that might be categorized as affect related statements. These two quotes indicated that Chad seemed to feel overwhelmed by the number of problems that are typically assigned in mathematics classes, and particularly by the ALEKS program. However, Chad also mentioned that he wished there were “less problems and maybe the problems were more complicated.” I wish I would have followed up on this statement, because it is not clear to me what Chad meant by “more complicated” mathematics problems. Did he mean applied problems? Was he bored by the repetition of the “simple” problems? Those answers cannot be determined from what he said. Chad: (FS16) The ALEKS program is weird to me because it really makes it daunting. . .. The number of topics is overwhelming. There are a lot. I wish there were less problems and maybe the problems were more complicated. Does that make sense? . . . So 75 problems, even if they are sort of simple, is a lot, to the point that you start evading it. (End of Course Interview, Fall Semester 2016) Lastly, Chad also said he felt like Sisyphus when it comes to mathematics, that he is always pushing a boulder up the hill only to have it roll back down. Chad: (SS17) . . . it's not that I don't want to work hard because I work hard in my other classes that I am better at and I like. So, it's not a work ethic thing, I don't think. I think psychologically you end up just, you know, feeling like Sisyphus, that Greek character. You're just rolling the boulder up the hill and it’s always rolling right down at you. Or that is what it feels like when I do math. . . (End of Course Interview, Spring Semester 2017) In summary, Chad’s affective statements about mathematics appear to indicate an overall lack of interest in the subject due to the lack of relevance, creativity, and perhaps the ongoing drill of mathematical procedures involved in the school mathematics that he has experienced. 111 Chad’s Academic Interactions (SS17) Chad’s academic interactions included two study strategies and the use of available resources. First, he took notes about important information as he worked in ALEKS to help him remember course content, but his notes were not organized and perhaps were not used a reference for future study. Second, Chad planned ahead to complete his weekly assignments by dividing up the number of assigned ALEKS topics into manageable amounts to be completed each day. Chad: I have this many topics and they’re due this day. What I was trying to do is plan and say, I will do this many this day, this many this day, and this many this day. But if you get behind, it is . . . extremely hard to catch up. (End of Course Interview, FS16) The primary resources Chad used to learn the content were those available within the ALEKS environment, such as the ALEKS examples, written explanations for the problems he missed, and short instructional videos. If he needed additional assistance, he went to the university Mathematics Learning Center. Summarizing Chad’s Engagement Chad did not pass the Intermediate Algebra course in Fall Semester 2016 and then retook the class in Spring Semester 2017 and passed with a 2.5. Consequently, his experiences provided a perspective of both failure and success in this online, CAI course. In FS16, the semester that he failed, Chad reported feeling overwhelmed by the number of ALEKS topics to be completed each week. To cope, he copied the ALEKS examples, imitated the example as he worked on the following ALEKS problems, and made little effort at understanding the content. On top of that, he had technological issues due to a data breach at the university, and he got behind in all of his classes. At the same time, the mathematics of the course was more challenging for him, so he gave up on his math class and focused on his other classes. When Chad re-took the course in 112 SS17, Chad took a different approach to completing the ALEKS assignments. He studied the ALEKS examples and solved the subsequent problems, only referring to the example if needed, and spoke of putting forth effort to understand the content. However, I note Chad’s SS17 passing grade with the caveat that there is not sufficient evidence to make the claim that his effort and strategies are directly related to his passing grade in the course. Chad’s cognitive interactions with the mathematics of different ALEKS topics fluctuated along the surface-to-deep continuum. This variation may have been related to whether the topic was directly related to a real life context involving proportions. Chad’s think aloud recordings included three ALEKS topics related to contexts (Think Aloud 2, 5, and 8), and two of these contexts were directly related to proportions (Think Aloud 2, percent discounts, and Think Aloud 8, solving proportions). In both of these, Chad utilized proportional reasoning to verify his problem solutions before submitting them to ALEKS, thereby demonstrating deeper cognitive interactions. Otherwise, for the most part, Chad’s cognitive interactions appeared to be generally surface level, reproducing the procedure (as opposed to imitating the step) the solution presented in the ALEKS example problems. Chad’s academic interactions included two study strategies and the use of course and university resources. First, he took notes about important information as he worked in ALEKS. Second, he made weekly plans to complete coursework by dividing up the number of assigned ALEKS topics into an amount to be completed each day. The resources he drew upon were those available within the ALEKS environment. If he was stuck and needed assistance, he went to the university Mathematics Learning Center. Chad displayed a tendency to freely express his feelings as well as his thoughts as he worked in ALEKS. He also freely expressed his feelings and opinions in the interviews. 113 Consequently, there was a substantial amount of affective interactions captured in the data. The number of Chad’s positive and negative expressions of confidence before solving a problem were relatively close (7 positive and 6 negative). His positive expressions after solving a problem outnumbered the negative 13 to 4, due to his tendency to celebrate his success. Yet, this positive affect during his work in ALEKS did not translate to an increased value of mathematics. He stated that college math would not be useful in his everyday life and career as a journalist and sports reporter. He lacked interest in mathematics because of his lack of success and because he felt “Math isn’t personal or creative. It just is.” However, Chad’s reflection on the course included positive emotion, such as “a satisfying moment” and a “significant feeling of accomplishment” in anticipation of passing the course as noted in the quote below. Chad: I’ve never liked or been good at math, so there wasn’t really any moment of inspiration wherein I decided to pursue math farther than the university requirements, but when I got a 91 on my second test after studying really hard for it, it was a satisfying moment. With math, I’m just trying to get through it. I didn’t like [intermediate algebra], but that’s not [intermediate algebra]’s fault. I anticipate that when I pass, there will be a pretty significant feeling of accomplishment (ECQ, #3, Spring Semester 2017) 114 CHAPTER 7 Tia “I like to get things done.” When I first met Tia, my impression was that she was rather quiet and shy. She caught on quickly to all of the technology requirements of participating in the study and seemed pleased that she could make the think aloud recordings independently on her own time schedule. Tia was a consistent and dedicated participant in this study, which she demonstrated by regularly submitting her weekly think aloud recordings and participating in the Observed and Extended Think Aloud Interview even though it was scheduled on her birthday. This case study begins with an overview of Tia’s demographic and mathematical history based upon her Math History Questionnaire (MHQ) responses and the subsequent interview. Then I provide an illustration of the nature of Tia’s engagement with ALEKS, the CAI software of the Intermediate Algebra course, based upon data from multiple screencast and pen-cast think aloud recordings, and an end of course questionnaire and interview. The nature of Tia’s engagement is organized around themes of her cognitive, academic and affective interactions with ALEKS. Tia is an African American female from an urban, in-state city. During the study, she resided on campus in one of the dorms. When Tia completed the Mathematics History Questionnaire (MHQ) early in the 2017 spring semester, it was her second semester as a freshman undergraduate at Michigan State University (MSU), and she listed her major as Apparel and Textile Design. In the MHQ, Tia responded “I dislike math” and that she was a “below average” mathematics student. In the quote below, Tia expressed that she did not like math because she was not good at it. 115 Tia: I’ve never been good at math and I have always needed to find extra help outside of class to pass tests. I don’t like math, mostly because I’m not good at it. Prior to her enrollment at MSU, Tia took four years of mathematics in high school, culminating with calculus her senior year. When asked to identify a statement that best described her high school mathematics experiences (MHQ, #18, Appendix D), she selected the statement: “My high school teacher both lectured to the class and provided activities in which students worked together in small groups to learn mathematics.” However, in the interview, Tia said that statement pertained to her first three years of high school mathematics, but not her senior year. During her senior year, she took AP Calculus in a flipped class format in which students were required to watch online video lectures at home and worked in groups to solve related calculus problems in class the following day. Tia commented that she preferred person to person lectures rather than online lectures because you could ask questions. Also, Tia stated the reason she got through four years of high school mathematics was because she had a tutor. Tia: 9th grade second semester, I was getting like an F. I went to the office and they recommended some people. I ended up with an older woman who I met with once a week until the end of that year, second year, and third year. My fourth year, I had a friend who was really good at math, and I met with him like twice a week. He was better because he would meet with me whenever I needed, plus I didn't have to pay him. That's like the only reason I got through all of those classes. Especially with Calc because that teacher, he was more like, here's the assignment and work with your peers and if you still don't get it, then ask me a question. He wasn't like, he didn't write on the board and teach it. He did YouTube videos that we watched at home. Stuff like that. Because it sounded like Tia’s high school calculus class utilized an online component, YouTube lectures, I was curious about Tia’s impression of that format. Jen: That's called a flipped classroom. How did that work for you? Tia: It never worked for me. Because you can't stop the video and ask the questions you need. At least in ALEKS you can press for an extra explanation or something. With him, I could re-watch the video, but it's not going to go into more detail or anything. 116 On the MHQ, Tia responded that she felt that although she had “previous exposure to the topics in this course,” she believed she “needed a refresher experience before proceeding to college-level mathematics.” In the interview, Tia mentioned difficulty remembering mathematics as the reason she did not question her enrollment in intermediate algebra even though she had taken calculus in high school. Tia: I have a really bad memory when it comes to math, so it made sense to me that I would be starting over, kinda. (Initial interview, follow-up to MHQ) In one of her statements near the end of the interview, Tia again mentioned that she did not remember mathematics concepts, plus she makes small mistakes (e.g. miscopying a problem or math fact errors) as she works on math problems. Tia: My main issue is remembering concepts. I make small mistakes and then after I'm like, ‘Why did I think that?’ (Initial interview, follow-up to MHQ) In the next section, an excerpt of one of Tia’s think aloud recordings illustrates what this challenge might look like. Tia’s Cognitive Interactions: Mixed Messages Tia expressed an orientation to things that do not interest her that may have influenced and limited her cognitive interactions with the online CAI Intermediate Algebra course. Tia: I never think too much about it, like why is it like that? I'm more like one of those people, who, like when you flick the light switch, I don't need to know how it happens. (Tia, Observed Extended Think Aloud) In other words, in the case of algebra, Tia was not concerned with understanding algebraic procedures, but was more inclined to put her efforts toward remembering what do do without considering how the procedures worked or why the procedures were correct. In this section, I provide excerpts of data from three different think aloud recordings of Tia (Think Aloud 3, 5, and 7). Also, I provide an analysis of these three think aloud recordings to 117 illustrate how Tia’s cognitive interactions appeared to remain at the surface level of the surface- to-deep continuum regardless of her degree of confidence about the ALEKS topic. Tia’s Think Aloud Recording 3 (Figures 26 & 27): Miscopied In this think aloud recording, Tia was solving problems from the ALEKS topic, Solving for a variable in terms of other variables using addition or subtraction with division. In general, solving for a variable was an algebraic procedure Tia was adept at carrying out, as demonstrated by her utterances and written work in Figures 26 and 27. Figure 26. Tia Think Aloud 3, Problem 2 Although Tia performed the procedures correctly for these two problems, there are two aspects of her interactions with ALEKS that one might consider concerning. First, in Problem 2 (Figure 26), Tia performed the correct calculation, entered her final answer into ALEKS as /=1#1+23+ /=1(142. , and ALEKS responded that this was correct. However, later in ALEKS and on exams in this course, Tia would be asked to answer this same question in simplified form, , otherwise it would be considered incorrect. Some might argue that students need to pay attention to directions when answering mathematics problems. Others might argue that 118 students need a consistent message about simplifying rational expressions. An additional argument might question whether simplifying rational expressions is a necessary skill to begin with. Based on these think aloud recordings, it was not obvious whether this ALEKS generated problem was intentional or a random fluke in the ALEKS algorithm. In the end, inconsistencies or unclear expectations such as this may potentially perpetuate student perceptions of algebraic manipulations as a set of rules without meaning. Original Problem First Attempt Second Attempt Figure 27. Tia Think Aloud 3, Problem 3 119 Tia’s interactions with this second problem illustrated the issue she mentioned in her first interview, her tendency to miscopy or make simple arithmetic errors. In this particular situation, although Tia had answered the problem correctly on paper, she mistyped her answer into the ALEKS program twice. The issue here is that ALEKS was not able to assess whether Tia was mistyping her answers and mistakenly attributed incorrect student responses to a lack of mathematical skill as opposed to other reasons (e.g. distraction, fatigue). Tia’s Think Aloud Recording 5 (Figures 28 – 30): I did it all wrong! In this think aloud recording, Tia was solving problems from the ALEKS topic, Solving two-step equations with signed fractions. As stated in the section before, Tia was proficient with the skills of solving for a variable (Figure 28, Transcript lines 4-13), but when reading the solution to this problem, she got confused about fraction operations (Figure 28, Transcript lines 14-20). She recognized that the ALEKS example included finding the least common denominator (LCD) to subtract two fractions, “They did the LCD of these” (Figure 28, Transcript line 18). However, she did not appear to understand the equivalent fractions used to find the least common denominator, “How did they get 12 over 9?” (Figure 28, Transcript lines 19-20). In the end, Tia said, “I don’t get it [inaudible]. I’ll try the next one” (Figure 28, Transcript lines 28-29). In the next problem (Figure 29), the first few steps of Tia’s solution method mirrored the prior ALEKS Example just discussed (Figure 29). Despite her confusion in the prior example, for this problem she found the LCD and the associated equivalent fractions correctly (Figure 29, Transcript lines 32-39). Unfortunately, in her next steps, she conflated the algorithm for addition of fractions with the algorithm for multiplying fractions, which produced the incorrect answer (Figure 29, Transcript Lines 40-46). As Tia entered her answer into ALEKS, she said, “I feel like 120 that’s wrong. I’m not good with fractions” (Figure 29, Transcript line 50). Next, Tia chose to look at the explanation for this problem to learn what errors she had made. Figure 28. Tia Think Aloud 5, ALEKS Example 121 Figure 29. Tia Think Aloud 5, Problem 1 When Tia reviewed the ALEKS Explanation for the problem for which she made an error, she exclaimed, “Oh. I wasn’t simplifying with the LCD. Oh. I did this all wrong [original verbal emphasis]” (Figure 30, Transcript lines 56 & 57). But Tia had not done the problem all wrong. She had been correct in her calculations up to the point where she conflated the fraction addition and multiplication algorithms (Figure 30, Transcript lines 40-46). Furthermore, Tia had attempted to follow the procedures that ALEKS had provided in the initial example problem (Figures 28), solving a linear equation using fraction operations. However, the solution method that ALEKS presented for this problem began by eliminating the fractions of the equations by multiplying both sides by the LCD (Figure 30). Tia did not appear to recognize that the ALEKS Example she had studied (Figure 28) and attempted (Figure 29) was an equivalent but different method than the ALEKS Explanation (Figure 30). The ALEKS Explanation (Figure 30) presented the fraction elimination method, and Tia then used this method to solve the next three problems correctly. 122 Figure 30. Tia Think Aloud 5, Problem 1 Explanation The issue here is that ALEKS did not help Tia understand and correct her error. Worse yet, this ALEKS sequence may have reinforced unproductive beliefs about mathematics. These potentially reinforced unproductive beliefs are contrasted with productive beliefs in Table 18. Table 18. Unproductive and Productive Beliefs about Learning Mathematics Unproductive Belief Correct answers are the sole purpose of mathematics learning. There is one best method to solve a mathematics problem. Productive Belief Understanding mathematics is a primary purpose of mathematics learning and typically results in correct answers. There are multiple ways to solve a mathematics problem. Some methods may be more efficient than others. 123 Tia’s Think Aloud Recording 7 (Figure 31): Misconception In this think aloud recording, Tia was Factoring a linear binomial. In this excerpt, Tia switched the order of two terms in her answer without paying attention to the operation, which was subtraction. Her answer was incorrect because subtraction is not commutative. When Tia read the ALEKS explanation for this problem, she exclaimed, “Wow! They marked it wrong because I switched the order . . . I’m pretty sure the answer would have been the same but OK. Note to self. Don’t switch the order” (Figure 31, Transcript lines 12-18). The issue here was that although ALEKS marked Tia’s answer as incorrect and presented a solution for her to review, Tia did not reconcile her incorrect solution with the ALEKS solution. In this way, Tia maintained the incorrect belief that the operation of subtraction is commutative, but at the same time, she resolved to abide by the meaningless rule, “Don’t switch the order.” Thus, in this case, ALEKS did not address Tia’s misconception. Figure 31. Tia Think Aloud 7, Problem 3 124 Tia’s Cognitive Interactions Summary In all of Tia’s think aloud recordings, her cognitive interactions with ALEKS remained at the surface level. In addition, due to the nature of the problems presented in ALEKS, Tia was never required to move beyond surface level thinking. An overview of Tia’s cognitive interactions discussed in this section is provided in Table 19 below. Problem Solving Phases Orientation Activities (Transcript Lines) Table 19. Patterns of Cognitive Interactions in Tia's Think Aloud 3, 5, & 7 (Figures 26-31) Patterns of Cognitive Interactions ALEKS Example(s) &/or Routine Exercises (Figure #) Imitate the Steps Think Aloud 3, Problem 2 [Correct] (Figure 26) Imitate the Steps Think Aloud 3, Problem 3 [Miscopied Her Correct Answer] (Figure 27) • Verify o External Authority (31-33) o • • Verify o External Authority (41-49) o • • Read & Copy Problem (24-26) • • • Read & Copy Problem (34-35) • • • • • Imitate Steps (27-30) • • • Imitate Steps (36-39) Generation Activities (Transcript Lines) Orientation Activities (Transcript Lines) • Read & Copy Example (1-29) o Backtracking (14-17) • • • Read & Copy Problem (30-31) • • • Read & Copy Problem (1) • • Imitate the Steps Think Aloud 5, ALEKS Example (Figure 28) Imitate the Steps Think Aloud 5, Problem 1 [Incorrect] (Figures 29 & 30) Imitate the Steps Think Aloud 7, Problem 3 [Incorrect] (Figure 31) • Verify o External Authority (50-55) o • Reflect (Figure 27) o Read Explanation (56-70) • Verify o External Authority (5-11) o • Reflect o Read Explanation (12-18) • • • Imitate Steps (32-49) • • • Imitate Steps (2-4) 125 Tia’s Academic Interactions Tia was a dedicated student who “liked to get things done.” For instance, I learned that she met with me for the Observed Extended Think Aloud interview on her birthday even when friends had tried to convince her otherwise. [During the Mid Semester Interview OETA, Tia got a text] Tia: Oh, I'm sorry. Let me respond to this. [pause while texting] I told my friends I wouldn't be done until 1:30pm, but they're ready to go. [Laughs] You see, it's my birthday. Jen: Thank you so much for doing this on your birthday! Tia: Oh, it's no problem. I don't mind. I'm the kind of person who likes to get things done. So, well, if it's gotta get done, I'm not going to push it back. My friends can wait. It's www.livescribe.com/player my [emphasis original] birthday anyway. [laughs] Download this PDF to your computer and go to On iOS, open the PDF in Livescribe+. In addition to her dedication and “get it done” attitude, Tia’s academic interactions included two academic strategies. First, to help her remember course content, she took notes about important information as she worked in ALEKS. Figure 32. Tia Note Taking Example (Think Aloud Recording 3) Second, Tia planned ahead to complete her weekly assignments by dividing up the number of assigned ALEKS topics into manageable amounts to be completed each day. Tia was also resourceful. She utilized the ALEKS online resources, but she also set up weekly appointments with a mathematics tutor, would ask a friend for help, and went to the university Mathematics Learning Center. 02.07.2017 11:52p Tiffanie Recording 3 2/7/17, 6:53 PM, 18m 34s 126 Tia’s Affective Interactions Tia generally demonstrated a lack of confidence in her ability to do mathematics. This was evident by saying, as she entered her answer into ALEKS, “Please be right.” or “I hope that’s right.” This lack of confidence was corroborated by some of Tia’s responses to the MHQ. Tia responded that she was a “below average” mathematics student and that she did not like math because she was not good at it. Tia: I’ve never been good at math and I have always needed to find extra help outside of class to pass tests. I don’t like math, mostly because I’m not good at it. Despite her lack of confidence in the course, Tia said that she liked structure of the course because she could learn at her own pace, appreciated the ALEKS explanations, was not required to go to class, and could set her own schedule. She reported these aspects reduced the pressure of the course and she felt more relaxed learning mathematics. Tia: I like [intermediate algebra ALEKS] because I could learn the lessons at my own pace and work around my schedule. I also liked the explanations and being able to get help on campus if I wanted – not being required to go to a class. My attitude with math changed because of these circumstances. I didn’t feel as pressured and it was more relaxed. (ECQ, #3) Tia: It [intermediate algebra ALEKS] made math more tolerable, but I still dislike it. (ECQ, #8) Tia reported that mathematics will play “somewhat” of a role in her career, Apparel and Textile Design. “When I have to measure out pattern pieces or make my own, I will use math.” She also reported that mathematics will play “somewhat” of a role in her everyday life, because “We all have to deal with finances.” (ECQ, Questions, #9-12). 127 Summarizing Tia’s Engagement Based on what I observed, Tia put a lot of effort and was persistent in her work in the CAI intermediate algebra course, and her efforts paid off. Tia earned a 2.0 grade; however, I note her final grade with the caveat that there is not sufficient evidence to make the claim that her effort and persistence are directly related to her grade in the course. As I reported at the beginning of the section on Tia’s cognitive interactions, Tia expressed that she did not concern herself with why things worked in general and this orientation that may have influenced and limited her cognitive interactions with the online CAI Intermediate Algebra course. Tia was not concerned with understanding algebraic procedures, but was more inclined to put her efforts toward remembering what do do without considering how the procedures worked or why the procedures were correct. As a result, in all of Tia’s think aloud recordings, her cognitive interactions with ALEKS remained at the surface level. In addition, due to the nature of the problems presented in ALEKS, Tia was never required to move beyond surface level thinking. Furthermore, Tia’s apparent view of mathematics as a set rules without meaning seemed to be reinforced by the ALEKS environment. This was more evident when Tia’s misconceptions were not addressed during her interactions with ALEKS (e.g. Figure 31, Think Aloud 7, Problem 3). Tia’s academic interactions included effort, persistence, and resourcefulness. She put effort into the use of two academic strategies. First, she took notes to refer back to as she worked in ALEKS. Second, she planned ahead to complete her weekly assignments by dividing up the number of assigned ALEKS topics into manageable amounts to be completed each day and was persistent until she had completed her goal. In her own words, Tia likes “to get things done.” The resources she used to learn the content were those available within the ALEKS environment, 128 such as the ALEKS examples, written explanations, and videos. Tia demonstrated further resourcefulness through the avenues that she used to get academic assistance with her mathematics. She applied for a personal mathematics tutor and they met for an hour once a week. If she needed further assistance, she would ask a friend or go to the university Mathematics Learning Center. Tia mentioned that she learned mathematics best when someone was sitting with her and teaching her how to do the problems. In Tia’s affective interactions, she demonstrated a lack confidence about mathematics. For example, as she entered her answers into ALEKS, she often made statements such as, “I hope that’s right,” or even seemed to plead at times saying, “Please be right.” In spite of her lack of confidence, Tia appreciated the flexible structure of the online course and said “It made math more tolerable, but [she] still disliked it.” Lastly, Tia stated that mathematics would play “somewhat” of a role in her everyday life and future career in Apparel and Textile Design. 129 CHAPTER 8 CROSS-CASE ANALYSIS Each of the prior three chapters consisted of a descriptive case of an individual participant’s interactions with ALEKS. The purpose of this chapter is to provide a cross-case analysis of these three descriptive cases. First, to better describe and understand students’ experience in ALEKS, this chapter begins with a comparative analysis. In the comparative analysis, I summarized and compare the student engagement indicators across the three cases. In the comparison, I identified those indicators that varied and those that remained constant. The comparative analysis is followed by a relational analysis of those student engagement indicators that were found to vary. The chapter concludes with a summary of the findings from the analysis. Comparative Analysis To analyze the data, I used an adapted form of Finn and Zimmer’s student engagement framework described earlier in Chapter 3, cognitive, academic, and affective interactions. It follows, then, that the comparative analysis has been organized into three corresponding sections, cognitive interactions, academic interactions, and affective interactions. Each section begins by revisiting the definitions of the relevant interaction theme (cognitive, academic, affective) specific to this research study and then summarizes the similarities and differences of the data across the three case studies. The purpose of this summary and comparative analysis is to better understand and describe the indicators of student engagement and to identify those indicators that vary and those that remain fairly constant. 130 Cognitive Interactions For the purposes of this study, mathematical cognitive interactions are defined as the thinking activities that are undertaken as one solves mathematics problems, perhaps within, but not limited to, a school mathematics environment. Specifically, mathematical cognitive interactions are the thinking activities that are carried out during the orientation, generation, and conclusion phases of mathematical problem solving. Cognitive interactions may be characterized along a continuum of surface to deep, which is contingent upon the nature of the aforementioned thinking activities. At one end of the continuum, surface cognitive interactions are described as the thinking activities that are limited to the bare essentials to memorize or imitate the steps of procedures. At the other end of the continuum, deep cognitive interactions are described as the intensive effort or study of mathematics to understand the concepts underlying mathematical procedures, and to make connections between mathematical contexts, symbols, ideas and representations. In the context of solving a high cognitive demand mathematical problem, deep cognitive interactions would include all of the activities summarized in Table 6: Problem solving phases and activities when solving non-routine mathematics tasks (p. 50). It is important to note that the nature of cognitive interactions is influenced by the cognitive demand of mathematical problem being solved. Cognitive Demand of Mathematical Tasks. The mathematics education researchers, Silver, Smith, and Nelson, (1995) were the first to distinguish between mathematical problems and mathematical tasks. Later, Boston and Smith (2009) define a mathematical task as “a set of mathematical problems or a single complex mathematical problem that focuses students’ attention on a particular mathematical idea” (Boston & Smith, 2009, p. 121). For the purpose of this study, a mathematical task corresponds to the set of mathematical problems organized within 131 an ALEKS Topic. Mathematical tasks have been characterized as either low cognitive demand or high cognitive demand tasks. Low cognitive demand tasks are memorization tasks or procedural tasks in which an algorithm is provided for students to follow and do not require students to explain or make connections to the underlying mathematical concepts for performing the task. On the other hand, high cognitive demand tasks are described as “procedures with connections” and “doing mathematics.” Procedures with connections require student attention on underlying mathematical concepts, ideas, or structure. Doing mathematics tasks require students to “create meaning for mathematical concepts, procedures and/or relationships” (Boston & Wolf, 2006). Because the task itself influences how students interact with the task, I first evaluated the cognitive demand of the ALEKS Topics using the IQA Potential of the Task Rubric (Boston & Wolf, 2006) (Appendix G). In ALEKS, each topic began with a worked out example on the ALEKS Learning Page (Appendix I), which was followed by a similar problem to be solved. As a result, each task was “limited to engaging students in using a procedure that is specifically called for or its use is evident based on prior instruction, experience, or placement of the task” (Boston & Wolf, 2006) (Appendix G). Therefore, all 57 ALEKS Topic recordings I viewed were low cognitive demand tasks. Although all of the recorded ALEKS Topics can be described as low cognitive demand tasks, this categorization did not capture the effort I observed in the recordings of students as they worked in ALEKS. However, my prior review and synthesis of the research literature about mathematical problem solving provided a way to parse students’ mathematical activities into problem solving phases that proved useful for categorizing and generalizing student approaches to solving the routine exercises in ALEKS. 132 Patterns of Cognitive Interactions. I had synthesized the idea of problem solving phases, orientation, generation, and conclusion, from prior theoretical and research literature (Polya, 1985; Schoenfeld, 1985; Garofalo & Lester, 1985; and NCTM, 2009), with the caveat that these phases do not necessarily occur in a linear order. In addition, the activities within each of these phases would be required to solve high cognitive demand mathematics tasks (Table 6, p. 50). Nevertheless, I posited problem solving phases would apply to all types of mathematics problems, but that the activities within these phases would vary based on the cognitive demand of the task. The usefulness of this framework for analyzing the nuances of student engagement in tasks is evident in the prior Chapters 5, 6, and 7 (see Tables 14 – 19). The observed patterns of cognitive interaction are provided in Table 20. Table 20. Observed Patterns of Cognitive Interactions with ALEKS problems. E C A F R U S Patterns of Cognitive Interactions Example(s) &/or Problem Set(s) Imitate Each Step of Procedure Provided Example(s) Routine Exercises Decipher the Procedure Provided Example(s) Routine Exercises Transcend the Procedure Provided Example(s) Routine Exercises • • • Problem Solving Novel Problems P E E D Problem Solving Phases Orientation Activities Generation Activities Conclusion Activities • Read & Copy Example • • • Read & Copy Example • Understand • • Read & Copy Example • Understand • Analyze & Connect • • • • Read • Understand • Analyze & Connect • • • Imitate Each Step • • • Reproduce Procedure • • • Recreate Procedure • • • • Explore • Plan • Execute Plan 133 • Verify o External Authority o • • Verify o External Authority o Internal Authority • Reflect & Connect • Verify o External Authority o Internal Authority • Reflect & Connect • • • • Verify o External Authority o Internal Authority • Reflect & Connect The three patterns of cognitive interaction outlined above require brief definitions and some elaboration, so here I define them and in the next paragraph I elaborate with references to some examples in this study. Imitating the steps of the procedure is defined as reading the problem and copying the presented procedure, using these notes to imitate each step of the procedure with no evidence of attempts to understand why the procedure works, and culminating with a reliance on an external authority to verify whether the solution is correct. Deciphering the procedure is defined as as reading the problem and copying the presented solution method, but with some evidence of effort in recognizing patterns in order to duplicate the complete procedure, and culminating with a reliance on an external authority to verify correctness and the potential use of internal authority primarily using intuition. Transcending the procedure is defined as reading the problem and copying the presented solution method, understanding why the procedure works as demonstrated by carrying out the solution method and culminating with internal verification that mathematically determines the solution is correct. I define imitating as surface level understanding and deciphering as just below the surface, because neither demonstrates knowing why the solution method is correct. I define transcending as a deeper level of understanding because mathematical verification demonstrates knowing why the solution method is correct. Due to the nature of the mathematical tasks presented in ALEKS, student interactions with 53 of the 57 ALEKS Topics were at the surface level. In other words, the students either imitated each step of the procedure, or deciphered the ALEKS Example to duplicate the complete procedure. In both of these patterns of cognitive interaction, students primarily relied on the External Authority of ALEKS to check the correctness of their solution. Furthermore, in two think aloud recordings, students acquiesced to the solution method presented in ALEKS, 134 thus relying on the external authority of ALEKS even when they may have been on a different, yet correct, solution path (Jade, Think Aloud 1, Figure 12, Transcript Lines 25-31; and Tia, Think Aloud 5, Figure 28, Transcript Lines 14-17). In 4 of the 57 ALEKS Topics I observed, two of the students, Jade and Chad, interacted with the mathematics at a deeper cognitive level. Two examples of these deeper cognitive interactions were illustrated in earlier chapters (Jade, Think Aloud 4, Figures 15 & 16; and Chad, Think Aloud 2, Figures 17-21). In these instances, the students brought their own knowledge and reasoning about procedures or real life contexts to solve the problem. In addition, students used their prior knowledge to verify their solutions before entering their answer into ALEKS, thus relying on their own Internal Authority. The key point here is that the students brought their own cognitive resources to their interactions with the mathematics and went beyond, or transcended, the procedure that was presented by the ALEKS program. In addition, due to the nature of the tasks presented in ALEKS, the students were never required to move beyond surface level cognitive interactions to be “successful” as defined by ALEKS. Thus, my hypothesis is that when students did engage in deeper cognitive interactions, it was due to their prior knowledge and experiences. Academic Interactions For the purposes of this study, mathematical academic interactions are defined as researcher observed and student stated behaviors related to interacting in class and with course material, such as study strategies and the use of various resources and tools to learn course content. For example, study strategies may include, but are not limited to, memory techniques, scheduling time to study, planning to complete a certain portion of their assignments each day, 135 taking notes and reviewing material. Use of resources may include, but are not limited to, the use of course texts, online resources, or university support services such as tutoring. Academic Study Strategies. All three participants utilized study strategies such as planning for assignment completion and note-taking. For instance, all three participants reported planning ahead to complete their ALEKS assignments. They each determined a daily goal for their ALEKS work by dividing the total number of ALEKS Topics to be completed each week by the number of days they planned to work in ALEKS. In addition, all three students took notes as they solved problems, but with varying degrees of organization. As noted in Jade’s descriptive case (Figures 12 & 13), she discussed and consistently demonstrated how she systematically took notes and presented her problem solutions in an organized manner (Appendix J, Figures 41 & 42). Likewise, Tia took notes as she worked in ALEKS and presented her problems solutions in an organized manner (Appendix J, Figures 45 & 46). Chad also took notes as he worked in ALEKS, but his work was not as easy to follow (Appendix J, Figures 43 & 44). As a result, the LiveScribe pen-cast recording was often necessary for me to interpret Chad’s written work. In addition to her systematic, organized note-taking, Jade used a unique memory strategy. She solved the first two of the three ALEKS problems using pencil and paper. Then for the third ALEKS problem, she solved the problem in her head, without writing anything down. Her stated belief was that this would help her remember how to solve similar problems in the future. Academic Resource Use. All three participants’ think aloud recordings revealed the sole use of ALEKS resources. These ALEKS resources included the use of Examples that introduced ALEKS Topics and ALEKS Explanations, which were available for each problem. Most often, participants utilized the ALEKS Explanations after they had entered an answer into ALEKS and their answer was incorrect. However, in the interviews all three participants reported that if they 136 could not figure out how to solve a particular problem using ALEKS resources, they would go to the Mathematics Learning Center. One participant, Chad, reported that he also utilized the ALEKS Instructional Videos and found them helpful. Another student, Tia, reported that she sometimes asked a friend for help with her mathematics. Surprisingly, none of the participants in this study were observed or reported using online resources that were outside of the ALEKS or university environment (e.g. Khan Academy, Wolfram Alpha). This research result contradicts the results of prior research conducted by Kraus and Putnam (2016), who found that students engaging with the online homework system, WebWork, regularly utilized other online resources, yet differed in their approach to the use of these additional online resources. However, a critical difference between ALEKS and WebWork is that ALEKS provides a suite of related tutorial resources (examples, explanations, tutorial videos) within the program, whereas WebWork does not. Affective Interactions For the purposes of this study, mathematical affective interactions are defined as emotional response to course content and participation, which includes feelings of confidence (or feelings of a lack of confidence and/or anxiousness) in tackling various assigned mathematical tasks, as well as a sense of mathematics course content value and usefulness. The sense of mathematics course content value and usefulness may include observations of unprompted student statements (e.g. “When will I ever use this.”) or student statements in response to the end of course interview prompts (e.g. “Please elaborate on how math will play a role in your career, or not.”). Affective Response to Mathematical Tasks. Student affective response, as exhibited by levels of confidence, to each set of problems in an ALEKS Topic varied widely. I categorized 137 student’s confidence on a continuum from low confidence to high confidence based on explicit confidence related statements that students made. Students sometimes made statements of confidence when they first saw the ALEKS Topic and Example, which was categorized as Confidence Before. An example of a statement of confidence before was, “I already know this, so this should be easy.” An example of a statement of a lack of confidence before was, “I don’t like exponents. This is already not going to be my chapter.” Students sometimes made statements of confidence after they solved an ALEKS problem as they entered their solution into ALEKS, which was categorized as Confidence After. An example of a statement of confidence after was, “I know this is true because…” An example of a statement of a lack of confidence after was, “If it’s wrong, which I expect it to be, then I’ll just go to the explanation.” At times, one student almost seemed to plead to some higher authority, saying, “Please be right.” as she entered her solution into ALEKS for verification, and these instances were recorded as a lack of confidence in her answer. To analyze the extent of students’ confidence regarding a specific ALEKS Topic, I quantified confidence by the number of explicit statements participants uttered that conveyed a level of confidence. Participants uttered explicit statements regarding confidence during interactions with 17 out of the total data set of 57 ALEKS Topics. At times multiple statements of confidence were uttered during interactions with the set of problems in a single ALEKS Topic. The number of statements of higher confidence per ALEKS Topic were quantified as a positive number, and the number of lower confidence statements were quantified as a negative number (Table 21). The data in this table suggests that Jade and Chad’s confidence varied, and that Tia appeared to generally lack confidence in mathematics. 138 Table 21. Quantification of Confidence Statements per ALEKS TOPIC in Critical Incidents Name Recording Session # ALEKS Topic Confidence Statements Before Confidence Statements After Jade Jade Jade Chad Chad Chad Chad Chad Chad Chad 1 2 4 2 4 5 6 6 8 8 Dividing rational exressions involving quadratics with a leading coefficient of 1. Introduction to square root mutliplication. Solving and equation of the form x^2=a using the square root property. Finding the original price given sale price and percent discount Solving a system of linear equations using elimination with addition. Power rules with positive exponents: Multivariate products Power and quotient rules with negative exponents: Problem type 1 Power and quotient rules with negative exponents: Problem type 2 Word problem on proportions: Problem type 2 Word Problem involving multiple rates -1 -1 -1 -1 1 1 Chad 10 Finding the nth root of a perfect nth power monomial 1 3 3 5 7 Multiplicative property of equality with fractions. Find x and y intercepts of a line given the equation in Standard [Ax+By=C] form. Solving for a variable in terms of other variables using addition or subtraction with division. Solving a two step equation with signed fractions Factoring a linear binomial OETA Adding rational expressions with common denominators and monomial numerators Tia Tia Tia Tia Tia Tia -1 1 3 2 1 1 -3 -1 3 -1 -2 -1 -2 -1 1 Affective Value of Mathematics. Another indicator of affective interactions was how students valued mathematics. In general, how the students valued mathematics seemed to be 139 based on their prior experiences and their beliefs about the nature of mathematics rather than by their experience with mathematics in this one course. For example, two of the students (Jade and Tia) reported that they felt mathematics would be useful in their everyday life and future career. Jade specifically mentioned that “math and science are closely related” and she expected algebra to play a role in her future career of molecular genetics. Tia’s view of how math would be useful did not include algebra but the everyday use of mathematics in finances and measurement of fabric in her future career of textile design. On the other hand, Chad did not view everyday mathematics or statistics as college-level content, so although statistics would be relevant to his future career of journalism, he did not see any relevance in what he viewed as “college-level mathematics” content in his life. Affective Response to Course Participation Structure. All three students mentioned that they generally felt that ALEKS helped them to learn the course content. However, two students (Jade and Chad) mentioned that they would have preferred a hybrid course structure. On the other hand, Tia appreciated the flexibility of the online course and said that she “didn’t feel as pressured” and that this structure and resources provided by ALEKS “made math more tolerable.” It is important to note that Chad spoke positively about his experience in ALEKS during the semester when he was passing the course. However, in the prior semester immediately after he had failed the course and stopped working in ALEKS, he mentioned feeling overwhelmed by the number of ALEKS topics he had to complete. Yet, even during the semester when Chad experienced success in the course, he reported that the ALEKS mathematics assignments made him feel like Sisyphus pushing the boulder uphill only to have it roll down at him again. For instance, when a student completes a certain number of ALEKS Topics, the program responds, 140 “Congratulations, [student name]! You have unlocked [some number] ALEKS Topics!” Often student responses to this message was a despondent or sarcastic, “Oh, yay…” Summary of Comparative Analysis In summary, I analyzed the data for student engagement in three categories, cognitive interactions, affective interactions, and academic interactions. I first examined the cognitive demand of the tasks, and found that all 57 of the ALEKS Topics recorded in this study were low cognitive demand tasks. Thus, it came as no surprise that the evidence of students’ cognitive interactions with ALEKS Topics indicated that 94% were surface level interactions largely because the ALEKS tasks did not require anything more. On the other hand, 6% of the students’ cognitive interactions with the ALEKS Topics were deeper along the surface-to-deep continuum, although still not what could be characterized as deep interactions. Furthermore, two of the students’ cognitive interactions varied along the surface-to-deep interactions perhaps depending their confidence about the ALEKS Topic, and the third student’s cognitive interactions appeared to remain at the surface level. Similarly, two of the students’ affective interactions with the ALEKS Topic varied between low and high confidence, and the third student’s confidence generally remained low. Lastly, the academic interactions of all three students remained constant throughout the semester. In summary, two dimensions of the analysis framework varied (cognitive interactions and affective interactions-confidence), and the third dimension (academic) remained constant for each student. Thus, in the next section, we analyze potential relationships between those two dimensions that vary, cognitive interactions and affective interactions-confidence. 141 Relational Analysis Thus far, I have described each student’s cognitive and affective interactions in the cases (Chapters 5, 6, and 7). Plus, the comparative analysis in the prior section of this chapter showed that cognitive interactions and confidence varied for two of the three study participants. This relational analysis extends that comparative analysis to examine the potential relationship between patterns of cognitive interactions and levels of confidence. Confidence and Cognitive Interactions Here I briefly describe how the data related to confidence and cognitive interactions were coded for this relational analysis. To analyze and code the extent of students’ confidence regarding a specific ALEKS Topic, I quantified confidence by the number of participants’ explicit utterances that conveyed some level of confidence. Of the 57 ALEKS Topics in this study, a subset of 17 included explicit utterances related to confidence. Participant utterances regarding confidence occurred both before and after solving one or more problems in that ALEKS Topic (Tables 21 and 22, fourth and fifth columns). For this relational analysis, I totaled the before and after confidence utterances to calculate the student’s general confidence level for that ALEKS Topic (Table 22, sixth column). In addition, I observed and coded three patterns of students’ cognitive interactions: imitating each step of a procedure, deciphering and reproducing a procedure, and transcending the procedure (Table 20). To analyze for a potential relationship between confidence and cognitive interactions, the data described above was recorded and represented in Table 22. However, the tabular representation did not provide a clear picture of the data. To further examine for potential relationships between confidence and patterns of cognitive interactions, a two dimensional graphic form was used (Figure 34). In this graphic representation, the horizontal axis represents 142 the level of confidence codes, and the vertical axis represents surface-to-deep cognitive interactions. The latter included the observed patterns of cognitive interactions: imitating steps, deciphering procedure, and transcending procedure. Table 22. Relationship between Confidence and Patterns of Cognitive Interaction Name Jade* Jade Jade* Chad* Chad Chad Recording Session # 1* 2 4* 2* 4 5 Chad* 6* Chad Chad Chad Chad Tia Tia Tia Tia* Tia* Tia 6 8 8 10 1 3 3 5* 7* OETA ALEKS Topic Dividing rational exressions involving quadratics with a leading coefficient of 1. Introduction to square root mutliplication. Solving and equation of the form x^2=a using the square root property. Finding the original price given sale price and percent discount Solving a system of linear equations using elimination with additi... Power rules with positive exponents: Multivariate products Power and quotient rules with negative exponents: Problem type 1 Power and quotient rules with negative exponents: Problem type 2 Word problem on proportions: Problem type 2 Word Problem involving multiple rates Finding the nth root of a perfect nth power monomial Multiplicative property of equality with fractions. Find x and y intercepts of a line given the equation in Standard [Ax+By=C] form. Solving for a variable in terms of other variables using addition or Solving a two step equation with signed fractions Factoring a linear binomial Adding rational expressions with common denominators and Confidence Statements Before Confidence Statements After Sum of Confidence Statements Patterns of Cognitive Interactions -1 -1 -1 -1 1 1 -1 1 3 2 1 1 -3 -1 3 -1 -2 -1 -2 -1 1 -1 Imitate 1 3 2 0 0 -4 -2 1 1 3 -1 -2 -1 -2 -1 1 Transcend Transcend Imitate to Transcend Decipher Imitate Imitate Imitate Transcend Imitate Decipher Imitate Imitate Imitate Imitate Imitate Imitate *Indicate those Critical Incidents illustrated in Chapters 5, 6, & 7 of this report. 143 In Figure 33 below, the data points in the lower left hand quadrant suggests that when students have low confidence, they are more likely to imitate the steps of the procedure presented in ALEKS. This stands to reason, particularly due to the course participation structure of ALEKS, which reinforces the imitation pattern of behavior. Tia Chad Jade Low Confidence Deep Transcend Procedure Decipher Procedure Imitate Steps Surface High Confidence Figure 33. Representation of Relationship: Confidence and Patterns of Cognitive Interaction In contrast, the data points in the right hand quadrant indicate that as student confidence level increases, it is hard to determine from the data whether the student would imitate the steps of the procedure, decipher and duplicate the procedure, or transcend the procedure. However, only when students expressed confidence about the ALEKS Topic did they progress to the somewhat deeper cognitive interactions of deciphering and reproducing procedures, and 144 transcending procedures presented to them. This suggests that in these instances, when students engaged in some type of mathematical reasoning, there may have been other influences on their learning interactions. One anomalous data point is particularly interesting because it represents the only instance in which the evidence indicated that the student progressed from deciphering and duplicating the steps of a procedure to transcending the procedure within a single ALEKS Topic (see dashed arrow in Figure 33). This example of progression in mathematical thinking was from the Chad’s Think Aloud Recording 2 (Figures 17-21). Chad’s cognitive interactions with this ALEKS Topic, Finding the original price given the sale price and percent discount, are notable because the problem solving activities he engaged in appeared to fluctuate along the surface-to- deep continuum as he made sense of how to solve the problems. As discussed in the prior chapter, Chad often used proportional reasoning to estimate whether his solution to the problem made sense, and even though his solution to Problem 1 did not make sense to him, he submitted his answer to ALEKS for verification. Then he chose to study the ALEKS explanation for Problem 1, and he wondered how to differentiate between the example problem and Problem 1 and how they were solved. As he worked on Problem 3, Chad made connections between the wording of the problems and the mathematical concepts they represented when he said, “If it’s 76% of then it’s 24% off.” Chad’s cognitive interactions stabilized (as opposed to fluctuate) at a deeper level as he then used this connection to reason through the solution to Problems 3 and 4. This anomaly in the data is somewhat similar to what a mathematics teacher might desire as result of student engagement in solving a mathematics task. 145 Research Questions and Findings Summary Overarching Research Question: What is the nature of students’ mathematical engagement in an online, CAI intermediate algebra course? 1. What is the nature of students’ cognitive interactions in mathematics in an online, CAI intermediate algebra course? a. What is the potential cognitive demand of the way CAI presents the mathematical tasks to students? Finding 1.a. All 57 of the mathematical tasks (ALEKS Topics) were Low Cognitive Demand. b. What is the nature of the patterns of cognitive interactions within the problem solving phases (orientation, implementation, verification) that students use to solve the CAI problems? Finding 1.b. Patterns of cognitive interactions varied but were mostly Imitating the Steps of the Procedure and Deciphering and Duplicating the Procedure. Only in four instances did participants Transcend the Procedure. 2. What is the nature of students’ academic interactions in an online, CAI intermediate algebra course? a. What academic study strategies do students use to learn the course content? Finding 2.a. All three students consistently used the study strategies of note- taking and planning and setting goals for studying. One student, Jade, used a memory strategy. b. What academic resources do students draw upon to learn the course content? Finding 2.b. All three students consistently used the resources provided by ALEKS and sought assistance from the Math Learning Center as needed. One student, Tia, arranged for a weekly tutoring appointment. 146 3. What is the nature of students’ affective interactions in an online, CAI intermediate algebra course? a. In what ways do students affectively respond to mathematical tasks of an online, CAI intermediate algebra course? Finding 3.a. Students’ expression of level of confidence in response to the ALEKS Topic varied. b. In what ways do students affectively respond to the course participation structure of an online, CAI intermediate algebra course? Finding 3.b. All three students reported ALEKS generally helped them learn course content. In addition, two reported that they would prefer having some type of face-to-face instruction in addition to ALEKS, and the third had arranged weekly appointments with a tutor. c. What is the nature of students’ value of mathematics in their lives and future careers? Finding 3.c. Students’ value of mathematics remained consistent between pre- and post- questionnaires. One student valued mathematics due to its connection with science and her future career. Another student valued everyday mathematics. The third student beliefs about the separation of school mathematics and everyday mathematics seemed to influence his belief that school mathematics is not useful. 4. Question Resulting from Cross-Case Analysis: What is the relationship between Confidence and Patterns of Cognitive Interactions as students engage with an online, CAI Intermediate Algebra Course? Finding 4. Participants’ expression of low confidence was related to the Imitating Steps pattern of cognitive interactions. Yet, participants’ expression of high confidence did not appear to be specifically related any of the three patterns of cognitive interactions. 147 DISCUSSION—TYING IT ALL TOGETHER CHAPTER 9 In Chapters 5 through 7 of this study, I presented the results from each individual case study, and Chapter 8 consisted of the cross-case analysis. In these chapters the student engagement indicators of cognitive interactions, academic interactions, and affective interactions, were foregrounded and were the the primary focus of my research (Figure 34 below). This focus was imperative to illuminate the nature of student engagement, but it is also important to consider other influences on students’ academic success so that we do not miss “the forest for the trees” so to speak. Sociocultural Influences Political and Social Environment: Culture, power, policy, economics STUDENT ENGAGEMENT INDICATORS Cognitive Interactions Problem Solving (Phases & Activities) Surface Deep Affective Interactions Level of Confidence Value of Math Social Interactions Participation in Academic Norms Academic Interactions Study Strategies Resources Structural Influences University Culture Policies Curriculum Assessment Discipline Student Background Support Family Lifeload Psychosocial Influences University Teaching Staff Support Workload Relationships Student Motivation Skills Identity Self efficacy Proximal Consequences Distal Consequences Academic Learning Achievement Affective Satisfaction Well-being Academic Retention Work success Lifelong learning Social Citizenship Personal growth Figure 34. Conceptual framework, student engagement indicators (adapted from Kahu, 2013; Finn & Zimmer, 2012) To rephrase this, the larger framework also includes sociocultural influences, as well as antecedents and consequences to student engagement (Figure 35). In this chapter, I discuss the 148 research findings from the prior chapters in relation to this larger sociocultural framework of student engagement as well as in relation to relevant research literature and theories of mathematics education. Sociocultural Influences Political and Social Environment: Culture, power, policy, economics Structural Influences University Culture Policies Curriculum Assessment Discipline Student Background Support Family Lifeload Psychosocial Influences University Teaching Staff Support Workload Relationships Student Motivation Skills Identity Self efficacy Student Engagement Cognitive Interactions Academic Interactions Affective Interactions Social Interactions Proximal Consequences Distal Consequences Academic Learning Achievement Affective Satisfaction Well-being Academic Retention Work success Lifelong learning Social Citizenship Personal growth Figure 35. Sociocultural Antecedents, and Consequences of Student Engagement (Adapted from Kahu, 2013, p. 766; Finn & Zimmer, 2012, p. 104) Sociocultural Influences Sociocultural influences permeate the context of this study. This framework recognizes that the larger national political and social climate influences the nature of student engagement. However, this study focusses on sociocultural influences related directly to mathematics education, power and privilege associated with mathematical success, and cultural assumptions about the nature of mathematical activities. Power and Privilege Associated with Mathematical Success In the prevalent United States culture, people who are successful at mathematics are generally believed to be more intelligent than those who are not successful at mathematics. 149 These sociocultural dynamics allocate power to mathematical prowess and privilege mathematical success. In addition, a common myth is that some people possess an innate mathematics ability and that others do not. This mathematics ability myth permeates U.S. culture and is perpetuated in the schooling system, in part due to tracking students by their perceived ability. As a result, students who do not experience mathematical success early on in their schooling typically surmise they simply are not smart enough to learn mathematics (Gutierrez, 2013). These sociocultural and sociohistorically constructed beliefs are perpetuated by the educational system in K-12 standardized testing and university placement testing practices because these tests often result in tracking students. Regarding this research study, consider the fact that all three of these students had taken either precalculus or calculus in high school, yet none of them questioned their placement exam score and none chose to retake the placement exam with the hope of placing into a more advanced course of study. This finding is consistent with prior research conducted by Larnell (2011, 2016). Two participants, Chad and Tia, chose to remain in intermediate algebra due to their belief that they were not good mathematics students who needed the review. For example, Chad had explicitly stated that he did not have a “math brain.” On the other hand, the third participant in this study, Jade, chose to remain in intermediate algebra even though she was confident she could succeed in college algebra. She had considered the pros and cons of this decision and deliberately made this choice to ease her transition to college. This deliberate choice could be interpreted as an act of agency on her part. In addition, the sociohistorically constructed perception that White males are more inclined to be mathematically talented may positively influence or negatively constrain students’ identity in relation to mathematics (Stinson, 2013). These sociocultural and sociohistorically constructed beliefs are perpetuated by the realities of institutional disparities in our educational 150 system. In the United States education system, working-class and poor students, as well as Black and Brown students, are more often relegated to K-12 schools with few resources and less experienced teachers. Additionally, mathematics education research rarely documents or analyzes the success stories of Black and Brown students, and by this omission has perpetuated these sociohistorically constructed belief systems (Martin, 2009b). Thus, these and other sociohistorically constructed beliefs about inherent mathematical ability were also influences that permeated the context of this study. A limitation of this study was that it did not include explicit documentation of these sociocultural influences on students’ experiences in this online, CAI intermediate algebra course. This study draws upon prior research conducted by Larnell (2011, 2016) which documented Black student experiences and identity development in a NCBMC, as well as the work of other critical perspectives in mathematics education (e.g. Gutierrez, 2013; Martin, 2009; Stinson, 2013) to substantiate these claims of sociocultural influences. Cultural Assumptions About the Nature of Mathematical Activity Commonly, mathematics is associated with certainty: knowing it, with being able to get the right answer quickly (Ball; 1988, Schoenfeld, 1985b; Stodolsky, 1985). These cultural assumptions are shaped by school experience, in which doing mathematics means following the rules laid down by the teacher; knowing mathematics means remembering and applying the correct rule when the teacher asks a question; and mathematical truth is determined when the answer is ratified by the teacher. Beliefs about how to do mathematics and what it means to know it in school are acquired through years of watching, listening, and practicing. (Lampert, 1990, p. 32) Lampert’s quote regarding cultural assumptions about mathematics is still relevant today, almost 30 years later. This quote from Lampert can serve as a lens through which to view the student interactions with ALEKS as documented in this research study. The ALEKS program appeared to be designed in accordance with these cultural assumptions about mathematics. Doing mathematics in ALEKS meant following the procedures presented by the program. Knowing mathematics in ALEKS meant remembering the procedure and applying it to similar 151 exercises. Mathematical truth was determined when students entered their answers into ALEKS for ratification. As such, student interactions with ALEKS software likely served to reinforce these cultural assumptions about the nature of mathematics. Antecedents to Student Engagement For this study, two documented antecedents to student engagement were the nature of the mathematics curriculum and related mathematics instruction, or in this study, CAI. A limitation of this study was that other antecedents to student engagement were not documented. Structural Influence: The ALEKS “Bite-Sized Pieces” Curriculum Mathematics curriculum design can be approached from several perspectives. Schoenfeld (1992) described one such perspective that he dubbed the content perspective of mathematics curriculum design. According to the content perspective of curriculum development, …the route to learning consists of delineating the desired subject-matter content as clearly as possible, carving it into bite-sized pieces, and providing explicit instruction and practice on each of those pieces so that students master them. From the content perspective, the whole of a student’s mathematical understanding is precisely the sum of these parts. (Schoenfeld, 1992, p. 342) The design of ALEKS curriculum can be compared to what Schoenfeld described as the content-focused perspective of mathematics curriculum. First, the developers of the program assembled a group of experts to delineate “the desired subject-matter content as clearly as possible into bite-sized pieces” (Schoenfeld, 1992, p. 342), in other words the specific ALEKS Topics. Next ALEKS Topics were arranged into ordered “precedence relations” represented by a large, complex directed graph (Falmagne et al., 2006, p. 4). Students are provided “explicit instruction and practice on each of those pieces [ALEKS Topics] so that students master them.” Lastly, the “whole of a student’s mathematical understanding” is related to the number of ALEKS Topics the student has mastered (Schoenfeld, 1992, p. 342). ALEKS utilizes Knowledge 152 Space Theory, which represents the curriculum as a directed combinatorics graph. In this graph, the vertices represent each bite-sized mathematics topic which are connected by arrows that are arranged in a predetermined precedence relation (Falmagne et al., 2013; Falmagne et al., 2006). In summary, the ALEKS curriculum is composed of bite-sized ALEKS Topics, and each topic includes explicit instruction followed by practice on similarly structured exercises. As a result of this inherent ALEKS participation structure, all 57 of the ALEKS Topics recorded in this research study were categorized as low-cognitive demand mathematical tasks according to the IQA rubric definition because “students follow demonstrated procedures” (Boston & Smith, 2009; Boston & Wolf, 2006). Psychosocial Influence: ALEKS Instruction and Participation Structure More than 25 years ago Schoenfeld (1992) described prevalent teaching practices and the related assumptions about teaching and learning, and these teaching practices continue to be prevalent today. These teaching practices include the presentation of techniques and related routine problems in the three steps. 1. A task is used to introduce a technique. 2. The technique is illustrated. 3. More tasks are provided so that the student may practice the illustrated skills. (p. 338) Similarly, the instruction provided by ALEKS also consists of the three steps described in Schoenfeld’s quote in the prior paragraph. 1. A specific ALEKS Topic is used to introduce a technique. 2. The technique is illustrated on the ALEKS Learning Page. 3. More tasks are provided so that the student may practice the illustrated skills. Schoenfeld went on to describe the basic assumptions about mathematical learning that relies on this 3-step sequence of instruction and associated participation structure. Having worked this cluster of exercises, the student will have a new technique in their mathematical toolkit. Presumably, the sum total of such techniques (the curriculum) 153 reflects the corpus of mathematics the student is expected to master; the set of techniques the student has mastered comprises the student’s mathematical knowledge and understanding. (Schoenfeld, 1992, p. 339) In other words, the assumption of this instructional model and participation structure is that knowledge can be transferred to the students by demonstrating techniques. Another assumption is that when students practice the demonstrated techniques and produce correct answers they demonstrate mastery of that technique. Furthermore, this participation structure assumes that the number of techniques that students can recall and reproduce correctly indicates the mathematical knowledge and understanding they have attained. Schoenfeld’s description of prevalent teaching practices and the assumptions about learning that underlie these practices can be aligned with ALEKS methods of instruction as well as the earlier forms of programmed instruction based on Gagne’s theories of hierarchy of content and cumulative learning. The Nature of Student Engagement in this Study In Chapter 8, I presented the cross-case analysis of the three individual case studies and the nature of student engagement they demonstrated. The patterns of cognitive interactions students demonstrated primarily included imitating each step of the procedure or duplicating the procedure. This was not surprising considering the low cognitive demand of the tasks in ALEKS. In only a few instances, when students explicitly expressed confidence, did their patterns of cognitive interactions demonstrate transcending the procedure. I posit that in these instances, it was the student’s own prior mathematical knowledge and experience that facilitated the transcending pattern of cognitive interaction. Furthermore, in one instance, Chad utilized proportional reasoning in relation to a real life context, to learn from and transcend the ALEKS procedure (see Chapter 6, Figures 17-21, Chad Think Aloud 2). In this instance, I claim that 154 Chad was able to learn and generalize from his interaction with ALEKS due to the combination of his own proportional reasoning and the real life context of the problem. Consequences of Student Engagement In the Kahu (2013) framework on the sociocultural nature of student engagement, there are proximal and distal consequences of student engagement. Here I focus primarily on the proximal consequences with the understanding that the proximal has an influence on the distal consequences. Kahu described two primary proximal consequences of student engagement, affective in relation to satisfaction and well-being, and academic in relation to learning and achievement. Although I acknowledge that student satisfaction and well-being are an important proximal consequence, those two aspects of affect seem to be related to marketing the benefits of higher education rather than reflection of the deeper purpose of higher education. Also, in Kahu’s framework, the affective and academic are presented as separate, yet I argue that they are inherently intertwined. Affective consequences of educational student engagement include beliefs about the nature and practices of disciplinary knowledge, the academic consequences. Affective Consequences: Beliefs about Mathematics In constructivist and sociocultural perspectives on learning, it is generally accepted that our life experiences help to form our knowledge and beliefs. Regarding beliefs about mathematics, Schoenfeld (1988, 1992) has expressed concern about how the K-12 students’ experiences solving literally thousands of routine exercises may result in unproductive beliefs about the nature of mathematics. Table 23 below, includes an adapted version of Schoenfeld’s unproductive beliefs and corresponding evidence of how student engagement with ALEKS may have only served to reinforce these unproductive beliefs. The students of this study likely had developed their own beliefs about the nature of mathematics before they entered this 155 mathematics class, and this study did not directly examine their beliefs about mathematics. Yet, in a few instances statements indicating beliefs about mathematics emerged unsolicited in the interviews with students. Also, the nature of the problem types in ALEKS and how students interacted with ALEKS corresponded with the unproductive beliefs outlined by Schoenfeld. Table 23. Unproductive Beliefs about Mathematics and Evidence from this Research (Adapted from Schoenfeld, 1992, p. 359) Unproductive Beliefs about Mathematics (Schoenfeld, 1992) Mathematics problems have one and only one right answer. There is only one correct way to solve any mathematics problem—usual the rule the teacher has most recently demonstrated to the class. Ordinary students cannot expect to understand mathematics; they expect simply to memorize it and apply what they have learned mechanically and without understanding. Mathematics is a solitary activity, done by individuals in isolation. Student who have understood the mathematics they have studied will be able to solve any assigned problem in five minutes or less. Related Evidence from this Research Study (Reference to Specific Evidence) In ALEKS, problems have only one correct answer. Two of the three participants acquiesced to the solution method provided by ALEKS during their recorded Think Aloud sessions. (Figure 13. Jade, Think Aloud 1, Example, Transcript Lines 25-41) (Figure 14. Jade, Think Aloud 1, Problem 1, Transcript Lines 46-50) (Figures 30 & 31. Tia, Think Aloud 5) Chad stated, “Math is like, [deep Darth Vader like voice] ‘This is the way, so learn it or die’” (p. 100). The participants’ most prevalent patterns of cognitive engagement were Imitating Steps and Duplicating Procedures, which suggests that interactions with ALEKS did not help to facilitate learning with understanding. In addition, during her OETA Tia explicitly stated that she needed to remember how to do the procedures, but not why the procedure worked. ALEKS is designed for individualized student learning and for students to work in isolation. The average time participants spent on each ALEKS problem was calculated to be less than 5 minutes per problem. (Jade’s average time per problem was approximately 3 ¼ minutes.) (Chad’s average time per problem was approximately 2 ½ minutes.) (Tia’s average time per problem was approximately 3 ½ minutes.) 156 Table 23. (Cont’d) The mathematics learned in school has little or nothing to do with the real world. Mathematics is not a creative endeavor. In the EOC Questionnaire two participants (Tia and Chad) did not see connections between the course content and the real world. The third participant (Jade) believed the course content would be useful in her science-based career because math and science are related. In the EOC Interview, one participant, Chad, explicitly stated that mathematics is not creative. Based on the evidence indicated in Table 23 above, it is likely that the students’ mathematical engagement with ALEKS would have reinforced unproductive beliefs about the nature of mathematics that the students may have already possessed. If unproductive beliefs about the nature of mathematics remain unchallenged, and if students ascribe to these unproductive beliefs about the nature of mathematics, an additional consequence of student engagement with ALEKS is that students who have successfully completed this course are not likely to be prepared to successfully engage in the type of mathematical thinking that is required in subsequent coursework or careers. Academic Consequences: Learning Goals for Collegiate Mathematics Obviously, it is important for introductory collegiate mathematics courses to prepare students to succeed in potential requisite multi-disciplinary coursework as well as requisite mathematics coursework. To help design courses to meet this goal, the MAA convened five groups of professors each from the following disciplinary content areas: agriculture, the arts, economics, meteorology, and the social sciences. These five disciplinary groups of professors served as content experts and each disciplinary group discussed the mathematical competencies necessary to be successful in their courses. The list of mathematical learning goals (see summary below) were strikingly consistent even though they were independently developed by each of these five disciplinary content areas (Ganter & Haver, 2011). 157 • Conceptual understanding and problem solving—communicating solutions to diverse audiences; precise and correct use of mathematics in presentations and reports (cid:1) • Arithmetic and basic mathematical equations—relationships between variables; percentages, proportion, and measurement; translation of words into appropriate formulas and equations; graphical representations; unit conversions (cid:1) • Problems in context—building analytical models and testing their viability; applying theory to real problems and evaluating alternative solutions; communicating and coordinating with disciplinary faculty to develop alternative problems; using context to inspire and create a need for mathematics (i.e., mathematics as a common technical language) (cid:1) • Estimation and approximation—use of experimentation and exploration to discover mathematical concepts (cid:1) • Statistics and quantitative data—measures of central tendency and standard deviation; analyzing data to make inferences and draw conclusions; presenting data as pictures (such as bar graphs, line graphs, and scatter plots) (cid:1) • Appropriate use of technology—spreadsheets; geometrical/graphical software (cid:1) (Ganter & Haver, 2011, p. 39) Unfortunately, the potential consequences of student engagement with ALEKS does not appear to support the learning outcomes recommended for collegiate mathematics by the Mathematical Association of America (MAA). Promises and Pitfalls of CAI The ALEKS program represents a notable technological achievement in the development of computer adaptive instructional software. Yet, as with any type of promising educational practice or technology, it is important to carefully examine the assumptions underlying those promising practices and consider alternative views. Johannes and Lagerstrom (2017) provided such an analysis of some “promises and pitfalls” of CAI, which are outlined in Table 24 below. Table 24. Summary of Promises and Pitfalls currently apparent in CAI (Johanes & Lagerstrom, 2017, p. 11) Promises Clarification Promise. Clarify the underlying content, skills, and dispositions needed to master a certain domain. Personalization Promise. Find personalized paths through the learning process for each and every student. Pitfalls Epistemological Pitfall. Limits the instructor, learner, and researcher conceptions of knowledge and knowing. Ownership/Security Pitfall. Mishandling learner data legally, ethically, and economically (intentionally or not). 158 Table 24. (Cont’d) Optimization Promise. Increase learning gains while reducing the time durations needed to achieve them. Equalization Promise. Learners from all backgrounds can receive the education they want in a manner they need. Instruction Promise. Teachers can be empowered and better supported in facilitating high-quality learning. Research Promise. The scale and nature of the collected data open up new research avenues in data as well as learning science. Development Pitfall. Creating an adaptive learning system can bankrupt an institution due to high cost in expertise, time, and capital. Discrimination Pitfall. Biased, opaque, and inscrutable models can discriminate against certain learners. Learning Pitfall. The models focus on a particular interpretation of learning that can be neglectful of social and physical learning. Deluge Pitfall. While the systems are collecting a lot of data, much of that data might not be mission-critical and/or meaningful for analysis. The promise and pitfalls that are central to this study are the clarification promise and epistemological pitfall. The clarification promise is related to earlier discussions about the nature of the content/curriculum that is comprised by CAI and that students interact with. The epistemological pitfall is related to the earlier discussions about beliefs regarding the nature of mathematics and knowing mathematics. Although ALEKS presents mathematics content in clearly delineated, easily digestible bite-sized bits, it is not clear that students make mathematical connections between these bits of mathematics. This is the essential epistemological pitfall of ALEKS—that learning and understanding mathematics is narrowly defined as the ability to recall and reproduce isolated bits of procedures. To clarify this issue, I draw upon Skemp’s (1977) definitions of “instrumental understanding” and “relational understanding” of mathematics. Instrumental understand is solving mathematical exercises using “rules without reason” (p.89). In contrast, relational understanding is solving mathematical exercises by “knowing both what to do and why” (p.89). Due to the nature of its design, ALEKS can only guarantee that students attain an instrumental understanding of mathematics under the tutelage of the program alone. 159 Implications There are many implications of this dissertation study, the first of which is the importance of “looking under the hood” of quantitative data analysis, so to speak, to investigate the same phenomenon using qualitative methods. The results of this research reveal that even if quantitative data may indicate positive outcomes regarding student learning, a deeper qualitative investigation may illuminate unforeseen results. Additional implications specifically for policymakers and mathematics departments, curriculum and CAI development, mathematics instructors, and mathematics education researchers are presented in the subsequent paragraphs. Implications for Policymakers and Mathematics Departments When considering developing or adopting CAI, it is important for policymakers and mathematics departments to consider the premises of CAI as well as both the promises and pitfalls of CAI (Johanes & Lagerstrom, 2017). Because CAI programs are commercially owned and marketed, policymakers and mathematics departments need to take the claims that are made about student learning in CAI with a grain of salt. For instance, current recommendations for undergraduate mathematics teaching and learning included calls for practices that encourage active learning environments. As a result, ALEKS has been marketed as being “based on active learning” (https://www.aleks.com/about_aleks/tour_ai_intro). It is important to pause and ask— What does “active learning” actually mean in the ALEKS CAI environment? Furthermore, in administrative journals specific to higher education, CAI has been touted as a “silver bullet” to the dilemma of undergraduate mathematics attrition and failure rates (Twigg, 2011). Perhaps student pass-fail rates for mathematics may improve with the use of CAI (this is not necessarily the case), but the question remains whether CAI adequately prepares students to apply mathematical knowledge outside of the CAI environment and to succeed in requisite courses and 160 careers. Thus, when evaluating research about CAI, it is important to ask—Does this research provide evidence that students can apply this knowledge outside of the CAI environment and in requisite coursework? Implications for Curriculum and CAI Development Curriculum and CAI developers need to also consider the implicit messages that their products convey about the nature of mathematics and learning. CAI software also needs to incorporate methods that help students develop conceptual understanding as well as skill development. This may be possible by incorporating mathematical tasks that require students to make mathematical connections in CAI software. For example, problems may ask students to make connections between mathematical representations (words, expressions and equations, tables, and graphs), or to evaluate multiple solution methods. The inclusion of more problems based in a real life context also may help students make sense of the mathematics, as Chad was able to do in his Think Aloud Recording 2. Furthermore, CAI software needs to include metacognitive components that encourage students to verify their solutions and reflect on their solution process in relation to the mathematical whole. For instance, a glaring shortcoming in the ALEKS presentation of mathematics is that students are not encouraged or instructed to verify their own solutions. This overt omission needs to be rectified by the ALEKS software developers. Also, the inclusion of prompts to facilitate student reflection would at least begin a move toward the inclusion of metacognitive processes that may nurture a more relational understanding of mathematics. Implications for Mathematics Instructors Mathematics instructors who are required to utilize CAI such as ALEKS need to be aware of the implicit messages about the nature of mathematics, as well as message about the nature of 161 learning and knowing mathematics, that their students receive from these programs. In addition, it is imperative that mathematics instructors explicitly work to counteract any unproductive implicit messages their students may receive through CAI. Instructors may choose to conduct whole class discussions about the strengths and weaknesses of the CAI program. Also, it is important to make students aware of what requisite courses will expect so students can make informed decisions about their learning interactions with the CAI program. Furthermore, instructors may want to use class time to log into the CAI program to model and encourage effective metacognitive learning strategies for students. Last but not least, it is imperative that mathematics instructors engage their students in high cognitive demand tasks that require students to make mathematical connections and to “do mathematics” (Bieda et al., 2013; Boston & Smith, 2009; Boston & Wolf, 2006). Implications for Mathematics Education Research This dissertation study demonstrated novel research methodology to document, analyze and illustrate student interactions with CAI software. This cutting-edge research is significant owing to the fact that few studies have investigated this phenomenon. To document student interactions within a CAI environment, the data collection method included the innovative, combined use of screencast and pen-cast video recordings. In addition, this dissertation study presents an original framework, Problem Solving Phases, with which to analyze the nature of an individual’s mathematical work. Although the foundation of this framework was based on research concerning the solving of non-routine mathematics tasks, I posited that the general overarching phases of the framework (orientation, generation, and conclusion) would be applicable to any type of mathematical task, even routine 162 exercises, but that the activities within these phases would differ. The results of this study suggest that this is the case, yet further research is required. The cross-case analysis revealed the potential relationship between confidence and patterns of cognitive interactions. A strength of this analysis was this potential relationship emerged unprompted from the data. In other words, students spontaneously made explicit statements concerning confidence about 17 of the 57 ALEKS Topics recorded in this study. Although the natural emergence of the confidence measure represented a strength of the study, at the same time this small number of ALEKS Topics with a confidence measure was a limitation of the confidence measure. Future research about the relationship between confidence and patterns of cognitive interactions in the context of ALEKS, as well as other contexts, is warranted. Furthermore, for future research it may be advantageous to utilize a Likert scale question about confidence regarding mathematical tasks students solve to obtain more comprehensive data. In addition, the results of this study are task specific. In other words, a potential limitation of this study is that the 57 tasks, or ALEKS Topics, that were analyzed represented approximately 13% of the over 428 ALEKS Topics that comprised this Intermediate Algebra course. Due to the nature and presentation of the ALEKS Topics, or mathematical tasks, that students solved in this study, it was surprising to find that for a few of the tasks students engaged with the mathematics at a deeper level. Further examination of these instances is warranted to determine what may have been contributing factors for those deeper interactions. Future Research and Conclusion Can students engage in deep learning of mathematics from worked examples? The results of this dissertation study thus far imply the answer is no. However, based on prior research I had 163 conducted with others (Gilbertson et al., 2016; Nimtz et al., 2015), I was confident that the answer to this question was that students can learn important mathematics from worked examples if those examples are carefully designed as hypothetical student work. Thus, I performed a thought experiment using the Phases of Problem Solving framework, my past research experience, and research conducted by Rittle-Johnson and Star (2011) to explore this question further (Table 25, p. 165). Note that the above cited research about examples of hypothetical student work is quite different from the plethora of research that exists on student learning from worked examples (Atkinson, Derry, Renkl, & Worthham, 2000). In contrast to the research summarized by Atkinson and colleagues, the research cited above examined examples of hypothetical student work and often includes multiple worked solutions that required students to compare, contrast, and analyze different solution methods. When examining the Table 25 below, keep in mind that the first three Patterns of Cognitive Interaction (Imitating, Deciphering and Duplicating, and Transcending), located above the dark, horizontal line in the middle of the table, were obtained empirically in this study. The three Patterns of Cognitive Interactions following this dark line in the middle of Table 25 are the result of a thought experiment. Lastly, I caution the reader that the list of Patterns of Cognitive Interactions in Table 25 is not intended to imply a linear curriculum or linear learning process. For example, a lesson or mathematical task involving Problem Solving may precede a lesson or mathematical task involving Compare and Evaluate Representations, Solutions, and/or Structure. 164 Table 25. Patterns of Cognitive Interactions: Observed and Thought Experiment Problem Solving Phases Orientation Activities • Read & Copy Example • • • Read & Copy Example • Understand • • Read & Copy Example • Understand • Analyze & Connect • Read • Understand • Analyze & Connect • Read • Understand • Analyze & Connect • Read • Understand • Analyze & Connect • Read • Understand • Analyze & Connect Generation Activities • • • Imitate Each Step • • • Reproduce Procedure • • • Recreate Procedure • • Plan (Identify Procedures) • Execute Plan • • Plan (Discriminate Procedures) • Execute Plan • Explore • Plan • Execute Plan • Explore • Plan • Execute Plan Conclusion Activities • Verify o External Authority o • • Verify o External Authority o Internal Authority • Reflect & Connect • Verify o External Authority o Internal Authority • Reflect & Connect • Verify o External Authority o Internal Authority • Reflect & Connect • Verify o External Authority o Internal Authority • Reflect & Connect • Verify o External Authority o Internal Authority • Reflect & Connect • Verify o External Authority o Internal Authority • Reflect & Connect Patterns of Cognitive Interactions Example(s) &/or Problem Set(s) Imitate Each Step of Procedure Provided Example(s) & Routine Exercises Decipher the Procedure Provided Example(s) & Routine Exercises Transcend the Procedure Provided Example(s) & Routine Exercises Identify & Differentiate Provided Examples & Routine Exercises Followed by Quiz or Test Compare & Evaluate Representations, Solutions, &/or Structure 2 or More Examples of the Same or Similar Problems Create & Assemble Representations, Solutions, &/or Structure Definition & 1 or More Examples Problem Solving Novel Problems E C A F R U S P E E D My point in creating this thought experiment is two-fold. The first purpose was to illustrate how multiple examples of hypothetical student work might be designed to facilitate deeper student interactions with mathematics. The second purpose was to illustrate how the Problem Solving Phases framework might be used to design mathematical tasks with specific goal of a deeper student interactions and to shift the authority for verifying solutions to the student as opposed to relying on the instructor or CAI program for verification. In these last pages, I provide a few examples and discuss some of the purpose and goals of the examination of hypothetical student work. 165 The mathematical task of examining hypothetical student work in Figure 36 was inspired by Tia’s interactions with ALEKS recorded in her Think Aloud Recording 5 (see Chapter 7) and falls under the category of Compare and Evaluate in Table 25. Recall that the excerpts from Tia’s Think Aloud Recording 5 suggested that she may have believed there is only one way to solve a mathematics problem. She stated, “I did it all wrong!” because her solution method did not follow the ALEKS Explanation, when in fact, her work was correct up until the last step of her solution. To counter this unproductive belief, that there is only one correct solution to a to Increase Students’ Mathematical Engagement Using Mathematics Examples mathematics problem, this mathematics task presents two different, yet correct solution methods Jennifer (Jen) Nimtz and asks the student to determine if both methods are correct, to explain how they know they are June 1, 2018 correct or not, and explain which method they prefer. 1 Compare and Evaluate: Example of 2 Solution Methods, Both Cor- rect Topic: Solving a two-step equation involving fractions. Two students solved this problem using two di↵erent methods. Are both methods correct? How do you know? Which method do you prefer? Explain your reasoning. Solve for v. 2 9 4 5 = v + 4 3 Simplify your answer as much as possible. Jackie’s Solution Diontae’s Solution 2 9 4 2 9 3 12 2 9 9 14 9 14 4 5 4 5 4 5 4 5 = = = = ✓ 5 4◆✓ 9◆ =✓ 35 18 = v v + v + 4 3 4 3 4 3 v v 5 4◆✓ 4 5 v◆ 2 9 4 5 = v + 4 3 2 45✓ 45✓ 9◆ = 45✓ 9◆ = 45✓ 2 4 5 4 5 4 v + 3◆ v◆ + 45✓ 4 3◆ 5(2) = 9(4)v + 15(4) 10 = 36v + 60 10 60 = 36v + 60 60 70 = 36v 70 = 36 v 36 36 35 18 = v Figure 36. Compare and Evaluate Two Correct Solutions 166 1 The mathematical task presented in Figure 37 is another type of Compare and Evaluate task. Two different solution methods with different answers are presented. In this task, the student is asked to determine which method is correct, to find the error and correct it, and explain their reasoning. The purpose of this mathematical task is to encourage students to conduct their own error analysis of a commonly seen error, and to make connections between symbolic and 2 Compare and Evaluate: Example of 2 Solution Methods with Di↵er- graphical representations. ent Representations Topic: Finding the intercepts of a radical function. Two students solved this problem using two di↵erent representations, one symbolic and the other graphical. They came up with di↵erent answers. Who is correct? Find the error and correct it. Explain your reasoning. Find the x and yintercepts, if they exist, for the graph of the function: f (x) = px + 9 + 2 Jake’s Solution The yintercept is when x = 0. f (0) = p0 + 9 + 2 f (0) = p9 + 2 f (0) = 3 + 2 f (0) = 5 So, the yintercept is the point (0, 5). The xintercept is when f (x) = 0. 0 = px + 9 + 2 0 2 = px + 9 + 2 2 (2)2 =px + 92 4 = x + 9 4 9 = x + 9 9 5 = x So, the xintercept is the point (5, 0). Del Juan’s Solution f (x) = px + 9 + 2 8 7 6 5 4 3 2 1 y = f (x) x 1 2 109 8 7 6 5 4 3 2 1 1 2 When I graphed the equation, I found one intercept, the yintercept of (0, 5). There is no x-intercept. Figure 37. Verifying and Evaluating Two Solutions, One Correct, and One Incorrect 167 (x4)2 x24 Domain: All Real Numbers EXCEPT x = 4. Domain: All Real Numbers EXCEPT x = 2, x = 2, and x = 4. The mathematical task presented in Figure 38 requires the student to Create and Jane’s Answer Assemble a mathematical example of a polynomial with roots at x = 1, 2, and 3. This is an g(f (x)) = 1 x24 intentionally open ended task that has multiple solutions—actually infinitely many solutions. The Domain: All Real Numbers EXCEPT x = 2 and x = 2. goal of this task is to facilitate discussion and evaluation of another student’s work. At the same time, depending on where this task is located in the curriculum, this task has the potential to introduce basic ideas of roots and polynomial factors that are essential to understand the Fundamental Theorem of Algebra. 4 Create and Assemble an Example Topic: Polynomials and Roots of Polynomials • Create an example of a function equation with a graph that has roots at x = 1, 2, and 3. • Swap your example with another student. • Evaluate and give feedback on your colleague’s problem • Together, come up with a di↵erent third example. Figure 38. Creating and Assembling Examples In summary, I agree with Boston and Smith (2009) who argued that low level cognitive demand tasks that are so prevalent in CAI mathematics environments have their place in the curriculum, but that these low level cognitive demand tasks encompass far too high a proportion 3 of students’ overall mathematical experiences. In other words, students spend too much time merely memorizing the procedures of how to solve specific types of mathematics problems, without understanding why the procedures work and without making connections between the procedures used in one problem type with how the same or similar procedures might or might not apply in the next problem type. 168 In contrast, working on complex mathematical tasks requires that students engage in mathematical thinking, make mathematical connections, and utilize important mathematical habits of mind to solve those tasks. Yet these types of mathematical tasks pose a longstanding challenge to educational technology. For instance, due to the open response design of the mathematical tasks outlined in Figures 36-38, they are not transferable to a CAI individualized learning environment such as ALEKS. In general, mathematical tasks that involve activities that encourage deeper student engagement (e.g. comparing, evaluating, exploring, creating, assembling) are more challenging to facilitate and assess in a computerized environment. Furthermore, communicating mathematics in online educational environments has been an ongoing challenge for online educators and program designers. Nason and Woodruff (2004) presented an overview of the challenges regarding facilitating mathematics discussions in an online class environment. 1. Inability of most “textbook” math problems [such as those prevalent in most CAI programs] to elicit ongoing discourse and other knowledge-building activity either during or after the process of problem-solving. 2. Limitations inherent in most [computer-supported collaborative learning] CSCL environments’ math representational tools and their failure to promote constructive discourse or other mathematical knowledge-building activities. (p. 104). To improve the potential of online mathematics instruction, Nason and Woodruff called for two primary innovations: 1. Authentic mathematical problems that involve students in the production of mathematical models that can be discussed, critiqued and improved. 2. Comprehension modeling tools that: (a) enable students to adequately represent mathematical problems and to translate within and across representational modes during problem solving, and (b) facilitate online student-student and teacher-student hypermedia-mediated discourse. (p. 104) It seems that Nason and Woodruff’s comprehension modeling tools would be necessary for online mathematics instruction and curriculum to incorporate authentic mathematical 169 problems. Fortunately, recent technological innovations such as online synchronously shared whiteboards, some with palettes for mathematics, are now becoming more widely available (Hodges & Hunger, 2011). Until these curriculum changes and mathematical communication/modeling tools are incorporated into online mathematics courses, these courses will remain limited regarding the mathematical learning opportunities that online environments provide for students. 170 APPENDICES 171 Appendix A: Research Participation Consent Form Case Studies of Undergraduate Students’ Interactions with an Online Computer Adaptive Instruction Intermediate Algebra Course Researchers: Jennifer L. Nimtz Doctoral Candidate Program in Mathematics Education A-721 Wells Hall, Michigan State University 517-862-2489 nimtzjen@msu.edu Sandra Crespo Professor Department of Teacher Education 362 Erickson Hall, Michigan State University 517-353-3035 crespo@msu.edu Ralph Putnam Associate Professor Department of Counseling, Educational Psychology and Special Education 511 Erickson 517-353-9285 ralphp@msu.edu Purpose: This form represents a request for your participation in a research study of an online intermediate algebra course (MTH 1825 at Michigan State University). The goal of this study is to gain more understanding of students’ experiences and learning interactions in an online intermediate algebra environment (MTH 1825) to develop theories about how students experience and learn about mathematics in this and similar courses. The study has been designed and is being conducted by a doctoral student in the Program in Mathematics Education (College of Education and the College of Natural Science) to fulfill the partial requirements of a doctoral program. Your Participation and Rights to Say “No” or Withdraw from the Study: As part of this project, you are being asked to participate in several ways. You may refuse to answer particular questions. You may terminate your involvement in any part of this study at any time. In other words, you have the right to say no. Overview of the Study Structure: First, the the researcher will distribute a questionnaire to all MTH 1825 students via the course email system. The purpose of the questionnaire is to help the researcher learn more about your personal and mathematics background as well as your attitudes towards mathematics. Following the questionnaire, 5 to 10 students will be asked to participate in a case study of this MTH 1825. The case study entails making a weekly 15 minute think aloud screen casts of your work in ALEKS using a free internet software called Screen-Cast-O-Matic. In addition, there will be a minimum of 3 interviews that last from 30 to 60 minutes. The time and commitment will vary; interviews will be arranged with the researcher. Please sign your initials if you understand and consent to the following: _____I may terminate my participation in this study at any time. _____I give my consent to be interviewed if asked. Each interview (maximum of four) will not 172 exceed 60 minutes. For the interviews, I also consent to be video-recorded for research purposes only. The interview will include topics related to personal experiences in schools, mathematics education background, and mathematical tasks (i.e., doing math problems related to the course content). _____Any information gathered from these interviews and used in this research will be reported under a false name (confidentiality). You may choose not to answer specific questions or to stop participating at any time. _____I agree to complete a questionnaire related to this project. None of my responses to this questionnaire will be shared with anyone associated with this course in ways that identify me. Any information gathered from my questionnaire will be reported under a false name. I give my consent for Jen to collect copies of my homework, quizzes, or tests (or other class materials) for the study if I am asked. Complete copies of these materials will not be shared with anyone. I give Jen permission to use portions of my work in her research, understanding that my real name will not be associated with those materials. _____I agree to return the Live Scribe Pen and Live Scribe Paper to Jen Nimtz at the end of the semester. Costs and Compensation: There is no financial cost to participate in this study. Similarly, you will not receive any monetary compensation, formal compensation (e.g., extra credit) for completing the survey. However, Students who are selected for the case study and commit their time and energy to record screen casts and participate in interviews will receive a $20 Amazon Gift Card and up to 4 hours of free one-to-one mathematics tutoring from the researcher, an experienced mathematics teacher and tutor. This tutoring can be scheduled near the end of this semester (November 28 through December 11) before final exams, or during the following semester. Potential Benefits to You: If you do choose to participate, this is an opportunity for you to reflect on your own mathematical experiences and other math-related experiences as a MTH 1825 student. Research shows that discussing academic experiences can be empowering, and our hope is that this will be an empowering experience. Over the course of the interviews, students who particulate will also be asked to talk about and do mathematics with the researcher. The mathematics will be directly related to the course material, and extra attention to the course content during these interviews will reinforce what students are learning course. Potential Risks to You: Just as discussing academic experiences can be empowering, discussing negative experiences in mathematics can also be a potential stressor. As part of the purpose of this study, however, we are supportive of students’ struggles and hope to help whenever possible. Otherwise, there are no foreseeable risks associated with participation in this study. Privacy and Confidentiality: Any materials we collect, any interview recordings, and any personal information will be kept 173 confidential to the maximum extent allowable by law. Data will be stored in the researcher’s university office; electronic information will be password-protected and physical data (e.g., notes on paper, written materials) will be locked in a file cabinet to which only the researchers have access. Your real name will not be associated with any personal information; instead, false names will be used. The results of this study will be published and/or presented at professional meetings, but the identities of all research participants will remain anonymous. Contact Information for Questions and Concerns: If you have any questions about this study, such as scientific issues, your role, or to report an injury, please contact the investigators, Jennifer (Jen) Nimtz, A-721 Wells Hall, MSU; If you have questions or concerns about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University's Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail irb@msu.edu or regular mail at 207 Olds Hall, MSU, East Lansing, MI 48824. Documentation of Informed Consent: Your signature below means that you voluntarily agree to participate in this research study. You will be given a copy of this form to keep. __________________________________________________ Student’s Name (Printed) ___________________________________________________ Student’s Signature __________________ Date 174 Overarching Research Question: What is the nature of students’ engagement (SE) with an online CAI intermediate algebra course? Appendix B: Research Design Summary Table Table 26. Research Questions, Student Engagement Indicators, Definitions, and Evidence Specific Research Questions SE Indicator Short Definition What is the nature of students’ cognitive interactions within an online CAI intermediate algebra course? What is the nature of students’ academic interactions within an online CAI intermediate algebra course? What is the nature of students’ affective interactions within an online CAI intermediate algebra course? Cognitive Interactions Student problem solving and thinking activities while solving mathematics problems Academic Interactions Student activities related to learning course material. Affective Interactions Student emotional responses to course content and class participation Not applicable in this context Social Interactions Student activities that are in accordance with explicit and implicit academic/class norms 175 Researcher Observed Evidence (Screencast & Pen-cast Think Aloud Recordings) Phases and Activities of Problem Solving SurfaceßàDeep Learning !"#$ ßà &'($!)#*'(+', *'( -"''$-#+"') Study Strategies Use of Resources and Tools Student Reported Evidence (Surveys and Interviews) Responses to researcher questions during Observed Extended Think Aloud (OETA). Study Strategies Use of Resources and Tools Explicit statements about confidence or lack of confidence Not applicable in this context Confidence level about mathematics and course Value of mathematics and course Not applicable in this context Appendix C: Research Design by Student Engagement Indicator Table 27. SE Indicator 1: Cognitive Interactions Data Source A: Think Aloud Recordings Coding Pass Unit of Analysis 1.A.Pass (i) Each Complete Think Aloud Recording 1.A.Pass (ii) ALEKS Topic 1.A.Pass (iii) Each Complete Think Aloud Recording 1.A.Pass (iv) Utterances and actions in each ALEKS Topic Utterances and actions within each Phase of Problem Solving Analysis Activity a. Identify ALEKS Topics in each recording. b. Identify the ALEKS Sequence. c. Identify Problems as correct, incorrect, or not attempted/completed. Identify Cognitive Demand of the Task. The math task is defined as the ALEKS Topic. Identify & Transcribe Critical Incidents. These are those ALEKS Topics in which the recording showed explicit evidence of student thinking. Document the beginning time and ending time for each ALEKS Topic and each problem within the topic. Identify Problem Solving Phases (PSP) by analyzing student utterances and actions in the ALEKS topic. Identify Activities for Critical Incidents. Type of Code a. Predetermined by ALEKS software b. Observed Student & CAI interactions, such as: ALEKS Example Problem 1 Incorrect ALEKS Explanation Problem 2 Correct Problem 3 Correct Problem 4 Correct Next Topic Predetermined by IQA Potential of the Task Rubric (Appendix F) Emergent: Critical incidents typically occurred when students’ got a problem wrong and was trying to correct their error. Critical incidents also occurred when the student’s work went over and above expectations in ALEKS PSP (Orientation, Generation, Conclusion) from synthesis of research literature Emergent & Descriptive: Imitate Reproduce Recreate 176 Table 28. SE Indicator 2: Academic Interactions Data Source A: Think Aloud Recordings Unit of Analysis Coding Activity Type of Code Utterances and actions within each ALEKS Topic Identify and code Study Strategies Identify resources used via screen cast for each ALEKS Topic Identify and code Resources Emergent & Descriptive: Notetaking Planning Memory Emergent & Descriptive: ALEKS Environment Personal Tutor Friend Mathematics Learning Center Indicator & Data 2.A 2.A Type of Code Emergent & Descriptive: Notetaking Planning Memory Emergent & Descriptive: ALEKS Environment Personal Tutor Friend Mathematics Learning Center Table 29. SE Indicator 2: Academic Interactions Data Source B: End of Course Survey and Interview Coding Activity Unit of Analysis Indicator & Data 2.B Student Response to Interview Questions: #4 & 5 Identify and code Study Strategies 2.B Student Response to Interview Questions: #4 & 5 Identify and code Resources 177 Table 30. SE Indicator 3: Affective Interactions Data Source A: Think Aloud Recordings Indicator & Data 3.A Unit of Analysis Analysis Activity Type of Code Utterances and actions within each ALEKS Topic Identify and code explicit statements evidencing Confidence Emergent & Descriptive: Confidence Before Lack of Confidence Before Confidence about Answer Lack of Confidence about Answer Table 31. SE Indicator 3: Affective Interactions Data Source B: End of Course Survey and Interview Indicator & Data 3.B 3.B Unit of Analysis Analysis Activity Student Response to Math History Questionnaire (Appendix D): #19, 22-25 Student Response to End of Course Questionnaire (Appendix E): #5-12 Student Response to End of Course Interview Questions (Appendix F): #2 & 3 Transcribe and compare pre- and post- responses. Transcribe responses. 178 Appendix D: Mathematics History Questionnaire (Adapted from Larnell, 2011) FS16 Survey Introduction_____________________________________________________ This survey is part of a research project documenting the experiences of students taking online Intermediate Algebra courses, such as MTH 1825. This study is being conducted by Jennifer (Jen) Nimtz, a doctoral student in the Program in Mathematics Education and to fulfill the dissertation requirements of a doctoral program. This survey will take about 10 minutes. Survey Contact: Jennifer (Jen) Nimtz, Program in Mathematics Education, Michigan State University, nimtzjen@msu.edu Your participation in this project is welcomed, but you are not required to participate. Also, this questionnaire is not connected to your grade or coursework in this or any courses that you are taking or may take in the future. Your responses to these items will be kept confidential and data will be assigned to a fake name. By completing and submitting this online survey, you are volunteering your responses. There is no monetary compensation associated with completing this survey. You may refuse to answer any questions or stop the survey at any time. Thank you in advance for your participation! SS17 Survey Introduction_____________________________________________________ Complete this survey to be entered to win a $50 Amazon gift card! This survey will help to improve students’ learning experiences in MTH 1825 and similar courses. This survey will take about 15 minutes. Survey Contact: Jennifer (Jen) Nimtz, Program in Mathematics Education, Michigan State University, nimtzjen@msu.edu Your participation in this project is important, but you are not required to participate. This questionnaire is not connected to your grade or course work in this or any course you are taking or may take in the future. Your response to these items will be kept confidential. By completing and submitting this online survey, you are volunteering your responses. You may choose not to answer any questions or stop the survey at any time. However, only those surveys with at least 75% of the questions accurately completed will be entered into the Amazon gift card drawing. You must be at least 18 years old to participate in this research and to win the drawing. Thank you in advance for your participation! 179 Yes __________ No __________ 1. Are you 18 years of age or older? 2. Are you also enrolled in MTH 100E this semester? 3. Would you be interested in learning more about participating in further research (a case Yes _____ No _____ Unsure_____ study) about your experiences in this online MTH 1825 course? SS17 Additional Text to this question: Benefits of participating in the research: • a $75 Amazon gift card,(cid:1) • up to 2 hours of free tutoring for the MTH 1825 Final Exam,(cid:1) • the activities of the research project may help you remember and learn the course content, • your participation would help future students who take similar courses. For more details, please see the information on the website at: sites.google.com/view/mth1825researchproject Yes _____ Maybe ______ No______ 4. What is your name? (your preferred name) 5. What is your MSU email? (required for entry into the drawing) ___________________________________________________________________ 6. What is your cell phone number? (required for entry into the drawing) ___________________________________________________________________ 7. What is the best way to contact you? Email __________ Text __________ 180 8. Please take a few moments to write about your experiences with mathematics both in school and outside of school. If you are having a hard time starting, consider the following questions: • Was there a particular time when your attitude toward math changed for the better or worse? o When was that? What happened? o Do you like math? o Why or why not? _____ Male _____ Female _____ Other _____ American Indian, Native American, or Alaska Native; _____ Asian or Asian-American; _____ Black (not of Hispanic origin) or African-American; _____ Hispanic, Latino, or Latino-American: _____ White (not of Hispanic origin) or Caucasian; _____ Other/None of the above. 9. What is your gender? (check all that apply) 10. If you answer other to the question, “What is your gender?” above, please specify. 11. With which racial/ethnic group do you identify? (check all that apply) 12. Please specify further about the racial/ethnic group with which you identify. ______________________________________________________________ 13. When did you graduate from high school? (month/year) _______/__________ 14. What is your current university classification? 15. Were you admitted to the university under the College Achievement Admissions _____Freshman _____Sophomore _____Junior _____Senior (+) Program (CAAP) or any other student service programs at MSU? Yes, CAAP _____ No _____ Unsure_____ 181 16. If you were you admitted to the university under a program other than CAAP, please state that program here. ________________________________________________________________ 17. What is your major? ________________________________________ 18. Which statement below best describes your high school mathematics experience. _____ My high school math teacher primarily lectured to the class and sometimes asked students questions related to the lecture. _____ My high school math teacher both lectured to the class and provided activities in which students worked together in small groups to learn mathematics. _____ My high school math teacher primarily provided activities in which students worked together in small groups to learn mathematics followed by whole class discussion. _____Freshman _____ Sophomore _____ Junior _____ Senior _____ Additional Year(s) _____ I liked math. _____ Indifferent (It was ok; no big deal) _____ I disliked math _____ I did not have an opinion about math 19. How would you best describe your view of mathematics in high school? 20. In what year(s) did you take mathematics in high school? (Check all that apply.) 21. What was the last mathematics classes you took in high school? _____Advanced Algebra or Algebra 2 _____ Integrated Math _____ Statistics _____ Pre-Calculus _____ Calculus _____ Financial Math 182 _____I enjoy math(cid:1) _____I am indifferent about math _____I dislike math _____Strong(cid:1) _____Above Average _____Average(cid:1) _____Below Average _____Weak 22. How would you describe your math ability? 23. How would you describe your attitude toward math? 24. How much do you think mathematics will play a role in your career? 25. How much do you think mathematics will play a role in your everyday life? 26. Did you take the SAT test? 27. If you took the SAT test, what was your mathematics score on the SAT? (If you can _____a lot _____somewhat _____not at all _____a lot _____somewhat _____not at all _____Yes _____No _____Not Sure remember it.) __________________________________________________ 28. Did you take the ACT test? 29. If you took the ACT test, what was your mathematics score on the SAT? (If you can _____Yes _____No _____Not Sure remember it.) __________________________________________________ 30. Do you think your score on tests like the SAT and/or ACT accurately reflects your mathematics ability? _____Definitely yes(cid:1) _____Probably yes _____Might or might not(cid:1) 183 _____Probably not _____Definitely not _____Yes _____No _____Not Sure 31. Did you take the MSU Mathematics Department Placement Exam? 32. If yes, how did you take the MSU Mathematics Department Placement Exam? 33. How influential do you feel your score on the MSU Mathematics Department _____On-line _____On campus with a proctor _____Not Sure Placement Exam on your placement into MTH 1825? _____Definitely influential(cid:1) _____Probably yes influential _____Maybe influential (cid:1) _____Probably not influential _____Definitely not influential 34. Do you feel the MSU Mathematics Department Placement Exam was a good indication of your knowledge about mathematics? _____Definitely yes(cid:1) _____Probably yes _____Might or might not(cid:1) _____Probably not _____Definitely not 35. Do you feel with better high school preparation your score on the MSU Mathematics Department Placement Exam would have improved? _____Definitely yes(cid:1) _____Probably yes _____Might or might not(cid:1) _____Probably not _____Definitely not 36. Do you feel that if you reviewed more before taking the MSU Mathematics Department Placement Exam, that your score would have improved? _____Definitely yes(cid:1) _____Probably yes _____Might or might not(cid:1) _____Probably not _____Definitely not 37. What best describes the reason for your placement into MTH 1825?(cid:1) 184 _____ I had previous exposure to the topics in this course, but need a refresher experience before proceeding to college-level mathematics courses.(cid:0) _____ I had no previous exposure to the topics in this course, and this experience will help me develop the skills to enter college-level mathematics courses. _____ Neither of the above. 38. Would you be interested in learning more about participating in further research (a case study) about your experiences in this online MTH 1825 course? SS17 Additional Text to this question: Benefits of participating in the research: • a $75 Amazon gift card,(cid:1) • up to 2 hours of free tutoring for the MTH 1825 Final Exam,(cid:1) • the activities of the research project may help you remember and learn the course content, • your participation would help future students who take similar courses. For more details, please see the information on the website at: sites.google.com/view/mth1825researchproject Yes _____ Maybe ______ No______ 39. Is there anything else that you’d like to say that we didn’t ask? We’d like to know! THANK YOU FOR COMPLETING THIS QUESTIONNAIRE!!!!! 185 Appendix E: End of Course Questionnaire This survey is part of a research project documenting the experiences of students taking online Intermediate Algebra courses, such as MTH 1825. This study is being conducted by Jennifer (Jen) Nimtz, a doctoral student in the Program in Mathematics Education and to fulfill the dissertation requirements of a doctoral program. Survey Contact: Jennifer (Jen) Nimtz, Program in Mathematics Education, Michigan State University, nimtzjen@msu.edu This survey will take about 10 minutes. Your participation in this project is welcomed, but you are not required to participate. Also, this questionnaire is not connected to your grade or coursework in this or any courses that you are taking or may take in the future. Your responses to these items will be kept confidential and data will be assigned to a fake name. By completing and submitting this online survey, you are volunteering your responses. There is no monetary compensation associated with completing this survey. You may refuse to answer any questions or stop the survey at any time. Yes __________ No __________ Thank you in advance for your participation! Section 1. Personal Information and Background 1. Are you 18 years of age or older? 2. What is your name? (your preferred name) 3. Please take a few moments to share about your experience in MTH 1825. If you are having a hard time getting started, consider the following two questions: • What there are particular time when your attitude toward mathematics changed for the better or for the worse? If so, when was that? What happened? • Did you like MTH 1825? Why or why not? 4. What is your major? Has your major changed this semester? 186 _____Strong(cid:1) _____Above Average _____Average(cid:1) _____Below Average _____Weak 5. How would you describe your math ability? 6. Please elaborate on how you describe your math ability in relation to MTH 1825. 7. How would you describe your attitude toward math? 8. Please elaborate on how your experience in MTH 1825 influenced your attitude toward _____I enjoy math(cid:1) _____I am indifferent about math _____I dislike math math (or not). _____a lot _____somewhat _____not at all 9. How much do you think mathematics will play a role in your career? 10. Please elaborate on how mathematics will play a role in your career (or not). 11. How much do you think mathematics will play a role in your everyday life? 12. Please elaborate on how mathematics will play a role in your everyday life (or not). _____a lot _____somewhat _____not at all 187 Appendix F: Interview Protocols Math History Questionnaire Follow-Up Potential Questions (Use the Math History Questionnaire to structure this interview) 1. In your survey, I noticed that you answered that____. Would you please tell me more about that? Observed and Extended Think Aloud Potential Questions 1. Here I noticed that you wrote _____. Would you tell me more about your thinking? 2. I noticed that you said _____. Would you tell me more about your thinking? 3. Based on the topic you just completed, and the topic you are just starting, how are the two related? a. When you work in ALEKS, do you think about how the topics might be related? Do you consider what connections there might be between the topics? 4. I noticed that you were having some difficulty on this problem. What do you do in this online course when you are having difficulty solving a problem? a. What resources do you draw upon? For example, do you read the electronic text? Watch a video? Do you go to the Math Tutoring Center for help? Do you ask a friend for help? 5. As you’ve worked in ALEKS, I noticed this (ALEKS feature). Have you ever used it? 188 End of Course Interview (Also discuss the End of Course Questionnaire) 1. So, you are almost done with this online Intermediate Algebra course. What would you like to share about your experiences? a. What worked best for you in this course? b. What did not work very well for you in this course? 2. What about this course needs to change and what needs to stay the same? a. Do you appreciate how the course was structured? b. Do you feel like working in ALEKS helped you learn what you needed to know? 3. Was the mathematics you needed to know valuable or useful to you? 4. Because you did not have a lecture class, how did you learn the material? a. What resources do you draw upon? 5. What did you do when you had difficulty? a. What resources did you use when you had difficulty with the work? For instance, did you go to the MLC for help? Did you ask a friend for help? 189 Appendix G: IQA Task Potential (Boston & Wolf, 2006) Academic Rigor RUBRIC 1: Potential of the Task Did the task have potential to engage students in rigorous thinking about challenging content? concepts. The task has the potential to engage students in exploring and understanding the nature of mathematical concepts, procedures, and/or relationships, such as: (cid:120) Doing mathematics: using complex and non-algorithmic thinking (i.e., there is not a predictable, well-rehearsed approach or pathway explicitly suggested by the task, task instructions, or a worked-out example); OR (cid:120) Procedures with connections: applying a broad general procedure that remains closely connected to mathematical The task  must  explicitly  prompt  for  evidence  of  students’  reasoning  and  understanding.   For example, the task MAY require students to: (cid:120) solve  a  genuine,  challenging  problem  for  which  students’  reasoning  is  evident  in  their  work  on  the  task;; (cid:120) develop an explanation for why formulas or procedures work; (cid:120) identify patterns and form and justify generalizations based on these patterns; (cid:120) make conjectures and support conclusions with mathematical evidence; (cid:120) make explicit connections between representations, strategies, or mathematical concepts and procedures. (cid:120) follow a prescribed procedure in order to explain/illustrate a mathematical concept, process, or relationship. The task has the potential to engage students in complex thinking or in creating meaning for mathematical concepts,  procedures,  and/or  relationships.  However,  the  task  does  not  warrant  a  “4”  because:   (cid:120) (cid:120) the  task  does  not  explicitly  prompt  for  evidence  of  students’  reasoning  and  understanding. students may be asked to engage in doing mathematics or procedures with connections, but the underlying mathematics in the task is not appropriate for the specific group of students (i.e., too easy or too hard to promote engagement with high-level cognitive demands); students may need to identify patterns but are not pressed for generalizations or justification; students may be asked to use multiple strategies or representations but the task does not explicitly prompt students to develop connections between them; students may be asked to make conjectures but are not asked to provide mathematical evidence or explanations to support conclusions (cid:120) (cid:120) (cid:120) The potential of the task is limited to engaging students in using a procedure that is either specifically called for or its use is evident based on prior instruction, experience, or placement of the task. There is little ambiguity about what needs to be done and how to do it. The task does not require students to make connections to the concepts or meaning underlying the procedure being used. Focus of the task appears to be on producing correct answers rather than developing mathematical understanding (e.g., applying a specific problem solving strategy, practicing a computational algorithm). OR There is evidence that the mathematical content of the task is at least 2 grade-levels below the grade of the students in the class. The potential of the task is limited to engaging students in memorizing or reproducing facts, rules, formulae, or definitions. The task does not require students to make connections to the concepts or meaning that underlie the facts, rules, formulae, or definitions being memorized or reproduced. The task requires no mathematical activity. Students did not engage in a task. ATTACH OR DESCRIBE THE TASK. IQA Mathematics Lesson Observation Rubrics and Checklists, Melissa Boston ©2012 For permission to use, contact Melissa Boston, bostonm@duq.edu, 412-396-6109 8 4 3 2 1 0 N/A 190 Appendix H: ALEKS Learning Mode (ALEKS, 2016, pp. 31-32) Figure 39. ALEKS Learning Mode Reference 191 Example of a Problem Page in Learning Mode Below is an example of a problem in Learning Mode that points out key areas of the page with a description. For more details, please select on the links to go to the applicable section in this document. 2 1 4 3 5 6 7 8 9 1 | Home: Returns students to the homepage. 7 | Resources: Students will have access to learning resources (i.e. tools on the right side of the page) while they are working on problems. 8 | Explanation: Opens a pop-up with an explanation of how to solve the problem. Using this button does not count against the student’s score. 9 | Check: Checks the answer submitted by the student. 2 | Slice Name. 3 | Topic Name. 4 | Topic Carousel Tab: Opens/closes the Topic Carousel where students can choose other topics to work on. 5 | Underlined Mathematical Terms: Links to the dictionary. Students can select any term to get a complete definition. 6 | Progress Indicator: Displays immediate feedback messages and a counter to show how many correct answers students need in a row. ©2016 McGraw-Hill Education | Reference Guide: Student Module | Last Updated: 12.21.2016 32 Figure 40. ALEKS Learning Mode Reference 192 Appendix I: Think Aloud Recording Data Table Headings Data from each think aloud recording was organized in a table with headings as listed below. • Logistical Information o Semester (FS16 -or- SS17) o Semester Week Number o Student First Name (Pseudonym) o Recording Type (Think Aloud -or- Observed Extended Think Aloud) o Recording Session Number o Date of Recording o Length of Recording (Minutes and Seconds) o Problems Per Topic o ALEKS Sequence • ALEKS Topic (Topic as listed in the software) ALEKS Example Problem 1: Correct Problem 2: Correct Problem 3: Correct Next Topic o Context (Real World Application) o Problems Correct o Problems Incorrect o Problems Not Attempted o Cognitive Demand of the Task (IQA Rubric) o Routinized (ALEKS Sequence proceeds smoothly) o Critical Events (ALEKS Sequence included detours, unplanned events, application problems, and observer noted difference.) • Cognitive Interactions • Academic Interactions online resources) o Resources (e.g. ALEKS Example, ALEKS Explanation, ALEKS Video, other o Study Strategies (e.g. Note-taking, memory strategies, planning strategies) o Before Problem Confidence Statement (before beginning work on a problem) • Affective Interactions § Positive (e.g. “I get this.”) § Negative (e.g. “I have never been good at fractions.”) o After Problem Confidence Statement (after completing a problem and as the problem is entered into ALEKS for evaluation) § Positive (e.g. “That seems right.”) § Negative (e.g. “I think that’s wrong.”) 193 Appendix J: Sample Pages of Each Participant’s Written Work Figure 41. Sample Page of Jade's Written Work (Think Aloud 1, p. 1) 194 Jade solved Problem 3 mentally. Figure 42. Sample Page of Jade's Written Work (Think Aloud 1, p. 2) 195 Figure 43.Sample Page of Chad’s Written Work (Think Alouds 1 & 2, p. 1) 196 Figure 44. Sample Page of Chad’s Written Work (Think Aloud 2, p. 2) 197 Figure 45. Sample Page of Tia’s Written Work (Think Aloud 3, p. 1) 198 Figure 46. Sample Page of Tia’s Written Work (Think Aloud 3, p. 2) 199 REFERENCES 200 REFERENCES ALEKS. (2016). New ALEKS Student Module Reference Guide Retrieved from https://www.aleks.com/resources/New_Student_Module_Ref_Guide.pdf ALEKS. (n.d.). Overview of ALEKS. Retrieved from https://www.aleks.com/about_aleks/overview - origine Allen, I. E., & Seaman, J. (2011). Going the distance: Online education in the United States, 2011 Arendale, D. (2005). Terms of endearment: Words that define and guide developmental education. Journal of College Reading and Learning, 35(2), 68-82. Ashback, H. (2013). Bridging the gap: Adpative learning program developed at UCI supplements and reinforces math and science skills. UCI News. Retrieved from https://news.uci.edu/2013/07/01/bridging-the-gap/ Ashby, J., Sadera, W. A., & McNary, S. W. (2011). Comparing Student Success Between Developmental Math Courses Offered Online, Blended, and Face-to-face. Journal of Interactive Online Learning, 10(3), 128-140. Atkinson, R. K., Derry, S. J., Renkl, A., & Worthham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181-214. Bahr, P. R. (2010a). Preparing the underprepared: An analysis of racial disparities in postsecondary mathematics education. The Journal of Higher Education, 81(2), 209-237. Bahr, P. R. (2010b). Revisiting the efficacy of postsecondary remediation: The moderating effects of depth/breadth of deficiency. The Review of Higher Education, 33(Winter 2010), 177-205. Bailey, T. (2009). Rethinking developmental education in community college. CCRC Brief, (40), 1-4. Retrieved from http://ccrc.tc.columbia.edu/ Bailey, T., Jeong, D. W., & Cho, S.-W. (2009). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29, 255-270. Retrieved from Elsevier website: http://www.elsevier.com/locate/econedurev 201 Bazeley, P. (2013). Qualitative data analysis: Practical strategies. Thousand Oaks, CA: Sage. Bieda, K., Herbel-Eisenmann, B., McCrory, R., & Sikorski, P. (2013). Transforming Developmental Mathematics Education in Partnership with Teacher Preparation. Michigan State University: National Science Foundation. Biggs, J. (1979). Individual differences in study processes and the quality of learning outcomes. Higher Education, 8(4), 381-394. Boston, M. D., & Smith, M. S. (2009). Transforming secondary mathematics teaching: Increasing the cogntive demands of instructional tasks used in teachers' classrooms. Journal for Research in Mathematics Education, 40(2), 119-156. Boston, M. D., & Wolf, M. K. (2006). Assessing academic rigor in mathematics instruction: The development of the Instructional Quality Assessment toolkit: Regents of the University of California. Brint, S., Cantwell, A. M., & Hanneman, R. A. (2008). The two cultures of undergraduate academic engagement. Research in Higher Education, 49(5), 383-402. Campbell, C. M., & Cabrera, A. F. (2011). How sound is NSSE? Investigating the psychometric properties of NSSE at a public, research-extensive institution. The Review of Higher Education, 35(1), 77-103. Case, J., & Marshall, D. (2004). Between deep and surface: proceedural approaches to learning in engineering education contexts. Studies in Higher Education, 29(5), 605-615. Center for Postsecondary Research. (2017a). NSSE 2017 Codebook: U.S. Version. Retrieved from Bloomington, IN: http://nsse.indiana.edu/ Center for Postsecondary Research. (2017b). NSSE 2017 Overview. Retrieved from Bloomington, IN: Chickering, A. W., & Gamson, Z. F. (1987). Seven Principles for good practice in undergraduate education. AAHE Bulletin, 39, 3-7. Clowes, D. A. (1980). More than a definitional problem. Journal of Developmental and Remedial Education, 4(1), 8-10. Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches, Second Edition. Thousand Oaks, CA: Sage Publications, Inc. 202 Doignon, J.-P., & Falmagne, J.-C. (1985). Spaces for the assessment of knowledge. International Journal of Man-Machine Studies, 23(2), 175-196. Drake, S. (2010). Placement into first college mathematics course: A comparison of the results of the Michigan State University Proctored Mathematics Placement Examination and the Unproctored Mathematics Placement Examination. (Doctor of Philosophy), Michigan State University, Michigan State University. Dweck, C. S. (2009). Mindet: The New Psychology of Success. New York: Balantine Books. Engstrom, C., & Tinto, V. (2008). Access without support is not opportunity. Change: The Magazine for Higher Learning, 40(1), 46-50. Fain, P. (2012). Placement tests still rule Inside Higher Ed. Falmagne, J.-C., Albert, D., Doble, C., Eppstein, D., & Hu, X. (Eds.). (2013). Knowledge Spaces: Applications in Education. New York: Springer. Falmagne, J.-C., Doigon, J.-P., Cosyn, E., & Thiery, N. (2006). The assessment of knowledge in theory and in practice. Paper presented at the 4th International Conference on Formal Concept Analysis, Dresden, Germany. https://www.aleks.com/about_aleks/Science_Behind_ALEKS.pdf Finn, J. D., & Zimmer, K. S. (2012). Student engagement: What is it? Why does it matter? In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of Research on Student Engagement (pp. 97-131). New York: Springer. doi:10.1007/978-1-4614-2018-7_5 Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109. Gagne, R. M. (1965). The conditions of learning (1st ed.). New York: Holt, Rinehart and Winston. Gagne, R. M. (1985). The conditions of learning and theory of instruction (4th ed.). New York: Holt, Rinhart and Winston. Gagne, R. M., & Briggs, L. J. (1974). Principles of instructional design (1st ed.). New York: Holt, Rinehart and Winston. Gagne, R. M., Briggs, L. J., & Wager, W. W. (1992). Principles of instructional design (4th ed.). New York: Holt, Rinehart and Winston. 203 Gange, R. M. (1968). Contributions of learning to human development. Psychological Review, 75(3), 177-191. Ganter, S. L., & Haver, W. E. (Eds.). (2011). Partner Discipline Recommendations for Instructory College Mathematics and the Implications for College Algebra. Washington D.C.: MAA. Garofalo, J., & Lester, F. K. (1985). Metacognition: Cognitive monitoring and mathematical performance. Journal for Research in Mathematics Education, 16(3), 163-176. Gilbertson, N. J., Edson, A. J., Grant, Y., Nimtz, J., Phillips, E. D., & Ray, A. (2016). An analytic framework for examining curriculum-generated student work. Paper presented at the American Educational Research Association, Washington, D.C. Givvin, K. B., Stigler, J. W., & Thompson, B. J. (2011). What community college developmental mathematics students understand about mathematics, part 2: The interviews. MathAMATYC Educator, 2(3), 4-18. Glesne, C. (2011). Becoming qualitative researchers: An introduction (4 ed.). Boston: Pearson Education, Inc. Goldrick-Rab, S. (2010). Challenges and opportunities for improving community college student success. Review of Educational Research, 80(3), 437-469. doi:10.3102/0034654310371063 Grossberg, S. (1980). Paper presented at the SIAM/AMS Philadelphia. Gutierrez, R. (2013). The sociopolitical turn in mathematics education. Journal for Research in Mathematics Education, 44(1), 37-68. Higbee, J. L. (1996). Defining developmental education: A commentary. In J. L. Higbee & P. L. Dwinell (Eds.), Defining Developmental Education: Theory, Research, & Pedagogy. Cold Stream, IL: National Association for Developmental Education. Hill, R. (2006). On the transition in mathematics from high school to Michigan State University. Retrieved from http://math.msu.edu/~hill website: http://math.msu.edu/~hill Hodges, C. B., & Hunger, G. M. (2011). Communicating Mathematics on the Internet: Synchronous and Asynchronous Tools. Tech Trends, 55(5), 39-44. Jaggars, S. S., & Bailey, T. (2010). Effectiveness of fully online courses for college students: Response to a Department of Education meta-analysis. Retrieved from New York: 204 Jimmerson, S., Campos, E., & Greif, J. (2003). Toward an understanding of definitions and measures of school engagement and related terms. The California School Psychologist, 8, 7-27. Johanes, P., & Lagerstrom, L. (2017). Adaptive learning: The premise, promise, and pitfalls. Paper presented at the 2017 ASEE Annual Conference & Exposition, Columbus, OH. https://www.asee.org/public/conferences/78/author_index/83158 Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38(5), 758-773. doi:10.1080/03075079.2011.598505 Kanter, M. J. (2009). Partitioning: Instructor mathematics philosophy shaping community college mathematics. Paper presented at the A Declaration of Numeracy: Empowering Adults Through Mathematics Education, Proceedings of the 15th International Conference of Adults Learning Mathematics, Philadelpia, PA. Keller, J., Bower, B. L., & Chen, P. D. (2015). Investigating Instructional Methods in Community Colleges Developmental Mathematics. MathAMATYC Educator, 7(1), 4-13, 57. Krause, A., & Putnam, R. T. (2016). Online calculus homework: The student experience. Paper presented at the Conference on Research in Undergraduate Mathematics Education, Pittsburgh, PA. Kuh, G. D. (2001). The National Survey of Student Engagement: Concpetual framework and overview of psychometric properties. Retrieved from Bloomington, IN: Kuh, G. D. (2009). The National Survey of Student Engagement: Conceptual and empirical foundations. New directions for institutional research, 141, 5-20. Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., & Emerson, R. M. (2008). Unmasking the effects of student engagment on first-year college grades and persistence. The Journal of Higher Education, 79(5), 540-563. Lampert, M. (1990). When the problem is not the question and the solution is not the answer: Mathematica knowing and teaching. American Educational Research Journal, 27(1), 29- 63. Larnell, G. V. (2011). More than just skill: Mathematics identies, socialization, and remediation among African American undergraduates. (Doctor of Philosophy Dissertation), Michigan State University. 205 Larnell, G. V. (2016). More than just skill: Examining mathematics identities, racialized narratives, and remediation among black undergraduates. Journal for Research in Mathematics Education, 47(3), 233-269. Latterell, C. M., & Regal, R. R. (2003). Are placement tests for incoming undergraduate mathematics students worth the expense of administration? PRIMUS: Problems, Resources, and Issues in Mathematics Undergraduate Studies, 13(2), 152-164. Lawson, M. H., & Lawson, H. A. (2013). New conceptual frameworks for student engagement research, policy, and practice. Review of Educational Research, 83(3), 432-479. Lockee, B., Moore, D. M., & Burton, J. (2004). Foundations of programmed instruction. In D. H. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology (pp. 545-569). Mahwah, NJ: Lawrence Erlbaum. Mann, S. J. (2001). Alternate perspectives on the student experience: Alienation and engagement. Studies in Higher Education, 26(1), 7-19. Martin, D. B. (2000). Mathematics success and failure among African-American youth: The roles of sociohistorical contex, community forces, school influence, and individual agency. Mahwah, NY: Erlbaum. Martin, D. B. (2009a). Liberating the production of knowledge about African American children and mathematics. In D. B. Martin (Ed.), Mathematics teaching, learning, and liberation in the lives of black children. New York: Routledge. Martin, D. B. (2009b). Researching race in mathematics education. Teachers College Record, 111(2), 295-338. Maxwell, J. A. (1996). Qualitative Research Design: An Interactive Approach. Thousand Oaks, CA: Sage Publications, Inc. McMahon, B., & Portelli, J. P. (2004). Engagement for what? Beyond popular discourses of student engagement. Leadership and Policy in Schools, 3(1), 59-77. Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jomes, K. (2009). Evaluation of Evidence- Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. Center for Technology in Learning, ED-04- CO-0040 Task 0006. Retrieved from Mesa, V. (2010). Student participation in mathematics lessons taught by seven successful community college instructors. Adults Learning Mathematics - An International Journal, 5(1), 64-88. 206 Meza, E. J. (2015). Programmatic practices that promote student success in community college math developmental education: A mixed methods study. (Doctor of Philosophy Dissertation), University of Washington. Michigan State University. (2016). MSU data breach exposed records. MSU Today. Retrieved from https://msutoday.msu.edu/news/2016/msu-data-breach-exposed-records/ Moses, R. P., & Cobb, C. E. (2001). Radical equations: Civil rights from Mississippi to the Algebra Project: Beacon Press. Nason, R., & Woodruff, E. (2004). Online collaborative learning in mathematics: Some necessary innovations. In T. S. Roberts (Ed.), Online Collaborative Learning: Theory and Practice (pp. 103-131). Hershey: Information Science Publishing. NCTM. (2009). Focus in high school mathematics: Reasoning and sense making. Reston, VA: Author. New Oxford American Dictionary. (2005). New York: Oxford University Press. Nimtz, J. L., Gilbertson, N. J., Ray, A., Edson, A. J., Grant, Y., & Phillips, E. D. (2015). Student work as a context for student learning. Paper presented at the 37th annual meeting of the North American Chapter of the International Group for the Psychology of Matheamtics Education, East Lansing, MI. Pace, C. R. (1984). Measuring the quality of college student experiences: An account of the development and use of the College Student Experiences Questionnaire. Los Angeles: Higher Education Research Institute. Park, O., & Lee, J. (2004). Adaptive instructional systems. In D. H. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology (pp. 651-684). Mahwah, NJ: Lawrence Erlbaum. Parsad, B., & Lewis, L. (2003). Remedial Education at Degree-Granting Postsecondary Institutions in Fall 2000. Statistical Analysis Report. Retrieved from Washington DC.; Rockville, MD: Piper, J. (1998). An interview with Martha Maxwell. The Learning Assistance Review, 3(1), 32- 39. Polya, G. (1985). How to solve it: A new aspect of mathematical method (Second ed.). Princeton, NJ: Princeton University Press. 207 Porter, S. R. (2011). Do college student surveys have any validity? The Review of Higher Education, 35(1), 45-76. Reiser, R. A. (2001). A history of instructional design and technology: Part II: A history of instructional design. Educational Technology Research and Development, 49(2), 57-67. Rittle-Johnson, B., & Star, J. R. (2011). The power of comparison in learning and instruction: Learning outcomes supported by different types of comparisons. Psychology of Learning and Motivation, 55, 199-225. doi:10.1016/B978-0-12-387691-1.00007-7 Rogoff, B. (2008). Observing sociocultural activity on three planes: Participatory appropriation, guided participation, and apprenticeship. In K. Hall, P. Murphy, & J. Soler (Eds.), Pedagogy and practice: Culture and Identities. Los Angeles, CA: Sage. Rose, M. (1989). Lives on the boundary: The struggles and achievements of America's underprepared. New York: Free Press. Rose, M. (2009). Blue-collar brilliance: Questioning assumptions about intelligence, work, and social class. The American Scholar, 78(3), 43-49. Schoenfeld, A. H. (1985). Mathematical problem solving. San Diego, CA: Academic Press, Inc. Schoenfeld, A. H. (1988). When good teaching leads to bad results: The disasters of 'well-taught' mathematics courses. Educational Psychologist, 23(2), 145-166. Schoenfeld, A. H. (1992). Learning to think mathematically: Problem Solving, metacognition, and sense-making in mathematics. In D. Grouws (Ed.), Handbook for Research on Mathematics Teaching and Learning (pp. 334-370). New York: MacMillan. Schrock, S. A. (1995). A brief history of instructional development. In G. J. Anglin (Ed.), Instructional Technology: Past, Present, and Future (pp. 11-19). Englewood, CO: Libraries Unlimited. Silver, E. A., Smith, M. S., & Nelson, B. S. (1995). The QUASAR project: Equity concerns meet mathematics education reform in the middle school. In W. G. Secada, E. Fennema, & L. B. Adajian (Eds.), New directions for equity in mathematics education (pp. 9-56). Skemp, R. R. (1977). Relational understanding and instrumental understanding. Mathematics Teaching, 77, 20-26. Skinner, B. F. (1954). The science of learning and the art of teaching. Harvard Educational Review, 24, 86-97. 208 Skinner, B. F. (1958). Teaching machines. Science, 128, 969-977. Skinner, B. F. (1968). Technology of Teaching. New York: Appleton-Century-Crofts Educational Division, Meredith Corporation. Snyder, T. D., Brey, C. d., & Dillow, S. A. (2016). Digest of Education Statistics 2014 Stein, M. K., Smith, M. S., Henningsen, M., & Silver, E. A. (2000). Implementing standards- based mathematics instruction: A casebook for professional development. New York: Teachers College Press. Stigler, J. W., Givvin, K. B., & Thompson, B. (2010). What community college developmental mathematics students understand about mathematics. MathAMATYC Educator, 1(3), 4- 16. Stinson, D. W. (2013). Negotiating the "white male math myth": African American male students and success in school mathematics. Journal for Research in Mathematics Education, 44(1), 69-99. Twigg, C. A. (2011). The math emporium: A silver bullet for higher education. Change: The Magazine for Higher Learning, 43(3), 25-34. Venkataramani, A., & Tsai, A. (2017). Dreams deferred - The public health consequences of rescinding the DACA. The New England Journal of Medicine, 377(18), 1707-1709. Vivien Beattie IV, Collins, B., & McInnes, B. (1997). Deep and surface learning: A simple or simplistic dichotomy. Accounting Education, 6(1), 1-12. doi:10.1080/096392897331587 Webel, C., Krupa, E., & McManus, J. (2015). Benny goes to college: Is the "math emporium" reinventing individually prescribed instruction? MathAMATYC Educator, 6(3), 4-14. Weller, K., Trouba, J., & Wood, R. (2015). A comparison of instructional delivery formats in intermediate algebra. MathAMATYC Educator, 7(1). Williams, D. R., & Medlock, M. M. (2017). Health effects of dramatic societal events - ramifications of the recent presidential election. The New England Journal of Medicine, 376(23), 2295-2299. Zavarella, C. A., & Ignash, J. M. (2009). Instructional delivery in developmental mathematics: Impact on retention. Journal of Developmental Education, 32(3), 2-13. Zepke, N. (2017). Student engagement in neoliberal times: Theories and practices for learning and teaching in higher education. Singapore: Springer. 209