INVESTIGATING HIGH SCHOOL MATHEMATICS TEACHERS’ FORMATIVE ASSESSMENT PRACTICES By Joanne Philhower A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Mathematics Education – Doctor of Philosophy 2018 ABSTRACT INVESTIGATING HIGH SCHOOL MATHEMATICS TEACHERS’ FORMATIVE ASSESSMENT PRACTICES By Joanne Philhower Although formative assessment practices have been shown to increase students’ achievement (Black & Wiliam, 1998), limited research has described or analyzed what these practices look like in mathematics classrooms—especially high school classrooms. Building on prior work (Black & Wiliam, 1998; Cameron, Gawroski, Eich, & McCready, 2011; Chappuis, Stiggins, Chappuis, & Arter; 2012; McMillan, 2007) this study conceptualizes formative assessment as a process that involves teachers and/or students gathering evidence about students’ thinking in order to make decisions to improve instruction and student learning. It focused on six practices: learning targets, questioning, feedback, self-assessment, peer assessment, and instructional decisions. I examined how six high school mathematics teachers implemented formative assessment practices in two series of lessons taught in their classrooms. Through background, pre-lesson, post-lesson, and exit interviews, data was gathered about which practices teachers were implementing, how they were implementing them in their classrooms, and what they saw as supports and obstacles when trying to implement these types of practices in their classrooms. The study nested the formative assessment practices onto a lesson structure framework I developed to show in which activity structures teachers incorporated each of these formative assessment practices and at what level of expertise these practices were implemented. I found that whole class discussions and pair/small group work included the highest occurrences of formative assessment practices. I also found that questioning and feedback were the practices most often implemented by this group of teachers. These teachers also saw professional development, teacher resources, and their students as supports for implementing formative assessment in their classrooms. In addition, they saw their students and time as the most common hindrances in their ability to implement these practices. ACKNOWLEDGEMENTS I would like to thank my dissertation committee, Dr. Jack Smith (chair), Dr. Corey Drake, Dr. Kristen Bieda, and Dr. Alicia Alonzo. I greatly appreciate the support and feedback from these four outstanding individuals. A special thank you to Jack for meeting with me on a regular basis to make sure my dissertation was completed and was something I could be proud of. The multiple rounds of feedback are most appreciated. A special thank you also to Corey for meeting on a number of occasions and providing valuable feedback on preliminary frameworks and on multiple chapters in their various forms of doneness. A huge thank you to my dear friend, Amy Ray, for coding with me so I could check inter-rater reliability. Your feedback and our conversations were incredibly helpful and interesting. Your support throughout this process has meant the world to me! Another huge thank you to my dear friend Rani Satyam for listening to me throughout this process, for letting me vent and offering support at every turn. I must also thank my loving family: Elmer and Cindy Philhower (my super awesome parents), Jason, Heather, Jarod, and Bailey Philhower, and Jenn, Jeff, Brelynn, and Brody Wise for supporting me not only throughout this dissertation process, but for their love and support throughout my Ph.D. journey. It has been quite the adventure and I definitely would not have survived it without them cheering me on every step of the way. I would also like to thank those who helped me recruit participants for my dissertation. I cannot name them at the risk of identifying my participants, but without their help, I would not have been able to find the six wonderful teachers who allowed me into their classrooms. Lastly, and probably most importantly, I would like to thank those amazing teachers and their students for allowing me to come visit and part of their iv classroom community, so I could collect valuable and interesting data about how teachers are using formative assessment in their mathematics classrooms. v TABLE OF CONTENTS LIST OF TABLES ........................................................................................................................ xii LIST OF FIGURES ..................................................................................................................... xiv Chapter 1 Introduction .................................................................................................................... 1 Chapter 2: Literature Review .......................................................................................................... 6 Definitions of Formative Assessment ......................................................................................... 6 Analysis. ................................................................................................................................ 11 Formative Assessment Practices in Mathematics ..................................................................... 13 Defining clear learning objectives. ........................................................................................ 14 Gathering evidence. ............................................................................................................... 15 Questioning ............................................................................................................................ 16 Peer Assessment .................................................................................................................... 18 Self-Assessment..................................................................................................................... 21 Making instructional decisions .............................................................................................. 24 Providing feedback ................................................................................................................ 27 Integrated Formative Assessment Practices in Mathematics .................................................... 30 Supports and Obstacles When Implementing Formative Assessment ...................................... 32 Factors That Support Formative Assessment Practices ......................................................... 33 Factors That Hinder Formative Assessment Practices .......................................................... 34 Chapter 3 Methods ........................................................................................................................ 38 Overview ................................................................................................................................... 38 Setting and Participants ............................................................................................................. 38 Data Collection .......................................................................................................................... 40 Background interview ............................................................................................................ 40 Lesson planning interview ..................................................................................................... 41 Debriefing interview .............................................................................................................. 41 Analysis of Classroom Observations and Interviews ................................................................ 42 Lesson Observations .............................................................................................................. 48 Lesson Structure Framework ................................................................................................. 49 Addressing research question 1 ............................................................................................. 50 Addressing research question 2 ............................................................................................. 51 Addressing research question 3 ............................................................................................. 56 Inter-Rater Reliability ............................................................................................................... 57 Chapter 4: Teachers’ Lesson Structure as Context for Formative Assessment ............................ 67 Charlie’s Lesson Structure ........................................................................................................ 68 Evan’s Lesson Structure ............................................................................................................ 70 Gwen’s Lesson Structure .......................................................................................................... 72 vi Lindsay’s Lesson Structure ....................................................................................................... 74 Lukas’s Lesson Structure .......................................................................................................... 76 Scott’s Lesson Structure ............................................................................................................ 79 Comparisons Across Teachers .................................................................................................. 81 Chapter 5: Teachers’ Formative Assessment Practices Findings ................................................. 84 Formative Assessment Practices by Activity ............................................................................ 85 Charlie’s Formative Assessment Practices ............................................................................... 89 Learning Targets .................................................................................................................... 89 Questioning types .................................................................................................................. 90 Self-assessment ...................................................................................................................... 91 Feedback ................................................................................................................................ 91 Instructional Decisions .......................................................................................................... 92 Summary ................................................................................................................................ 92 Evan’s Formative Assessment Practices ................................................................................... 93 Learning Targets .................................................................................................................... 94 Questioning types .................................................................................................................. 94 Self-assessment ...................................................................................................................... 95 Feedback ................................................................................................................................ 95 Instructional Decisions .......................................................................................................... 96 Summary ................................................................................................................................ 96 Gwen’s Formative Assessment Practices .................................................................................. 96 Learning Targets .................................................................................................................... 97 Questioning types .................................................................................................................. 97 Feedback ................................................................................................................................ 98 Instructional Decisions .......................................................................................................... 98 Summary ................................................................................................................................ 98 Lindsay’s Formative Assessment Practices .............................................................................. 99 Questioning types ................................................................................................................ 100 Feedback .............................................................................................................................. 101 Instructional Decisions ........................................................................................................ 101 Summary .............................................................................................................................. 101 Lukas’s Formative Assessment Practices ............................................................................... 102 Learning Targets .................................................................................................................. 103 Questioning types ................................................................................................................ 103 Feedback .............................................................................................................................. 103 Instructional Decisions ........................................................................................................ 104 Summary .............................................................................................................................. 104 Scott’s Formative Assessment Practices ................................................................................. 105 Learning Targets .................................................................................................................. 106 Questioning .......................................................................................................................... 106 Self-assessment .................................................................................................................... 107 Feedback .............................................................................................................................. 107 Instructional Decisions ........................................................................................................ 107 Summary .............................................................................................................................. 107 Formative Assessment Practices Across Teachers.................................................................. 108 vii Learning targets ................................................................................................................... 111 Questioning .......................................................................................................................... 111 Self-assessment. ................................................................................................................... 112 Feedback .............................................................................................................................. 113 Instructional decisions ......................................................................................................... 114 Chapter 6: Formative Assessment Practices in Context ............................................................. 118 Charlie’s Experiences and Beliefs About Teaching ................................................................ 118 School context ..................................................................................................................... 118 Teacher background ............................................................................................................ 119 Teaching assignment and curriculum .................................................................................. 119 Classroom context ............................................................................................................... 119 Definition of formative assessment ..................................................................................... 119 Professional development opportunities .............................................................................. 120 Evan’s Experiences and Beliefs About Teaching ................................................................... 120 School context ..................................................................................................................... 120 Teacher background ............................................................................................................ 120 Teaching assignment and curriculum .................................................................................. 120 Classroom context ............................................................................................................... 121 Definition of formative assessment ..................................................................................... 121 Professional development opportunities .............................................................................. 122 Gwen’s Experiences and Beliefs About Teaching .................................................................. 122 School context ..................................................................................................................... 122 Teacher background ............................................................................................................ 123 Teaching assignment and curriculum .................................................................................. 123 Classroom context ............................................................................................................... 124 Definition of formative assessment ..................................................................................... 124 Professional development opportunities .............................................................................. 124 Lindsay’s Experiences and Beliefs About Teaching ............................................................... 125 School context ..................................................................................................................... 125 Teacher background ............................................................................................................ 125 Teaching assignment and curriculum .................................................................................. 125 Classroom context ............................................................................................................... 126 Definition of formative assessment ..................................................................................... 126 Professional development opportunities .............................................................................. 127 Scott’s Experiences and Beliefs About Teaching ................................................................... 127 School context ..................................................................................................................... 128 Teacher background ............................................................................................................ 128 Teaching assignment and curriculum .................................................................................. 128 Classroom context ............................................................................................................... 128 Definition of formative assessment ..................................................................................... 129 Professional development opportunities .............................................................................. 129 Lukas’s Experiences and Beliefs About Teaching .................................................................. 130 School context ..................................................................................................................... 130 Teacher background ............................................................................................................ 130 Teaching assignment and curriculum .................................................................................. 130 viii Classroom context ............................................................................................................... 131 Definition of formative assessment ..................................................................................... 132 Professional development opportunities .............................................................................. 132 Teachers’ Descriptions of Activities and Strategies Related to Formative Assessment ......... 133 Activity Structures................................................................................................................... 133 Warm-up .............................................................................................................................. 133 Introduction ......................................................................................................................... 135 Presenting student work....................................................................................................... 135 Pair/small group work ......................................................................................................... 136 Summary .............................................................................................................................. 137 In-Class Activities ................................................................................................................... 137 Goal check ........................................................................................................................... 138 Mini whiteboards ................................................................................................................. 138 Silent squares activity .......................................................................................................... 139 Jigsaw activity ..................................................................................................................... 139 Desmos ................................................................................................................................ 140 Team strategies .................................................................................................................... 140 Summary .............................................................................................................................. 141 Assessments ............................................................................................................................ 141 Learning checks ................................................................................................................... 142 Homework quizzes .............................................................................................................. 142 Pre-assessment ..................................................................................................................... 143 Homework ........................................................................................................................... 143 Summary .............................................................................................................................. 145 Instructional Strategies ............................................................................................................ 146 Activating prior knowledge ................................................................................................. 146 Monitoring ........................................................................................................................... 147 Students’ sharing out answers ............................................................................................. 148 Restating peers’ ideas .......................................................................................................... 148 Looking for multiple strategies ............................................................................................ 148 Summary .............................................................................................................................. 150 Analysis of Teachers’ Descriptions and Observed Formative Assessment Practices ............. 150 Learning Targets ..................................................................................................................... 150 Charlie ................................................................................................................................. 150 Gwen .................................................................................................................................... 152 Lindsay ................................................................................................................................ 153 Scott ..................................................................................................................................... 153 Lukas ................................................................................................................................... 154 Analysis ............................................................................................................................... 155 Differences based on observations ...................................................................................... 156 Questioning ............................................................................................................................. 156 Charlie ................................................................................................................................. 156 Evan ..................................................................................................................................... 159 Gwen .................................................................................................................................... 160 Lindsay ................................................................................................................................ 162 Scott ..................................................................................................................................... 164 ix Lukas ................................................................................................................................... 165 Analysis ............................................................................................................................... 167 Differences based on observations ...................................................................................... 168 Self-assessment ....................................................................................................................... 170 Charlie ................................................................................................................................. 170 Evan ..................................................................................................................................... 170 Gwen .................................................................................................................................... 171 Lindsay ................................................................................................................................ 171 Scott ..................................................................................................................................... 171 Lukas ................................................................................................................................... 172 Analysis ............................................................................................................................... 173 Differences based on observations ...................................................................................... 173 Peer assessment ....................................................................................................................... 175 Charlie ................................................................................................................................. 175 Evan ..................................................................................................................................... 175 Gwen .................................................................................................................................... 176 Lindsay ................................................................................................................................ 176 Scott ..................................................................................................................................... 177 Lukas ................................................................................................................................... 177 Analysis ............................................................................................................................... 177 Differences based on observations ...................................................................................... 178 Feedback.................................................................................................................................. 179 Charlie ................................................................................................................................. 179 Evan ..................................................................................................................................... 179 Gwen .................................................................................................................................... 180 Lindsay ................................................................................................................................ 180 Scott ..................................................................................................................................... 180 Lukas ................................................................................................................................... 180 Analysis ............................................................................................................................... 181 Differences based on observations ...................................................................................... 182 Instructional Decisions ............................................................................................................ 183 Charlie ................................................................................................................................. 183 Evan ..................................................................................................................................... 184 Gwen .................................................................................................................................... 184 Lindsay ................................................................................................................................ 185 Lukas ................................................................................................................................... 186 Analysis ............................................................................................................................... 186 Differences based on observations ...................................................................................... 187 Chapter 7: Charlie’s High-Quality Formative Assessment Practice ........................................... 190 June 1 Lesson .......................................................................................................................... 191 June 2 Lesson .......................................................................................................................... 198 June 3 Lesson .......................................................................................................................... 203 Summary ................................................................................................................................. 211 Chapter 8: Supports and Obstacles to Implementing Formative Assessment Practices ............. 216 x Supports in Implementing Formative Assessment .................................................................. 216 Writing learning targets ....................................................................................................... 218 Determining questions ......................................................................................................... 218 Feedback .............................................................................................................................. 219 Self-assessment .................................................................................................................... 220 Peer assessment ................................................................................................................... 221 Instructional decisions ......................................................................................................... 222 Summary .............................................................................................................................. 223 Obstacles When Implementing Formative Assessment .......................................................... 226 Learning targets ................................................................................................................... 227 Questioning .......................................................................................................................... 228 Feedback .............................................................................................................................. 229 Self-assessment .................................................................................................................... 231 Peer assessment ................................................................................................................... 232 Instructional decisions ......................................................................................................... 234 Summary .............................................................................................................................. 236 Supports and obstacles......................................................................................................... 239 Chapter 9: Discussion ................................................................................................................. 241 Lesson Structure ...................................................................................................................... 242 Formative Assessment Practices ............................................................................................. 244 Activity structure as a context for formative assessment .................................................... 245 Formative assessment practices levels of sophistication ..................................................... 255 Supports and Obstacles for Implementing Formative Assessment ......................................... 258 Summary ................................................................................................................................. 261 Implications ............................................................................................................................. 263 APPENDICES ............................................................................................................................ 265 Appendix A: Background Interview Protocol ......................................................................... 266 Appendix B: Lesson Planning Interview ................................................................................ 268 Appendix C: Lesson Debriefing Interview ............................................................................. 269 Appendix D: Classroom Video Formative Assessment Practices Coding Framework .......... 270 Appendix E: Lesson Activity Codes and Definitions ............................................................. 275 Appendix F: Coding Framework External Supports and Obstacles ....................................... 276 REFERENCES ........................................................................................................................... 277 xi LIST OF TABLES Table 1 Teachers’ Demographic Information ............................................................................... 39 Table 2 IRR Formative Assessment Practices .............................................................................. 58 Table 3 IRR Formative Assessment Practices by Lesson Segment .............................................. 61 Table 4 Formative Assessment Practices by Lesson Segment and Level of Practice .................. 63 Table 5 Charlie’s Total Time Per Activity Structure .................................................................... 69 Table 6 Evan’s Total Time Per Activity Structure ....................................................................... 71 Table 7 Gwen’s Total Time Per Activity Structure ...................................................................... 73 Table 8 Lindsay’s Total Time Per Activity Structure ................................................................... 75 Table 9 Lukas’s Total Time Per Activity Structure ...................................................................... 77 Table 10 Scott’s Total Time Per Activity Structure ..................................................................... 80 Table 11 Teachers’ Percentage of Time Per Activity Structure ................................................... 81 Table 12 Formative Assessment Practices by Activity Structure ................................................. 85 Table 13 Teachers’ Formative Assessment Practices by Activity Structure ................................ 87 Table 14 Teachers’ Percentage of Total Activities Including Formative Assessment Practices 109 Table 15 Teachers’ Formative Assessment Weighted Averages ................................................ 109 Table 16 Teachers’ Level of Sophistication for Learning Targets by Activity Structure .......... 111 Table 17 Teachers’ Level of Sophistication for Questioning by Activity Structure .................. 112 Table 18 Teachers’ Level of Sophistication for Self-Assessment by Activity Structure ........... 113 Table 19 Teachers’ Level of Sophistication for Feedback by Activity Structure ....................... 113 Table 20 Teachers’ Level of Sophistication for Instructional Decisions by Activity Structure . 114 Table 21 Formative Assessment Weighted Averages by Activity Structure .............................. 115 Table 22 Charlie’s Questioning Practices ................................................................................... 158 xii Table 23 Evan’s Questioning Practices ...................................................................................... 159 Table 24 Gwen’s Questioning Practices ..................................................................................... 161 Table 25 Lindsay’s Questioning Practices .................................................................................. 163 Table 26 Scott’s Questioning Practices ...................................................................................... 164 Table 27 Lukas’s Questioning Practices ..................................................................................... 167 Table 28: The June 1 Lesson ...................................................................................................... 192 Table 29: The June 2 Lesson ...................................................................................................... 198 Table 30: The June 3 Lesson ...................................................................................................... 204 Table 31 Supports in Implementing Formative Assessment ...................................................... 216 Table 32 External Supports to Implementing Formative Assessment ........................................ 217 Table 33 Obstacles to Implementing Formative Assessment Practices ...................................... 226 Table 34 External Obstacles to Implementing Formative Assessment ...................................... 227 Table 35 Lindsay’s and Scott’s Percentage of Time Per Activity Structure .............................. 242 Table 36 Evan’s and Gwen’s Percentage of Time Per Activity Structure .................................. 243 xiii LIST OF FIGURES Figure 1 Activity Structures .......................................................................................................... 67 Figure 2 Charlie’s Activity Structures by Lesson ......................................................................... 69 Figure 3 Evan’s Activity Structures by Lesson ............................................................................ 70 Figure 4 Gwen’s Activity Structures by Lesson ........................................................................... 72 Figure 5 Lindsay’s Activity Structures by Lesson ........................................................................ 74 Figure 6 Lukas’s Activity Structures by Lesson ........................................................................... 77 Figure 7 Scott’s Activity Structures by Lesson ............................................................................ 79 Figure 8 Charlie’s Formative Assessment Practices by Lesson ................................................... 89 Figure 9 Evan’s Formative Assessment Practices by Lesson ....................................................... 94 Figure 10 Gwen’s Formative Assessment Practices by Lesson .................................................... 97 Figure 11 Lindsay’s Formative Assessment Practices by Lesson .............................................. 100 Figure 12 Lukas’s Formative Assessment Practices by Lesson ................................................. 102 Figure 13 Scott’s Formative Assessment Practices by Lesson ................................................... 106 xiv Chapter 1 Introduction “Formative assessment can, indeed, transform the way a teacher teaches” (Popham, 2005, p. 273). This implies that if teachers choose to implement formative assessment practices in their classrooms, it will likely lead to a shift in their teaching practices. Implementing formative assessment practices, is not a new idea, though. Over twenty years ago, Black and Wiliam (1998) published their foundational work that found student achievement was raised when teachers incorporated formative assessment practices. Very little research has been conducted since then on how teachers are using these practices in their classrooms, though, so it would appear that the field still has much to learn. We need to conduct research studies and find ways to learn more about what formative assessment practices teachers are using and how they are implementing formative assessment in their classrooms. I became interested in formative assessment after reading a book by Grant Wiggins entitled Assessing Student Performance: Exploring the Purpose and Limits of Testing (Wiggins, 1999). During my tenure as a high school mathematics teacher, I had not learned about formative assessment, therefore, this book was eye-opening to me and redefined my graduate program trajectory. I am passionate about what happens in mathematics classrooms and want to find ways to support teachers in helping students learn mathematics. Implementing formative assessment practices in classrooms seemed like a meaningful way to improve instruction and support student learning. I have spent the rest of the time in my Ph.D. program learning more about formative assessment and finding ways to incorporate these practices into my collegiate level teaching. I have also spent the last few years supporting pre-service teachers in developing their own knowledge about and ability to implement formative assessment practices. When I thought about 1 what the focus of my dissertation should be, it was obvious that I should investigate how teachers were using formative assessment in their mathematics classrooms. Assessment has become even more of an integral part of the conversation over the last ten years. Studies that Campbell (2013) reported on in her handbook chapter still express concerns about teachers’ preparation, knowledge of assessment, and knowledge of implementing and interpreting assessment results. It is likely that teachers will be expected to develop stronger assessment practices over the next few years, “given the current climate of school accountability and data-driven decision making” (Campbell, 2013, p. 81), therefore, teachers need to focus on developing their assessment practices in order to modify instruction and improve student learning. Since formative assessment has been shown to promote higher achievement for students (Black & Wiliam, 1998), incorporating these types of practices into instruction seems like a productive idea. Changing teaching practices to implement more formative assessment practices is not likely an easy task, though. Teachers willing to embark on this type of classroom “end up thinking about instruction in a manner profoundly different from the way most teachers have traditionally conceived of teaching” (Popham, 2005, p. 273). It can require a change in teaching philosophy, including the role of teachers and students, as well as an initial time commitment to develop the necessary tools (Popham, 2005). Even though schools and teachers may see formative assessment as a valuable resource for improving instruction, teachers are often faced with obstacles that hinder their ability to implement formative assessment (Heritage, Kim, Vendlinski, & Herman, 2009; McMillan, 2003; Suurtamm & Koch, 2014; Suurtamm, Koch, & Arden, 2010). Research has shown, though, that if teachers are able and willing to make changes 2 to their classroom culture and instruction and overcome obstacles, then students are likely to blossom. Due to the obstacles that we know teachers face when implementing formative assessment practices, we need to know more about what implementation of formative assessment looks like in classrooms. These examples could demonstrate how teachers who are interested in implementing formative assessment practices have used their formative assessment knowledge to incorporate these practices into their classrooms. This could then provide a model for other teachers to follow who also want to implement formative assessment practices in their own classrooms. Examples from the current study are diverse, therefore, teachers and researchers could see the multiple ways in which formative assessment could be incorporated into mathematics classrooms. Past research on formative assessment has included multiple content areas (Volante, 2010), research specific to science teachers’ practices (Box, Skoog, & Dabbs, 2015; Cowie & Bell, 1999), or studies with science and mathematics teachers (Gotwals, Philhower, Cisterna, & Bennett, 2015; Wiliam, Lee, Harrison, & Black, 2004). Some articles have focused on particular aspects of formative assessment in mathematics, such as self-assessment (Brookhart, Andolina, Zuza, & Furman, 2004; Ross, Hogaboam-Gray, & Rolheiser, 2002) or feedback (Rakoczy, Harks, Klieme, Blum, & Hochweber, 2013; Shih & Alexander, 2000). However, there has been very little research on teachers’ formative assessment practices as a whole in mathematics classrooms (Peterson & Siadat, 2009; Suurtamm, Koch, & Arden, 2010: Suurtamm & Koch, 2014; Suurtamm, 2004). Most of these studies focus on elementary or middle school mathematics classrooms, meaning high school mathematics is not well- represented in the literature. Therefore, there is reason to focus on high school mathematics 3 teachers because there is a gap in the research that needs to be addressed. Investigating teachers’ formative assessment use is also an important issue to consider since incorporating formative assessment likely involves a shift in teachers’ instructional practices, therefore, seeing how teachers have implemented these types of practices would be helpful to others hoping to implement them in their own classrooms. I also examined what teachers viewed as helping and hindering their ability to implement formative assessment practices in their classrooms. To address these issues, the following research questions guided this study: 1) What formative assessment practices are some high school mathematics teachers implementing in their lessons to support students’ learning of the mathematical content? 2) How do a sample of high school mathematics teachers make sense of and implement formative assessment practices, in the context of their views of their subject, their students and school, and their broader practice? 3) What do some high school mathematics teachers see as supports and obstacles in implementing formative assessment practices in their classrooms? The first two questions are related, but they address different aspects of teachers’ formative assessment practices. In the first question, the focus is on whether the formative assessment practices were present in each teacher’s observed lessons based on the researcher’s perspective. In the second question, the focus is on how teachers made sense of these formative assessment practices and how they enacted them in their mathematics classrooms. This means I provided a detailed description of the contexts in which particular formative assessment practices were observed. It is important to note that everyone makes sense of ideas differently and that just as there is variation across the teachers, there is likely to be variation between myself and the 4 teachers. Any differences between the teachers’ interpretations of formative assessment and myself are addressed in the results section for the second research question. Chapter 2 is a literature review discussing previous research on how formative assessment has been implemented. Chapter 3 explains the methods I used to conduct this dissertation. Chapter 4 provides findings using the lesson structure framework I developed for mapping the formative assessment practices. Chapter 5 discusses the formative assessment practices that participating of teachers implemented in their mathematics classrooms. Chapter 6 provides contextual information to situate each teacher’s formative assessment practice use. Chapter 7 discusses one particular teacher’s high-quality formative assessment practices in greater depth than what is provided in the previous chapters. Chapter 8 explains the supports and obstacles teachers described in their ability to implement formative assessment in their classrooms. Chapter 9 is the discussion of the results and provides implications for the future. 5 Chapter 2: Literature Review This chapter summarizes the literature on teachers’ use of formative assessment, especially in mathematics and science classrooms. I begin by reviewing definitions of formative assessment presented in previous research. Then, I described how I used these publications to synthesize the definition of formative assessment that I used to guide this study. Next, I review research framed by two different conceptualizations of formative assessment, first as single practices and then as a single practice that integrates multiple elements. I close the chapter with research describing the supports and obstacles the influence teachers’ implementation of formative assessment. Definitions of Formative Assessment Formative assessment is not a well-defined term. Some definitions describe formative assessment specifically as a process, (Gotwals, Philhower, Cisterna, and Bennett, 2015), whereas others use it to refer to a specific tool or activity (Ross, 2006). These slight differences in the definition can make it difficult for researchers to communicate because they may not be talking about the same type of practice. Therefore, the first task of this chapter is to consider how different scholars have conceptualized formative assessment before stating the definition that has oriented this study. Paul Black and Dylan Wiliam are commonly referred to as the founding fathers of formative assessment, and it is their work that many other researchers have built on to further the conversation about the benefits of formative assessment. In their groundbreaking literature review on classroom formative assessment, Black and Wiliam (1998) defined formative assessment as, “all those activities undertaken by teachers, and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in 6 which they are engaged” (pp. 2-3). It would be another ten years before an influx of articles and books about formative assessment would be published. As the editor of the book, Formative Classroom Assessment Theory into Practice, McMillan (2007) defined formative classroom assessment “as a set of skills and activities that are undertaken by teachers to provide feedback to students to enhance their motivation and learning by designing instruction to meet student needs” (p. 1). This definition is similar to that of Black and Wiliam in that it talks about a set of activities that teachers use to provide feedback to their students. There is also a focus in both definitions on modifying instruction to support students. Ginsburg (2009) also saw a connection between formative assessment and improving instruction. He stated formative assessment “should be used to gain information that can help the teacher plan effective instruction, particularly for the individual” (p. 110). However, Ginsburg described two different kinds of formative assessment: organized and informal. In organized assessment, teachers use established tools to gather information, whereas informal assessment is more spontaneous, such as when the teacher asked students questions in the moment. Ginsburg’s (2009) definition is also different in that he discussed four different areas in which teachers should gather information about students that would be helpful in assessing them: performance, thinking/knowledge, learning potential, affect/motivation. Performance related to the students’ level of understanding of the content. Thinking/knowledge sought to understand the students’ processes, weaknesses, and misconceptions. Learning potential related to students’ readiness to learn the content. Affect/motivation referred to students’ interest and feelings about mathematics. Ginsburg (2009) claimed the most important aspect of formative assessment was “its actionable character: it is assessment that forms or informs instruction in a principled and effective manner” (p. 111). 7 Ginsburg saw specific methods for formative assessment; observation, task, and clinical interview. Observation “involves the attempt to gather information about naturally unfolding behavior without influencing it” (p. 112). Observation also included “the gathering and subsequent analysis of products, like the child’s written work in school or markings on the blackboard (p. 112, italics in original). Task “involves an adult-determined task that the child is asked to solve” (p. 112). This could be a task all the students complete or the teacher could give the students different tasks. These tasks might be similar to the activities Black and Wiliam (1998) and McMillan (2007) discussed in their definitions of formative assessment. The clinical interview started with a task chosen by the teacher. Then the teacher “observes the child’s response, behavior, affect, and anything else that might be relevant, and develops an interpretation of the process underlying the child’s behavior” (p. 113). The teacher then created new tasks to assess their interpretation and asked questions to gain information about the students’ thought processes. This specific method was meant to be student-centered and individualized to each child. Ginsburg saw great value in clinical interviews, stating, “interviews concerning issues of direct relevance for teaching should be at the heart of formative assessment” (p. 115). As was true in these three definitions of formative assessment, Good (2011) has recognized the importance of formative assessment informing instruction. He stated, “formative assessment is commonly understood to occur during instruction with the intent to identify relative strengths and weaknesses and guide instruction” (p. 1). However, he focused on the fact that formative assessment is a noun phrase, when it is often considered a process and therefore not a noun, making this phrase confusing and misleading about what formative assessment is. Good noted, “we need to be explicit that in a system with formative value, what goes on around 8 the time the student takes the assessment is as important as what goes on during the assessment” (p. 2). His definition of a formative system is also unique compared to the previous authors because he stated that there were three important components to a formative system: content, context, and strategies. The content of assessments should contain items that are appropriate for students, include item diversity, and align with instruction. He believed that students, not just teachers, should be able to use assessment information to determine students’ level of understanding. The context is the current instruction. Good (2011) stated, “assessment items that have formative value need to be tightly aligned to the identified learning goals broadly and to current instructional targets specifically” (p. 3). Good argued that four strategies were needed in a formative system: providing quality, descriptive feedback; using effective questioning techniques; assessing prior knowledge and misconceptions; and implementing student goal setting, self-regulation, and self-evaluation (p. 4). Though these were not the only strategies available, they are important for an effective system. He also claimed these strategies are often connected to each other in practice. It is possible activities could be created to use these strategies, therefore, this aspect of his definition is similar to Black and Wiliam and McMillan who spoke of activities as part of formative assessment. Good argued that “formative use of assessment information” is a more appropriate phrase to use to describe formative assessment as a process and not a noun. He stated, “adding information to the phrase makes explicit that what students and teachers learn about a student’s understanding goes beyond the response to a particular item” (p. 5). This new phrasing implies that students have a role in the formative assessment process, which is not explicit in the previous definitions, except Black and Wiliam who spoke of students using activities. In 9 addition, Good explicitly mentioned feedback as a required element of the formative assessment process as did Black and Wiliam and McMillan. In an NCTM publication, Cameron, Gawroski, Eich, and McCready (2011) stated, “Formative assessment is an ongoing process designed (a) to assess where a student is in the learning process and (b) to help a teacher use students’ responses to determine the instructional activities necessary to further the student’s learning” (p. 9). In a more recent practitioner-focused publication, Chappuis, Stiggins, Chappuis, and Arter (2012) defined formative assessment as, “formal and informal processes teachers and students use to gather evidence for the purpose of improving learning” (p. 4). Both definitions portray formative assessment as a process that helped teachers make changes to instruction to improve students’ learning but neither explicitly mentions providing or using feedback. The difference between these definitions is the role of students in the process. Cameron et al. (2011) mentioned students as a secondary figure in that teachers use students’ information to make decisions. The first part of their definition is unclear in who is doing the assessing of where students are, which could imply that students are an active participant in this aspect or this is another role of the teacher. Chappuis et al. (2012) included students with teachers as those who are gathering evidence making it clear that the students are an active participant in the formative assessment process. Like the previous definitions, Reinholz and Gillingham (2017) noted the role of modifying instruction in formative assessment, stating, “assessment must impact instruction to be considered formative” (p. 9). Their article was different because they developed a framework for explaining the different aspects of formative assessment related to eliciting student thinking in relation to when assessment occurred. This framework had three parts: reactive, active, and proactive assessments where “reactive assessments take place after instruction, active 10 assessment take place during instruction, and proactive assessments take place before instruction” (p. 9). The authors noted that when information is gathered from students impacts how that information can be used. Reactive assessments can be used to provide feedback or determine when a lesson needs to be re-taught. However, if teachers always relied on reactive assessments, then they would never use student thinking in the moment to guide instruction. An example of a reactive assessment the authors described was a reflective paper students wrote after a lesson. Active assessments provided instantaneous information but required teachers to respond immediately. The authors found this could be challenging for teachers to do in the moment. Examples of active assessments the authors discussed were the Mathematics Assessment Project (MAP) lesson plans and questioning (Good, 2011). Proactive assessment was the most versatile because teachers could use the information at any point during the lesson. The authors stated that proactive assessments provided the best opportunity for including formative assessment because teachers could use the information to plan effective future lessons. Meaningful use of proactive assessments could also benefit the other assessment forms as to “advance knowledge of student thinking can enhance the use of information when it is elicited, strengthening active assessments” (p. 11). An example of a proactive assessment the authors described was a lesson study experience by a group of teachers in Japan. Some of the assessments described by the authors could be activities the students complete to provide teachers with information, which aligns with the definitions stated by Black and Wiliam and McMillan. Reinholz and Gillingham also discussed how the different types of assessment could be used to gather evidence of students’ understanding and provide feedback to students. Analysis. These definitions have some important similarities and differences. First, all the publications focused on the need to improve or modify instruction to have formative 11 assessment. Second, five of the seven definitions included the explicit requirement of improving student learning. Third, four of the seven definitions mentioned providing feedback to students or using it to inform instruction. Fourth, Chappuis et al. (2012) explicitly stated that evidence should be gathered in order to make decisions, whereas the other definitions implied this was happening through the use of activities or other methods. Fifth, the first three definitions and the last one infers there is a process to using formative assessment because evidence was being gathered and teachers were making decisions accordingly. The last two definitions, though, clearly framed formative assessment as an integrated process and not a specific activity, meaning they believe formative assessment requires action. This usage also aligned with Good’s conception of formative assessment. Lastly, all the definitions mention teachers and students, however the role of the student is described differently in the different definitions. Some definitions described students as active participants in the formative assessment process, whereas other definitions had only the teacher as the main user of assessment information. A combination of these definitions was used to conceptualize formative assessment in this study. I view formative assessment as a process because the different practices can be implemented together in an integrated way within a lesson. Previous research discussed the role of teachers and students working together within the process of formative assessment, therefore it seems important to include both within my working definition. One key to formative assessment is gathering information about students’ thinking, which then allows teachers to make instructional decisions to support their students. Hence the following definition was adopted to embrace these important aspects. Formative assessment is a process that involves teachers acting directly or through students to gather information about students’ thinking to 12 inform instruction and support student learning. It typically involves multiple methods (or specific practices ) to achieve these goals. Formative Assessment Practices in Mathematics There is limited research about formative assessment practices specific to mathematics instruction. Of this research, an even smaller number of studies have been conducted in high schools. Most studies cited here were conducted in elementary and middle school classrooms, as well as in non-mathematics specific publications. They are still relevant, though, because the types of practices examined in these studies could also be implemented in high school mathematics classrooms. I am conceptualizing formative assessment as a process that involves teachers acting directly or through students to gather information about students’ thinking to inform instruction and support student learning. Embedded in this definition is the notion of feedback, gathering evidence and instructional decisions. Self-assessment, peer assessment, and questioning are methods of gathering evidence. Along with learning objectives, these aspects were discussed in the previous definitions as formative assessment practices. Gathering evidence is an exception. It is not a formative assessment practice, but the goal of multiple specific practices. Though I view formative assessment as a process typically involving multiple practices, research studies have typically focused on one specific practice. It is possible the specific practice is embedded in a larger formative assessment process, but most articles I have reviewed have focused on a particular practice. Therefore, this section is divided into sub- sections describing the characteristics of the formative assessment practices that have often been examined individually. This section also considers research focusing on formative assessment as an integrated process, that is, as involving multiple practices at the same time. 13 Defining clear learning objectives. Providing students with clear learning objectives or targets is the foundation for developing formative assessment practices (Chappuis, 2009). Stein and Smith (2011) also recognized the importance of learning goals, stating that goals are “a critical starting point for planning and teaching a lesson” (p. 13). Clear learning objectives are important for teachers and students. The NCTM publication, Principles to Action, stated, “clear goals not only guide teachers’ decision making during a lesson but also focuses students’ attention on monitoring their own progress toward the intended learning outcomes” (2014, p. 12). In her practitioner-focused publication, Chappuis (2009) argued that if teachers and students are unclear on the goals of the lesson, then it is unlikely teachers will be able to assess students effectively and accurately and unlikely students will be able to assess themselves or track their progress. Due to the importance of shared goals from the start, teachers and students need clear and similar expectations for the lesson; “the mathematical purpose of a lesson should not be a mystery to students” (NCTM, 2014, p. 13). Chappuis (2009) suggested three steps for providing students with learning objectives: 1) Sharing the learning target with students, 2) use language students understand, and 3) introduce students to the language and concepts of the rubrics you use (p. 22). Chappuis et al. (2012) asserted there were important benefits for teachers and students if these recommendations were followed. For teachers, clear objectives provide guidelines for what to teach, activities they should implement, and strategies for assessing students’ knowledge of the concepts. For students, such objectives provide a clear sense of what they were expected to learn; a vehicle for responding to feedback; and opportunities for self-assessment, goal-setting, tracking and reflecting on their progress. The teachers in a study conducted by Brookhart, Moss, and Long (2008) also found that sharing the learning goals with their students was an important 14 part of the formative assessment process. They argued that teachers should include connections to the learning goals in all aspects of the lesson, including discussions and assignments. Gathering evidence. There are often many opportunities during a class period to collect data about students’ understanding of the mathematical concepts they are working on (Chappuis et al., 2012). There are the tasks that students are completing, the conversations students are having with their peers and the teacher, and other interactions between students that may not be specific to that particular activity. When gathering data about students, Chappuis et al. (2012) stated that it was important for teachers to have a plan for how they will keep track of the extensive information they gather. They offered three suggestions to help organize the data so it could be useful to the teacher and student: 1) Organize data by learning target, 2) gather evidence that is content-specific and behavior-related separately, and 3) record raw scores, as opposed to percentages. For the first, since the teacher and student should have already established clear learning objectives, the data should be organized according to each objective to ensure that activities address the particular learning objectives, and teachers can easily monitor students’ progress over time. For the second, gathering evidence should not be related strictly to content. There are other aspects that need considered, such as work habits and social interactions. Including these shows that formative assessment “takes into account the progress of each individual, the effort put in and other aspects of learning which may be unspecified in the curriculum; in other words, it is not purely criterion-referenced” (Harlen & James, 1997, p. 372). Therefore, gathering multiple kinds of evidence could help students recognize the importance of these aspects to their learning. As with content-specific learning objectives, When the learning objectives involve work habits and social interactions, students should have clear ideas of how their work in these 15 areas will contribute to their overall performance and how they will be assessed (Chappuis et al., 2012). Keeping data about these aspects separate from achievement results is their second recommendation because “raising or lowering the grade stands as the only solution to a disparate range of learning problems” (Chappuis et al., 2012, p. 314). The authors’ third suggestion was to record data about students’ academic performance as a raw score, such as the number of points earned out of the total number of points available, whenever possible – as an alternative to recording scores as percentages. The authors argue that raw scores provide the most detailed information about a student’s performance on a particular activity or when looking at multiple activities related to the same learning objective because raw scores tell the students’ point total out of the possible points as opposed to a percentage, which treats all assignments as equal. Teachers can use this information when determining students’ progress towards a particular learning target. Questioning. Asking students questions is perhaps the best-known method of gathering evidence of how students are thinking about all content ideas, not just mathematical concepts. Hodgen and Wiliam (2006) stressed that, “by listening more to pupils, teachers learn more about what pupils know and how well they know it” (p. 15). But not all questions provide the same amount or kind of information, so it is important for teachers to vary the questions they ask based on the type of information they seek (Boaler & Brodie, 2004). Framing and posing high-level questions in the moment is no trivial task, though, because teachers must consider many factors, such as how the student’s response connects to the learning goal and the direction the teacher had planned for the discussion (Boaler & Brodie, 2004). High-level questions focus on students’ mathematical reasoning and not just the solution they found (Boaler & Brodie, 2004). These kinds of questions can also “guide students’ attention to previously unnoticed features of a 16 problem or they can loosen up their thinking so that they gain a new perspective on what is being asked” (Stein & Smith, 2011, p.62). Examples of high-level questions included, “It is still unclear how you figured out that 20 was the scale factor, so can you explain it another way?” or “How does that array relate to multiplication and division?” (Stein & Smith, 2011). Boaler and Brodie (2004) argue that the questions teachers ask “shap[e] the nature and flow of classroom discussions and the cognitive opportunities offered to students” (p. 780). Asking thoughtful questions also provided teachers with opportunities to think carefully about how they want to respond to students or what instructional decisions they will make based on what they hear (Hodgen and Wiliam, 2006). How teachers respond to students’ answers give students information about what the teacher was valuing, therefore it is important for teachers to listen carefully to students to make sense of their thinking (Hodgen and Wiliam, 2006). From their work with physics teaching, Minstrell and van Zee (2003) stated that questioning can be used as a tool for on-going assessment depending on the kinds of questions teachers ask students. Questioning could be used to “first diagnose the state of students’ thinking and then to prescribe an appropriate next step to guide students toward a deeper understanding” (p. 61). One type of questioning they discussed was elicitation questions. These “provide an opportunity for students to articulate the conceptions and reasoning with which they begin their study of a particular topic” (p. 62). The authors believed that these types of questions help students develop their thinking. In addition, the authors thought that in listening to students’ responses teachers gain valuable information that can help them make instructional decisions. It is also important for teachers to model for students how to respond to different types of questions. Exploring other possible answers, wondering about connections, and not knowing the answer right away are all appropriate ways to respond to questions (Chappuis et al., 2012). 17 Helping students see that they should respond directly to their peers instead of waiting for the teacher to respond is a productive way for developing a culture of questioning. Chappuis et al. (2012) believe that, “[students’] contributions can reveal a great deal about their levels of knowledge, conceptual understanding, and reasoning abilities” (p. 275). Having students respond directly to their peers’ questions helps teachers to determine students’ level of understanding. In addition, teachers could have students restate their peers’ questions or responses as another way to check students’ comprehension of the concepts. In a study of three elementary teachers, Franke, Webb, Chan, Ing, Freund, and Battey (2009) found that these teachers routinely asked students to explain their mathematical thinking. The authors found that by asking different follow-up questions, teachers were able to gather different levels of understanding by the students. Probing and specific questions enabled teachers to gather more information about students’ mathematical thinking; asking only probing questions supported students in reaching the correct answer after they had been incorrect. In a study of twenty elementary and secondary teachers from all content areas, Volante and Beckett (2011) found that teachers often used questioning to guide instruction and planning. Peer Assessment. Peer assessment is an activity that allows students to assess, evaluate, and provide feedback to their peers, effectively and meaningfully. With support of their teachers, peer assessment can provide information about students’ understanding that teachers can use to make instructional decisions. After students peer assess, teachers can review the results and adjust his or her plans accordingly. As with the previous practices, to implement peer assessment, students need to have clear learning objectives and be trained in how to assess their peers’ work in a productive manner (Black, Harrison, Lee, Marshall, & Wiliam, 2003; Topping, 2013). To assist students in the process of peer feedback, teachers can provide examples of 18 student work that has already been assessed and have conversations with students about the process and the type of feedback that was given or teachers can provide students with rubrics (Topping, 2013). Peer assessment can be a valuable practice for multiple reasons (Black et al., 2003; Topping, 2009). There are many more students in a classroom than teachers, therefore, more feedback can be provided in a short amount of time, which means students can receive specific and individualized feedback quickly (Topping, 2009). Due to the abundance of peer assessors, this may take some of the strain off the teacher because she can review the peer assessments more quickly than providing all the students her own feedback. Peer assessment is also beneficial for teachers because it leads them to think critically about the assessment process by clarifying lesson objectives, the intentions of the activities, and the assessment methods. They must be able to explain the process to students in a way that makes sense to them instead of just having the information for themselves (Topping, 2009). Peer assessment should be a reciprocal process, so that every student has the opportunity to have their work assessed, as well as to assess the work of one of their peers (Topping, 2013). Peer assessment should not just focus on the solutions students produce, but also on the process that students complete to solve a problem (Topping, 2013). It is important for students to recognize that the process is just as important, if not more important, as the product because, “Combining assessment of product and process can enhance student understanding of the consistency or mismatch between these and different ways of learning beyond their own” (Topping, 2013, p. 398). It should not be uncommon for students to experience a new strategy in the process of reviewing their peers’ work. When this happens, peer assessment becomes a learning experience in itself for students. 19 Black et al. (2003) found many benefits for peer assessment. First, it increases students’ motivation because they know their peer is going to be reviewing their work. Second, when students talk with each other, they use language that makes sense to them, which is often a different way from how the teacher talks with the students. This leads to students who “often accept, from one another, criticisms of their work that they would not take seriously if made by their teacher” (p. 50). Third, peer assessment empowers students and gives them a voice in their classrooms. The authors stated that it can also open an enriched dialogue between the teacher and students because the students are the ones who are assessing the work, therefore, this provides the teacher with additional feedback of what students know about the concepts. As with responding to their peers’ questions, how students give feedback to their peers provides the teacher with valuable information about students’ understanding. For example, if students provide feedback that is inaccurate, then the teacher will know that the student does not have a strong understanding of the mathematical concepts. Peer assessment can also be used to support students’ learning and self-assessment. Reinholz (2016) developed a framework to show the connection between peer assessment and self-assessment. The framework includes six activities: task engagement, peer analysis, feedback provision, feedback reception, peer conferencing, and revision. While he saw this as a cyclical process, he stated that the order was flexible and not all activities were required for the process to be effective. Task engagement requires the students to work through the task they will be providing feedback on, so they have an opportunity to experience the task on their own first. Reflective questions could be incorporated into these activities to help students self-assess. Peer analysis requires students to make decisions about the quality of their peer’s work to provide feedback. Reinholz thought this was an important activity to help students self-assess because 20 “without the opportunity to see these flaws in her peers’ work, she may never have developed the appropriate lens for identifying them in her own work” (p. 306). Feedback provision requires the students to explain their feedback to their peers. This can be done through written or verbal feedback; however, verbal feedback is preferred because it gives students practice explaining their thinking in a way that makes sense to others and that feedback is immediate. This activity promotes self-assessment because students can determine quickly if their explanations are understandable, which allows them to make quick adjustments. Feedback reception allows students to see how another person views their work. This feedback helps students see their areas for growth. Like others, Reinholz agrees that students need advice on what types of feedback are productive to give to their peers. Peer conferencing provides students opportunities to talk with their peer about their feedback. Revision allows students to revise their work after receiving feedback before they submit their work to the teacher. This revision step “may influence the feedback [students] give and how they perceive feedback they receive” (Reinholz, 2016, p. 307). Students can focus their feedback on areas where their peers need to improve, which in turn can motivate them to make needed changes before they submit their final product. Revision also supports self-assessment because it allows students to take feedback and improve their work. This peer assessment framework also provides students with opportunities to improve their collaboration and communication skills because they must interact with their peers and explain their thinking and feedback (Reinholz, 2016). Self-Assessment. This is an important aspect of formative assessment, because “it is only the learner who can do the learning” (Hodgen & Wiliam, 2006, p. 21). Volante and Beckett (2011) report that teachers in their study believed that “involving students in the assessment process is vital to student learning” (p. 246). Self-assessment, however, requires careful 21 preparation for the students. Self-assessing or self-reflection are not natural processes for students, so teachers need to provide them with the tools, such as rubrics or examples of student work that has already been assessed, to make these effective and meaningful practices (Cameron et al., 2011). Clear learning objectives and an idea of where the lessons are going, helps students be more prepared to assess their own performance (Chappuis et al., 2012). These criteria are necessary because “unless [students] come to understand their strengths and weaknesses, and how they might deal with them, they will not make progress” (Harlen & James, 1997, p. 372). This emphasizes the importance of students understanding themselves as learners. Ross (2006) claimed that four important components are necessary for students successfully engage in effective self-assessment. First, students must be involved in the process of defining the criteria and language they will use to self-assess. Second, students must be taught how to self-assess using the specified criteria. Third, they need feedback from teachers on the self-assessment process to ensure students are approaching and implementing the process accurately and meaningfully. Fourth, students need guidance in learning how to use their self- assessment to improve their learning. Ross (2006) also suggested that if teachers help students focus on goals related to learning the content and not on how they perform in comparison with their peers, self-assessment becomes more meaningful. Ross also stressed that students need to feel safe in rating themselves and reporting low scores. Teachers can help students by creating a welcoming classroom culture that shows students that the process of self-assessment can be productive when they provide honest interpretations of their work. Fontana and Fernandes (1994) taught Portuguese elementary teachers self-assessment strategies they could implement with their students in all content areas, but the article focused on mathematics. A control group of teachers who were not taught the strategies was used to assess 22 the impact of the strategies. These strategies included the following elements: the details of the learning objectives to be reached and criteria for assessing these objectives, opportunities to choose learning tasks to meet these objectives, and the scope for monitoring and assessing outcomes (p. 408). The students taught by the teachers who implemented the self-assessment strategies demonstrated higher achievement and showed more growth from the pre- to post-test than those taught by the teachers in the control group. Ross, Hogaboam-Gray, and Rolheiser (2002) described multiple studies where elementary and one middle school teachers were successful in training their students to develop self-assessment skills. The authors conducted their own study with fifth and sixth grade students and found that students who received training related to self-assessment demonstrated positive growth on their achievement in mathematics. This training consisted of four steps: 1) involving students in defining evaluation criteria, 2) teaching them how to apply those criteria, 3) giving them feedback on their self-evaluations, and 4) helping them use evaluation data to develop action plans (p. 11). The authors reported that for this type of training to be successful it had to be long-term and students needed support from teachers throughout the process in order to achieve these results. Although their study lasted twelve weeks, the authors did not explicitly indicate what direction of time would be sufficient. Brookhart, Andolina, Zuza, and Furman (2004) conducted a similar study with third- graders. In conjunction with completing their weekly timed tests on multiplication facts, the students predicted their scores and then graphed their predictions on a bar graph. Once they received their scores, they graphed the actual results next to their predictions. They used this information to then reflect on their performance and the accuracy of their prediction. The researchers found that over time, the students became more accurate with their predictions. The 23 students reported positive reactions to this self-assessment experience because they liked being able to notice the progress of their scores according to their graphs. Making instructional decisions. After teachers have gathered evidence from their students, they can use this information to make instructional decisions. These decisions could include in-the-moment changes to the lesson, determine about how to group students, or adjustments to the next day’s lesson plan. To make these kinds of decisions, teachers need knowledge of how children learn mathematics. Morris (2006) studied elementary and secondary pre-service teachers’ interpretations of students’ thinking. She used videotaped mathematics lessons and asked the 30 volunteers to make claims about how the lesson affected students’ learning. She found that pre-service teachers could make reasonable suggestions for improving a videotaped mathematics lesson, however, they were not always able to provide evidence to support those claims. It was challenging for the pre-service teachers to offer reasons because they were still developing their ability to pay attention to multiple interactions at one time (Morris, 2006). The ability to make instructional decisions is an aspect of knowledge of content and teaching (Ball, Thames, & Phelps, 2008). For the participants in Morris’ study, the changes were made after the lesson had been completed, but this does not always have to be the case. These modifications are often made in the moment and are a direct reaction to questions, ideas, or comments made by students (Ball et al., 2008). The ability to make instructional decisions was also examined by Kohler, Henning, and Usma-Wilches (2008), who studied 150 elementary and secondary pre-service teachers from multiple content areas via a performance assessment called a teacher work sample (TWS). This assessment included seven different aspects of high-level teaching, including instructional decision-making. For this aspect, the pre-service teachers reflected on two instances where they 24 noticed a student struggling and made a modification to the lesson based on that student’s need. They had to identify what the student said or did that made them think an instructional decision was warranted, and then describe and justify their modification. The most common instructional decisions arose from listening to students talking or observing their behavior. Most concerned the work of the whole class, with only a small percentage addressing a small group or individual student. The most common modifications were related to instructional activities, such as explaining the concepts more thoroughly, asking different questions, or making alterations to students’ activities, such as changing the directions or grouping structure. Although it is helpful for students when teachers explain why they are making instructional decisions, such explanations are infrequent (Kohler et al., 2008). In that study, pre-service teachers gave students a rationale less than half of the time they made instructional decisions. Nathan and Koedinger (2000) asked from than 100 teachers, including elementary, middle, and high school teachers, to rank story problems, equations, and word equations in order of expected difficulty for algebra students. The researchers found that high school teachers believed equations would be the easiest for these students, but students reported the opposite. This suggests that teachers’ inability to predict where their students are struggling has serious implications for classrooms and their ability to make instructional decisions (Nathan & Koedinger, 2000). If teachers are incorrect about their assumptions, then it is not likely that their lessons will meet the needs of their students. Gearhart and Saxe (2004) engaged elementary teachers in a professional development program to increase their pedagogical knowledge in mathematics. The professional development focused on the Integrating Mathematics Assessment program, which emphasizes formative assessment. The program combines three different activities: Teachers’ Mathematics, Children’s 25 Mathematics, and Implementation. During the Teachers’ Mathematics activities, teachers solved problems to increase their mathematics knowledge. Then, they discussed the problems they had solved, made comparisons between solutions, and talked about key mathematical ideas. After the discussion, teachers reflected on the activity and how they could apply it to their own teaching practice. For the Children’s Mathematics activities, teachers watched videos of children solving problems and examined student work. These activities built on the activities teachers completed during the Teachers’ Mathematics activities. The teachers discussed how they made sense of the students’ mathematical thinking. During some of these conversations, teachers agreed on the students’ level of understanding: in others they interpreted the students’ thinking in different ways. As part of this activity, “teachers brainstormed assessment strategies that could provide more information” (p. 307). During the Implementation phase, the focus was on incorporating assessments that would continually gather information about students’ thinking. Some assessments were suggested by the teacher leaders; at other times, teachers shared their own ideas. They had discussions about different assessment methods, such as observations, whole class discussions, and peer assessment. The authors found that as teachers’ knowledge increased, their classroom practices became stronger. They stated, “[teachers] need to investigate children’s thinking, and build activities and discussions upon children’s understandings” (p. 310). This suggests that teachers need to use evidence of their students’ understanding to help them make instructional decisions. The authors also found that as teachers strengthened their knowledge, they saw more value in assessment to make sense of their students’ thinking. Heritage, Kim, Vendlinski, and Herman (2009) studied more than 100 sixth-grade teachers’ completion of mathematical tasks. To assess their pedagogical knowledge, researchers had the teachers respond to questions about instructional decisions they would make or feedback 26 they would give students based on assessment information. The authors found that teachers were weakest in making decisions about how to use information from students to plan upcoming lessons. The researchers stated that, “Action is dependent on teachers’ knowledge of how learning develops in the domain and on their pedagogical content knowledge” (p. 31). This implies that strong teacher knowledge of student learning is needed to make appropriate instructional decisions. Schneider and Gowan (2013) assessed a group of elementary teachers’ ability in four areas related to pedagogical knowledge: determining what an item measures, analyzing student work, providing targeted feedback, and determining next instructional steps (p. 194). Teachers were divided into one of three groups: control, treatment one (those who analyzed student work with a rubric), and treatment two (those who analyzed student work with a rubric after watching a professional development video “on providing feedback to students that encourages students to review, reflect, and revise.” (p. 195)). The researchers found that providing targeted feedback was the most challenging aspect of the process, with determining next steps the next most challenging. They noticed that teachers mostly gave feedback correcting the students’ errors and not feedback supported students’ revision of their work to improve their learning. The authors found statistically significant correlations between: analyzing student work and feedback, analyzing student work and determining next steps, and feedback and determining next steps. The authors found statistically significant correlations between analyzing student work and feedback, analyzing student work and determining next steps, and feedback and determining next steps. Providing feedback. Meaningful and effective feedback is an important aspect of formative assessment because feedback provides students with information about their progress 27 towards their current learning objectives (Chappuis, 2009; Hodgen & Wiliam, 2006). Chappuis (2009) argued that good feedback practices have five key characteristics: 1) it directs attention to the intended learning, pointing out strengths and offering specific information to guide improvement, 2) it occurs during learning, while there is still time to act on it, 3) it addresses partial understanding, 4) it does not do the thinking for the student, and 5) it limits corrective information to the amount of advice the student can act on (p.57). Nicol and Macfarlane-Dick (2006) agreed that feedback should help to clarify what good performance is, deliver high quality information to students about their learning, and provide opportunities to close the gap between current and desired performance. Those authors also thought that teachers’ feedback to students could be used to help shape teaching (p. 205). For feedback to be helpful to students, it should provide them with information on where they are in their learning and give them a plan for how to improve their understanding. The timing of feedback is important as well. It should occur at a time that allows students to make changes and further develop their understanding. It is also important for students to have an opportunity to process the feedback they received. Hodgen and Wiliam (2006) stated that teachers need to provide students time to make sense of the feedback, ask questions to clarify feedback, and determine how to address the issues. As Wiggins (1999) so eloquently stated, “we do not know what students really know until we examine their answers with them and see how they respond to our responses” (p. 197). This implies that it is important for students to have conversations with others about the feedback they have received after they have had time to make sense of the feedback on their own. While marking students’ work to determine accuracy could be considered evaluative feedback, it does not provide students with as much information as teachers’ comments on their work and reasoning. Research has shown that students’ performance improved when teachers 28 provided formative feedback, instead of only marking their work for accuracy (Butler, 1988). It was helpful for students when their teachers provided information about what they were doing correctly, what they needed to work on, and how to take the next steps (Hodgen & Wiliam, 2006). Simply providing students with a score did not give the student any information about how to improve their learning (Hodgen & Wiliam, 2006). Volante and Beckett (2011) also found that the teachers in their study thought it was important to provide feedback that was not evaluative. Only marking students’ responses as right or wrong does not support their learning because it does not give them direction for how to improve. In addition, feedback should focus on comments or questions that push students’ thinking forward. Dweck (2003) found that different types of feedback also influenced students’ motivation. If teachers focused on the students’ ability or intelligence (i.e., You’re so smart!), that decreased students’ motivation because they believed that their intelligence was fixed and nothing could change that. However, if teachers focused on students’ effort or strategies, then motivation was increased. This suggests that teachers should focus their feedback on students’ effort to promote positive mathematical beliefs. Shih and Alexander (2000) studied Taiwanese fourth grade students to determine the effect of feedback on their self-efficacy. They found that students who had received self- referenced feedback (i.e., feedback specific to their performance) performed better on the fraction activities than students who had received social-referenced feedback (i.e., feedback in comparison to other children of comparable ability). The students in the self-referenced feedback group also had more positive self-efficacy, were more mastery focused, and set higher goals. Rakoczy, Harks, Klieme, Blum, Hochweber (2013) compared the effects of different types of feedback (process-oriented and social-comparative) on German ninth graders’ learning. They 29 found that process-oriented feedback promoted a positive sense of competence and increased students’ interest in mathematics, but it did not improve achievement. Process-oriented feedback was more helpful to students, though, when compared to social-comparative feedback. These findings support what other authors have said about feedback—that it should focus on the individual student’s thought processes and learning and provide each with opportunities to reflect. Taken together, these studies suggest that feedback is a valuable and critical aspect of the formative assessment process. Feedback was a useful tool for teachers to use to communicate to students where their work stood in relation to the learning goals and what their next steps should be to improve and be more successful. But the character of the feedback that teachers provide matters, because not all feedback is productive in developing students’ mathematical understanding. Marking students’ answers simply as incorrect does not support their mathematical development. As the reviewed studies have shown, it is important to give students descriptive and actionable feedback with an appropriate amount of information for them to make sense of and also give them time to digest it. Students need this type of feedback to adjust their learning before they are summatively assessed. Integrated Formative Assessment Practices in Mathematics In contrast to the predominant focus in research on particular formative assessment practices, some researchers have seen and studied formative assessment as a sequence of practices. Peterson and Siadet (2009) conducted a study to determine the effect of formative assessment practices on college students’ achievement in an introductory algebra class. They found that students whose instructors implemented formative assessment practices, such as frequent quizzes and timely constructive feedback scored higher on a standardized test than those 30 students whose instructors had not implemented these practices. The students in the formative assessment group also had higher passing rates for the course and greater retention rates. Suurtamm, Koch, and Arden (2010) conducted a longitudinal study of Canadian mathematics teachers’ classroom practices. They found that teachers successfully used different formative assessment practices, such as feedback, questioning, and student self-assessment, to interpret middle and high school students’ understanding of the mathematics content. These teachers made assessment an integral part of their classrooms, making it impossible to distinguish between instruction and assessment activities. This allowed teachers to gather information about students’ mathematical thinking on a continual basis. The teachers demonstrated these practices in three important ways, “by providing students with feedback on their mathematical thinking, by developing students’ ability to self-assess and act on assessment information, and by providing teachers with information to guide their classroom practice” (p. 413). With colleagues (Gotwals, Philhower, Cisterna, and Bennett, 2015), I conducted a study with thirteen mathematics and science teachers to determine which formative assessment practices were more visible in classrooms and at what level of expertise they implemented them. The practices were learning targets, questioning, feedback, self-assessment, peer assessment, and instructional decisions. The eight mathematics teachers in the study demonstrated more formative assessment practices and higher level of practice than the five science teachers. We speculated that mathematics teachers implemented formative assessment in a manner that was more observable than the science teachers, such as incorporating more whole class discussions that included multiple questioning strategies and teachers offering feedback to students. The mathematics teachers also made their instructional decisions more explicit to their students and 31 observers by stating reasons for the actions they were taking. We also noticed that with the implementation of multiple formative assessment practices and achieving higher levels of practice, formative assessment became more of a process and less about individual practices being implemented on separate occasions. Andersson and Palm (2017) discussed a professional development program designed for Year 4 mathematics teachers that conceptualized formative assessment as a “unity of integrated strategies”. They shared five key strategies: clarifying learning intentions and criteria for success, engineering effective classroom discussions and other learning tasks that elicit evidence of student understanding, providing feedback that moves learners forward, activating students as instructional resources for one another, and activating students as owners of their own learning (p. 95). Through this framework, teachers and students worked together to support learning. The authors found that the students of the teachers who had participated in the professional development improved their mathematics achievement more so than the students of the teachers who had not participated. Supports and Obstacles When Implementing Formative Assessment Though teachers may be interested in implementing formative assessment practices in their mathematics classrooms, there are often internal and external factors that affect their ability to enact them. Internal factors occur within the teachers’ control, such as their beliefs or philosophies about teaching and learning. External factors occur outside the teachers’ control, such as curriculum requirements or school expectations. This section reviews the factors that support teachers’ implementation formative assessment and the obstacles that hinder them in doing so in their classrooms. 32 Factors That Support Formative Assessment Practices. Teachers’ knowledge, communities of practice, and professional development workshops support teachers in implementing formative assessment in their classrooms (Hill, Rowan, & Ball, 2005; Hodgen & Marshall, 2005; Suurtamm & Koch, 2014; Suurtamm, Koch, & Arden, 2010). Hill, Rowan, and Ball (2005) defined mathematical knowledge for teaching (MKT) as, “the mathematical knowledge used to carry out the work of teaching mathematics” (italics in original, p. 373). Examples include the ability to explain mathematical concepts to students, interpret students’ thinking, use multiple representations, and provide students with effective examples of mathematical concepts (Hill et al., 2005). This means that teachers must know how and why different mathematical concepts work, how they relate to each other, and how to introduce them to students in sensible ways. Teachers must also be prepared to interpret students’ multiple strategies that are often incorrect and possibly unknown to the teacher at a moment’s notice, so they can respond to students quickly and meaningfully (Hill et al., 2005). This is important then because “teachers’ ability to interpret and make judicious strategic use of assessment information from many sources is a critical factor in their instructional effectiveness” (National Research Council, 2001, p. 350). Teachers need strong mathematical knowledge for teaching because, “teachers’ MKT is strongly related to the mathematical quality of their instruction” (Hill & Ball, 2009, p. 70). Hodgen and Marshall (2005) also found that strong pedagogical content knowledge (a closely related construct to MKT) supported the mathematics teacher in their study in making sense of students’ solutions, recognizing connections between different correct and incorrect strategies, and using these ideas in following lessons. They stated that, “the teacher’s profound knowledge provided an effective structure for her in turn to develop an effective scaffold for the 33 pupils’ learning” (p. 169). This implies that this type of knowledge is important for teachers to implement formative assessment practices successfully. Suurtamm and Koch (2014) found that developing communities of practice providing a regular time for teachers to meet and talk about assessment practices was integral in the teachers’ development. In these communities, teachers “learned new ways of assessing students and challenged their own thinking about what assessment means and could look like” (p. 283). Teachers also had the time and opportunity to try out new ideas in their classrooms and then report back to the group on the experience. Another benefit of these communities was that they were teacher-focused and teacher-led, which empowered teachers and positioned them as experts in their classrooms. The researchers did not evaluate the teachers’ responses or assessment work, choosing instead to ask guiding questions and encouraging everyone to participate in the discussions. This allowed the teachers to speak freely and not worry about being judged on how they were implementing the assessment practices. Suurtamm, Koch, and Arden (2010) found that when teachers received similar messages from curricula and professional development opportunities, they felt encouraged to implement formative assessment practices in their classrooms. When teachers used curricula materials that encouraged multiple forms of assessment and making sense of students’ thinking, and attended professional development promoting these same views, then teachers were better prepared to incorporate these types of practices into their classrooms. Factors That Hinder Formative Assessment Practices. Pressure from high-stakes testing, curricular demands, teachers’ lack of knowledge, expectations of parents and students, and school context or expectations can effectively hinder teachers in implementing formative 34 assessment (Box, Skoog, & Dabbs, 2015; McMillan, 2003; Suurtamm & Koch, 2014; Volante, 2010). High-stakes testing can restrict teachers’ ability to implement assessment practices they desire (Box, Skoog, & Dabbs, 2015; McMillan, 2003). Though they may not agree with the methods of high-stakes testing, “there was a sense of resignation as teachers decided to change the nature of their classroom assessments” (McMillan, 2003, p. 37) to align with the nature and content of high-stakes testing practices. Box, Skoog, and Dabbs’s (2015) reported similar results during their study, finding that “[a science teacher] knew there were other ways to assess and would rather teach and assess for understanding than rote memorization” (p. 972). But she still let high-stakes tests guide her decisions. Since the teachers perceived the tests as requiring low- level knowledge, they tended to limit their use of formative assessment, focusing instead on summative assessment. In connection to the pressure from high-stakes testing, the demands for curricular “coverage” can hinder teachers’ ability to implement formative assessment practices (Box, Skoog, & Dabbs, 2015; Volante, 2010; Webb & Jones, 2009). Box et al. (2015) found that one of the greatest pressures on teachers was this feeling that they must “cover” everything in the curriculum so students would be prepared for high-stakes testing. This often forestalled delving deeper into the curriculum or implementing formative assessment practices. Webb and Jones (2009) also found that teachers felt this pressure to push through the curriculum even when they wanted to spend more time discussing the concepts. Weak teacher knowledge can also restrict teachers’ implementation of formative assessment. Suurtamm and Koch (2014) identified two types of obstacles (the authors called these “dilemmas”) that related to teachers’ knowledge. The first was conceptual dilemmas. These 35 were defined as situations that, “arise as teachers attempt to understand the conceptual underpinnings of inquiry-based mathematics teaching and learning and of current views of assessment” (p. 272). The researchers saw these dilemmas arise when teachers talked about how one problem could elicit many different types of student responses. The second type was pedagogical dilemmas, where were defined as situations that, “arise as teachers deal with the “how” of enacting current assessment ideas” (p. 274). The researchers saw these occur when teachers talked about finding time to assess students and determining how they would record students’ responses or their own observations. Box, Skoog, and Dabbs (2015) also found that gaps in teachers’ knowledge impacted how they responded to students. They found that teachers’ lack of knowledge for how students learned influenced their decisions and how they responded to students. This often led to teachers’ lecturing to students instead of leading discussions for students to discover ideas on their own. Volante (2010) found that parents’ and students’ assessment expectations made it challenging for them to accept some of the formative assessment practices, such as peer and self- assessment. Parents and students were used to seeing numbers and grades; therefore, practices that do not result in these types of results were difficult to accept. Volante believed teachers will continue to face this conflict “if students and parents are skewed toward a completely work- based function of schooling” (p. 74). He thought teachers must work to convince students and parents to accept formative assessment practices as meaningful ways to measure learning to overcome this obstacle. Suurtamm and Koch (2014) also found that others’ expectations of assessment hindered teachers’ practices. They called these cultural dilemmas, that were defined as situations that, “focus on changes in classroom and school culture with regard to assessment practice” (p. 275). The researchers saw cultural dilemmas arise when teachers talked about 36 changing the role of students in the assessment process or dealing with parents’ or administrators’ views of appropriate assessment. Suurtamm and Koch (2014) also found that school context or expectations could also hinder teachers’ ability to implement formative assessment. They called these political dilemmas – situations that, “arose as teachers’ thinking and practice were juxtaposed with provincial, district, and school assessment policies” (p. 277). Political dilemmas occurred when teachers talked about how they were required to report students’ grades, including the types of comments they could make on report cards. This chapter summarized prior research on formative assessment. The central task of defining formative assessment is challenging due to the differing perspectives of formative assessment as a process versus specific individual practices. This chapter provided research on both stances to support a working definition for my research study. Most studies focused on individual practices including, defining clear learning objectives, gathering evidence, questioning, peer assessment, self-assessment, making instructional decisions, and providing feedback. Although only four publications supported formative assessment as a process or a series of integrated practices, this is an important perspective to consider. Both stances on formative assessment influenced my study, but my analysis focused primarily on individual practices. The chapter concluded by identifying the factors that either support (teachers’ knowledge, communities of practice, and professional development workshops) or hinder (pressure from high-stakes testing, curricular demands, teachers’ lack of knowledge, expectations of parents and students, and school context or expectations) teachers’ ability to implement formative assessment practices in their classrooms. 37 Chapter 3 Methods The purpose of the current study was to determine what formative assessment practices a sample of high school mathematics teachers used and what those practices looked like when implemented in their classrooms. This study concentrated particularly on high school teachers because they were underrepresented in the research literature. Formative assessment has many meanings for both researchers and practitioners. We know that practices like formative assessment can be implemented in many ways, varying based on teachers, content, and school contexts. Here, I explored the range of practices observed by a sample of high school mathematics teachers who all claimed to be enacting formative assessment practices. Overview Six high school mathematics teachers from four different schools in Mid-Michigan participated in the study. The goal was to have teachers from different school and with different years of experience to have a diverse group of teachers. I looked for teachers who were focused on implementing formative assessment practices in their classrooms, either by their own choice or based on a school mandate. Because the focus was on the use of formative assessment practices, teachers were recruited because they stated that they were currently using formative assessment in their classrooms. Although the study primarily focused on what these practices looked like in action, it was important to first consider which specific practices teachers were using. Setting and Participants In recruiting teachers, I contacted three leaders in the mathematics education community to help me find teachers who were implementing formative assessment practices. These individuals sent out emails to teachers they had worked with or sent me the names and emails of 38 district leaders who might be able to help me recruit teachers. I also sent out emails to teachers I had worked with in the past to see if they would be interested in participating in my study. Some of these teachers also forwarded my information to their colleagues who they thought would be interested in participating in my study. The emails sent out to potential participants included a brief description of what their participation would entail and noted that I was specifically looking for high school mathematics teachers who were implementing formative assessment practices. The goal was to recruit ten teachers, however, only seven teachers agreed to participate. One teacher had to drop out before data collection started for personal reasons, therefore, only six teachers participated in the study. Because I stated in my initial contact with the teachers that I was looking for participants who were using formative assessment in their classrooms, there was no reason to doubt that this group of teachers were currently using formative assessment. Table 1 shows the demographic information for the six teachers who agreed to participate in my study. Also included in the table are the two mathematics content areas for which I observed each of the teachers with their focus class identified. The findings discussed in this dissertation concentrate on the focus class. As the table shows, the school settings represent diverse student populations and there was a range in participants’ teaching experience. Table 1 Teachers’ Demographic Information Name School School Information Lindsay Scott Alexander High School Alexander High School Rural, 900 students, 70% free/reduced lunch Lucas Newson High School Rural, 1000 students, 74% free/reduced lunch 39 Subjects Observed Years of Teaching Experience 16 8 Algebra 2 (Focus) Geometry Geometry (Focus) Algebra 2 Geometry Pre-Cal/Coordinate Geometry (Focus) 15 Table 1 Evan Gwen (cont’d) Sanderson High School Sanderson High School Thompson High School Charlie Data Collection Urban, 500 students, 88% free/reduced lunch Suburban, 1800 students, 52% free/reduced lunch 6 2 Algebra 1 (Focus) Algebra 2A Trigonometry Algebra 2 (Focus) Algebra 2H (Focus) Algebra 1 10 This section describes the methods I used for data collection, including four different types of interviews and classroom observations. Background interview. I conducted a background interview (Appendix A) with participants to gather information about their background, how they defined formative assessment, how they were using practices often associated with formative assessment, any professional development opportunities they had had in connection to formative assessment, and what they saw as supports and obstacles in implementing formative assessment practices in their classrooms. Interviews were audio and video recorded and then transcribed. The overall objective of the background interview was to gather information about how teachers viewed the role of the different formative assessment practices in their classroom. How teachers talked about the supports and obstacles they faced could also explain how they chose whether to implement particular practices, as well as the ways in which they implemented those practices. At the end of the background interview, the teachers chose two classes of different content for me to observe. They also chose which class would be their focus class and which days they wanted me to observe for the first round. Observations were video recorded and the teachers wore an audio recorder. The observations lasted approximately three consecutive days at two different times between March and June 2017, approximately one month apart. The main 40 purpose of these observations was to gather descriptive examples of formative assessment practices in action in the different school and subject area contexts. Lesson planning interview. Before observing the teachers, I met with each teacher for a lesson planning interview (Appendix B). This occurred only for the class they had chosen as their focus class. This interview was in relation to the three lessons in that round that I would be observing. A lesson planning interview occurred before each of the observation rounds. The goal of these lesson planning interviews was two-fold. First, it was to determine how the teacher planned for incorporating formative assessment practices into their lessons. Second, prior research stresses the importance of teachers’ knowledge and skills in shaping their implementation of classroom practice; therefore, this interview helped gain information about the teachers’ knowledge related to the mathematical content and how they would respond to the needs of their students. Debriefing interview. At the end of each lesson observation of the focus class, the teacher and I had a short conversation to debrief the lesson I observed (Appendix C). The teacher was asked to share his/her interpretation of the lesson and reflect on how the lesson progressed in connection to what they planned. The teacher also shared how s/he planned to use the data s/he collected during the lesson to make decisions about the next class session. These conversations were video-recorded to allow teachers to show materials as needed while debriefing the lesson. I transcribed these conversations. My purpose for these interviews was to gather information about how the teacher viewed the lesson and determine how the teacher made instructional decisions that may not have been obvious by just watching the lesson. There were multiple data sources collected for this study. First, formative assessment is not well-defined in the literature, so the initial interviews provided valuable information about 41 how the teachers understood formative assessment. The background interviews provided information about teachers’ prior experiences related to assessment, such as attending professional development focused on formative assessment. The interview questions focused on specific formative assessment practices, which were based in the formative assessment literature, so that the practices teachers described were aligned to the literature. Analysis of Classroom Observations and Interviews To determine which formative assessment practices teachers implemented and at what level of expertise they were implemented, I used a modified version of the Formative Assessment for Michigan Educators (FAME) rubric (Gotwals et al., under revision; Gotwals et al., 2015; see Appendix D), which the FAME project used for coding classroom videos in all content and grade levels. This rubric aligned with the formative assessment literature and included the following top-level categories: Learning targets, Eliciting student understanding, Feedback, and Instructional decisions. Each of these four dimensions included one formative assessment practice (the one that matches its name), except Eliciting student understanding and Feedback. Eliciting student understanding included: Type of question, Teacher questioning strategies, and Self-assessment. Feedback included Feedback in the moment and Peer assessment. Each practice was classified on a continuum demonstrating the level of skill, with 1 being a low-level or novice practice and 4 considered high-level or expert practice. I made changes to the original FAME rubric to better align with the focus on teachers’ practices; these changes included deleting some sub-categories, the location of peer assessment, and the name of one of the sub-categories. Instead of focusing on two different aspects of teacher questioning, I focused just on the type of question teachers asked because including the additional category did not add any additional information to the description of teachers’ 42 questioning strategies. Peer assessment was originally listed under the Feedback dimension, but I moved it to Eliciting student understanding because it could be a way for teachers to gather information about students’ thinking. I also changed the name of sub-category of feedback in the moment under the Feedback dimension to make it clearer to what that practice was assessing, since verbal feedback was the only kind of feedback observed in the moment. Within each of these dimensions are formative assessment practices that can be implemented to support that dimension. For the Learning targets, Feedback, and Instructional decisions dimensions, the practices listed are the same as the dimensions (i.e., Learning targets is learning target use). For the Eliciting student understanding dimension, the practices included questioning, self-assessment and reflection, and peer assessment. The dimensions and formative assessment practices are displayed below in an indented format to visually represent which practices are part of each dimension. Further explanation of the practices is provided after the diagram. I. Defining Learning Targets II. Eliciting Student Understanding a. Questioning b. Promoting self-assessment and reflection c. Promoting peer assessment III. Providing Feedback IV. Making Instructional Decisions Currently the only one practice listed beneath the Defining learning targets dimension is use of learning targets, which considers how teachers shared the learning target with the students during the lesson. For example, does the teacher give the learning target a passing mention or 43 does s/he make an explicit connection for the students between the learning target and the lesson activities? If the learning targets were just posted somewhere in the classroom, then that was not coded as using learning targets because the teacher did not explicitly mention the learning targets to the students as part of the lesson. The second dimension, Eliciting student understanding, contains the following three formative assessment practices: questioning, promoting self-assessment and reflection, and promoting peer assessment. Questioning is focused on the type of question and cognitive demand of the teacher’s questions (Webb, 2002). Cognitive demand is important to consider because the level of question a teacher asks provides different kinds of information about students’ thinking. For example, a closed yes/no question would provide very different information than an open-ended question that asked students to explain their thinking. I documented all the teachers’ questions and then classified questions as low or high level. The other two practices that are part of Eliciting Student Understanding are promoting self-assessment and reflection and peer assessment. Although these are practices completed by students, the teacher must provide students with opportunities and the necessary tools to accomplish them. Due to my focus on what teachers do to implement formative assessment, I focused on how teachers provided students with opportunities for self- and peer assessment, and not on what students were doing or saying during these activities. Students near the camera could be heard during these activities, but I recognize that this is not enough information to make claims about the students’ role in these formative assessment practices, therefore, the focus was strictly on the teacher’s role. For promoting self-assessment and reflection, teachers could just tell students their grade, which does not provide the student with much support or structure in how to self-assess or reflect on their progress. Or teachers might provide students with structured 44 opportunities and support in reviewing their own work and adjusting their thinking. Teachers could use the information from students’ self-assessments and reflections to determine groups for activities or modify instruction if students are struggling. For promoting peer assessment, students may grade each other’s papers, which does not provide either student with much support or structure in developing their understanding. Teachers might also provide structured opportunities and support for students to do peer assessment using a rubric or other specific criteria that helps students make connections to the learning goals. The teacher may then use the information from students’ peer assessments to modify instruction and support students’ learning. The third dimension, Providing feedback, only contains the formative assessment practice of providing verbal feedback in the moment. The rubric considers the type of feedback teachers gave students, such as evaluative, descriptive, or actionable and whether the feedback focused on the student or the learning process. The focus was on verbal feedback since that occurred during the classroom observations. The fourth dimension, Making Instructional Decisions, only contains the formative assessment practice of making adjustments to teaching and/or learning. The rubric considers whether the teacher made his/her instructional decisions clear to the students. This practice can be hard to observe because these decisions are often made in a teacher’s head. The debriefing interviews helped make these decisions visible because I asked the teachers why they responded in particular ways during the lesson. Since I met with the teacher for the planning interview, I knew how the teacher expected to implement the lesson, so once I observed the lesson, I asked the teacher about any discrepancies between the planned and enacted lessons. I analyzed the classroom observation videos to identify how often teachers were implementing these particular 45 practices during the days of observation. This provided numerical information about which practices teachers were implementing and at what level of practice. The FAME rubric distinguished four levels of implementation for each practice, and I applied these levels to my data as FAME defined them. In general, all subsequent codes included and surpasses the expectations of the prior levels. FAME’s definitions of each level for five of the six practices are given below. For Learning Targets at Level 1, the teacher made a passing mention of the learning target but did not actually read it to students. For Level 2, the teacher read the learning target aloud to the students. For Level 3, the teacher reminded students of the learning target and then made an explicit connection between the learning target and that day’s lesson activities. To achieve a Level 4, the teacher reminded students of the learning target, made an explicit connection to the day’s activities, and informed students of the types of knowledge, skills, and abilities they must master to meet the learning target. For the coding the Self-Assessment practice, a Level 1 just required students to track their own grades. A Level 2 also required teachers to encourage students to look for their own mistakes and make corrections. For a Level 3, the teacher provided students with a structure to reflect on their learning and provided them opportunities to practice self-assessment, as well as track their grades and search for mistakes. To achieve a Level 4, teachers provided students with structures and opportunities, as well as provided prompts to students to think deeper about their learning. For the Peer Assessment practice, a Level 1 just had students grade each other’s work without providing correct solutions. Level 2 required students to share the correct solution with their peer. For a Level 3, the teacher provided students with a structure for providing feedback to their peer regarding their work. To achieve a Level 4, the teacher provided students with a 46 structure and opportunities to practice peer assessment, as well as connecting it to the learning goals. When coding the Feedback practice, a Level 1 included the teacher only giving evaluative feedback, such as telling the student if they were correct or not. For Level 2, teachers’ feedback was mainly evaluative and focused on the solution. For Level 3, the feedback is descriptive and focused not only on the solution, but also students’ solution process. This type of feedback may also provide students with some guidance on how to move forward in their learning but may do some of the thinking for them. To achieve a Level 4, teachers offered descriptive feedback focused on the process and the task and provided students with enough guidance to move students’ learning forward and for students to determine their own next steps. For the Instructional Decisions practice, a Level 1 was when the teacher provided no rationale for their decision-making process. For a Level 2, the teacher was vague about their rationale. For a Level 3, the teacher provided a vague rationale for their decision, making some connection to students’ learning in their decision-making process. To achieve a Level 4, the teacher was explicit in their rationale and made direct connections to students’ understanding and how that led to their decision-making process. However, the levels for the Questioning practice require more explanation. Questioning practices coded as Level 1 only included low-level questions, which had one correct response, such as “what’s the square root of 4?” or was a yes or no question. The rest of the levels included a mix of low and high-level questions to varying degrees. Level 2 questioning practices included mostly low-level questions, but the teacher did ask at least one high-level question. For example, this level of questioning included teachers asking students “does it make sense?” to check their understanding. Level 3 questioning practices included many high-level questions, often 47 scaffolding from low-level to high-level questions and included about 50% high-level questions. This level of questioning also pushed students to explain their thinking. However, judgments were made in coding for this level in that some instances had less than 50% high-level questions but were coded as Level 3 because of the particular high-level questions that were asked. There were also activity segments that had 50% high-level questions that were not coded as Level 3 because students were not pushed to explain their thinking. Level 4 questioning practice included more high-level than low-level questions, as well as questions that pushed students to do more than just explain their solutions. For example, asking students to make compare and contrast judgments (“What is different about this problem compared to the last one?”) or connections between representations (“Do you see any connections between your equations and your graphs?”). However, if the teacher only asked five or fewer questions and the majority of them were high-level, then judgements were made to code these as Level 3 and not code Level 4, because the sample size of questions was so small. Lesson Observations. During the observations, I positioned myself in the back of the classroom (except in one classroom, where I was stationed in the front) with the video camera on a tripod. The goal was to situate myself out of the way, so I would not be a distraction to the students. I did not walk around the classroom with or without the video camera. The video camera stayed focused on the teacher as s/he moved around the classroom, except when the teacher was talking to another teacher or when s/he was working on her/his computer. I took some field notes during the lesson. These focused mostly on questions I wanted to ask the teacher during the debriefing interview. Other times they were notes I made about what I witnessed, such as when the teachers were observed by administrators during my observations or 48 information about activities that occurred during class. The teachers wore an audio recorder on a strap around their neck to document conversations that were not picked up by the video camera. Lesson Structure Framework. Before looking at which formative assessment practices teachers used in their classrooms, I developed a lesson structure framework (See Appendix E) in order to segment the classroom videos into separate activity structures. This framework was necessary for a few reasons. First, it provided reasonable time segments to break down the lessons. Second, it created a structure for discussing in which activity structure the different formative assessment practices occurred during the lessons. Through an iterative, open-coding process, I found that nine codes could represent all the activity structures implemented in the teachers’ classrooms. Warm-up was an activity some of the teachers used to start the class to get students thinking about the mathematical content of that day’s lesson. Introduction to class and lesson was another way that teachers started their classes by sharing a verbal or written description of the plan for that day’s class. This introduction often included the learning targets for that day’s lesson. Interactive lecture was a teacher-led activity teachers used to introduction new content to the students that included low-level participation by students. Individual activity was when students were instructed to work alone on an in-class activity. Pair/small group work was when students were instructed to work with a partner or small group of students on an in- class activity. Presenting student work was when students or the teacher shared students’ work with the whole class using a document camera or whiteboard. These presentations were often followed by a discussion of the students’ work that was led by the student or teacher. Whole class discussion was when teachers and students participated in a discussion of the work students had completed, such as in-class activities or homework. Teacher giving information was when the teacher read the correct answers to an assessment or other piece of student work to the 49 students without discussing the solutions or solution process. Summary occurred at the end of the lessons and was when teachers provided final thoughts or takeaways to that day’s lesson. I then calculated the amount of time each teacher spent implementing each of these activity structures. Then, I used these activity structures and times to development visual representations for each teacher’s lessons. I used this framework as the foundation for determining when teachers were using the different formative assessment practices, by mapping these practices onto the lesson structure bars. Addressing research question 1. Research question 1 asked: “What formative assessment practices are some high school mathematics teachers implementing in their lessons to support students’ learning of the mathematical content?” The video-recorded classroom observations provided the bulk of the evidence of the teachers’ formative assessment practices. Even though I took field notes about the activities I saw in the classrooms, they were not detailed enough to provide information to support the video-recordings. First, I divided the lesson into segments using the activity structures described above. This was done by watching the videos and documenting when the activity structure changed. If the teacher told the students to work together, then I coded the activity as Pair/Small group activity, even if not all the students worked with their group. The grouping structure designations were based on what the teacher told the students to do. I created a spreadsheet for each teacher’s lessons to document the order of the activity structures and the start and ending times for the segments. I used this spreadsheet to determine the total amount of time for each activity structure for each lesson. Then, using the formative assessment coding framework described previously, I determined which formative assessment practices teachers used and at what level of expertise these practices occurred during each of these activity structures. To do this, I watched each lesson and coded for learning target 50 use, self-assessment, peer assessment, and instruction decisions. This round of coding was straight-forward, except coding instructional decisions was challenging at times because teachers had to make their decisions verbally and it had to be clear to me that a decision had been made to be coded. Next, I watched each lesson again and documented each question the teacher asked and coded that formative assessment practice. Coding this practice was challenging at times because some teachers asked questions and then answered them right away without giving students time to answer, so those questions were not counted. Teachers had to ask a question and give students a chance to answer before the questioning code was used. During this pass through the videos, I also coded for feedback. This practice was also challenging at times because sometimes teachers told students what to do, instead of providing them with suggestions for how they could solve the problem themselves. If the teacher did the work for the student, then that was not coded as feedback because the student did not have any thinking to do on their own. Lastly, I mapped the formative assessment practices onto the lesson structure bars to create a visual representation of each teacher’s lessons showing where they implemented these formative assessment practices. I used a second coder to analyze the data for this research question to check reliability (see IRR section at the end of this chapter). Addressing research question 2. Research question 2 asked: “How do a sample of high school mathematics teachers make sense of and implement formative assessment practices, in the context of their views of their subject, their students and school, and their broader practice?” The question of “how” formative assessment was implemented was examined in multiple ways. First, as described in addressing research question one, I examined where in the teachers’ lessons the formative assessment practices occurred. I completed the lesson activity structure analysis and then placed the observed practices within that structure. Then I also identified at what level of 51 sophistication the teachers’ implemented these formative assessment practices to determine where high-quality practices occurred. Thirdly, I investigated the relationship between particular activity structures and formative assessment practices to determine how the activity structures influenced which formative assessment practices are used. All the interviews and the classroom observations helped answer this question. The background interviews provided information about teachers’ school contexts, discussions of how they implemented activities related to the formative assessment practices discussed above, and what they viewed as supports or obstacles in implementing formative assessment. I asked the teachers questions about the subjects they were teaching, what curriculum they were using, and what professional development opportunities they had experienced. I summarized the information teachers provided to write a description about each teacher to provide context for their classroom practices. To unpack their views of these different formative assessment practices, I asked questions such as how they determined their lesson goals, whether they provided students with feedback, and how they made decisions about adjusting their instruction. I asked these questions in a way that did not imply teachers should be implementing all these practices. I summarized the information teachers shared and used the responses to these questions to determine how the teachers viewed each of the formative assessment practices. Then I examined all of their responses to determine similarities and differences between teachers’ beliefs and implementations. The planning interviews provided context for the particular lessons I observed. Teachers talked through how they planned the lesson and then I asked questions, such as how often the teacher had taught the lesson and if they had made any changes to the lesson based on prior implementations. This information spoke to the teacher’s experience with this particular content 52 and how they had made decisions about adjusting their instructional plan in the past. I also asked questions connecting back to the formative assessment practices, such as how they determined the lesson goals, what questions they will ask students, and how they will close the lesson. I summarized this information as needed to support teachers’ descriptions of how they used the different formative assessment practices. The debriefing interviews also provided context for the lessons that I observed. I have data via three different perspectives of the same lesson (i.e., planning-before, observations- during, and debriefing-after). The debriefing interviews included information about how teachers reflected on the lessons, how they made sense of the information they gathered from students, and how they determined what should happen the next day based on the information they gathered from students. I specifically asked teachers how they determined whether students had met the lesson goals or not and how the next day’s lesson would be influenced by what happened in the current lesson. This descriptive data provided another dimension to the classroom observation data because teachers’ decision-making is often not visible to others. Asking teachers to talk about how they planned for the next lesson based on the current lesson was helpful in determining how teachers used information from their students to guide their instruction, which is an important factor in formative assessment. I summarized this information as needed to support teachers’ descriptions of how they used the different formative assessment practices. The classroom observation videos provided information about what these formative assessment practices looked like when teachers enacted them in their classrooms. I used this information I observed in these lesson videos to write detailed descriptions of how these teachers implemented the formative assessment practices in different ways. Although the classroom 53 videos provide information about how I interpreted the teachers’ practices, it was possible that the teachers could have interpreted formative assessment differently than me. During the exit interviews, I asked the teachers if they thought different activities, activity structures, and ideas they implemented in their classrooms was related to formative assessment. I used this information to determine how the teachers’ views of formative assessment was different from my own. I also compared the responses from the teachers to determine any similarities and differences in their views. For example, many teachers had their students working in groups and some teachers saw this as promoting formative assessment and others did not. I report my results for this research question first by teachers, so that I can provide a context for each of their classrooms. I share information about their school context, background, teaching assignment/curriculum, definition of formative assessment, and professional development opportunities. The second way I report my results is by the formative assessment practices, so that I can provide descriptions of how the teachers talked about how they have implemented them in their classroom. Then, I provide comparisons among the teachers for how they have implemented these practices in similar and different ways in the context of teachers’ understandings, experiences, and school contexts. After presenting which formative assessment practices each teacher implemented and the level of their implementation, the focus of my dissertation shifted to examine some examples of high-quality practice. Given my focus on what formative assessment in secondary mathematics classrooms looks like in practice, it made sense to investigate more deeply examples of high- quality formative assessment practice, as these may serve as models of how high school mathematics teachers have successfully implemented formative assessment. 54 After coding all the teachers’ focus class lessons, I looked for teachers who consistently had lessons including multiple formative assessment practices at Level 3 or Level 4. Then I determined which lessons included these high-quality practices. Charlie was the only teacher whose practice fit this description. I chose three consecutive lessons to focus on because between these three lessons Charlie reached Level 4 for four of the six formative assessment practices and Level 3 for the fifth practice (as previously noted, Charlie did not implement peer assessment in his observed lessons). For each lesson, almost all the other formative assessment practices are Level 3. These three lessons showed that it was possible to consistently implement many formative assessment practices at a high-level during a mathematics lesson. Although Charlie used the same activity structures in all the lessons, he implemented them in different ways. During the lesson on June 1, students were learning new content, whereas during the lesson on June 3, students were reflecting on what they had learned in preparation for their unit test. The June 3 lesson was the only observed lesson where Charlie focused the whole class time on students’ reflecting and reviewing for an assessment. I introduced Charlie’s three lessons when I undertook a closer examination of his formative assessment practices when it was particularly high-quality in Chapter 7. When focusing on Charlie’s three lessons, I analyzed the relationship between the activity structures and the formative assessment practices implemented during those activity structures. I also delved deeper into these three lessons to determine whether the particular lesson activities played a role in what formative assessment practices were implemented. Did the activities or tasks Charlie implemented in his classroom promote opportunities for formative assessment practices to be more visible? In addition, I considered how Charlie’s views of 55 formative assessment were enacted in how he implemented these practices in his classroom as well as how he talked to students about their learning. Addressing research question 3. Research question 3 asked: “What do some high school mathematics teachers see as supports and obstacles in implementing formative assessment practices in their classrooms?” The background interviews provided information about what teachers saw as supports and obstacles in their ability to implement these formative assessment practices. Even though the teachers were in different school contexts and teaching different subjects, they had some similar experiences related to supports and obstacles, therefore, the results report the experiences of all the teachers to determine in what ways these similarities do exist across contexts. I coded the supports and obstacles using an iterative, open-coding process. The factors teachers saw as helping and hindering their ability to implement formative assessment were identified and then grouped into categories. Eight codes could identify all the supports and obstacles stated by teachers (See Appendix F). Class norms were factors that were part of the teacher’s regular classroom practices. Content were factors related to the mathematics content. Logistics were factors related to making decisions about in-class activities, assessments, and pacing. Resources for teachers were resources available to the teacher for their use that included people and materials. Resources for students were resources available to the student for their use. School were factors related to the school structure. Students were factors related to students’ interactions, experiences, and participation in the classroom setting. Time were factors related to the issue of time. 56 Inter-Rater Reliability A colleague served as an additional rater to check the reliability of my lesson structure and formative assessment practices coding. This was necessary to determine whether another person would see the same formative assessment practices and at the same levels of expertise in the same lessons as I did. This provided evidence that these formative assessment practices were visible in the lessons to someone who was not in the classrooms during the observations and did not have the background knowledge of the teachers or their classrooms that I did. I worked with this person on a previous research project related to assessment where we coded interview transcripts and had to check our coding process to come to an agreement on the appropriate codes. She also has focused on formative assessment while working on a curriculum project for the last few years. I conducted two short training workshops with this person to introduce her to the lesson structure framework and formative assessment coding framework. I had her code a 15-minute clip of one of the videos, then we discussed the codes we both assigned to the activities. Then she coded the rest of the video on her own after reviewing the lesson structure and formative assessment practices frameworks. We met again to discuss the codes we both assigned to the rest of the video. We held another round of training where she coded an entire classroom video on her own, and then we met to discuss the codes. Once we had successfully reconciled our codes and she felt comfortable with the frameworks, I chose six videos for her to code on her own. This represented 19% of the rest of the videos. I selected this set of videos for several reasons. First, each teacher was represented within the set. Second, the set contained lessons that included a variety of activity structures for her to have experience with as many different activity structures as possible. Third, the set contained a 57 variety of formative assessment practices, in that each practice that I coded was represented in this set. Fourth, the videos contained a range of activity structures, meaning that some videos had a small number of activity structures and others contained a large number of activity transitions. The same is true for the formative assessment practices, such that some lessons had very few instances of formative assessment compared to others which contained many instances. Finally, the set also contained three videos from the first round of the data collection and three videos from the second round of data collection because it is possible that teachers’ practices could be different during the stages of data collection since some observations were conducted mid- semester, whereas others were conducted at the end of the school year. I assessed inter-rater reliability at three different levels. First, I considered whether we saw the same formative assessment practices during each individual lesson. This round did not account for where in the lesson the particular practices appeared. Table 2 shows the results of this comparison. Table 2 IRR Formative Assessment Practices Teacher Charlie Learning Targets Questioning Self- Assessment Peer Assessment Feedback Instructional Decisions Coder 1 Coder 2 Percent 3 4 2 0 2 1 3 100.00% 4 100.00% 2 100.00% 0 100.00% 66.67% 3 1 100.00% 58 Table 2 (cont’d) Learning Targets Questioning Self- Assessment Peer Assessment Feedback Instructional Decisions Learning Targets Questioning Self- Assessment Peer Assessment Feedback Instructional Decisions Learning Targets Questioning Self- Assessment Peer Assessment Feedback Instructional Decisions Learning Targets Questioning Self- Assessment Evan Gwen Lindsay Lukas 1 100.00% 3 100.00% 1 100.00% 0 100.00% 2 100.00% 3 66.67% 0 100.00% 6 100.00% 0 100.00% 0 100.00% 4 66.67% 0 100.00% 0 100.00% 4 100.00% 0 100.00% 0 100.00% 2 66.67% 1 100.00% 1 3 1 0 2 2 0 6 0 0 6 0 0 4 0 0 3 1 1 11 0 1 100.00% 11 100.00% 0 100.00% 59 Table 2 (cont’d) Peer Assessment Feedback Instructional Decisions Learning Targets Questioning Self- Assessment Peer Assessment Feedback Instructional Decisions Scott 0 11 0 100.00% 90.91% 10 1 0 4 0 0 3 0 1 100.00% 0 100.00% 4 100.00% 0 100.00% 0 100.00% 66.67% 2 0 100.00% Table 2 shows a high-level of reliability between the two coders. For each teacher, there was only one formative assessment practice that was not the same. Interestingly, for five of the six teachers, this formative assessment practice was feedback, with instructional decisions being the formative assessment practice for the sixth teacher. Feedback can be a challenging practice to code because teachers’ feedback can come in different forms, such as questions, explanations, or comments, which can be hard for the coder to interpret. It is important to note, though, that the differences in coding never exceeded one for any practice or teacher. The second round of inter-rater reliability I considered was whether we saw the same practices during the same lesson segments. The second coder worked from my parsing of the lessons into segments, however, she was asked to verify that she agreed with the activity structures for each lesson segment. The agreement for the two coders on the lesson segments was 100%. Table 3 shows the comparisons. 60 Table 3 IRR Formative Assessment Practices by Lesson Segment Coder 1 Coder 2 Introduction LT Individual Act LT, Q, SA LT, Q, SA, ID Whole Class Pair/Sm Group Q, F Whole Class Q, F Intro None none Warm-up LT, Q, SA LT LT, Q, SA, F LT, Q, SA Q, F Q, F, ID none ID LT, Q, SA none none QT QT, F QT, F F QT, F QT none QT, F QT, F Intro Whole Class Q, F, ID Q, F, ID Individual Whole Class Q, F, ID Q, F, ID Whole Class QT Present Student Lecture Individual Whole Class QT, F Lecture QT, F Pair/Sm Group Pair/Sm Group Whole Class QT, F QT, F, Pair/Sm Group ID Present Student Summary Intro Giving Info Warm-up QT, F none LT F none QT, F none LT, F F none QT QT QT, F ID QT, F QT, F QT 61 Teacher Charlie Evan Gwen Lindsay Lukas Practices Same 1 3 3 2 2 0 0 3 3 0 3 1 2 1 0 2 2 2 1 1 3 2 0 1 1 6 Practices Different % Agree 100 0 1 1 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0 1 0 0 75 75 100 67 100 0 100 100 100 100 100 100 50 0 100 100 100 100 50 100 100 100 50 100 100 Table 3 (cont’d) QT, F QT QT QT, F QT, F none QT, F QT, F QT, F, ID QT, F QT, F QT, F QT none none none QT 2 1 1 2 1 6 2 2 3 2 2 2 1 0 0 6 1 0 1 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 100 50 100 100 50 100 100 100 100 100 100 100 100 0 0 100 100 QT, F none QT, F 2 6 1 0 0 1 100 100 50 QT, F QT none QT, F QT, F QT Present Student Lecture Pair/Group Present Student Lecture Pair/Group Present Student Pair/Group Present Student Pair/Group Present Student Individual Pair/Sm Group Introduction Pair/Sm F Group Individual none Whole Class QT Pair/Sm QT, F Group none Individual Whole Class QT QT, F QT, F QT, F, ID QT, F QT, F QT, F QT F Scott Table 3 also shows a relatively high-level of agreement. Similar to the previous round of IRR, feedback and instructional decisions were the only formative assessment practices that were different between the two coders. For Charlie, Lukas, Scott, the two coders characterized practices in three activity structures differently. For Gwen, they did so in two. For Evan and Lindsay, the practices were coded differently in only one activity structure. These differences 62 occurred in many activity structures; there was not one particular activity structure for which the coders did not agree on the formative assessment practices. The third round of IRR considered at what level the teachers implemented the different formative assessment practices within the particular lesson segments. The levels represent the level of expertise for the implementation of the formative assessment practices. They ranged from Level 1 (novice) to Level 4 (expert). Table 4 shows the comparisons. Table 4 Formative Assessment Practices by Lesson Segment and Level of Practice Teacher Coder 1 Coder 2 Levels Same Levels Different % Agree Introduction LT1 Individual Act LT3, Q3, SA4 LT4, Q2, SA3, ID3 Q2, F3 Charlie Whole Class Pair/Sm Group Intro Whole Class Individual Whole Class Q2, F2 Intro Warm-up none none LT3, Q2, SA3 Q3, F1, ID2 none Q2, F1, ID2 Whole Class Whole Class QT2 Present Student Lecture Individual QT2, F1 QT2, F1 F1 Evan Gwen 1 3 3 2 1 6 0 3 2 6 2 1 2 1 0 LT1 LT3, Q3, SA4, F2 LT4, Q2, SA3 Q2, F3 Q2, F3, ID2 none ID1 LT3, Q2, SA3 Q2, F1, ID2 none Q1, F1, ID2 QT2 QT2, F1 QT2 none 63 0 1 1 0 2 0 1 0 1 0 1 0 0 1 1 100 75 75 100 33 100 0 100 67 100 67 100 100 50 0 QT3, F2 none LT2, F1 F1 none (cont’d) Whole Class QT2, F1 Lecture QT2, F1 Pair/Group QT2, F2 Pair/Sm Group Whole Class QT2, F1 QT3, F2, Pair/Sm Group ID2 Present Student Summary Intro Giving Info Warm-up Present Student Lecture Pair/Group Present Student Lecture Pair/Group Present Student Pair/Group Present Student Pair/Group Present Student Individual Pair/Sm Group Introduction Pair/Sm F1 Group Individual none Whole Class QT3 QT2, F2 QT2, F2 QT2, F2, ID3 QT2, F2 QT2, F2 QT2, F1 QT1 QT2, F1 QT1 none QT3 QT2, F1 QT2, F1 QT3 F1 Table 4 Lindsay Lukas Scott 2 2 2 1 0 2 2 6 1 1 6 2 1 1 2 1 6 2 2 3 2 2 2 1 0 0 6 1 QT2, F1 QT2, F1 QT2, F2 QT3 QT3 QT3, F3 ID2 QT3, F2 none LT2 F1 none QT2, F2 QT2 QT1 QT2, F1 QT1, F1 none QT2, F2 QT2, F2 QT2, F2, ID3 QT2, F2 QT2, F1 QT2, F1 QT3 none none none QT3 64 0 0 0 0 2 1 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 100 100 100 100 0 67 100 100 50 100 100 100 50 100 100 50 100 100 100 100 100 100 100 100 0 0 100 100 Table 4 (cont’d) Pair/Sm QT2, F1 Group Individual none Whole Class QT2 QT2, F2 none QT2, F1 1 6 1 1 0 1 50 100 50 Table 4 again shows a relatively high-level of agreement. Aside from the differences in presence of feedback and instructional decisions that was considered in the previous round, there were also differences in how the coders assessed the level of questioning. In the three instances where the questioning levels did not match, the coders differed only by one level. It is possible this is due to how many questions each coder considered to be high-level, since this was a factor in determining the questioning level. The most common differences were in level of feedback since both coders did not always have feedback present in an activity structure. For most of these cases, one coder did not see feedback and the other coder had a Level 1 feedback, which was strictly evaluative feedback. This difference is possibly due to the coders’ interpretation of the teacher’s speech. For example, a teacher could say “okay” after a student’s response and then continue talking. This could be seen as a transitional word by the teacher (not coded as feedback) or as low-level feedback to the student. It is also possible that these evaluative words (good, yes, etc.) could have been missed by one of the coders because they occurred quickly. It is also worth noting that the differences occurred for both coders, therefore, one coder was not the one to always have observed a formative assessment practice that the other coder did not see. Across the three levels of IRR, the agreement was relatively high. Feedback, questioning, and instructional decisions were the only formative assessment practices where the coders differed. In all cases, the differences in the coding was one or two apart. When considering the final round of IRR, which assessed the level of formative assessment practice within the lesson segment, the differences were only one level apart, therefore, the coders did not interpret the 65 lessons in drastically different ways. These three formative assessment practices were likely the most challenging to achieve agreement because they could be interpreted in different ways. Feedback depends on the teachers’ tone and speech patterns since comments could be interpreted as responding to students or transitioning to the next topic. Questioning depends on how the coders assigned low and high-level status to the questions. Instructional decisions required the teachers to be explicit about their decision-making process, but this could be interpreted by observers in different ways based on their knowledge of the teacher. 66 Chapter 4: Teachers’ Lesson Structure as Context for Formative Assessment This chapter presents a descriptive analysis of the activities the teachers used to structured their lessons. This presentation is necessary because in the rest of the results chapters, the teachers’ formative assessment practices will be mapped onto their lesson structure. The lesson structure framework, therefore, provides context for understanding teachers’ formative assessment practice, specifically in providing opportunities for each activity structure to implement different formative assessment practices. Without first providing some lesson-level context for teachers’ formative assessment practice, it seemed difficult to present and characterize that practice in a meaningful and accessible way. The lesson structure distinguishes nine separate activity structures, with an additional category for Logistics, which includes the teacher taking attendance or talking with students about topic unrelated to the lesson. Figure 1 presents a color-coded key to the ten activity structures. Figure 1 Activity Structures By way of introduction, I first briefly describe each teacher’s activity structures in the lessons I observed before presenting them in greater detail and then discussing the key similarities and differences across the six teachers. Scott used the least number of activity structures, only 3 or 4 activity structures within each lesson. His lessons generally followed the pattern of Introduction, Pair/small group work, and Whole class discussion. Lukas demonstrated 67 the most diversity, using six or seven activity structures within each of his lessons. The only exception was a review day where he only used three activity structures. Lukas started every lesson with an Introduction and then most often transitioned to a Warm-up activity, followed by the Presentation of student work. Evan’s and Gwen’s activity structures varied across their lessons. Evan used between two and six activity structures within a class period and Gwen incorporated between two and five activity structures. Evan usually used fewer activity structures on the days he only had his students for 45 minutes, except the one day where he incorporated two activity structures, using almost the entire 90-minute class period to have students working together on a project. He started almost every class with the Individual activity of a homework quiz. Introduction activities often followed this. Interestingly, Gwen used more activity structures on the days she had her students for 45 minutes; she shifted more frequently between structures when she had less time with her students. She also did not seem to have a pattern to her use of the activity structures. She started almost every class period differently, except for using Introduction activities for two of the five observed lessons. What was consistent across almost all her lessons was the use of Interactive lecture. Charlie’s Lesson Structure I observed Charlie’s Algebra 2 Honors class for six lessons across the two rounds of data collection almost exactly one month apart. His class met every day for 60 minutes. Charlie used a variety of activity structures, implementing every activity structure, except providing the students with information or ending with a summary, which is shown in Figure 2. 68 Figure 2 Charlie’s Activity Structures by Lesson Charlie started almost every class with an introduction to the class or lesson (yellow), with one exception of starting the May 9th class with a warm-up. During the first round of data collection, Charlie used interactive lecture (red) once a day for each of the three days and had students present their work or shared student work (purple) on the first two days of observation. The rest of the observations follow a similar pattern with incorporating individual work time (orange), pair or small group work time (green), and whole class discussions (blue). Table 5 shows the total amount of time (rounded to the nearest half of minute) Charlie spent on each of these activity structures. Table 5 Charlie’s Total Time Per Activity Structure Day 1: May 6 0 Day 2: May 9 4.5 Day 3: May 10 0 Day 1: June 1 0 Day 2: June 2 0 Day 3: June 3 0 Total 4.5 4 4 0 36 3 0 4 23.5 0 15 13 12.5 2 5 28 0 0 26 3 0 12 21 0 25 69 3 0 3 29 0 26 1 0 11 26 0 13 13 77.5 112 18 22 124.5 Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work Whole Class Discussion 0 0 60 0 0 59.5 0 0 61 0 0 61 0 0 61 0 0 60 0 0 362.5 Table 5 (cont’d) Teacher giving information Summary Total As shown in Table 5, Charlie spent most of his class time orchestrating whole class discussions, followed closely by having students work in pairs or small groups, and then providing students time to work individually. Each of these three activity structures totaled more than an hour of his class time, with the whole class discussions totaling more than two hours. The rest of the activity structures each totaled less than twenty minutes and two of the activity structures did not occur during the six days of observation. Evan’s Lesson Structure I observed Evan’s Algebra 1 class for six lessons across the two rounds of data collection almost exactly one month apart. His school follows a modified block schedule, so Evan saw all his classes on Mondays for 45 minutes each and then saw each class twice more on alternating days for 90 minutes each. Evan used multiple activity structures, which is shown in Figure 3. Figure 3 Evan’s Activity Structures by Lesson Evan started almost every class with an individual activity (orange), which consisted of a homework quiz. This was often followed by an introduction to the class or lesson for the day (yellow), however this activity structure was not limited to the beginning of the class, since Evan transitioned to different topics by introducing them to students to help with the shift in 70 mathematical ideas. For more than half of the lessons, Evan provided students with time to work with a partner or small group (green) and incorporated a whole class discussion (blue). Only two of the six lessons included Evan lecturing (red) to the students. It is interesting to note that Evan only incorporated pair or small group on the days where their class was 90 minutes long. Table 6 shows the total amount of time (rounded to the nearest half of minute) Evan spent on each of these activity structures. Table 6 Evan’s Total Time Per Activity Structure Day 1: April 25 2.5 Day 2: April 26 0 Day 3: April 28 0 Day 1: May 23 0 Day 2: May 24 0 Day 3: May 26 Total 0 2.5 8 0 4 0 0 25 12 0 18.5 18.5 0 40 5 32 22 24 0 7 3 13.5 27.5 0 0 0 4.5 0 37 29 0 17 0 0 32 54 0 0 32.5 45.5 141 125.5 0 89 0 1 91 0 0 44 0 0 87.5 0 0 86 0 1 437 Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work Whole Class Discussion Teacher giving information Summary Total 0 0 39.5 0 0 89 As shown in Table 6, Evan spent most of his time providing students with time to work individually, working with a partner or small group, or orchestrating whole class discussions. These were the same activity structures that Charlie used most often in his classrooms, however, the two teachers spent very different amounts of time implementing them. Evan spent almost twice as much of his time having students work individually compared to Charlie. In addition, 71 Charlie spent almost 36 minutes more having whole class discussions than Evan did. The two teachers spent about the same amount of time having their students work in pairs or small groups (e.g., there was only about a 14-minute difference). Each of these activity structures totaled between 1.5 hours and 2.5 hours. The rest of the activity structures each totaled less than an hour, with two activity structures (presenting student work and teacher giving information) not being implemented during these observed lessons. Gwen’s Lesson Structure I observed Gwen’s Algebra 2 class for five lessons across the two rounds of data collection almost exactly one month apart. Gwen was only observed for two days during round one of data collection due to scheduling conflicts. She is at the same school as Evan, so she also follows a modified block schedule, so Gwen saw all her classes on Mondays for 45 minutes each and then saw each class twice more on alternating days for 90 minutes each. Gwen’s activity structure is shown in Figure 4. Figure 4 Gwen’s Activity Structures by Lesson For four of the five classes, Gwen spent time at the beginning of class taking attendance or having conversation with students that did not relate to the lesson, including significant time spent on the last two classes where Gwen gave students time to talk amongst themselves while she redesigned the lesson in the moment due to impending student absences the next day or talking about the final exam schedule. Gwen spent most of her time incorporating interactive 72 lecture (red) or individual activity (orange), which were each utilized four out of five lessons. Only two of the five days included a whole class discussion (blue) or partner/small group work (green). The extended individual activity on April 27th was a unit test that the students took on computers. Table 7 shows the total amount of time (rounded to the nearest half of minute) Gwen spent on each of these activity structures. Table 7 Gwen’s Total Time Per Activity Structure Day 1: April 25 3 Day 2: April 27 0 Day 1: May 23 0 Day 2: May 25 0 Day 3: May 27 Total 0 0 22 6.5 0 0 9.5 1 33 50 0 0 0 0 17.5 0.5 10.5 5 7 5 34 38 0 0 0 3 6 106.5 95 0 0 0 76 86.5 0 0 5 16.5 Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work Whole Class Discussion Teacher giving information Summary Total 0 0 41 0 0 84 0 0 40.5 0 0 77 0 0.5 76.5 0 0.5 319 As shown in Table 7, Gwen spent most her class time incorporating interactive lecture, individual activities, and pair/small group activities, with each of these activities lasting about 1.5 hours or longer. It is important to note that pair/small group activities were only incorporated on two days with the other instance lasting only about ten minutes, so much of the pair/small group work was implemented on one specific day. The rest of the activity structures were used for a total of less than twenty minutes each, with teacher giving information being the only 73 activity that was not used during the observed lessons. Three of the activity structures (warm-up, presenting student work, and summary) were each used only once. It is important to note that Gwen only had five observations whereas Charlie and Evan each had six, therefore, this influenced the amount of time Gwen spent on each activity structure. These three teachers had two activity structures in common as their most often implemented: pair/small group and individual activity. While Charlie and Evan spent similar amounts of time using pair/small group, Gwen did not use it as often. Evan spent almost 40 minutes more on this activity structure than Gwen. Evan also spent more time having his students work individually than Gwen did by 46 minutes, however, Gwen spent more time than Charlie by about 18 minutes. What makes Gwen unique out of these three teachers is the extensive amount of time she spent lecturing to her students. She spent about 8 times more minutes than Charlie and more than twice as much as Evan. Even though Gwen and Evan taught at the same school, they used their class time differently. Lindsay’s Lesson Structure I observed Lindsay’s Algebra 2 class for six lessons across the two rounds of data collection about five weeks apart. Her class met every day for approximately 53 minutes each day. Lindsay’s activity structure is shown in Figure 5. Figure 5 Lindsay’s Activity Structures by Lesson 74 Lindsay spent many class sessions starting with taking attendance or on the two Friday observations, she spent time allowing the students to share good news. As shown by the large amounts of green in Figure 5, Lindsay provided students with ample time to work with a partner or small group. She often started the class with a small group activity or an introduction to the lesson or class. Individual activity (orange) and lecture (red) were each incorporated once and having students present their own work (purple) occurred during half of the lessons. Lindsay ended most of the classes with a summary (brown) of the big ideas of the lesson. Table 8 shows the total amount of time (rounded to the nearest half of minute) Lindsay spent on each of these activity structures. Table 8 Lindsay’s Total Time Per Activity Structure Day 1: March 30 0 Day 2: March 31 0 Day 3: April 1 0 Day 1: May 12 0 Day 2: May 13 0 Day 3: May 16 Total 0 0 7 0 8 22 0 13.5 0 0 0 41 4 8 5.5 0 0 37 3 1 4 4 0 29 0 3 0 0 34 2 13.5 6.5 7 0 0 26.5 4 8 44 207 0 0 9 42.5 Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work Whole Class Discussion Teacher giving information Summary Total 0 0 50.5 0 1 54 0 1 47.5 0 0.5 51 0 0 45.5 0 1 52 0 3.5 300.5 As shown in Table 8, Lindsay spent a large majority of her class time having students work with a partner or small group, spending almost 3.5 total hours on this activity, which is four 75 times the amount of time spent on the next most common activity structure, which was whole class discussions. Aside from introduction to class or lesson, which totaled almost 30 minutes, the rest of the activity structures each lasted a total of nine minutes or less. Warm-up and teacher presenting information were the only two activity structures that Lindsay did not implement during the observed lessons. Lindsay’s lesson structure was very different from the previous three teachers. While all four teachers implemented pair/small group work as one of their most common activity structures, the amount of time Lindsay spent implementing this activity structure was considerably higher than the previous teachers. She used this 1.5 times more often than Evan and about twice as often as Charlie and Gwen. Like Charlie and Evan, whole class discussion was one of Lindsay’s most common activity structures, however, again, her usage of this activity structure was very different than the other teachers. Charlie implemented this activity structure almost three times more often and Evan used it twice as often as Lindsay. Lukas’s Lesson Structure I observed Lukas’s Pre-Calculus/Coordinate Geometry class for five lessons across the two rounds of data collection almost exactly one month apart. Lukas was not observed for Day 2 of Round 2 because he was giving his students a unit test during the entire class period, therefore he requested I not observe that day because it would be awkward for students to be video- recorded while they were taking a test. His class met every day for approximately 70 minutes each day. Lukas used a variety of activity structures, often moving through different structures within the same class period, which is shown in Figure 6. 76 Figure 6 Lukas’s Activity Structures by Lesson Lukas started every class with an introduction to what the students would be doing that day, which was followed the first three days by a warm-up activity (gray), with an additional warm-up activity incorporated on a fourth day after going over the assessment. Some of these introductions were followed by the teacher providing information to the students (navy blue), which consisted of reading the answers to the previous day’s assessment to the students while they looked at their own papers. Lukas often had students come to the board to share their work (purple), incorporating this activity four out of five days, often multiple times during the class period. Every day during the data collection period, Lukas had the students work with a partner or small group (green), and four out of five days, he orchestrated whole class discussions (blue). Only three of the five days included Lukas lecturing (red) to the students. Table 9 shows the total amount of time (rounded to the nearest half of minute) Lukas spent on each of these activity structures. Table 9 Lukas’s Total Time Per Activity Structure Day 1: April 19 5 Day 2: April 20 9 Day 3: April 21 5 Day 1: May 18 0 Day 3: May 20 Total 4 23 1 5 0 1.5 0 24 1 5 9.5 1 0 0 4 20 0 8.5 30 33.5 77 Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Table 9 (cont’d) Pair/ Small group work Presenting student work Whole Class Discussion Teacher giving information Summary Total 0 32.5 22 21 29 5 21 5 11 7 0 0 66 0 0 57.5 4 0 67.5 22.5 13 91.5 0 0 0 56 6 9 43 69.5 14 0 70 18 0 317 As shown in Table 9, Lukas spent most of his class time having students work with pairs or small groups (91.5 total minutes) or orchestrating whole class discussions (69.5 minutes). Lukas incorporated five of the nine activity structures for a total of between 20 and 45 minutes, with only one activity structure less than ten minutes (introduction to class and lesson). Lukas never utilized the summary activity. Lukas had the most common activity structures as Charlie and Evan, however, he spent different amounts of time implementing these. It is important to note, though, that Lukas only had five observations and Charlie and Evan each had six, therefore, that influenced the differences in the amount of time on each activity structure. Charlie spent almost twice as much time as Lukas orchestrating whole class discussions. Evan spent almost 35 minutes more than Lukas having students work with partners or in small groups. In addition, Lindsay spent more than twice as much time using this activity structure compared to Lukas. Evan also spent considerably more time having his students work individually compared to Lukas, with Evan’s spending more than four times as many minutes that Lukas. Charlie spent more than twice as much time on this activity structure as Lukas. 78 Scott’s Lesson Structure I observed Scott’s Geometry class for five lessons across the two rounds of data collection almost exactly one month apart. Scott was only observed for two days during Round 1 due to scheduling conflicts. Scott was at the same school as Lindsay, therefore his class met every day for approximately 53 minutes each day. Scott did not use a variety of activity structures, which is shown in Figure 7. Figure 7 Scott’s Activity Structures by Lesson Scott started every class with taking attendance and the Pledge of Allegiance. Since this was first hour, the school required teachers to start each day with the Pledge. Scott followed a fairly strict schedule each day as seen in Figure 7. After the logistical activities, he would introduce the lesson (yellow) and then provide students with most of the rest of the class period to work in their small groups (green) on the lesson. For four out of the five lessons, Scott ended the lesson with a whole class discussion (blue) based on what the students discovered during their small group work. Only once did Scott end class with a summary (brown, March 21) or having a student present his work (purple). During one other lesson, Scott had the students work individually (orange), however those three instances were the only time that Scott broke from his standard routine. Table 10 shows the total amount of time (rounded to the nearest half of minute) Scott spent on each of these activity structures. 79 Table 10 Scott’s Total Time Per Activity Structure Day 1: March 21 0 Day 2: March 22 0 Day 1: May 3 0 Day 2: May 4 0 Day 3: May 5 Total 0 0 3 0 0 41 0 3.5 2 0 0 39 0 5 6.5 0 0 38 2 2 2 0 5 31 0 6.5 3 0 0 16.5 0 5 40 189 0 4 2 21 Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work Whole Class Discussion Teacher Giving Information Summary Total 0 0.5 48 0 0 46 0 0 48.5 0 0 44.5 0 0 47 0 0.5 233.5 As shown in Table 10, Scott spent a significant amount of his time (more than 3 hours out of a total less than 4 hours) having students work in pairs or small groups. The next most common activity structure, whole class discussions, only totaled about 20 minutes. Aside from introduction to class and lesson, which lasted a total of about 16 minutes, the rest of the activity structures each lasted a total of five minutes or less. There were three activity structures (warm- up, interactive lecture, and teacher giving information) that were not utilized by Scott. Scott and Lindsay, who taught at the same school, had the same activity structures as the most commonly implemented in their classrooms. It is important to note that Scott only had five observations, whereas Lindsay had six, therefore, this influenced the amount of time he spent on each activity structure. Based on these five days, though, it is likely a sixth observation would contain similar amounts of time spent on the different activity structures. Lindsay spent about 18 80 minutes more having her students work in pairs or small groups, so it is excepted that Scott’s amount of time would be higher had he had six observations, since he spent at least 30 minutes on this activity structure in all the observed lessons. Lindsay spent twice as much time leading whole class discussions as Scott. Based on how much time Scott spent on this activity structure during the five observations, it is likely Lindsay still would have about twice as much time using this activity structure as him. This is similar for the introduction activities, since Lindsay spent about ten minutes more on this activity structure compared to Scott. On average, he only spent about three minutes on this activity structure, therefore, it is likely Lindsay would still have spent more time on this activity structure. Comparisons Across Teachers Even though the teachers came from four different schools and taught five different subjects, the nine activity structures represented all their lesson activities. However, the teachers implemented these activity structures to varying degrees. Table 11 shows the percentage of time each teacher spent incorporating each of the activity structures. 1.23% 0.58% 1.04% 0.00% Lindsay Gwen Charlie Evan Table 11 Teachers’ Percentage of Time Per Activity Structure Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work 30.92% 21.24% 28.73% 3.49% 3.53% 26.95% 32.22% 29.69% 10.37% 33.29% 7.45% 2.01% 0.00% 1.60% 5.12% Lukas 7.44% Scott 0.00% 2.74% 9.39% 10.53% 6.81% 0.00% 2.04% 28.63% 81.05% 13.72% 0.84% 8.67% 1.31% 2.71% 69.17% 2.98% 81 34.48% 20.42% 5.26% 14.15% 21.88% 9.09% 0.00% 0.23% 0.00% 0.16% 0.00% 1.02% Table 11 (cont’d) Whole Class Discussion Teacher Giving Information Summary 0.00% 0.17% There are important differences to notice in how the teachers spent their class time. Scott 0.00% 0.00% 5.67% 0.00% was the only teacher who basically used one activity structure during all of observations. He spent more than 80% of his time having students work with partners or small groups. He spent less than 10% of his time on each of the other activity structures. Pair/Small Group work was one of the three most common activity structures used by each of the six teachers, however, they used this with very different amounts of time. Scott spent 10% more time on this activity structure than the next teacher, who was his colleague, Lindsay (69%). The other four teachers each spent about 30% of their time having students work with a partner or small group. Whole class discussions were another common activity structure that was implemented across the teachers, however, there was some variance to the percentage of time they spent on this activity structure. Gwen was the only teacher who did not have whole class discussions as one of her three most common activity structures. Gwen and Scott each spent less than 10% and Lindsay spent less than 15% of their time compared to the other three teachers who each spent more than 20% of their time with Charlie spent the highest percentage of time (35%) on this activity structure. Scott was the only teacher who did not implement interactive lecture, however, for the teachers who did implement this activity structure there was a large range of time spent on this activity structure. Lindsay spent less than 2% of her time compared to Gwen who spent one-third of her time incorporating this activity structure. This was three times the amount of time as the 82 next teacher (Evan), who happened to be at the same school as her. There was also a varied amount of time spent on having students work alone from Lindsay and Scott (who were at the same school) each spending about 2% of their time to Lukas who spent about 10% of his time to the other three teachers who each spent two or three times Lukas’s amount of time (Charlie – 20%, Evan – 32%, and Gwen – 30%). There was also quite a range of time spent on having students present their work or teachers sharing student work. Gwen and Scott were each around 1% of their time, however, Lukas spent almost 14% of his time on this activity, which was almost three times as much as Charlie (5%). 83 Chapter 5: Teachers’ Formative Assessment Practices Findings This chapter addresses the first research question, which asked, “What formative assessment practices are some high school mathematics teachers implementing in their lessons to support students’ learning of the mathematical content?”. This chapter identifies which formative assessment practice these teachers implemented and then builds on the lesson structure framework from the previous chapter to map out where the teachers were implementing the formative assessment practices. The chapter starts by focusing on which activity structures contained the different formative assessment practices, then transitions to each particular teacher’s practice. Also, included is a discussion of findings related to the different levels of sophistication for each of the formative assessment practices. The chapter closes with a comparison across all the teachers and a rationale for selecting two of Charlie’s lessons for further analysis as examples of relatively “high-quality” formative assessment practice. Overall, the activities of Whole Class Discussion and Pair/Small Group Work included the most occurrences of formative assessment practice, whereas Warm-up and Summary had the fewest occurrences. When frequency is addressed by teacher, Whole Class Discussion was the activity structure that included the most formative assessment practices for Charlie and Evan. Pair/Small Group Work was the activity structure that Lindsay and Scott used to implement the most formative assessment practices. For Gwen, Interactive Lecture included the highest number of formative assessment practices, whereas for Lukas, it was Presenting Student Work. None of the teachers implemented peer assessment, and some of the teachers did not implement other formative assessment practices, such as learning targets. These formative assessment practices were also implemented at various levels of sophistication ranging from Level 1 to Level 4, with very few occurrences of Level 4 practice. Charlie was the only teacher who had more than one 84 occurrence of a Level 4 formative assessment practice; he achieved this level of practice at least once for every formative assessment practice he implemented. Gwen and Lindsay each had one occurrence of a Level 4 formative assessment practice. Evan, Lukas, and Scott only achieved a Level 3 for their formative assessment practices. In addition, most of Gwen’s and Lukas’s formative assessment practices were Level 1 or Level 2, therefore, most of the practices they demonstrated were low-level. There were six formative assessment practices that were coded in this analysis: learning target use (LT), questioning types (QT), self-assessment (SA), peer assessment (PA), feedback (F), and instructional decisions (ID). Each of these practices was coded into four levels of practice ranging from novice (level 1) to expert (level 4). Formative Assessment Practices by Activity All the activity structures included at least one of the formative assessment practices. However, the implementation of these practices varied across the different activity structures. Table 12 shows where each of the formative assessment practices occurred during the nine activity structures as well as the total amount of time spent on each of these activity structures to provide context. Table 12 Formative Assessment Practices by Activity Structure Total Time for Activity Structure 34 102 199 359.5 Warm-up Introduction Interactive Lecture Individual Activity Learning Targets 0 18 Question- ing 2 10 Self- Assess- ment 0 1 Peer Assess- ment 0 0 Feedback 1 7 Instructional Decisions 0 4 2 4 22 15 0 2 85 0 0 10 13 1 1 Total FA 3 40 35 35 Table 12 Pair/ Small Group Present Student Work Whole Class Discussion Teacher Giving Info Summary Total (cont’d) 811.5 78 364.5 18 5 1971.5 0 0 2 0 0 26 44 19 48 0 3 163 2 0 1 0 0 6 0 0 0 0 0 0 30 13 26 4 0 104 3 4 3 0 0 16 79 36 80 4 3 315 Learning targets were predominately shared with students during the introduction activities (N=18), but were also mentioned during individual, interactive lecture, and whole class discussions. Sharing learning targets during the introduction activities made sense because this activity structure was often used at the beginning of the class period and was where the teachers introduced the plan for that day’s class, often reading an agenda that included the learning targets. Feedback was the most common (and only) formative assessment practice for teaching giving information. This made sense because during this activity structure the teacher provided overall class feedback on an assessment or assignment while reading the solutions. Questioning was the most common formative assessment practice for the rest of the activity structures with Whole class discussion (N=48) and Pair/Small Group (N=44) accounting for more than half of the questioning practices used by teachers. Questioning as the most common formative assessment practice for seven of the nine activity structures made sense for most of the activity structures. During many of the activity structures, the teachers would need to gather information about students’ understanding and questioning was the best way to do this. Questioning as the most common formative assessment practice for presenting student work is slightly surprising since during this activity structure the students were sharing their work with the class, therefore, 86 it seems that feedback would have been a common practice. However, the counts for questioning and feedback were close, therefore, feedback was a practice that was often used during this activity structure. Questioning as the most common formative assessment practice for interactive lecture also seemed slightly surprising because it was not expected that much formative assessment would occur if the teacher was lecturing. However, during this activity structure teachers did check in with students at different times, expecting them to participate at some level. Pair/Small Group and Whole Class Discussion had the most occurrences of formative assessment practices and they also had the largest amount of time spent on them during the lessons. These two activity structures had almost the same amount of formative assessment practices, even though Pair/Small Group (811 minutes) was used more than twice as often as Whole Class Discussion (364.5 minutes). Introduction and Presenting Student Work had the next highest occurrences of formative assessment practices, however, there were two activity structures (Interactive Lecture and Individual) that had more time spent on them. Looking at the data more closely shows during which activity structure each teacher used the formative assessment practices. Table 13 displays the total number of formative assessment practices each teacher used during the different activity structures. Charlie Lindsay Table 13 Teachers’ Formative Assessment Practices by Activity Structure Teacher Warm-up Introduction Interactive Lecture Individual Activity Pair/ Small Group N/A 5 26 15 7 7 7 4 1 3 7 3 0 12 7 1 8 4 8 1 5 16 1 3 Evan Gwen Lukas 15 Total Scott N/A 7 N/A 0 19 3 40 35 35 79 87 (cont’d) 4 22 N/A 19 2 5 6 14 22 12 2 8 36 80 N/A N/A 62 N/A 0 40 Table 13 Present Student Work Whole Class Discussion Teacher Giving Info Summary Total 4 3 315 Charlie implemented formative assessment practices during all the activity structures that N/A 0 52 N/A 3 58 4 N/A 67 N/A 0 36 he implemented, with Whole Class Discussions and Individual activities being the activity structures that contained the most formative assessment practices. He also implemented the second most number of formative assessment practices. Like Charlie, Whole Class Discussion was the activity structure in which Evan implemented the most formative assessment practices. Gwen implemented the second fewest number of formative assessment practices, however, interactive lecture was the activity structure that provided her the most opportunities to incorporate these practices. Like Charlie, Lindsay implemented formative assessment practices in every activity structure that she implemented. In contrast to Charlie, though, pair/small group work was the activity structure that included the most formative assessment practices. Lukas used the highest number of activity structures to implement his formative assessment practices, with summary being the only activity structure that did not contain formative assessment practices. Like Lindsay, pair/small group work was the activity structure implemented by Scott that had the highest number of formative assessment practices, however, he demonstrated the fewest number of formative assessment practices out of the six teachers. It is important to note that Gwen, Lukas, and Scott had one lesson observation than the other three teachers, so this influenced the number of practices they could implement. practices. 88 Given the overall differences in teachers’ uses of formative assessment practices described above, in the next section, I present each teachers’ use. It is interesting to note the variations across teachers not only in which formative assessment practices they implemented during the different activity structures, but also in the level of sophistication each teacher demonstrated regarding each of the formative assessment practices. Charlie’s Formative Assessment Practices Charlie implemented every formative assessment practice, except peer assessment. He incorporated these practices throughout his lessons with many activity structures including multiple formative assessment practices, as seen in Figure 8. His primary activity structures across lessons were whole class discussion, pair/small group work, and individual activity, collectively accounting for 87% of overall lesson time. Figure 8 Charlie’s Formative Assessment Practices by Lesson Learning Targets. Charlie introduced students to the learning targets 8 times during the observed lessons. In all of Charlie’s initial introduction activities (yellow), he shared the learning targets with the students, though with varying levels of practice. In the last two observed lessons, Charlie only made a passing mention of the learning targets (Level 1) compared to higher level practices demonstrated in the three other lessons starting with an introduction activity. In these instances, Charlie was often explicit with students about what the learning targets were and how they connected to the lesson’s activities. To achieve a Level 4 practice (N=2) on May 10, Charlie 89 provided students with more information about the type of knowledge and skills that would be developed through the lesson activities that would help students meet the learning targets. The introduction activities were not the only place that Charlie talked with students about the learning targets. Charlie also achieved high levels of practice related to learning targets on June 3 during the individual activity (orange, Level 3) and whole class discussion (blue, Level 4). The final instance of learning targets occurred on May 10 during the second individual activity when Charlie reminded students of the learning targets. Questioning types. Charlie used questioning to elicit student thinking 29 times during the observed lessons. Out of the seven activity structures that Charlie implemented, questions were used to elicit student thinking in six of them; the introduction was the only one that did not include questions. These question types ranged from Level 1 (low-level questions) to Level 4 (mix of low and high-level questions with scaffolding from low to high level questioning) depending on the lesson activity. During the warm-up (gray), Charlie had a Level 1 practice because he used only low-level questions that had one correct answer. This was the only time he was at a Level 1 practice for questioning. This means that every other activity structure that included questioning contained at least one high-level question. At the other end of the spectrum, Charlie achieved a Level 4 practice twice, once during the presenting student work activity (purple) on May 9 and once during a whole class discussion (blue) on June 2. During all the pair/small group activities (green), Charlie achieved a Level 2 practice (N=13), which consisted of mainly low-level questions, but started including some high-level questions. For all the individual activities (orange) and interactive lectures (red), with the exception of an individual activity on May 10 and June 3 and interactive lecture on May 9, Charlie also had a Level 2 practice. For these exceptions and all but two of the whole class 90 discussions (blue), Charlie achieved a Level 3 practice (N=13), which means he incorporated a mix of low and high-level questions that pushed students to justify their solutions. This mix of questions often included almost equal amounts of low and high-level questions. This variation in levels of questioning practices made sense based on the activity structure. Lower-level questions were often asked during the interactive lecture, which would not likely contain high-level questions, or when students were working on their own. On the other hand, higher-level questions were asked during activity structures that focused on students sharing their ideas with the class, such as whole class discussions or presenting student work. Self-assessment. June 3 was the only day that Charlie had his students complete a self- assessment. This was done at a Level 4 during the individual activity (orange), which means that the teacher provided students with opportunities to self-assess and think about their thinking that followed a specific structure aligned with the learning targets. During the whole class discussion (blue), Charlie reached a Level 3 because the activity was not as focused on the individual student’s learning. Feedback. Of the seven activity structures that Charlie implemented, he provided students with feedback during six of them for a total of 18 instances. These practices ranged from Level 2 to Level 4. The one instance of Level 4 feedback occurred during the second individual activity (orange) on June 1. To achieve this level of practice, Charlie provided the student with descriptive feedback about his solving process and the mathematical task the student was completing with enough detail that the student would be able to take the feedback and move forward on the task and advance his thinking. There was only one instance during the introduction activities (yellow) where Charlie provided students with feedback and this was in relation to a quiz the students had previously 91 taken. This instance reached a Level 2 feedback practice, meaning it was mainly evaluative and focused on the solution, however it also included some information about next steps. Charlie also had a Level 2 practice (N=8) during many of the other activities, including presenting student work, interactive lecture, individual activity, and whole class discussions. During all the pair/small group activities (green), some of the whole class discussions (blue), and some of the individual activities (orange), Charlie achieved a Level 3 practice (N=9). This means that the feedback was descriptive and focused not only on the solution, but also the strategy students used to complete the task and included suggested steps to help students move forward in their thinking that are not as clear as those in the Level 4 practice. Instructional Decisions. There were four activity structures (introduction, presenting student work, individual activity, and whole class discussion) that included instructional decisions, ranging from Level 2 to Level 4, for a total of 5 instances. When Charlie implemented a Level 2 practice (N=2) during the introduction on June 2 and the presenting student work activity on May 9, he provided students with a vague rationale for the decisions he was making in the moment. Charlie reached a Level 3 practice twice, once during an individual activity on May 10 and once during a whole class discussion on June 3. To achieve a Level 4 practice, like Charlie did during the introduction on June 1, he was explicit with students for the decision he was making, providing them with clear evidence he had gathered about their mathematical thinking. Summary. Charlie implemented all the formative assessment practices, except peer assessment using seven of the nine activity structures. He achieved the highest level of formative assessment practice with all the practices he implemented; reaching a Level 4 in learning targets, questioning, feedback, self-assessment, and instructional decisions. Almost all of Charlie’s 92 formative assessment practices were Level 2 or Level 3, with more of them being classified as high-level (Levels 3 and 4). He had only three instances of Level 1 practice during the observed lessons. Questioning and feedback were Charlie’s most commonly implemented formative assessment practices. For both practices, Charlie had more than half of the instances coded as Level 3 or Level 4, which means he consistently demonstrated high-level practices for these two formative assessment practices. Charlie almost always introduced the learning targets to his students during the Introduction activities. Charlie stated during the interviews that questioning was important to him; therefore, it was not surprising that he asked questions during six of the seven activity structures, with most of the instances occurring during Whole Class Discussions. Charlie was one of three teachers who implemented self-assessment and both times he implemented this formative assessment practice, it was at high-levels. Feedback was another formative assessment practice that was regularly implemented in Charlie’s classroom, occurring during six of the seven activity structures. All instances were either Level 2 or Level 3, with one occurrence reaching Level 4. It is not surprising that most of Charlie’s formative assessment practices were visible during Whole Class Discussions, since this was the activity structure he most often implemented in his lessons (34% of the time). Evan’s Formative Assessment Practices Evan implemented every formative assessment practice, except peer assessment. He incorporated these practices throughout his lessons with many activity structures addressing multiple formative assessment practices, as seen in Figure 9. Like Charlie, Evan’s primary activity structures were individual activity, pair/small group work, and whole class discussion. But in contrast, Evan began with individual activities and devoted more time to it than Charlie 93 (and less time for whole class discussion). Two of the activity structures (Warm-up and Summary) Evan implemented did not include any formative assessment practices. Figure 9 Evan’s Formative Assessment Practices by Lesson Note: The * represents LT2, F1 Learning Targets. Evan had 9 instances of sharing the learning targets with his students. During almost all his introductions (yellow), Evan introduced the learning targets to students at either a Level 2 or 3. This means he either read the learning target to students (Level 2, N=5) or connected that day’s activities to the learning target (Level 3, N=3). Evan made mention of the learning targets during other activities as well. During the third whole class discussion (blue) on April 26, Evan read the learning targets to students and during the interactive lecture (red) on May 23 he made a passing mention to it (Level 1, N=1). Questioning types. Evan used questioning 24 times to elicit students’ thinking. He used a range of questions during the observed lessons achieving between a Level 1 and Level 3 depending on the activity. Half of the interactive lectures had Level 1 questioning (N=7) because Evan was only asking low-level questions. These types of questions were also asked during two of the introductions, one individual activity (orange), and one whole class discussion. There were almost the same number of instances where Evan achieved a Level 3 questioning (N=5), mixing low and high-level questions and asking students to explain their answers. This was done during 94 four of the whole class discussions and one of the pair/small group activities. This means that Evan reached higher levels of questioning types almost exclusively during the whole class discussions. Most of the time, Evan reached a Level 2 (N=12) in his questioning, which means he still focused more on low-level questions, but started to include some high-level questions. This was done during one of the introductions and individual activities, roughly half of the whole class discussions, and almost all the pair/small group work. For example, during the introduction, Evan asked eight low-level questions and two high-level questions, which resulted in a Level 2 for his questioning during this activity structure. Self-assessment. Evan reached a Level 3 for self-assessment twice during the observed lessons, once during the second introduction activity (yellow) on the first day and once during the individual activity (orange) on the second day of data collection. To achieve a Level 3, Evan provided the students with structure in the form of a rubric and opportunities to self-assess. Feedback. Evan provided his students with feedback 14 different times during the observed lessons. He demonstrated a range of practices regarding feedback achieving between a Level 1 and Level 3 depending on the activity, however, most of his practices were at a Level 1 (N=8), which means that he provided evaluative feedback that was focused on students’ answers and not on their process. He used this practice during many of the activities, such as most of the whole class discussions (blue), two individual activities (orange), one introduction (yellow), and one interactive lecture (red). Evan achieved a Level 2 practice (N=5) during many of these same activities, two whole class discussions, one individual activity, one interactive lecture, and one pair/small group activity. During one of the whole class discussions (April 26), Evan reached a Level 3 practice for feedback because he provided students with descriptive feedback that 95 focused not only on the solution, but also the process that the student took in solving the problem. Instructional Decisions. Evan achieved a Level 2 practice for instructional decisions three different times during the observed lessons, twice during whole class discussions on the first day and once during the introduction on May 24. To achieve a Level 2 practice, Evan provided the students with a vague rationale for the decisions he was making in the moment. Summary. Evan implemented all the formative assessment practices, except peer assessment using five of the nine activity structures. Like Charlie, Evan’s most commonly implemented formative assessment practices were questioning and feedback. In contrast to Charlie, though, most of Evan’s uses of these formative assessment practices were Level 1 or Level 2, which means most of the time, he demonstrated low-level formative assessment practices. An exception to this was when Evan implemented self-assessment, which like Charlie, he did at a high-level both times. Most of the time, when Evan shared the learning targets with students, it was during Introduction activities. Evan asked questions during five of the activity structures with most of the instances occurring during Whole Class Discussions. Evan provided his students with feedback during five of the activity structures, with half of these instances occurring during Whole Class Discussions. Like Charlie, Whole Class Discussions was the most common activity structure where Evan implemented formative assessment practices, however, the difference was that this was not the most common activity structure for Evan. Gwen’s Formative Assessment Practices Gwen implemented most of the formative assessment practices, except self and peer assessment. She incorporated these practices throughout her lessons with some activity structures 96 addressing two or three different formative assessment practices, as seen in Figure 10. In contrast to both Charlie and Evan, Gwen used more interactive lecture and very little whole class discussion. Summary was the only activity structure Gwen implemented that did not include any formative assessment practices. Figure 10 Gwen’s Formative Assessment Practices by Lesson Learning Targets. Gwen achieved a Level 2 or 3 for learning targets during the observed lessons. Twice learning targets were shared with students during an introduction activity (yellow) and once during an interactive lecture (red). To reach a Level 3 like she did twice, Gwen was explicit with students on what the learning target was and how it connected to the lesson’s activities. Questioning types. Gwen used questioning 20 different times during the observed lessons. She achieved between a Level 1 and a Level 3 for her questioning types, with all but three of her questions being coded as Level 2, meaning she mostly asked low level questions, but did include some higher level questions as well. Gwen focused her Level 1 questions (N=2) on activities such as the warm-up (gray) and introduction (yellow). Gwen incorporated Level 2 questions (N=17) into six out of the eight activities that she implemented. All, but one of the activities (the final lecture) from May 25 included these types of questions. The final lecture was where Gwen achieved her highest level of questioning, reaching a Level 3 because she included more of a mix of low and high-level questions. She also used Level 2 questions during all the 97 whole class discussions, pair/small group work (green), and the presentation of student work (purple). Feedback. Gwen had 15 instances of providing her students with feedback during the observed lessons. She also achieved Level 1 or 2 for her feedback practices. Level 1 feedback (N=9), which was strictly evaluative, was used during most of the interactive lectures and individual activities. It was done during one of the whole class discussions and the presentation of student work. Gwen used Level 2 feedback (N=6) during all the pair/small group work, some of the interactive lectures and individual activities, and one of the whole class discussions. Instructional Decisions. Gwen achieved higher levels of practice for her instructional decisions, reaching a Level 3 during one of the interactive lectures and a Level 4 during the introduction, both of which occurred during the same lesson. To reach these levels of practice, Gwen was explicit with students regarding the decisions she was making in the moment and provided them with evidence of their learning that supported her decision making. Summary. Gwen implemented all the formative assessment practices, except self and peer assessment using seven of the activity structures. Like Charlie and Evan, Gwen’s most commonly implemented formative assessment practices were questioning and feedback. In contrast, she used different activity structures to implement these formative assessment practices. While Gwen achieved high-level practices five times during her observed lessons, most of her formative assessment practices were Level 1 (N=11) or Level 2 (N=24), which means most of the formative assessment practices she demonstrated were low-level. Evan and Gwen taught at the same school, so it may not be surprising that their level of sophistication in formative assessment was similar in that both demonstrated lower-level practices. In contrast though, Evan 98 had twice as many practices coded as high-level compared to Gwen. He also had five more years of teaching experience than her. Gwen rarely shared the learning targets with her students, but when she did it was usually done during the Introduction activities, which was also when Charlie and Evan shared them with their students. Gwen asked questions during all the activity structures that she implemented, except summary. Gwen provided feedback to her students during five of the activity structures. Instructional decisions were the only formative assessment practice in which Gwen achieved only high-levels of practice. It is interesting to note that both instances occurred during the same class period. Gwen spent more time using Interactive Lecture (33% of her class time), therefore, it is not surprising that most of her formative assessment practices occurred during this activity structure. Lindsay’s Formative Assessment Practices Lindsay implemented half of the formative assessment practices, not using learning targets, self-assessment, or peer assessment. She incorporated these practices throughout her lessons with some activity structures addressing two or three different formative assessment practices, as seen in Figure 11. Like Charlie and Evan, Lindsay’s primary activity structures were pair/small group work and whole class discussion. In contrast, she used pair/small group work considerably more often than the previous three teachers, using it almost 70% of the time. 99 Figure 11 Lindsay’s Formative Assessment Practices by Lesson Note: The + represents QT3, F1 Questioning types. Lindsay used questioning 38 different times during the observed lessons. She implemented a variety of questioning types reaching all levels of practice during the observed lessons. Her Level 1 practices (N=5) were used during three different activities, once for interactive lecture (red), once for presenting student work (purple), once for introduction (yellow), and twice during pair/small group work. Most of Lindsay’s questioning, though, was at a Level 2 or Level 3. During six of the seven activities that Lindsay implemented, she reached a Level 2 practice (N=12), which means she asked mainly low-level questions, but included some high-level questions. These activities included the two introductions (yellow) where questions were asked, half of the presentations of student work, and four of the whole class discussions. Three activities (one individual activity, almost all the pair/small group activities, and most of the whole class discussions) included Level 3 practices (N=20) because Lindsay asked a mix of low and high-level questions, often asking students to explain or justify their solutions. Once Lindsay achieved a Level 4 questioning, which occurred during the final pair/small group activity on March 30. This was a Level 4 because Lindsay asked her students to make and verify predictions as well as make connections between different mathematical ideas while asking a mix of high and low-level questions. 100 Feedback. Lindsay had 17 instances of providing her students with feedback during the observed lessons. She also demonstrated a range in her feedback practices achieving Level 1 through Level 3 during the observed lessons. Only once did Lindsay reach a Level 3 practice, which was during the last pair/small group activity for the first day of observation. To achieve this level of practice, Lindsay provided descriptive feedback to the students not only on their solution, but also on their solving process, so they would have an idea of how to move forward when solving these types of problems. Lindsay focused on evaluative feedback (Level 1, N=6) during about half of the activities that she implemented, including one of the individual activities (orange), one of the pair/small group activities (green), two of the introductions (yellow), and two of the whole class discussions (blue). Most of Lindsay’s feedback was at a Level 2 (N=10), though, so she went beyond providing only evaluative feedback to students to include some suggestions for moving forward that might have been too directive. Three different activities (whole class discussions, pair/small group work, and one of the presentations of student work) included this level of practice. Instructional Decisions. Lindsay achieved a Level 2 practice for instructional decisions three different times during the observed lessons. Twice this occurred during pair/small group activities and once while students were presenting their work. This practice was seen on three of the six observation days. To reach a Level 2 practice, Lindsay provided students with a vague rationale for why she was making the decisions she was during class. Summary. Lindsay implemented half of the formative assessment practices (questioning, feedback, and instructional decisions) using seven of the activity structures. Like the previous three teachers, questioning and feedback were the formative assessment practices most often implemented by Lindsay. More than half of her questioning practices were coded as high-level 101 (most were Level 3). However, only one instance of feedback was high-level, therefore, her feedback practices were mostly low-level. Overall, Lindsay’s sophistication implementing formative assessment was either Level 2 (N=25) or Level 3 (N=21). This means she demonstrated many high-level practices and many others that were moving towards high-level. Questioning was the only formative assessment practice in which Lindsay had more instances of high-level practice (N=21) than low-level practice (N=17). Lindsay asked questions during all the activity structures that she implemented, with most of these instances (N=15) occurring during Pair/Small Group Work. Lindsay provided her students with feedback during five of the activity structures, with most instances occurring during Pair/Small Group Work. Since Lindsay spent most of her class time using Pair/Small Group Work, it is not surprising that most of her formative assessment practices occurred during this activity structure. Lukas’s Formative Assessment Practices Lukas implemented most of the formative assessment practices, not incorporating self- assessment or peer assessment. He incorporated these practices throughout his lessons with many activity structures addressing two or three different formative assessment practices, as seen in Figure 12. Like Charlie and Evan, Lukas’s primary activity structures were pair/small group work, whole class discussion, and individual activities. Figure 12 Lukas’s Formative Assessment Practices by Lesson Note: + represents LT2, F1; ! represents QT1; @ represents QT1, F1; $ represents QT2, F1; # represents QT2; and & represents QT1 102 Learning Targets. Lukas achieved a Level 1 or 2 practice for learning targets, with this formative assessment practice being coded on three occasions. The Level 1 practice (one instance) occurred during the individual activity (orange) on April 20, which was when the students took a quiz. This means that Lukas just made a passing mention of the learning targets as students prepared to take their quiz. Both Level 2 practices occurred during introduction activities (yellow), which was when Lukas read the learning targets to the students. Questioning types. Lukas had 33 instances of using questions to elicit student thinking during the observed lessons. He achieved either a Level 1 (N=9) or Level 2 (N=24) practice for his questioning types with more than twice as many instances reaching a Level 2, which means he incorporated some high-level questions amongst the low-level questions. Level 1 question types were used during multiple activities, such as pair/small group work (green), whole class discussions (blue), and one interactive lecture (red). These same activity structures also included Level 2 practices, with most of the interactive lectures reaching these levels. All the presentations of student work (purple), all of the whole class discussions, except those on May 20, and four of the pair/small group activities contained Level 2 questions. Feedback. Lukas had 29 instances of providing students with feedback during the observed lessons. He had a range in his feedback practices reaching Level 1 through Level 3 during the observed lessons. There was one instance of a Level 3 practice, where Lukas provided descriptive feedback on the students’ process along with the solution during one of the pair/small group activities on May 18, which contained a test review activity. There were six different activity structures that contained Level 1 practice (N=11), such as an individual activity (orange), one of the teacher giving information (navy blue), and one warm-up (gray). There were four activities in which Lukas consistently reached a Level 2 103 practice (N=17), meaning he was moving beyond just providing evaluative feedback, but his feedback on the process might be too vague for students to be clear on how to move forward. These activities included teacher giving information, presentations of student work, pair/small group work, and whole class discussions. Instructional Decisions. On two occasions Lukas achieved a Level 3 practice for instructional decisions, which means he was explicit with students regarding the decisions he was making in the moment and used evidence he had gathered about their learning to make these decisions. Both instances occurred during presentations of student work, once on April 20 and once on April 21. Summary. Lukas implemented all the formative assessment practices, except self- assessment and peer assessment using eight of the activity structures, which was the highest number of activity structures implemented by one of the teachers. Like all the previous teachers, questioning and feedback were the formative assessment practices most often implemented by Lukas. All of Lukas’s questioning practices were low-level, which contrasted with some of the other teachers, like Lindsay and Charlie who often demonstrated high-level questioning practices. Lukas demonstrated similar levels of sophistication for feedback, with most of his practices Level 1 (N=11) or Level 2 (N=17), however he did achieve a high-level of practice once during the observed lessons. This level of sophistication was like that of Evan and Lindsay, who also demonstrated more low-level and one high-level feedback practice. Similar to Evan and Gwen, though, overall most of Lukas’s formative assessment practices were Level 1 (N=21) or Level 2 (N=43). This means that most of the formative assessment practices he implemented were low-level. 104 Lukas rarely shared the learning targets with students, but when he did, it was usually during Introduction activities, which was the same activity structure used by the other teachers. Lukas provided his students with feedback during every activity structure that he implemented with most of the feedback occurring during Presenting Student Work (N=9) and Pair/Small Group Work (N=7). It is not surprising that much of his feedback was during Pair/Small Group activities, because Lukas stated during the interviews that having students work in groups made it easier for him to check in with them and provide feedback to the group since he did not have time to go around and talk with every student during every lesson. Instructional decisions were the only formative assessment practice where Lukas had only high-level practices (Level 3). It is interesting that both instances occurred during Presenting Student Work as it implies that something in the student’s work led Lukas to make a decision about the direction of the lesson. It is surprising that much of Lukas’s formative assessment practices occurred during Presenting Student Work since this was not the most common activity structure implemented by him. He only spent about 14% of his class time on this activity. This implies that having students share their work in class was a useful structure for Lukas to implement formative assessment practices. Scott’s Formative Assessment Practices Scott implemented every formative assessment practice, except peer assessment. He incorporated these practices throughout his lessons with some of his activity structures addressing two formative assessment practices, as seen in Figure 13. Like Lindsay, Scott’s primary activity structures were pair/small group work, whole class discussion, and introduction to class/lesson. Even though all the previous teachers implemented pair/small group work as one of their most common activity structures, Scott stood out because he mostly only used this activity structure, using it more than 80% of his lesson time. Individual and Summary were the 105 two activity structures that Scott implemented that did not include any formative assessment practices. Figure 13 Scott’s Formative Assessment Practices by Lesson Note: @ represents LT1, F1; & represents SA3, QT3; and *representing QT3, F1 Learning Targets. Whenever Scott shared the learning targets with the students, it was always during the introduction activities (yellow). All three of these instances, though, only achieved a Level 1 because he made a passing mention to them without actually reading the learning target to students. Questioning. Scott asked questions during 19 different instances across the observed lessons. All of Scott’s questioning practices were either Level 2 (N=8) or Level 3 (N=11). Four out of the six activities that Scott implemented included questions with individual activities (orange) and summary (brown) as the exceptions. During two of the introduction activities, two of the pair/small group activities (green), and most of the whole class discussions (blue), Scott achieved a Level 2 questioning because he asked mainly low-level questions but included some high-level questions. To reach a Level 3 practice, like he did during the presentation of student work and more than half of the pair/small group and three of the whole class discussions, Scott incorporated a mix of low and high-level questions. 106 Self-assessment. Scott had two instances of providing students opportunities to self- assess and both reached a Level 3, meaning that he provided students opportunities to self-assess using a structure that encouraged them to reflect on what they had learned during the lesson. Feedback. Scott had 11 instances of feedback across the observed lessons that reached a Level 1 (N=7) or Level 2 (N=4) practice, which means that his feedback was mainly evaluative and focused on students’ solutions. To reach a Level 2 practice, he started to give some feedback on process, but it might have been too directive or vague for students to do the thinking on their own of what they should do next. Scott provided feedback during four different activities. Level 1 feedback was provided during the presentation of student work, two of the introduction activities, and one pair/small group and whole class discussion. In most his pair/small group activities, though, Scott achieved a Level 2 practice. Instructional Decisions. Scott had one instance of making instructional decisions during the observed lessons, which occurred during the second pair/small group activity on the first day of observation. This was a Level 1 practice because Scott did not provide the students with any kind of rationale for his in-the-moment decision making. Summary. Scott implemented all the formative assessment practices, except peer assessment using four of the activity structures, which was the fewest number of activity structures used by one of the teachers. Like all the other teachers, questioning and feedback were the formative assessment practices most often implemented by Scott. All his questioning practices were Level 2 (N=8) or Level 3 (N=11), which means he achieved high-level practices more often than low-level. He demonstrated less sophistication implementing feedback, though, since all his practices were Level 1 (N=7) or Level 2 (N=4), which means these were all low- level practices. This was like Gwen who also only demonstrated low-level feedback practices. 107 However, overall Scott had more Level 3 practices (N=13) than any other level of practice. His distribution of levels of practice was very similar across all three levels as Level 1 had 11 instances and Level 2 had 12 instances. This means that Scott had twice as many formative assessment practices coded as low-level compared to high-level, therefore, he demonstrated mostly low-level sophistication with formative assessment. If Scott shared the learning targets with the students, it was always during the Introduction activities, which was the same activity structure as all the other teachers. Scott asked questions during all four of the activity structures he implemented, however, most of them occurred during Pair/Small Group Work (N=9). Self-assessment was the only formative assessment practice where Scott achieved only high-levels of practice (Level 3), with both instances occurring during Pair/Small Group Work. Scott provided feedback to his students during all the activity structures that he implemented, however, most of these instances (N=7) occurred during Pair/Small Group activities. It is not surprising that most of Scott’s formative assessment practices were implemented during Pair/Small Group activities since this was the activity structure he implemented almost the entire time. Formative Assessment Practices Across Teachers As described above the teachers implemented varying amounts of formative assessment practices to varying levels of expertise. Table 14 shows the percentage of total activities in which each teacher implemented the formative assessment practices. For example, Charlie had 36 total activities across the six observed lessons, and 8 of those activities included learning targets, therefore, his percentage for learning targets was 22%. 108 Charlie Evan Gwen Lindsay Lukas Table 14 Teachers’ Percentage of Total Activities Including Formative Assessment Practices Formative Assessment Practice Learning Targets Questioning Self- Assessment Peer Assessment Feedback Instructional Decisions 4% Questioning was the only formative assessment practice that was used in more than 50% 12% 73% 0% 81% 0% 63% 0% 36% 6% 62% 0% 55% 13% 83% 0% 42% 0% 30% 0% 50% 14% 20% 52% 22% 81% 6% Scott 0% 8% 4% 0% 7% 8% 6% 4% 0% of each teacher’s activity structures. Three teachers (Gwen, Charlie, and Lindsay) used questioning in more than 80% of their activity structures. Feedback was also a common formative assessment practice implemented by the teachers with three of them (Gwen, Lukas, and Charlie) incorporating it into more than 50% of their activity structures. This shows the dominance of these two formative assessment practices compared to all the other formative assessment practices in the teachers’ lessons. Another aspect that was discussed above was the level of sophistication that each teacher demonstrated when implementing the different formative assessment practices. Table 15 shows a weighted average for each of the teachers across the formative assessment practices. Table 15 Teachers’ Formative Assessment Weighted Averages Formative Assessment Practice Learning Targets Questioning 0.00 2.45 2.22 1.92 2.67 1.95 2.63 2.55 Lindsay Charlie Gwen Evan Lukas Scott 1.67 1.73 1.00 2.58 109 (cont’d) 3.50 0.00 2.61 3.00 0.00 1.50 0.00 0.00 1.40 0.00 0.00 1.71 0.00 0.00 1.66 3.00 0.00 1.36 2.80 2.00 3.50 2.00 3.00 1.00 Table 15 Self- Assessment Peer Assessment Feedback Instructional Decisions The levels progressed from novice (Level 1) to expert (Level 4) with the top two levels considered higher level practices. A level of zero occurs when teachers did not implement that practice during the observed lessons. Charlie was the only teacher to show consistent levels over 2.5, indicating more sophistication in his formative assessment practice overall. However, almost every teacher had at least one practice that was greater than 2.5, therefore, everyone demonstrated some higher level practices during the observed lessons. Lindsay was the only teacher who did not have a weighted average greater than 2.5 and she did not implement three of the formative assessment practices, therefore, she likely demonstrated the least sophisticated formative assessment practices. It is interesting to note that for the three teachers (Charlie, Evan, and Lukas) who implemented self-assessment, all of them demonstrated higher level practices. This means that even though this practice was not implemented very often, these teachers were able to do so in a way that was meaningful for students. While the previous paragraph focuses on the teachers’ levels of formative assessment implementation by specific practice, it is also important to consider the teachers’ level of practice during the different activity structures. A key idea discussed in this dissertation is the relationship between the formative assessment practices and the activity structures, therefore, considering the sophistication of the teachers’ practices as a group for the different activity structures provides valuable information about this relationship. The following section provides 110 information about the levels for each formative assessment practice for each of the activity structures. Learning targets. Table 16 shows the levels of practice across the five teachers (Lindsay did not discuss learning targets with her students) for learning target use during the four activity structures in which that practice occurred. Table 16 Teachers’ Level of Sophistication for Learning Targets by Activity Structure Level 1 5 Level 2 Level 3 Level 4 Total 6 18 1 Introduction Interactive Lecture Individual Whole Class Total 1 1 0 7 0 2 1 9 6 1 1 0 8 0 0 1 2 2 4 2 26 There was a fairly even distribution of levels of practice for learning target use across the four activity structures. The exception is Level 4, which only occurred during one Introduction and one Whole Class Discussion. Most learning target use was during the Introduction activities, which included all four levels of practice. Individual activities also included a range of practice for learning targets since these activities had three of the four levels of practice coded. Interactive Lecture and Whole Class Discussion each had one low-level and one high-level practice, showing potential for these activity structures to contain high-level practice for learning target use. Questioning. Table 17 shows the levels of practice across the six teachers for questioning during the eight activity structures in which that practice occurred. Teacher Giving Information was the only activity structure that did not include questioning. 111 0 0 2 3 Level 1 Level 2 Level 3 Level 4 Total 2 4 Table 17 Teachers’ Level of Sophistication for Questioning by Activity Structure Warm-up Introduction Interactive Lecture Individual Pair/Small Group Present Student Work 1 4 Whole Class 0 Summary Total 24 19 48 3 163 14 21 1 86 3 22 2 50 5 1 7 0 0 0 0 1 1 1 0 3 2 10 22 15 44 0 6 15 11 18 18 As mentioned previously Whole Class Discussions and Pair/Small Group Work included more than half of the questioning practices. Both activity structures had all four levels of questioning practice, with most instances falling in the middle of the spectrum, being classified as either Level 2 or Level 3. Presenting Student Work was the only other activity structure that had all four levels of practice, with most of the instances (N=14) coded as Level 2. Interactive Lecture and Individual activities each had levels of practice ranging from Level 1 to Level 3. However, most of the practices for these two activity structures were coded as Level 2. These two activity structures did have teachers achieve high-level practice, though, so it is possible to include high-level questions in these activity structures. Warm-ups and Introduction activities only contained low-level questioning practices (Levels 1 and 2). The Summary activities included low (N=1) and high-level (N=2) questioning practices, with more of the instances being coded as high-level. Self-assessment. Table 18 shows the levels of practice across the three teachers (Charlie, Evan, and Scott) for self-assessment during the four activity structures in which that practice occurred. 112 Level 1 Level 2 Level 3 Level 4 Total 0 0 Table 18 Teachers’ Level of Sophistication for Self-Assessment by Activity Structure Introduction Individual Pair/Small Group Whole Class Total 0 0 0 2 1 5 2 1 6 0 0 1 0 0 0 1 2 1 1 0 1 0 0 Self-assessment was the only formative assessment practice that had only high-levels of practice for this group of teachers. Each of the four activity structures, aside from Introduction activities, provided opportunities for direct interaction between the teacher and students. All these activity structures included one Level 3 practice with Pair/Small Group Work having two instances of Level 3 practice. Individual activities, which required students to work on their own, was the only activity structure to achieve a Level 4 practice. Feedback. Table 19 shows the levels of practice across the six teachers for feedback during the eight activity structures in which that practice occurred. Summary was the only activity structure that did not include feedback. 1 7 0 0 0 1 Level 1 Level 2 Level 3 Level 4 Total 1 6 Table 19 Teachers’ Level of Sophistication for Feedback by Activity Structure Warm-up Introduction Interactive Lecture Individual Pair/Small Group Present Student Work 6 Whole Class 9 Teacher Giving Info Total 4 104 3 50 0 12 7 13 10 13 1 41 13 26 6 0 4 0 0 0 6 7 5 0 0 0 1 4 3 19 0 2 30 0 1 113 Most the feedback practices used by these teachers were low-level practices (N=91). Only three of the activity structures – Individual, Pair/Small Group, and Whole Class Discussion – accounted for the 13 instances of high-level feedback practices. Ten of those high-level feedback practices were implemented during Pair/Small Group Work or Whole Class Discussion, which means that teachers more often provided high-level feedback to groups of students as opposed to individual students. Although these two activity structures had high-levels of practice, most of the instances were Level 2 practices. Even though Individual activities included high-level feedback practices, this was a rare occurrence, as most of the instances (N=7) were Level 1. Presenting Student Work had a fairly even split between Level 1 and Level 2 feedback practice, however there was one more instance of Level 2 practice. Teacher Giving Information also had slightly more Level 2 practice than Level 1 practice, however there were only four total instances. Introduction and Interactive Lecture each had Level 1 or Level 2 feedback practice. Both activity structures had more instances of Level 1 practice. Instructional decisions. Table 20 shows the levels of practice across the six teachers for instructional decisions during the six activity structures in which that practice occurred. Table 20 Teachers’ Level of Sophistication for Instructional Decisions by Activity Structure Level 1 Level 2 Level 3 Level 4 Total 0 2 0 2 4 0 0 Introduction Interactive Lecture Individual Pair/Small Group Present Student Work 0 0 Whole Class Total 1 1 0 0 2 2 2 8 1 1 0 2 1 5 1 1 3 4 3 16 0 0 0 0 0 2 114 For this formative assessment practice, the counts for low and high-level practice was almost evenly split, however there were more low-level (N=9) than high-level practices (N=7). All the activity structures, except Pair/Small Group Work included at least one high-level practice. Interactive Lecture and Individual activities each included one instance of instructional decisions, which was a high-level practice (Level 3). Introduction, Presenting Student Work, and Whole Class Discussion each included low and high-levels of practice for instructional decisions. The counts for low and high-levels of practice were equal for Introduction and Presenting Student Work, however, more of the Whole Class Discussion instances were low-level practices for instructional decisions. After considering the counts for each of the formative assessment practices by activity structure, Table 21 shows the weighted average across all the levels of practice. Table 21 Formative Assessment Weighted Averages by Activity Structure 2.00 2.00 Learning Targets Warm-up 0.00 Introduction 2.17 Interactive Lecture Individual Pair/Small Group Present Student Work Whole Class Discussion Teacher Giving Info 0.00 Summary 0.00 0.00 0.00 3.00 Questioning 1.00 1.60 Self- assessment 0.00 3.00 Peer assessment Feedback 0.00 0.00 1.00 1.14 Instructional Decisions 0.00 3.00 1.86 2.13 2.30 2.21 2.42 0.00 3.50 3.00 0.00 3.00 0.00 0.00 0.00 0.00 0.00 1.40 1.77 2.03 1.54 1.81 3.00 3.00 1.67 2.50 2.33 0.00 2.67 0.00 0.00 0.00 0.00 1.75 0.00 0.00 0.00 For learning targets, Whole Class Discussion was the only activity structure that had a high-level of practice weighted average (3.00), however Introduction had a 2.17 weighted 115 average, therefore that is moving towards higher levels of practice. None of the activity structures for questioning had a weighted average that was considered a high-level practice. Summary, though, had the highest weighted average (2.67), which could be considered high level because it is greater than 2.5. Whole Class Discussion also had a weighted average close to this (2.42), which means that teachers’ questioning practices as a group during this activity structure were close to being high-levels of practice. All the activity structures that included self- assessment had weighted averages of at least 3.00, with Individual activities having the highest average (3.50). None of the activity structures for feedback had a weighted average that was considered a high-level practice, however, Pair/Small Group Work (2.03) had the highest average. Three of the activity structures – Introduction, Interactive Lecture, and Individual – had weighted averages demonstrating high-levels of practice (3.00 for each one) for making instructional decisions. Presenting Student Work had a weighted average of 2.50, which could be considered borderline high-level practice for instructional decisions. Looking across the nine activity structures, seven of them had at least one formative assessment practice with a weighted average classifying it as a high-level practice. This shows that almost every activity structure that teachers implement in their classrooms provide opportunities to include formative assessment practices. Warm-up and Teacher Giving Information were the only practices that did not include a high-level formative assessment practice, however, it is possible for these activity structures to include formative assessment practices depending on how they are implemented. Introduction and Individual included high- level formative assessment practices for self-assessment and instructional decisions. For Whole Class Discussion, the formative assessment practices were learning targets (3.00) and self- assessment (3.00). For Interactive Lecture and Presenting Student Work, the high-level formative 116 assessment practice was instructional decisions (3.00 and 2.50, respectively). For Pair/Small Group Work, the formative assessment practice was self-assessment (3.00). For Summary, the formative assessment practice was questioning (2.67). 117 Chapter 6: Formative Assessment Practices in Context This chapter addresses the second research question, which asked, “How do a sample of high school mathematics teachers make sense of and implement formative assessment practices, in the context of their views of their subject, their students and school, and their broader practice?”. First, I will provide background information for each teacher that describes their school context, how they used the different formative assessment practices in their classrooms, and how they define formative assessment. Next, I discuss specific activities and strategies the teachers implemented in their classrooms and whether they thought those related to formative assessment or not. Then, I will share information about how the teachers viewed the different formative assessment practices and specific examples of how teachers successfully implemented those practices during the lessons I observed. Charlie’s Experiences and Beliefs About Teaching Charlie chose one of his Algebra 2 Honors classes as his focus class for this research study. This class was approximately 60 minutes long and met every day of the week. I observed this class six times (two sets of three consecutive days). The observations occurred about a month apart. During both rounds of data collection, the class was discussing probability. The first round of data collection was the beginning of the unit, so students were learning about experimental and theoretical probability and sample spaces. The second round of data collection was at the end of the unit when students were learning about permutations and combinations and preparing for their final unit test of the school year. School context. Charlie teaches at Thompson High School, which is a suburban school with a diverse cultural and socio-economic student population. There were approximately 1,800 students enrolled in the high school and 52% of its students were on free or reduced lunch. 118 Teacher background. Charlie earned his Bachelor’s and Master’s degrees from a large state university. He taught at Thompson for about five years before leaving his classroom to become an instructional coach. After working with other teachers to improve their classroom practices for about seven years, he decided to return to the classroom. Teaching assignment and curriculum. At the time of this study, Charlie had been back in his own classroom for about five years and was teaching Algebra 2 Honors and Algebra 1. He had been teaching Algebra 1 for his entire teaching career but had only been teaching Algebra 2 Honors for about three years. His school did not have a textbook series for their curricula, instead they had developed a teacher-created curriculum over the years and continued to use that with modifications over time to align with state standards and restructure courses as needed. Classroom context. Charlie’s classroom consisted of desks and large tables. The desks were divided in half and faced each other whereas the tables were in the back of the classroom. Students sat at the desks, except during small group work time, students could move to the tables to work together. Definition of formative assessment. Charlie sees formative assessment as a process and not as one particular event. He stated, “it’s a way of thinking about how you get information from kids and how kids get information from you. And the idea is you want to make decisions based on what kids do to make your instruction better and most importantly we want them to change what they’re doing to get better” (BI, lines 748-751). He also sees the students having an important role in the formative assessment process, which has been a “concentrated goal” of his since he came back to his classroom (BI, line 138). Students in his classroom are held accountable for monitoring their own learning and coming to him when they realize they need extra support. 119 Professional development opportunities. Charlie had extensive professional development related to formative assessment. He was a teacher leader for a research study about formative assessment and at the time of the study was leading professional learning communities at his school about formative assessment. He has also given presentations at conferences related to formative assessment to share his experiences with other teachers and mathematics educators. Evan’s Experiences and Beliefs About Teaching Evan chose his Algebra 1 class as his focus class for this research study. His school used a modified block scheduling, so he saw all his classes for 45 minutes on Mondays, and then saw the even class periods Wednesdays and Fridays and the odd class periods on Tuesdays and Thursdays with each class period lasting 90 minutes. The observations occurred Mondays, Tuesdays, and Thursdays (three consecutive class sessions) at two different points in the semester about one month apart. During the first round of data collection, the students were learning about quadratic expressions and equations. During the second round of data collection, the students were working on data analysis. School context. Evan teaches at Sanderson High School, which is an urban, low socio- economic high school with about 500 students enrolled and 88% of their students receiving free or reduced lunches. Even though Sanderson is small, it is culturally diverse with students from over twenty countries. Teacher background. Evan earned a Bachelor’s degree from a large state university. He taught for five years at a different high school before coming to Sanderson, making this his first year at his current school. Teaching assignment and curriculum. At the time of the study, Evan was teaching Geometry, Algebra 1, Algebra 2A, and Algebra 2B. The Algebra 2A and B courses represent 120 Algebra 2 taught over two years, so Algebra 2A would be the first semester of Algebra 2 and 2B is the second semester. Evan taught Geometry and Algebra 2B at his previous placement, however this year was his first time teaching Algebra 1 and Algebra 2A. All his classes used a discovery-focused curriculum. There were also additional hard copy resources that could be purchased to accompany the textbook, including a skills practice that was used by all the mathematics teachers for homework problems. The textbook also included an online component that they were supposed to be using, but the teachers had stopped using it because students were not getting anything out of it. Students realized if they did not know how to work a problem, then they could just enter random numbers as quickly as possible to use up the question bank and the program moved them on without them getting correct solutions. The teachers realized that students “weren’t demonstrating learning, so we’ve reduced the amount of time they’re supposed to spend on that quite significantly” (BI, lines 80-81). To find more reliable material, the teachers searched online to find other resources to supplement their curriculum, as well as finding another adaptive computer program to pilot in hopes of replacing their current curriculum-aligned program. Classroom context. Evan had his students sitting in groups of four with the expectation that they would work together during class on the lessons or homework. This happened to varying degrees based on the mix of students who were sitting together. Definition of formative assessment. Evan sees formative assessment as “the constant assessment you’re doing of students, um, tracking of the progress they’re making um to gauge where you need to be” (BI, lines 469 – 471). For Evan, he considered observing students and having discussions with them as the main ways he included formative assessment into his classroom. 121 Professional development opportunities. Evan had not attended any professional development sessions focused on formative assessment, however he did think he had attended short one hour workshops related to assessment, but nothing stood out in his memory. He had attended PD related to planning mathematics lessons, which incorporated anticipating students’ strategies and misconceptions and planning questions to ask students. At the time of the study, Evan, along with the other math teachers at his school, were enrolled in an online blended learning course. In addition, the mathematics teachers were required to meet twice a week during their common planning time to discuss any necessary information, such as determining unit goals or common assessments. Gwen’s Experiences and Beliefs About Teaching Gwen chose one of her Algebra 2 classes as her focus class for this research study. She is at the same school as Evan, so she sees all her classes for 45 minutes on Mondays, and then sees the even class periods Wednesdays and Fridays and the odd class periods on Tuesdays and Thursdays with each class period lasting 90 minutes. The observations occurred Monday and Wednesday for the first round of data collection (a third day could not be included due to scheduling conflicts) and Monday, Wednesday, and Friday (three consecutive class sessions) for the second round of data collection. The observations occurred about one month apart. During the first round of data collection, the students were working on graphing rational functions. During the second round of data collection, the students were learning how to simplify and solve radical equations. School context. Gwen teaches at Sanderson High School with Evan, where their mathematics department consists of four teachers. The teachers at Evan and Gwen’s school are required to observe each other once a week. Since all the mathematics teachers have the same 122 planning period, unfortunately they are unable to observe each other, so they visit other content teachers. They complete an online feedback form for the teacher they are observing, but they do not have conversations with the other teacher in person about what they observed or the feedback they provided. This is the same evaluation form that the principal uses when he observes the teachers, however the peer evaluations do not count for anything in terms of the teachers’ overall evaluation. Gwen likes this practice because she feels like she learns something from every teacher that she observes. She is also very comfortable having others come into her classroom to observe her. Teacher background. She graduated with a Bachelor’s degree in engineering and came to teaching as a second career. While she was substitute teaching, she was also working on a Master’s degree in the field of education. She then spent about seven years teaching at a community college, before moving to a different state. She then taught at a middle school for a year, before moving to yet another state. When she first moved to Michigan, she taught at a community college for a few years before coming to Sanderson to teach high school for the first time, therefore, her participation in this study was during her first year at Sanderson. Teaching assignment and curriculum. At the time of the study, Gwen was teaching Algebra II and Trigonometry, teaching both subjects for the first time. The two classes used different curricula series, with the Algebra II textbook more discovery-focused and the Trigonometry textbook more traditional. The Algebra II textbook included an online component that they were supposed to be using, but the teachers had stopped using it because it did not align well with the textbook. The online portion was also particular about how students entered their responses, making it challenging for students to use, “they’re just guessing to try and get through it. They’re not actually putting forth the effort” (BI, lines 86-87). Gwen and her colleagues had 123 turned to other resources for additional support, finding worksheets and activities online to supplement their curriculum. Gwen was also an active user of Google classroom, providing her students a place to view her worked out homework solutions and videos of her working out problems or notes from the class period. The textbook for the Trigonometry class was a concern for Gwen because the students spent three years working out of exploratory-focused textbooks, and then switched to a traditional textbook. It had been challenging for her students to use this textbook because it was very different from the textbook series they had used in their previous mathematics courses. Classroom context. When I observed Gwen, she had her students sitting in rows, but mentioned that she had had them in groups earlier in the year. She stated that she liked to change up the seating arrangement. Definition of formative assessment. Gwen thinks formative assessment is “constantly checking in just to make sure that what I taught they’re actually grasping” (BI, lines 670-671). She thinks of summative assessment as being at the end of instruction and formative assessment is the during part that leads up to the summative assessment. Professional development opportunities. Similar to Evan, Gwen had not had professional development specifically related to assessment. She was part of a professional learning community with teachers from other content areas that met six times this school year with each lasting a full day. These workshops covered multiple topics including building a professional learning community, questioning, and looking at student data. As previously mentioned, Evan, Gwen, and the other two mathematics teachers met twice a week as a department to discuss what was happening in their classrooms and whether they needed to bring in the curriculum director or principal to discuss any upcoming issues. They also used this time 124 to learn from each other. For example, if one of them attended a conference or workshop, then that person shared what they learned with the rest of the group. Lindsay’s Experiences and Beliefs About Teaching Lindsay chose one of her Algebra 2 classes as her focus class for this research study. This class met every day for approximately 50 minutes. I observed this class six times (two rounds of three consecutive days each) with observations occurring about six weeks apart. During the first round of data collection, the students were working on graphing polynomial functions. For the second round of data collection, students were learning about data and statistics, discussing such topics as measure of center and spread. School context. Lindsay teaches at Alexander High School, which is a rural school with approximately 900 students enrolled in the high school and about 70% of its students are on free or reduced lunches. At Alexander, all the mathematics teachers have students working in groups and assign group roles to their students, creating a classroom culture of what it means to do work in mathematics classrooms at their school. Teacher background. Lindsay did not start out as a mathematics teacher. Instead she holds an MBA and had worked in banking for many years before coming to teaching after deciding she needed a change. Lindsay had been teaching for 16 years, though, so she is not new to the profession. Teaching assignment and curriculum. At the time of the study, Lindsay was teaching Algebra 2, Geometry, AP Statistics, and Math Lab. She had taught Algebra 2 and Geometry for her entire teaching career and had significant experience teaching AP Statistics, having done it for twelve years. Her Algebra 2 and Geometry classes used a student-centered, exploratory curriculum, which had been implemented for the last four years. They originally had a more 125 traditional textbook series but switched because the current curriculum was not meeting the needs of their students or the expectations of the teachers. The teachers started supplementing the curriculum, which “can be very dangerous” (BI, line 92) because it created gaps in students’ knowledge. For the AP Statistics, Lindsay mostly used a teacher-created curriculum because the traditional textbook had not been very helpful. She mostly built from the College Board website and the expectations of the AP exam. In the future, she will be working with a textbook series to develop a more student-centered curriculum for the AP Statistics course and is very excited about the opportunity. Classroom context. In Lindsay’s classroom, the students sat in groups of four with individual desks pushed together. When students were absent and groups had fewer students, Lindsay moved these students to other groups so they would have more students to work with during class. She thought it was important for students to share their ideas with others, so she wanted the groups to be as evenly balanced as possible. While she thought this was important, she was concerned that students got overconfident in their mathematical understanding based on group think. This made it important for her to check in with individual students while they are working in groups to ensure each person had a clear understanding of the content. She also used a random name generator to call on students, so they all were held accountable for the work their group completed. She thought this was helpful because “it tells [students] right away whether it’s the group that gets it or them” (BI, line 253). This also gave her valuable information about the students’ understanding. The teachers at Lindsay’s school also used common summative assessments. Definition of formative assessment. Lindsay thinks formative assessment is “not graded” and “somebody does something with the information” (BI, line 467), with the somebody 126 being herself or her students. Lindsay would like her students to take more of the lead in using the information to improve their learning, but that is not always the case. She also thinks that part of formative assessment is helping students move to the next step in their learning. Lindsay originally thought that formative assessment was about exit slips and pre-tests, but after hearing someone speak, she realized that questioning was also a formative assessment practice. She thought formative assessment had to be “something concrete”, but this presentation broadened her ideas of what formative assessment was. Professional development opportunities. Lindsay has had many opportunities for professional development, although not much of it was focused on assessment. At the time of the study, she was part of a coherence team that had been working to brainstorm ways to improve their county’s program K-12. This groups met a couple of times a year and discussed many educational issues, such as assessment and the role of their curriculum and whether it was serving the expected purpose. She was also part of a school specific coherence team that had met once this year. Lindsay had even given presentations at national conferences related to the curriculum she was using. Lindsay, Scott, and the rest of the Algebra 1 teachers at her school were also participating in a lesson study experience where they planned a lesson together and then observed one of them (not Lindsay or Scott) teaching the lesson. Scott’s Experiences and Beliefs About Teaching Scott chose one of his Geometry classes as his focus class for this research study. This class met every day for approximately 50 minutes. I observed this class five times (two consecutive days for round one and three consecutive days for round two) with observations occurring about six weeks apart. During the first round of data collection, the students were working with three-dimensional shapes, including finding the surface area and volume of prisms. 127 For the second round of data collection, two different topics were discussed. The first day was the final lesson about probability, then the next two days the students were building on their prior knowledge to talk about a different kind of three-dimensional shape (i.e. polyhedron). School context. Scott taught with Lindsay at Alexander High School. Teacher background. He earned his Bachelor’s and Master’s degrees from two different large state universities. He had been teaching for eight years, all of them at Alexander. Teaching assignment and curriculum. At the time of the study, Scott was teaching Geometry, Algebra 1, and Algebra 2. All his classes used a student-centered, exploratory curriculum, which had been implemented for the last four years. They originally had a more traditional textbook series but switched because the current curriculum was not meeting the needs of their students or the expectations of the teachers. Their test scores were low and they needed to find a curriculum that would help “to provide instruction that helped retention” (BI, line 32). Using the old, traditional curriculum, Scott found that “teaching at that point wasn’t fun” because it mostly involved him lecturing to students (BI, line 48). When they switched to the new, student-center curriculum, though, Scott found the change invigorating, saying, “I just kind of re-energized myself” (BI, lines 50-51) because he could interact with his students, leaving the lecturing behind. Classroom context. In his classroom, Scott had students sitting in groups of four. He changed their groups and roles at the end of every chapter, so every student could work with all the other students and hold each of the four group roles throughout the school year. The group roles were suggested in the new curriculum, which had been an important change for Scott 128 because it provided structure for having students work in teams, which had been missing for him in the past. Definition of formative assessment. Scott’s view of formative assessment has changed over the course of his teaching career. When he first learned about formative assessment, he thought of it as “an exit slip type of thing where [I] give a problem and I take a look at it and maybe provide some feedback” (BI, lines 390-391). The student-centered curriculum his school used has changed his thinking about formative assessment. He now thinks he gets “more feedback from students through observations, through the interactions, through the talking, it’s immediate” (BI, lines 393-394). Scott no longer thinks exit slips are as valuable because the feedback to students is delayed. Scott also discussed how as an athletic coach, he uses formative assessment by giving his players immediate feedback while he is observing practice or during games. He believes this immediate feedback “kind of sticks for the students” because they receive direction on their performance right away. Scott said that he had “developed [his] interpretation of formative assessment through practice through the actual teaching part and coaching” (BI, lines 415-417). Professional development opportunities. Scott has had some professional development opportunities related specifically to assessment. His whole school was part of a book study where they read a book by Rick Stiggins and then participated in a webinar led by the book’s author. Along with Lindsay, Scott has participated in a lesson study with the other Algebra 1 teachers, although he has not had the opportunity to teach the lesson. This is the first year the teachers have tried this and Scott sees great value in doing this saying, “it gives an opportunity to open your doors and other people come in, it’s kind of neat” (BI, line 482), which Scott thought 129 helped make teachers feel less isolated. The mathematics department at Alexander also used common assessments, so the teachers get together to create them. Lukas’s Experiences and Beliefs About Teaching Lukas chose his Pre-Calculus/Coordinate Geometry class as his focus class for this research study. This class met every day for approximately 70 minutes. I observed this class five times (three consecutive days for round one and two non-consecutive days for round two) with observations occurring about four weeks apart. During the first round of data collection, the students were working with parabolas and circles, including writing them in standard form and graphing. For the second round of data collection, two different topics were discussed. The first day was the unit test review about all the conic sections (parabolas, circles, ellipses and hyperbolas), then the following day was their unit test (I did not observe on the test day.). On the last observation day, Lukas introduced the last unit of the school year, which was matrices. School context. Lukas teaches at Newson High School, which is a rural high school with about 1,000 students enrolled in the high school and about 74% of its students on free or reduced lunches. Teacher background. Lukas earned his Bachelor’s and Master’s degrees from the same large state university. He had been teaching for 15 years, however the first six years were at Alexander High School (the school where Lindsay and Scott taught). Teaching assignment and curriculum. At the time of the study, Lukas was teaching Geometry, Pre-Calculus, and a non-mathematics course related to his minor. He had been teaching Geometry for almost his entire teaching career (thirteen years), but only recently started teaching the Pre-Calculus course (five years). When Lukas started teaching at Newsom, they were using a teacher-created curriculum for all their mathematics classes. However, three years 130 ago, the administration insisted that they choose a textbook series and no longer use their teacher-created materials. The Geometry classes had been using the same student-centered, exploratory curriculum that is now used at Alexander High School. Lukas was happy with the change to this new curriculum, because he and the other mathematics teachers “felt like it accomplished a lot of things we as a department valued philosophically, but executed in many ways not perfectly, but executed it in many ways better than what we felt” (BI, lines 47-49). The teachers wanted a curriculum that provided students with opportunities to discover the mathematics, collaborate with each other, and make connections to their knowledge and believed this new curriculum met these requirements. The Pre-Calculus class, which is an elective course and not the typical college-bound Pre- Calculus course, still used a teacher-created curriculum that had been modified over the years to align to standards and re-structured content as needed. Lukas had also adapted it during the time he had taught the course to make it more his own. The course was for students who needed another mathematics course to ensure they had four years of mathematics before they graduated high school. The teachers wanted to offer a course for students who would not necessarily be taking Calculus but would still offer those students a meaningful mathematical learning opportunity. Lukas explained that the content of the course included topics that did not fit nicely into the other mathematics courses, but still had some importance for students’ learning them. This course also helped prepare students for enrolling in college mathematics after graduation. Classroom context. In his classroom, Lukas had tables, so students sat in groups of four or five, with the expectation that students would work together. Like Scott, Lukas attributed the 131 improvement in group work to the curriculum, saying it “helped provide some structure for making them work together more” (BI, lines 64-65). Definition of formative assessment. Lukas thinks formative assessment “is anything that you use to help inform your instruction” (BI, line 640). This could be information gathered about whether students are making sense of the content or not, which helped him decide if the students had the necessary knowledge to be successful on the next topic. He used conversations with students or looking at their written work as feedback to himself of students’ readiness for continued learning. Lukas also thought an important factor for formative assessment was that “it cannot be something you’re giving them a summative grade on” (BI, lines 648-649). Lukas thought many teachers used formative assessment without realizing it because teachers are routinely “reading your students, you’re reading their expressions or their comments, their questions” (BI, lines 665-666) and using that information to make decisions about their instruction. Professional development opportunities. Lukas has had multiple opportunities for professional development related to assessment. At his previous school, he heard a presentation about standards-based grading that also related to formative assessment, which also came with a book about classroom assessment and student learning. Lukas was also part of a school improvement team at his current school focused on standards-based grading and formative assessment. As part of this team, he attended a few meetings focused on formative assessment that was part of a research project at a large university. Unfortunately, the focus of the school improvement team changed, so they are no longer concentrating on formative assessment. Because of this shift, Lukas has taken the initiative to find his own avenues for developing his knowledge about formative assessment, such as attending state mathematics conferences. 132 Teachers’ Descriptions of Activities and Strategies Related to Formative Assessment During the exit interviews, I asked teachers about specific activities and concepts I saw in their lessons to see if they thought those were related to formative assessment. These were not necessarily activities or concepts I thought related to formative assessment; instead these were activities that were either consistent across a teacher’s lessons or something that stood out as different. This section describes these specific activities and concepts for all the teachers. The first section describes specific activity structures defined previously that the teachers saw as relating to formative assessment. The second section discusses in-class activities that the teachers implemented and whether the teachers saw these as being related to formative assessment. The third section explains different types of non-summative assessments the teachers gave in their classrooms and whether they saw those as relating to formative assessment. The fourth section describes instructional strategies the teachers implemented and whether they think those relate to formative assessment. Activity Structures The teachers talked about four activity structures (Warm-up, Introduction, Pair/Small Group Work, and Presenting student work) that they saw as being connected to formative assessment. This section details those four activity structures and how the teachers saw or did not see them as being related to formative assessment. Warm-up. Charlie, Lukas, Evan, and Gwen all implemented at least one warm-up activity during the observed lessons. When I asked the teachers if they saw this activity structure as being related to formative assessment, they had mixed responses. Lukas and Gwen both stated that they thought warm-ups were related to formative assessment. Lukas stated, “That is intended for me to get some feedback of how are you guys doing from what we did yesterday, cause if 133 you don’t remember anything we did yesterday, or this is telling me you didn’t understand what we did yesterday, it tells me okay, we’ll change what we’re doing” (Exit Interview, lines 215- 217). Therefore, he used the information he gathered from the warm-up as feedback to make instructional decisions. Lukas also saw the warm-ups as a self-assessment opportunity for students, stating, “It’s meant to be a how am I doing, you know? Can I do these problems?” (Exit Interview, lines 220-222). Gwen thought warm-ups related to formative assessment because of one specific activity that she has used in the past that required students to analyze another student’s work. She saw this as being related to formative assessment because the student would have to know how to solve the problem in order to make sense of their peer’s work. Charlie and Evan did not always think that warm-ups related to formative assessment. Charlie thought that some of the warm-ups he implemented did not relate to formative assessment because of the kinds of problems he used. On one day, he thought the problems were a “cheat” for students who had not done the homework, therefore, he did not think this related to formative assessment. However, when he has students work alone and he can monitor what they are doing, then those do relate to formative assessment. He stated, “I get to circulate and see what they’re writing. So, I get an idea of what types of things maybe they’re doing or they’re not doing” (Exit Interview, 79-80). Evan questioned whether the way he used the warm-ups allowed him to implement formative assessment, stating, “the way that I use it probably not that much because I’m not necessarily looking at what students are doing and making a whole lot of adjustments” (Exit Interview, lines 17-18). He did state, though, that he goes over the warm-up with his students, so they get feedback on how they solved the problems. Like Lukas, he also thought students could use the warm-up as a way to self-assess leading into the new lesson. 134 Introduction. Charlie and Evan both started most of their class sessions with what they called an agenda. I classified these activities as the Introduction to class or lesson because the teachers used their agenda to set the stage for that class session. Both teachers thought this activity structure provided them with an opportunity to talk about the learning goals with their students. For Charlie, the agenda was a way to provide his students with some structure, stating, “But I want there to be a routine of I come in today, I figure out which goals I’m working on, I can link this goal to this task, and I know what it is I’m gonna do in terms of the outcomes for the day” (Exit Interview, lines 189-191). Evan thought the goals were an important part of the agenda because those were what he was assessing students on and what they were assessing themselves on. It is interesting that the learning goals were the focus of this activity structure for these two teachers because for the most part, this was the only activity structure that included learning goals. Presenting student work. Lukas and Gwen both had their students working on the whiteboards in their classrooms quite often. For Lukas, it was having students present their work to the whole class, whereas for Gwen, it was students working in pairs or trios to practice their mathematical skills. However, both teachers saw this activity structure as being related to formative assessment. Lukas thought this related “because it gives them a chance to either show or explain their thinking to others” (Exit Interview, line 229). He also thought having his students analyze their peers’ work was an important component of having students share their work with the class because it allowed the students to make sense of someone else’s thinking. This activity structure is helpful to Lukas, too, because it gave him information about who could understand their peers’ thinking and whether a student could find his/her own mistake in their work. Like Lukas, Gwen saw the value in students being able to explain their thinking to their 135 peers or teach their peer how to solve the problem was an important aspect of having students work together to solve problems on the boards. Pair/small group work. I asked all the teachers, except Scott about grouping structures during the exit interview and their connection to formative assessment. The teachers’ views about the role students working together played in relation to formative assessment was mixed. Lukas, Gwen, and Lindsay all thought having students worked together related to formative assessment. For Lukas, hearing students talk in their groups provided him with information about the students’ understanding. Having students work in groups also made it easier for him because he did not have to check in with individual students, saying, “it can make the assessment piece happen faster because they’re collectively stuck and not just a bunch of individuals where I would have to check on each individual” (Exit Interview, lines 202-204). Gwen thought having students work together was a great tool for determining if students could explain their thinking to another person and have that person understand the explanation. Lindsay thought that the teamwork she used in her classroom was a great way for students to “bounce ideas off each other” (Exit Interview, line 63). She also thought that working with others was a great way for students to assess themselves. Charlie had mixed feelings about the role grouping structures played in formative assessment. He thought the mix of students in the group impacted the kind of information he could gather from them. When students are working together, he is more focused on the questions they ask him, than what they have written on their paper “because I feel like that gives me better idea of where the kids are at” (Exit Interview, line 233). Evan was the only teacher who did not think the grouping structures related to formative assessment. He said that he saw 136 that as an opportunity for them to work together and nothing more. However, he did say if he was interacting with the group, then he would consider it relating to formative assessment. For Lindsay and Scott, group roles were an important aspect of students working in teams. Lindsay thought the group roles were breaking down by the time I observed but stated that earlier in the year she had specific questions for each role to ask the group at different times during the class period to help them stay focused and hold everyone accountable. Scott did not think the group roles related to formative assessment; instead he saw it as a structure that was “hopefully engraining a mindset that uh, that you have to, they have to stick together, they have to work together, they have to problem solve together” (Exit Interview, lines 124-125). Interestingly, Scott also stated that he thought the group roles were breaking down by this time of the school year. Summary. The teachers who talked about these activity structures during their exit interviews all thought these structures related to formative assessment (except for Evan and grouping structures). Based on the wording of the questions I asked teachers, it is impossible to determine whether they believed these activity structures themselves were formative assessment, however they did see value in implementing them in their classrooms. Based on my definition of formative assessment, these activity structures would not be formative assessment practices, however, they do provide structure for teachers to implement the different practices as discussed in the previous chapter. In-Class Activities This section focuses on in-class activities I observed in the teachers’ classrooms and whether the teacher thought they related to formative assessment or not. Most of these activities 137 occurred during only one of the observed lessons, which made me wonder what made this activity special, so I asked the teachers about it. Goal check. Charlie was the only teacher who implemented this activity. For this activity, he listed the three goals on the board and students had to write what each goal was and how they could be successful at that goal. He thought it related to formative assessment because “It felt like it was, how well do you understand the goals that we have? Do you know what it is you’re supposed to know? And then also the part of like, where are you?” (Exit Interview, lines 389-390). As part of this activity, Charlie had students working together. If students had questions, then they wrote those on the board to discuss class. If multiple students had the same question, then they put a star next to the original question. Charlie thought aspect of the activity also related to formative assessment because it allowed him to see what students’ questions were. Having students put the stars next to the questions was especially helpful because it showed him where most of the students were struggling. Mini whiteboards. Charlie, Lukas, and Lindsay all implemented activities in their classrooms involving mini whiteboards. In Charlie and Lindsay’s classes, each student had their own whiteboard. In Lukas’s class, the students worked in pairs or trios sharing one whiteboard. All the teachers see these mini whiteboard activities as being related to formative assessment. Charlie and Lukas both talked about how it provided them an opportunity to give quick and immediate feedback to students. Lukas saw the immediacy as important, saying, “I find out right away how the kids are doing and they get to find out right away how they’re doing” (Exit Interview, lines 382-383). Charlie also saw it as a way for students to get feedback on their own because if many students have a different answer displayed than you do, then you know that there is likely something wrong with your answer. Lindsay thought her students were more 138 willing to take risks using the mini whiteboards than if they were using paper and pencil. She especially thinks the mini whiteboards are useful when students are first learning the content because “they don’t have any problem writing it down on the whiteboard because they can get rid of the evidence really quickly” (Exit Interview, lines 111-113). Silent squares activity. Lindsay was the only teacher who implemented this activity. Students worked together to create four squares of different sizes, however, they were unable to talk to their teammates, so they had to communicate in different ways. Students could not take pieces from their teammates, instead, they had to offer pieces they thought would help their teammates complete their squares. Lindsay thought this activity was a way for them to “formatively assess their teamwork” (Exit Interview, lines 20-21). She thought it would help them determine “how a team functions together and for them to kind of reflect on their own participation in a team” (Exit Interview, lines 23-24). Based on her explanation, it sounds like Lindsay does not see this activity as relating to formative assessment in terms of content or student thinking, but in relation to their peer interactions. Jigsaw activity. Another activity Lindsay implemented was a jigsaw activity. She had students number off in their teams and then each person was assigned a vocabulary word. The students met in their expert groups to learn about their vocabular word, so they could then teach their teammates. The students also created posters in their expert groups as a resource. Lindsay saw this activity as being “a good formative assessment” because the students realized how well they understood their vocabulary word they were an expert on when they go back to their original groups. The students had to be able to explain the vocabulary word to their teammates and answer any questions they had. Lindsay stated, “they’re each responsible for learning the material from each other” (Exit Interview, line 96). 139 Desmos. Lindsay and Evan both talked about using Desmos with their students. Lindsay took her students to the computer lab to work on the activities, whereas, Evan shared a mobile computer lab with another teacher, so his students worked in the classroom on laptops. I did not observe Lindsay using Demos with her students, but she talked about using an activity about inequalities with her Algebra 1 students and called the Desmos activities “wonderful formative” (Exit Interview, line 230), which leads me to believe she thinks the activities themselves are formative assessment. Evan also used Desmos with his Algebra 1 students. When asked if he saw this program as a tool he could use for formative assessment, he said yes. He continued saying, “You can track what the students have done the whole time and I think address certain issues that come up. Um, but it’s nice cause you can see the progress they’re making through it. It’s not just a final” (Exit Interview, lines 96-98). Both teachers saw great value in using Desmos to promote formative assessment in their classrooms. Team strategies. Lindsay and Scott were the only ones who implemented these activities in their classrooms, which is not surprising since they were recommended in their textbook. There were many different activities included in the team strategies, however, Lindsay specifically talked about ambassadors and huddle and Scott spoke about them more broadly. Both teachers, though, thought these types of activities related to formative assessment. Ambassadors is when Lindsay would send a student to another group to help them if she is not able to get around to every group. She believed this related to formative assessment because it gave students a good idea of whether they understood the content or not. This seems to relate to self-assessment, since it tells the student information about their own understanding. Huddle has two different versions. One is when Lindsay would call multiple group of students together to explain something to them all at the same time. Another way a huddle could be implemented was 140 demonstrated by Scott. He called a single person from each group together and talked with them about a problem and sent them back to their teammates to share the explanation with them. Lindsay thought that huddles were related to formative assessment because, “they self-evaluate and decide do I know this enough to leave or should I stay and ask another question because I have to go back and answer everybody else’s questions” (Exit Interview, lines 146-147). Scott thought the teaching strategies related to formative assessment because they often got more students involved in the conversation than a having a whole class discussion. He can listen to what students are sharing with their partners to see how students are making sense of the concepts, saying, “hopefully that, that one-on-one setting is easier to pull more out, more information out of them for formative assessment”. (Exit Interview, lines 108-109). He also thought these activities not only provide him with information about students’ thinking, but also gives the students information because they had to pay attention to what their partner is saying. Summary. The teachers implemented many activities during their lessons, so this is just a sample of what was observed. However, the teachers saw these activities as opportunities to implement formative assessment in their classrooms. Some of the teachers saw the activities themselves as being formative assessment, which differs from how I define formative assessment. None of the activities themselves were coded as formative assessment, but many of them included formative assessment practices. These are just a few examples of examples that teachers could use to include formative assessment practices in their classrooms. Assessments Some of the teachers assessed their students on a regular basis either using assessments such as learning checks (Charlie) or homework quizzes (Evan). This section discusses how the 141 teachers saw these different assessments as being related to or did not relate to formative assessment. Learning checks. Charlie was the only teacher who assessed his students using learning checks, which he said were “based on formative assessment” (Exit Interview, line 118). These are similar to quizzes for Charlie’s students because they are intended to assess where students are in their learning. He gives students feedback on these “because I use them to adjust my instruction, because I hope they use them to adjust their instruction” (Exit Interview, lines 121- 122). Charlie thought these learning checks were valuable not just for himself, but also for his students, saying, “they tend to give me a good idea of where they’re at, they give them a good idea where they’re at, and I can assess earlier than I would with a quiz” (Exit Interview, lines 124-125). Charlie saw the learning checks as different from quizzes in that the learning checks were not summative grades like quizzes are. He frequently used learning checks to assess students’ understanding, however, the scores for these were flexible based on how students performed throughout the unit. Charlie used standards-based grading, therefore, students could improve their score by retaking the learning checks. During his Algebra 1 class, Charlie had students complete a practice learning check. He thought this also related to formative assessment and served as a self-assessment for students. He saw it as more beneficial for the students than himself because it gave the students information about where they were in their thinking and what they needed to work on before the real learning check the following day. Homework quizzes. Evan and Gwen administered homework quizzes to their students. For Evan, this was an almost every day occurrence. Evan saw the quizzes as being formative because “it’s my check to see the progress students are making through the course of a unit” 142 (Exit Interview, lines 76-77). He also wrote students’ scores on their learning scales “so they have that to monitor and track as well” (Exit Interview, line 78). Based on this explanation, Evan saw the homework quizzes as beneficial to himself and his students for different reasons, but the focus was on students’ progress. Even though Gwen did not use the quizzes as often as Evan, she still saw them as relating to formative assessment because, “I can see what they understand and what they don’t understand. I think it’s a good, good way for me to, for me to kind of do that spot check. Do you understand this concept?” (Exit Interview, lines 151-153). However, Gwen did express concern about the authenticity of the quizzes since she allowed students to use their homework to complete the quizzes. She said she did this because so few of them did the homework, it was not much of an advantage. She worried, though, that students had copied their homework from the online solutions she posted, so their quiz would just be copied from her work and not demonstrate the students’ actual understanding. If students copied everything, then she did not think the quizzes related to formative assessment. Pre-assessment. Before the start of every chapter, Gwen had her students complete a pre-assessment. Gwen stated that the teachers realized these pre-assessments were not formative because the students did not put any effort into completing them. She does believe, though, that these pre-assessments have potential to be formative saying, “I think they could be great if they actually gave effort” (Exit Interview, line 210). Gwen stated that the pre-assessments could provide her with information about what students already know and with which concepts she should spend more time on during the unit, but that is not possible with students’ current strategy for completing these. Homework. Charlie, Lukas, Gwen, and Evan all talked about giving their students homework, however, they had mixed views on whether it was related to formative assessment or 143 not. Lukas and Gwen thought homework was related to formative assessment. At the start of the class, Lukas walked around to look at every student’s homework so he could mark the level of completeness because “it’s practice and that’s why I grade it the way I do” (Exit Interview, lines 186-187). This allowed him to see whether students just had solutions or if they had left problems blank. For the problems that were blank, Lukas asked the students if they were blank because they did not know how to do the problem or because they did not get to it. He has found that students are honest with him, which is helpful because it gave him better information about the students’ understanding of the content. Students were able to submit homework late for full credit, so there was no reason for students not to be honest with Lukas. When multiple students had questions on the same problem, Lukas stated, “that’s my formative assessment right there” (Exit Interview, line 155). Gwen did not collect students’ homework, so she saw it as more of a self-assessment for students because she posted the solutions on her class website. Students were responsible for checking their homework themselves. For Charlie, it depended on the purpose of the homework or how he planned to use it. For example, his students had a project to complete that counted as a take home test grade and Charlie thought this related to formative assessment because “they were able to like, assess what they knew about combinations and permutations and then use that on other things and decide to ask questions and things like that” (Exit Interview, lines 300-302). Charlie stated that he did not think he always used homework in a way that supported formative assessment, since he did not always collect it. He thought the reviews worked well, but for other assignments, students did not see the value in completing them if they were just for practice. Charlie said that sometimes he used homework as a learning check and let students know ahead of time that he would be collecting it the next day to see where students are in their learning. Part of the reason Charlie 144 did not see homework as relating to formative assessment is because “I don’t feel like I get a ton of information partly because of how many kids complete it” (Exit Interview, line 317). He did find that if the homework counted as a learning check grade, then more students completed it. Evan did not see homework as relating to formative assessment. He did not collect homework and like Gwen, he gave the students the answers ahead of time. He saw homework as a tool for students to practice their mathematical skills, however, he did not expect them to complete the entire assignment. He thought students should work enough problems that they feel confident in their understanding of the material. Based on Evan’s explanation, it seems like since he did not use the homework for anything that is why he did not see it as relating to formative assessment. Summary. The teachers used multiple tools for assessing students’ thinking with many of them being relating to formative assessment. It is interesting to see how the teachers viewed the different assessments that multiple ones of them implemented. Gwen and Evan were at the same school and they both used homework quizzes, however, they used them in different ways, which impacted whether they saw quizzes as relating to formative assessment. Even though four teachers talked about homework, they saw the connection to formative assessment in very different ways. It all depends on how the teachers used the different assessments in their classrooms. Based on my definition of formative assessment, none of these assessments would be formative assessment, however, some of them could include formative assessment practices depending on how they were implemented. 145 Instructional Strategies This section discusses instructional strategies, such as activating students’ prior knowledge or looking for multiple solution strategies, the teachers implemented and whether they saw them as relating to formative assessment. Activating prior knowledge. Charlie, Lukas, and Gwen all mentioned the importance of activating students’ prior knowledge in connection to supporting students’ learning. Based on a professional development opportunity, Charlie thought activating students’ prior knowledge was related to formative assessment because, “we’re trying to get them to understand what they know already and for them to kind of reposition their understanding in the course” (Exit Interview, lines 107-108). He stated that he often used the warm-ups to access students’ previous knowledge. Gwen also saw connecting to students’ prior knowledge as relating to formative assessment because, “They need to be able to access it. It’s there. They need to realize that they’re building blocks” (Exit Interview, lines 362-363). Gwen thought this was important as a way for her to determine where students had gaps in their understanding, so she could help them be successful. She also made an explicit connection between students’ prior knowledge and formative assessment saying, “that’s the whole thing with the whole formative assessment. You’re seeing what they know and you’re building upon it” (Exit Interview, lines 373-374). Lukas had mixed views on the relationship between connecting to students’ prior knowledge and formative assessment. First, he stated that he thought the two did relate because “it gives me feedback” (Exit Interview, lines 422-423). He stated that if he makes a connection for students but notices they are not seeing the connection, then he needs to change what he is doing and provide them with more information to refresh their memories before he can continue the discussion. However, he also stated, “It’s not necessarily formative in itself, it just gives me 146 guidance of how I can help them make connections and uh, use their understanding” (Exit Interview, lines 429-430). Monitoring. Although all the teachers monitored students while they were working at some point during the observed lessons, Charlie and Lukas were the only ones who answered a question specifically asking about the connection between monitoring and formative assessment. Charlie thought monitoring related to formative assessment depending on what he did during this time. He stated that sometimes he is more planned about who he talks to and what he asks students, which makes it more related to formative assessment than the times he is not as organized. He said when he did it well, then it gave him information about where students were in their thinking. He also saw monitoring as a great time to ask questions to students, which provided him with different information than he gained during a whole class discussion. Lukas also saw monitoring as relating to formative assessment because “I’m watching what they’re doing, what they’re writing down. I’m listening to what they’re saying, what they’re talking about. Um, so that kind of guides me a little bit” (Exit Interview, lines 435-437). Lukas also used this time to judge the pacing of the lesson to determine if students needed more time to work or if they were ready to move on, stating, “formative assessment isn’t just understanding, it’s when they’re going to be ready for whatever’s next” (Exit Interview, lines 444-445). Based on his explanation, it seemed like Lukas used monitoring to gather evidence about students’ understanding of the content as well as their readiness to change activities. Lukas also said that he considered how students were working to make decisions about giving students more time. For example, if students were on task and working to make sense of the activity, then he gave them more time. However, if students were not finished because they were off task, then he did not give them extra time to work before moving on. 147 Students’ sharing out answers. Sometimes Charlie would go around the room and have the students read the answers to the worksheet. He thought this had a weak connection to formative assessment because “I don’t know that I get a ton of information and I don’t know that they do” (Exit Interview, line 341). However, Charlie did say that this process gave him information from a large sample of students. If students had the correct solutions, then that told Charlie that students were likely understanding the content and if students had a wrong or incomplete answer, then Charlie knew those students needed more support. Restating peers’ ideas. Charlie was the only teacher who answered questions specifically asking about the connection between having students restate their peers’ ideas and formative assessment because this was a regular practice in Charlie’s classroom. Charlie thought there was a relationship to formative assessment because “it’s about listening to each other. But also, um, about do you understand this idea that this other human being is saying?” (Exit Interview, lines 324-325). This process gave students’ information about their own understanding, but it also provided Charlie with evidence of students’ understanding or lack of understanding. When choosing students to restate ideas, sometimes Charlie would decide based on students’ body language. If he saw a student who looked confused, he would ask that student to restate what they heard to help them make sense of their peers’ ideas. Other times, he would call on someone else and if the student who looked confused started nodding, then he would have that student restate the ideas to see if they now understood the concepts like they thought they did. Looking for multiple strategies. Charlie and Lukas often used class time to have students share different strategies and saw this as an important part of their class. When I first asked Charlie is he saw a connection between having students share multiple strategies and 148 formative assessment he did not think there was because “by formative assessment definition no, cause I’m not doing it on purpose” (Exit Interview, lines 348-349). By this he meant that he was not having students share different ideas as an assessment strategy. As he talked more about his thinking, he started to see it slightly differently. He stated, “it feels like, process formative assessment. Like do you have a process? Do you have a better process than that person or a worse process? Do you have a different way of thinking about it? Is it a better way?” (Exit Interview, lines 360-362). Without any prompting from me, Charlie continued thinking through how he used multiple strategies. He then came to a new conclusion, stating, “So I think it’s giving them feedback on strategy as opposed to answer, which is probably just as important, so I would say probably yeah that it’s a formative assessment. I’ve gone full circle; I think it is (Exit Interview, lines 373-375). Charlie often asked his students if there was another way to solve a particular problem. He saw value in having students share multiple solution strategies, so he made a point of asking students for different methods. Charlie stated that this was a goal he had for his students. He wanted students to hear strategies from their peers to help them think about different ways to approach the problems. He found this especially helpful in the probability unit they were working on since the problems could be approached in different ways depending on how you made sense of the problems. This also connected to a larger goal Charlie had for his students, which was for them to be flexible mathematical thinkers. Lukas also thought having students share multiple strategies related to formative assessment because “it gives you, see the different kids see it in different ways and I think that’s good for other kids to see, too” (Exit Interview, lines 281-282). He made time during his class for students to share different strategies. When students were presenting their work on the board, 149 he would often ask students if someone had a different strategy and have that student also share their work. This way the students could see two different, valid methods that work and use the one that makes sense to them. Lukas also made a point of validating students’ strategies that were different from his own. He let students know they could choose which one worked best for them and they did not have to use his strategy just because that was the one he showed them. Summary. The teachers used many different instructional strategies during their lessons, however, those discussed here are just a sample. The teachers saw these strategies as relating to formative assessment, but if often depended on how they used them in their classrooms. Based on how formative assessment is defined in this dissertation, none of these strategies alone are formative assessment, however, they could include formative assessment practices, such as feedback or instructional decisions, as discussed by the teachers. Analysis of Teachers’ Descriptions and Observed Formative Assessment Practices This section provides specific descriptions of how each teacher viewed the formative assessment practices based on questions they answered during the background interview, unless noted otherwise. Following this is an analysis of the similarities in their beliefs. Then, I share descriptions of how the teachers implemented the same formative assessment practices in different ways based on the class observations. Learning Targets Charlie. The teachers at Charlie’s school worked together to develop the learning goals for their courses and often shared resources. They had a very collaborative environment. When determining the goals for the unit, Charlie tried to limit the number of goals, so he could assess them multiple times throughout the unit. He also considered the size of the goal in that the goal needed to be big enough that he could assess it multiple times, but small enough that the students 150 were able to take feedback on the goals and know what to do. He referred to some of the goals as “formative assessment goals” explaining that “we wanted to be able to give assessments to kids and the language and size of them needed to be in a certain way to make that usable for us” (BI, lines 323-325). He had two goals that he often included in every unit because those were important and foundational goals that were more process-oriented. Goal 1 was “I can mathematize a situation” and Goal 2 was about students being able to explain their mathematical reasoning. He also had content -based goals, such as “I can write a linear rule”. All his goals were written as “I can” statements and were in student friendly language. As he moved through a unit, the goals followed a structure starting with a context, then working through abstract situations, and then ending back in context. An example of this progression Charlie gave was “the context, how do we describe this, what are all the tools that we can have, now let’s use the tools” (BI, lines 204-205). Charlie often shared the goals with his students, but he might not share all of them at the beginning because he wanted “students to drive the curriculum as much as they can” (BI, line 235). He would then share the goals with students as they moved through the unit and find that a new goal was needed to address the concepts they were talking about. After students had completed the first unit and had a better idea of the structure of Charlie’s class, he liked to have them think about where the concepts should go next by asking them, “What do you think makes sense next? What might be the goals we look at?” (BI, lines 250-251). He did this because, “I’d like them to see that mathematics has a structure to it that makes sense and they can predict what we’re going to do next” (BI, lines 251-253). Evan. Evan worked with the other teachers in his department to determine the learning goals for each unit. They dissected the Common Core State Standards to determine the important 151 goals and used those to write learning targets. They focused on the standards that repeated and were broad, so they could account for multiple standards within a larger one. The teachers tried “to write standard goals for students that are more specific [than the Common Core Standards], um towards what we’re doing in that actual unit” (BI, lines 243-244). They also limited the number of goals to about four or five per unit to keep the amount of information students needed to keep track of to a minimum. Evan always shared the learning targets with the students at the start of every new unit. He did this by giving them a document (called their learning scales) that had all the goals for the unit listed in student friendly language. Examples of the learning targets for one of the chapters for the Algebra 1 class I observed included, “I can solve quadratic equations.”, I can use math vocab related to naming polynomials.”, and “I can add or subtract two expressions.” The school was going one-to-one during the following academic year, so Evan stated that he would switch to having students complete their learning scales online. Gwen. Gwen worked with Evan and the other mathematics teachers to write learning targets for the classes they had in common. Even though the textbook had learning goals provided within it, the teachers did not often use these. They thought these goals were not worded the way they preferred and there tended to be many more of them. It was important to the teachers to limit the number of learning goals students were being assessed on for each unit. Like Evan, Gwen also provided her students with learning scales at the start of each unit, which included the learning targets for that particular unit. Examples of learning targets for the Algebra 2 class I observed included, “I can identify the zeroes and horizontal and vertical asymptotes of rational functions from a graph.” or “I can solve radical equations.” 152 For the trigonometry course that Gwen taught, she used a different practice in determining the learning targets. She stated that this course was not a Common Core aligned course, so she just used the goals from the textbook. At the start of the chapter, she would give the students the goals, however she did not provide them with scales like with her other classes. Lindsay. In determining her learning targets, Lindsay mostly followed what her textbook had listed as the goals of each lesson. For her AP Statistics course, though, Lindsay referred to the AP syllabus to develop learning targets. Lindsay also shared the learning targets with her students in the form of a focus question that she wrote on the whiteboard. The intention of this question was to “help [students] figure out what it is they’re supposed to be understanding by the end of the class” (BI, lines 186-187). These focus questions also came from the textbook and were the headings for the sections. Examples from each round of data collection for the Algebra 2 class included, “How can I describe the graph?” and “How can I compare results?”. There were some days during the observations that Lindsay had the learning targets or standards for mathematical practice on the screen when students walked in, but she did not verbally mention these to students. Some examples of these from the Algebra 2 class I observed included: “I can measure spread using the range, interquartile range (IQR), and standard deviation.” or “I can use appropriate tools strategically.” Scott. Like Lindsay, Scott also determined his lesson goals based on what the curriculum had listed as the objectives for that particular section. During the observations, I noticed he also wrote the focus questions from the textbook on the whiteboard in the back of his room. Examples from his Geometry class for the two rounds of data collection included, “How can I measure it?”, “How can I build it?”, and “What if the sample space is too large?” Sharing the goals with students happened most of the time but might not be an everyday occurrence. If the 153 goals were shared with students, then this occurred during the introduction of the lesson where “we’ll kind of explain, kind of where we were in the past and kind of where we’re going to” (BI, lines 124-125). The administration at Alexander required teachers to post their learning targets. Scott found that with the student-centered curriculum they adopted, his approach to learning targets was different. With the traditional textbook, there would be one objective for each lesson and he would post that on the board. However, when they adopted the new curriculum, posting the goals became problematic. He found that there would be multiple days needed to address a particular learning objective, so it was challenging to post the goals because they would not likely change every day. They made a compromise with their administration that they would post focus questions that came from their curriculum every day that would link to the learning targets at help guide instruction. Lukas. For his Geometry class, Lukas consulted the textbook to determine the learning targets, however he also knew where these goals fit with the Common Core Standards. The teachers worked together to determine which of the optional lessons they would include to focus on learning targets they believed were important for students to know. They also strayed from the textbook, so they could condense some of the content, which also allowed them to concentration on learning targets that were necessary for understanding future mathematical concepts. Lukas had been working on creating “I can” statements for the learning targets and used the content of the lesson to determine them, considering “what are the things the students are supposed to learn today” (BI, line 385). For his Pre-Calculus class, Lukas was not as concerned about connecting to specific Common Core Standards like he was for his Geometry class because this course was an elective. However, he did believe that the “fundamentals of the course are still tied to the Common Core” 154 (BI, line 318). For the learning goals, since he had been working with this curriculum for longer than the Geometry one, he thought he was better prepared to unpack the objectives. His focus was breaking down the standards into student friendly language and writing them as “I can” statements. These statements gave students an idea of “what [they] should get out of today” (BI, line 325). Lukas wrote the learning targets on his whiteboard for each of his classes, however, he stated that he could do a better job of verbalizing the learning targets to students because as of now, students were expected to look at the board to see what the plan was for that day. Most of the learning targets changed day-to-day, however, there were some occasions where the same goal would apply to multiple lessons. Some examples of the learning targets he had written on the whiteboard for the Pre-Calculus class I observed included: “I can find the dimensions of a matrix.” or “I can identify key characteristics of a conic section from its equation or graph.” Analysis. Charlie, Evan, and Gwen all talked about working with their colleagues to write the learning targets for the units. They were also concerned with keeping the number of learning targets to a minimum, so students did not have to keep track of many different mathematical ideas during the unit. It is not surprising that Evan and Gwen talked about working together to develop their learning targets since this was one of the foci of their daily co-planning time. Lindsay and Scott, who taught at the same school, and Gwen relied heavily on their textbook to determine the learning targets. For the elective courses that Gwen and Lukas taught, they were less concerned about aligning these courses with the Common Core State Standards than their college-prep courses. Charlie, Evan, Gwen, and Lukas all wrote their learning targets as “I can” statements, making sure they were in student friendly language. Lindsay and Scott, though, used focus questions to guide their instruction. It is interesting that Lindsay and Scott were required by their administration to post the learning targets, yet they rarely verbally shared 155 them with students during the observations. They did have the focus questions posted on their whiteboards, though, so that likely met the administration’s expectations. It is also worth noting that during one of my visits to Evan’s classroom, he was being observed by his principal, who took photos of the learning targets posted in the back of Evan’s classroom. This makes it seem like the administrators at Evan and Gwen’s school also expect to see learning targets posted in their teachers’ classrooms. Differences based on observations. Charlie, Evan, and Gwen all displayed high level practices for sharing learning targets with their students. Charlie and Evan both started most of their class sessions by sharing the learning targets with their students. They often connected the learning targets directly to that day’s activity. This set the stage for the students for what they were expected to learn during that lesson. These teachers shared the learning targets with the students on a regular basis, making it an important part of their classroom practice. Although Gwen had the learning targets posted in her classroom, she rarely discussed them explicitly with her students. However, on one particular day, she spent a good portion of the class period focused specifically on the learning targets as a review for the students’ unit test that occurred the second part of class. Gwen read each learning target to her students and then worked out problems on the board with the students that addressed each learning target. She checked in with students after each problem to see if students felt prepared to answer questions related to that learning target. Questioning Charlie. Charlie said that he used many different types of questions during his instruction. For some class sessions, he would launch class with a big question that had a context to guide that day’s instruction. An example of this type of question he gave during the 156 background interview was related to the work of Dan Meyer, “How many of these cups would line up, you know, and you show the cups, and they say, I want to know this, so they come up with a question and that’s the question we’re trying to ask.” He described “hip pocket questions” he used during the explore section of the lesson to push students’ thinking. Clarifying questions were also used during the explore section and helped him better understand what students were asking him. During the summary portion of his lessons, Charlie said his questions were about “connecting usually to what we’ve already done or connecting to what we’re gonna do and so we try and have questions that would shift level” (BI, lines 370-371) so that students were speaking either more specifically or generally depending on the situation. Charlie did not give examples of these three types of questions during the interviews. He also referred to formative assessment questions, which he used to target specific students to determine where they were in their thinking in relation to the learning goals. Although he did not give specific examples of what these questions would be, he did say he often asked these questions when he was monitoring students working. Charlie saw these different question types contributing to formative assessment in different ways. The two times he saw questioning as being most related to the formative assessment process was during the explore time when he was talking with individuals or small groups because this gave him information on where they were in their thinking and during the summary where they were connecting mathematical ideas. Charlie had questions prepared ahead of time that he wanted to ask students while they were working that usually started with a checking in type of question and then he used the students’ response to determine the next question. He also used the questions students asked him to gather information about where they were in their thinking, which helped him determine what to ask them. 157 During the six observations of Charlie’s teaching, he asked questions during all the activity structures he implemented except Introduction; he did not implement Teacher Giving Information or Summary. He also asked a large number of questions, with almost 36% of those questions being classified as high-level. Charlie had the largest percentage of high-level questions out of all six teachers. Table 22 shows the number of low and high-level questions Charlie asked during each of the nine activity structures. Table 22 Charlie’s Questioning Practices Low High Total 1 0 38 41 118 24 0 0 11 19 25 23 1 0 49 60 143 47 181 146 327 Activity Warm-up Introduction Interactive Lecture Individual Pair/Small Group Presenting Student Work Whole Class Discussion Teacher Giving Info Summary Total 0 0 403 0 0 224 0 0 627 As Table 22 shows, Charlie asked most of his questions during Whole Class Discussion, with almost 45% of those questions classified as high-level. The number of questions he asked during this activity structure was more than twice as many questions as he asked during the next activity structure, which was Pair/Small Group Work. Most of the questions asked during this activity structure were low-level (83%). Interactive lecture and Individual activity also included 158 mostly low-level questions. Warm-up was not an activity structure where Charlie asked many questions as he only asked one low-level question during this time. During the Presenting student work activity structure, Charlie had an almost even split on the number of low and high- level questions. Evan. Evan said he did not usually plan questions ahead of time, instead came up with them “on the fly”. Since he was teaching four different classes, he found it challenging to plan questions for all his classes ahead of time. However, he believed his past teaching experience had helped him think about questions to ask saying, “I feel like a lot of what I ask now in class is based off things that have happened over the course of the last 5 years where I’ve seen students have trouble on something or um I just I try and gauge where students are at and ask questions to direct them in whatever direction they think they need to go” (BI, lines 304-307). Many of Evan’s questions came from his textbook because it was inquiry-based, so students answered the questions in the book and Evan monitored while they were working and talked to them about what they were doing. During the six observations of Evans’s teaching, he asked questions during all the activity structures he implemented except Warm-up and Summary; he did not implement Teacher giving information or Presenting student work. He also asked many questions, with about 19% of those questions being classified as high-level. Table 23 shows the number of low and high-level questions Evan asked during each of the nine activity structures. Table 23 Evan’s Questioning Practices Activity Warm-up Low High Total 0 0 0 159 18 60 8 45 0 3 6 2 14 0 21 66 10 59 0 201 54 255 0 0 332 0 0 79 0 0 411 Table 23 (cont’d) Introduction Interactive Lecture Individual Pair/Small Group Presenting Student Work Whole Class Discussion Teacher Giving Info Summary Total Similar to Charlie, the activity structure where Evan asked the most questions was during Whole Class Discussion, with 62% of his questions occurring during this activity structure. This was almost four times as many questions as he asked during the next activity structure, which was Interactive lecture. It is interesting that Evan talked about making up questions in the moment and then during the observations, most of his questions were low-level. It can be challenging to think of high-level questions on the post, which could by why Evan had so few high-level questions. Gwen. Gwen stated that she determined the questions she would ask students based on how things were going in the class. She did not use the same questions for every class. She tried to use open-ended questions but had stated that this was something she was working on getting better at. Gwen often relied on thinking of questions in the moment and did not plan questions ahead of time, stating, “I have planned things that I want them to get out of the lesson, but I don’t have necessarily the questions that I’ll ask” (BI, lines 351-352). Gwen also thought it was 160 important to whom teachers focused their questions saying, “I think as teachers [we] need to be more aware of is how we present the questions, so we’re not just singling out somebody and everyone else is shutting down” (BI, lines 360-361). During the five observations of Gwen’s teaching, she asked questions during all the activity structures she implemented except Summary; she did not implement Teacher Giving Information She also asked many questions, with about 21% of those questions being classified as high-level. Table 24 shows the number of low and high-level questions Gwen asked during each of the nine activity structures. Table 24 Gwen’s Questioning Practices Low High Total 3 10 180 32 68 9 37 0 3 49 7 20 2 7 3 13 229 39 88 11 44 Activity Warm-up Introduction Interactive Lecture Individual Pair/Small Group Presenting Student Work Whole Class Discussion Teacher Giving Info Summary Total 0 0 339 0 0 88 0 0 427 As Table 24 shows, Gwen asked more than half of her total number of questions (54%) during Interactive lecture, with 21% of those questions being considered high-level. The next most common activity structure for Gwen asking questions was Pair/Small group work, which 161 accounted for 21% of her questions. This means these two activity structures contained almost three-fourths of the total number of questions Gwen asked during the observed lessons. During the background interview, Gwen stated that she was working on trying to ask better questions, therefore, it is not surprising that she had many more low-level compared to high-level questions. Lindsay. Lindsay thought questioning was an important part of formative assessment. She often used her textbook to determine the questions she would ask students during the lesson. She thought her experience with the curriculum had helped her determine which questions to use. Lindsay also considered what questions she would ask based on the grouping structure of her students. She stated that she did not ask as many questions to the whole class unless they were of the “can you share your ideas” type of question (BI, line 212). Lindsay said that she planned and used more of her questioning strategies while students are working in small groups. Like Charlie, Lindsay referred to formative assessment questions as the types of questions students ask each other when working in their teams. Lindsay used this phrase during the exit interview. She said that she assigned each group role a different set of questions. Some examples are, “Does anybody have any questions?” or “Can somebody explain this a different way?” (lines 79-80). She said her goal for these types of questions was for her students to “know whether or not they can explain it” (Exit Interview, line 84). She expected that these questions pushed her students thinking and made them have to explain their ideas to her or their peers. During the six observations of Lindsay’s teaching, she asked questions during all the activity structures she implemented; she did not implement Warm-up or Teacher Giving Information. She asked largest number of questions out of the six teachers, with about 34% of those questions being classified as high-level. Table 25 shows the number of low and high-level questions Lindsay asked during each of the nine activity structures. 162 Table 25 Lindsay’s Questioning Practices Low High Total 0 11 3 5 0 5 0 6 0 16 3 11 351 169 520 7 6 13 79 48 127 Activity Warm-up Introduction Interactive Lecture Individual Pair/Small Group Presenting Student Work Whole Class Discussion Teacher Giving Info Summary Total 0 1 457 0 3 237 0 4 694 As shown in Table 25, most of Lindsay’s questions (75%) occurred during Pair/Small group work, with about 33% of those questions classified as high-level. Whole class discussion was the activity structure with the next largest number of questions (127), which included almost 38% high-level questions. Lindsay was the only teacher who had multiple activity structures with more high-level than low-level questions. During the Individual activities, more than half of her questions were high-level. In addition, during Summary, three out of the four questions she asked were high-level. Like Charlie, Lindsay had an almost even split in the number of low and high- level questions during the Presenting student work activity structure. The dominance in the number of questions during the Pair/Small group work is not surprising since Lindsay mentioned during the background interview that she liked to ask more questions during this activity structure than any other. 163 Scott. Scott stated that he determined many of his questions in-the-moment based on responses he received from students when he asked general questions that required them to explain their thinking or process. At times, he used the questions provided in the curriculum’s teacher notes. He also worked to ensure his questions were moving students towards achieving the learning goals. These types of questions often occurred when students were working in their groups. Scott did not give a specific example of this type of question. When Scott was asking questions to the groups, he said that he tried to ask questions to different group members, so he could gather information about all the students’ thinking. During the five observations of Scott’s teaching, he asked questions during all the activity structures he implemented except Individual and Summary; he did not implement Warm-up, Interactive lecture, or Teacher Giving Information. He also asked the fewest number of questions compared to all the teachers (though it should be noted he had five instead of six observations), with about 35% of those questions being classified as high-level, which was the second highest percentage across the teachers. Table 26 shows the number of low and high-level questions Scott asked during each of the nine activity structures. Table 26 Scott’s Questioning Practices Activity Warm-up Introduction Interactive Lecture Individual Pair/Small Group Low High Total 0 4 0 0 0 3 0 0 0 7 0 0 160 80 240 164 2 23 2 18 4 41 0 0 189 0 0 103 0 0 292 Table 26 (cont’d) Presenting Student Work Whole Class Discussion Teacher Giving Info Summary Total Like Lindsay, the activity structure that included most of his questions (82%) was Pair/small group work, with about 33% of those questions being classified as high-level. Whole class discussion contained the second highest number of questions (14%) with almost 45% of those questions being high-level. Like Lindsay and Charlie, Presenting student work had an even split of low and high-level questions. Scott also had an almost even split for the questions he asked during Introduction activities. The dominance of questions Scott asked during Pair/Small group work is not surprising, since this is the activity structure he implemented most often. Lukas. Lukas had the big ideas of the lesson in his mind during class and then based his questions on what he heard from students, saying, “I should be asking this question of them because, uh, they’re not understanding this part or they got that really fast, so I can ask this next question because they got that faster than I thought they would” (BI, lines 400-402). He had questions planned ahead of time that he wanted to ask during the lesson, but he also came up with many of the question on the spot. Lukas also used his questioning strategies to ensure that students were moving towards achieving the learning targets for that day, however, he did not give a specific example of this type of question. Lukas said sometimes he could gain valuable information about students’ thinking without them verbal answering questions. He used students’ 165 facial expressions and body language as a gauge for who was understanding the material and who was not. Lukas found the curriculum for his Geometry class helpful because many of the problems in the textbook required students to explain their answers. When the students were working in their groups, he stopped and checked in with students and asked them probing questions related to the work they had done. Then if their response was incomplete, he knew he needed to ask follow-up questions to clarify their thinking to ensure they were making the necessary mathematical connections in the problem. Lukas did not give specific examples of what these questions would be during the interviews. Lukas thought that students often “superficially answer a question” and “it takes some time to develop the understanding that, no, we want you to really be making connections” (BI, lines 435-436). Lukas said that he also made a point of asking all students questions when he was talking to different groups, so that all students were held accountable for the information. He did this in small groups so as not to put students on the spot during whole class discussions. He tried to ask them questions that he was certain they knew the answer to, to provide the student with some confidence in themselves. These questions tended to be more opinion-based, such as “do you prefer tree diagrams or area models when solving probability problems” and “why is that your choice”, so there was no correct answer and therefore no risk of being wrong for the student. During the five observations of Lukas’s teaching, he asked questions during all the activity structures he implemented except Warm-up, Introduction and Teacher giving information; he did not implement Summary. He also asked many questions, with about 11% of those questions being classified as high-level. This was the lowest percentage of high-level 166 questions among the six teachers. Table 27 shows the number of low and high-level questions Lukas asked during each of the nine activity structures. Table 27 Lukas’s Questioning Practices Low High Total 0 0 37 3 63 60 176 0 0 6 1 9 17 10 0 0 43 4 72 77 186 Activity Warm-up Introduction Interactive Lecture Individual Pair/Small Group Presenting Student Work Whole Class Discussion Teacher Giving Info Summary Total 0 0 339 0 0 43 0 0 382 Similar to Charlie and Evan, the activity structure where Lukas asked the most questions was during Whole class discussion (48%), with about 5% of those questions being classified as high-level. The next most common activity structure for Lukas asking questions was Presenting student work, which included 22% high-level questions. Analysis. All the teachers used questioning throughout the observed lessons. They all asked students questions to justify their solutions or explain their thinking to varying degrees. Lindsay, Scott, and Lukas all stated that when students were working in groups, which was most of the time in these classrooms, that they needed to ask questions of all students to ensure everyone understood the mathematical concepts. Gwen, Evan, and Scott all said that they relied 167 more on determining questions in the moment and not going into a lesson with very many planned questions. Charlie, Lindsay, and Scott all talked about asking different types of questions. They recognized that their questions served different purposes either based on the grouping structure or the goal of their questions. It is interesting to note that these three teachers also had the highest percentages of high-level questions. It is possible their focus on asking different types of questions for different reasons and in different situations allowed them to ask more higher-level questions than the teachers who were not as focused on asking multiple types of questions. It is not surprising that Charlie asked so many questions during the observed lessons, since he stated that he thought questioning was an important part of formative assessment. The data from the observations seem to confirm that Charlie focused on using questioning as a way to gather evidence from his students. Charlie and Lindsay were the only teachers who used the phrase formative assessment question. Although the teachers used it in different contexts, they had similar meanings in that they saw these questions as pushing their students’ thinking forward. It is interesting to note that Lindsay and Scott, who are at the same school, both asked the most questions during Pair/Small group work. This is not surprising, though, since these two teachers most often implemented this activity structure. It is surprising to note that Lukas stated he was more comfortable asking questions when students were working in groups, however, he asked twice as many questions during whole class discussions than during group work time. Differences based on observations. Although all the teachers asked questions during their lessons, there were some teachers who used their questioning strategies in different ways. Lindsay had found value in asking questions about what students noticed about particular ideas. Lindsay saw this line of questioning as important for her students as a way for them to 168 “formatively assess [themselves]. Are they making connections between the real world, and, and are they making, are they making things on their own or are they just going through motions in class?” (Exit Interview, lines 50-52). For example, she started the lesson by showing her students three polynomial functions and a graph and asked her students what they noticed about the equations and the graph. She gave them some time to think individually before talking with their groupmates and then had students report out what they had noticed. She did not give them any guidelines ahead of time because she wanted to see what her students would pay attention to on their own. She likes to use this line of questioning because, “they’ll know what direction the lesson’s going by just looking at, the looking at a few things a couple of equations and a diagram” (Debrief March 30, lines 271-272). She also thought her students had a lot valuable and interesting ideas to share if they were given the opportunity, so this line of questioning gave her students a voice in their learning. Since her students were often able to identify the important mathematical ideas, she was able to make a smooth transition to the day’s lesson by building on the students’ ideas. Charlie also asked his Algebra I students what they noticed to help them make sense of the exponent problems they were solving. One specific example of this was when Charlie was talking with an individual student who was struggling. He noticed that she was not seeing a connection between any of the problems on the worksheet and was treating each problem as its own mathematical idea. He wanted her to notice which other problems on the worksheet were similar to the problem they had just worked through together. He wanted her to make the connection that a negative exponent means division, so she would be able to use that knowledge on future problems. He saw this line of questioning as “a way to move kids forward that were maybe just at that beginning level” (Exit Interview, lines 467-468). 169 Self-assessment Charlie. Charlie used self-assessment in different ways in his classroom. Sometimes it was informally where students completed a ticket out the door reflecting on where they were in their thinking. Other times, it was more formal where students used a rubric or evaluated themselves based on a scale and provided reasons for why they thought they were at a particular level. In the beginning of the year, he often used a yes/no checklist because he thought students were better at that to start with because it was more straightforward. Then as the year progressed, students got better at self-assessing and could use rubrics. Charlie thought, “it’s hard for them to understand that idea [using rubrics] really early on … that gets better as they’ve written more, as they’ve gotten feedback, they’ve seen other people’s writing and things like that” (BI, lines 503- 505). He tried to scaffold students into completing self-assessments to ensure they were prepared to effectively track their own progress. Evan. As previously mentioned, Evan provided his students with their learning scales at the start of each unit. The students then assessed themselves on the learning goals using a scale of 1-4 at the start of the unit and then again at different points during the unit. When students self-assessed, they submitted their scales to Evan to review and he recorded grades based on quizzes, so students could see how their self-assessment compared to their actual performance. Evan did not think students were currently put much effort into thinking about the learning goals and often considered their past performance in determining their ratings. For example, a student who had struggled in mathematics tended to rate themselves as a 1 and thought they were always at a 1, whereas students who had been successful in mathematics tended to rate themselves high regardless of whether the content was new or not. Evan had been working on making students’ self-assessments “more effective and drive some of their learning” 170 (BI, line 380) and he wanted to “push them to do something” (BI, lines 383-384), but he was not yet where he wanted to be in terms of ensuring students were benefiting from this process. Gwen. Like Evan, Gwen used the learning scales as an opportunity for students to self- assess in connection to the learning targets. Unlike Evan, though, Gwen did not use the learning scales as often, nor did she have students monitor their progress in relation to the goals as they progressed through the unit. She claimed that she would like to make this more of an integral part of her classroom practice, though. Lindsay. Lindsay did not provide many opportunities for her students to formally self- assess. One experience she described, though, was to provide her students with model work for them to compare their work to. The curriculum also came with learning logs that provided students opportunities to reflect on particular mathematics concepts, and Lindsay sometimes used these to help students think about what they had learned during that specific lesson. An example of one of these learning log questions is, “Record your descriptions of center, shape, spread, and outliers.” The log also included discussion questions for students to consider, including “When do we use a histogram and when do we use a scatterplot?” and “How does the shape determine which statistics we will use?” Lindsay thought she needed to work on this practice, though, because she did not think she had “students to the place where they reflect” (BI, lines 426-427). She did not think students took reflection opportunities seriously. Scott. At the end of every lesson, there is a review/preview section, so Scott used this as homework and an opportunity for students to self-assess. He gave the students the answers to these problems, so they could check their work, but also so they focused on the solution process. Having the answers ahead of time also allowed students to self-assess because after they worked through the problem, they could check their answers and know if they were correct or not. If 171 students had questions on these problems, they could ask their team members or the teacher to help clarify any confusion. Scott also thought that having the students work in groups allowed them opportunities to self-assess. Through conversations with their team members, students could determine whether they were understanding the content or not. Scott stated, “if they don’t have much to contribute, then they know they need to, to get some work caught up” (BI, lines 241-242). Like Lindsay, Scott was observed using the learning logs from the textbook to provide his students opportunities to reflect on what they had learned. An example of one of these learning log questions is: “What is volume? What does it measure?”. Lukas. Lukas stated that he did not often use self-assessment in his classroom. Sometimes he used thumbs up/thumbs down to have students rate themselves 1-5 on how they were doing in relation to a particular concept. Another example of how Lukas provided opportunities for his students to self-assess was by giving them the answers to some homework problems ahead of time, so students could focus on the solving process. Lukas saw this as putting some of the responsibility on the student to take some ownership in their learning. However, Lukas used self-assessment more often in his previous position but had not had the time to implement many of those activities into his current teaching. In the past, he would give his students a list of all the topics as part of their unit test review and a scale of 1-4 for them to rate themselves on whether they understood the material (4) to they needed lots of help (1). He did not grade this activity; it was used as a resource for students to help them determine where they needed to spend their time studying. His students liked this activity and found it incredibly beneficial as a study tool because it helped them focus their attention on the concepts they needed to study. 172 Analysis. Across the teachers, two common themes are seen; self-assessment is not a big part of their classroom practices and the teachers wish they incorporated this practice more often. This coincides with what was observed during the lessons as well. Only three teachers implemented self-assessment during the observed lessons, therefore, it is clear that this is a practice that teachers are not using very often. When the teachers did provide students opportunities to self-assess, it mostly involved the students using a rubric or other structure. Charlie and Lukas both talked about giving students a rubric with a 1-4 scale to rate themselves. Evan and Gwen, who are at the same school, both connected their self-assessments directly to the learning targets by giving students scales at the start of each unit with all the learning targets listed. Lindsay and Scott, who are at the same school, both used resources from their textbook to provide their students with opportunities to self-assess. This is not surprising since both teachers mentioned they relied heavily on their textbook for information about learning targets and questioning. Lindsay and Evan both expressed concerns about students’ effort or motivation in completing self-assessments in a meaningful way. They did not think their students took these opportunities seriously and therefore were not gaining the benefits to their learning. Charlie also recognized the impact students had on this process and found ways to scaffold them into effective self-assessment by starting with a checklist and then moving to a rubric as students gained experience. Differences based on observations. Since most of the teachers noted during the background interview that they did not often use self-assessment and that this was a formative assessment practice they would like to use more often, it is not surprising that only three of the teachers (Charlie, Evan, and Scott) provided students with opportunities to self-assess during the observed lessons. Charlie and Evan both connected the self-assessment to the unit’s learning 173 targets, however they did so in different ways. Charlie had students self-assess in preparation for their unit test. He had the students spend some time reflecting on the learning targets on their own. The students had to write what was needed to accomplish the three learning targets for the unit and then assessed where they were in relation to those learning targets. Then they had a whole class discussion where students shared the key ideas for each of the learning targets. Next, students worked in groups on a review activity where they only had to focus on the questions related to the learning targets they needed more practice with. The class ended with another whole class discussion where Charlie answered any final questions students had about the learning targets or the problems on the review. Evan provided all his students with their learning target scales, which is a document that consisted of the unit’s learning targets and columns for students to self-assess at different times throughout the unit. Students first self-assessed before starting the unit to have a baseline for their understanding. Then after their quizzes, students would self-assess again to monitor how they were progressing through the unit. Evan would also record students’ actual scores from their quizzes on the scales so students could see how their self-assessments compared to their actual performance. He would like to do more with this, though, and hold students more accountable for how they are rating themselves. He has considered having students provide explanations for their ratings instead of just providing a number in hopes that this would require them to think more about the content of the learning targets and their reasons for their scores. Scott had students reflect on their learning in their small groups. As part of his curriculum, there were learning logs at the end of some of the lessons. This was an opportunity for students to think about what they learned during the lesson and provided them with a place to record their thoughts. This document also served as a review for students because it contained 174 many of the key ideas in the lesson. He thought that reflecting was challenging for students, so providing them with a structured opportunity was beneficial to them. Peer assessment Charlie. Charlie originally had students do peer assessment in an informal way but found that “they’re not super great at peer assessments”, so he tried to help students in two different ways. First, he gave them something to complete that had specific questions students must answer. Second, he showed anonymous student work on the document camera and had a whole class discussion based on the work. He has found that students were “really willing to be hyper- critical” of work in that type of situation (BI, line 652). He had students decide on a score for that particular piece of work based on the learning goals and then he shared what score he would give. Then they had a conversation about their reasoning in determining the score in relation to the teacher’s expectations. Not all his students were comfortable with him sharing their work, however, he made it clear from the beginning that this was a norm for his class and this was a way to get feedback on your work in a safe environment. He used examples of high and low quality work to provide students with exemplars of both types of work, which allowed them to have better discussions and helped students have a clearer understanding of how they could achieve the learning goals. Charlie believed that all students received the benefit of peer assessment even when they were not the ones specifically getting the feedback. Evan. Evan did not intentionally structure opportunities for students to peer assess, but since they sit and often work together in groups, he hoped that they were providing each other with feedback on their learning. He would “like better communication and collaboration between students” (BI, lines 436-437). Evan had stated this is something he would like to incorporate more often into his classroom practices. He is hoping with the adoption of Chromebooks for next 175 school year that he can incorporate online forums where students post work and get feedback from their peers. Gwen. Gwen did not use this practice often in her classroom and would like to incorporate more opportunities for students to peer assess. She used to use the activity, “My Favorite No”, where she would put up an incorrect solution and the students would have to determine what was wrong. Her students enjoyed this activity and “they help each other a lot so I think they do provide each other good feedback” (BI, lines 611-612). More often, she allowed students to work together, so she excepted that they are providing each other with feedback while they were working with partners or small groups. Lindsay. Lindsay did not provide many opportunities for peer assessment, but she had students trade their homework and then for a particular problem they must “say two good things and one thing to improve” (BI, lines 354-355). She always made sure to have a conversation with students about what was appropriate feedback to write. At the start of the school year, students focused on non-mathematical, unhelpful aspects like handwriting, but as they get more experience, students were able to provide meaningful feedback to their peers. She found this practice to be easier to implement with her AP students than with her younger student in large part to the maturity of the students. Like Gwen, Lindsay also used the activity, “My Favorite No”, which she used with her mathematics lab students. Her students enjoyed this activity and thought it would help them be able to provide feedback to their peers in the future. The activity focused on what was good about the work before focusing on the mistake the student made. 176 Scott. Scott used different activities to provide students with the opportunity to peer assess. One example was called hot potato where students took turns solving a problem one step at a time. If a student made a mistake, then the pairs had a conversation to correct the mistake in order to continue solving the problem correctly. Another example was called a pairs check where one student was the writer and the other partner was the solver. The writer, who was not allowed to talk, must write down everything their partner said to solve a problem. After a certain amount of time, the pairs were allowed to talk and provided each other with feedback to correct any mistakes or ask any questions about the solving process. Scott thought these activities were beneficial to students saying, “it’s kind of an eye opener for different methods and different ways to solve because they all think differently and now they are kind of forced to think like their partner, which might not normally see that” (BI, lines 299-301). Lukas. Lukas’s use of peer assessment was more informal. He had students sitting in groups, so he would often have them check their answers with those at their table. He has found that having tables in his classroom as opposed to individual desks had made students more willing to share their thinking with their peers. Analysis. Like self-assessment, peer assessment was one of the formative assessment practices that the teachers said they did not use very often in their classrooms and wanted to find ways to use more often. This was also visible in the observed lessons since none of the teachers implemented peer assessment in their focus class lessons. When the teachers talked about using peer assessment, it was mostly informal implementations. Evan, Gwen, and Lukas all mentioned that they expected that students were providing peer assessment to their classmates when their students were working together in class in groups or partners. Although the particular activity was not observed during data collection, Gwen and Lindsay both discussed having used “My 177 Favorite No” with their students as a way to practice providing peer assessment. The activities described by Scott were also not observed. Similar to how Lindsay used model work to help her students self-assess, Charlie talked about using examples of student work to help students peer assess. For these two teachers, having completed student work seemed an important part of self and peer assessment. Lukas stated that he thought when his students worked problems on the board, that the other students evaluated the work, which could be seen as peer assessment. However, none of these instances were coded as peer assessment during the observations because that practice was not explicit to the viewer nor was it verbally stated to the students that they should assess their peer’s work. Lukas did ask students if they agreed or disagreed with the solution, however that seemed more like part of the conversation about the student’s work than a request for evaluation. The intention of the activity seemed to be on the sharing of and discussion of students’ work and not on giving that student peer feedback on their performance. Differences based on observations. None of the teachers provided students with opportunities that were coded as peer assessment during the observed focus lessons. However, Charlie did implement peer assessment in the additional class that I observed that related to the example he provided during the background interview. This was an Algebra 1 class that was learning about graphing rational functions. Charlie collected the students’ homework and anonymously showed different pieces of student work on the document camera and asked the students to rate the work on a scale of 1-4. Charlie reminded the students what each score meant, such as expert (4) and proficient (3). The students held up their fingers to display the score they would give each particular piece of student work. Then students shared reasons for how they decided to rate the work. Charlie also shared his own thinking about the work to help the students see how their expectations align with his. During the activity, the students were engaged 178 in the activity, displaying their scores, and sharing their reasoning. None of the students questioned whose work was displayed, which Charlie noted as norm for that class. The students were not focused on the individual; instead they were focused on evaluating the work. Feedback Charlie. Charlie provided his students feedback in multiple ways, including verbal and written. He used verbal feedback to ensure students were noticing the mathematical ideas he wanted them to be paying attention to. Sometimes he used a rubric to provide feedback to classify students on a scale from 1-4, with 4 being an expert. The feedback told students “where they’re at and in terms of the goals that we’re looking at” (BI, lines 549-550). Other times he assigned codes to different feedback comments and wrote the codes on students’ papers, so they knew which comments applied to them, which helped him focus students’ attention on particular feedback comments. Charlie wanted his students to be able to take the feedback he gave them and use it to fix their mistakes and develop a stronger mathematical understanding. Charlie also assessed students early in the unit and then throughout the unit so students had many opportunities to receive feedback on where they were in relation to the goals. He and his students could then use this feedback to determine next steps for the students to progress their learning. Evan. Evan provided verbal feedback daily to his students, which he considered to be valuable. This happened through the conversations he had with them while he was monitoring and connected to what they were working on. He also tried to return quizzes to students the same day or by the next day so that he could provide written feedback quickly. In addition, he provided comments on their scales, so his feedback was specifically aligned to the learning targets. 179 Gwen. Gwen regularly gave her students quizzes and handed them back the next day with feedback on where they went wrong. She also went over the quizzes in class right after students handed them in while the questions were still fresh in their minds, so they could receive verbal feedback as a whole class on how they did. Students were allowed to retake quizzes, but they must have completed all of the homework and met with Gwen to discuss the mistakes they made. She used these conversations as an opportunity to give specific verbal feedback to the student. The students took their chapter tests online, so they instantly received evaluative feedback from the program on a score, but they did not know which questions they answered incorrectly. Lindsay. Lindsay provided a lot more verbal than written feedback to her students. She tried to provide written feedback on their homework “to take them to the next step” (BI, line 440). She often used questions as a form of verbal and written feedback to help students determine what they should do next or how to correct a mistake they have made. Most of her verbal feedback was provided to students while they were working in their groups. Scott. Scott mostly provided his students with verbal feedback, which happened in the moment while students were working. However, on summative assessments, Scott provided written feedback via a rubric, which “give[s] them a score based on the rubric so they kind of see where they’re at on that problem” (BI, lines 262-263). This feedback came in the form of the following categories: expert, proficient, beginner, novice, or incomplete. Students would also receive information detailing why their work was assigned that particular category. Lukas. Lukas focused mostly on providing students with verbal feedback through having conversations with students to help “them to see not just the answer, if it’s incorrect, but to understand where the problem is” (BI, lines 556-557). Showing their work was imperative for 180 Lukas to provide students with helpful feedback, because if they only had an answer, he could give them much direction of where they went wrong, since he did not have any work to look at. When students took assessments, Lukas provided written feedback to let students know where they went wrong in their solving process. It could be a calculator error or a process error, but regardless, he would write a note, so the student knew everything was correct and they might have entered it into the calculator wrong or they made a mistake in their work. Analysis. All the teachers talked about providing written and verbal feedback to their students, however, most of the teachers focused on verbal feedback. Charlie and Evan both tried to align their feedback with the learning goals, so they could support students in achieving those goals. Although, Lindsay did not talk specifically about alignment to learning goals, she stated that she focused her feedback on helping students determine the next steps. Evan and Gwen worked to ensure their feedback on quizzes was returned to students quickly, so the mathematical ideas would still be fresh when students reviewed their work. During the observed lessons, Evan accomplished this by handing back quizzes either during the same class period or during the next class period. Gwen only gave one quiz during the observed lessons, which she handed back quickly. Scott and Charlie often used a rubric to provide students with written feedback that classified their performance on a 1-4 scale. During the observations, only verbal feedback was coded since that was the type of feedback that was visible during the lessons. If the teachers gave students written feedback during the data collection period, then it was not discussed with me during any of the interviews, except for Charlie and Lindsay. With the teachers’ focus on providing feedback to students, it was not surprising this was one of the most common formative assessment practices seen in the observed lessons. 181 Differences based on observations. The focus of this formative assessment practice was on verbal feedback since that is what was heard during the observed lessons. However, many of the teachers talked about giving written feedback to their students. During one debriefing interview each, Charlie and Lindsay both talked through some written feedback they had provided to their students. For Charlie, the written feedback was on a learning check the students had received back that day in class. When we had our debriefing after school, a student was working on another assignment in the room and heard us talking and allowed the teacher to share his learning check with me to talk through some examples of the written feedback. Charlie does not usually identify the correct answer for students; instead he provides them information about a mistake they made or asks them a question. For one of his comments, Charlie said he did not think it was a good example of feedback; he had written “don’t round” on the student’s paper. For a different problem, Charlie had multiple notes. First, he had written “This part is important” and underlined part of the student’s response in reference to an explanation the student had written. Second, he also circled a word in the original problem to identify a key idea the student did not address in his response. Lastly, Charlie crossed off the word “sometimes” in the student’s response because that was not the correct answer; the answer was “never”. However, at times, he does put a check mark next to students’ work to let them know they have a correct answer or method. Charlie stated that, “I try and just not give a ton of feedback other than that’s good, um unless there’s something specific in their writing I want to point out” (Debriefing Interview, June 1, lines 329- 331). For a different problem, Charlie disagreed with the student, writing “Yeah, these can… not why we subtract”. This told the student that part of his response was correct, but his justification was not. An example Charlie gave that was not included on this student’s paper was that 182 sometimes he would set up a probability on one problem for a student who was struggling to write the probability equations to give them an example to refer to, but he would only do it for that one problem, since he did not want to do the work for the student. The written feedback Lindsay discussed during the debriefing interview was for her AP Statistics class. The students worked together to create posters solving different problems related to probability. For one group, the students had the correct answer, but that was not enough for Lindsay. She stated that this group of students had communication issues because they did not provide enough information about how they solved the problem. She wrote the following three comments on their poster: “Show all work”, “How did you assign digits?”, and “Could you be more clear… so others understood your method?”. For a different group, Lindsay wanted the group to read their conclusion to see if it contradicted itself, which she stated that it did, so the students could revise their work. Lindsay stated that most of the time when she provides written feedback it is in the form of a question because it “will hopefully um, help them move forward with their answer” (Debriefing Interview, March 30, line 380). Instructional Decisions Charlie. Charlie thought about making instructional decisions based on two different timelines; during the class session and for the next day’s lesson. Even though Charlie taught the same subjects more than once a day, he based his decisions on each particular class, therefore, it was likely that all his classes of the same subject were in different places within the unit. This often occurred more towards the beginning of the unit, though. In thinking about making decisions, Charlie said, “I have a sequence in my head of what I’d like kids to do, but these are sort of movable parts and based on what we’ve seen, I can absolutely make changes” (BI, lines 726-728). He had different assignments in mind and he thought through them to see which ones 183 would be most appropriate for the next lesson. He also considered whether he needed to create a new activity, instead of using a previously created one. When thinking about making in-the-moment decisions, Charlie did this at times to check in with his students. For example, if he saw multiple students making the same mistake, then he would stop the class and bring them back together. Sometimes he would change the focus of the lesson, telling students, “I’d like you guys to answer these instead of what we’re going to do next. I want you to answer these four questions. Let’s talk about this right now” (BI, lines 733- 734). These decisions were based on the goals of the lesson, but if past goals were interfering with students making progress on new goals, then Charlie would take time to address these issues before having students move on. Evan. Evan gave daily quizzes and used the information from these to determine whether he needed to reteach the material or not. Evan did not share what factors would help him make these decisions. He also used discussion quite often “so students are communicating with me, um me asking them questions to kind of see where they’re having trouble and go from there” (BI, lines 463-464). Gwen. Gwen said she was “constantly adjusting and modifying” her lessons and that it was “based on student understanding” (BI, line 618). Many of Gwen’s students did not complete the homework on a regular basis, so she ended up giving them time in class to work because she thought it was important for them to practice these skills before they moved onto other concepts. During one of the days I observed Gwen, she made an instructional decision at the start of class to change the whole plan for the day because she found out that most of her students would be gone the next day for a field trip and she had planned on starting a new lesson. To ensure this large number of students did not miss the new topic, she changed her plan to teach the new 184 concept that day and decided they would spend the next day, when most of the students were gone, reviewing the lesson. Lindsay. Lindsay was not asked about how she made instructional decisions. Scott. Scott was fairly diligent in following the pacing guide laid out within the curriculum and as a department, they tried not to stray from it very often. In the past, the teachers would want to review or teach the lesson in a different way if many of the students were struggling, but they realized this was not best for the students because if often led to them lecturing. Scott thought that even if students were confused at the end of a lesson, “there’s still a lot of benefit to that” (BI, lines 319-320). So, now the teachers “put [their] faith in the curriculum, even though we may not, uh, think it, the day, or the lesson went well” (BI, lines 323-324). Scott did think there was room to make minor changes if they were necessary, such as focusing only on the problems identified in the curriculum as core problems and not having time to work on the extensions problems. He also pulled the class back together for a discussion if he noticed that multiple groups were struggling on the same problem. Another strategy he used, that was suggested in the curriculum, was to pull one member of each team together and talk with those students and then send them back to their group to teach their peers. Scott thought this strategy worked well for students because there was “something about when, uh, a student or a team member explains it, just it’s easier to understand than the teacher explaining it” (BI, lines 351-352). He also determined the student he chose from each group in different ways. Sometimes he chose all of one group role, such as facilitators, whereas other times, he chose a student who was quiet or shy to force them to be a leader in their group. 185 Lukas. Lukas used verbal and non-verbal responses from students to make decisions about students’ level of understanding. He also used his experience with the curriculum to determine which topics would take longer than the textbook suggested, saying, “I know many lessons don’t expect mastery right away, um, but same time, I look, okay, we can’t just move on and hope that they’re going to pick this up from the homework questions later. We need to invest a little bit” (BI, lines 605-607). In addition, Lukas used his prior teaching knowledge to know with which concepts students would struggle. He then told students that they were introducing the topic today, but they would come back to it the next day, so it was okay if they were still slightly unsure after the first day because there would be more opportunities to make the necessary connections. He also believed that giving students a chance to sleep on the information was helpful and could help them make sense of the concepts. Analysis. All the teachers talked about making adjustments to their teaching based on evidence from their students. Much of this evidence was gathered through conversations with students, but sometimes it was based on students’ written work. Charlie and Scott both talked about identifying concerns while students were working and then pulling the class back together to address students’ needs. This was observed in both of their classrooms. Charlie stopped the class and called them back together, whereas Scott called up one student from each group to talk with them before sending them back to teach their teammates. True to his commitment to follow the textbook, Scott did not make any major adjustments to his instruction; instead he only made minor alterations like those describe above. Consistent with Charlie’s flexibility in the flow of his classroom, during our planning interviews, he often listed multiple tasks the students would complete depending on how they progressed through them. He had a rough idea of what he 186 wanted students to accomplish during the days I observed, but he was not committed to following a specific path. Differences based on observations. This was one of the most challenging formative assessment practices to observe because teachers often made decisions in their head and did not verbally tell students why they were doing what they were doing. These two examples show two different ways that teachers made instructional decisions. In Gwen’s case, it was a complete change in her lesson plan. For Charlie, it was planning the activities focused on students’ thinking and where they needed support to be successful on their test. Gwen achieved a Level 4 practice for instructional decisions when she scrapped her planned lesson to teach a different lesson because she found out most of her students would be absent the next day. She was explicit with her students about the reasons for the changes. This was described above in Gwen’s section. During the debriefing interview for that lesson, I asked Gwen about her decision to change the plan for class and she admitted that she had not been completely prepared because she had not officially planned for that lesson since she thought she could be teaching it the next day. However, she thought it was more important for her students to have experience with the lesson since she would not see them again before their unit test. The lesson did not flow as she had hoped because it was her first time teaching that particular lesson. Because this class is her last hour of the day, she is used to having taught the lesson multiple times before she sees these students, so this was a new experience for her. As part of the lesson, she had planned on teaching word problems, but she cut that part of the lesson while she was teaching it. When I asked her about this decision, she stated, “I wanted to make sure they were having success with the solving before I threw the word problem in too because I know that’s a struggle for a lot of them” (Debriefing Interview, May 25, lines 54-55). The fact that Gwen 187 changed her entire lesson plan for that class period in the moment and then also made changes during the lesson shows that she considered the impact of the lesson on her students when making instructional decisions. Charlie achieved a Level 3 for instructional decisions during the observed lessons. He had his students complete a goal check where they worked individually to reflect on the three unit goals. Students had to write concepts they needed to know in relation to each of the goals and then determine their level of understanding of each of the goals. After the students worked individually, Charlie led a whole class discussion where students reported out their ideas. This activity was also a review for the students’ upcoming test. Charlie was explicit with the students that the next activity where they would work with an assigned partner was to help them review for the test and focus on the concepts they need additional help on. He also intentionally assigned partners so that every pair had a student with a strong understanding and a student who needed extra help with hopes that the partners could support each other. He told the students while they were working to write questions on the board and they discussed those at the end of class, showing students that their understanding and questions guided the direction of the discussion and the intention of the activity. As shown in this and the previous chapter, Charlie was the only teacher who consistently demonstrated high-level implementation of multiple formative assessment practices. He also regularly used multiple practices within several different activity structures in individual lessons. In addition, he made explicit connections between different activity structures, so students might appreciate the flow and logic of the lessons, instead of participating in disjointed activities. In the lessons observed on June 1 and June 3, Charlie achieved Level 4 implementation for almost every formative assessment practice he used. These lessons represent the high-level practices 188 that Charlie often implemented during his lessons. The use of multiple practices throughout these lessons also makes the process of formative assessment more visible. The following chapter details the activity structures, formative assessment practices, and specific activities that Charlie implemented during these two lessons. The purposes of this chapter are to characterize “formative assessment practice” as more than the frequency of individual practices and to appreciate “high quality” formative assessment practice in terms of one teacher’s high-level implementation and the connections between his individual practices. 189 Chapter 7: Charlie’s High-Quality Formative Assessment Practice As shown in previous chapters, the analysis, practice by practice, indicated that Charlie’s use of formative assessment in teaching was more sophisticated than the other teachers in the sample. So, it makes sense to look more closely at his practice in the observed lessons. The objective of this chapter is to discuss how one teacher implemented high-level formative assessment practices in multiple lessons. The focus is on three consecutive lessons because between these lessons Charlie reached Level 4 for four of the six formative assessment practices and Level 3 for the fifth practice. (As previously noted, Charlie did not implement peer assessment in his observed lessons.) For each lesson, almost all the other formative assessment practices were scored at Level 3. The middle lesson is a slight exception because only half of Charlie’s practices were high-level. These lessons showed that it was possible to consistently implement many formative assessment practices at a high-level during a mathematics lesson. Although Charlie used the same activity structures in all three lessons, he implemented them in different ways. In the June 1 lesson, students were developing a stronger understanding of how to use permutations and combinations to find probabilities. The focus of this lesson carried into the next, on June 2. On June 3, Charlie provided students with the opportunity to reflect on what they had learned in preparation for their unit test. This lesson focused on wrapping up the unit. The June 3 lesson was the only observed lesson where Charlie focused the whole class time on students’ reflecting and reviewing for an assessment. In my analysis, I analyzed the relationship between the activity structures in each lesson and the formative assessment practices implemented during those activity structures. One goal was to bring the reader closer to the mathematics and the classroom activity than has been possible in prior analyses. Another was to see “high-level” formative assessment practices up 190 close. I also delved deeper into these three lessons to determine whether the particular lesson activities played a role in what formative assessment practices were implemented. Did the activities or tasks Charlie implemented in his classroom make formative assessment practices to be more visible? In addition, I considered how Charlie’s views of formative assessment were enacted in how he implemented these practices in his classroom as well as how he talked to students about their learning. The following sections describe the lesson activities and formative assessment practices Charlie implemented during the observed lessons. June 1 Lesson The Algebra 2 Honors class that was observed for this study was the second one of the day for Charlie, so he could make changes to the 4th hour lesson based on what happened in his 2nd hour class. These lessons occurred during the last week of the probability unit and towards the end of the school year. The students had taken a learning check (a quiz in Charlie’s classroom) related to permutations, combinations, and expected value. At the start of the June 1st class, Charlie handed back the learning checks, so students could see how their performance aligned with the three unit goals (as stated, I can find probabilities; I can use properties of probabilities; and I can find expected value.). The first goal required students to find probabilities in different situations, including using permutations and combinations. The second goal expected students to be able to use characteristics of probability, such as independence and mutually exclusivity to solve problems. The third goal wanted students to solve problems involving expected value. Based on students’ performance, Charlie realized that students were still struggling to use permutations and combinations correctly to find the probability of different events, therefore he focused the class period on providing students with more experience 191 working on these types of problems. He noticed that students still seemed unsure of when to use permutations and when to use combinations (Planning Interview and stated in class to students). Table 28 presents the activity structures in Lesson 1, the time Charlie spent on each, the formative assessment practices he implemented, and their level of sophistication. It also provides a more detailed description of the lesson activities than given in previous chapters and the mathematics involved. This table provides a deeper description of what was happening during the lesson than what was shared in previous chapters. Table 28: The June 1 Lesson Time (min) 3 4 3 21 Activity Structure Introduction to class/lesson • Presented the day’s agenda • • • Related these questions to students’ work on the Identified learning goals and upcoming assessment. Identified specific two questions as focus for the day learning check Individual activity • Students focused on one question from the learning check (finding the probability for different situations related to students in a class having one of three jobs) • Teacher’s purpose seemed to be to get the students thinking before working in small group • Asked students for answers and explanations to the Whole class discussion question above. Pair/Small group work • Purposefully assigned students to partners. • Partner work on the question above to develop multiple • Monitor partner work, query pairs, and answer solution strategies. questions. Formative Assessment Practices Learning Targets (Level 3) Feedback (Level 2) Instructional Decisions (Level 4) Questioning (Level 2) Questioning (Level 3) Feedback (Level 2) Questioning (Level 2) Feedback (Level 3) 192 Table 28 (cont’d) 22 Whole class discussion • Asked students for answers and explanations from partner work. • Discussed multiple solution strategies (e.g., writing as a • single fraction and using combination notation) Identified common mistakes (e.g., reporting probabilities > 1 and combining multiple notations) from the learning check • Connected discussion to current assessment • Shared multiple strategies students had used on the learning check that he thought were acceptable (e.g., finding permutation of one job plus permutation of other job or using multiplication) • Discussed additional problem about expected value • Provided feedback to the class on rest of learning check Questioning (Level 3) Feedback (Level 3) Questioning (Level 2) Feedback (Level 4) 8 Individual activity assignment. • Students had the rest of class to start on their • Monitored the class and checked in with students. During this lesson, Charlie achieved high-level practice (Levels 3 or 4) for four of the formative assessment practices (learning targets, questioning, feedback, and instructional decisions). Charlie was the only teacher who consistently demonstrated high-level formative assessment practice, achieving high levels in multiple practices for every observed lesson. In this lesson, Charlie reached Level 3 for learning targets because he started the lesson with an agenda that explicitly included the learning targets. By contrast, other teachers in the study typically did not mention the learning targets for the day, even though all the teachers had them posted in their classrooms. He told his students that many had struggled with one particular problem on the learning check, so he wanted to spend some time in class focusing on that problem and its related learning goals. The problem stated that there were 23 students in a class and there were three jobs in the classroom. The students had to determine the probability that 1) a particular student had a job, 2) the particular student was line leader, 3) the particular student was line leader and his friend was the teacher helper, and 4) the particular student was the teacher helper or the 193 problem solver1. He also told them they would review an additional problem (using expected value to solve a lottery problem) from the learning check later in the class period. Because his plan for that day’s class was based on evidence of students’ lack of understanding of the mathematical concepts, Charlie was evaluated at Level 4 for instructional decisions. He was explicit with the students on what they were doing for that day and why he had chosen those specific questions from the learning check as well as the lesson activities. After framing the lesson in this way, he then gave the students time on their own to think about how they solved the first focus problem and to consider an additional way to solve the problem (four minutes of Individual activity, Table 28 above). During the whole class discussion that followed (three minutes, Table 28 above), Charlie routinely asked students “Why?” when they shared a solution. Because he pushed students for an explanation, instead of just accepting a solution, he achieved a Level 3 for his questioning practices. When the class moved to pair work, he assigned partners based on students’ work on these tasks. He wanted to ensure that students who were struggling would have a partner who had demonstrated understanding of the concepts, so they would have a better chance of getting clarification on their confusion (Debriefing Interview). Charlie stated that he thought the pairings were successful stating, “overall I thought they were good. Like most of the groups when I came, they had a group question not an individual question, which was good. They pointed at both of their papers as they were talking which I thought was good” (Debriefing Interview, lines 248-250). During the paired work (21 minutes of Pair/Small group work, Table 28 above), Charlie monitored the students’ work and asked questions when necessary. As in the whole class 1 This is a paraphrase of the actual problem text. 194 discussion, he asked students to provide explanations for their solutions. He also asked them to explain their solution and consider what they would do in other contexts, using questions such as, “How would you allow that to be in any order?,” “How do you take the order out?,” and “How do we also do that by using combinations?” Charlie thought it was important for students to have a chance to discuss the mathematics with a peer, so he allowed about one-third of the class time to be spent working in pairs. Providing different avenues for students to share and hear ideas is an important part of Charlie’s overall teaching practice—including individual think time, an opportunity to work with a peer, and a whole class discussion (Planning Interview). Mathematics education research also agrees that providing students with opportunities to share their ideas in a whole class discussion is a valuable use of class time (Smith & Stein, 2011). During the second and longer Whole class discussion (22 minutes, Table 28), Charlie repeatedly asked the students to provide explanations. Also, when he worked part of a problem himself, he always asked the students to provide justification for the step. Some representative questions were, “Why isn’t it 23?” and “Why aren’t I multiplying in this case?” This lesson occurred on a Wednesday and the students had a take home learning check due on Friday related to expected value. Charlie made a point of connecting the questions they were discussing during class to the take home learning check to help students recognize the similarities in the concepts. Not only was Charlie’s questioning practices a Level 3, but his use of feedback was also coded as a Level 3, as he provided descriptive feedback that focused on the task, and on students’ processes in solving the task. An example of task-focused feedback was his feedback to the whole class about what he noticed about their work on the learning check. He worked through the first part of the problem with the class to find the solution, then while he was showing another way to solve the problem, 195 he identified student mistakes he had seen on that part of the problem. For example, some students had probabilities greater than one, which he told them that was “a big problem”. Other students set up part of the problem but did not completely solve it, so he told them what else they needed to do to finish the problem or correct their mistake. After completing all four parts of the problem, Charlie told students he wanted them to be flexible in their thinking because some of them thought they had to use a specific method to solve this problem, which was not the case. Their focus on only using permutations seemed to hinder their ability to solve the problem, so Charlie wanted them to consider different possibilities for how to solve the problem, such as using trees, Venn diagrams, or tables. During the debriefing interview, Charlie mentioned that this part of the lesson took much longer than he had expected. However, he allowed the time for it because he knew it was important for the students’ understanding of the concepts. Charlie ended the class by giving students time to start working on the next assignment (8 minutes of Individual activity, Table 28) which would also give them more practice finding probabilities using permutations and combinations. Because he noticed that students were struggling to decide whether to use permutations or combinations, for this next assignment, all the problems were written so that they could be solved via permutations. After students had solved every problem, Charlie wanted the students to change the problems so that they would require combinations. He thought that telling students to use permutations to start with would help them make sense of the problems and allow them to focus on how to solve these types of problems (Planning Interview). Then if the students had to change the problems to make the combination problems, it would help them see the difference between permutation and combination problems (Planning Interview). He also told the class that he thought it would be easier for them to decide if they need to use a permutation or a combination. 196 During this Individual activity, Charlie was scored at Level 4 for feedback, as he provided descriptive feedback on the students’ process as well as the task itself. His feedback also included steps to move forward that required them to think about what to do next. He had multiple conversations with students about their specific work on the learning check, assignment, or the take home learning check. He provided descriptive feedback related to the strategies they used to solve the problems as well as the problems themselves. But I was unable to view students’ work, therefore, I, nor the camera, could see how students were solving the problems. The only evidence is the conversation between Charlie and the students. Instead of telling students how to solve the problems, I observed that he guided them in making the decision themselves. For example, in a conversation with a student about his take home learning check (the take home learning check related to the lottery), Charlie told the student that he was on the right track, but he needed to make changes, using questioning and feedback to help guide the student. Charlie was the only teacher who achieved a Level 4 for feedback and he only did it once, suggesting that this type of feedback is difficult for teachers to provide. During our debriefing interview, Charlie stated that, “much more of the lesson was on the learning check than I had initially intended, but they needed more time” (lines 2-3). The fact that Charlie allowed the students the necessary time they needed to make sense of the problems, even though it went against his plan for the class is evidence that this lesson was structured around students’ understanding. Based on the different interviews I conducted with Charlie and observing his classroom, it was clear that gathering evidence from his students and making decisions based on that evidence was an important part of his classroom practice. I generally did not see this in the other teachers in this study, many of whom followed their textbook pacing guide regardless of students’ level of understanding during the implementation of the lesson. 197 June 2 Lesson This lesson was a continuation of the June 1 lesson. Charlie had different tasks for students to work on to continue developing their knowledge around using permutations and combinations to find probabilities. The focus of the class shifted at the end when students received a review assignment to help them prepare for their unit assessment. Table 29 shows the amount of time Charlie spent on each activity structure and which formative assessment practices, including levels of sophistication he implemented during each lesson segment. The table also provides a detailed description of the activities in the lesson. Table 29: The June 2 Lesson Time (min) 3 Introduction to class/lesson Activity Structure Formative Assessment Practices Learning Targets (Level 1) Instructional Decisions (Level 2) Questioning (Level 4) Feedback (Level 3) Questioning (Level 2) Feedback (Level 3) Whole class discussion • Presented an agenda for the day. • Discussed homework assignment and work on additional task related to permutations and combinations. • Reminded students of their upcoming assessment and that their take home assessment was due the next day. • Made a passing mention of what the goals were. • Showed the answers to the homework assignment and provided explanations for some of the problems. • Asked questions back to the class when students asked questions. • Connected problems to real life (e.g., asked students in what card games the order would matter) • Focused second part of discussion on how students would change the problems to make them combination problems. Pair/Small group work • Students chose their own partners. • Students completed probability problems using permutations and combinations. • Monitored the class, checked in with students, and answered questions. 198 17 28 Table 29 (cont’d) 10 Whole class discussion • Displayed the solutions and provided explanations for some of them. Other times he asked students for explanations. • Provided suggestions to help students not make common mistakes (e.g., the numerator is not always 1, work through the numerator and denominator separately) • Students started working on a review for their upcoming Questioning (Level 3) Feedback (Level 2) None 3 Individual activity assessment. In this lesson, Charlie achieved high-level practice for two of the formative assessment practices (questioning and feedback). Although he started the lesson with an agenda, Charlie made only passing mention of the learning goals, which were the same goals as the previous lesson. However, like the previous day, Charlie was clear on what was happening during the class period, including telling students that he thought they needed more work with permutations and combinations, so they would spend that day working on an additional task focused on those concepts. After introducing the plan for the day, Charlie led a Whole class discussion (17 minutes, Table 29) over the homework, which asked students to solve problems using permutations and then change all the problems to require combinations. During this discussion, Charlie achieved high-level practice for both questioning (Level 4) and feedback (Level 3). To reach a Level 4 for questioning, Charlie had to ask a mix of low and high-level questions, with more questions requiring students to demonstrate higher level thinking. In addition, students needed to explain their thinking and Charlie had to use questioning to push them to extend their thinking. More than half of Charlie’s questions during this discussion were scored as high-level. He asked questions such as, “What’s different about this problem compared to the last one?”, “Why would we multiply those instead of add?”, and “How would that make a difference?”. 199 The nature of the assignment also enabled Charlie to extend the students’ thinking because for every problem, students were expected to change the problem to require combinations instead of permutations. This pushed the students’ thinking to a new level because they had to use their knowledge of permutations and combinations in order to change the problem to require a different mathematical idea. Charlie also asked the students to make a connection between the mathematical concepts and real-life by asking them in what card games order would matter and to provide justification for their choice. In the Whole class discussion, Charlie was scored at Level 3 for feedback, as he provided descriptive feedback on the task and the students’ processes. He provided this type of feedback throughout the second half of the discussion when they focused on changing all the permutation problems to combination problems. For example, the first problem on the task asked the students to determine how to rank the best drummer from a group of ten bands. One student suggested not having there be an order. Charlie asked him questions to help him think through what that would look like and then student stated that they could just put the best drummers in a group without assigning a ranking. Another student then asked whether ranking them within that group would be a combination, but Charlie clarified for him that ranking is what makes it a permutation. The student struggled to explain his idea and Charlie gave him time to think and asked him questions, such as “What do you mean?” and “Do you have an idea of a way to reword it?” to help him clarify his thinking. He worked with the student to help him make sense of the mathematics, instead of moving on and leaving the student confused. Almost half of the class time (28 minutes of Pair/Small group work, Table 29) was spent having students work with a partner or small group on a permutation and combination task. Although his questioning was scored at Level 2, as he asked a mix of high and low-level 200 questions, but more low-level, Charlie asked some high-level questions that are worth noting. As in previous lessons and activities, Charlie consistently asked students to explain their answers. There were multiple times where he asked students how they would do the mathematics of the problem (“How do you calculate that?”). He also asked students, “Why are you doing 30 choose 5?” During this Pair/Small group work, Charlie achieved a Level 3 for feedback. While students were working with their partners, Charlie monitored the classroom, asking questions, and checking in with students. He often provided feedback through questioning, by asking students to explain their answers. In doing this he gave them the opportunity to think about their strategies. At times, Charlie would start by providing evaluative feedback because students often asked if their answer or process was correct. But he then asked questions about the students’ mathematical processes. For example, in one case, Charlie noticed in their work that they had found the denominator and had not found the numerator. He talked with the pair of boys about their solution strategy and what they needed to do to correct their mistake. They connected this current problem to those on the take-home learning check that was due the next day. At the end of the conversation, Charlie stressed to them that they needed to think about the numerator and denominator separately. During the debriefing interview, Charlie said that he was surprised by how helpful the task was in providing him information about students’ understanding. He stated, “I liked those questions more than I had initially planned on liking those questions … In terms of what they did and what I got out of it in terms of watching them do that” (lines 57, 59). During the second Whole class discussion (10 minutes, Table 29), Charlie reached a Level 3 on questioning. The first six questions Charlie asked during the discussion started with “why”, which shows his consistency with having students explain their thinking and not just 201 provide an answer. He asked questions such as, “Why is that a combination problem?”, “Why would there be 60?”, and “Why do I have to divide by the 10?” He also asked many “how” questions, which also required students to explain the mathematics. Examples of these questions included, “How do we actually calculate this combination?” and “How did I get 9?”. The last question that he asked was “Did anybody write this differently?”, which is an example of how Charlie was focused on finding multiple solution methods to the problems. He mentioned multiple times to his students in the observed lessons and to me during our interviews that he wanted students to be flexible in their thinking, so he focused class time on having students share different solution strategies. Charlie and Lukas were the only teachers who saw this need for students to share multiple strategies as an important part of their mathematics classrooms. Both teachers also saw a connection between this practice and formative assessment as described in the previous chapter. At the end of class (three minutes of Individual activity, Table 29), Charlie gave students the unit assessment review to start working on. He let students know that they would be spending the next class period focusing on reviewing for their assessment, but they should complete at least part of the review before coming to class. He wanted them to determine which problems they needed help or practice on, so they would be prepared to ask questions in class the next day to ensure they were ready for their assessment. Charlie was pleased with how students seemed to be progressing in their understanding of permutations and combinations. He stated, “more of the kids were able to correctly identify combinations and permutations and then once they did that, most of the kids by the time they got to the end they were able to just write the formula out in one of the ways. The other thing is, I saw more ways they were writing it. Like different kids were writing it in multiple ways than 202 what we were doing before” (Debriefing Interview, lines 70-74). This showed that he had been gathering evidence of students’ understanding and the strategies they were using throughout the lesson. During multiple conversations, Charlie demonstrated that he had paid attention to what students were doing during individual and pair/small group activities to help him gather information about what they understood and where they were still struggling. Then he used this information to make decisions about what activities he would have students complete during the next class period. In contrast, many of the other teachers in the study followed their curriculum pacing guide very closely and did not change plans based on students’ understanding or lack of understanding of the mathematics concepts. Often other teachers indicated they would discuss the topics again later since the concepts built on each other, so students would have additional experience and therefore they did not need to spend an extra day on the topic. June 3 Lesson The focus of this lesson was on students’ self-assessing in relation to the three unit goals (as stated, I can find probabilities, I can use properties of probabilities, and I can find expected value.) in preparation for their unit assessment. Students then spent the rest of the class period working together on their review assignment, focusing on the areas they needed the most help. Charlie ended the class with a whole class discussion about the questions and concerns students still had about the content, so they would feel prepared to take their unit assessment the following Monday. Table 30 shows the amount of time Charlie spent on each activity structure and which formative assessment practices, including levels of sophistication, he implemented during each lesson segment. As before, the table also provides a detailed description of each lesson activity. 203 Table 30: The June 3 Lesson Time (min) Activity Structure Introduction to class/lesson • Presented an agenda for the day. • Reminded students of their upcoming assessment. • Made passing mention of the goals for the day. Individual activity • Students are working individually to self-assess related to the three goals of the unit. • Monitored the class, checked in with students, and answered questions. Whole class discussion Formative Assessment Practices Learning Targets (Level 1) Learning Targets (Level 3) Self-Assessment (Level 4) Questioning (Level 3) Learning Targets (Level 4) Self-Assessment (Level 3) Questioning (Level 2) Instructional Decisions (Level 3) Questioning (Level 2) Feedback (Level 3) Questioning (Level 2) Feedback (Level 2) 1 11 8 26 14 • Summarized what he heard from students. • Asked students to list things they needed to know to show proficiency for each goal (e.g., permutations, combinations, mutually exclusive). • Reminded students that during the review time (the next activity) they needed to focus on the areas where they were struggling. Pair/Small group work • Students chose their own partner(s) • If they had a question or wanted to see a problem worked out, then they could write that number or topic on the smartboard. • Monitored the class, checked in with students, and answered questions. Whole class discussion • Discussed the different questions/concepts students had written, including definitions of terms. • Connected back to a previous activity the students had done to help answer a question. In this lesson, Charlie achieved high-level practices for all five of the formative assessment practices that he implemented (learning targets, questioning, feedback, self- assessment, and instructional decisions). Charlie was the only teacher who reached high-level practices for five formative assessment practices within the same lesson. The introduction for this lesson was very brief (one minute of Introduction to class/lesson, Table 30) where Charlie 204 showed the agenda for the class period, reminded students of their unit assessment on the following class day, and made a brief mention of the learning targets. During the Individual activity (11 minutes, Table 30), all three of Charlie’s formative assessment practices were scored at high-levels, with learning targets and questioning at Level 3 and self-assessment at Level 4. For this individual activity, students self-assessed in relation to the three learning goals for their current unit. Charlie called this self-assessment “goals writing.” He stated that he has students complete this type of exercise two or three times a unit and often does it before an assessment (Debriefing Interview). For this activity in particular, he had students do the goal writing with the hope that they would approach their review differently. He wanted them to think critically about the goals before completing the review, so they would focus their attention on the concepts they need the most help on. He stated, “I wanted the review, to be, seen as something that was made to be helpful to them as opposed to an assignment I made them do” (Debriefing Interview, lines 287-288). On the screen, he displayed the three goals (I can find probabilities, I can use properties of probabilities, and I can find expected value). For each, he wanted students to “1) explain what it means, 2) explain what things you need to know in order to show proficiency on that goal, and 3) identify what things in the list for 2 that you do well and which you need to still work on for our test on Monday”. This activity was scored as a Level 3 for learning targets because Charlie focused specifically on the learning targets and what knowledge or skills students needed to master them. While students worked, he monitored the class, checked in with students, and answered questions. He only asked three questions during this activity, but two of them were high-level, therefore this practice was coded as Level 3. He asked questions, such as, “If I say 205 you need to find probabilities, what do you need to do to do that?” and “If you’re looking at Question #1, what’s that about?” Charlie reached a Level 4 for self-assessment because the activity was designed specifically to provide students with an opportunity to self-assess in relation to the learning goals. He also provided students with an explicit structure (the three questions) that directed and focused their work. During the Whole class discussion (8 minutes, Table 30), Charlie scored highly on all three of the practices he implemented (learning targets, self-assessment, and instructional decisions). He used questioning as well during this activity structure, but his practice was evaluated at a Level 2. In the previous activity, Charlie reached a Level 3 for his learning target use. During this activity, students were required to list the knowledge and skills necessary to master each of the learning goals, so his use of learning targets was coded as Level 4. The fact that students had to make sense of the mathematics needed to achieve these learning goals is what led this use to be evaluated at a higher level than in the previous activity structure. Charlie was the only teacher who reached Level 4 for his use of learning targets, achieving this twice during the observed lessons. Since the Whole class discussion focused more on the combined observations of the students, Charlie’s use of self-assessment was coded at Level 3 and not Level 4 like in the previous activity structure. Although this discussion is still focused on the learning targets and what students need to know to be successful on their assessment, the focus is more on the whole class and not as much on the individual student, which led to it being coded at a lower level. 206 During the discussion, Charlie asked many questions, however most were low-level. He started the discussion by stating what he had heard from students while he was monitoring the class, which was that the goals were interrelated. He agreed that this was the case but wanted students to be as specific as possible when listing the skills and knowledge needed to be successful for each particular goal. For the first goal (I can find probabilities), students suggested trees (that is, tree diagrams), Venn diagrams, analyze data, union, intersection, conditional probability, tables, combinations, and permutations. Charlie stated that “analyze data” could be a top-level heading with everything else listed as ways to analyze the data. For the second goal (I can use properties of probabilities), students stated independence, complementary, and mutually exclusive. Charlie stated that many students put the conditional probabilities in this category and that was fine as well. He reminded students that these two goals were what the unit assessment would assess. Their take home learning check addressed the third goal, therefore they needed to focus their attention on the first two learning goals. For the third goal (I can find expected value), students suggested expected value, which led Charlie to ask students what expected value was. During this discussion, Charlie reached a Level 3 for instructional decisions because he was explicit with the students that the following activity tied directly to what they had just done and what they needed to work on to improve their understanding. The students would work with a partner or small group to work on the review assignment they received at the end of the previous class period. They needed to focus on the problems they had questions on and not necessarily just work through the whole review. If students had additional questions, then they could write those problem numbers on the board or write down mathematical concepts they needed more explanation on. Charlie created this review activity specifically for these students. 207 He stated that he chose the problems on the review assignment based on how students performed on the previous learning check (Debriefing Interview). For the Pair/small group work (26 minutes, Table 30), students chose their own partner(s). Charlie implemented two formative assessment practices (questioning and feedback). Feedback was scored at a high-level (Level 3). Although Charlie asked a lot of questions during this activity, he asked very few high-level questions, therefore his questioning practice was evaluated at a Level 2. Some of the high-level questions he asked included, “Do you know why that happens?” and “What’s it asking about?” While students were working, Charlie monitored the groups and checked in with them. Students went to the board at different times to write problems or concepts they needed more help on; one student wrote most of the items on the board. Charlie thought it was important for the students to write their questions on the board, so they could be more active in the lesson, which helped him gather information about their thinking. He stated, “it gives me a better idea in terms of what they’re doing” (Debriefing Interview, line 24). During this activity, some students asked Charlie about the take-home learning check they had just turned in. This is where he achieved a Level 3 for feedback. One student asked Charlie to explain part of the problem. As he did, she stated that she had completed the problem that way but kept getting 1.9, which she did not think made sense. Charlie thought that was strange as well and told the student he wanted to look at what she had done saying that was “a different kind of error”. He went to his desk and pulled out her paper and sounded shocked (making sounds of shock and saying “you weren’t kidding”) when he looked at her paper. He first stated that she had incorrectly set her equation equal to one, but she explained that was a previous attempt she had made on the problem. Charlie continued asking her questions to see if 208 he could make sense of what she had done. Through their conversation and looking at her work, he reasoned that she had forgotten to multiply, so he allowed the student to make corrections to her paper. Because he asked questions and helped the student reason through her work to find her mistake, this exchange was coded as Level 3 feedback because it was descriptive to her solution and strategy. When giving students feedback, Charlie regularly used questioning to help guide students in making sense of the mathematics and their solution strategies. Another example of Charlie’s high-level feedback occurred when a student asked for clarification on when to use permutations and combinations. First Charlie asked a clarifying question to make sure he knew what the student needed, asking, “Permutations versus combinations or permutations and combinations versus a tree?”. The student stated that she knew the difference between permutations and combinations was whether order mattered or not but wondered if there was another method, because sometimes she found the context of the problem confusing. Charlie focused her attention on specific problems on the review to help her think about the difference and asked her questions about how she would solve them. The student then connected the mathematical ideas back to an activity they had completed a few days prior (the problem discussed in the June 1 lesson). This exchange was coded as high-level because his feedback focused on the specific problems the student was solving and connected to her prior knowledge. During the second Whole class discussion (14 minutes, Table 30), Charlie reached Level 2 for both formative assessment practices he implemented (questioning and feedback). His questioning was a Level 2 because only one-fourth of his questions (four out of sixteen) were high-level, however, he did ask two questions that are noteworthy. The first one was, “Can somebody maybe say more about what this meant?”, which was asking the students for 209 clarification. Someone had written expected values on the board as a topic they wanted more information about, so Charlie asked this question, so he would know how to help the students. Another question was, “What about the information that is given says a tree makes sense here?”, which required the students to think critically about the problem to determine how making a tree would be an appropriate solution path. This question related to a specific problem on the review, which asked about a student playing tennis and the different probabilities of their serve going in or out. Charlie appreciated the students asking questions about specific math concepts and not just asking him to work different problems on the review (Debriefing Interview). What they wrote on the board provided him different information about their thinking and understanding. He stated, “I thought the question they had on the board of like, not like how do you do number 3, but what’s the tree for number 3 was like a different level of question. Um, and so I appreciated that that was sort of where they got to” (Debriefing Interview, lines 52-54). Since his feedback was evaluated as a Level 2, this means that it was mostly evaluative and either did not provide the students with a clear path on how to move forward or was more directive on how they should solve the problem. When Charlie explained the different topics students had written on the board, he mostly told them what solution path to use, which meant he did most of the thinking for the students instead of having them determine the solution paths. This was a rare occurrence in Charlie’s classroom; he often put the thinking on the students. During the Debriefing Interview, Charlie talked about the growth he had seen in his students over the previous few days in their understanding of probability. He stated, “one of the girls turned in essentially a blank learning check less than a week ago and was able to do this entire paper” (lines 131-132). He also noted that he thought “80% of them are proficient at that [permutations and combinations] right now and another out of that like you know 50% are 210 expert” (lines 200-201), so he was feeling much better about how they would perform on their unit assessment. Summary This chapter provided a deeper look into Charlie’s formative assessment practices and how he was able to implement multiple formative assessment practices within his lessons. It also showed how he routinely gathered evidence of his students’ understanding or lack of understanding and used that information to design tasks that would support students’ mathematical learning. This chapter discussed specific lesson activities that Charlie implemented that allowed him to incorporate high-level formative assessment practices into his lessons. In addition, this section compares Charlie’s formative assessment practices to other teachers in the study to show how his use was different. Charlie regularly chose the lesson activities intentionally based on students’ understanding of the mathematics content. The tasks students worked on during the June 1 lesson were based on how they performed on their previous learning check. He used the evidence he gathered from students on the subsequent lessons to make decisions about what tasks and activities students would complete during class. This demonstrated how he regularly made instructional decisions based on students’ understanding. Not only did he make clear instructional decisions during each of these three lessons, but also five of his six observed lessons included this formative assessment practice at some point during the first half of each lesson. This demonstrates Charlie’s consistency in making decisions based on his students’ level of understanding. Charlie also used multiple activity structures in each of his three lessons to provide different types of interactions between students and between himself and students. Almost all his 211 lesson included individual activity, pair/small group activity, and whole class discussion. As discussed in previous chapters, these activity structures provided opportunities for teachers, not just Charlie, to implement multiple different formative assessment practices. Questioning and feedback, in particular, are two formative assessment practices often implemented during these activity structures. For all the teachers, these two practices were most often implemented in the same activity structure. For example, Charlie and Evan implemented them during whole class discussion. Lindsay and Scott used Pair/Small group work to ask questions and provide feedback. In contrast, activity structures such as interactive lecture, teaching providing information, and summary provided few opportunities for teachers to incorporate formative assessment practices. However, Gwen is the exception here because she most often used interactive lecture to ask questions and provide feedback to her students, which is not surprising since that was the activity structure she most often implemented. Within these different activity structures, Charlie tended to locate the cognitive work in the students while providing appropriate levels of support. By providing students individual think time and opportunities to work with a partner or small group, he required students to take on most of the mathematical thinking in the lesson. This was visible across all his observed lessons. He required students to think critically about the mathematics they were learning. This led to students making connections on their own between their current work and tasks they had completed in the past. His interactions with them during these activity structures, whether that was through questioning or via feedback, allowed them to develop their mathematical knowledge. Charlie regularly implemented multiple formative assessment practices within activity structures, and often they were at high levels. This demonstrated that it was not only possible to 212 use different formative assessment practices within the same lesson activity, but it was also possible to implement them with high levels of sophistication. Although many of the other teachers implemented multiple formative assessment practices within their activity structures, they were not at the frequency or level of sophistication seen in Charlie’s practice. Even within specific formative assessment practices, such as learning targets, Charlie showed it was possible to seamlessly incorporate these into his lessons, using an agenda to start the class period. Evan was the only other teacher who used an agenda to start class and he also had the learning targets listed. Both teachers made the learning targets front and center for their students from the beginning of the lesson during the Introduction to class/lesson. As mentioned in a previous chapter, all the teachers were required to have the learning targets posted in their classrooms, but only Charlie and Evan regularly verbalized them to their students. In contrast, other teachers posted them, but never stated them to the students, suggesting that students were expected to read the learning targets themselves and determine how they connected to their lessons. Charlie and Evan both made learning targets a priority in their classrooms but used them very differently. Although Evan used learning targets slightly more often (9 instances compared to 8 for Charlie), most of his were scored at low levels (Levels 1 or 2). Charlie, however, achieved high level practice for most of his learning target use (5 out of 8 instances). Charlie also had more of a range in learning target use compared to Evan in that he achieved all four levels of sophistication whereas Evan never reached Level 4 in his use of learning targets. In contrast, Evan was more likely to discuss the learning targets multiple times in an individual lesson. During two of his observed lessons, he mentioned the learning targets, to varying degrees, within three activity structures (once within consecutive activity structures and once spread out 213 throughout the class period). This shows that in those two lessons, Evan was more deliberate about reminding students throughout different activity structures what the goals of the lessons were. Although Charlie discussed the learning targets multiple times in two lessons (May 10 and June 3), he did so only twice during one lesson in non-consecutive activity structures (May 10) and in the other (June 3) in three consecutive activity structures where the focus of the activity was self-assessment. Both teachers demonstrated that was it possible to share the learning targets with the students in meaningful ways throughout the lesson activities. Questioning was one formative assessment practice where Charlie’s use stands out as different and more productive than the other teachers. Charlie often used questioning to explore alternate methods of a solution. During whole class discussions, he regularly asked students to explain how else they could solve a particular problem. Charlie stated that he wanted students to be flexible in their thinking and to determine different methods for solving the mathematical problems (multiple interviews). He also stated he would be concerned if every student solved a problem in the exact same way: “If everybody’s doing the same strategy, I did something wrong, cause I just pushed every kid to do the exact same thing, when maybe that works well all the time and maybe it doesn’t” (Exit Interview, lines 381-383). When originally asked if this teaching practice related to formative assessment, he stated, “I’ve never thought about that. Um, so by formative assessment definition no, cause I’m not doing it on purpose right” (Exit Interview, lines 348-349). When pressed to explain what he meant by he was not doing it on purpose, he stated that he was not using it as an assessment strategy. As we worked to unpack his thinking about how he used multiple strategies, he changed his mind and determined that it was formative assessment because it related to providing students with feedback on their mathematical thinking. 214 Lukas also incorporated the discussion of multiple strategies into his lessons, but unlike Charlie, did not use questioning as the method. Sometimes during his Whole class discussions Lukas would have students share an additional solution strategy. However, most of the instances occurred during presenting student work where students would come to the board to work problems in front of the whole class. During this activity structure, he would have multiple students come to the board to show different methods for solving the same problem. Like Charlie, Lukas also thought this teaching practice related to formative assessment because it allows students to see alternative mathematical strategies. 215 Chapter 8: Supports and Obstacles to Implementing Formative Assessment Practices This chapter addresses the third research question: What do some high school mathematics teachers see as supports and obstacles in implementing formative assessment practices in their classrooms? The teachers were asked what they saw as helping or hindering them in being able to implement the different formative assessment practices, including writing learning targets, determining questions they would ask students, providing feedback, self- assessment, peer assessment, and making instructional decisions. First, I will discuss the supports that teachers talked about and then I will discuss the obstacles that teachers shared. Supports in Implementing Formative Assessment The supports that teachers talked about were categorized into two initial categories; internal and external. Internal factors related to teachers’ beliefs and views of teaching, learning, and themselves. External factors related to aspects outside the teacher, such as resources or school contexts. Table 31 shows how many of the teachers’ supports were considered internal or external and for which formative assessment practice they occurred. 2 2 0 Internal External Table 31 Supports in Implementing Formative Assessment Total Writing learning goals Determining questions Providing verbal feedback Providing written feedback Having students self- assess Having students peer assess Adjusting instructional plan Total 8 47 6 9 0 3 7 7 9 0 0 3 5 5 9 9 5 6 8 11 54 216 As shown in Table 31, most of the supports that teachers talked about were classified as external supports. Only three of the seven formative assessment practices (learning goals, questions, instructional decisions) had internal factors that supported teachers. The rest of this section discusses specific supports teachers named related to each of the above formative assessment practices. Since there were a substantial number of external factors mentioned by teachers, these supports were classified into eight sub-categories with common themes (Table 32). See Appendix F for definitions of each sub-category. The sub-categories represent the columns while the formative assessment practices are the rows. The most common external support teachers named was resources for teachers, which could be other people or materials. The next most common was students, which related to their students’ actions or behaviors in class. Table 32 External Supports to Implementing Formative Assessment Class Norms Content Logistic Resource for Teachers Resource for Students School Student Time To -tal 0 0 1 0 1 0 0 0 0 0 0 0 1 1 0 2 6 2 3 0 217 0 0 0 0 3 0 0 0 0 0 0 1 4 1 2 1 0 1 0 0 3 7 9 5 6 FA Practice Writing learning goals Determ- ining questions Providing verbal feedback Providing written feedback Having students self- assess Table 32 (cont’d) Having students peer assess Adjusting instruct- tional plan Total 1 5 1 1 0 2 3 16 1 6 2 3 0 11 0 2 8 46 2 0 0 0 2 1 3 0 8 Writing learning targets. Teachers named two internal supports for writing learning targets. Charlie identified that having a clear idea of where the lesson needed to go helped him write learning targets. Lukas named the other internal support: having prior teaching experience with the content made writing learning targets easier. Teachers identified three external supports: two Resources for Teachers and one Time. Charlie, Evan, Gwen, and Lindsay all stated that having other teachers to work with was helpful in writing learning targets. Gwen, Lindsay, and Scott all mentioned that their textbook helped them in determining the goals for their lessons. Evan stated that having 90 minutes for his planning time helped him write learning goals. Determining questions. Determining what questions to ask students was associated with one of the highest number of supports (N=9): two internal and seven internal supports. The internal supports were having prior teaching experience, which was mentioned by Evan, and self-awareness, mentioned by Lindsay. She stated, “I’ve gotten a lot more aware of the questions that I’m asking” (BI, lines 635-636). Teachers named seven external supports for determining questions: six Resources for Teachers and one Students. Charlie, Evan, and Lukas all stated that professional development opportunities had been helpful to them in determining the questions they asked students. Scott 218 mentioned that the lesson study he and his colleagues were participating in had been helpful to him. Similarly, Charlie mentioned that seeing other teachers who are good at asking questions helped him think of the kinds of questions he would want to ask his students. Evan and Lindsay viewed their textbooks as a helpful resource in determining questions for their students. Gwen reported using additional resources outside of her textbook, and Lindsay reported using research to help her identify appropriate questions for students. Lukas mentioned the lone support related to students, asking questions to small groups of students, which he thought he was better at as compared to asking questions of the whole class. Feedback. Teachers did not mention any internal supports when discussing their provision of written or verbal feedback to students; however, they mentioned 14 external supports across the two types of feedback. Verbal feedback included five of the sub-categories and written feedback included three sub-categories. There were three sub-categories in common between verbal and written feedback; these included Logistics, Resources for Teachers, and Students. Evan thought that having an extended class time (90 minutes twice a week) provided him with more opportunities to give his students verbal feedback. He also believed that giving students time in class to work allowed him opportunities to walk around during class and talk with students; therefore, he could reach more students during this time than would have been possible with a shorter class period. Lukas stated that since verbal feedback is done in the moment and has a quick turnaround, it helped him use this type of feedback in his classroom. Scott said that his textbook and the lesson study he was participating in with the other Algebra 1 teachers helped him provide verbal feedback to his students. His textbook had suggestions within the teacher’s edition for how to respond to different student strategies and talking with his 219 colleagues also gave him ideas for thinking about students’ work. Charlie, Gwen, Lindsay, and Lukas all identified supports related to students’ behaviors or actions. Charlie thought being able to talk one-on-one with students was helpful because he could provide them specific feedback to improve their learning. He also reported “it definitely changes the dynamic, there’s a back and forth you can talk to them” (BI, lines 936-937). Gwen appreciated her students’ willingness to talk to her. Her students were comfortable coming to talk with her about their performance, which allowed her to provide verbal feedback to them. Similarly, Lindsay thought the fact that her students were receptive to her feedback allowed her to share her thinking with students. Lukas thought providing verbal feedback was a strength of his because he could read students’ body language and said he “can kind of tell right way if they understand what I just said” (BI, line 834); therefore, he knew how to respond to them in the moment. The three sub-categories for the supports related to written feedback were Teacher Logistics, Students, and Resources for Teachers. Evan thought limiting the number of questions on his quizzes helped him provide written feedback to students. Gwen stated that being able to see students’ worked out solution methods helped her provide written feedback. Just as with verbal feedback, Scott looked to his textbook and his lesson study team as supports for writing feedback to students. Charlie also saw a benefit in working with other teachers to determine what makes good written feedback, which was a focus of his professional learning community at one point. Self-assessment. This formative assessment practice also did not have any internal supports mentioned by the teachers. The teachers shared six external factors, including three different sub-categories (Classroom Norms, Students, and Resources for Students). Evan thought that because he had made clear learning targets and the learning scales document a classroom 220 norm, he provided a structure for his students to self-assess, saying “now I have something for them to self-asses themselves on” (BI, lines 591-592). Charlie and Lindsay identified supports related specifically to students’ behaviors or actions. Charlie thought providing students with opportunities to self-assess will help them get better at self-assessing, which then made it easier for him to incorporate this practice in his classroom. Lindsay stated that students’ willingness to engage in self-assessment allowed her to incorporate this formative assessment practice, which her AP Statistics students more willing to do than her other students. Charlie, Gwen, and Lindsay all stated that having resources for their students allowed them to incorporate more self- assessment in their classrooms. For Charlie, this was in the form of rubrics, whereas for Gwen it was online resources that her students had access to. Lindsay provided her students with examples of model work to compare their own work to; she thought her AP Statistics students used the model work more productively than her other students. Peer assessment. Similar to self-assessment, the teachers did not provide any internal supports for implementing peer assessment. There were eight different external factors mentioned, representing four sub-categories: School, Resources for Students, Classroom norms, and Students. Gwen thought having small class sizes helped her provide students opportunities to peer assess. Similar to self-assessment, Charlie thought having a structure such as a rubric allowed him to have students peer assess. He and Lindsay also thought that having model work available for students to look at allowed students to peer assess. Scott thought the classroom norms he had set of students working in groups most of the time and having group roles allowed him to provide his students with opportunities to peer assess. Charlie, Lindsay, Gwen, and Lukas all thought that students’ behaviors or participation in class helped them incorporate peer assessment in their classrooms. Similar to self-assessment, Charlie thought providing students 221 with experience in peer assessing helped them improve, which allowed him to use this practice. The biggest benefit Charlie saw in incorporating peer assessment was “every single kid gives and gets feedback” (BI, lines 950-951). Lindsay agreed with Charlie in that she thought her students needed practice using peer assessment to get better at it. Gwen stated that her classes had a good rapport, which enabled her to have them peer assess. Lukas discussed that sometimes students could explain a strategy to their peers better than him, so these opportunities could support his students’ understanding. Instructional decisions. Teachers named three internal factors and eight external factors for supporting their ability to incorporate making instructional decisions into their classrooms. Charlie and Evan shared the internal factors. Charlie thought that his mindset of what teaching should be and his prior teaching experiences had helped him be more prepared to make adjustments to his instructional plan in the moment. Evan also believed that his previous teaching experience had helped him make decisions in his classroom. He said, the “more experience you have I think that helps because I have some idea um where I need to make some adjustments or alter what I’ve done or make changes to what I’ve done in the past based on how students have done” (BI, 642-644). The eight external factors were included in five of the sub-categories (Classroom Norms, Content, Resources for Students, School, and Resources for Teacher). Scott thought the classroom norm of having his students work in groups helped him make instructional decisions because he could have groups work on different activities based on how they were performing. He made decisions based on groups instead of individual students, which made this easier. Evan stated that his school would be going to one-to-one technology the following school year and that this would help him make adjustments to his instructional plan because he would be able to 222 provide students with different assignments that they can work on online. Lukas thought the importance of the mathematics content helped him make adjustments to his plan because if it was a concept that was important to future learning, then he made changes that would allow him and his students to spend more time discussing this topic. Charlie and Scott stated that seeing others teach or having conversations with their fellow teachers helped them make adjustments in their instruction. Whether it was an issue in his classroom or with the curriculum, Scott found talking with others incredibly helpful. He stated, “just having those conversations where, where [we’re] not just isolated in our own room that helps with that” (BI, lines 605-606). Evan had found resources outside of his textbook, such as through online websites, to help him make instructional decisions. Charlie and Gwen stated external factors that related to their schools more broadly. For Charlie, it was the structure of their mathematics department: since they have a teacher-created curriculum, it is expected that teachers would make decisions and adjustments as they moved through the different units. For Gwen, it was support from her administration, “knowing that I’m supported and that I can slow it down and make sure the students are getting what they need” (BI, 954-955), helped her to make instructional decisions in her classroom because she was not pressured to move through the textbook without students understanding the content. Summary. When discussing what has helped them implement formative assessment practices in their classrooms, the teachers named internal and external supports. Three formative assessment practices, writing learning goals, determining questions, and adjusting instruction, included internal supports. Having prior teaching experience was seen as a support for all three of these formative assessment practices. All the teachers, except Gwen and Scott, named an 223 internal support. This shows that most of the teachers saw something in themselves that would help them implement formative assessment practices in their classrooms. Most (87%) of the supports the teachers named, though, were external. Teacher resources and students combined accounted for almost 60% of the factors mentioned by teachers. Teacher resources (textbooks, colleagues, professional development, and additional resources) helped the teachers implement all the formative assessment practices, except self-assessment and peer assessment. All the teachers named at least one resource that supported their ability to determine questions they would ask their students. All the teachers, except Lukas, mentioned a resource they used to write the learning targets. Four of the teachers (Charlie, Evan, Lukas, and Scott) named a resource that supported their ability to adjust their instruction. This shows that good teacher resources were valuable to this group of teachers and necessary for them to be able to implement formative assessment practices in their classrooms. How the students engaged with or participated in activities relating to the formative assessment practices was also an important support for teachers. Writing learning goals and making adjustments to instruction were the only formative assessment practices that did not include a mention of students. All the teachers, except Evan and Scott named at least one way that students supported their ability to implement these formative assessment practices. When students had opportunities, showed their work, had good rapport with their teachers and peers, and were willing to engage in the activities, teachers could implement formative assessment practices. For these teachers, students played an important role in whether they could incorporate formative assessment in their classrooms. This is interesting because based on how formative assessment is defined in this dissertation, students must be active participants in the process. It seems these teachers also saw that connection. 224 Writing learning targets had the fewest number of supports (N=3) named by teachers. These were either resources for the teacher (N=2) or having extra planning time (N=1). Gwen, Lindsay, and Scott agreed that their textbooks were helpful in writing learning targets. Lindsay and Scott were at the same school and spoke highly of their textbook, so it was not surprising that they both relied on it for support. When determining questions to ask their students, the teachers saw their resources as the most helpful option. Of the nine factors named (2 internal and 7 external), six of the external supports were their teacher resources, which included their colleagues and textbooks. Lukas named the other support, which was having students work in small groups, so he could ask more specific questions. The external supports mentioned by teachers for providing verbal and written feedback represented the same sub-categories, except verbal feedback also included Classroom norms and time. However, there were twice as many supports named for verbal feedback compared to written feedback. Students were the most common sub-category for verbal feedback, whereas, resources for teachers was the most common for written feedback. Teachers named only two supports (textbooks and lesson study experiences) for both types of feedback. Self-assessment and peer assessment had similar counts for the number of supports teachers mentioned. They also had three supports in common: providing a structure to students, giving students experience, and having examples or model work for students to see. The supports named by teachers for adjusting their instruction represented five different sub-categories, which was the same number of sub-categories included in verbal feedback. Therefore, two formative assessment practices showed the most diversity in what teachers saw as supporting their ability to implement these practices in their classrooms, however, resources for teacher and classroom norms were the only sub-categories they had in common. 225 Obstacles When Implementing Formative Assessment The obstacles that teachers talked about were categorized into two initial categories; internal and external. There is an additional none category because twice teachers stated that nothing hindered them in implementing a particular formative assessment practice, so this did not fit the internal and external categorization. Table 33 shows how many of the teachers’ obstacles were considered internal or external and for which formative assessment practice they occurred. Table 33 Obstacles to Implementing Formative Assessment Practices Internal External None Total 3 4 0 4 2 2 4 4 3 5 10 12 1 1 0 0 0 0 8 9 3 9 12 14 Writing learning goals Determining questions Providing verbal feedback Providing written feedback Having students self-assess Having students peer assess Adjusting instructional plan Total 3 18 7 45 0 2 10 65 As shown in Table 33, most of the obstacles that teachers talked about were classified as external supports. However, except for providing verbal feedback, all the practices were associated with at least one internal factor that hindered teachers. The rest of this section discusses specific obstacles teachers named related to each of the above formative assessment practices. Since teachers mentioned a substantial number of external factors mentioned, these obstacles were classified using the same eight sub-categories as the supports (Table 34). The most common external obstacle teachers named was students, which 226 could be related to their behavior, attitudes, or participation in the classroom setting. This accounted for almost half of the obstacles (22 out of 45) that teachers named. The next most common was time. Table 34 External Obstacles to Implementing Formative Assessment Class Norms Content Logistic Resource for Teachers Resource for Students School Student Time Total 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 2 0 1 1 1 0 0 0 0 0 0 0 0 1 1 2 0 0 0 0 1 0 1 3 2 7 7 1 2 0 1 1 1 4 4 3 5 10 12 0 2 2 8 FA Practice Writing learning goals Determ- ining questions Providing verbal feedback Providing written feedback Having students self- assess Having students peer assess Adjusting instruct- tional plan Total 0 2 1 1 1 4 7 45 Learning targets. Although Gwen stated that nothing hindered her in being able to write 2 22 0 3 1 3 learning targets, other teachers mentioned three internal and four external factors that hindered them in writing learning targets. Evan (N=2) and Lukas (N=1) named the three internal factors. Evan stated that in the past he did not see value in having learning targets and he did not spend 227 much time writing them. Lukas stated that writing learning goals had not been a priority for him, which then hindered his ability to determine them. The four external factors represent three of the sub-categories (Resources for Teachers, Time, and School). Although Lindsay was able to write learning targets on her own without obstacles, she has found that working with other teachers to determine learning targets for their lesson study was challenging because everyone came to the meeting with different ideas about what the focus of the lesson should be. She stated that “we were kind of nice to each other so when we went to plan the lesson we didn’t really have a focus” (BI, lines 596-597). Lukas named the other three external obstacles to writing learning goals. He stated that he did not have the time he wanted to work on determining learning targets for the courses he is teaching. He also noted the lack of structural support from the administration was a hindrance. Teachers were expected to work on their learning goals during department meetings; however, other issues tended to come up and then the teachers did not have that time to work. Lukas also stated that it was challenging to write learning targets for a new course, which was something he was doing this year. As an additional obstacle, the course he was teaching was not in mathematics, although it was related to his teaching minor. Questioning. Just as with the learning targets, Gwen’s initial reaction was that nothing hindered her in determining questions for her students; however, she did mention an external factor later in our conversation. Teachers mentioned four internal and four external factors that hindered them in determining questions they wanted to ask during their lessons. Lindsay and Scott named the four internal factors. Lindsay stated that sometimes she could be in a hurry during class and asked questions that were not as open-ended as she would have preferred. Lukas stated that determining questions is a challenging formative assessment practice for him. Lukas 228 thought it is easier for him to ask questions in small groups but that during whole class discussions he sometimes struggled with asking questions. He had also struggled with sustaining what he learned during professional development. He found it challenging to follow through on the strategies he had been provided for more than a few days. Evan, Gwen, and Lukas named the four external factors related to determining questions for students. Gwen’s obstacle related to students’ lack of prior knowledge interfering with their understanding. Since her students were unable to build on previously learned mathematics concepts, it was challenging for her to determine the appropriate questions to ask them. Evan and Lukas both stated that time was a hindrance for them. Evan was teaching four different courses, so he struggled to find time to determine questions in advance for each of these courses. Not only did Lukas wish he had more time to find resources to help him write better questions for his students, he also wished he had other people to talk to as resources. Feedback. Verbal feedback was the only formative assessment practice that did not have any internal factors named by the teachers. Gwen, Evan, and Lindsay named the three external factors related to students. For Gwen, students who were quiet or were not able to stay after school to meet with her made giving verbal feedback challenging. Evan stated that some of his students needed more attention than others; therefore, he ended up spending more time working with these students and was not able to give verbal feedback to all of his students. Her students not responding to her questions or comments made it challenging for Lindsay to give verbal feedback. She stated that how her students responded “has a lot of impact on whether or not I do it” (BI, line 647). Teachers named four internal and five external factors that hindered their ability to provide written feedback to their students. Charlie (N=1), Evan (N=1), and Lukas (N=2) all 229 stated internal factors. For Charlie and Lukas, it was their own terrible handwriting. Evan did not make written feedback a priority in his classroom. Lukas identified the other two internal factors: being a slow reader and trying to find a balance when giving written feedback. Providing students with quality written feedback was important to Lukas, who stated, “I want to get timely feedback to them, but [I] also want to give meaningful feedback to them, so finding that balance between timeliness and meaningfulness is tough for me sometimes” (BI, lines 827-829). The five external factors represented three sub-categories: Time, Logistics, and Students. Evan, Lukas, and Scott all stated that providing written feedback was time-consuming, which hindered their ability to use this formative assessment practice. For Gwen, the mathematics teachers’ testing format hindered her ability to give written feedback because the tests were multiple choice; therefore, she was unable to see the students’ thought processes. Scott stated that an obstacle to providing written feedback was that it was not immediate for students, which was a logistical issue because it had to do with the feedback process and not the students themselves. Lindsay and Lukas mentioned the two factors related to students. Lindsay had started using less written feedback in her classroom because “kids don’t look at it” (BI, line 655). However, she was trying to hold her students more accountable and force them to read the feedback, so she was incorporating a new method for providing written feedback on students’ summative assessments. She did not provide a score the first time she reviewed their work, instead providing feedback, to which the students were required to respond in order to get their assessments scored. As of now, she was only providing feedback on work that needed to be improved. She said her goal with this was that “they’ll try and figure out what went wrong” (BI, line 669). Lukas had a similar response in that a hindrance of giving written feedback was that it 230 was up to the student to determine how to respond to the feedback and take the next steps. He stated that, “[in] most cases when they look at written feedback, they’re not going to follow up and ask you about it you know, so you kind of get one shot with that” (BI, lines 836-837). Self-assessment. The teachers named two internal factors and ten external factors that hindered their implementation of student self-assessment. This formative assessment practice had the second most factors named as hindrances. Evan and Lukas described the two internal factors. Evan’s obstacle was his own lack of knowledge, “I just didn’t know how to have them self-assess, I’d never really thought about it before” (BI, lines 593-594). For Lukas, providing students with an opportunity to self-assess had not been a priority for him. He stated, “other things have gained greater priority in my mind than at this point than that, not that I don’t want to do it, but it’s one more thing” (BI, lines 849-850). The ten obstacles named by teachers represented four different sub-categories: Classroom Norms, Time, Resources for Students, and Students. All the teacher, except Evan, mentioned at least one external obstacle to student self-assessment. Scott, Lukas, and Charlie named the external factors representing Classroom Norms, Time, Resources for Students. Scott stated that he needed to set a classroom culture that promoted students’ self-assessment, which was something that he was working on. Lukas stated that he had not had the time at his current school to implement self-assessment in the way that he would like. Charlie needed to find a structure that worked for students in order for them to be prepared to self-assess in a meaningful way. The seven other external factors all relate to students’ behavior, attitudes, and/or participation in class. All the teachers, except Evan and Lukas, mentioned an external factor that related to issues with students. For Charlie, there were two factors. First, he struggled with getting students to understand the depth of the self-assessment process. Students’ self-assigned 231 scores were always much higher than Charlie’s assessment of their work and this pattern seemed to continue throughout the unit. He needed to find a way for their view of their learning to be more aligned with his view of their learning. His second issue was as he referred to it, the mindset of his teenage students in that they were not putting much effort into the self- assessments. Lindsay’s obstacle was that her students were not willing to take the self- assessment process seriously. Scott mentioned two factors that hindered his ability to implement self-assessment. Scott was concerned that not all his students were comfortable with self- assessment and that they would need trained on how to self-assess effectively. He thought some of his students were natural self-assessors. However, not all of them were and it was this group of students that hindered his ability to incorporate self-assessment. In contrast, Gwen had two different factors that served as obstacles in her ability to implement self-assessment. The first obstacle was that many of her students did not have internet access at home. This was a problem because she posted answers to the homework on her school’s webpage, and students are unable to check their work to see how they performed. The second issue was that many of her students did not do their homework; therefore, they did not have the opportunity to self-assess their work beyond what was done in class. Because they are not practicing their mathematical skills, Gwen said she was unable to move to the next topic because the students did not quite understand the previous mathematical concepts. Peer assessment. The teachers named two internal and twelve external factors that hindered their implementation of peer assessment. This was the formative assessment practice with the most obstacles named by the teachers. Lukas and Evan named the two internal factors. Lukas said that he needed to find a better way for students to peer assess in an effective manner. 232 Evan mentioned that peer assessment was not a priority for him because there were more important concerns to address in his classroom. The twelve external factors represented six of the sub-categories: Logistics, Classroom Norms, Time, Resources for Students, School, and, Students. All the teachers mentioned at least one external obstacle to implementing peer assessment. Lukas identified a logistical issue in that he wanted to protect students’ privacy, stating, “I try to be careful what types of things [I] have [students] peer assess cause of laws and things like that” (BI, lines 858-859). As Scott stated regarding self-assessment, he was working on creating a classroom culture in which peer assessment was a regular part of the classroom activities. Evan stated that a lack of time inhibits his ability to incorporate peer assessment. Charlie had the same obstacle with peer assessment as with self-assessment: finding a good structure. Gwen named the one school related hindrance, which was the large size of some of her classes. The seven other external factors all relate to issues of students’ behavior, attitudes, and participation. All the teachers, except Lukas and Scott, mentioned obstacles related to students. For Gwen, the obstacle was that her students were not willing to participate or be engaged in the peer assessment process. Two of Charlie’s three obstacles were the same as his self-assessment obstacles: getting students to understand the depth of the peer assessment process and the mindset of his teenage students. The other obstacle that he mentioned was status issues between students, stating, “they think that kids are really good based on things I don’t even understand like I don’t know why” (BI, lines 1013-1014). He found that students would make claims that a peer’s work was good because that student was seen as always being smart or that a peer’s work was wrong because that student was seen as not being good at math. The students made 233 judgments based on these ideas about their peers and not the actual mathematical work their peers were doing. Similarly, Lindsay reported that the maturity levels of her students hindered her ability to implement peer assessment. Her younger students struggled with providing their peers with meaningful feedback, which made it challenging for Lindsay because she had to remind them every time what appropriate comments looked and sounded like. She also found that her students’ ability level had an impact on the peer assessment process. When a student was struggling, s/he did not feel qualified to provide peer assessment. Lindsay believed that students needed time to be better at peer assessment saying, “I definitely think it’s a long-term project and not something that they’ll be able to do overnight” (BI, lines 405-406). Similarly, Evan had found that his students had gaps in their mathematical understanding; therefore, they did not have the knowledge that would allow them to meaningfully assess their peers. Instructional decisions. Teachers named three internal and seven external factors hindered them in making adjustments to their instructional plans. Charlie was the only teacher who did not state an obstacle to making instructional decisions. Lindsay (N=2) and Scott (N=1), who teach at the same school, stated the three internal issues. Scott did not believe this that making instructional decisions was a strength of his because he saw this as being related to differentiated instruction, which he found challenging. Lindsay believed that a lack of confidence in herself hindered her ability to make instructional decisions because, “I think sometimes letting kids go where they want to go it can be a little frightening” (BI, lines 825-826). This was concerning to her because she could not be sure in what direction students would take the conversation and whether it would relate to the goals she had for the lesson. Lindsay also wished that she could address her students’ mathematical issues more quickly. There were some aspects, 234 such as how she did homework, that she knew needed to be improved, but she was unsure how to fix them. She stated frankly, “I wish I modified plans quicker” (BI, line 844). The seven external factors represented five of the sub-categories: Logistics, Content, Resources for Teachers, Time, and, Students. Of the five teachers who named obstacles to making instructional decisions, all of them had at least one external factor. A logistical issue for Scott was staying on pace with the other teachers who were teaching the same subjects. Because the teachers used common assessments, it was important for them to stay together when moving through the curriculum Gwen’s obstacle was covering enough of the mathematical content that her students were prepared for their next mathematics course. While there was not pressure from her administration to move on without students’ understanding the content, she also had to provide students with opportunities to learn the concepts they needed so they could be successful in future mathematics courses. Lukas’s lack of additional resources hindered his ability to make decisions to adjust his instructional plans. He had extra resources for some of the topics he was teaching, but not all of them; it was more challenging for him to spend extra time on a topic for which he did not have prepared resources to use. The other two sub-categories, Time and Students, each had two obstacles named by teachers. Lindsay and Lukas each reported an issue related to time. Lindsay’s obstacle was not having the time to address all the needs of her students; whereas Lukas’s obstacle was not having enough time to find or create resources to help him make adjustments to his instructional plans. Evan stated both obstacles related to students’ knowledge and abilities. The first was that there were a number of students in his classroom that needed accommodations. The second was the diversity in students’ abilities within the same class. Both obstacles contributed to Evan thinking, “it almost becomes overwhelming at times to try and figure out okay how am I going to meet the 235 needs of all my students” (BI, lines 647-648). The drastic differences among his students made it challenging for Evan to make adjustments to his instructional plans that would benefit all of his students. Summary. When discussing what has hindered them in implementing formative assessment practices in their classrooms, the teachers named internal and external obstacles. Providing verbal feedback was the only formative assessment practice that did not include an internal obstacle. Teachers named the most internal obstacles for determining questions to ask students and providing written feedback. The most common internal obstacle mentioned by teachers was not making a particular formative assessment a priority in their classroom. Teachers reported this obstacle for four of the formative assessment practices: writing learning goals, providing written feedback, self-assessment, and peer assessment. This shows that teachers’ preferences play a role in what formative assessment practices they implement in their classrooms. Most (69%) of the obstacles named by teachers were external. Students were named as the most common obstacle (49%), accounting for almost half of the obstacles. All the teachers mentioned students as an obstacle for at least one of the formative assessment practices. Four teachers (Charlie, Gwen, Lindsay, and Scott) stated students were an obstacle for them implementing self-assessment. Some of these issues related to students included: students’ mindset, unwillingness to participate, and students being uncomfortable with the process. Four teachers (Charlie, Evan, Gwen, and Lindsay) named students as a hindrance for incorporating peer assessment. While there were some obstacles named for both self- and peer assessment, such as mindset and lack of participation, there were other obstacles, like status issues and gaps 236 in students’ knowledge that made peer assessment a challenging practice for teachers to implement. In addition, half of the teachers named students as an obstacle for providing verbal and written feedback. Lindsay was the only teacher who named students for both types of feedback. Interestingly, there were no overlaps in obstacles for the two kinds of feedback. Verbal feedback had obstacles such as students who were not willing to talk to the teacher or needed more attention from the teacher. Obstacles for written feedback included: getting students to read teachers’ feedback and students choose how to or if they will respond to the feedback. Based on the data, students played a major role in teachers’ ability to implement formative assessment practices, therefore, it seems it would be important for teachers to help students see the value in these practices, so teachers could implement these practices more often. Time was the next most common obstacle (18%) for teachers when implementing these formative assessment practices. Providing verbal feedback was the only formative assessment practice that did not have time named by teachers as an obstacle. Four of the six teachers (Evan, Lindsay, Lukas, and Scott) mentioned time was an issue for them. Lindsay only mentioned time as an issue when she was making adjustments to her instructional plans. Evan was the only teacher who said time hindered his ability to incorporate peer assessment in his classroom. Evan, Lukas, and Scott named time as a hindrance when providing written feedback to their students. Lukas mentioned time more often than any of the other teachers, naming it as an obstacle for five of the formative assessment practices (writing learning targets, determining questions, providing written feedback, self-assessment, and adjusting instructional plans). This shows that time was a serious obstacle for some of the teachers, therefore, helping students figure out how to 237 incorporate the formative assessment practices into their current teaching methods would likely help alleviate the pressure of needing more time. Providing opportunities for students to peer assess had the most sub-categories (N=6) and most obstacles (N=12) of all the formative assessment practices. As mentioned previously, students were the most common obstacle, but other sub-categories included: classroom norms, resources for students, and school issues. Examples of some of these obstacles were having a large class size or creating a classroom culture that supported peer assessment. Making adjustments to instructional plans had the next highest number of sub-categories (N=5) and the third highest number of total obstacles (N=7). This shows quite a bit of diversity because there were not very many obstacles named within each sub-category. Some of these sub-categories included: content and resources for teachers. Examples of these obstacles were needing to cover enough content for students to be successful in the next mathematics class and lack of additional resources for the teacher. Providing opportunities for students to self-assess had the second highest number of obstacles (N=10), but the third highest number of sub-categories (N=4). Although this seems to imply there were multiple obstacles for each sub-category, it is important to note that seven of the ten obstacles were from the same sub-category (students) and the other three sub-categories (classroom norms, resources for students, time) each represented one obstacle. An example of an obstacle for incorporating self-assessment was not having a good structure for students to use. Writing learning goals, determining questions, and providing written feedback each had obstacles represented by three sub-categories. Time was the only sub-category these three formative assessment practices had in common. Teachers named four obstacles each for writing learning objectives and determining questions, whereas they named five obstacles for providing 238 written feedback. Examples of obstacles for writing learning objectives included: teaching a new course and lack of structural support. Teachers named students’ lack of prior knowledge and not having other teachers to talk to as obstacles to them determining questions to ask their students. Hindrances for providing written feedback included giving multiple choice tests and feedback not being immediate for students. All the obstacles named for providing verbal feedback to students were from the same sub-category (students). Supports and obstacles. The teachers named about the same number of external supports (N=45) as obstacles (N=46), however, there was a much greater difference in the number of internal supports (N=7) and obstacles (N=18). This seems to imply that teachers see themselves as more of an obstacle to implementing formative assessment than a support. Six of the formative assessment practices had internal obstacles named by teachers, however, only three of the practices had internal supports mentioned. Teachers named nine external supports and three external obstacles to providing verbal feedback to students. This was one of two formative assessment practices that had more supports than obstacles mentioned by teachers. This is interesting because it seems to imply that teachers saw many supports and did not see very many obstacles to their ability to give verbal feedback, therefore, this seems like a practice they could incorporate more easily into their classroom practices. Teachers also named more supports (N=11) than obstacles (N=10) for adjusting their instructional plans. This seems encouraging for teachers as they have many supports to help them make decisions that could hopefully counteract the obstacles that stand in their way. Teachers mentioned the same number of supports as obstacles (N=9) for determining questions to ask their students. However, there was an even split between internal and external obstacles (N=4), but teachers named many more external (N=7) than internal (N=2) supports. This shows 239 that external factors were more helpful to teachers, but there was a balance between external obstacles and the teachers themselves. It is interesting to note that the teachers did not name any internal supports for implementing self-assessment and peer assessment, however, for each of these practices, teachers mentioned two internal obstacles. The teachers also named almost twice as many obstacles as supports for each of these formative assessment practices. This suggests that self-assessment and peer assessment are challenging formative assessment practices for these teachers to implement. Although the teachers mentioned the same number of external supports as obstacles (N=5) for providing written feedback, they also named four internal obstacles, which implies that teachers see themselves as hindering their ability to give written feedback. 240 Chapter 9: Discussion This chapter discusses and analyzes the results described in the previous four chapters. First, I describe the different activity structures teachers implemented during the observed lessons to provide context for their classroom structure. I found that pair/small group work and whole class discussion were the activity structures most often implemented by the teachers. In addition, none of the teachers implemented every activity structure in their individual observed lessons. Then, I explain how the lesson structure impacted teachers’ formative assessment practices. I found that whole class discussions and pair/small group work included the most formative assessment practices. Also, every activity structure included at least one of the formative assessment practices. In addition, seven of the nine activity structures had at least one formative assessment practice that averaged a high-level practice. However, not every teacher implemented all the formative assessment practices and none of the teachers used peer assessment. The implementation or lack of implementation of these formative assessment practices was consistent with previous research (Gotwals et al., 2015). Although there were not a lot of high-level practices observed, almost every teacher averaged a high-level practice in at least one formative assessment practice. Next, I provide information related to teachers’ classroom activities that promoted and constrained formative assessment practices use. The school context and the role of curriculum were influential in Lindsay’s and Scott’s classrooms, but not as much in Evan’s and Gwen’s classrooms. Teachers also stated that they thought particular activities or strategies related to formative assessment, however, those activities and strategies were not often coded as including or being formative assessment practices. This demonstrates the differences in the teachers’ views of formative assessment and my own. Finally, I discuss the supports and obstacles that teachers saw as helping or hindering their ability 241 to implement formative assessment practices in their classrooms. While some of the supports and obstacles named by teachers overlapped with previous research (Box, Skoog, & Dabbs, 2015; McMillan, 2003; Suurtamm and Koch, 2014; Suurtamm, Koch, and Arden, 2010; Volante & Beckett, 2011), the most common supports and obstacles mentioned where different from prior research. The most common supports named by teachers included teacher resources and students. Students dominated the list of obstacles named by teachers to their ability to implement formative assessment. Lesson Structure Chapter 4 focused on the activity structures teachers implemented in their lessons. The lesson structure was divided into nine different activity codes. Not every teacher implemented all the activities, however. All the teachers used the following four activity structures at least once during the observations: introduction to class and lesson, individual activity, pair/small group work, and whole class discussion. Beyond that, however, the number and type of activity structure varied by teacher. Lindsay and Scott taught at the same school; however, they were observed for different subjects (Lindsay was Algebra 2 and Scott was Geometry). They used the same curriculum series for both content areas. Table 35 shows the percentage of total time each of these two teachers spent on the different activity structures. Table 35 Lindsay’s and Scott’s Percentage of Time Per Activity Structure Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Lindsay 0.00% Scott 0.00% 8.67% 1.31% 2.71% 69.17% 6.81% 0.00% 2.04% 81.05% 242 Table 35 (cont’d) Presenting student work Whole Class Discussion Teacher Giving Information Summary 2.98% 14.15% 0.00% 1.02% 0.84% 9.09% 0.00% 0.17% It is interesting to note that even though these lessons included different mathematical content, these two teachers had very similar lesson structures. Neither incorporated a Warm-up or Teacher Giving Information. They spent most of their class time on pair or small group work, including having their students sitting in groups. The example of Lindsay and Scott suggested that the use of activity structures was influenced, at least in part, by curricular and school context. However, it seems lesson influenced by the specific content of the class (e.g., Algebra vs. Geometry). The impact of school context is not surprising, since their mathematics department is very collaborative and the teachers regularly worked together to stay on the same track in their classrooms and had similar classroom norms, such as group work and group roles. But, the story is not that simple. Evan’s and Gwen’s cases suggest that even colleagues working in the same building can have substantially different lesson and activity structures. Table 36 shows the percentage of total time each of them spent on the different activity structures. Evan Gwen Table 36 Evan’s and Gwen’s Percentage of Time Per Activity Structure Activity Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work Whole Class Discussion Teacher Giving Information Summary 1.04% 2.01% 33.29% 29.69% 26.95% 1.60% 5.26% 0.00% 0.16% 0.58% 7.45% 10.37% 32.22% 28.73% 0.00% 20.42% 0.00% 0.23% 243 Like Lindsay and Scott, Evan and Gwen were not observed for the same subject areas. Evan was observed for Algebra 1 and Gwen for Algebra 2; however, they used the same textbook series for both these courses. Neither Evan nor Gwen implemented Teacher Giving Information. Both teachers spent very little time on Warm-up or Summary activities. This shows both the similarity and difference in their practices in that both teachers spent significant time having students work alone and with their peers. However, Evan spent much of his time having discussions with students, whereas Gwen used that time to lecture. Evan also had his students sitting in groups, while Gwen had her students sitting in rows. Like Lindsay and Scott, these two teachers met regularly and they had common unit assessments, however, the collaborative nature of their department did not seem to have an impact on their lesson structures. Having established both similarities and differences in lesson structures, I then turned in Chapter 5 to analyze the kinds of formative assessment practices teachers enacted in order to understand the relationships between activity structure and formative assessment practices. Formative Assessment Practices Chapter 5 focused on which of the six formative assessment practices (learning targets, questioning, self-assessment, peer assessment, feedback, and instructional decisions) teachers implemented in their lessons and in which activity structures these practices occurred. One of the most interesting findings was that none of the teachers implemented peer assessment during their observed lessons. This was interesting because Volante and Beckett (2011) also found that teachers struggled with implementing this formative assessment practice in their classrooms. There was also very little self-assessment (only six instances, two each by Charlie, Evan, and Scott). But, the self-assessment I did observe often included the use of a specific structure, such as a rubric or teacher-created document. The lack of self-assessment was not completely 244 surprising, though, because Gotwals et al. (2015) also found that self-assessment was rarely seen in classrooms. There were also not very many instructional decisions observed during the lessons (16 instances with at least one from each teacher). Again, this was not totally surprising because Gotwals et al. (2015) also found that this particular practice was often challenging to observe during lessons because many of these decisions happened inside a teacher’s head. This was the case for the current study as well because there were times that I implied an instructional decision had been made, however, those instances were not coded since they were not explicit to the students. For each of the teachers – and looking across all teachers, questioning and feedback were the most commonly implemented formative assessment practices. This was not surprising because many mathematics teachers often use questioning to elicit students’ thinking and to gather information about how students are thinking about the mathematics. Volante and Beckett (2011) also found that questioning was the dominate formative assessment practice implemented by the teachers in their study. Feedback, then is often provided to students in response to their answers to teachers’ questions. Activity structure as a context for formative assessment. The lesson activity played a role in which formative assessment practices were incorporated into the lessons. Learning targets were coded 26 times across all the teachers’ lessons, however 18 of those instances were during the Introduction activities. Therefore, if teachers did not mention learning targets during the introduction to the lesson, they were rarely mentioned during the rest of the class period. The only other activity structures that included learning targets were Interactive Lecture (N=2), Individual Activity (N=4), and Whole Class Discussion (N=2). Some of the teachers had the learning targets posted in their classrooms, however they did not verbally share them with 245 students, therefore, they were not included in the coding process. This lack of discussion is interesting because learning targets are often seen as an important start to the formative assessment process (Brookhart, Moss, and Long, 2008; Chappuis, 2009; Stein and Smith, 2011). As mentioned previously, questioning was the most commonly implemented formative assessment practice (N=163), which is encouraging since questioning is an important part of formative assessment (Hodgen & Wiliam, 2006). Questioning was used in every activity structure, except Teacher Giving Information, which made sense since this was the only activity structure that did not include interaction between the teacher and students. Whole Class Discussion (N=48) and Pair/Small Group work (N=44) were the activity structures where teachers had the highest overall count of questioning codes. These two activity structures provided teachers with many opportunities to interact with their students and ask them questions about the mathematics they were working on. These interactions are important because they enable teachers to gather information about the different levels of understanding students have (Franke et al., 2009). It is interesting to note that more question codes were applied during Interactive Lecture than Individual Activity, since by nature lecture was more of the teacher doing the talking. There were very few times, however, that the Interactive Lecture was completely teacher led. The teachers would often ask at least a few questions to check in with students as they shared information about new content with their students. It seemed like when students were working individually teachers wanted to give students time to work on their own without interruption and then reserved their questions for the whole class discussion that would often follow the individual work time. It is also interesting to note that Presenting Student Work had the fourth highest occurrence of the questioning code. During this activity students were often sharing their work in 246 front of the class, which provided the teacher with the opportunity to ask questions of the student about his/her work, as well as ask the rest of the class about their peer’s work. Warm-up (N=2) and Summary (N=3) had the lowest occurrence of the questioning code, which is interesting since these two activity structures often open and close the lessons. It is likely that teachers used the warm-up as a chance for students to work on their own or with others without interaction with the teacher and then waited to ask questions during the whole class discussion that often followed the warm-up. The Summary activities were often used to wrap up the lesson, which was usually done quickly, therefore, there were not many opportunities to ask questions. Although the teachers’ used extensive questioning practices, most of the specific questions they asked were low-level as discussed in Chapter 6. Teachers asked the most questions during Pair/small group work and Whole class discussion. Gotwals et al. (2015) also found that the mathematics and science teachers in their study asked predominately low-level questions. In addition, some of their science teachers only asked low-level questions during their observations. All the teachers in the current study asked some high-level questions, however, there were teachers who did not ask many, such as Lukas, who had only about 11% of his questions coded as high-level. This suggests that even though teachers are asking many questions during mathematics lessons, they need more support in determining high-level questions that would allow them to gain more information about students’ thinking. Self-assessment was only implemented by three of the teachers (Charlie, Evan, and Lukas) for a total of six instances during four different activities. Teachers had students self- assess twice for each of the following two activity structures, Individual and Pair/Small Group Activities. In addition, Introduction and Whole Class Discussion each had one instance of the self-assessment code. When teachers provided students with opportunities to self-assess, students 247 were often given a specific structure, such as a rubric aligned with the learning targets in Evan’s classroom or a reflective learning log in Scott’s classroom. This can be a helpful way to incorporate self-assessment in classrooms, because having a structure provides students with a starting place and more guidance on how to reflect on their learning (Cameron et al., 2011). Charlie listed the three unit goals and asked students to reflect on the unit to determine what was needed to be successful on each of the goals and then to consider where they were in their own learning. Based on students’ high-level of engagement during the activity, this was likely a great learning experience for them because they were required to think about what they had learned throughout the unit and how those different mathematical concepts aligned with the unit’s learning targets. Both Charlie and Evan aligned their students’ self-assessments with the learning targets, which gave the students specific aspects to self-assess on as well as provided them with opportunities to make explicit connections between what they had learned and the learning targets. Feedback was provided to students during all the activity structures, except Summary. This made sense since during this activity the teacher was wrapping up the lesson and therefore not giving students feedback on their thinking or performance. There were 104 activities that were coded as the teacher providing feedback to their students. More than half of these instances occurred during the Pair/Small Group Work (N=30) or Whole Class Discussion (N=26). These activity structures provided the teacher and students many opportunities to interact around the mathematics students were working on. These activity structures also provided students time to make sense of teachers’ feedback and ask questions, which Hodgen and Wiliam (2006) thought were important aspects of feedback. Feedback was provided to a lesser degree during Individual Activity (N=13), Presenting Student Work (N=13), and Interactive Lecture (N=10). It is 248 interesting that these activity structures had similar frequencies of feedback, since lecture is often not thought of as providing students with information about their thinking. The teachers who implemented this lecture structure, though, did expect some level of participation from students, therefore, there were opportunities for teachers to provide feedback on students’ thinking. Similar to questioning, it seemed teachers focused their feedback at the small group or whole class level and not at the individual student level. Since Lukas was the only teacher who implemented Teacher Giving Information, he accounted for the four instances of feedback during this activity. This was also the only formative assessment practice that was implemented during this particular activity structure. This is important because the nature of this activity structure could provide opportunities for the inclusion of learning targets, however it never did. During this activity structure the teacher read the solutions to an assessment or assignment and occasionally provided feedback on students’ overall performance before he read the answers to different problems. Connections between the problems and the learning targets could have been made before the solutions were read to the students. Instructional decisions were made during six of the nine activity structures, with Warm- up, Teacher Giving Information, and Summary not containing this formative assessment practice. This formative assessment practice was not viewed as often as the other practices. It was not surprising that few instructional decisions were visible because Heritage, Kim, Vendlinski, and Herman (2009) also found that teachers struggled to make decisions about future lessons based on information they had gained from students. However, half of the instructional decisions were made during Introduction (N=4) or Presenting Student Work (N=4). Pair/Small Group and Whole Class Discussion each had three instances of teachers making instructional decisions. Interactive Lecture and Individual Activity each had one occurrence of the teacher making an 249 instructional decision. This means that almost all the teachers’ instructional decisions were made based on interactions with students. This shows that teachers were using evidence they gathered from their students to make decisions about the flow of their lesson or what happened during the class period. It was likely that teachers used questioning to gather this evidence, since almost all instances of instructional decisions also had questioning. Teachers often use questioning to help them make instructional decisions (Hodgen &Wiliam, 2006; Volante & Beckett, 2011). Gotwals et al. (2015) also found that many of their mathematics teachers made their instructional decisions visible to students by verbally stating what was happening during the lesson. An example discussed in their article that was also seen in the current study was teachers noticing many students were struggling while they were working on their own or in a small group and called the class back together to address the issue. Based on this data, whole class discussions (N=80) and pair/small group work (N=79) provided the teachers with the best opportunity to implement formative assessment practices. Whole class discussions included all the formative assessment practices, except peer assessment, which was not seen during any of the activity structures. Pair/Small group work included all the formative assessment practices, except learning targets and peer assessment. Most of the questioning and feedback codes occurred during these two activity structures, therefore, these two activity structures provided teachers with many opportunities to interact with students and talk about students’ mathematical thinking. There was little difference in which formative assessment practices teachers implemented during these two activities, however, there were slight differences in the levels of sophistication teachers demonstrated when implementing these practices. One main difference in formative assessment practices was that whole class discussions included two uses of learning targets and pair/small group work did not include any. 250 Another difference was that whole class discussions had more questioning codes, however, teachers asked almost 150 more questions during pair/small group work. This means that more of the whole class discussions included questioning compared to the pair/small group work, however, during those lesson segments, teachers asked more overall questions during the pair/small group work. One possibility is because when students are working in groups, the teacher might be asking each group the same questions (which would count multiple times), whereas when they are leading a whole class discussion, they only ask the question once. Another option is that during small group work time, teachers have more time to talk with each group and ask more questions to gather evidence of their understanding than would be possible during a whole class discussion. During whole class discussions, teachers demonstrated higher levels of sophistication for questioning (2.42 vs. 2.30) and instructional decisions (2.33 vs. 1.67). Even though teachers asked more questions during pair/small group work, they asked slightly more higher level questions during whole class discussions, which led to a higher weighted average. Although teachers implemented instructional decisions three times during each activity structure, their practices during whole class discussions were higher. This makes sense because it is expected that the teacher would make decisions about the flow of the lesson based on the majority of the students instead of a small number of students. Whole class discussions enable teachers to gain information about more students at one time than would be possible with small groups. Feedback was the only formative assessment practice for which pair/small groups had a higher weighted average than whole class discussions (2.03 vs. 1.81). This is not surprising because when students are working in small groups teachers can provide more specific feedback to those students’ thinking than would be possible in front of the whole class. When teachers 251 provided feedback to the whole class, it likely needs to be appropriate for all of them, therefore it would be more general. Introductions had the third highest count (N=40) of formative assessment practices and the highest occurrence of learning targets. This means that this activity provided teachers with the best opportunity to share the learning targets with students and set the stage for how those learning targets connected to the activities students would be completing during that day’s lesson. Introduction activities included all the formative assessment practices, except peer assessment, therefore teachers also had opportunities to implement practices other than learning targets during this activity structure. This means this activity structure could be a good place for teachers to start their lessons because they could share the learning targets with students at the start of class and then ask questions or provide feedback to gather information about students’ thinking at the beginning of the lesson. This is important to formative assessment because these types of activity structures provide teachers and students a foundation to start each class talking about the learning targets and how what they have planned for class supports students’ learning of those goals. This also allows them to come back to the learning targets throughout the class period reminding students of what they should be learning, which would help students monitor their own learning and self-assess. Presenting Students’ Work had the fourth highest count (N=36), however it only incorporated three of the formative assessment practices (questioning, feedback, and instructional decisions). Having students share their work or having the teacher share students’ work for them was a great way for teachers to ask questions and provide feedback on students’ mathematical thinking. Lukas thought this activity in itself related to formative assessment “because it gives [students] a chance to either show or explain their thinking to others” (Exit 252 Interview, line 229). This showed him how well students understood the concepts, because they had to be able to explain their thinking to others. This also gave him information about the other students because they had to make sense of their peer’s work and then ask questions if they were unsure of the solution or method. This type of activity structure is important for mathematics classes because it allowed students to share their thinking, provided them with the mathematical authority in the classroom, as well as required other students to make sense of someone else’s thinking, which could be a different method than they used. This activity structure provided teachers with many opportunities to implement multiple formative assessment practices. These types of experiences are also important to the formative assessment process because they help develop students’ ownership in their learning. They also help develop their communication skills, which is helpful for when the teacher is trying to gain information about the students’ level of understanding. It is interesting to note that Interactive Lecture and Individual activities had the same amount of formative assessment practice codes (N=35), with a similar breakdown regarding the specific practices, apart from self-assessment, which was not implemented during the lectures. Both activity structures had questioning and feedback as the most common formative assessment practices. In addition, Interactive Lecture had two instances of learning targets and Individual Activity had four instances. Both activity structures also each had no instances of peer assessment and one instance of instructional decisions. This implies that these two activities provided teachers with similar opportunities for implementing formative assessment. This similarity is likely because teachers seemed to approach these two activity structures in similar ways. While there was some interaction between the teacher and students during these activity structures, the 253 focus seemed to be to either provide students with new information or let students explore ideas on their own. Warm-ups (N=3), Teacher Giving Information (N=4), and Summary (N=3) provided the least opportunity to incorporate formative assessment practices. This is interesting because all three activity structures could provide opportunities for incorporating more formative assessment practices depending on how they are implemented. During the exit interviews, some of the teachers stated that they considered the warm-up activities to be related to formative assessment. Lukas said that warm-ups are “intended for me to get some feedback of how are [students] doing from what we did yesterday” (Exit Interview, lines 215-216). He also believed that the warm-ups provided students a chance to self-assess regarding the previous day’s content. Charlie also saw the warm-ups as an opportunity for him to gather information about his students’ thinking. He stated, “I get to circulate and see what they’re writing. So, I get an idea of what types of things maybe they’re doing or they’re not doing” (Exit Interview, lines 79-80). It is possible this activity structure did not include many formative assessment codes because the teachers’ practices were not verbal. The teachers often monitored the students during this activity structure, however, they did not have many conversations with students. It is likely that teachers were looking at students’ work, though, and gathering evidence in their head of what they were seeing. However, since this work was not done verbally, it would not have been coded as formative assessment. While Lukas was the only teacher who used teacher giving information and feedback was the only formative assessment practice used, other formative assessment practices, such as learning targets could have been implemented. While reading the solutions to students’, teachers could remind students of how the problems on the assessment or assignment relate to the 254 learning targets to make an explicit connection for students. This helps students monitor their own learning and keep the goals at the forefront of their minds. It was also slightly surprising that the summary activities did not include many formative assessment practices since the nature of these would likely provide opportunities for questions, feedback, or learning targets. However, Lindsay was the only teacher who included formative assessment practices during this activity structure and all of them were questioning. If teachers use questioning during this activity structure, they could also provide feedback to students’ responses. In addition, just as teachers start the class with the learning targets, they could end the class reminding students of what they should have learned during the lesson. It is interesting that all these activity structure could have included additional formative assessment practices, especially learning targets. This implies that teachers need more support in sharing the learning targets with students and making explicit connections between the learning targets and what happens in their classrooms. Formative assessment practices levels of sophistication. When considering the levels of formative assessment practice implemented by the six teachers, almost all of them had at least one practice that was classified as high-level. Charlie demonstrated the most consistency in implementing high-level formative assessment practices, since he achieved levels greater than 2.50 for all the practices that he implemented (peer assessment was the only practice he did not implement). It is interesting that the three teachers (Charlie, Evan, and Scott) who were the only ones who provided their students an opportunity to self-assess, all achieved high-level self- assessment practices. Even though these teachers did not often implement self-assessment, it seems they are knowledgeable on how to do it well. All three teachers had structures in place for 255 students to use to self-assess, which likely made it easier for them to implement this formative assessment practice in a way that was meaningful for students. Gwen (2.67) and Charlie (2.63) were the only teachers who reached high levels of practice for learning targets, which means that when they shared the learning targets with students, they often made connections between the lesson’s activities and the learning targets. This comparison may be misleading, though, regarding the regularity of the teachers’ use of this formative assessment practice. Gwen only had three instances of learning targets during the observed lessons, with one at Level 2 and two at Level 3. Her use of learning targets also occurred on only two of the five observation days. Charlie had eight instances of learning targets with at least one occurrence on five of the six observation days. Five of his eight uses of learning targets were high-level (Level 3 or 4), which shows much more consistency in his practice compared to Gwen. The activity structures also contributed to the levels of formative assessment practice. Feedback was the only formative assessment practice that did not reach high levels of practice, however, Pair/Small Group Work (2.03) had the highest weighted average. Whole Class Discussion (1.81), which was the other activity structure with the most instances of feedback had the next highest weighted average. Since neither of these activity structures achieved high levels of practice, this means that much of the feedback was evaluative or focused on the students’ responses and not their method. In addition, this implies this is the formative assessment practice that teachers could use the most support in developing. The teachers provided higher quality feedback during pair/small group activities, which implies that it was easier for teachers to focus their feedback when talking to a group of students compared to the whole class, where they may have focused on big picture feedback, which would be lower level to relate to every student at 256 the same time. When learning targets were enacted during the whole class discussions, they tended to be high-level, but when questioning and feedback were enacted during the whole class discussion, they tended to be lower level. This is surprising because whole class discussion involved a lot of dialogue between the teacher and students, therefore, it was expected that questioning and feedback would be enacted at high-levels. It was slightly surprising that learning targets were high-level for this activity structure because they are not often associated with whole class discussions, however, it is possible that during these discussions, teachers connected back to the learning target to help students see the relationship between their discussion and what they were expected to learn from the activity. When self-assessment was enacted during pair/small group work, it tended to be high-level, but when questioning and feedback were enacted during pair/small group, they tended to be low-level. This was slightly surprising because self-assessment is often thought of as an individual activity but working with a small group was beneficial to students when they were reflecting on their learning because they could talk with their peers to check if their thinking was on track. Like whole class discussion, it was surprising that questioning and feedback were not implemented at high-levels during pair/small group work. When questioning was enacted during the summary activities, though, it was at high-levels (2.67). It was not surprising that the summary activities included high-level questioning practices because during this activity structure, it is likely that the teacher was checking students’ understanding one last time regarding the lesson content, therefore, s/he would want to be deliberate in the questions they are asking. When instructional decisions were enacted during the introduction activities, they tended to be at a high-level, but when learning targets were enacted, they tended to be low-level. This is surprising because it was expected that learning targets would be high-level, since that was when teachers most often shared the learning 257 targets with their students. All four activity structures that included self-assessment all had high levels of practice (3.00 or 3.50), which means these activity structures allowed the teachers to provide clear expectations for how students should be self-assessing. This also means that these teachers had the knowledge to implement this formative assessment practice at a high-level, which often meant using an established rubric or other tool. When instructional decisions were enacted during interactive lecture, it tended to be at high-levels, but when the other practices were enacted, they tended to be low-level. This is surprising in that the nature of lecture would not suggest the teacher making changes to their lesson based on that activity structure. It is likely that the participation of students in this activity structure influenced the flow of the lesson. It is not surprising, then, that the other formative assessment practices were low-level because the teachers were not asking high-level questions or providing feedback to their students during this activity structure. There were four activity structures that included high-level practices for teachers making instructional decisions. This shows that many of the activity structures provided teachers with opportunities to share their decision-making process with students to provide them with context for the reasons teachers had for determining the lesson’s activities. In addition, teachers were not limited by the activity structure in their ability to share their thinking about the flow of the lesson with their students. Supports and Obstacles for Implementing Formative Assessment Chapter 8 focused on what teachers perceived as helping and hindering them in implementing formative assessment practices in their mathematics classrooms. These supports and obstacles included both internal and external factors, with the large majority of them being classified as external. The teachers named internal factors that supported their implementation of formative assessment in terms of writing learning goals (N=2), determining questions to ask 258 students (N=2), and adjusting their instructional plan (N=3). Some of these internal supports included teachers’ previous teaching experience and their beliefs about themselves. McMillan (2003) and Suurtamm, Koch, and Arden (2010) also found that teachers’ beliefs helped them incorporate formative assessment in their classrooms. Multiple researchers (Cowie & Bell, 1999; McMillan, 2003; Hodgen & Marshall, 2005; Box, Skoog, & Dabbs, 2015; Gotwals, Philhower, Cisterna, & Bennett; 2015) have stated that a strong knowledge base for teachers supported their ability to implement formative assessment, yet none of the teachers in the current study named their own knowledge as a support. It is possible that the teachers were incorporating the development of knowledge when they talked about their prior teaching experiences, therefore, they did not explicating mention their own knowledge. External factors were broken down into eight sub-categories – Class Norms, Content, Logistics, Resources for Teachers, Resources for Students, School, Students, and Time. Teacher resources (N=16) and students (N=11) provided the most support for the teachers in this study for implementing formative assessment in their classrooms. Teachers stated resources for themselves regarding all the formative assessment practices, except self and peer assessment. Some of these supports included other teachers to talk or work with, their textbook, and professional development. This need to have conversations with peers and have others to share resources with was also found by Suurtamm and Koch (2014). In addition, Suurtamm, Koch, and Arden (2010) found that teachers’ curriculum supported their ability to implement formative assessment. Professional development was a common factor named by other researchers (Suurtamm, Koch, & Arden, 2010; Volante & Beckett, 2011; Box, Skoog, & Dabbs, 2015). Students supported teachers’ implementation of formative assessment by being willing to participate in the activities, showing their mathematical thinking via written work, and sharing 259 their strategies with the teacher and their peers. Interestingly none of the research articles I found stated how students supported teachers’ implementation of formative assessment, yet in the current study, students were an important factor. Teachers stated more obstacles (N=65) to implementing formative assessment than supports (N=53). They also named more than twice as many internal factors (N=17) that hindered their ability to use formative assessment than supported it (N=7). This seems to imply that the teacher’s beliefs, knowledge, and expectations are more likely to hinder than help their ability to implement formative assessment in their classrooms. The teachers listed internal factors that hindered their ability to implement formative assessment for all the practices, except providing verbal feedback. Some of the internal factors named by teachers included a lack of confidence in themselves, not knowing how to follow-up with professional development in their classrooms, the formative assessment practice not being a strength of the teacher, and a fear of not understanding students’ mathematical thinking. This lack of teacher knowledge or beliefs was consistent with previous research (Cowie & Bell, 1999; Volante, 2010; Box, Skoog, & Dabbs, 2015; Gotwals, Philhower, Cisterna, & Bennett; 2015). There was also one teacher (Gwen) who stated that nothing hindered her ability to write learning goals or determine questions for her students. The external factors were broken down using the same eight sub-categories as the internal factors. Students (N=22) were the greatest hindrance for these teachers when implementing formative assessment, as it had more than three times as many factors named as the next category, which was time. Students was the only external category that was greater than the number of internal factors stated by teachers. Students’ lack of prior mathematical knowledge, unwillingness to participate in class activities or do homework, status issues, and 260 multiple ability levels in one class period were all reasons teachers gave for how students hinder their ability to implement formative assessment in their classrooms. Students were not a common factor named in the previous research as hindrances to teachers implementing formative assessment, however, Volante (2010) did find that students’ perspectives of assessment hindered teachers in their study. Time was also a concern for teachers in that they did not think they had the time to plan or find resources to help them implement formative assessment. Box, Skoog, and Dabbs (2015) and Suurtamm and Koch (2014) also found that time hindered the teachers in their study in being able to implement formative assessment. It is interesting to note that high stakes testing was a common factor mentioned in previous research (Cowie & Bell, 1999; McMillan, 2003; Wiliam, Lee, Harrison, & Black, 2004; Volante & Beckett, 2011; Suurtamm & Koch, 2014; Box, Skoog, & Dabbs, 2015), yet none of the teachers in the current study named it or standardized testing of any kind as a hindrance in their ability to implement formative assessment. Summary This study investigated six high school mathematics teachers from four different high schools who claimed to be using formative assessment in their classrooms to see what these practices were and how they were implemented in the different contexts. Very little research has been conducted with high school mathematics teachers, therefore, this study contributes to the mathematics education research community by providing more information about which formative assessment practices mathematics teachers are implementing in their mathematics classrooms. There is also little research describing how teachers had implemented formative assessment in their classrooms and this study provided descriptions not only of the formative assessment practices, but backgrounds about the teachers and their school contexts. In addition, 261 an important contribution of this study is the discussion of the relationship between the formative assessment practices and the activity structures. Previous research has not discussed what activity structures provided teachers opportunities to implement different formative assessment practices, therefore, the current study introduces the idea that the activity structure could play a role in how teachers implement formative assessment practices in their classrooms. The most common formative assessment practices were questioning and feedback, which were expected since many mathematics classrooms include the teacher asking students questions. The least common practices were self and peer assessment, which was also not surprising, because previous research had also found limited implementation of these practices. It was surprising, though, that across the 33 focus lessons, none of the teachers used peer assessment. Even though the teachers did not implement formative assessment as often as expected, they did implement these formative assessment practices at varying levels of sophistication. For some of the formative assessment practices, such as self-assessment and questioning, teachers implemented them in different ways, which showed that there is not one way to implement these practices in mathematics classrooms. This study also showed that teachers in different school contexts can implement formative assessment in their mathematics classrooms, therefore, these types of practices are not limited to one type of school or student population. The supports and obstacles in implementing formative assessment named by teachers aligned well with previous research with a few exceptions. In the current study, external factors for supports and obstacles were largely influenced by students, however, previous research rarely named students as a factor. High-stakes testing was often named as a hindrance to teachers in previous studies, yet this was never mentioned by any of the teachers in the current study. 262 Implications This study introduced the possible relationship between formative assessment practices and activity structures. However, only six teachers participated in the study, therefore, more research needs conducted to gain more information about this relationship. Although the teachers in the current study claimed to be using formative assessment in their classrooms, limited occurrences of the different practices were observed. These practices were also seldom demonstrated using high-levels of sophistication. Therefore, more research needs conducted on what formative assessment practices teachers are using in their classrooms, as well as, what levels of sophistication they are implemented. Like previous research (Gotwals et al., 2015), self- assessment and peer assessment were rarely seen in the current study, which implies these practices are used less often than other formative assessment practices. Hence, additional research to investigate these specific practices could provide valuable information why these practices are not implemented as often. Previous research (Suurtamm, Koch, & Arden, 2010; Volante & Beckett, 2011; Box, Skoog, & Dabbs, 2015) has found that professional development has been shown to support teachers’ implementation of formative assessment. In this study, I found that teachers stated that professional development helped them implement formative assessment if it related directly to their classrooms, therefore, it seems that it might be useful for professional development to focus on suggestions for how to implement specific formative assessment practices in their classrooms. The teachers also stated the need to have others to talk with, therefore, professional development workshops should include opportunities for teachers to work together or follow-up with how they have implemented the different practices. Since research has shown that incorporating formative assessment practices into teachers’ classrooms requires a shift in thinking (Popham, 263 20050, it seems like it would be helpful to provide teachers with opportunities to focus on one formative assessment practice at a time as a way to scaffold them into using these practices more often and at higher levels of sophistication. While the current study provides information about how this group of six high school mathematics teachers used formative assessment in their classrooms, continued research on teachers’ formative assessment uses is necessary to gather more evidence of these types of practices and to provide examples for how other teachers could implement these formative assessment practices in their own classrooms. Like the current study, it is suggested the teachers from multiple school contexts should participate in the research, however, a larger sample of teachers is needed. Video-recording classrooms is imperative to allow for multiple viewing to ensure a more accurate coding process. Interviews with teachers are also necessary to gather more information about their thought processes and how they view the different formative assessment practices, since these likely influence how they implement the practices. 264 APPENDICES 265 Appendix A: Background Interview Protocol 1) How long have you been teaching? a. Total and at this particular school 2) Have you worked in the field of education in any other capacity? 3) What grade levels and courses have you taught? a. How long for each? 4) What courses are you currently teaching? 5) What curriculum are you using? a. How long have you been using it? 6) Could you tell me a little bit about your school? a. Population b. Demographics 266 7) Could you tell me a little bit about your educational background? 8) How is assessment used in your mathematics classroom? 9) How are the goals of the lesson determined? 10) Do you share the goals of the lesson with your students? If so, how? 11) How do you determine what questions you will ask students during mathematical a. For example, in whole class, small group, or individual 12) How do you know if students are making sense of the mathematical concepts you are 13) How do you determine students’ level of mathematical understanding during the lesson? 14) Do students self-assess in your mathematics class? a. Can you provide an example? 15) How do students know if they have met the goals of the lesson? 16) How do students determine their own strengths and weaknesses in relation to the goals of lessons? teaching? the lesson? 17) Do you provide feedback to your students? If so, how? 18) Do students provide each other with feedback? If so, how? 19) How do you decide if you need to adjust or modify your instructional plan? a. Can you provide an example? 20) How do you think about formative assessment? 21) What led to your interest in formative assessment? 22) Have you had any professional development opportunities related to assessment or questioning strategies? There are a lot of different strategies that teachers might employ to use students’ mathematical thinking in their teaching. I’ve listed some here… [writing learning goals, determining appropriate questions to ask students, providing feedback to students, having students self-assess, having students peer assess, making sense of students’ mathematical thinking, determining students’ level of understanding, adjusting or modifying your instructional plan based on students’ thinking] 1) Which of these are easy for you to implement? What makes them easy? 2) Which of these are hard for you to implement? What makes them challenging? 3) I’d like to go through each of these one by one… If this is something that you do or would like to do in your classroom, what do you think has helped or hindered you in doing it? 267 Appendix B: Lesson Planning Interview2 The teacher will be asked to explain their planning process and to walk me through the lesson I will be observing. After they have completed their explanation, the following questions will be asked if they were not mentioned during the initial explanation. 1) What resources do you use to plan the lesson? 2) How often have you taught this particular lesson (series of lessons)? 3) How have you changed the lesson from previous implementations? a. What led to these changes? 4) What are the goals of the lesson? a. How did you choose the particular goal(s) for this lesson? b. What do you want students to be able to do based on this lesson? c. What do you already know about students' current ideas about the topic? a. Probe for structure of lesson (i.e., exploration, practice time, group work, 5) How are the math practices addressed in the lesson? 6) How will you introduce the lesson? 7) What will students be doing during the lesson? discussion) 8) Do you have any questions planned that you will ask? a. What are they? b. How did you determine them? 9) How will you close the lesson? 10) What kinds of challenges do you anticipate students having and how will you address them? 11) What mathematical strategies do you anticipate students using? 2 Taken and modified based on ERGO Pre-Lesson Planning Interview Protocol 268 Appendix C: Lesson Debriefing Interview3 1) Describe the lesson. a. What was the lesson about? b. What were students doing? c. Did the lesson flow as you had planned it? 2) What kinds of student strategies and challenges did you observe? a. Provide an example of a student strategy you observed. b. Provide an example of a challenge a student faced. 3) What, if anything, surprised you about the lesson? 4) Do you think your students met the goals of the lesson? a. In what ways did students meet the goals? b. In what ways are they still working on the goals of the lesson? c. Provide examples 5) How might you revise the lesson in the future? 6) Based on what happened today, what will you do in tomorrow’s class? 7) What is the goal for tomorrow’s lesson? Questions to be asked after the last lesson debriefing interview (Note: These are subject to change based on what occurs during the observations and prior interviews.) 1) Do you think one subject provides more opportunities to incorporate formative assessment practices than the other? a. Explain and provide examples. 2) I noticed you did or had students do …, do you consider that formative assessment? 3) How have you used formative assessment during the lessons I’ve observed? 4) What other tools or strategies do you think are part of formative assessment? 5) What would you like to learn more about or focus on going forward? 3 Taken and modified based on ERGO Post-Lesson Interview Protocol 269 Appendix D: Classroom Video Formative Assessment Practices Coding Framework Defining Learning Targets Use of Learning Target Lower Anchor (Novice) 1 The teacher makes passing mention of the learning target (e.g., “you all know the learning target and so we will keep moving on…”) but does not go over the learning target or relate it explicitly to the class activities 2 The teacher reads the learning target aloud, but does not explicitly relate the learning target to class activities (e.g., “you know that our learning target is ‘I can …’. So now we are moving on to…”) 3 The teacher explains and/or reminds students of the learning target and relates it to their activities but does not thoroughly describe the types of knowledge, skills, and abilities students will be expected to master (e.g., “you know our learning target is ‘I can…’ and today we will do ___ activity to get at this learning target…”) Upper Anchor (Expert) 4 The teacher thoroughly explains and/or reminds students of the learning target and what will be taught, learned, and the types of knowledge, skills, and abilities students will be expected to master over the course of the lesson and/or unit (e.g., “you know our learning target is ‘I can…’ and today we will do ___ activity to get at this learning target. Specifically, you should be able to do _____ by the end of this activity…”) Teacher asks students what the learning targets are for the day (i.e., student involvement in creating their own learning goals Note: Do not count learning targets posted on the wall, unless the teacher explicitly mentions it 270 Questioning: Type of question / information elicited Promoting self- assessment4 and self- reflection Lower Anchor (1) The teacher asks only low depth of knowledge (DoK) questions that only have one correct answer (e.g., recalling and reproducing others’ ideas) or are skill/concept questions (Webb, 2005, 2009) The teacher does not ask students to explain their ideas, but may ask students “do you understand?” or “does this make sense?” Teachers tell students their evaluation of the students’ learning. Teachers have students keep track of their own grades Eliciting Student Understanding 2 The teacher mainly asks lower DoK questions, but has some higher- DoK questions included (or asks students to explain their responses sometimes). The teacher may still ask “does this make sense?” as evidence for student understanding 3 The teacher asks a mix of questions (perhaps building from lower-DoK questions to higher DoK questions as scaffolding tool). The teacher asks student why they arrived at an answer (e.g., a reflective toss), but may not push students to be metacognitive about their reasoning process. Upper Anchor (4) The teacher asks a mix of question (perhaps building from lower-DoK questions to higher DoK questions as scaffolding tool). The teacher asks the students to explain their ideas (asking “why”) and pushes students to be metacognitive in how they arrived at their answer. Teacher may push students to advance their thinking (e.g., “will your method always work?” (Crespo, 2002) Teachers encourage students to correct their own work and look for mistakes. Teachers teach the process of reflection and provide opportunities for students to practice. Students review their work. Teacher encourages students to check their thinking and adjust understanding OR OR Teacher provides structure and support for reviewing own work Teachers teach the process of reflection and provide opportunities for students to practice. Students learn how to think about thinking (metacognition) through teacher modeling. Students develop the ability to self-assess and monitor their learning progress. Teachers provide prompts for students to think deeper about their learning (MP, 2010). Teacher structure and support the process of self-assessment and connect it to learning goals. OR Teacher provides scaffolding to promote student metacognition or reflection on their learning or supports students in asking their own questions 271 Promoting peer assessment Teachers have students grade their peers’ work without informing their peers of the correct response Teachers ask students to do peer assessment, but do not provide them with supports or structures (e.g., rubrics or criteria). Teachers support students in providing descriptive feedback to partners (e.g., they have a rubric or criteria to guide them). Teacher provides structure, guidelines, and the use of criteria as a frame of reference (MP, 2010). Teachers structure and support the process of peer assessment and connect it to learning goals. OR Teachers have students only provide other students with the answer 272 Upper Anchor (4) Teacher provides descriptive feedback about the process for completing the tasks (strategies used) and on the task itself Feedback is specific enough for student to know what to do next, but not so specific that the task is done for them (giving students the answer) (Brookhart, 2008) Teacher provides feedback that moves learning forward – working with students to provide them the information they need to better understand problems and solutions (including promoting metacognition). Teacher uses students’ questions as a means to provide rich feedback Providing verbal feedback in the moment Providing Feedback Lower Anchor (1) Feedback is mainly evaluative (e.g., correct/incorrect). Teacher may provide the right answer if the response is incorrect and/or Feedback focuses on student(s) rather than on the learning or the tasks (e.g., “great, you are so smart”) Teacher does not use students’ questions as a means to provide feedback (either answers them or does not engage with them) 2 Feedback is mainly evaluative and focused on the outcome (the answer) of the task and not the process for getting the answer (problem-solving skills), but the teacher may provide ideas for moving forward (but the feedback may be too directive (essentially gives the answer) or too vague (it is not “actionable”)). There may be not enough or too much feedback throughout the class Teacher uses students’ questions as a means to provide feedback 3 Feedback is descriptive and focuses on both the outcome of the task (the answer) and how the student arrived at the answer (the process). The teacher may provide feedback on how to move forward, but the feedback may be either too directive (essentially gives the answer) or too vague (it is not “actionable”) Teacher uses students’ questions as a means to provide feedback 273 Making Instructional Decisions5 Adjustment to teaching and/or learning 2 Teacher is vague about rationale for instructional decisions (e.g., “I am going to do this again to help you understand this…”) but does not provide students with a clear evidence for why she is making decisions Lower Anchor (1) There is little or no attempt by the teacher to collect evidence of student learning in the lesson that is connected to the learning goals or criteria for success. AND/OR The teacher has no basis for making instructional decisions 3 Teacher provides a rationale for instructional decisions using vague evidence from students (e.g., “You just finished solving some problems and I am going to review those that I think were most difficult for you because you will need to know this for the exam”) OR The teacher suggests students’ adjustments in their learning, but is not systematic or does not provide guidance Upper Anchor (4) Teacher is explicit in her rationale for making specific decisions and uses evidence of student understanding for making the decision (“We will take some more time going over ____ because I noticed that some of you struggled with _____ and this is a big idea …”. OR The teacher explicitly involves students in making their own learning adjustments. 5 Note: Instructional decision must be explicit and/or apparent (e.g., it is clear to the observer that the teacher notes that she is making a change or it is clear from the agenda or planned course of the instruction that there are changes being made). Examples of instructional decisions: Re-teaching; teaching differently; engage students differently in learning: provide students with successful learning experiences, enrichment activities that expand their learning; grouping students differently; providing different activities; analyzing data from assessments with students; activating students’ prior knowledge; and adjusting assessments to better assess learning) 274 Appendix E: Lesson Activity Codes and Definitions Activity Code Warm-up Introduction to class and lesson Interactive lecture Individual activity Pair/ Small group work Presenting student work Whole Class Discussion Teacher giving information Summary Definition An individual or small group activity at the start of class that gets students thinking about the mathematical content; usually either connects to relevant prior knowledge or introduces the lesson's content A verbal or written description of the plan for the class that is discussed by the teacher; could include a mention or reminder of the learning goals, could also include the teacher making connections to the previous day's activity or lesson Teacher led instruction of the new lesson where students have low level participation by answering questions or offering suggestions on how to solve the problems Students are instructed to work individually on an activity Students are instructed to work with a partner or small group, such as 3-4 students on an activity Teacher shares student work or the students share their own work with the rest of the class (likely via a document camera or on the white board) followed by a discussion about the student's work, often led by the student Teacher and students participate in a discussion of the work students have done during class or on homework; this is often teacher led, but could be student led Teacher reads the correct answers to an assessment, in-class activity, or homework assignment, with no new content shared or discussion with students. Teacher shares a final thought or takeaway for the day's lesson 275 Appendix F: Coding Framework External Supports and Obstacles Definition These are factors that are part of the teacher's regular classroom practices. These are factors related to the mathematics content. These are factors related to making decisions about activities, assessments or pacing. These are resources that are available to the teacher for their use; this includes people and materials Resources for Teachers Resources for Students These are resources that are available to the students for their use. School These are factors related to the school structure. These are factors related to the students' interactions and experiences related to the classroom setting. These are factors related to the issue of time. Code Class Norms Content Logistics Students Time 276 REFERENCES 277 REFERENCES Andersson, C., & Palm, T. (2017). The impact of formative assessment on student achievement: a study of the effects of changes to classroom practice after a comprehensive professional development programme. Learning and Instruction, 49, 92-102. Ball, D.L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What makes it special?. Journal of teacher education, 59(5), 389-407. Black, P., Harrison, C., & Lee, C. (2003). Assessment for learning: Putting it into practice. Berkshire, England: Open University Press. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: principles, policy & practice, 5(1), 7-74. Boaler, J., & Brodie, K. (2004, October). The importance, nature, and impact of teacher questions. In Proceedings of the twenty-sixth annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (Vol. 2, pp. 774-790). Box, C., Skoog, G., & Dabbs, J. M. (2015). A case study of teacher personal practice assessment theories and complexities of implementing formative assessment. American Educational Research Journal, 52(5), 956-983. Brookhart, S. M., Andolina, M., Zuza, M., & Furman, R. (2004). Minute math: An action research study of student self-assessment. Educational Studies in Mathematics, 57(2), 213-227. Brookhart, S., Moss, C., & Long, B. (2008). Formative assessment. Educational Leadership, 66(3), 52-57. Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task‐involving and ego‐involving evaluation on interest and performance. British journal of educational psychology, 58(1), 1-14. Cameron, A., Gawroski, J., Eich, M., & McCready, S. (2011). Using classroom assessment to improve student learning. Reston, VA: National Council of Teachers of Mathematics. Campbell, C. (2013). Research on teacher competency in classroom assessment. In J.H. McMillan (Ed.), SAGE Handbook of Research on Classroom Assessment (pp. 71-85). Thousand Oaks: SAGE Publications, Inc. Chappuis, J. (2009). Seven strategies of assessment for learning. Boston, MA: Pearson, Inc. 278 Chappuis, J., Stiggins, R., Chappuis, S., & Arter, J. (2012). Classroom assessment for student learning. Boston, MA: Pearson, Inc. Choppin, J., Davis, J., Drake, C., & Roth McDuffie, A. (2012a). ERGO Background Interview on Teachers’ Perceptions of the CCSSM [Interview Protocol]. Unpublished Instrument. University of Rochester, Western Michigan University, Michigan State University, Washington State University Tri-Cities: National Science Foundation. Choppin, J., Davis, J., Drake, C., & Roth McDuffie, A. (2012b). ERGO Post-Lesson Interview on Teachers’ Perceptions of the CCSSM [Interview Protocol]. Unpublished Instrument. University of Rochester, Western Michigan University, Michigan State University, Washington State University Tri-Cities: National Science Foundation. Choppin, J., Davis, J., Drake, C., & Roth McDuffie, A. (2012c). ERGO Pre-Lesson Interview on Teachers’ Perceptions of the CCSSM [Interview Protocol]. Unpublished Instrument. University of Rochester, Western Michigan University, Michigan State University, Washington State University Tri-Cities: National Science Foundation. Cowie, B., & Bell, B. (1999). A model of formative assessment in science education. Assessment in Education: Principles, Policy & Practice, 6(1), 101-116. Dweck, C. S. (2003). Ability conceptions, motivation and development. British Journal of Educational Psychology, 13-27. Fontana, D., & Fernandes, M. (1994). Improvements in mathematics performance as a consequence of self‐assessment in Portuguese primary school pupils. British Journal of Educational Psychology, 64(3), 407-417. Franke, M. L., Webb, N. M., Chan, A. G., Ing, M., Freund, D., & Battey, D. (2009). Teacher questioning to elicit students’ mathematical thinking in elementary school classrooms. Journal of Teacher Education, 60(4), 380-392. Gearhart, M., & Saxe, G. B. (2004). When teachers know what students know: Integrating mathematics assessment. Theory into Practice, 43(4), 304-313. Ginsburg, H. P. (2009). The challenge of formative assessment in mathematics education: Children’s minds, teachers’ minds. Human Development, 52(2), 109-128. Good, R. (2011). Formative Use of Assessment Information: It's a Process, So Let's Say What We Mean. Practical Assessment, Research & Evaluation, 16. Gotwals, A. W., Lane, J., Cisterna, D., Philhower, J., Bennett, S., Kintz, T., & Roeber, E. (under revision). Testing hypotheses about teaching: A practice progressions approach to formative assessment. Educational Assessment Journal. 279 Gotwals, A. W., Philhower, J., Cisterna, D., & Bennett, S. (2015). Using video to examine formative assessment practices as measures of expertise for mathematics and science teachers. International Journal of Science and Mathematics Education, 13(2), 405-423. Harlen, W., & James, M. (1997). Assessment and learning: differences and relationships between formative and summative assessment. Assessment in Education, 4(3), 365-379. Heritage, M., Kim, J., Vendlinski, T., & Herman, J. (2009). From evidence to action: a seamless process in formative assessment? Educational Measurement: Issues and Practice, 28(3), 24-31. Hill, H., & Ball, D. L. (2009). The curious—and crucial—case of mathematical knowledge for teaching. Phi Delta Kappan, 91(2), 68-71. Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42(2), 371- 406. Hodgen, J., & Marshall, B. (2005). Assessment for learning in English and mathematics: a comparison. The Curriculum Journal, 16(2), 153-176. Hodgen, J., & Wiliam, D. (2006). Mathematics inside the black box: Assessment for learning in the mathematics classroom. London, England: GL Assessment Limited. Kohler, F., Henning, J. E., & Usma-Wilches, J. (2008). Preparing preservice teachers to make instructional decisions: An examination of data from the teacher work sample. Teaching and Teacher Education, 24(8), 2108-2117. McMillan, J. H. (2003). Understanding and Improving Teachers' Classroom Assessment Decision Making: Implications for Theory and Practice. Educational Measurement: Issues and Practice, 22(4), 34-43. McMillan, J. H. (2007). Formative Classroom Assessment: The Key to Improving Student Achievement. In J. H. McMillan (Ed.), Formative Classroom Assessment Theory into Practice. New York, NY: Teachers College Press. Minstrell, J., & van Zee, E. (2003). Using questioning to assess and foster student thinking. Everyday assessment in the science classroom, 61-73. Morris, A. K. (2006). Assessing pre-service teachers’ skills for analyzing teaching. Journal of Mathematics Teacher Education, 9(5), 471-505. Nathan, M. J., & Koedinger, K. R. (2000). An Investigation of Teachers' Beliefs of Students' Algebra Development. Cognition and Instruction, 18(2), 209-237. 280 National Council of Teachers of Mathematics. (2014). Principles to Action Ensuring Mathematical Success for All. Reston, VA: National Council of Teachers of Mathematics. National Research Council. (2001). Adding it up: Helping children learn mathematics. Washington, D.C.: National Academy Press. Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2), 199-218. Peterson, E., & Siadat, M. V. (2009). Combination of formative and summative assessment instruments in elementary algebra classes: A prescription for success. The Journal of Applied Research in the Community College, 16(2), 92-102. Popham, W. J. (2005). Classroom assessment: What teachers need to know. Boston, MA: Pearson/Allyn and Bacon. Rakoczy, K., Harks, B., Klieme, E., Blum, W., & Hochweber, J. (2013). Written Feedback in Mathematics: Mediated by Students' Perception, Moderated by Goal Orientation. Learning and Instruction, 27, 63-73. Reinholz, D. (2016). The assessment cycle: a model for learning through peer assessment. Assessment & Evaluation in Higher Education, 41(2), 301-315. Reinholz, D.L. & Gillingham, D. (2017). Forms of formative assessment: Eliciting and using student thinking. For the Learning of Mathematics, 37(1), 9-11. Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research & Evaluation. Ross, J. A., Hogaboam-Gray, A., & Rolheiser, C. (2002). Student self-evaluation in grade 5-6 mathematics effects on problem-solving achievement. Educational Assessment, 8(1), 43- 58. Schneider, M. C., & Gowan, P. (2013). Investigating teachers’ skills in interpreting evidence of student learning. Applied Measurement in Education, 26(3), 191-204. Shih, S.-S., & Alexander, J. M. (2000). Interacting effects of goal setting and self-or other- referenced feedback on children's development of self-efficacy and cognitive skill within the Taiwanese classroom. Journal of Educational Psychology, 92(3), 536. Smith, M. S., & Stein, M. K. (2011). Five practices for orchestrating productive mathematics discussions. Reston, VA: National Council of Teachers of Mathematics. 281 Suurtamm, C., Koch, M., & Arden, A. (2010). Teachers’ assessment practices in mathematics: Classrooms in the context of reform. Assessment in Education: Principles, Policy & Practice, 17(4), 399-417. Suurtamm, C., & Koch, M. J. (2014). Navigating dilemmas in transforming assessment practices: experiences of mathematics teachers in Ontario, Canada. Educational Assessment, Evaluation and Accountability, 26(3), 263-287. Suurtamm, C. A. (2004). Developing authentic assessment: Case studies of secondary school mathematics teachers’ experiences. Canadian Journal of Math, Science & Technology Education, 4(4), 497-513. Topping, K. J. (2009). Peer Assessment. Theory into Practice, 48(1), 20-27. Topping, K. J. (2013). Peers as a Source of Formative and Summative Assessment. In J. H. McMillan (Ed.), SAGE Handbook of Research on Classroom Assessment. Thousand Oaks: SAGE Publications, Inc. Volante, L. (2010). Assessment of, for, and as learning within schools: Implications for transforming classroom practice. Action in Teacher Education, 31(4), 66-75. Volante, L., & Beckett, D. (2011). Formative assessment and the contemporary classroom: Synergies and tensions between research and practice. Canadian Journal of Education/Revue canadienne de l'éducation, 34(2), 239-255. Wiggins, G. P. (1999). Assessing student performance: Exploring the purpose and limits of testing. San Francisco, CA: Jossey-Bass. Wiliam, D., Lee, C., Harrison, C., & Black, P. (2004). Teachers developing assessment for learning: Impact on student achievement. Assessment in Education, 11(1), 49-65. Webb, N. L. (2002). Depth-of-knowledge levels for four content areas. Language Arts. Webb, M., & Jones, J. (2009). Exploring tensions in developing assessment for learning. Assessment in Education: Principles, Policy & Practice, 16(2), 165-184. 282