#3er9. z. A .K ¢o. . :1 .1 II .F . we . z... Swag!“ rum, 2 e. L. ..! $.....u;..h...h...... ... $.91” _ l 3 Q . . .3! L.» X; .p L. $32. ask: s :3 f ‘ v. .r.Jv.:!.xl ‘ . nu»... 5.4? 3.5%. ‘I. 225.12.... 1.. no . a... » .1 1.3 Lgfiifiy. ; ’ 23:0 LIBRARY Michigan State _ University This is to certify that the dissertation entitled BREAKING THE MOLD: PREPARING GRADUATE TEACHING ASSISTANTS TO TEACH AS THEY ARE TAUGHT TO TEACH presented by SARA A. WYSE has been accepted towards fulfillment of the requirements for the PhD degree in Plant Biology Ecology, Evolutionary Biology & Behavior fl”; far-flu, , - Maj Professor’s Signat e 5 [71“, 20/0 Date MSU is an Affirmative Action/Equal Opportunity Employer PLACE IN RETURN BOX to remove this checkout from your record. To AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE DATE DUE DATE DUE 5’08 K:/Prolecc&Pres/ClRC/DateDue.indd BREAKING THE MOLD: PREPARING GRADUATE TEACHING ASSISTANTS TO TEACH AS THEY ARE TAUGHT TO TEACH By Sara A. Wyse A DISSERTATION Submitted to Michigan State University in partial fulfillment for the degree of DOCTOR OF PHILOSOPHY Plant Biology Ecology, Evolutionary Biology 8: Behavior 2010 ABSTRACT BREAKING THE MOLD: PREPARING GRADUATE TEACHING ASSISTANTS To TEACH AS THEY ARE TAUGHT TO TEACH By Sara A. Wyse Increasingly, graduate teaching assistants (TAs) are relied upon to teach introductory undergraduate biology courses at many large universities. This change in the role of the TA in the 19805 prompted the development of professional development programs for science, technology, engineering and mathematics (STEM) TAs. However, many of these programs, workshops and courses do not consider the TA in their dual role as learner and teacher, nor are they based on theories of how people learn. In addition, to date, there are little data published that. document TAs changes in behavior in response to TA professional development. This dissertation is the first comprehensive research on biology TAs teaching introductory biology (Biol) at a large university with very high research activity. This dissertation begins with research on Biol TAs, specifically, what do Biol TAs believe about effective teaching and student learning? Data from surveys of 30 Biol TAs revealed that they held beliefs about how teachers teach that opposed their beliefs about how students learn. These data, along with data collected through surveys about the effectiveness of their current professional development (i.e., traditional) influenced how we reformed TA professional development. Second, this dissertation describes a novel model of TA professional development (i.e., reformed) based on how people learn and data on Biol TAs (see Appendix A). Third, this dissertation evaluates the reformed model of TA professional development using data from surveys, TA—designed learning objectives and assessments, and TA classroom practice. Together, these data provided support for the efficacy of this model of biology TA professional development. Reformed professional development statistically significantly improved the cognitive processing levels TAs asked their students to achieve via learning objectives and assessments, and improved the degree of learner-centered instructional practices occurring in Biol TAs’ classrooms. Finally, results from a case study of one TAs who was prepared to teach under both traditional and reformed models of professional development indicated that the expectations of the TA professional development program play a key role in determining the instructional practices of a TA, more so than his own beliefs. This research presents the first reported data on biology TAs exploring the relationship among TA beliefs, professional development and practice. In addition, it also provides a novel reformed model of professional development (Appendix A) and the only data on evaluation of biology TA professional development that is not self-report The results from this research indicated the importance of critically examining and changing TA professional development as faculty work to reform undergraduate biology courses. Copyright by Sara A. Wyse 2010 DEDICATION This dissertation is dedicated to my God and Savior. It is through Him that all wisdom, knowledge, and understanding are possible. I also dedicate this dissertation to my husband, family and friends who have continually supported and encouraged me. ACKNOWLEDGEMENTS There are many people I must personally thank for their instrumental role in my dissertation research. First, I thank my advisor, Diane Ebert-May (PhD), for her guidance, encouragement and mentoring. Second, I thank the support of the research group at Michigan State University for their critical evaluation of my work, and for their active role in the reform of TA professional development. These people include: Drs. Tammy Long , lenni Momsen, and Elena Bray Speth. Third, I thank my committee members; Drs. Rique Campa, Natasha Speer and Andy Jarosz for their thoughtful comments and questions about my work throughout my duration as a graduate student. Fourth, I thank the Biological Sciences Program and Director John Merrill (PhD) for allowing me to conduct this research as an innovative teaching assistant (TA). Without the support and participation of Sonya Lawrence, Chuck EIzinga (PhD) and the TAs in Biological Sciences this research would not have been possible. Thank you. Finally, I acknowledge the funding support I received from Tammy Long’s NSF Grant (DUE 08172 24) and from the Future Academic Scholars in Teaching (FAST) program at Michigan State University. vi TABLE OF CONTENTS LIST OF TABLES x LIST OF FIGURES xi KEY TO ABBREVIATIONS xiii CHAPTER 1 INTRODUCTION 1 1.1 Introduction 1 1.2 Relevant Background and Statement of the Problem 2 1.3 Purpose and Research Questions 5 1.4 Significance 6 1.5 Outline 7 CHAPTER 2 CHARACTERIZING EPISTOMOLOGICAL BELIEFS OF BIOLOGY TAS ......... 9 2.1 Introduction 9 2.2 Relevant Theories 12 2.2.1 Preperties of the Construct of Beliefs 13 2.2.2 Distinction between Beliefs and Knowledge 14 2.2.3 Beliefs Classification 14 2.2.4 Belief Change 14 2.3 Methods 16 2.3.1 Course Selection 16 2.3.2 Experimental Design 16 2.3.3 Description of Participants 17 2.3.4 Survey Sampling 18 2.3.5 Coding and Data Analysis 20 2.4 Results 23 2.4.1 What do Biol TAs Believe about Effective Teachers? 23 2.4.2 What do Biol TAs Believe about the Role of the Teacher? ...................... 25 2.4.3 What do Biol TAs Believe about the Purpose of Teaching? .................... 27 2.4.4 What do Biol TAs Believe about Student Learning? 28 2.4.5 What do Biol TAs Believe about the Role of the Learner? ...................... 29 2.4.6 What Variables Influence TA Classroom Practice? 30 2.5 Discussion 31 2.6 Implications for TA Professional Development 36 CHAPTER 3 EVALUATING REFORMED TA PROFESSIONAL DEVELOPMENT .............. 40 3.1 Introduction 40 3.2 Evaluating Professional Development 41 3.3 Traditional and Reformed Professional Development 44 3.4 Research Design and Participants 45 ' 3.4.1 Study Design 45 3.4.2 Study Participants 47 vii 3.5 3.6 3.7 3.8 CHAPTER 4 A CASE STUDY OF A BlOl TA'S BELIEFS AND PRACTICES ......................... 4.1 4.2 4.3 4.4 4.5 4.6 4.7 Methods 3.5.1 Level 1: TA Reactions 3.5.2 Level 2: TA Learning 3.5.2.1 Learning Objectives 3.5.2.2 Assessment of Student Learning 47 47 49 49 50 SO 3.5.2.3 Assessing Cognitive Processes 3.5.3 Level 3: TA Application 3.5.3.1 Reformed Teaching Observation Protocol 52 53 3.5.3.2 Statistical Analyses S4 Results 3.6.1 Level 1: TA Reactions 56 56 3.6.2 Level 2: TA Learning 3.6.2.1 Learning Objectives 3.6.2.2 Assessment of Student Learning 57 S7 58 3.6.2.3 Alignment of Objectives and Assessments 3.6.3 Level 3: TA Application 3.6.3.1 Comparison between Traditional and Reformed .................... 3.6.3.2 Comparison between Reformed 1 and Reformed 2 ................ Discussion 3.7.1 Level lzTA Reactions 3.7.2 Leve12:TA Learning 3.7.3 Level 3: TA Application Conclusions Introduction Professional Development for Graduate TAs Unique Challenges of TA Professional Development Research Design 61 62 65 66 70 70 72 73 75 78 78 79 80 82 4.4.1 Focal Course - Biology 1 82 4.4.2 Jack 83 4.4.3 lack’s Professional Development 4.4.3.1 Traditional Professional Development Model 84 84 4.4.3.2 Reformed Professional Development Model 86 Methods 4.5.1 Surveys 4.5.2 Videotapes of Iack’s Classroom Practices 4.5.3 Interview Data Analysis 4.6.1 Coding of Survey Results and Interview Transcripts 4.6.2 lack's Classroom Practice Videotapes 4.6.3 Jack’s Review Results 4.7.1 What does lack Believe about the Teacher and Teaching? ............ 4.7.2 What does jack Believe about the Learning and Learning? ........... viii 89 90 90 91 92 92 92 93 93 .......... 93 .......... 98 4.7.3 How did Traditional Profession Development Influence jack? .............. 99 4.7.4 How did Reformed Professional Development Influence jack? .......... 101 4.7.5 Motivation for jack's Classroom Practices 102 4.8 Implications and Future Directions 107 CHAPTER 5 CONCLUSIONS 110 5.1 Summary 110 5.2 Unique Contributions 112 5.3 Limitations and Future Research 113 APPENDICIES 116 A Reforming TA Professional Development in Biology 116 B Surveys 147 C Coding Categories for Qualitative Data 152 D Lesson Plan for TAs 159 E Interview Questions 163 REFERENCES 164 ix 2.1 3.1 3.2 3.3 3.4 4.1 A.1 LIST OF TABLES Coding scheme example for “The qualities of an effective teacher are" ........... 22 Points of comparison between professional development models (Part 1) ...44 Bloom's taxonomy of the cognitive domain 51 Pairwise comparisons of Bloom's ratings for TA-designed assessments ......... 61 AN OVA table for semesters of TA professional development by RTOP ............ 63 jack’s beliefs about teaching and learning 95 Point of comparison between professional development models (Part 2) .. 143 LIST OF FIGURES 1.1 Model of data-driven chapters and their relationships 8 2.1 Coding categories for TA survey responses relating to beliefs 21 2.2 Spatial map of Biol TAs' beliefs about effective teachers 24 2.3 Spatial map of Biol TAs' beliefs about the role of the teacher 25 2.4 Spatial map of Biol TAs’ beliefs about the purpose of teaching .......................... 27 2.5 Spatial map of Biol TAs' beliefs about how students learn science ................... 29 I 2.6 Spatial map of Biol TAs' beliefs about the role of the learner 30 2.7 Variables influencing Biol TAs’ classroom practices 31 2.8 Summary of Biol TA beliefs across the five continua 32 3.1 Kirkpatrick’s Evaluation Hierarchy 42 3.2 TA reactions to professional development models 56 3.3 Bloom's level for traditional laboratory student learning objectives ................ 57 3.4 Bloom’s level for reformed laboratory student learning objectives .................. 58 3.5 Distribution of TA-designed assessments in traditional professional development 59 3.6 Bloom’s level for TA-designed assessments in reformed professional development 60 3.7 Mean RTOP score for TAs by semester of professional development ............... 63 3.8 Mean scores on five RTOP sub-scales by semester of professional development 64 3.9 Comparison of RTOP scores for TA 1 between traditional and reformed (R1) TA professional development models 65 3.10 Comparison of tape 1 RTOP scores for 4 TAs teaching during R1 and R2 ....... 67 3.11 Comparison of tape 1 instructional design sub-scale score for 4 TAs teaching during R1 and R2 67 xi 3.12 Comparison of tape 1 content knowledge sub-scale score for 4 TAs teaching during R1 and R2 68 3.13 Comparison of tape 1 student-teacher relationship sub-scale score for 4 TAs teaching during R1 and R2 68 3.14 Comparison of tape 2 RTOP scores for 4 TAs teaching during R1 and R2 ....... 69 4.1 Reformed professional development for Biol TAs 87 4.2 Design of data collection during Iack’s three semesters in Biol .......................... 90 4.3 Model of Jack's beliefs about teaching and student learning 101 4.4 Model of influences on lack’s classroom practice 107 Al Designing TA professional development 123 A.2 Reforming TA professional development in Biol 127 xii TAs EAs Biol RTOP R1 R2 STEM KEY TO ABBREVIATIONS Graduate Teaching Assistants Undergraduate Educational Assistants Introductory Biology 1 Course Reformed Teaching Observation Protocol Traditional Model of TA Professional Development First Semester of Reformed TA Professional Development Second Semester of Reformed TA Professional Development Science, Technology, Engineering and Mathematics xiii CHAPTER 1 INTRODUCTION 1-1 lnILQdufliQn Biology of the 21st century is complex and interconnected. One challenge college biology instructors face is how to effectively teach students about complex systems and interconnections in the classroom. The more than two decades emphasis on reforming undergraduate biology (e.g., Boyer 1998, NRC 2003, AAAS 2009) points to the fact that many of our current practices in the classroom are not effective for teaching students the science they need to become practitioners, teachers, researchers and informed citizens. While faculty professional development programs were implemented in response to these needs, data from these reform efforts are generally sparse. One recent study examining the influence of intensive professional development on university faculty teaching practices found that the majority of faculty teach biology to undergraduates through lecture with only minor student interaction (Ebert-May et al. submitted). This study (Ebert-May et al. submitted) provides some of the first evidence that very few changes have transpired in college classrooms despite repetitive calls to teach biology as biology is practiced. Ebert-May et al. (submitted) also provided data to support the claim that in general faculty who have been teaching longer use fewer learner-centered pedagogies than early-career faculty. Results such as these suggest that the focus of the reform might have more influence if it were targeted at early-career faculty, post-doctoral scholars and graduate students. If quality learner-centered 1 professional development is provided to graduate students and post-docs as they Ieam how to teach, then it may still be possible to change the face of undergraduate biology to more accurately reflect the science we teach (Handelsman et al. 2004). Recently, Bruce Alberts reflected on the current state of undergraduate science education stating: ...When I taught, I rarely sought to build on what other teachers had developed before me. This difference between how scientists approach their research and their teaching goes a long way, I believe, to explain why the quality of university science education lags so far behind the quality of science itself (Alberts 2010). We must, as scientists, make instructional decisions based on data about how students learn and how to effectively teach them. Therefore, changing the quality of undergraduate science requires a fundamental change in the way we prepare our future college instructors to teach 'f we must prepare them to teach in the manner we wish they would teach in their classroom. Stated another way, we must prepare them to teach scientifically by modeling scientific teaching (Handelsman et al. 2004) in their professional development opportunities. This research provides some initial evidence that when quality professional development is provided to biology instructors, they begin to change their teaching practices; instructors begin to teach as they are taught to teach (Lortie 1975). 1.2 32W Acknowledging that early-career faculty are not as effectively prepared to teach as they are to do research (Adams 2002), a growing body of research emerged exploring the role of graduate programs in preparing students for their careers. In the 19605, when TAs first began assuming roles at universities, there was virtually no TA training (Chism 1998). As teaching responsibilities became more closely aligned with those of faculty, faculty began to discuss the logistics of how to properly evaluate TAs, and how to assign TAs to particular courses (Chism 1998). It was not until the 19905 that research on TAs drawing from cognition and teacher development literature as theoretical frameworks, considered how TAs grow and develop as instructors (Sprague & Nyquist 1991). This groundbreaking work provided the first models of TA development and set the stage to fundamentally refocus TA preparation. Therefore, the professional development I focus on in this research is that which prepares graduate students specifically for the aspects of classroom instruction, specifically pedagogical approaches. In 1989, Abbott, Wulff and Szego laid out future directions for research on TA training, such as the need for more empirical research on TA training, research on longitudinal TA training, and how to integrate knowledge of the student learner into TA training. A review of the literature reveals that many of these questions remain unanswered (Nyquist et al. 1991, Speer et al. 2005). Universities are addressing a need for improved TA training in numerous disciplines, including biology, chemistry, physics, geology and mathematics (Druger 1997, Etkina 2000, DeFranco and McGivney-Burelle 2001, Hammrich 2001, Stamp and Pagano 2002). Particularly in biology, seminars, workshops and courses were developed for TAs that emphasized teaching science as science is practiced (Handelsman et al. 2004, Miller et al. 2008). However, these training opportunities have not coincided with rigorous evaluation of their impact on classroom practices. Often, evaluations of TA training programs/ workshops are based on self-report information, typically an end-of-workshop survey. TAs, like teachers and college faculty, may respond that they will apply what they learned in their classrooms (Kane et al. 2002). These self-report data, in one case, only explain 28% of the variance in reformed teaching practice for one cohort of university faculty (Ebert- May at al. submitted). Therefore, it is important to consider other variables, such as aspects of classroom practice and professional development that may influence teaching practices. Studies focused on these variables will be some of the first to use classroom data to confirm whether individual instructors actually implement what they have learned through professional development. Despite the growing interest in TAs over the past two decades, only a few research studies evaluate the impact of TA training on TA beliefs about teaching and learning and their classroom practice (Abbott et al. 1989, Nyquist et al. 1989, Speer et al. 2005). In the K-12 setting, teachers' beliefs about teaching and learning were important to ascertain how teachers develop, and in some cases, beliefs were found to correlate with observed instructional practice (e.g., Anning 1988). No such link is established for science TAs. TAs’ pre-existing beliefs about teaching and learning should be considered when designing TA professional development, because these beliefs may have the potential to influence the professional development outcomes, just as Iearners' pre-existing conceptions influence learning and must be considered when designing instruction (Pajares 1992, Thompson 1992, Calderhead 1996, Kember and Kwan 2000). Establishing relationships among TAs beliefs about teaching, learning and the translation of these beliefs to classroom practice is important; only then can TA training become more meaningful and help facilitate professional development of TAs, giving rise to instructors who are able to teach in the ways that students learn best. Given that most faculty were once TAs (Nyquist et al. 1989) and the students in their courses are the next generation of citizens, understanding the relationship between TA professional development, beliefs and classroom practice is of utmost importance. 1.3 Eumcsundfimamhfiuestigns The purpose of this research is to: (l) evaluate the traditional professional development for TAs teaching Introductory Biology 1 at a large, research-intensive university in the Midwest, (2) propose and implement a new model (reformed) of professional development for TAs, and (3) evaluate the impact of the new model of professional development on TA beliefs and classroom practice. The research questions that are driving this dissertation are: 1) What do Introductory Biology 1 (Biol) TAs believe about teaching and student learning? 2) How are Biol TAs traditionally prepared to teach? 3) What is the resulting TA classroom practice when TAs are prepared traditionally? a. What levels of cognitive processing are students asked to achieve? b. How do TAs assess student learning? Are assessments aligned with objectives? c. To what degree is TA instruction learner-centered? 4) How are Biol TAs prepared to teach under reformed professional development? 5) What is the resulting TA classroom practice when TAs are prepared with reformed professional development? a. What levels of cognitive processing are students asked to achieve? b. How do TAs assess student learning? Are assessments aligned with objectives? c. To what degree is TA instruction learner-centered? 6) ,What is the experience of one graduate teaching assistant teaching Biol while participating in both traditional and reformed professional development? 1-4 Significance The need to prepare TAs to teach in learner-centered courses is occurring in response to national reports for reform and accountability for learning (e.g., AAAS 2009). This research represents one of the first studies in biology to investigate TA beliefs about teaching and student learning, the impact of a reformed professional development program on TA practice, and the relationship between TA beliefs and their classroom practices. This work builds upon studies of K-12 teachers’ beliefs and extends it to a new population of early career instructors who differ from university undergraduates preparing to obtain their teaching certification (i.e., pre- service teachers) in a couple ways. First, biology TAs are not solely focused on becoming a teacher; they have a primary responsibility to conduct science research. As a result, TAs have varying interest in learning how to teach or in teaching. Secondly, TAs are embedded in a research-intensive culture which may not support efforts of graduate students in the classroom (Latulippe 2007). This culture may differ significantly from that of the environment pre-service teachers, and as a result, may influence the process of TAs learning to teach. As a result of these differences, TAs may have unique beliefs about teaching and Ieaming that may contribute to understanding the relationship between beliefs and classroom practices. The findings from this research will inform the development of future TA professional development programs. 1.5 Outline This research is part of a broader initiative involving a team of biologists working to implement research-based scientific teaching (i.e., active and learner- centered) in an introductory biology course (Biol) at a large research university. My research uses mixed methods to evaluate the effectiveness of TA training. My dissertation is divided into this introduction chapter, three data-driven chapters in response to my research questions, and a conclusion chapter (Figure 1.1). In Chapter 2, I explore my first research question by characterizing TAs teaching Biol in terms of their beliefs about teaching and student learning (i.e., their epistemological beliefs). These data informed the development of a reformed model of TA professional development that is described in theory and in practice in Appendix A. F Chapter 3 ] I Chapter 2 J \ [ Chapter 4 ] Evaluating Professional / \ Development K Epistemological \ TA Case Study Beliefs of TAs -What is the nature of TA What is the classroom practice when experience of one What do TAs they are prepared TA in believe about tradItIonally? (R02 8‘ 3) Introductory teaching and , Biology 1 student learning? ~What '5 the nature 0f TA professional (RQl) classroom practice when development? they are prepared to teach ' K j With reformed professional k (R0 5) j development? (R0 4 & 5)) Figure 1.1. Model of data-driven chapters and their relationships. Chapter 2 considers TA beliefs, which form part of the foundation for the reformed mode of TA professional development. This reformed model is described in detail in Appendix A. Chapter 3 provides evidence to support the efficacy of reformed TA professional development Chapter 5 investigates the experience of being a TA participating in professional development. In Chapter 3, I present the evaluation of the reformed model of professional development in comparison to the traditional approach to TA professional development in Biol. This chapter focuses on how TAs responded to professional development, what they learned and what they applied from their professional development in their classroom practices. In Chapter 4, I present a case study of one TA teaching Biol across both traditional and reformed professional development. Finally, in Chapter 5, l summarize the results of my dissertation work; provide insight into the novel contributions of this work, the limitations of this work, and the future research directions derived from this research. CHAPTER 2 CHARACTERIZING EPISTEMOLOGICAL BELIEFS OF BIOLOGY TAS 2-1 W Across a dIVersity of disciplines, researchers are interested in determining the nature of the relationship between people’s beliefs and their actions. Specifically in education, researchers are interested in epistemological beliefs - teachers’ beliefs about knowledge and Ieaming. Beliefs teachers hold about what constitutes effective teaching have been widely researched since the 19805, especially in undergraduates pursuing a K-12 teaching certification (i.e., pre-service teachers). The results of this research have contributed to understanding the role of teacher beliefs and their relationship to classroom practices. Researchers discovered that beliefs are important in learning to teach, and in a teachers' classroom practice. It is unclear whether stated beliefs are consistent with beliefs expressed through classroom practice (Shirk 1972, Anning 1988, Short and Short 1989, Smith and Neale 1989), or whether stated beliefs are inconsistent with practice (Galton et al. 1980, Thompson 1982, Cooney 1985). One hypothesis as to why some studies report alignment between beliefs and practice and others do not is that analyses were not directed at a level that would elicit such alignment (Speer 2008). It may be that inconsistencies are really differences in definitions between the researcher and the teacher (Speer 2005), or that studying independent beliefs rather than a “collection of beliefs" does not allow the researcher to elucidate the true beliefs of the teacher (Speer 2008). Thus, more research is needed to determine the nature of the relationship between beliefs and practices. In addition, studies of pre-service teachers also provide insight into the role of beliefs in professional development. For example, Holt-Reynolds (1992) interviewed nine pre-service teachers to elicit beliefs about what makes “good teaching." Holt-Reynolds found that this sample of pre-service teachers' beliefs about “good teaching" was largely influenced by their own experiences as students. These experiences were cited as evidence by pre-service teachers for rejecting pedagogy presented in their education coursework. Following coursework focused on learner-centered instruction, pre-service teachers cited their own beliefs about student learning as rationale for why they did not agree with the learner-centered pedagogies. Specifically, they believed that an individual students' interest and attention during class meant they were engaged in active Ieaming. Therefore, they concluded that a lecture was an active approach to teaching. A review of the literature on pre-service teachers' beliefs shows that pre- service teachers have solid beliefs about teaching and learning developed through their years of observing teachers (Lortie 1975). These pre-existing beliefs have the potential to influence classroom practice and teachers' response to professional development (Brown and Borko 1992, Richardson 1995, Richardson 1996, Kane et al. 2002). Teachers who believe active learning is required for student learning provide a different set of classroom activities than those teachers who hold a more traditional, passive belief of student Ieaming (Anning 1988). Additionally, research on pre-service elementary teachers suggests that beliefs influence what concepts 10 and Ieaming theories pre-service teachers are willing to accept in their teacher preparation (Borko and Putnam 1996). If beliefs have important implications for a teacher's classroom practice and their response to professional development, it is important to ascertain what teachers believe. While several studies have examined beliefs in pre- and in-service (i.e., practicing teachers) K—12 teachers (Pajares 1992, Barkastas-Tasos and Malone 2005, Beswick 2007), few have considered beliefs of instructors at the collegiate level (Kane et al. 2002). Specifically, science graduate teaching assistants (TAS), one population of teachers in higher education, have received relatively little attention in the research on teacher beliefs. These arguably distinct college-level instructors are in the process of teaching and Ieaming to teach with little to no guidance provided to them. Biology TAs often teach courses to meet graduation requirements by their department, or in return for financial support which enables completion of their degree. This results in a population of teachers who may or may not be interested in teaching and who have received little (if any) formal training in teaching methods. Thus, TAs may have markedly distinct beliefs about teaching and student learning when compared to pre- and in-service teachers. It is important to consider the beliefs of TAs because TAs teach a large number of undergraduates, especially in STEM disciplines, in their introductory courses (Allen and Rueter 1990). These courses often include recitations, discussions or laboratory sections (Chism 1998), which offer the opportunity for instruction and interaction with a significantly smaller group of students than the larger lecture classes. Knowing what TAs believe about teaching and student 11 Ieaming has the potential to influence the design and implementation of professional development opportunities provided to TAS, and subsequently, the instruction of large numbers of undergraduates. There are few studies investigating TA beliefs about teaching and student learning, and even fewer studies that incorporate TA beliefs into models of professional development (Speer et al. 2005). Therefore, the aim of this research study is to characterize and describe biology TA beliefs about teaching and student learning. What do biology TAs believe about teaching and student learning, and what do TAs report as variables that influence their classroom practice? Results from this work have the potential to impact the development and/or refinement of professional development for TAs. 2.2 Western The study of peoples' beliefs cuts across many fields of study. For example, anthropologists study cultural or religious beliefs; social psychologists may study the formation of beliefs, and researchers in education may study epistemological beliefs - beliefs about knowledge and learning. Belief research focuses on what an individual believes about an object/concept/theory that is of interest to the researcher. In the case of teacher beliefs, for example, researchers are interested in what teachers believe about teaching (e.g. Calderhaed 1996, Pajares 1992), student Ieaming (e.g. Borko and Putnam 1996), or the nature of knowledge (e.g. Calderhead 1996), for example. Despite all the research on teacher beliefs, beliefs remain a challenging construct to define. To complicate things further, beliefs are inconsistently referred 12 to in the literature, sometimes reported as values, attitudes, knowledge, convictions or judgments (Pajares 1992). For example, Richardson (1996) defines beliefs as “psychologically held understandings, premises or presuppositions that are held to be true" (Richardson 1996), where as DeFord’s (1985) definition of beliefs is that they are “attitudes which direct perceptions and behaviors" (DeFord 1985), and Green (1971) defines beliefs as a “proposition that is accepted as true by the individual holding the belief" (Green 1971). I use Green's definition of beliefs to guide my research on TA beliefs about teaching and student Ieaming. 2.2.1 Properties of the Construct of Beliefs While an agreed upon definition of beliefs is challenging to achieve (e.g., Green 1971, DeFord 1985, and Richardson 1996), there is a better consensus on the properties of the belief construct. First, beliefs are individualized to each person (Thompson 1992). Second, beliefs are held with varying degrees of conviction. For example, beliefs that are both verbally described and visible in practice (i.e., “core beliefs”), may be more tightly held than peripheral beliefs (i.e., beliefs verbally defined but not also visible in practice) (Haney and McArthur 2002). Core beliefs are often associated with one's identity or derived from vivid personal experiences; these beliefs are more resistant to change than peripheral beliefs (Haney and McArthur 2002, Kane et al. 2002). Third, beliefs are affective, often based on memories or experiences (Thompson 1992). Fourth, beliefs are not static. They are subject to change and restructuring (Schram et al. 1998, Hollingsworth 1989). Finally, an individual's beliefs act as a lens for viewing and responding to new information (Kane et al. 2002). 13 2.2.2 Distinction between Beliefs and Knowledge In psychology, beliefs and knowledge are often used interchangeably. However, in philosophy, beliefs are distinct from knowledge for reasons most clearly articulated by N espor (1987). First, Nespor states that beliefs are based on things that are true only to the individual while in contrast; knowledge is a “social construct" that is based on presumptions that are agreed upon by others. Second, beliefs are views of an alternative, sometimes ideal state, which is often in conflict with the realities of the world. Third, beliefs are strongly affective and subjective; in contrast, knowledge is not subjective. Fourth, beliefs are associated with memories in one's personal life; knowledge, on the other hand, may be indifferent to personal memorable moments (Nespor 1987). Due to these four key distinctions, 1 base my research on the philosophy framework that knowledge and beliefs are distinct constructs. 2.2.3 Belief Classification Since individuals can hold beliefs about a broad range of ideas, it is helpful to classify beliefs into categories. Beliefs about teaching and student learning are often classified into numerous categories (Borko and Putnam 1996; Calderhead 1996). For the purposes of this research, I focus on two categories of beliefs, as defined by Calderhead (1996) that will enable me to understand Biol TA beliefs: (1) beliefs about the teacher and teaching, and (2) beliefs about the learner and Ieaming. 2.2.4 Belief Change Belief change is a complex process partially due to the dual role beliefs play in the process. Often, beliefs are both the target of change, and the mechanism for 14 change in classroom practice. Some researchers propose that changes in beliefs contribute to changes in practice while others propose that teachers must change their practice first and their beliefs will change second (Richardson 1995). Change models identify‘common developments in ones' thinking that must occur in order to facilitate change (e.g., Posner et al. 1982, Guskey 1986, Gerber and Hoffmann 1998). These developments include (1) learning that there are issues and or problems with their current ways of performing a task or job, (2) realizing that there are alternative ways to do the task, (3) understanding what is required to make the change, and (4) growing towards the “tipping point” (coined by Morton Grodzins), or the point where one will act and apply the desired change (Posner et al. 1982, Gerber and Hoffmann 1998, Roling and Wagemakers 1998). Others suggest that change will likely occur only when the desired change is consistent with their beliefs, not too difficult to use or learn, and tried out before adaptation is required (Rogers 1995). While change models differ, it is clear that change occurs along a continuum over time, where individuals make a conscious decision to apply the new way of thinking in their practice. Research on pre-service and in-service teachers has shown that changes in beliefs are possible (Schram et al. 1988, Hollingsworth 1989). For example, a study by Hollingsworth (1989) showed that pre-service teacher beliefs changed from a teacher-centered perspective to a learner-centered in response to professional development. Therefore, this research makes the assumption that beliefs can and do change. 15 2.3 Mejhgds 2.3.1 Course Selection My research focuses on a 100-level introductory biology course (Biol) in genetics, evolution and ecology, at a large public university with a very high degree of research activity (Carnegie Foundation 2005). Biol is offered throughout the academic year and students enroll simultaneously in the lecture and laboratory. Lecture sections are large (150-500 students) and are taught by university faculty. Laboratory sections are much smaller, with enrollment capped at 32 students. TAs teach the laboratory portion of this course. Each academic year Biol serves over 2,000 students, including majors and non-majors. This course employs about 15 TAs each semester from a variety of life science departments, such as plant biology, zoology, and fisheries and wildlife, each semester to teach the laboratories. Often, TAs teach courses with just a few (2-7) other TAS, so having a sample size of about 15 TAS each semester makes this relatively large and diverse group of TAs in Biol an ideal population in which to study TA beliefs. 2.3.2 Experimental Design The TAs participating in this study taught Biol during one or more of the following semesters: Spring 2008, Summer 2008, Fall 2008 or Spring 2009. In addition to participating in university-wide TA training at the start of their first semester in graduate school, TAs in Biol participated in professional development provided by the Biol program. This professional development consisted of a one- and-a-haIf-day workshop at the beginning of each semester introducing the Biol program, course, job expectations, laboratory safety, and preparation for teaching 16 the first lab of the semester. Weekly, TAs participated in a 3-hour meeting to recap the prior week of teaching, and prepare for the next week of laboratory instruction. At the end of the semester, TAS participated in a final preparation meeting to reflect on the semester, gather feedback for improvements and turn in final student grades. TAs were expected to attend every professional development meeting as part of their job description. Their evaluation in Biol included participation in these meetings. To gain insight into TA beliefs, TAs completed surveys at the beginning and end of each semester they worked as a TA in Biol. 2.3.3 Description of Participants Thirty-six Biol TAs were presented with the option of participating in this research study, and 33 self-selected to participate. In Biol, TAs are appointed by their departments as TAs. Therefore, participating TAs are a sub-sample of biology TAs in a university setting with this type of appointment system. Of these 33 TAs: o 18 TAs (55%) completed their undergraduate degrees at institutions classified as Research Universities - Very High Research Activity, 0 5 TAS (15%) completed their undergraduate degrees at institutions classified as Research Universities — High Research Activity, 0 2 TAs (6%) completed their undergraduate degrees at institutions classified as Doctoral/ Research Universities, 0 2 TAs (6%) completed their undergraduate degrees at institutions classified as Master’s Colleges (both medium and large programs), 0 2 TAs (6%) completed their undergraduate degrees at institutions classified as Baccalaureate Colleges (Arts and Sciences), and 17 o 4 TAs (12%) completed their undergraduate degrees at international universities with no Carnegie Classification available (Carnegie Foundation 2005). Seventeen TAs in this study were male, sixteen were female. Twenty-three TAs were working toward their PhD, and 10 working toward their Masters degree. Six TAS taught Biol labs during two semesters of this study; one TA taught Biol for three semesters. TAs participating in the study self-report to have an average of 2.2 1 0.63 (mean 1 SE) semesters of teaching experience outside of their current university teaching appointment, teaching mainly outdoor education (31%) at local nature centers or parks, or K-12 (9%) classes. TAs taught for an average of 1.9 i 0.23 semesters in Biol. Only 7 (21%) TAS took a semester-long course on teaching, whereas 13 (39%) took at least one workshop /seminar on teaching, namely the university seminar required by TAs at this research university. 2.3.4 Survey Sampling To understand TA beliefs, I created surveys that gave me data on TA beliefs about teachers and teaching, and the learners and learning. 1 adapted my survey questions (Appendix B) from questions raised by Carroll (1977) and Richardson (1995, 1996). Together with a team of researchers, we refined the survey to achieve greater clarity, objectivity and validity. Specifically, we focused on vocabulary used in the questions, trying to make certain the words in the question prompts did not bias the TAS toward providing a certain answer. In the end, we created questions that were as open-ended as possible in order to prevent bias. For example, statements asked TAs to complete the sentence "Students learn science best by...". 18 These open-ended responses allowed TAs to answer the question in any manner they wish. Prior to the start of this research, I conducted a pilot study of the survey questions on 23 TAs not teaching Biol. Through this pilot study I sought to determine if the items elicited consistent responses, and if they provided data that enabled me to answer my research questions. Pilot study results informed me of how TAs interpreted the questions and whether the responses to the questions were actually relevant to my research questions. I clarified survey questions based on feedback from this TA sample (e.g., corrected typos, added examples in parentheses to clarify) and created final surveys (Appendix B). All but the preparation effectiveness survey were administered through an online course management system. A total of 31 (of a possible 33) TAs completed online surveys focusing on their beliefs about teaching and student learning prior to their first semester teaching Biol. Twenty-five of the 31 TAs completed a second survey, administered at the end of the semester, focusing on TA beliefs about the role of the teacher and student. Data from one TA was excluded from analysis because this individual missed greater than 60% of the preparation meetings, and failed to spend enough time-on-task in the preparation meetings to be included in the study. Therefore, the TAs was excluded from the study making the total sample size 30 TAs. Taken together, these survey data provide insight into TAs' beliefs about teachers and learners and the instructional process. 19 2.3.5 Coding and Data Analysis My research objective is to articulate what Biol TAs believe about teaching and student learning. To gain insight into these beliefs, I analyzed surveys completed by TAs using qualitative methods since the survey responses are free- response. First, I de-identified TA responses and copied them verbatim into separate Word documents for each survey question number. Instances where TAs did not provide an answer to a particular question (e.g., they did not know how students learned because they had never taught) were omitted from the data set. Following the method recommended for coding by Bogdan and Biklen (1998), I read through each response numerous times and noted patterns and common responses to the question. Similar methods for developing coding schemes are found in Raymond (1997) and B02 (2008). From patterns in TA responses, I developed coding schemes to categorize and quantify their responses (Appendix C) to the following questions: (1) What are the qualities of an effective teacher? (2) What is the role of the teacher? (3) What is the purpose of teaching science? (4) How do students learn science? (4) What is the role of the learner? By applying this coding scheme (Figure 2.1), I was able to attribute numerical values to the TA statements along a continuum for comparison. On one end of the continuum, there are beliefs that are more focused on the teacher, called teacher-centered. Examples of traditional beliefs include: teachers tell science to their students (i.e., transmit knowledge to students), and teachers give lectures. TAs may also have traditional views about students, for example, 20 students learning science through reading text books, and the role of the student in the classroom is to receive knowledge (see left end of Figure 2.1). Qualities of an Effective Teacher Personality Content Clear Creates Helps Considers Engages Traits Knowledge Communication Community Students Students Students '~¢"ri '31:, ,';‘-u.¢’.- I . I ". I, _ .. .. 4' I? .1" n .5. . i 'l . e -. ._ .-‘ ‘33-, v3. .- p p. .1?ng z. .1. . Hewitt'r .. g. '1. 1 2 3 4 5 6 7 Role of the Teacher Transmit Knowledge Support Student Learning “97- r" . . . l-- “t I , 7‘ ."T~ ,- I g _ - _- ,1 . ~ - xe-' ‘ ‘ , I LI. . ._ *:~"_-,- .. ' e‘ ”’43 . :Ill'i' "' ' : 1- _'-" '3 ‘I‘I' W"? V" .5. fl ‘ V " I ' ‘1 ’ A: '» ‘: J ' t """"" .— . ”7-4: Iii-{.1wigs"):(Irfi‘i’rtlfeit-He Halt- .fIraI- , 4' pf . {"1 _-. .‘ _._ alibi“ i 1} . ‘_ ._.‘ _ ,9, ... ‘_. u... .1. _'«__I _ .i,‘ 5 “4.1.. . 9 I .i _A s -:--. [Lt-g .._. "- ; . . ,._.. , .,, If,“ ._...1_, , ,1 :‘ ., Purpose of Teaching Transmit Knowledge Inspire Future Prepare Informed Facilitate Learning Scientists Citizens/Thinkers __ - .. qr-“ hug-cu In; -.1_- 1“ x, s‘n‘r.’ "I .5: fer" a: :gi_—'.'."'-I t i‘, -:,-.'"'1"»":" ;: 5e I» e e... :_.;;r_-’- _ ' _ _I- "flute: 1.11:3.” . ...Z-‘I __ ' .n "u' ‘l:‘.‘..‘§fi-T!?PQ.£IIV'9'T:L‘ .-: 1:254 ;‘;.-€.- .H n: :,=' .....:¢.-i-iir:‘.-‘~ ‘ _ .._ - . . o . , . . '. . '_ , _ . - - . .' ‘ ' ..g,‘.£,.e-,.\; -. ..- "’i '7 .. m n- - . .- » ----- $3.1“. r 1 2 3 How Students Learn Science Reading Lecture Multiple Making Active Learning Methods Connections or Hands-On .- , ._ , _. . '_ , ,x . _ , .3 T . . ,.' , ,- ' ’z'a . . .- _ .‘ - , . _._ ". . r553;- ', '. E-"a"""': . ._u'l-t T: - '_ , j - _ , u .. h, _._ _.‘ . . , , . ' < ~' «'- -\; ~ - *. v.'." 91"." . -' i ‘- -"4 »a' '- I. —‘ ' I ‘3‘, '. . .. _ .205 we! '3 -; 'i'. ' .' s ' = a.“ ;z . L’s x‘ 'r . _‘ _.. ‘ ‘.' /_ -. _ I IV' =. , _ . 1_g .3 v x. _ “f‘ .p I -, -- _ ,3 g' - s, " i :4- uigliJr.Itiliflrtr'IaIPI-O‘;i'.i§ ‘g.-.~".‘-fg,"_...J.‘_,}9!3.I.I..flif=!)jl ‘r "-;' 'la!'.."v ’ '. 1 '2... - c -- ":3. t. .- . .".’ .I'I ‘5. ' a. " - .: 2:1;- . . . g-9.. 1:. 3, 3:5; - .4 . - .‘ ' : . ug'. . ;-s‘_c_°,.:I§~.r :,::'__.'s ‘-‘_;:*.,'|.*_.','.';,vv_-':_. PEI-{Vi -.:. _ '3'w§?~;:'_:\-" _‘I . if. _' - 4' - ‘9. " ' "|“‘-' ' « nv- “2;?! _ ' ‘ .. =. .. .'._.‘. r " ' - - :1:_" a ' ' ‘ » 1 2 3 4 Role of the Learner Receive/Acquire Responsible Construct Knowledge Learner Understanding .r.. r.- ‘1 .1l.-_:-_-. _5.: ..r_.._ 1‘ 1- . _:‘ ..'..,.‘..(.,..,-3e._.‘....._.,."._. 3., . 'T’ ' ' ‘ ‘i - .. .. I ""5" '33” -‘- «Lit-:45: t . '.-- -‘-. .- '. .34 . _' » f A. eeee - . ‘ 3,5, ‘ r. , . _ I. l 2,'.'-'l'.- .h; : .r.. "u 1 . .I a'tititd'ztl:rillé 1 -' n -'.: -'_ '_'__ ‘-.‘ \. ‘ ”‘3“ c Figure 2.1. Coding categories for TA survey responses relating to beliefs. Categories emerged from TA responses and were appropriately placed on each continuum. Each TA response was coded using this scheme and assigned the numeric value associated with the coding category. 21 On the other end, there are beliefs that are focused not on what the teacher will do, but on the students. These learner-centered (i.e., non-traditional) beliefs include: teachers engage students in inquiry-based instruction, and teachers prepare students to be critical thinkers. From the student perspective, non- traditional beliefs about the students include: students learn science through hands- on and active means, and the role of the student in the classroom is to construct understanding (see right end of Figure 2.1). While my codes are represented as a continuum, a TA's placement along the continuum does not indicate an “either/or" response. Rather, the placement of the TA on the continuum represents an average of TA responses. An example of my coding scheme provides insight into how this average score is derived: TA; - An effective teacher is, ”knowledgeable, patient, able to explain things in multiple ways, interested/excited about the content.” Two raters, myself and another trained rater, calibrated on the use of the coding scheme (Appendix B), and obtained a Cohen's Kappa of 0.93 for inter-rater reliability. Each rater coded every statement in the TAs’ response to each question along each continuum (Table 2.1). Table 2.1. Coding scheme example for “The qualities of an effective teacher are....". Each statement was assigned to a coding category by each rater and then assigned a numeric code that was associated with each response. Scores for each statement were averaged across raters and then averaged to obtain the final score for the TA. Statement Rater 1: Coding Category Rater 2: Coding Category Average (Numeric Code) (Numeric Code) Knowledgeable Content Knowled:ge (2) Content Knowlige (2) 2 Patient Personality Trait (1) Personality Trait (1) 1 Explain in multiple Considers the Student (6) Passing on Information (3) 4.5 ways Interested/excited Personality Trait (1) Personality Trait (1) 1 Final Code 2.125 22 For each statement, I averaged the numerical codes across raters. For “explains in multiple ways" statement, rater 1 assigned it a 6, and rater 2 assigned it a 3. The average in this case of 6+3 is 4.5. The final rating for the TA is an average of the values of all the statements in the response. In this example, (2+1+4.5+1) / 4 = 2.125. This final coding score is the value used for analysis. If a particular TA provided answers to survey questions for more than one semester, their scores are calculated separately for each semester of instruction (e.g., Fall 08 and Spring 08). The result of the coding process yielded a score (e.g., 2.125) for each question to which the TA provided a response. I used these final scores to map the location of each TA on the corresponding continuum. Following the example in Table 2.1, TA1 scored 2.125. I plotted TAI's location on the continuum, which in this case ranged from 1-7. These visual maps allowed me to examine the patterns in TA beliefs. I created these spatial maps using R (R Development Core Team 2008). I created additional coding categories for the remaining questions on the surveys focusing on the supporting and demographic information TAs provided in their responses. For example, one question asked TAs to list variables that influence their teaching practice. Responses included “my experiences as a student", “good teachers that I had”, and “constraints on my time”. These responses were binned into categories independent of the continuum (Appendix B). 2.4 13251111: 2.4.1 What do BioI TAS believe about efl’ective teachers? TAS teaching Biol expressed beliefs that clustered more closely to traditional beliefs about what makes an effective teacher (Figure 2.2). These traditional beliefs 23 focus on a teacher having desirable personality traits (e.g., friendly, approachable, etc), being knowledgeable of the content, and being a clear communicator. For example, TAs 264 and 159 expressed the most traditional beliefs about characteristics of effective teachers (Figure 2.2). TA 159 wrote that effective teachers are "committed, self-motivated, respectful", all personality traits focused on the teacher. [x — ,9 TA 234 (D = (O — (D m g TA 191 :2 TA 145 '0 '0 - 9 IT TA 141 TA ‘95 C O 2 TA 259 TA 133 v —1 TA 194 TA 214 TA 213-F08 ..e _a_a TA 201 76 m -—+ TA 211 TA 286 In TA 157 TA 125 TA 184 g TA 219 TA ‘44 % N 4 TA 272-F08 9 TA 100 i.- TA 200 TA 205 TA 229-F08 TA 229-808 TA 112 TA 217 TA153 TA 254308 TA 22° ‘— — TA 159 TA 1150 TA 264 Figure 2.2. Spatial map of Biol TAs' beliefs about effective teachers. Each data point on the map represents 1 TA’s score on this question. Map is uni-dimensional, but spreads out along the x-axis only to accommodate the clustering of several TAS at a particular point on the continuum. Traditional beliefs include clearly communicating ideas in lecture, being prepared for class, and being knowledgeable. Non-traditional (i.e., learner-centered) beliefs include characteristics such as being a guide, or facilitator of student learning. 24 A few TAs expressed non-traditional beliefs, such as an effective teacher “enables students to construct their own learning", “lets students ask questions”, “engages students”, and “pays attention to students’ understanding.” For example, TA 234, averaged a score of 6.33 for beliefs about effective teachers, displaying the most non-traditional beliefs of all Biol TAs (Figure 2.2). Specifically, TA 234 believed, that an effective teacher “listens well to the needs of the students”, and "shows the students how to think, not just what to think." 2.4.2 What do Bi01 TAS believe about the role of the teacher? When Biol TAs responded to the question "what is the role of the teacher?” their responses placed the majority of TAS in the middle of the scale, transitioning between teacher and learner-centered beliefs (Figure 2.3). .52 g 3 c9 - TA 213-F08 TA 272-Sp09 a C O a 8 ,: TA 213-Sp09 I C 2 TA 211 TA 272-F08 TA 160 TA 206 TA 144 TA 153 N 1 TA 159 TA 200 TA 254-8009 TA 112 TA 133 TA 254-F08 TA 1454:08 5 TA 157 '0': (D a S g .— 9 TA 229-F08 TA 264 TA 286 E .— Figure 2.3. Spatial map of Biol TAs' beliefs about the role of the teacher. Each data point on the map represents 1 TA's score on this question. Map is uni-dimensional, but spreads out along the x-axis only to accommodate the clustering of several TAs at a particular point on the continuum. Most TAs believe the role of the teacher to be that of both a teacher-centered and learner-centered instructor. 25 For example, TA 160 falls right at this interface (Figure 2.3). TA 160 stated that the role of the teacher is to “be knowledgeable about the material, be enthusiastic about teaching the material be willing to listen to student input, and help students learn how to think....” Here, TA 160 expressed beliefs that a teacher needs to be knowledgeable and enthusiastic about passing that knowledge on to students (more traditional beliefs) yet at the same time, willing to listen to students and help them learn how to think (more non-traditional beliefs). Many other TAs had scores in the center of the continuum, indicating they hold transitional views of the role of the teacher. One response read: The teacher's role is to convey meaningful information for the student to increase their understanding of the subject. They should also be trying to facilitate meaningful discussions between the students. Here, this TA expresses a role that is traditional (i.e., transmit/ convey information to students) and one that is non-traditional (i.e., facilitate discussions between students). Many other Biol TAs exhibited similar responses. TAs who held strongly traditional beliefs wrote, “I think that it is my role to transmit a message to my students”, “to disseminate information in class” or “answer their questions.” This is in contrast to TAs, like TA 213, who expressed non-traditional beliefs about the role of the teacher (Figure 2.3). This TA saw themselves as a facilitator, “It is my role as the teacher to help students and help students learn from each other." Other TAs who expressed non-traditional beliefs indicated that the role of the teacher is to “ask the students questions that get them to think about the material”, or “guide them”. 26 2.4.3 What do Bi01 TAS believe is the purpose of teaching? When asked to explain the purpose of teaching, TAs again expressed transitional beliefs with most of the TAs clustering in the middle of the scale (Figure 2.4). These TAs expressed beliefs that part of their responsibility is to share knowledge with the next generation, while simultaneously developing thinkers capable of inquiry in science. a .2 g v — TA 211 a S :32 TA 159 TA 264 E l— g <9 ~ TA 254-F08 2 TA 153 TA 160 TA 213-F08 TA 254-Sp09 TA 272-F08 TA 125 TA 206 N a TA 145-F08 TA 272-Sp09 TA 286 .2 % TA 133 no TA 112 a S :15: 1- ~ TA 144 TA 157 TA 200 TA 213-Sp09 TA 229.1:03 ; Figure 2.4. Spatial map representing Biol TAs’ beliefs about the purpose of teaching. Each data point on the map represents 1 TA's score on this question. Map is uni- dimensional, but spreads out along the x-axis only to accommodate the clustering of several TAs at a particular point on the continuum. While most Biol TAs cluster around a transitional belief embracing both teacher and learner centered beliefs, some TAs express purely traditional beliefs about the purpose of teaching; that is, to transmit knowledge. For example, one TA wrote that the purpose of teaching is: To continue to pass information and understanding to future generations, and to help them develop into strong, functional citizens. 27 In this response we see a traditional belief of teaching as passing on information to future generations, but we also see a belief that teaching is to help develop citizens who are functional and can make decisions. There are some TAs who strongly believed the purpose of teaching is “to impart knowledge to the next n M generation , to introduce students to information and skills”, and “to increase the knowledge of students”, and these TAs cluster around 1 on the scale (Figure 2.4). On the other side, there are a couple TAs that cluster on the non-traditional end ”of the scale (e.g., TA 211, TA 213 and TA 264). These TAs believed that the purpose of teaching is to “facilitate student learning”, “help others learn” and "give students a way to think for themselves”. 2.4.4 What do Bi01 TAS believe about student Ieaming? When asked to think about the learner, Biol TAs expressed more non- traditional beliefs (i.e., Ieamer-centered) than traditional beliefs about how students learn science (Figure 2.5). For example, most TAs believed that students learn science best by hands-on interaction with the material, through experimentation and actually doing the process of science (Figure 2.5). However, there are a handful of TAs (clustered between 3-4 on the y-axis) who believed that students learn science best when multiple modes of learning are combined. For example, a student may learn science best when they read a textbook and conduct an experiment. 28 '0 - TA 153 TA 159 TA 191 TA 194 TA 211 TA 219 TA 220 TA 234TA 264 TA 272-F08 TA 286 TA 254808 TA 133 V T TA 141 gig 41433420280.“ 2...... A Non-Traditional Beliefs «9 - TA160 TA217 TA125 .3 TM? 7.2 :3 E. ._. TA206 ’— Figure 2.5. Spatial map of Biol TAs’ beliefs about how students learn science. Each data point on the map represents 1 TA's score on this question. Map is uni- dimensional, but spreads out along the x-axis only to accommodate the clustering of several TAs at a particular point on the continuum. Most TAS hold non-traditional beliefs about how students learn science, or hold beliefs that students need to learn science through multiple means. 2.4.5 What do Bi01 TAS believe about the role of the learner? Bi01 TAs reported beliefs about the role ofthe learner in the classroom that placed them on the middle of the scale (Figure 2.6). Beliefs tended to cluster around being a "responsible learner”, which Includes characteristics of student expectations, such as being prepared and attending class, and also include more active components, such as contributing to class and asking questions. An example of non-traditional beliefs of a TA include that students should "contribute to the learning of others”, “be active in their learning”, and "provide feedback, and try to relate the material to novel problems/ concepts”. On the other end of the continuum, TAS holding traditional beliefs about the role of the learner in the classroom report that students need to "pay attention , read assigned materials", and “speak to the teacher if confused”. 29 .2 .2 To _ - m c") TA 272 Sp09 '6 5 TA 16o TA 272-F08 :e 'U 8 TA 254-Sp09 '2 TA 206 TA 125 0 TA 145—F08 2 TA 133 TA 144 TA 153 TA 157 m 4 TA 112 TA 211 TA 213-F08 TA 254-F08 TA 159 .2 .2 ii an To 5 3% '- -— TA 200 TA 213-Sp09 TA 229-F08 TA 264 TA 286 E .— Figure 2.6. Spatial map of Biol TAs’ beliefs about the role of the learner. Each data point on the map represents 1 TA’s score on this question. Map is uni-dimensional, but spreads out along the x-axis only to accommodate the clustering of several TAs at a particular point on the continuum. TAs cluster around 2.0, and more TAs express non- traditional beliefs than traditional beliefs. 2.4.6 What variables influenced TA classroom practice? When Biol TAs reported on what variables influenced their classroom practices (Figure 2.7), TAs most often cite “past experiences” (code #2). That is, their classroom practices were most influenced by their own experiences in the classroom. For example, my teaching is strongly influenced by, “past excellent teachers I’ve had”, “paying attention to what teachers do that I like and dislike”, and “my own undergraduate experiences”. Overwhelmingly, TAs cite their years as a student as the most influential in driving their current teaching practices. Only three TAS mentioned that the student population or the course they are teaching (code #3) influences their classroom practice. 30 4o 30 1’ 25 ~ 20 1s -- 10» Number of Responses Coding Category Figure 2.7. Variables influencing Biol TAs’ classroom practices. Data are number of responses give to each of the following categories: (1) Personal Interest, (2) Past Experiences, (3) Time, (4) Students, (5) Courses / Literature, and (6) Epistemological Beliefs. 2.5 Discussion TAs represent an important, and understudied population of teachers, and this research presents one of the first insights into biology TAs’ beliefs about teaching and student learning. The results of this study reveal that a majority of Biol TAs hold beliefs about teaching and Ieaming that are inconsistent. These results reveal that Biol TAs simultaneously hold traditional and non- traditional beliefs about teaching and student learning (Figure 2.8). The small “X’s" represent two randomly selected TAs. The large “X" is the mean of all TAs (Figure 2.8). The two TAs are included to provide an indication of how representative the mean is for the entire Biol population. 31 Teacher-Centered (Traditional) Learner-Centered (Non-Traditional) Role of the Learner Figure 2.8. Summary of Biol TA beliefs across the five continua. Each continuum progresses from teacher-centered beliefs (i.e., traditional) on the left end, towards Ieamer- centered beliefs (i.e., non-traditional) on the right end. Three data points are represented on each continua: the black small “x” represents TA 159, the small grey “x” represents TA 133, and the larger "X" depicts the relative location of the majority of Biol TAs along the five continua. Looking more closely, we see TA 159 (small black “x”, Figure 2.8) expressed traditional beliefs about the characteristics of effective teaching, transitional beliefs about the role of the teacher and the purpose of teaching, non-traditional beliefs about how students learn and transitional beliefs about the role of the learner. Similar patterns exist for TA 133 (small grey “x", Figure 2.8). This TA expressed transitional beliefs about the characteristics of effective teaching and the role of the teacher, more traditional beliefs about the purpose of teaching, non-traditional 32 beliefs about how students learn, and transitional beliefs about the role of the learner. Considering these two randomly selected TAs (i.e., TA 159 and 133), they both express teacher-centered beliefs about being an effective teacher. That is, they believe that effective teachers have desirable personality traits, and are knowledgeable about the content. They are less polarized on the role of the teacher, expressing beliefs that they are to support student learning — support that occurs by both telling students information and by engaging students in a supportive learning environment. In contrast, they believe students learn best by engaging with the materials (i.e., hands on), but the role of the student is simply to do their work and show up in class (Figure 2.8). The patterns expressed in beliefs of these two TAs are similar to that of the whole group of TAs (large “X", Figure 2.8). Therefore, the mean is a fairly robust indicator of beliefs in this context. Taken as a whole, Biol TAs have more teacher- centered beliefs (i.e., traditional) about what it means to be a teacher, yet they hold more learner-centered beliefs (i.e., non-traditional) about student learning. This dichotomy offers some unique challenges to Biol TAs as they learn to teach. For example, if a Biol TA believes that students learn science best through active . learning, yet believes that the role of the teacher is to convey knowledge, which belief will direct their classroom practice? Will they lecture? Will they include active learning? Will their classroom practice embrace both? What explains the inconsistency of their beliefs? These inconsistencies may be a result a couple of things worth further exploration. First, TAs have spent nearly 33 their entire school career as a student, and are just beginning to think as a teacher. As a student, they recall what worked well for them to learn and may have assumed that what worked for their learning works for their students too. For most, what worked was taking notes in lecture and memorizing enough details to pass exams. However, as graduate students, nearly all TAs in this study reported they learn best by doing; that the experience of learning in their laboratories through trial and error has been the best learning experience. Thus, their beliefs about learning from a textbook or lecture may be in the process of changing to active, learner-centered approaches (as was reflected in their survey responses). Second, since TAs are just beginning to think as a teacher, they may still hold onto the belief that a teacher is the source of knowledge and has the responsibility of imparting that knowledge to students. This is the predominant belief based on what TAs experienced in their apprenticeship of observation (Lortie 1975). Therefore, it is possible that TAs may not yet have had the opportunity to reflect on the conflict of these two belief sets, or have yet to even discover they are in conflict since they have not done much classroom instruction. The inconsistencies found in TA beliefs are interesting, although not surprising because similar inconsistencies in beliefs were found in research studies focused on pre-service teachers. Seaman et al. (2005) replicated Collier's 1972 study of pre-service teacher beliefs finding that while pre-service teachers in 1996 held more learner-centered, active beliefs about effective teaching and learning than their predecessors in 1972, they continued to express inconsistencies in their beliefs (Seaman et al. 2005). Teachers in both studies believed that a mathematics teacher 34 should work through problems for their students before assigning work to them. Simultaneously, these same teachers believed that students need mathematics problems to solve that challenge their thinking and creativity (Seaman et al. 2005). In addition, these pre-service teachers held traditional beliefs of mathematics (e.g., mathematics is a collection of rules/ processes) while simultaneously believing mathematics is flexible and creative. In 2007, Cady and Rearden investigated pre-service teacher's beliefs about mathematics and science. In mathematics, these teachers described their beliefs about the role of the student as passive but believed the teacher needed to instruct students by assigning practical application problems. Another study (B02 2008) focused on pre-service mathematics teachers found a dichotomy similar to the results presented in this paper. Some pre-service teachers expressed both non-traditional and traditional beliefs about effective instruction and the role of the teacher. Thus, it is not surprising that TAs, too, hold inconsistent beliefs about the role of the student and role of the teacher in the classroom. The inconsistencies in Biol TA beliefs are worth further consideration and investigation. Beliefs are a ' challenging construct to investigate because they are a latent variable, meaning, researchers cannot directly measure or sample ones’ beliefs. While there are diverse methods researchers employ to investigate beliefs (e.g., surveys, narratives, stimulated recall, think-aloud session, etc.), each method has its own strengths and limitation. 35 In the context of this research, I used surveys to explore the latent variable “beliefs". I used two different surveys (see Appendix B), so that I could look for patterns in TA responses across multiple responses to questions framed in different ways. This way, I could look for evidence of their beliefs from different perspectives, and when patterns emerged, I was able to see if these same patterns appeared in other similar questions. The format of my surveys was designed to ask TAs direct questions to which they would be able to respond rather than ask them to define or articulate their beliefs. For example, I did not ask, “what do you believe about your role as a teacher?" Instead, I asked, “what is the role of a teacher?” By choosing to ask survey questions in this fashion, I made inferences from their responses that what they wrote was a reflection of their beliefs. In this context, TAs provided answers to my questions which I interpreted as illuminating their beliefs about teaching and student learning. Therefore, I am interested in using additional, diverse methodologies to investigate Biol TA beliefs as we strive to better understand why Biol TAs have inconsistencies in their beliefs about effective teaching and student learning. 2-6 W This research makes a unique contribution by providing the first data on science TA beliefs. These results indicate that Biol TAs hold dichotomous beliefs about the teacher and the learner. This insight is important to consider as TAs learn to teach. Since TAs teach a growing percentage of undergraduates, especially in the 36 STEM disciples, at universities across the country, TA beliefs must be considered when designing and implementing TA professional development. This research informs professional developers of the beliefs of Biol TAs. Through exploring TA beliefs in this study, it is clear that TAs arrive at their teaching opportunities with well-formed but inconsistent (even conflicting) beliefs about teaching and Ieaming. These results provide us with valuable data to consider as we seek to improve TA professional development. First, this research highlights that it is important for professional developers to recognize that TAs are not blank slates when it comes to beliefs (Kettle and Sellars 1996). TAs will arrive in their professional development opportunities with prior notions of what it means to be an effective teacher, and how they expect students to learn. These beliefs are often derived from their years of being a student, critically observing what things their teachers did that they liked and what pedagogies they deemed ineffective (Lortie 1975). Second, it is also important for professional developers to understand TA beliefs because incoming beliefs have been shown to influence the receptiveness of a teacher to professional development materials (Holt-Reynolds 1992, Bramald et al. 1995, Calderhead 1996, Fajet et al. 2005). A teacher’s beliefs about effective instruction provide a lens through which they evaluate all content and professional development experiences, and as such, may only be able to adopt new practices that are in harmony with their pre-existing beliefs. Thus, incoming beliefs must be considered when designing professional development, just as student pre- conceptions are considered when designing a course. 37 Third, after considering pre-existing TA beliefs in the design of professional development, professional development experiences need to include opportunities for TAs to articulate and reflect upon their beliefs. In the hectic pace of the professional development with the focus on content, beliefs often get overlooked. However, there is research that suggests a strong link between what a teacher believes and their classroom practice (e.g., Pajares 1992, Speer 2008). Therefore, it seems important to provide opportunities for TAs to confront their beliefs and reflect on them. Research on teacher beliefs has shown that beliefs can change, and opportunities to confront existing beliefs (including inconsistencies among beliefs) can facilitate change in teacher beliefs (Hollingsworth 1989, Richardson and Hamilton 1994). However, beliefs are individualized and built by memorable events in one's own experience and changing beliefs takes time (Collier 1972, Shirk 1972). Even though measureable change may not take place during the course of short- term professional development, designing professional development with opportunities to explore one’s beliefs about teaching and learning is a start. In this research, 7 TAs took belief surveys over multiple semesters. While changes are challenging to detect, survey results indicate a small degree of change in their beliefs. For example, TAs 254 and 272 show a trend towards more non- traditional beliefs about the role of the student in the learning process (i.e., students are active participants in their learning) during their second semester teaching Biol than during their first (Figure 2.6). On the other hand, TA 213 shows the opposite 38‘ trend, becoming more teacher-centered in beliefs during this TAs’ second semester teaching Biol. There is also great potential within a TA professional development opportunity to provide experiences that will positively shape TA beliefs along the lines of how research shows students learn best. For example, TAs cite “personal experiences” as the number one influence on their teaching. To the TA, professional development likely qualifies as a personal experience. Thus, there is the opportunity to model during the professional development the kind of instruction that embodies learner-centered pedagogies. TAs may have never experienced a learner-centered classroom during their “apprenticeship of observation" (Lortie 1975). Allowing them to participate in one during their professional development could positively influence their beliefs about effective teaching and learning. The realization that TAS may arrive at their first teaching opportunity with inconsistent beliefs about teaching and student Ieaming provides professional developers with an opportunity to design meaningful professional development experiences where TAs explore these conflicting beliefs, and experience learner- centered instruction. Thus, the design of professional development may have potential to positively influence TA beliefs towards a higher degree of consistency. 39 CHAPTER 3 EVALUATING REFORMED TA PROFESSIONAL DEVEOPMENT 3-1 16mm Each academic year, graduate teaching assistants (TAS) teach a significant number of undergraduates in universities, especially in STEM disciplines (Fink 1977, Lawrence et al. 1992, Rushin et al. 1997). In contrast to K-12 teachers, TAs have little to no formal preparation to teach undergraduate biology courses (Druger 1997, Lumsden 1993, Rushin et al. 1997). Elementary and secondary school instructors receive coursework training, complete student teaching, and pass licensing exams to become qualified for teaching positions (Ballou and Podgursky 1999). Often, during their first few years of teaching, K-12 teachers have mentors and a team of teachers with whom they meet regularly to discuss teaching pedagogy and other issues of classroom practice. TAs, however, are regarded as qualified to teach solely because they are in a graduate program in a particular field of study (Lumsden 1993). Arguably, TAs likely have disciplinary knowledge, but, research indicates that a teacher needs more than content knowledge to teach effectively (Ball and Bass 2000, Whitcomb 2003, Wilson et al. 2002). With the increased understanding that TAs need better preparation to teach, new professional development opportunities were developed, implemented and evaluated at universities across the country (for examples, see Marincovich et al. 1998). While these professional development models often communicated the latest pedagogies (Baumgartner 2007, Caris and Merchant 1991, Etkina 1999, Hammrich 2001, Hiiemae et al. 1991, Marincovich et al. 1998, McManus 2002), the 40 Ieaming experiences were typically passive lectures, with limited interaction, practice (Baumgartner 2007) and feedback. While there has been a push in K-12 teacher preparation to clearly define and evaluate “effective” professional development (Wayne et al. 2008, Desimone 2009), no standards exist for TA professional development. Most often, conclusions of effectiveness are based on self-report information, typically an end-of-workshop survey (Black and Kaplan 1998, Chism 1998, Guskey 2002, Miller et al. 2008, Pfund et al. 2009, Winternitz and Davis 2000). These self-report data indicate that our current modes of TA professional development are not effectively preparing graduate TAs to teach (Worthen, 1992, Luft et al. 2004, Baviskar and Beardsley 2006). In order to better understand why the current professional development models are not meeting TAs needs, additional research is needed investigating what is occurring in TA professional development, and the application of their professional development in their teaching practices. Significant improvements to understanding this process may be possible by considering other sources of evaluation data in addition to self-report survey data. In this study, I use Kirkpatrick’s model (Kirkpatrick 1994) to evaluate the effectiveness of TA professional development using both survey and observational data. Taken together, these data provide the first insights on biology TA professional development and the application of training in TA classroom practices. 3.2 MW While there are many evaluation plans to choose from to evaluate professional development (see Boulmetris and Dutvvin 2000), I chose Kirkpatrick’s 41 Framework for Evaluation (Figure 3.1) because of its four-tiered approach drawing from multiple data sources. Indicator of Effectiveness 1.10113 PUB PEOPIJOM Figure 3.1. Kirkpatrick’s evaluation hierarchy. One commonly used framework for evaluating the effectiveness of professional development is the Kirkpatrick Evaluation Framework. Moving up the hierarchy achieves better estimates of effectiveness, but also increases the required work and effort to obtain these results. While Kirkpatrick’s framework does have critics (Holton 1996, Bates 2004), it is the most widely applied framework for evaluating professional development, especially in the business sector, to date. In addition, Kirkpatrick's model has also been applied in assessing professional development in higher education (e.g., Thackwray 1997, Steinert et al. 2006). The model is hierarchical, where each level (e.g., reactions, learning, behaviors and results) builds on the one before (Kirkpatrick 1976, 1994). Evaluating at higher levels of the hierarchy often requires greater investment of resources (e.g., time and money), but the outcome is a better understanding of the effectiveness of the training. 42 Briefly, Kirkpatrick (1994) argues for evaluating the effectiveness of training programs by considering multiple levels of evidence. First, it is important to consider feedback from the participants about their reactions to professional development. Typically, this is the only level considered when evaluating professional development. A second measure of effectiveness focuses on learning, that is, how did the professional development influence the Ieaming of the participants? Third, what does the participant do with the knowledge they have acquired? This level of evaluation is much more time-intensive, as it requires an in- depth look at what the participant applied from their training in their teaching practices. For example, if professional development focuses on teaching TAs pedagogies for engaging students in active learning, do TAs implement these strategies in their classroom? Finally, in Kirkpatrick’s model, the success of a professional development model is judged on the basis of its impact on the bottom line. In the business application of this model, this is a financial bottom line - how much money was saved or made as a result of the changes? For us, the bottom line is whether or not student Ieaming improves as a result of changed TA instructional practices. This level has many potential considerations and variables that make it difficult to assess in practice. In this study, for example, Introductory Biology (Bio 1) students are simultaneously enrolled in multiple courses, all of which contribute to their ability to critically think, reason, problem solve and acquire knowledge. This research evaluates the reformed professional development model described in detail in Appendix A. 43 3-3 Wm In order to evaluate the effectiveness of the new model of TA professional development, I compare data between the traditional and reformed approaches for preparing Biology 1 (Biol) TAs to teach biology laboratories. These two models of professional development are fundamentally distinct from one another in eight ways (Table 3.1). Table 3.1. Points of comparison between professional development models (Part 1). Reformed professional development is grounded in constructivism and cooperative Ieaming and the components of the model are derived from these two Ieaming theories. Point of Comparison Traditional Professional Reformed Professional Development Development Underlying Behaviorism Constructivism, cooperative Theory (transmission model) learning Motivation Program needs TA needs Design Lecture-based, answer-driven, Collaborative, inquiry-driven, Characteristics Goals of Preparation TA Role TA Ownership Observation of desired teaching practices Formal Reflection protocol-oriented Content-focused Passive, listener None None End—of-semester surveys process—oriented Student-learning focused Active, participant Partner Moderated fishbowl Iterative, discussions First, this new model is based on constructivism and cooperative learning theories; two theories of learning that have support in the research literature for how students learn (Bransford et al. 2000). Second, the model is motivated by TA needs - it considers their incoming beliefs and their experiences in the design of professional development rather than solely focusing on departmental needs. 44 Third, this new professional development models the expected active-learning, student-centered pedagogies that TAs are to apply in their classroom practices. Fourth, the goals of this reformed professional development focus on the TAs as learners and instructors, rather than solely on the content. Fifth, this new professional development has TAs as active participants in their professional development. This means that TAs are talking, doing labs, interacting and experiencing rather than listening to a lecture. Sixth, TAs partner with the designers of the course to have ownership. In this case, TAs redesigned learning objectives and assessments allowing them to have some ownership within the course. Seventh, TAs have an opportunity to observe what teaching will look like through the moderated fishbowl, and have a chance to experience it when they themselves engage in active Ieaming throughout the professional development meeting. Finally, TAs actively reflected on their Ieaming throughout the semester. For the first 15 minutes of lab, TAs discussed their learning from the perspective of the teacher and the student. This contrasts with the traditional model where reflection happened at the end of the semester. These contrasting approaches to preparing TAs to teach Biol offer an opportunity to compare TAs reactions to, Ieaming about, and application of their professional development. 3.4 Remmhflesignandflauiflnanm 3.4.1 Study Design This study focuses on an introductory-level biology course (Biol) covering the content domains of genetics, ecology and evolution. Students concurrently 4S enroll in lecture and laboratory. Over the course of a year, Biol teaches biology to over 2,000 students. TAs teach the laboratory portion of the course to sections of 25-32 students. Weekly, TAs attend a 3-hour professional development meeting to prepare them to teach the following week. In both the traditional and reformed professional development models, TAs also participate in an orientation meeting prior to the start of the semester, and have a concluding meeting during finals week to turn in grades and provide feedback about the semester. For traditional professional development, the pre-semester orientation consisted of a lab safety review, an overview of job responsibilities, a chance to get to meet the other TAs, and an overview of the Biol course. In reformed professional development, the logistics were still covered, but in addition, they received training in pedagogies on topics including active learning, assessment, cooperative groups (Johnson et al. 2000, Smith et al. 2005), Backward Design (Wiggins 8: McTighe 1998) and Bloom’s Taxonomy (Bloom 8: Krathwohl 1956) (see Appendix A). In the spring and summer of 2008, I collected data on the traditional (1‘) model of professional development in Biol. Beginning in the fall of 2008, I collected data as Biol professional development began the reformation process. In my study, Fall 2008 is considered “reformed”; however, it is important to note that this semester was highly transitional. Fall 2008 involved TAs re-designing learning objectives, assessments and corresponding rubrics, as well as reforming laboratory exercises in light of their professional development training. Thus, it is distinct from the second semester of reformed professional development operating in Spring 46 2009. I consider these as two different semesters of reform: reform 1 (R1, Fall 2008) and reform 2 (R2, Spring 2009). These semesters will, unless otherwise noted, remain separate for the purpose of data analysis. 3.4.2 Study Participants In total, 33 TAs (of 36 total) volunteered to participate in this research over the course of the three semesters of study. Of these TAs, 5 taught across multiple semesters, and agreed to participate in the study in all their semesters in Biol. The majority of TAs in this study (70%) have are working toward their PhD in either Plant Biology, Zoology, and Fisheries and Wildlife. Other departments providing TAs include anthropology and forensic science. These 33 TAs are considered to be a representative sample of life sciences TAs available to teach Biol at this institution. 3.5 Methgds Using Kirkpatrick’s model for evaluation (Kirkpatrick 1994), I investigated the differences among TA professional development at three levels. 3.5.1 Level] - TA Reactions To assess TA reactions to professional development, TAs in Biol completed a "Final Survey” evaluating the professional development they received at the end of the semester (Appendix B). I derived survey questions from prior surveys administered to Biol TAs, and modeled them after literature from marketing on how to effectively capture participant reactions (Marder 1997). For each item on the Final Survey, TAs placed an “x” along a continuum for each question, indicating their agreement or disagreement with the statement. For example, “The weekly lab meetings helped prepare me to teach next week’s laboratory exercise.” I quantified 47 the location of each “x” TAs placed on the continuum for each question. The numeric values corresponding to their response ranged from 0.0 (100% agreement) to 13.0 (0% agreement). TAs also explained their response. These data accompany the quantitative measures for each question and provide an understanding of why a TA chose to place their “x” in a particular location. Prior to administering the survey to the Biol TAs, I pilot tested the survey with another group of TAs teaching introductory biology who were not involved in my study (n=2 3). After reviewing their responses, I made revisions in word choice to questions before solidifying the final version. At the conclusion of each semester, TAs completed this evaluation survey. I informed the TAs that the Biol staff would not see their individualized responses. I have a total of 35 surveys post-professional development across traditional and reformed professional development: T (n=11), R1 (n=13), and R2 (n=ll). For each question, I averaged all the TA responses within a semester to yield an overall agreement measure. I was particularly interested in three items: (1) Friday prep meetings prepared me to teach next weeks’ lab exercise, (2) Friday prep meetings increased my confidence as a teacher, and (3) Friday prep meetings improved my teaching skills. Additionally, I read through the qualitative "responses to these same questions and generated a list of common responses TA provided. Using R, l evaluated each question’s differences in quantitative reactions between the two models of professional development (across all three semesters; i.e., T, R1, R2) using a KruskaI-Wallis Test (R Core Development Team 2008). I 48 investigated significant differences among these three semesters using pair-wise Mann-Whitney Tests (i.e., Wilcox Rank-Sum). 3.5.2 Level 2: TA Learning I chose to evaluate TA learning by evaluating classroom materials TAs created in response to professional development about assessment. I collected the classroom materials produced by the TA for his / her lab section students after receiving either traditional or reformed professional development. These materials include assessments (in-class, homework and exams) and learning objectives. To assess the materials TAs produced as part of their classroom practice, I collected assessment items designed by TAs to determine: (1) the level of cognitive processing TAs are asking their students to achieve (i.e., learning objectives), (2) the types of assessments TAs are using to understand their students’ learning, (3) the levels of cognitive processing at which TAs assess their student learning, and (4) the alignment between the cognitive level of learning objectives and assessments. 3.5.2.1 Learning Objectives Traditionally, TAs taught Biol from a laboratory book with pre-established objectives that TAs did not contribute to developing. I obtained these objectives from the laboratory manual and transcribed them into one document. In total, there were 68 Ieaming objectives (from 11 labs) articulated by Biol staff. During reformed professional development, TAs had co-ownership of the lab design and collectively they articulated a set of Ieaming objectives for what their students would know and be able to do upon completing the lab activity. In total, TAs defined 41 Ieaming objectives (from 9 labs) for students in Biol. 49 3.5.2.2 Assessment of Student Learning During traditional professional development, Biol TAs had 180 points to allocate to diverse assessments of their choosing (e.g., quizzes, homework assignments, projects, etc.), and 100 points dedicated to a mid-term and final exam worth 60 and 40 points respectively. In total, I collected 78 assessments containing 922 questions items from 10 TAs teaching during the traditional professional development model. I categorized the 78 assessments into assessment types: quiz, mid-term exam, final exam, in-class assignment and homework. During reformed professional development, TAs received preparation about assessment development, and collaborated to design assessments to evaluate the achievement of students on their articulated learning objectives. TAs converged on common pre-lab (n=8/semester) and post-lab assessments (n=8/semester) through which students could demonstrate their understanding in relation to learning objectives. The same objectives and assessments were used in reformed 1 (R1) and reformed 2 (R2); thus, I collapse the two reformed categories in this case. I collected all the pre- and post-labs created and used by the TAs during the two reform semesters and randomly sampled 1 pre-lab and 2 post-labs. 3. 5.2.3 Assessing Cognitive Processes To determine the cognitive processing required by each objective and assessment question (here after referred to as an item), I applied Bloom’s Taxonomy for Educational Objectives (Bloom and Krathwohl 1956)to each item and objective. Bloom’s Taxonomy was developed in the 19505 as a means for test developers to communicate in the same language when designing tests with items of 50 varying degrees of cognitive complexity. Bloom’s Taxonomy focuses on cognitive processing skills and uses key phrases typically present in test items to categorize them into varying levels of cognitive processing (Table 3.2). Table 3.2. Bloom’s taxonomy of the cognitive domain. Description of the six levels of cognitive processes in Bloom’s Hierarchy (Bloom and Krathwol 1956). Bloom Cognitive Level Process Description 1 Knowledge Recall facts 2 Comprehension Explain phenomena 3 Application Apply concepts to a novel situation 4 Analysis Interpret and analyze data 5 Synthesis Combine multiple data sources to generate conclusions 6 Evaluation Assess theories or evidence and make recommendations Two trained independent raters with established inter-rater consistency (lntra-Class Correlation, 0.782) rated the level of cognitive processing of each objective or assessment item using Bloom’s Taxonomy. Each rater was presented with items that were de-identified from semester of professional development and thus rated items blind to the treatment. Each rater rated (1) all the objectives and assessment items collected from traditional professional development, (2) all of the objectives and the sample subset of assessments from reformed professional development. The rating for each objective and assessment item represents the mean of the two raters’ scores. I compared Bloom’s levels (on items and objectives) within the traditional model of professional development (i.e., across assessment types), and between the traditional and reformed models using a non-parametric ANOVA (i.e., Kruskal-Wallis). I investigated significant differences in pair-wise comparisons 51 using Mann-Whitney Test (i.e., Wilcox Test) with a continuity correction using R (R Core Development Team 2008). 3.5.3 Level 3: TA Application To determine the application of professional development to a TA’s teaching, I examined TA classroom practice to see if TA instructional patterns changed in response to professional development. I hypothesized that reformed TA professional development would yield more reformed TA classroom practices. Thus, I investigated TA classroom practice in their laboratory sections to provide insight into this third level of evaluation. To assess TA classroom practice, I videotaped TAs twice during each semester of participation. The labs videotaped were the same for all TAs within a semester, but varied across traditional and reformed professional development. Labs were selected based on two criteria: (1) the time during the semester (e.g., early vs. late), and (2) the approach to the laboratory lesson (e.g., prescriptive vs. inquiry). During traditional professional development, I taped a lab early in the semester (tape I) focused on predator-prey population dynamics. This lab followed a strict protocol, where students collected data in a simulated predator-prey habitat. The second tape during traditional professional development was a more inquiry driven lab focused on using gel electrophoresis data to construct a phylogentic tree of relatedness. This second lab was later in the semester and contrasted with the approach of the first tape. During the summer session (traditional), scheduling changed, and for the second tape I videotaped an inquiry-focused lab on pollination biology. 52 For the reformed professional development semesters (R1 and R2), the first videotape was of the cellular reproduction lab; an inquiry-focused lab on the cell cycle. The second tape was a more prescriptive lab on animal diversity that took place towards the end of the semester. This lab asked students to compare four diverse marine animals and draw conclusions. 3.5.3.1 Reformed Teaching Observation Protocol While there are some tools available to help with observations of teaching practice, I chose the Reformed Teaching Observation Protocol (RTOP) specifically because this instrument provides a valid and reliable (Sawada 1999) way to quantify a science teacher’s classroom practice based on the degree of active, inquiry-based pedagogy (i.e., reformed practices) used in their classroom (Lawson 2002, Maclsaac and Falconer 2002). The RTOP composite scores ranges from 0-100, and is based on five subscales: (1) instructional design and implementation, (2) content knowledge, (3) procedural content knowledge, (4) communicative interactions and (5) student- teacher relationships. Each sub-scale and thus the composite score is designed to measure the degree to which this aspect of a teachers’ classroom practice is Ieamer- centered. The sub-categories in RTOP award more points to instructors using student-centered (i.e., reformed) pedagogies. For example, the greater percentage of time that students engage in interactive, inquiry-based explorations, the higher the score. The goal of the RTOP instrument is not that each teacher achieve a 100 on the scale, but rather to see progression towards the learner-centered end of the continuum by tracking changes in teachers’ instructional practice over time. 53 To assess the degree of learner-centered instructional practice taking place in TAs’ laboratories, l assigned each videotape a random identification number so that the raters were blind to the semester or type of TA training. Each videotape was rated by two trained raters with established inter-rater consistency (ICC = 0.70) using the RTOP to describe instructional design and implementation, content knowledge, communication and student/teacher relationships observed in TA classroom practice (Sawada et al. 2000). Over the course of seven months, each rater rated 57 video tapes of TA classroom practice. 3.5.3.2 Statistical Analyses Since TAs in Biol are from similar disciplines and at similar stages in their graduate careers (i.e., Biol employs a large number of first year graduate students), I predicted that possible confounding variables will have little impact on the outcome variable, teaching practice, as evaluated through RTOP. To verify this, I compared demographic data on traditionally prepared TAs to TAs prepared using reformed professional development using a Chi-Square test for independence for categorical data (e.g., gender), and ANOVA for continuous data (e.g., number of semesters of teaching experience). Variables included: semesters of prior teaching experience at this institution, semesters of teaching experience outside of this institution, semesters of teaching experience in Biol, undergraduate institution, undergraduate major, degree (MS/ PhD), degree-granting department, gender, year in program, course on teaching, and seminar on teaching. Each of these factors was considered fixed for this analysis. Prior to using these statistical tests, I verified 54 assumptions (e.g., homogeneity of variance, normality of residuals, points of high leverage, outliers) and found that the assumptions were not violated. The result of this demographic comparison showed two variables to be marginally significant: (1) gender (x2=13.l6, df=2, p=0.001) and (2) semesters of teaching experiences outside of the current institution (AN OVA, F 2,54=4.17, p=0.0206). While I had approximately equal representation of males and females in the study as a whole (n=17 and 16 respectively), the five TAs that appeared in the study over multiple semesters were male, and therefore when considering all data points, the distribution of gender across treatments differs. To determine whether gender or semesters of teaching experience outside of this institution had a significant influence on the model, I included both these variables in my linear model. When I ran the model with the covariates, neither variable was a significant predictor of a TAs’ RTOP rating (p<0.05). Therefore, I proceeded in analysis without including these covariates in the model. I used an ANOVA to investigate differences in RTOP ratings by both semester (i.e., traditional, reformed l, or reformed 2) and tape number (i.e., tape 1 vs. tape 2). The AN OVA model included testing for interactions. I used a post-hoc Tukey’s HSD test to identify which semesters and tape numbers differed from one another. To assess the five subscales, I again used an ANOVA when I found no violations in assumptions. I assessed post-hoc pairwise comparisons using Tukey’s HSD test. Finally, I created graphical representations of repeating TAs’ RTOP ratings to visually observe any patterns in teaching practices. I could not perform statistical analyses on these repeating TAs because of the small sample size (n=5) and thus 55 low power to detect differences. I ran all statistical analyses using R (R Core Development Team 2008). 3.6 Realms 3.6.1 Level 1: TA Reactions TAs were asked whether professional development (1) prepared them to teach, (2) increased their confidence, and (3) improved their teaching skills. TAs in the second semester of reformed professional development indicated a significant level of agreement with the statement (p<0.05) when compared to the traditional and first semester of reformed professional development (Figure 3.2). — ITraditional IReformed(1) 21Reformed(2) b Disagree H H H H OHNUJAU'IOTNNSDOHNW Agree—m Preparation Confidence Teaching Skills Figure 3.2. TA reactions to professional development models. Specific TA responses about how well they thought their professional development (1) prepared them to teach, (2) increased their confidence, and (3) improved their teaching skills during three contrasting semesters of professional development. Letters represent significant differences at p<0.05. Specifically, TAs found that they felt more prepared and had increased confidence in their ability to teach as a result of reformed professional development 56 (R2). Additionally, they also reported that reformed professional development (R2) increased their teaching skill set. However, TAs in reformed semester 1 (R1) showed more disagreement with these three statements than those in the traditional professional development and the second reformed semester of professional development (see Figure 3.2). 3.6.2 Level 2: TA Learning 3. 6.2. 1 Learning Objectives Learning objectives for students in traditional Bi01 laboratories were pre- determined by the Biol staff. Each of the 11 lab exercises had between 3 and 13 stated Ieaming objectives. The mean Bloom’s level for these objectives was 2.13 1 0.04 (mean 1 SE, for all values reported) (Figure 3.3). Most objectives targeted knowledge or comprehension levels in Bloom’s Taxonomy. Average Bloom's Level Lab 1 Lab 11 Figure 3.3. Bloom’s level for traditional laboratory student Ieaming objectives. Bars represent the average of the Ieaming objectives for each lab. The black line represents the mean (2.13) Bloom level across the entire 11-lab sequence. 57 Learning objectives for students in reformed Biol (R1 and R2 combined) exhibit an average Bloom's level of 3.65 1 0.19 (Figure 3.4); a significantly higher cognitive processing level than the learning objectives articulated for students during the traditional model (Wilcox Rank Sum Test, W=463.5, p<0.001). 6 . Average Bloom's Level w Labl Lab9 Figure 3.4. Bloom's level for reformed laboratory student learning objectives. Bars represent the average of the learning objectives for each lab. The solid black line represents the mean (3.65) Bloom level across the entire 9-lab sequence, and the dotted-black line is the mean (2.13) of traditional laboratory objectives. The mean of reformed laboratory objectives is significantly higher than that of traditional (dashed) objectives (Wilcox Rank-Sum Test, p<0.001). 3. 6.2.2 Assessment of student learning Across both types of professional development, TAs reported assessing students with formal and informal means. From a formative perspective, TAs noted in their surveys that they pay attention to students' "body language", "facial expressions” and "questions raised during or after lab" as a way to be mindful of 58 how students are processing course content. TAs also mentioned using grades or achievement on assessments as a more formal, summative, means of assessing student learning. In addition to these typical assessment methods, one TA under traditional professional development mentioned that he evaluated student learning based on response length stating, “short answers mean blatant memorization; longer answers mean they learned, and expand in their minds." For summative assessments, TAS in traditional professional development assessed their students through in-class assignments, homework, and quizzes in addition to the mandatory mid-term and final exam, with the majority of assessment items categorized as exams (n=493) and quizzes (n=287) (Figure 3.5). 2% Quizzes (31%) Mid-Term Exam (31%) iii Final Exam (23%) In-Class Assignments (13%) Homework (2%) Figure 3.5. Distribution of TA-designed assessments in traditional professional development. The majority of assessments are in-class (e.g., quizzes, mid-term and final exam), only a small proportion are out-of-class. During traditional professional development, TA-designed assessments primarily asked students to demonstrate knowledge and comprehension, with a few application questions found on homework and in-class assignments (Figure 3.6). 59 5 .. 4 4 i 5 —| VI E 3 o o E 2 .. 1 - l l I 0 ‘ “T“ """‘"i"""' 1.2-72.2. "'"I'wu‘ """T’T'V" _ """" I FinalExam Mid-term Quiz Class Homework Assignment Figure 3.6. Bloom’s level for TA-designed assessments in traditional professional development. TAs assess student understanding of Biol concepts through low-level cognitive processes (Bloom’s levels 1 & 2). Average Bloom's level for TA-designed assessments is 1.51 1 0.02. On average, TAs targeted low—levels (1.51 1 0.02) of Bloom's Taxonomy across assessment types. The small standard error around each assessment type (see Figure 3.6) is related to the small amount of variation in the levels of cognitive processing tasks TAs ask their students to demonstrate on assessments. Traditionally prepared TAs asked students to apply slightly more complex cognitive processing (e.g., level 2) on in-class and homework assignments (Table 3.3). Although statistically significant, it is arguable whether they are meaningful in the context of classroom assessment since they are still assessing low-level cognitive processes (see Zheng et al. 2008). 60 Table 3.3. Pairwise comparisons of Bloom's ratings for TA-designed assessments. Comparisons were made using Wilcox Test with continuity correction of small sample sizes. * are significant at a=0.05 Post-Hoc Comparison p-value ln-Class Assignments > Final Exam* <0.0001 ln-Class Assignments > Mid-Term Exam* <0.0001 In-Class Assignments > Quizzes“ <0.0001 ln-Class Assignments and Homework 0.78 Homework > Final Exam* <0.0001 Homework > Mid-Term Exam* <0.0001 Homework > Quizzes“ <0.0001 Quizzes > Final Exam* <0.001 Mid-Term Exam > Quiz* <0.0001 Mid-Term and Final Exam 0.48 TAs prepared to teach in reformed professional development (R1 and R2), assess their student learning primarily through a pre-lab (n=8/semester) and post- lab (n=8/semester). Collectively, the mean Bloom level for TA-designed assessments under reformed professional development (n=25 items) is 4.1 1 0.21 compared to TA-designed assessments under traditional professional development, which averaged 1.51 1 0.02 (Wilcox Test, p<0.001). 3.6.2.3 Alignment of Objectives and Assessments Bloom’s Taxonomy was initially created for the purpose of determining alignment (in terms of cognitive complexity) between what students are expected to know and be able to do (objectives) and the assessment of their ability to achieve these objectives (Bloom and Krathwohl 1956). Following reformed professional development, I predicted that TA-designed assessments will be more closely aligned to objectives than those produced during traditional professional development. 61 For traditionally prepared TAS, there was no alignment between their stated laboratory objectives (2.13 1 0.04) and their assessment of student Ieaming (1.51 1 0.02) (Wilcox Test, p<0.001). The significant results indicate TAs assessed their students at a lower-level (i.e., level 1) than they were expected to be able to perform at the end of the semester (i.e., level 2). Although there was a statistical difference between the Bloom’s level of the objectives and that of the assessments, both level 1 and level 2 are low-level cognitive processes (see Zheng et al. 2008). During reformed professional development, TAS assessed student learning (4.1 1 0.21) at the same level they asked students to achieve (3.65 1 0.19) (Wilcox Test, p=0.1020). The larger standard errors surrounding these measures may correspond to the larger range of Bloom’s levels targeted by TAs. In other words, TAs asked students to achieve across multiple Bloom’s levels in each lab, ranging from 1-5 whereas traditional labs had a much smaller range, focusing on Bloom’s levels 1-3. 3. 6.3 Level 3: TA Application In the full linear model, only type of professional development (i.e., T, R1 or R2) was a significant predictor of composite RTOP score (F2,54=6.08, p<0.005). Other covariates of interest (i.e., gender and semesters of teaching experience outside of the current institution) were non-significant predictors, as was the time of videotaping (i.e., tape 1 vs. tape 2). Average RTOP ratings across all TAS for each semester of professional development showed a significantly greater degree of learner-centered instruction 62 in both semesters of reform (R1 and R2) than in the traditional semester (Table 3.4, Figure 3.7; ANVOA, F2,54=6.18, p=0.005). Table 3.4. ANOVA table for semesters of TA professional development by RTOP. DF Sum of Mean F p-value Squares Square Model of Professional 2 654.39 327.19 6.18 0.003 Development Residuals 54 2858.16 52.93 100 ~ 90 1 ITraditional lReformedl Reformedz 80 ~ 70 ~ 60 ~ 50 — 4o - RTOP Score 30- 10~ Professional Development Model Figure 3.7. Mean RTOP score for TAs by semester of professional development. Bars represent the average RTOP scores for all TAs in each semester of professional development. Letters indicate significant (p<0.05) differences based on Tukey‘s HSD. The RTOP sub-scales of (1) instructional design and implementation (ANOVA, F2,54=8.92, p=0.0005), (4) classroom culture: communicative interactions (ANOVA, F2,54=3.99, p=0.024) and (5) classroom culture: student-teacher 63 relationships (ANOVA, F2,54=9.88 p=0.0005) are significantly higher in reformed semesters than in traditional professional development (Figure 3.8). 18 ‘ I Traditional l Reformed 1 I Reformed 2 RTOP Sub-Scale Score 1 2 3 4 5 Sub-Scales Figure 3.8. Mean RTOP scores for TAs by semester of professional development. The three bars (with standard errors) for each sub-scale represent the average RTOP score for all TAs on that subscale within each particular semester. Letters above bars represent significant (p<0.05) differences among semesters (as determined by ANOVA and post-hoc pairwise comparisons using Tukey’s HSD). Subscale (1) is instructional design and implementation. Subscale (2) is content knowledge, (3) is procedural knowledge, (4) is communicative interactions and (5) is student-teacher relationships. In addition, five TAs taught Biol across multiple semesters. One TA taught during the traditional and reformed professional development models, and four TAs taught during the two semesters of reformed professional development. 64 3.6.3.1 Comparison between Traditional and Reformed Considering first the TA teaching during the traditional and reformed (R1) professional development, we see improvements in his overall RTOP score between the two semesters (Figure 3.9). 100 - I Traditional I Reformed 1 90 - 80 ~ 70 ~ 60 — 50 ~ 40— RTOP Score 30- 20~ 10 ~ 0 _- eta-1‘1 RTOP 1 2 3 4 5 Figure 3.9. Comparison of RTOP scores for TA 1 between traditional and reformed (R1) professional development models. The "RTOP” bar represents the composite RTOP score of Tape 1 and Tape 2 combined for TA 1. The 5 following bars correspond to the 5 subscales of RTOP: (1) lesson design, (2) content knowledge, (3) procedural knowledge, (4) communicative interactions, and (5) student-teacher relationships. TA 1 also made substantial improvements in RTOP scores on the subscales of communicative interactions (4) and student-teacher relationships (5) (Figure 3.9). The subscales of instructional design and implementation (1), content knowledge (2) and procedural knowledge (3) exhibit only slight positive changes between the 65 two models of professional development. These scales were likely influenced by a change in the content of the labs (recall that during reformed professional development, TAs taught a cellular reproduction and animal dissection lab, wherein traditional professional development, TAs taught a lab on predator-prey dynamics and phylogenetic tree construction). However, the substantial improvements in the communicative interactions (4) and student-teacher relationships (5) are less likely to be influenced by a change in the content focus of the laboratory (Figure 3.9). 3. 6.3.2 Comparison between Reformed 1 and Reformed 2 There were four TAs who taught across both semesters of reformed professional development (R1 and R2). For the composite RTOP score, three of the four TAs showed improvement from the first time teaching the cellular reproduction lab to the second time (Figure 3.10). In particular, TAs 1, 2, and 3 increased their RTOP scores in the lesson design and implementation (Figure 3.1 1), content knowledge (Figure 3.12), and student-teacher relationships subscales (Figure 3.13). TA 4 was the only TA to exhibit a decrease in RTOP scores between R1 and R2, and this decrease is most pronounced in the instructional design and implementation sub-scale (Figure 3.11). 66 O 8 _ — TA1 "" TA2 """" TA3 o """ ' TA4 m —1 O 3 8 _ .......... o o ........... m ................... g ............................... . l- o _ ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, m V gnu—1:12: _________ e were ____________ _9 8: --------------------------------------------- """" O D _ N o _. I I R1 Tapel R2 Tape1 Figure 3.10. Comparison of tape 1 RTOP scores for 4 TAs teaching during R1 and R2. Three of the four TAs exhibit increases in their RTOP score from R1 to R2. O _ N —— TA1 ---- TA2 ........ TA3 ------ TA4 ID _ 0 _ Instruction Design and Implementation Score O— I ' I R1 Tape1 R2 Tape1 Figure 3.11. Comparison of tape 1 instructional design sub-scale scores 4 TAS teaching during R1 and R2. Three of the four TAs show increases in their RTOP sub-scale score for instructional design and implementation. 67 20 15 Content Knowledge Score 1 0 l o— l I R1 Tape1 R2 Tape1 Figure 3.12. Comparison of tape 1 content knowledge sub-scale scores 4 TAs teaching during R1 and R2. Three of the four TAs show increases in their RTOP sub-scale score for instructional design and implementation. 8 - — TA1 2 o 0 a) .3 9 - .C U) c .o E as 2 — 3 .I: 0 «I o S .n — c o .313, o — l l R1 Tape1 R2 Tape1 Figure 3.13. Comparison of tape 1 student-teacher relationships sub-scale scores 4 TAS teaching during R1 and R2. Three of the four TAs show increases in their RTOP sub- scale score for instructional design and implementation. 68 Interestingly, TA 3 began noticeably higher on the RTOP subscale for student-teacher relationships (Figure 3.13) than the other three TAS, yet continued to make improvements between the two semesters of reformed professional development In the second taping, TAS taught the structure-function relationships of animals. Three of the 4 repeating TAS displayed a slight decline in their total RTOP score (Figure 3.14). This decline may be a result of the content of this lab, and the fact that this lab is more prescriptive in nature. o e - — TA1 TA2 -------- TA3 o ------ TA4 co _. 2 o _ o co 0 a) O. o ---------------------------------------------------- 3 9 o _ ______________________________________________________________________________ a: V @2223; """""""" ____________________________ o o _ N O -l l l R1 Tape2 R2 TapeZ Figure 3.14. Comparison of tape 2 RTOP scores 4 TAs teaching during R1 and R2. Three of the four TAS show decreases in their RTOP composite score between R1 and R2. Combining tape 1 and tape 2, these 4 TAS improved their RTOP scores while involved in reformed professional development Their mean RTOP score during R1 is 37.18 1 1.06 and their mean RTOP score during R2 is 40.31 :t 5.53. 69 3.7 Discussion Results from this research provide the first multi-level evaluations of professional development provided to biology TAS. Typically, professional development programs are evaluated based on self-report survey data. In this case, considering just TA reaction data (Level 1) during the first semester of reformed professional development (R1) revealed that the reformed professional development model was less effective in preparing TAS to teach introductory biology labs than the traditional approach (Figure 3.2). However, when we consider data targeting other levels of Kirkpatrick’s evaluation framework, such as what is actually happening in the TAs' classrooms, we see a different picture. In R1 and R2, TAs asked their students to achieve to higher cognitive processing levels (see Figure 3.4), and more closely aligned their assessment of student learning with the intended level of cognitive processes. In addition, TAs taught in more learner-centered approaches during the reformed professional development than when they were prepared by traditional professional development (see Figure 3.7). Specifically, TAs made significant improvements in the design of their lessons and their interactions with students (see Figure 3.8). Therefore, self-report data are not good predictors of what TAs do in their classroom practices. 3. 7.1 Level 1: TA Reactions By the second semester of reformed professional development, TAs felt that their weekly preparation effectively prepared them to teach, increased their confidence as instructors, and improved their teaching skills (Figure 3.2). However, 70 this was not the case during the first semester of reformed professional development, where TAS reported feeling less prepared, less confident and having not improved their teaching skills. This significant change in satisfaction is likely a reflection of the significant levels of frustration among TAs during this first semester of reform. The first reformed semester (R1) involved a large number of changes - in the ways students were assessed, in the format of the laboratory experience, in the leadership of the preparation meetings, in the roles among the Bi01 staff, and in TA responsibilities. Since many of these TAs had previously taught in traditional Biol over the past few years, TAs were resistant to change. This was particularly noticeable in Friday meetings when TAS voiced opinions about moving to a common pre- and post-lab assessment format rather than being able to make their own quizzes for student. Much of the meeting time was devoted to group decision making, which raised TA frustration levels. On the one hand, TAS wanted to have a voice, a say in what happened in Bi01 - they wanted ownership. On the other hand, TAs wanted someone else to make a decision, and felt we spent too much time “arguing" over rubrics and making decisions. Therefore, TAs in the first semester of reform reported that {there was no time to actually improve or focus on their teaching skills because the meetings were not focused on pedagogies. During the reform, the leaders were also trying to determine a model of professional development that met the needs of the TAs. Weekly, the leaders tried new approaches and then sought feedback from TAs on how this particular approach helped them prepare to teach. For example, during the first semester of 71 the reform, we asked TAs to lead the professional development meetings and create lesson plans. This proved, however, to be too much for TAS to take on while simultaneously developing new course materials. We continued to respond to their feedback, and our iterative process finally converged on using a moderated fishbowl. Through the moderated fishbowl (see Appendix A) TAs finally were able to see what it looked like to teach in learner-centered ways, something they had little experience observing during their “apprenticeship of observation" during their K-16 years as a student (Lortie 1975). This moderated fishbowl model was in place for three-quarters of the second semester of reformed professional development (R2), and the TA reactions to their professional development reflect their appreciation of how this model helped their classroom practice. 3. 7.2 Level 2: TA Learning After receiving training in backward design, TAs in reformed professional development created assessments that asked students to achieve to similar cognitive processing levels as their objectives; both of which target significantly higher cognitive processing levels than those created during traditional professional development (see Figures 3.3 and 3.4). During traditional professional development, TAs received limited instruction about assessment. TAs were given the freedom to create assessments as they desired and prior examples of assessments (especially the mid-term and final lab exam) were made available to the TAs. However, TAs were not provided with instruction that outlined alignment between learning objectives and assessments, 72 nor were they provided Bloom's Taxonomy for Learning in order to determine cognitive processes required by certain items. TAs did have an opportunity to get feedback from their colleagues during a preparation meeting preceding the administration of high-stakes assessments (i.e., exams). Much of this conversation, however, focused on creating multiple forms of exams in order to prevent cheating. Once TAs learned about Bloom’s Taxonomy and set objectives that asked students to achieve to higher cognitive levels, the challenge became designing assessments that provided acceptable evidence of students' achievement of the Ieaming goals. Working collaboratively, TAs created assessments for students based on the stated objectives, and spent time refining their assessments based on peer feedback. TAs in reformed professional development demonstrated their Ieaming about assessing student’s knowledge by clearly articulating learning objectives and aligned assessments. 3. 7.3 Level 3: TA Application TAs in reformed professional development (both R1 and R2) had significantly higher RTOP scores than TAs prepared in traditional professional development (see Figure 3.7). This means that TAs prepared to teach through the new model of professional development taught in more learner-centered ways than TAs who were prepared to teach though traditional professional development. This was reflected in significant improvements in the lesson design and implementation, communicative interactions, and student-teacher relationship sub-scales of RTOP (Figure 3.8). .73 Since the goal of the RTOP instrument is to be able to detect the degree of learner-centered instruction, it is encouraging that TAs trained through reformed professional development made large improvements in student-teacher relationships. Post review of the videos, the RTOP raters commented two contrasting approaches to teaching observed in the videos. Rater 1 mentioned that in many tapes, it seemed like TAs were just talking to an empty room, that there was no consideration of the students in the classroom at all. In contrast, other tapes revealed TAs who interacted with the students in their classes. This result is not surprising, as reformed professional development (especially R2) emphasized discussion among both students and the TA. Collaborative groups were used as the starting point for discussions, and TAs applied group-based pedagogy (adapted from Smith et al. 2005) to engage the whole class in discussion. Therefore, it was encouraging to see significant improvements in the way TAS relate to their students (i.e., engaging in a conversation with them vs. talking at them) after training in reformed professional development. When we looked at data for TAs who taught over multiple semesters, we saw improvements in overall RTOP scores, and individual subscales. The one TA who taught during traditional and reformed professional development made large improvements in all five subscales with the largest gains in student-teacher relationships (see Figure 3.9). The four TAs teaching between both semesters of reformed professional development also exhibited growth (see Figures 3.11-3.14). However, one TA had lower RTOP scores during the second semester of the reform 74 than the first There are many possible variables that influence any one instance of TA classroom practice, and thus these data represent just one snapshot of TA instruction. However, to date, no study has quantified TA classroom practice and these data are the first to quantitatively evaluate TA professional development. 3.8 Conclusions Self-report survey data is the standard by which most programs evaluate professional development, including TA training (see Marincovich et al. 1998 for examples). This study demonstrates that considering self-report survey data alone would have led us to make a different, and incomplete, conclusion about efficacy of reformed professional development. During the first semester of the reform, TAs reported a decrease in confidence and teaching skills in response to professional development (Figure 3.2). Based on these results, we would have concluded that our reformed professional development was not meeting TA needs. These results, in combination with the resistance by the TAs to changing the approach to instruction in Bi01, may have been enough for us to revert back to the traditional model. However, when we evaluated what was happening in the TAs' classrooms, we observed that reformed professional development was associated with positive changes in TA classroom practice. For example, TAs in reformed professional development had lessons that engaged students in the process of science and conducted class through a discussion-based pedagogy increasing the interaction among students and teachers. Although the changes in the degree of Ieamer- centered instruction taking place in the Bi01 classrooms increased significantly 75 during the reformed professional development period, it is important to note that there is still room for substantial growth in all five of the RTOP subscales. These data provide a first glimpse into evaluation of TA professional development that actually considers application (level 3) data (Kirkpatrick 1994); that is, how are TAs applying their training in their classrooms? Nationally there is considerable attention to improving college-level science courses, particularly in the first year (see AAAS 2009, for example). A significant number of first-year biology students, particularly at universities with very high research activity, are being taught by graduate TAS. lf programs are serious about efforts to improve the quality of first-year college science they must pay attention to the quality of TA training. The implications from this research provide clear expectations for evaluating TA professional development - at the very least, we need to assess professional development at two levels in addition to self-report data. First, a detailed analysis of course materials created by TAs (e.g., Bloom’s Taxonomy applied to assess cognitive processing levels) provides insight into the level of cognitive processing required of students in the TAs' courses. Second, videotapes of classroom practice with actual analysis are necessary to quantitatively communicate changes taking place in teaching. Videotapes are often used as part of professional development feedback but often only include informal feedback Analyzing these videotapes using valid and reliable tools (e.g., RTOP) provide data upon which to make instructional and professional development decisions. While this research represents the first study to evaluate TA professional development in biology, additional research is needed to understand the impact of 76 reforming TA practices on student learning (Kirkpatrick’s 4th level: Impact). By considering data from multiple sources in evaluating professional development, a clearer and more representative picture may be formed of the effectiveness of TA professional development. This research provides one example of how multiple data sources can contribute to making the picture of TA professional development more clear. 77 CHAPTER 4 A CASE STUDY OF A 3101 TA'S BELIEFS AND PRACTICES 4.1 Introduction In recent years, numerous reports advocating reform of undergraduate STEM education have shined the spotlight on the ineffectiveness of traditional lectures for promoting critical thinking, scientific reasoning and understanding of the complex systems in STEM disciplines (NRC 1999, NRC 2003, Alberts 2008, Wood 2009). Research over the past decade has clearly demonstrated that students understand science best when they are able to confront their prior conceptions, interact with the content and with their peers, and formulate their own conceptual models of understanding (Brainard 2007, Bransford et al. 2000, Freeman et al. 2007, Hake 1998, Michael 2006, Prince 2004). In order to provide such opportunities to students, instructors often attend professional development meetings or workshops where they are exposed to research-based best practices for enhancing student Ieaming (Pfund et al. 2009). One often overlooked population of instructors in this process are the graduate teaching assistants (TAS). TAs teach large numbers of undergraduates, especially in STEM disciplines, across the country. While literature focused on TA professional development, especially in biology, is sparse, research shows that many TAs are dissatisfied with the current professional development afforded to them (Baviskar and Beardsley 2006, Luft et al. 2004). Considering that we know little about graduate TAs' beliefs and practices as instructors of undergraduate science courses it is not surprising that designing 78 professional development that meets the needs of the TA is not driven by evidence. While a wealth of research has illuminated characteristics of successful professional development for K-12 teachers (Borko 2004, Desimone et al. 2002, Fiszer 2004, Loucks-I-lorsley et al. 2009), it is possible that TAs have distinct characteristics that influence their response to professional development in ways that differ from pre- and in-service teachers. Research investigating what TAS believe about effective instruction, how TAs teach in their classrooms and what variables influence their beliefs and classroom instruction is needed to help us develop effective professional development. This case study focuses on the experience of one graduate TA as he participated in professional development designed to prepare him to teach in learner-centered classrooms. To better understand the context of this research, I first discuss TA professional development and the relevant theories. Then, I describe the context in which this TA teaches, and the professional development he receives. Finally, I consider data from surveys, interviews and classroom practice as evidence of what this TA believes about teaching and learning, how this TA teaches, and how professional development may influence both his beliefs and practices. 42 WW Historically, science TAs first assumed roles in the classroom as assistants to professors - serving as graders and course managers (Allen and Rueter 1990, Chism 1998). As the role of the TA expanded, TAs assumed responsibility for teaching laboratory sections, discussion sections and even independent courses. TAS employed for such courses were often at the beginning of their graduate student 79 experience, and therefore one of the main concerns was whether the TA understood the science well enough to adequately teach it. Hence, the priority of early professional development was to insure that TAs understood the subject matter. While goals for TA professional development vary (Gray & Buerkel-Rothfuss 1991), many TA training opportunities focus on biology content rather than on the intersection of content and pedagogy. As a result, it is common practice in professional development sessions to have TAs listen to lectures about content they are to convey to their students. The focus of the preparation meetings centers around the leader’s presentation rather than on the TAS as learners. Since these preparation meetings are often lecture-based, they fail to model reformed instructional design that promotes learner-centered teaching; the very pedagogical approaches that are increasingly implemented in biology courses across the country. This presents challenges to the TAS as their professional development fails to align with current and future course expectations (Adams 2002). 4.3 llniouejhallongesoflAflofossionaDeoeloomont TA professional development is a relatively new research area. Decades of research on preparing future K-12 teachers, in addition to continuing education for in-service teachers provide some insights into how one might approach TA professional development. While a useful place to start, findings from K-12 research may not be entirely applicable to TAS, who represent a unique and understudied population of teachers. TAs are graduate students employed as teachers during their time as Masters or PhD students. Thus, their primary focus is on research and on becoming a 80 researcher. While many graduate students are interested in gaining teaching experience, there are a small portion of graduate students who have no desire to teach (in this study, 6% of TAs solely taught for financial support or departmental requirements) and find themselves as TAs only to fulfill a departmental requirement or to secure funding to remain in school. In addition, the culture of the research- intensive university may not adequately support graduate students as TAs (Latulippe 2007). In contrast, pre-service teachers are primarily focused on teaching and Ieaming methods to become a better teacher. These external pressures and differences in motivation may have a significant impact on TAs beliefs about teaching and student learning. Moreover, the teaching preparation of graduate students differs significantly from pre-service teachers. Pre-service teachers take coursework focused on how students learn, and how to think about their content area in the context of teaching, focusing on Pedagogical Content Knowledge (PCK) (Ball and Bass 2000, Shulman 1986). TAs often teach their first course to undergraduate students without similar preparation in teaching methods and theory (Pillar et al. 2008, Prieto et al. 2007). Thus, TAs may have very different views about what makes effective instruction compared to pre-service teachers. Since TAs are a unique population of teachers, have distinct methods of teacher preparation from pre-service teachers, and are understudied in the literature, the goals of this field study are to define and understand the relationships among TA preparation, beliefs and classroom practice. 81 4.4 Bomnchfiosign In this research, I analyzed and evaluated the relationships among TA preparation, beliefs and classroom practice for one particular TA, jack. Specifically, I am interested in determining: 1. What does lack believe about teaching and student learning? 2. How do these beliefs influence his classroom practices? 3. What is the influence of traditional and reformed professional development on jack’s beliefs and classroom practices? I start by describing the course jack is teaching, jack himself, and the professional development he is receiving. Then, I describe the methodology used to elucidate jack’s experience as a TA. Following this, I discuss the results and interpretations of these data. 4.4.1 Focal Course - Biology 1 Introductory Biology 1 (Biol) at a large Midwestern, research-intensive university, focuses on the core concept areas of genetics, evolution and ecology. Each week, students in Biol attend 2-3 lectures taught by faculty members and a single three-hour laboratory taught by TAS. During the past two years, Biol underwent curricular changes in instructional design to make the course more learn-centered and active for students. More specifically, in the fall of 2008, the lecture portion was reconceptualized to include active learning class meetings (formerly called lectures), where students work in collaborative groups to investigate biological concepts of genetics, evolution and ecology. 82 Beginning in the fall of 2008, the Biol laboratories were redesigned towards learner-centered and active inquiry by students. The transition of the labs towards these learner-centered methods took one full academic year, and also required significant changes in the professional development given to TAs to teach these labs. Since Bi01 already has an established TA preparation program, Biol labs provided an ideal course in which to investigate the experience of a TA under two models of professional development. 4.4.2 Jack This research follows a case study design (Yin 2008) focusing on a single TA, jack. jack is a 2nd year Masters student teaching Biol (data from surveys, see below). Like many graduate students, he arrived in graduate school with minimal teaching experience; he has no formal training about teaching and Ieaming. In his first semester as a graduate student, jack was a TA in a non-majors environmental science laboratory, where he taught a stand-alone course with little to no pedagogical preparation from the supervising professor or the other TA teaching a second section. In jack’s second semester, he became a TA in Biol where he continued to teach for two additional semesters. jack indicated his primary interests are in his research, and he does not know whether or not he will go on to be a teacher in any capacity (see section 4.5.1). He also had aspirations of completing his Master's degree in a two-year time frame, and was told by faculty in his department that teaching should not get in the way of his research; however, at the present time, jack will take longer than 2 years to finish 83 his degree due in large part to the time commitment he has made to teaching in addition to his research (see section 4.5.3). While jack is one of 13 TAs teaching Biol during the spring of 2009, jack has been a TA in this biology course for three semesters, during which he experienced both the old (traditional) and new (reformed) models of TA professional development. Therefore, jack is uniquely positioned to have views on both models of professional development. Since jack has been a TA before, it is important to note that the experience of being a TA is not new for lack, although he is in a new course (Bi01) during the duration of the study. Therefore, it is possible there are other influences on jack’s thinking about teaching and student learning beyond the types of professional development jack participated in during this study. 4.4.3 jack’s Professional Development 4.4.3.1 Traditional Professional Development Model During the first semester of this research, jack and the other TAS participated in what 1 term “traditional" professional development. This professional development is considered traditional because this model is what historically was offered to TAs in this program. Traditional professional development consists of a weekly 3-hour preparation meeting. The goals of this preparation were to make sure all TAs were (1) clear about the logistics of the laboratory activity, (2) ready to deliver a lecture on the biology content required to complete the lab exercise, and (3) properly use the equipment needed for the lab activity. I videotaped each professional development meeting, and also took observational notes during each meeting in order to inform my description of the models of professional 84 development. However, these data are not used in the analysis of my specific research questions about lack and his experiences as a TA. In traditional professional development TAs typically arrived, sat at the lab benches and listened to a lecture than could last as long as 2 hours. This lecture was led by a leader, either the lab coordinator, or an experienced TA who had taught this particular lab before and was willing to assume the role of a leader. During the lecture, some TAs took notes while others were observed paying little or no attention to the leader. PowerPoint slides of the lectures were made available to each TA through a course management system. Following the presentation of the content, the leader walked the TAs through the laboratory exercise. The focus during this time was on making sure the TAS understood the equipment, logistics, important steps in the procedure, and could walk the students through the lab. This traditional model, however, failed to consider the students. The highly scripted protocol implied that it was the TAs' job to direct students through a predetermined experience rather than an open-ended, perhaps even unpredictable exploration. There was little mention of the intellectual development of the students enrolled in these labs. Rather, TAs received a clear message in these meetings - undergraduates could not be expected to figure the laboratory experiment out through exploration. This model is familiar to many who have taught in large lecture classes in American universities, especially in the sciences (Wood 2009). Survey data from other TAs, like jack, indicate they left these meetings feeling less than prepared to teach, and often were annoyed both at the length of the 85 meeting and the quality of preparation (see Appendix A). lack recalled that he often went home and prepared an additional 10 hours over the weekend to get ready to teach his sections the following week. 4. 4.3.2 Reformed Professional Development Model Based on the shortcomings of traditional professional development as well as the changes taking place in Biol, a team of researchers (myself included) applied backward design (Wiggins & McTighe 1998) to reform TA professional development (see Appendix A). Briefly, we first established goals and objectives for TA professional development that aligned with the goals of Biol and considered data (see Chapter 2) on the incoming beliefs and experiences of Biol TAs. Second, we determined what evidence we would accept to evaluate the effectiveness of professional development in achieving the stated objectives (see Appendix A). Finally, we developed weekly professional development meetings that modeled learner-centered, cooperative-learning approaches that facilitate TA obtainment of professional development objectives (see Appendix A). A model of the reformed professional development reflects the components of the reformed professional development jack participates in during this study (Figure 4.1). 86 interact . e W . 5‘99? Influence ‘v’ Leader(s) Er) and Content demonstrates 1 Influence TAs’ receptiveness to Figure 4.1. Reformed professional development for Bi01 TAs. This model of professional development considers incoming TA beliefs and experiences as influences on the TA and their response to professional development. Within the context of professional development, a complex system of interactions exists. TAS are learning in community and interacting both with the content and with their peers to construct an understanding of pedagogy and content. Leaders are providing feedback and facilitating throughout the process. Using videotapes and observations of reformed professional development meetings, I was able to characterize and describe the reformed professional development in which lack participated. Reformed professional development in Biol is TA-centered and active: TAs work in collaborative groups and learn through talking with one another. Furthermore, the TAs’ experiences in their Friday prep meetings are similar to the learning experiences they are encouraged to create in their labs during the week. The weekly prep meetings are still guided by a leader, but the TAs are now actively involved in the meeting. Leaders (using pseudonyms) include a science researcher (Kelsey), and two staff of the Biol program; one a science faculty member (Michael) and another a staff worker with responsibilities 87 for professional development of TAs (Bea), who is intermittently present at prep meetings. The TAs sit in heterogeneous base groups composed of one TA who has experience teaching Biol, one TA who is new to Biol, and undergraduate educational assistants (UEAs) who are working with each of the TAs. These base groups are the focal point of the preparation meetings. At the beginning of each weekly meeting, the TAs and UEAs talk about teaching strategies and reflect on the prior week’s lab activity, then report back some highlights to the group. After this reflection, the TAs get ready to participate in the upcoming lab activity. The UEAs model the undergraduate students in the TAs labs later in the week One prep meeting leader (most often, Kelsey) models how she would teach her own lab. The TAs observe the dialogue between the leader and the UEAs, and an additional leader (most often Michael; the moderator) facilitates a discussion about what is happening in this “mock classroom". This moderator directs TAs’ attention to the teaching strategies used by the leader, decisions the leader is making (that may or may not be apparent) as they instruct and interact with the responses from the UEAs. We named this pedagogical strategy a “moderated fishbow ” because the TAs watch the classroom as if they are peering through a fishbowl. It is moderated because throughout the observation, TAS are reflecting on the pedagogical approaches and student responses taking place in the mock classroom. Before the TAs and UEAs work together to complete the laboratory, there is time for questions directly about what they are observing in the moderated fishbowl. TAs are encouraged to ask clarifying questions at this time. Occasionally 88 TAs will interject questions or comments while the fishbowl activity is occurring; although this happened more commonly when the pedagogy was first introduced in the prep meetings. While the base groups work on the lab, they are focusing on the purpose of the lab, the laboratory content (e.g., concepts, skills and tools), and the pedagogies they can apply to elicit student thinking during the lab. The preparation meeting concludes with a wrap-up discussion, announcements and logistics. This reformed model (see Appendix A) emphasizes creating community among TAs through collaborative groups (johnson and johnson 1994), focuses on having TAs listen to student responses and allowing their responses to dictate the direction of the conversation (von Glasersfeld 1991), challenges TAs to become knowledgeable about the subject matter, and encourages TAs to probe student thinking. This reformed format places the TAs as active reflectors on observations, participants in discussions and lab activities — they draw their own conclusions rather than receiving an answer key-, and creators of structuring their lab session meetings by determining the pedagogies they wish to apply in their own labs. 4.5 Methods lack chose to participate in this study for three semesters as a TA in Bi01. I collected data on his teaching and his beliefs through surveys, observations and interviews (Figure 4.2). During traditional professional development, jack did not allow me to videotape his classroom teaching, but later agreed to videotaping for both semesters of reformed professional development. 89 Traditional Model Reformed Model Spring 2008 Fall 2008 Spring 2009 Jack’s 1St Jack’s 2"d Jack’s 3rd Semester Semester Semester Data Collected: Data Collected: Data Collected: Surveys Su rveys Surveys Classroom Observations Interview Classroom Observations Figure 4.2. Design of data collection during jack's three semesters as a TA in Biol. During jack’s first semester (T) I collected survey data from him at the beginning and end of this semester. During the two semesters of reformed professional development, I collected surveys at the beginning and end of each semester, as well as four observations of classroom practice (2 during each semester). Finally, jack participated in a structured interview during his final semester as a TA for Biol. 4.5.1 Surveys jack completed three surveys at the beginning of each semester focusing on his beliefs about teaching and student learning, his academic background, and teaching experiences. At the conclusion of each semester, jack took additional surveys on his beliefs about teaching and student learning, and his perspective on the effectiveness of the provided teaching preparation (Appendix B). I developed these surveys based on the literature about teacher beliefs (Carroll 1977, Richardson 1995, Richardson 1996), and pilot tested with a similar cohort of TAS prior to having jack take them. 4.5.2 Videotapes of jack’s Classroom Practice During Iack’s second and third semesters teaching Biol, I videotaped two different labs. Each semester, I videotaped jack teaching once at the beginning of 90 the semester (tape 1; Cellular Reproduction), and once towards the end of the semester (tape 2; Animal Diversity). jack knew one week in advance that I would videotape his class. I chose two distinct laboratories that focused on different scales of biological organization and also organisms. In the first laboratory (tape 1) students tested hypotheses about mitosis, and during the second laboratory (tape 2) students investigated the structure-function relationship among 4 marine species through dissection; these labs were the same across each semester. In particular, these labs allowed me to see lack interacting with his students in two distinct content labs, with varying degrees of inquiry (i.e., lab 1 exhibits a greater degree of inquiry than lab 2). In addition, each week, I videotaped the professional development meetings where jack was a participant. 4.5.3 Interview After I initially reviewed videotapes of jack’s professional development and his classroom teaching, I developed questions about jack’s beliefs, teaching practices and experiences in professional development (Appendix E). I used these questions as my research questions on jack's beliefs about teaching and learning, jack’s classroom practices and jack’s experiences in traditional and reformed professional development. My interview questions were reviewed and refined by a team of researchers, students in a graduate qualitative methods course, and Dr. Suzanne Wilson, the professor of this qualitative methods course, and an expert in qualitative research in education. Although this was a structured interview where my questions directed the flow of the interview, I did allow jack to direct the conversation occasionally. 91 jack and I sat down together for an hour-long interview at the start of his third semester teaching in Biol. l videotaped this interview and took field notes during the interview focusing on jack’s experiences in professional development across the three semesters he served as a TA. 4.6 DataAnalxsis 4. 6.1 Coding of Survey Results and Interview Transcript I compiled the responses from jack’s surveys and transcribed jack’s interview within one week of the interview time. As I read through jack’s survey responses and interview transcript, patterns emerged surrounding his beliefs. As a result of these patterns, I developed a coding scheme (Bogdan & Biklen 1998). I coded for (l) metaphors jack used to describe and characterize his beliefs about teaching and learning, (2) influences (i.e., those things or people that influenced him to develop and hold certain beliefs) on his beliefs and teaching practices, and (3) the experience of both traditional and reformed professional deveIOpment models experienced by lack during his time as a TA in Biol. Additional responses, such as jack’s career aspirations, influences on his beliefs and classroom practices were noted from his written responses. 4. 6.2 jack’s Classroom Practice Videotapes I transcribed the beginning of each of jack's 4 videotapes (2 from each semester). I focused on the beginning of the lab, as that is the time that jack introduces the lab and gives instructions to his students. The remainder of the time jack is interacting with students in small groups where conversation is not audible enough to transcribe. I watched the entire video of his classroom practice focusing 92 on how jack interacted with his students. Specifically, I looked for sections of video that were illustrative of the beliefs Jack verbally expressed to me, or in contradiction to those beliefs. For example, jack expressed a belief that as a teacher he is part of the community with his students rather than an authority over his students. Therefore, as I watched the videos, I looked for instances where his behavior was consistent with that of being a part of a community (i.e., interacting with students, respectfully listening to what they had to say, generating discussion with them) as well as instances of behaviors that are inconsistent (i.e., telling students what to do, making decisions without student input, not listening to students). I flagged instances that were particularly telling, and many of these are described in the results section below. 4. 6.3 jack’s Review Often, case or ethnographic studies provide the subject being studied an opportunity to review the results from the research, called a “member check” and provide comments and feedback (Elliott et al. 1999, Rossman and Rallis 1998). jack read a copy of this chapter and noted his agreement with what is written here. In addition, he pointed out some corrections (e.g., clarifications of his first TA assignment) that I made to the manuscript. 4.7 Results 4. 7.1 What does lack believe about the teacher and teaching? Compiling data from Iack’s interview, surveys and classroom practice videos, I put together a description of jack’s current beliefs about teaching and student 93 learning. The beliefs included in the summary are those that lack repeatedly stressed in conversation (strong emphasis, or mentioned more than 3 times), articulated in survey responses, and demonstrated through classroom practices. For example, here is a clip from the interview transcript: I think that’s incredibly important that you need to be able to answer questions. And, if you can’t, you need to address it, but you need to find out. You have to be that source for them, that can be, that can answer their questions, and to do that you need to be studied and you need to know what you’re doing; so prepared, I guess, is a good way to put that. lack stressed here that it is “incredibly important" to him that as a teacher you answer questions for your students. The “incredibly important" in his statement led me to believe that what followed was one of Jack's beliefs about effective instruction; thus, the section quoted above was one I flagged for analysis. once flagged, I re-read this section a few times to get a handle on what jack was saying here. I identified his own word “source" as being key to understanding this statement of his beliefs. Then, I tried to define what jack meant by “source" based on the description he gave in the interview. In this case, I defined “source" as teachers needing to be knowledgeable about the biology content This does not mean that the teacher needs to know everything, jack mentions that a teacher needs to be willing to look and discover answers to help their students. Once I had a definition for source, I used this definition to determine other instances of “source" in the transcript I repeated this process with other beliefs stressed by lack and summarized them in Table 4.1. 94 Table 4.1. jack’s beliefs about teaching and learning. jack’s beliefs about teaching and student learning originate from personal experiences in the classroom and‘teaching opportunities. Jack’s Metaphors about Description of Jack's Beliefs about Origin of Beliefs Teachingand Learning: Teaching and Learning Teacher as source Teachers need to be knowledgeable Personal experience Teacher as inquirer Teacher as community creator Teacher as applier Teacher as listener Students as doers about biology content Teachers need to probe students to think about biology Teachers need to establish a community of learners; the teacher needs to be a part of the community, not an authority over the community Teachers need to make content relevant to students’ lives by applying it to real world situations Teachers need to listen to students without interrupting Students need to have opportunities to interact with the biology content; hands-on learning is essential Personal experience in high school physics Personal experience in high school physics Personal experience and interests Personal experience in high school physics Personal experience in high school physics, and graduate school Throughout the results discussed here, I highlighted sections of interview transcript that illustrate jack’s beliefs described in Table 4.1. Central to jack’s beliefs about teaching is the idea of creating a classroom community, which involves listening to his students and making them feel welcome and comfortable. To effectively create community, jack clearly defines his role as a part of the community, rather than an authority over the community. He believes that his interactions with students help create the interactive classroom model intended to help students learn. 95 ”If you came into my class, you’d probably see me constantly walking around, I don’t know if you’ve seen that yet on your videos, but, just constantly moving and, sitting, and talking and not just standing looking over shoulders like (imitates a stern pose with crossed arms). You got to get down and talk.... Get down and actually, hunker down and talk to them....So, I guess I kind of become members of the group.” “...if you sit back and wait, they’ re never going to get comfortable with you, and no matter what you’re teaching you’re never going to get past the like “here’s what we’re going to do today” and ”hand it in at the end of class”. You [will] never get into where they are willing to ask you questions.” Essential to forming community to facilitate Ieaming, lack believes he needs to listen to his students. [In my classroom, when I am walking around the room] ”the first thing I do is, a lot of the times, when I stop, especially at the beginning of the semester, I’ll listen for key words. I’II listen for “I don’t get it”, you know, just a statement like that, or ”this doesn’t seem right” or “oh, yea, this makes sense”. [Then] you jump in, and you say ”what makes sense, what are you guys talking about?" So, I’ll just ask and see what’s going on, and they’ll just start talking, and I'll listen, and one thing I make sure is that I listen to the entire thing they say. I don't interrupt them - because, that’ s disrespectful, you know, you, you set that tone of disrespect.” As a teacher, jack believes his role is to bridge content with real world applications of that content to his students. First, jack believes teachers must be knowledgeable of and confident in the content area. jack believes that a teacher is the source of information for the students; not that the teacher must give them the information, but that the teacher needs to be knowledgeable in the subject matter they are teaching. ”I think a lot of times we [as TAs] can walk into situations without being an expert, we don’t take the time to learn everything that we might need to know. I think that’s incredibly important that you need to be able to answer questions. And, if you can't, you need to address it, but you need to find out. You have to be that source for them, that can be, that can answer their questions and to do that you need to be studied and you need to know what you’re doing.” 96 Second, jack discussed the need to repeatedly push students to think deeply; to inquire into their understandings to help them learn to think on their own, not just regurgitate facts that the teacher has already told them. Preparing to teach, jack purposefully seeds his lesson plans with probing questions, as well as anticipating hypothetical student responses. ”... [In my planning] I’ll list questions, a lot of times they are the questions from the lesson plan. Something I actually do now is, I’ll have a question, and then I’ll have possible answers. I’ll have answers that I think they might possibly have, so this way it gets me thinking about how to respond and how to connect and make them think further on those topics.” In observations of jack’s classroom practice, the forethought jack puts into thinking about student responses to questions is readily apparent. [lack] [Class] [Jack] [Jack] [Jack] [Jack] [Class] ”What are we doing today? lack asks his students. Virtually the entire class responds, rather enthusiastically, ”Dissecting!” Jack smiles. ”T hat’ 5 right,” he says, ”Specifically, what are we dissecting?” A few students mumble responses, Jack picks up on what they’ve said and asks a follow-up question, ”Are they fresh or marine species?” The discussion carries on, as Jack continues to interact with his students through a series of questions. He continues, “What are the life challenges that organisms face?” Students are able to articulate a few challenges, and Jack writes these down on the chalkboard; organisms need a way to reproduce, a way to obtain and use nutrients, and a way to protect themselves from predators. Jack tells the students there are two more big challenges; and he re-asks the question in another way making it possibly more relevant to the students, ”What else do you have to do every day to survive?” II II Students then respond with the final two: ”Breathing , Moving”. 97 Finally, jack believes that part of his role as a teacher is to shatter the glass wall between the classroom and real life. ”Whether it’ 5 directly on topic or not, you can always connect what you’re doing with something outside that you know personally, because you are, at this point, if you’re teaching you’re an expert in something — you have to be — or, you’re becoming an expert as you teach, so you should have some sort of background information. I think as a teacher, you need to be able to take this little bubble that is the classroom and kind of pop it, you know, and bring in anything else that you can.” 4. 7.2 What does jack believe about the student and student Ieaming? In addition to jack's beliefs about the role of the teacher, jack also has clearly articulated beliefs about the role of the students in his classroom. jack believes the best way for students to learn science is to practice science, and this belief largely stems from his own experiences learning science by doing it starting in high school. ”I’ve been a good student my entire life and I think a large portion of it up until that point was because I could read a book and memorize details. In that [high school physics] class, I think I gained a lot of self confidence in my abilities to actually think as a scientist. I know that sounds cliché, or whatever, but uh, it’ 5 really true, it’ 5 just that you step away from this “you read your chapter and take an exam” on it the next week and you walk into this idea of experiments, and learning through experiments. Um, so I think it really helped me build confidence as a person, let alone as a student, just in my abilities to do something other than the standard book and test type class.” jack also believes that an effective learner is open-minded, attentive and willing to learn. Part of their willingness to learn, he believes, is their ability and willingness to ask questions. As jack creates classroom community, he strives to let his students know that asking questions is important to their education. At the beginning of the semester, he initiates interaction with his students, and by creating 98 a community where he is a part of the group, he believes students are more willing to ask questions. 4. 7.3 How did traditional professional development influence jack? jack arrived as a TA in Biol with beliefs about teaching and student learning that were formed by his own experiences as a student, interactions with colleagues in the university, and prior experiences teaching. At the time of his arrival, jack's beliefs were a combination of the transmission model of instruction and active learning. jack wrote it was his “role to transmit a message to my students, and engage them in active learning." jack personally knew about the positive influence of active learning in his own “apprenticeship of observation" (Lortie 1975), but also knew the role of a teacher was to teach students biology content For the semester he participated in the traditional model of TA preparation, jack taught in ways that seemingly contradicted his beliefs about active learning. ”That first semester I would stand up there and dictate for 30 minutes, and have a power point slide and really just lecture at them, I guess, is a better way to put it. And that worked for me, it was fine - it was my job, that’s what I thought we were doing. What I did not do, at all, was ask them [the students] as we went along, because here was this power point I made and I was going to get through it and get them into [the lab activityl.” While jack clearly had beliefs about engaging students in active learning, and creating a community (see Table 4.1), under the traditional approach to professional development, jack did not fully express these beliefs in his classroom practice. Instead, jack more heavily emphasized the role of the teacher as “the source." jack attributes this to the organizational structure of the Biol labs, which 99 did not facilitate teaching in ways that aligned with all his beliefs. Instead, Biol labs emphasized the need for the teacher to transmit a message to the students. ”When I came to Biol, it was very structured, and there were some good experiments, and I thought Michael [the prep meeting leader during his first semester teaching] did a good job on trying to get people to think about talking to the students and engaging, but I don’t think the class was structured for that. It was structured like a high school chemistry course - you’re given a sheet, you’re asked to fill it out, and there’s very little interest besides that.” When jack refers to the structure, he is describing the format and layout of the lab activity. Biol labs traditionally were formatted so the first hour was a lecture, followed by the students completing a hands-on lab activity related to the content the TA lectured on at the beginning of the lab. lack articulates here that while there were good experiments and suggested ways to engage students, the design of the lab itself (i.e., lecture followed by a follow-the-directions lab) prevented the instructor from having the freedom to guide students through an inquiry activity. Not only did jack recognize the structure was stifling to him as an instructor, he felt that the students struggled in this traditional approach as well. ”So, I learned later on after their exams and their lack of understanding of any concepts that I ever went over in the PowerPoints that this was not a successful way to do a lab because it’ s not the mindset the students are in when they come to lab. They are not ready to be lectured to, you know, they’ re ready to do hands on work, and we were taking away from that.” One possible reason for the misalignment between what jack articulated as his beliefs about teaching and Ieaming and his practice is found in the unique role of a graduate student researcher who is also working as a TA. jack did not come to graduate school with the intent of being a teacher; rather teaching was a means of financial support As a result, lack saw his role as a TA as a job, wherein he 100 respected and responded to the employers by complying with their expectations. Perhaps the structure of Biol as imposed by his employers was enough for jack to teach in ways that did not entirely align with what jack believed makes effective teaching. 4. 7.4 How did reformed professional development influence jack? For jack, the reformed professional development more closely aligned with his beliefs about teaching and student learning. lack believed that teachers need to create community, listen to their students and “push" students’ thinking about concepts (Table 4.1, Figure 4.3). ”Real World” (outside the classroom Interact ‘ Listen Figure 4.3. Model of lack’s beliefs about teaching and student Ieaming. The items enclosed in the circle represent jack’s beliefs about what happens inside the classroom - specifically those relationships between the students, himself as the teacher and the biology content. The model is based on data presented in this paper. The reformed Bi01 also embraced these ideas. Therefore, the professional development jack and other TAs received to teach in Biol focused on how to create community, effectively establish collaborative groups, manage issues within 101 collaborative groups, facilitate discussions to engage students in discourse surrounding the biology concepts under investigation, and think about the bigger picture of the biology concepts by situating the content into a real-world context Recall that during reformed professional development, TAs participated in a moderated fishbowl, where they reflect on how a teacher progresses through a class discussion and how students respond to the questions raised. Sitting in on jack's lab on animal diversity, jack applied a very discussion-based approach with his students, using almost the same questions rehearsed in the preparation meeting. jack began his class with a series of questions designed to get the students talking in his class. His questions came off in a rapid-fire sequence and students responded by calling out answers. The answers given were typically one word, but lack would begin a dialog with the students that built upon the answer the student provided. 4. 7.5 Motivation for jack’s Classroom Practices Although jack had clearly articulated beliefs about effective teaching and learning, when confronted with a structure of preparation that conflicted with those beliefs, lack taught in those conflicting ways. However, when the structure was there that aligned with his beliefs, and when there was training in the pedagogy to go along with that, lack was able to begin to teach in ways that align with his beliefs. “You know, now I don’t give them the information — I don’t have PowerPoints, I don’t use them. I think I used one slide this year, and it was to show a model of meiosis because we hadn’t covered it in [the lecture portion of the] class. It’ s a lot more of me, asking, and I, I wasn’t sure, how successful this would be when I kind of starting doing this the first semester we changed over [reformed approach]. It would have been very easy for me to continue with the PowerPoint presentations because I had % of them made.... 102 In addition, jack's own personal experiences with active Ieaming may have contributed to his adoption of more learner-centered pedagogies. ”My personal successes in life have come from growing as a scientist because of [teaching]_methods like this [referencing his high school and college instructors who applied these learner-centered methods]; and to me, this is what college ' should be.” While the reformed structure of Biol preparation and laboratory activities changed in a way that afforded jack the opportunity to develop teaching practices aligned with his teaching beliefs, jack also articulated the hesitancy he had with a new approach, which is primarily based on the constraints as a researcher first, and a teacher second. "Let’s face it, its [reformed teaching] harder, and as a TA, as a person who’s doing research, we’re not necessarily looking for the hardest thing in our teaching, you know, we’re trying to do our research, right? One of the things [when I came into do my research here] they said, well, you know you’re going to teach, but it shouldn’t be too much of a time commitment and well, you should be able to get done [with your degree] in 2 to 2.5 years, and [for me] that’ s not going to happen.” Ironically, while jack believed the reformed approach to teaching took a larger intellectual investment in thinking about his approach to teaching, in reality, jack's time commitment for preparing to teach in this new method significantly decreased. “50, when [the prep meetings] got more organized...we have lesson plans, we talk about what we're going to do, and I think that I spend like, maybe an hour now prepping for class, and that’s like creating prep materials or [modifying] my lesson plan.... so I spend a lot less time outside the prep meeting and I think that’s because the prep meetings have improved at building that connection between the lab and science, like why are we doing this Iab....” 103 Despite time commitment concerns, jack did engage with the reformed pedagogical approaches. His motivation to try the new pedagogies is a result of three factors. First, he was inclined to try: “So, I guess, at first I was probably, you know I hate change, so I was probably not the most receptive to it [reformed practice] at first, but, and I [didn’t feel forced to change] because I knew there were other TAs who were still saying “well forget what they’ re saying we should do, we’re still going to do power points and we’re still going to teach how we want”, I said, ”well, they’ re changing this class for a reason, and maybe we should change.” Second, he felt like he understood the rationale for the changes: “So I think once we, once I figured out why we were doing it, and how this was going to help change the course in general, and hopefully other courses, um, that was probably the moment there, [when I decided to change].” jack also noted how well supported he felt under the reformed model of professional development: ”...That’s one thing I'll really give this class credit for is that they’ve tried to address everything that the teachers need, maybe to a point where it’ 5 almost over the top in some cases, right, but you know, we’ve had these great group discussions about how to teach, and styles of teaching, like we had that seminar [Karl Smith workshop on cooperative learning] there at the beginning of the semester....” jack’s needs as a TA were met, as opposed to the traditional professional development which left him on his own to prepare for teaching, and thus, he felt willing to give new pedagogies a try. Third, jack’s role as an employee in the Biol program contributed to his desire to implement reformed pedagogy: ”Like I said, it would have been very easy for me to continue with the power points because I had % of them made. But I honestly felt like Kelsey and others were making a large effort to push towards the new system and I didn’t want to be someone to hold that back... I’d rather help than hurt what you guys were trying to do.... I felt it would be wrong to take, as a person who’s employed by this program 104 and has been given opportunities to be able to teach and to be able to research, it wouldn’t be fair to me to try to deter what seemed to be a positive change..... There were so many people putting so much work into it, why would I want to hurt that - and there must have been a reason that we were trying to put so much work into it, and I had had success that way.” Yelon et al. (2004) described three essential components to change - credibility, practicality and need. For Jack, when he learned the rationale for the reform and how teaching in reformed ways were aligned with what we know about how students learn and thus could positively impact student learning, the reform idea was given credibility. Through jack’s prior experiences using discussion-based approaches and observing the reformed course meetings in Biol (which were also embracing reformed pedagogies), lack saw the practical success of these methods, and was motivated to change. jack mentioned during our interview: “As a teacher I saw the benefits [of teaching in reformed ways by observing Kelsey’ 5 course], and I began to, maybe even more at that point have those personal interactions [with students] maybe that’s where I really started to get rid of old Biol and come in with the new one - quit the lecturing in the labs...now that lecture wasn’t even a lecture, you know, lab shouldn’t be one; I shouldn’t sit there and lecture at them... So, I think it [reformed teaching] is a good style, and [attending lectures] has improved how I thought about this teaching style, I can tell you that.” In both instances above, jack also discovered the need for teaching in learner- centered ways. Recall that TAs attended class meetings; jack attended class meetings for Kelsey’s course; as he sat in these classes, he also heard positive student feedback. ”I hear things in class, one day sticks out because I hear a student behind me who didn’t know I was a TA (this was last semester), ”man, you know, I love the way this goes, I love this class because it’ s not like a lecture, it’s like we’re doing stuff.” I’m like, ”well, how ’bout that!” Or, and then another student says ”man, I can say II II one thing, I don’t get bored in this class . 105 A fourth dimension to factors motivating lack was job responsibility. jack is employed by Biol as support for his graduate education; his first priority is research - he is not even sure he wants to teach. As a result of these conditions, lack felt a responsibility to his job - to do what was asked of him — since it is this job that is allowing him to do his research. The unique position of a teaching assistantship — getting paid to teach to support his role as a researcher - may have given jack more of a feeling of responsibility to do what was asked of him, than if teaching were his primary job responsibility and he were in charge of his own course. lack emphasized one motivating factor for him trying reformed pedagogies in his classroom was not wanting to resist what he saw as a positive transition to reformed teaching methods because it would not be fair of him to do so as an employee. lack has a respect for his employing unit, Biol, and as such, feels a need to comply with their course structure and requirements. just as lack was willing to teach in ways that did not align with his personal beliefs when Biol was traditionally teacher-centered, jack is also willing to try new teaching approaches when the structure of Biol promotes these changes. lack also mentioned during our interactions that as a TA one has a different role than a professor; a TA is not in charge of his own course, and has responsibilities to fulfill the expectations of his employing program or course. This interaction between teaching and job responsibilities makes the TA unique from other pre and in-service teachers and warrants further investigation. 106 4.8 I l' . ”7- D' . lack presents a unique case of the experience of one TA teaching in an undergraduate biology course at a large university. Here, we see lack has a well- developed set of beliefs about effective teaching, largely influenced by his own personal experiences; but, when working as a TA under two very different models of professional development, jack’s teaching practices more closely reflect the model of professional development than they do his beliefs about teaching and learning (Figure 4.4). Years of schoofing 1 creates Personal Inlfluencf Beliefs: + lob Experiences Teaching Responsibility &Learning creates influences additional ‘0 09¢ 90 Professional ccs‘e’s Classroom Development .Qoe“ Practice \(\ Figure 4.4. Model of Influences on Iack’s classroom practice. Jack's classroom practice is influenced by his own person experiences, his beliefs about teaching and student learning, the professional development he receives and the responsibility he feels towards his job. While lack’s case is just an n=l, further investigations would determine whether or not other TAs respond similarly to various types of professional development, and if the four factors influencing change in classroom practice apply in additional situations. Understanding the complexities behind TA beliefs, 107 preparation and classroom practice may offer additional and distinct insights into preparation of TAs. If TAs’ beliefs may be less important to their classroom practice than the actual structure of the professional development, we must carefully reconsider the design and implementation preparation programs for TAs at large universities. By changing the nature of TA professional development to reflect instructional practices based on data about how students learn (see Bransford et al. 2000), TAs at the beginning of their teaching careers can learn to teach using research-based pedagogies. Though having a semester of experiences in reformed professional development, TAs can uncover their beliefs about teaching and student learning, practice teaching in learner-centered ways, and find support in a collaborative community as they learn to teach. Many of the features that could be a part of reformed TA professional development closely align with what data suggest are key components of effective professional development (see Fiszer 2004, Loucks- Horsley et al. 2009). The model of reformed professional development (Appendix A) is such a model that TA programs can apply and adapt as they design TA professional development. By first clearly articulating goals and determining evaluation, existing TA professional development programs can be modified to model active, learner- centered instruction. Through professional development, TAs have a unique experience to learn about teaching and student learning, and learn to teach scientifically. If universities around the country reformed their professional 108 development, the impact could make a significant difference in reforming undergraduate education in STEM. TAs are a unique population of teachers; some will not continue in an academic appointment and will never teach again in formal settings. Nonetheless, they are an integral component to undergraduate education at research-intensive universities. By studying TAs like jack, we can better understand what tools and preparation methods may meet the needs of the TA and serve to advance long-term reform in undergraduate biology courses. 109 CHAPTER 5 CONCLUSION 5.1 Summary The purpose of this dissertation was to introduce and evaluate a reformed method of professional development used to prepare TAs to teach introductory biology. This reformed model is based on the theories of constructivism (von Glasersfeld 1984), cooperative learning (johnson 1984) and variables characterizing the TAs participating in TA professional development The design of the model follows the principles of Scientific Teaching (Handelsman et al. 2004) and Backward Design (Wiggins & McTighe 1998). The ultimate goal of the model is that TAS will teach as they are taught to teach. In Chapter 2, I presented research focused on articulating beliefs of TAs teaching Biol. Specifically, these Biol TAs express beliefs that are both teacher- centered (i.e., they believe effective teachers must be knowledgeable, and able to clearly communicate content to their students), and student-centered (i.e., students learn best by exploring the content). These results revealed that TA professional development needs to provide TAs with an opportunity to uncover their-beliefs and reflect on their beliefs throughout practice. The results from this study informed the development of the novel model for professional development of TAs. Reformed professional development for Biol TAs was learner-centered and active and was a new approach to helping TAs in their process of learning to teach (see Appendix A). In reformed professional development, TAs were both learners and teachers, and were actively engaged in the development, process and 110 instruction of Biol. Through the experiences of observing someone teaching using the expected pedagogies, TAs were able to see firsthand what learner-centered instruction looked like and reflected on instructional decisions made by the teacher in response to student feedback. Data from TAs surveys at the end of the semester indicated that TAs found the moderated fishbowl approach helpful in preparing them to teach using active Ieaming. Analysis of multiple data sources revealed that reformed professional development improved TA classroom practice (see Chapter 3). First, TAs in the reformed model designed learning objectives that asked their students to complete more cognitively complex tasks than TAs did when traditionally prepared to teach (see section 3.6.2). Secondly, TAs in the reformed model assessed their students at cognitive levels more closely aligned to their stated objectives than did the TAs prepared under the traditional model (see section 3.6.2). Third, TAs who learned to teach via the reformed model taught in more learner-centered ways (e.g., discussion) than did those TAs who were prepared to teach via traditional professional development (see section 3.6.3). Specifically, TAs made significant improvements in the design of their lesson and interactions with students. Finally, TAs reported that reformed professional development (especially R2) was more effective than the traditional model in improving their confidence, helping them feel more prepared to teach, and improving their teaching skills. In Chapter 4, I presented a case study of jack, one TA who taught in Biol under both traditional and reformed professional development jack clearly articulated beliefs about teaching and student learning that were more Ieamer- 111 centered than teacher-centered. However, during traditional professional development, jack taught in ways that were more teacher-centered than learner- centered. The trend reversed when lack was prepared via reformed professional development jack's case study indicates that he was willing to teach using pedagogies that contradicted his own personal beliefs about how students learn in response to the expectations of professional development. For example, jack did not believe lecture was an effective pedagogical approach to teaching, but when traditional professional development prepared him to teach in such a way, he felt compelled to teach through a lecture because those were the expectations of his job. While jack is only one TA in Biol, this result is interesting and warrants further exploration to see if this trend is a common pattern among TAs. If this pattern extends to more TAs, reformed professional development programs become the lead vehicle through which change in teaching practices are widely implemented in undergraduate biology courses. Collectively, these research results indicated that reformed professional development can influence TA classroom practice. 4.2 llniousLontnioutions The reformed professional development model introduced and evaluated in this dissertation offers the first comprehensive study of TA professional development in undergraduate biology. Much of the research on TAs is focused in mathematics and offers some insights to the study of biology TAs. However, the practice of science is much different than the practice of mathematics in terms of the application of the scientific method; such differences have implications for the preparation and training of TAs. 112 In addition, this dissertation makes a unique contribution to the field by considering multiple data sources to evaluate professional development. Typically, most professional development opportunities are evaluated solely through self- report survey data (e.g., Kirkpatrick 1976; Pfund et al. 2009). In this case, if we had only considered self-report data from Biol TAs, we would have abandoned the reform when TA responses indicated they were not gaining teaching skills or confidence (see Chapter 3). However, because we considered multiple data sources following Kirkpatrick's evaluation framework, this research offers a more robust understanding of the effectiveness of TA professional development on aspects of TA classroom practice. 63 WW There are three identified limitations of this dissertation that lend themselves to additional inquiry. First, the concept of TA learning in response to professional development merits more thorough investigation. In this study, TA Ieaming was the translation of acquired knowledge into practice, as assessed by objectives and assessments created by TAS post-professional development. Of additional interest is how TAs change their understanding of the nature of science and science concepts as a result of their teaching experience. Can we find evidence to support the claim that teaching science improves the teacher’s understanding of the content and nature of science? Second, this study did not consider the fourth level of Kirkpatrick's evaluation framework - impact Impact is the long-term influence of professional development; this research focused solely on evaluation of the efficacy of TA 113 professional development (a short-term outcome). In assessing the impact of TA professional development, we seek to understand the degree to which changes in TA practice brought about by reformed professional development improves student learning. Research indicates that when teachers teach in active, learner-centered ways, student learning improves (Michael 2006), however, no attempt was made in this dissertation to measure student learning. Additional work is needed to evaluate the influence of student learning in response to TA professional development. This research direction and design requires careful thought and planning, as there are many confounding variables to consider in order to make robust claims about the influence of TA professional development on student learning. Nonetheless, it is an interesting and worthwhile line of inquiry. Third, the case study exploring Iack’s experiences as a TA in Biol led to the conclusion that the constraints and requirements of being a TA may have more influence on a TAs’ teaching practice than their own beliefs. This insight is particularly interesting when considering the role of professional development in influencing TA practice and is worth further exploration. Can we determine whether other TAs are also more highly influenced by their job description and job expectations than their own beliefs about effective instructional practices? Finally, while the results of this dissertation show significant improvement in the degree of learner-centered instruction taking place in laboratory classes taught by TAs, overall, TAs are still primarily teaching with limited student interaction. Continued implementation and refinement of the reformed professional 114 development model has the potential to make significant changes in the way undergraduate biology students are taught. 115 Appendix A - Reforming TA Professional Development in Biology A-1 Introduction Over the past 20 years, numerous calls to reform undergraduate biology in American universities (Boyer 1998, NRC 2003, Wood 2003, DeHaan 2004, Trigwell and Shale 2004, AAAS 2009) ask university faculty to engage students in the process of science, to actively involve them in their own learning, and to teach in the way science is practiced (Handelsman et al. 2004). After nearly two decades of “reform efforts", college faculty are largely still teaching as they were taught; that is, through lecture—based (i.e., teacher-centered) instruction. A recent pioneering study evaluated professional development outcomes by using observational data of faculty teaching science after workshop participation. This study concluded that faculty still teach through lecture with only minor student interaction (Ebert-May et al. submitted). In addition, there was a strong negative correlation between number of years teaching experiences and active, learner-centered, instructional practices. Therefore, there is an opportunity to truly make changes in the way undergraduate education happens if we teach graduate teaching assistants (TAS) to teach science as they practice science in their research labs. Graduate students serving as TAs currently teach a large number of undergraduate students, and the models that are currently in place for teaching graduate students how to teach are not sufficient for adequately preparing TAs to teach. Self-reported data suggest that TAs leave their preparation programs feeling overwhelmed and ill prepared to teach the undergraduates in their classrooms or laboratories (e.g., Luft et al. 2004). 116 Despite the pervasiveness of the ineffectiveness of many TA professional development programs, to date, there is little research on how to effectively prepare biology TAs to teach. While a handful of reports exist in the literature describing models of TA professional development (e.g., Etkina 2000, French and Russell 2002, McManus 2002), little information is present from these studies about how to effectively design and implement meaningful TA professional development This appendix describes the novel model of professional development for TAs teaching introductory biology that builds upon theories of how people learn and the professional development literature from K-12 teacher preparation and was evaluated as part of this dissertation research. The development of this model for the design of TA professional development relied heavily on implications from theories of constructivism (von Glasersfeld 1989) and cooperative learning (Johnson 1984, johnson and johnson 1994, johnson et al. 2000). These two theories are the basis for research that indicates student learning increases when teachers instruct using active, constructivist pedagogies (Michael 2006). Together these two theories of how people learn challenge the traditional notion of a teacher-centered classroom in three key ways. First, constructivism and cooperative learning promote a classroom environment that is interactive. In a constructivist approach to teaching and Ieaming, students create their understandings through experiences (Dewey 1916, Piaget 1932, Piaget and Inhelder 1969, Piaget 1972, von Glasersfeld 1989). 117 Therefore, these theories encourage teachers to promote engagement rather than passivity in the classroom. In this View, classrooms become important “discovery centers" for learning (Bruner 1966). Second, these theories promote placing students at the center of their learning, and thus, the classroom. Since students are the ones who must assimilate and accommodate their conceptual understanding as they interact with new concepts, teachers are encouraged to step aside and allow students to interact with the content and their peers so they can develop and practice their own thinking (Piaget 1972). Third, these theories encourage a sharing of power between teachers and students in the classroom. In the traditional view of teaching, the teacher is seen as the source of information and answers. Applying the theories of constructivism and cooperative learning requires a paradigm shift where teachers are no longer the sole provider of knowledge. These frameworks propose that students engage with materials and each other (johnson and johnson 1994) to build their understanding. This necessitates that teachers believe that problems can be investigated through a variety of approaches (von Glasersfeld 1989). Teachers are encouraged to converse with students about their approach to a problem and why they chose their approach. Through this dialogue, the teacher emerges with a better understanding of approaches students take to solve problems. This new knowledge affords the teacher insight into “how” students think, and should inform his /her instructional decisions. 118 A3 {..2' I'II' ..il. 1‘ not!” I ;_ 0‘ ll: I“Hn‘i Constructivism and cooperative learning have important implications for approaching the design of professional development First, it is important to recognize that teachers have prior knowledge about teaching. Teachers do not enter teacher preparation programs as “tabula rasas”(i.e., blank slates), rather, they come with constructed knowledge about teaching and student learning from years of classroom observation and personal experiences (Piaget 1972, Lortie 1975, Holt- Reynolds 1992). Teachers have well developed conceptions about what makes an effective teacher, how to teach, the purpose of teaching and how students learn (Borko and Putnam 1996). When new information is encountered, teachers work to assimilate and accommodate their conceptual understanding with these new details. Second, since learning is a collaborative process professional development opportunities need to engage teachers with content and their peers. Under these frameworks, teachers build understanding by interacting with the content they will teach and with their peers. In this way, professional development for teachers becomes learner-centered. Preparation for teaching using new pedagogical techniques should afford teachers an opportunity to engage with the content in the same way they are expected to teach it and to practice the teaching strategies they are learning. The collaborative learning framework further suggests that teachers in training should engage in dialog with the professional development leader(s) as they develop their understanding of the new teaching methods. This interaction 119 should include opportunities for teachers to reflect upon and discuss how they are assimilating these new ideas in their current understandings (Richardson 1996). A-4l'lfilEfi'lDl l' The best professional development results are achieved when professional development programs provide teachers opportunities to engage in repetitive, consistent professional development over a longer time period (i.e., longitudinal professional development) (Fiszer 2004). Longitudinal professional development is gaining popularity in primary and secondary education for a few reasons. First, longitudinal professional development provides teachers a community with which to build relationships. Second, this type of professional development gives teachers a chance to learn something new, try implementing it and then get feedback from others who may have tried the same pedagogy. Third, longitudinal professional development promotes the understanding that change takes time and requires a community with which one can engage and seek encouragement while trying new methods (van Driel et al. 2001). Additionally, professional development programs work well when attendees are interactive in the learning process. In other words, when they do something at the workshop, seminar or meeting. Inactive, passive lectures about great teaching strategies are ineffective in promoting professional development in teachers because they do not get a chance to try the strategies and receive feedback based on their implementation (van Driel et al. 2001). 120 A.5 W This research first presents a new model for designing professional development for TAs. This model draws from several sources: 0 Constructivism and cooperative learning frameworks o Backward design (Wiggins and McTighe 1998), and 0 Feedback and input from TAs After presenting the framework for designing professional development for TAs, I provide an example that demonstrates the application of this framework in the context of one TA professional development program. Of course, this implementation of the framework considers the practical constraints specific to this course, such as the course content, TA population, historical context of TA preparation in this program, and available resources (e.g., time, talent and financial support). Arguably, these considerations are not unique to our program, and exist across many universities wishing to improve their TA professional development programs as well. Below, I briefly describe the course of study, the professional development in place prior to the reform and the approach for designing reformed, learner-centered, professional development for TAs. A.5.l Course Overview TAS in this study teach in the Introductory Biology sequence at a large Midwestern university with very high research activity (Carnegie Foundation 2005). The course of interest, Introductory Biology 1 (Biol), covers the content domains of genetics, evolution and ecology focusing on the organism, population, community 121 and ecosystem scales. The course includes two components: a lecture taught by university faculty, and a weekly 3-hour laboratory, taught by TAs. In 2008, Biol began a comprehensive reform to make instruction more learner-centered and active. One of the reform goals was that TAs would learn to apply learner-centered pedagogies in their own instruction of the labs. Therefore, this course offers a unique opportunity to evaluate a new model of TA professional development in a course undergoing reform. A.5.2 Overview of Traditional Professional Development in Bi01 Historically, TAs in this program attended a weekly 3-hour preparation meeting (prep meeting) on Friday afternoons to receive training for teaching the upcoming laboratory exercise. This prep meeting consisted of one leader (sometimes a lead TA) delivering a lecture to TAs about the content students needed to be familiarized with before completing the laboratory exercise. Following the lecture, TAs had an opportunity to familiarize themselves with the laboratory equipment and methods and ask questions related to content and logistics. TAs were prepared to teach in this fashion for at least five years based on the administration of the program. M W A.6.1 Step 1: Articulate/Determine Goals for TA Professional Development The first step in developing professional development for TAs is to determine the broad semester-long goals for TAs to achieve (Figure A.1). Creating measurable goals and objectives serves to inform the development of effectiveness measures, and ultimately the design of the professional development meetings (Wiggins and 122 McTighe 1998). Goals for the TAs are influenced by many factors. First, goals are influenced by the TA population. Consider the TAS: What departments do they come from? How much teaching experience do they have? How many hours a week will they work in this course? What responsibilities (other than teaching) will they have with this course (e.g., office hours, grading, review sessions, etc)? What do they need/want in terms of professional development? Course Acceptable Evidence Assessment i . Professronal , . Development . ”$393.1 Mg} 1“". ~T-3‘T;;¢"'"“0"H:“x‘. 3:7. ‘~..'-‘L--,- . o L s/ . Leaders fl "“ §-;.Jidul\‘y.n.§. ‘i ' ' film: refit - . TA Population " Figure A.1. Designing TA professional development. Steps follow Backward Design (Wiggins and McTighe 1998) and are influenced by variables such as the TA population, the course of interest, the type of assessment/evaluation data collected, time, talent and financial resources at each step. Second, consider the course: What are the goals for the course? Who are the students taking this course? What is the role of the course in the university curriculum? Who are the faculty teaching this course? Historically, what was done to prepare TAs in this course? Thinking through these questions will help determine what important goals are for TAS to know and be able to do as teachers during the 123 course of interest, and what they ideally will take with them post-professional development. Once broad semester-long objectives are articulated, narrow the focus towards determining weeldy objectives. These objectives are influenced by laboratory content, course deadlines, and development of the TAs over time. For example, objectives for the first professional development meeting of the semester are likely influenced by (1) the fact that many TAs have never taught before (or, have never taught before in this particular course), (2) the expectations of the laboratory activity, and (3) the needs (i.e., knowledge and skills) of the TAs so they are able to teach their first laboratory section. A.6.2 Step 2: Devise an Assessment Plan Assessing whether or not TA professional development is effective is a critical component in designing professional development Ultimately, the goals and objectives will influence the type of assessment plan that will provide acceptable evidence for whether TAs achieved the desired professional development goals. There are many ways to evaluate effectiveness of professional development, yet there is not one agreed-upon standard (Desimone 2009). Effectiveness deals solely with the short-term outcomes of professional development asking whether the goals were achieved and to what degree (Boulmetis and Dutwin 2000). For example, at the end of the professional development, did TA classroom practices reflect constructivism and cooperative learning? There may be interest in long-term effects, or influence, of professional development depending on the program’s goals and objectives. For example, do TAS continue to teach in future courses using active 124 learning strategies after they have completed their professional development? The type of assessment, either effectiveness or impact, will provide help in selecting an evaluation model. A.6.2.1 Kirkpatrick’s Evaluation Framework One framework for evaluating effectiveness of professional development is Kirkpatrick's Four Levels (1976, 1994). Kirkpatrick's model was originally developed for evaluating professional development in the business world yet it offers a framework for thinking about evaluating professional development for TAs with only a slight change of focus in the highest tier. In the business model, the fourth level focuses on the bottom line, that is, the financial gains to the business post-professional development In the case of TA professional development, the bottom line often is change in student learning. In practice, professional development for teachers is generally evaluated through surveys given to participants at the completion of professional development (Gibbs and Coffey 2004, Pfund et al. 2009). These surveys assess how participants liked the professional development and may ask them questions about what they learned or hope to apply from their professional development According to Kirkpatrick (1994), simply surveying the participants only provides data on the most basic levels of effectiveness. By administering an end-of- workshop survey, one captures both participant reactions (i.e., what did participants like/dislike), and possibly aspects of learning (i.e., what did they learn after having completed this professional development? What is the evidence that demonstrates learning?) (Baumgartner 2007). However, these surveys do little to 125 address the higher levels of effectiveness, such as application (i.e., did participants implement new strategies) and impact (i.e., did the training make a difference on student learning?) In designing an evaluation plan for a professional development program, it is beneficial to consider how to assess effectiveness at each level of the evaluation plan (e.g., Kirkpatrick’s 4 Levels). To do so, ask, what types of data will inform the effectiveness at each level? How will these data shed light on whether or not TAs achieved the stated goals and objectives? A.6.3 Step 3: Design TA Professional DevelopmentActivities The final step, after determining goals and objectives and an assessment plan, is to design and implement TA professional development. The particular instructional design that fits the stated goals and objectives of the professional development is also influenced by the time, money, space and people involVed. For example, is there an opportunity to work with TAs before the semester begins? Will the TAs meet weekly? How much total time will they have on task? What kind of budget is available for professional development meetings? Who can provide pedagogical training to TAS? Where will the group meet? Is the meeting space flexible - is the furniture movable? Who are the TAs? What knowledge do they bring to TA professional development? What do they want from professional development? These questions provide just a few examples of the types of variables that influence the design of TA professional development In addition to considering these practical variables of TA professional development, also consider the learning theories that underlie effective instruction 126 that improves student learning (Michael 2006). The same theories also underlie the design and implementation of TA professional development. Given that teachers tend to teach as they are taught (see Lortie 1975), professional development needs to be constructed in such a manner that it models constructivism and cooperative learning. Applying the research-supported theories of constructivism and cooperative learning to professional development activities has the potential to positively influence TAs to teach as they are taught. A.7 : f i l e 'Bi (us will: 0 1. Partner with leaders & TAs in the reform. \ f ‘ . ° Learner- 2. Apply scientific 1‘ Level ,1“ TA Centered teaching 8. backward Reactions . design to design ~---~; ““ > . Active learning objectives, 2‘ Level ,2 TA L‘. g implement learner- Leamlng { g . TAS redesign centered pedagogy & 2 [abs evaluate student items}: B'JA IE; learning. I pp ca Ion °- . Moderated Z fishbowl 3. Reflect on their 4' :.evel 4' '3 Ieaming, 8t the mpad ) 0 Personal learning of the : reflection students through 7 L J discussion, reflection \ & feedback. J Figure A.2. Reforming TA professional development in Biol. Our reformed professional development model has assessments that are aligned with goals. The activities taking place during professional development meetings help TAs to achieve these goals. 127 My colleagues and I designed a reformed model for TA professional development in Biol that applies the theoretical framework articulated above (Figure A.2). Below, I describe each of the steps in our model. A.7.l Step 1: Articulate/Determine Goals Biol has a history of offering professional development for approximately 15 TAs per semester. This professional development, however, has not met the TAs needs. TA feedback from surveys taken during traditional professional development capture the dissatisfaction TAs expressed towards this approach to TA professional development. Comments include, "I leave prep meetings feeling: 0 overwhelmed, especially when we were doing something that I wasn’t particularly familiar with, o tired, bored and wanting to go home, 0 concerned about teaching new material, 0 mostly prepared, but I'd like more teaching tips, and o I didn't learn anything new, it's interesting to share my experiences with others, but I wish I could learn something new.” These comments support the research documenting a lack of satisfaction with current professional development opportunities available to TAs (e.g., Baviskar and Beardsley 2006, Luft et al. 2004). Comments like these provided a portion of the motivation for reforming the way TAs are prepared to teach Biol. In addition, Biol was undergoing a reform to become active and learner- centered. A critical component to the success of the reform was having TAs begin to 128 teach in learner-centered approaches. Therefore, we collaborated to begin to develop a reformed model of TA professional development Given these specifics (i.e., TAs not feeling prepared and the broader course reform) for our TA population and the Biol course, we converged on three over- arching goals for our TA professional development program. Specifically, TAs will: 0 Partner with leaders and TAs in the reform of Biol; 0 Apply principles of scientific teaching and backward design to develop learning objectives, implement learner-centered pedagogy and evaluate student learning; and, o Reflect on their Ieaming and the learning of the students through discussion, reflection and feedback. In addition to our program-level goals, we specified objectives for each weekly preparation meeting that were directly related to the stage of TA development and the lab content One overarching goal that remained constant from week to week was “each TA will leave the Friday prep meeting ready and confident with the content they are teaching and how they will teach it.” Specifically, objectives from our very first prep meeting of the semester (Spring 2009) included: 0 Apply and practice pedagogy for establishing cooperative groups; 0 Develop networks for support (among TAS, undergraduates and meeting leaders); 0 Apply and practice pedagogy for establishing cooperative groups; 0 Practice the jigsaw activity for laboratory safety; 129 0 Practice pedagogies for stimulating student discussion about variation; and o Gain confidence and clarity in assessing student learning. A. 7.2 Step 2: Devise Assessment Plan Consistent with Kirkpatrick's assessment model (Kirkpatrick 1994), we decided to assess the overall effectiveness of professional development in ways that aligned with each of Kirkpatrick’s four levels: A. 7.2.1 Level 1: TA Reactions We assessed TA reaction to their weeldy prep meetings through a preparation effectiveness survey (Appendix B). This survey was developed to assess the degree to which we achieved the goals and objectives set forth for the prep meetings. On the survey TAs place an “x" along a continuum from agree to disagree. We quantified the location of their “x" to obtain their reaction to their professional development For example, one statement reads: The lab meetings helped me to gain confidence in teaching the next weeks' lab exercise. This directly measures TA response to one of our weekly goals, “TAs feel confident" about the content and how to teach the upcoming lab. A.7.2.2 Level 2: TA Learning We evaluated TA learning by measuring how TAS used their training on Bloom's Taxonomy for Educational Objectives (Bloom and Krathwohl 1956) in the context of assessments. We assessed their learning by measuring the learning objectives and assessments created by TAs after instruction. Through evaluating 130 these objectives and assessments we gain insight into (1) the levels of cognitive processing TAs are expecting their students to achieve (objectives), (2) the levels of cognitive processing at which TAs assess their students (e.g., assessment questions), and (3) the alignment between cognitive levels of objectives and assessments. In our evaluation, two qualified independent raters with established inter- rater reliability rated the level of cognitive complexity of each item and objective using Bloom’s Taxonomy (Bloom and Krathwohl 1956). The scores for each item were averaged to obtain a single rating for each item within an assessment. We expected that after receiving professional development about Bloom's Taxonomy and assessment of student learning that Biol TAs would develop objectives and assessments that were aligned. That is, if they say they want their students to demonstrate an ability to analyze data (level 4), do they assess their students at the same level? We compared mean Bloom's level between objectives and assessments to indicate the degree of alignment A. 7.2.3 Level 3: TA Application To assess how TAs applied their professional development, we observed TA classroom practice to determine the degree of learner-centered teaching done by the TAs. To gather these data, we videotaped each TA twice during their semester of professional development. Videotapes were evaluated using the Reformed Teaching Observation Protocol (RTOP), which quantifies the degree of learner- centered classroom practice on a scale of 0-100 (Sawada et al. 2000). Scores closer to 100 represent a more reformed, learner-centered classroom. RTOP is a valid and reliable instrument (Sawada 1999) that measures the following as subscales of 131 classroom practice: lesson design and implementation, content knowledge, communication, and student/teacher relationships. Two independent raters were trained and calibrated using RTOP training videos (available online at: W11), and then rated each videotape. We compared RTOP scores over time and by TA to determine the application of our professional development on TA classroom practice. A. 7.2.4 Level 4: Impact Ultimately, the desired impact of changing professional development is to see an improvement in student learning as a result of changed teaching practices. However, it is beyond the scope of this dissertation to evaluate impact of professional development on student learning in Biol. Evaluating a change in student learning in this context is confounded by many factors. First, students are co-enrolled in the lecture and the lab at the same time. Second, students are participating in reformed practices in both the lecture and lab, and this impacts our ability to tease apart the impacts of the lab reform from that of the reform happening in the lecture. Third, students in the courses arrive with different levels of biological knowledge and scientific thinking, which require special consideration when tracking student growth. Relating student progress to TA professional development would require careful experimental design to minimize and control for these confounding variables. Therefore, in our assessment plan, we saw the application of the learner-centered teaching practices (i.e., Level 3 in Figure 3.2) as the first step in improving student learning. 132 One area through which we can assess impact is by keeping abreast with what the TAs in Biol do with their knowledge and experiences after they have participated in our reform. To date, one TA has a feature in Frontiers in Ecology and the Environment, and another two are working to submit the laboratory materials they created to an online peer-reviewed library. A. 7.3 Step 3: Designing Professional Development The final step in our professional development process is to design prep meetings that enable TAs to achieve our stated goals and objectives. Below, I (1) provide an overview of our semester in the form of meeting agendas, (2) describe the materials we used to train the TAs and then (3) describe the pedagogical approaches we applied in our professional development meetings. A. 7.3.1 Semester Outline A. 7.3.1.1 Pre-Semester Training The pre-semester training for reformed professional development consisted of a 2-day workshop for TAs. The first day began with the TAs thinking and reflecting on these questions: (1) What do you expect to gain from the workshop? (2) What are your major teaching challenges? (3) What challenges do your students face in learning? (4) How did you learn science? (5) How do your students learn science? After the TAS responded to these questions, they received feedback from professional development leaders. The leaders proceeded to model the very lesson design we planned to have the TAS use in their classroom instruction. 133 #1 The workshop was designed to model a Ieaming cycle (Ebert-May et al. 1997, Lawson et al. 1989); specifically, the 5 E's (Engage, Explore, Explain, Elaborate, Evaluate), which we adapted for our purposes. Engagement - To engage the TAs, we asked the TAs to respond to a series of 5 statements. One statement included, “Active learning strategies enable students to learn science better than passive lectures or labs." TAs responded by placing a number on a post-it note from 1 (strongly agree) to 5 (strongly disagree). When TAs completed their response, they placed their post-it note with their number on to a large graph to create a histogram of the their responses. Explain - Following the creation of these graphs, the professional development leader walked through each response and explained through a mini- lecture and discussion what research says about each of these statements. Following with the active learning example, we specifically discussed “How People Learn" (Brandsford et al. 2004) and determined what constitutes an active vs. passive classroom. By discussing these questions, TAs learned about Scientific Teaching (Handelsman et al. 2004) and how teaching scientifically mirrors the authentic practice of science, which is collaborative and inquiry-driven. During the engagement activity, TAs defined critical thinking and experienced working in collaborative groups. Specifically, the workshop was designed to practice cooperative learning strategies. TAs were seated in groups of 4 and during this phase (“explain") of the workshop, the leader first posed a question to the groups and gave the groups 2-4 minutes to talk about the question. The protocol was that the leader would 134 randomly call on a group to report what they discussed. This collaborative group experience was designed to mimic what TAs were expected to do in their classrooms. Explore - By this point, the TAs’ interest in changing and improving undergraduate education in Biol was evident. TAs were provided with an opportunity to think about and try designing instruction. We posed a question to the groups, “How do we go about developing a lab? How would you start? What would you do?” TAs discussed these questions in groups, then groups reported their feedback. We linked their responses to the Backward Design process (Wiggins and McTighe 1998), and then began to step through each phase of the design process. The TAs' first task was to write the learning objectives for a particular lab activity. We assigned each collaborative group a lab, and the TAs worked together to articulate lab objectives stating what students will know and be able to do. When all groups had at least one objective written, each group shared their objectives. Most TAs’ objectives at this point began with, “students will learn...”, which led us into a discussion on assessment - that is, how do we know whether or not they “learned"? What evidence will we accept to show they have the desired knowledge and skills we stated in the objectives? TAs refined their learning objectives after a small bit of instruction focused on what “learning" means. This discussion focused on Bloom's Taxonomy for Educational Objectives (Bloom and Krathwohl 1956). TAs applied Bloom's Taxonomy to their learning objectives and then determined how they would assess their students at similar cognitive processing levels. If their objective was that 135 students synthesize concepts, what data would show students could synthesize information? TAs spent the remaining half of the first day, and the majority of the second day developing laboratory objectives in collaborative groups. Expand - The second day, TAs continued to expand on their Ieaming by sharing and refining laboratory objectives for three modules for the Biol labs: genetics, evolution and ecology. In addition, there was a short instructional component on developing and using rubrics. This included processing three questions: 1. How do we make our expectations for assignments clear? 2. How do we provide constructive feedback? 3. How do we assess fairly and consistently across sections of Biol? TAs first discussed these questions in groups. Then each group was assigned a component of the rubric to develop. Groups worked independently to articulate assignment expectations and assessment criteria on their portion of the rubric. Then, each group brought their piece back to the large group to form the rubric as a whole (this activity is a jigsaw). Together, TAs refined and clarified sections of the rubric for Biol students. Evaluate - To evaluate the first professional development meeting, we returned to our original goals (see A .7 .1) and created a survey for TAs to complete based on these goals. We desired to know the degree to which we achieved our stated goals. At the completion of the two-day workshop, TAS left with experience using cooperative groups, practice writing and refining learning objectives, experience creating assessments and the beginnings of rubrics to use in Biol. 136 A.7.3.1.2 Weekly Meetings The second component to the reformed professional development model included weekly, 3-hour preparation meetings. These meetings had one overarching objective: TAs will apply, practice, reflect on pedagogies and practice cooperative Ieaming techniques during the process. How this looked in practice varied from week to week based on the content and nature of the lab design, however, I will provide a description of the first preparation meeting below which served as a template for subsequent meetings. The goals for the first meeting were that TAs apply and practice pedagogies to: (1) establish cooperative groups, (2) implement the jigsaw activity, (3) facilitate student discussion about variation and (4) assess student learning. Each meeting began with TAs receiving an agenda that outlined the major objectives, the flow of the meeting and administrative announcements. First, in groups TAs reflected about what they learned and remembered from their two-day pre-semester workshop. These collaborative discussions helped develop a network among TAs and their undergraduate assistants (EAs) as they reflected on their learning, what worked and in what areas they needed further clarification. The leader randomly selected a group to share the highlights from their discussion with the rest of the class, and in this way, they modeled group pedagogy with the TAs. Second, TAs received a lesson plan (see Appendix C) to aid them in their instruction. This lesson plan included learning objectives, materials they needed for lab, reminders for email communications with their students, potential challenging areas of the lab for their students (derived from TA comments and feedback), and a 137 lesson design. TAS used this lesson plan as a template and were able to follow along with the lesson plan throughout the preparation meeting. The majority of the preparation meeting was devoted to the lesson — the experience of the lesson, discussions of teaching approaches, and practice. Each meeting concluded with an opportunity for TAs to have discussions with their peers and EAs. Through these interactions, TAs asked questions, clarified teaching approaches, and engaged their EAs in their role in the classroom. What emerged from these discussions was a networked community - a community that collaborated together for the purposes of teaching. TAs felt comfortable emailing one another outside of lab, posting materials on our common course management software page and offering support to one another in terms of their teaching (and often other areas of graduate school). A. 7.3.1.2 Final Meeting The final meeting from the semester involved TAs turning in grades, reflecting on the final presentations from the last week of lab, and providing feedback to Biol about their professional development in the form of surveys and discussions. A. 7.3.2 Materials for TAs A. 7.3.2.1 Pre-Semester Workshop TAs received a binder at their pre-semester workshop which included logistical information (i.e., agenda, teaching assignments for the semester, TA contact information, etc.). In addition, TAs were expected to read three reading selections prior to training: (1) Chapter 1 of “Scientific Teaching” (Handelsman et al. 138 \. 2007), (2) Chapter 1 from “How People Learn”(Bransford et al. 2000) and (3) Essay on Scientific Teaching from Science (Handelsman et al. 2004). TAs also received a copy of Bloom’s Taxonomy table (from Bloom and Krathwohl 1956). A. 7.3.2.2 Weekly Meetings At each of the weekly preparation meetings, TAs were provided with: (1) copies of the lab, (2) lesson plans, and (3) agendas. In the event that we focused on grading, TAs were provided with de-identified student work (a sample of 5) and copies of the rubric for practice. A. 7.3.2.3 Final Meeting TAs took online surveys at the final meeting about their experience in TA professional development (see “Final Survey" in Appendix B). A. 7.3.3 Pedagogical Approaches A. 7.3.3.1 Pre-Semester Training The key to the pre-semester training was to give TAs an opportunity to experience active learning. Thus, this workshop was not a lecture, rather, TAs were the “students" in the active learning classroom. TAs did a lot of things at the workshop, which gave them a chance to apply their training right away, get feedback and refine their approaches. By counting off by numbers, we placed TAs into “random" groups at the beginning of their workshop. They used these groups as “base groups" to have discussions and process their reading and learning. Later, TAs re-assembled into task groups that were content-area specific, so, for example, those that were interested in re-designing lab materials about ecology were grouped together. 139 In addition to this active participation, there were built-in opportunities for reflection. Specifically, we asked: (1) What is your muddiest point? (2) What do you wish we would have done? These questions are adapted from Angelo and Cross (1993) A. 7.3.3.2 Weekly Meetings The pedagogical technique we applied with most success was a moderated fishbowl. In this approach, TAs observed a mock classroom session modeled by a leader (e.g., Kelsey) and moderated by a second leader (e.g., Michael). The “students" in the mock classroom were undergraduate educational assistants (EAs) who worked with TAs in Biol. During the fishbowl, the moderator’s job was to ask probing questions about lesson design and implementation to the TAS observing the lesson. Below, I provided a narration (in italics) of one preparation meeting fishbowl. The leader here is Kelsey, and she is interacting with a handful of students in a mock classroom setting introducing digital organism evolution with Avidians: [Leader] ”Who can describe for me what an Avidan is?” Kelsey, the leader, nods towards Amanda, an EA, sitting to my right about half way up the bench. Amanda answers, and Kelsey probes deeper, [Leader] “What do you mean it competes for resources? Can you tell me what you mean?" As the dialog unfolds between Kelsey and Amanda, the TAs scribble down notes about this discourse. Another contributor, Jack pipes in an answer, [Student 1] “I heard they’re kind of like bacteria” he responds to Kelsey’ 5 original question about the Avidians. Kelsey takes the same approach with Jack, 140 [Leader] [Student 1] [Leader] [Student 2] [Leader] [Student 2] [Leader] [Leader] [Student 3] [Leader] “How are they like bacteria? These organisms are digital... how can they be like bacteria?" The MS sitting to my right continue to write notes down on their yellow sheets of paper. lack responds, “Well, they’re asexual.” Kelsey asks, “What does that mean?” Another person, this time in the back of the room, to my left, answers, “Asexual reproduction is like a mitotic division.” Kelsey follows up with another question, "Jim, how are bacteria then different from eukaryotes [as a side note to TAs Kelsey mentions they might need to define eukaryotes for their students]?” TAs continue to take notes. The same student in the back of the room, responds, “Well, bacteria clone themselves.” Kelsey asks another question, “How about their chromosomes? What do the chromosomes of eukaryotes look like compared to chromosomes of bacteria?” Kelsey moves to the open port of chalkboard, to the left of the overhead projector screen, and begins to draw a visual representation of eukaryotic chromosomes. As she does, she explains the arrangement of the chromosomes for eukaryotes, and then proceeds to draw a single loop of bacterial DNA. “What does this arrangement mean for expression of alleles in bacteria?” Kelsey asks the group of TAs. Kate, 0 student at the front of the room inquires, “So there’s no dominant/recessive in bacteria?" Kelsey interjects, “dominant and recessive alleles don’t come into play here; we’re dealing with the fact that there’s only 1 loop of DNA and so whatever allele bacteria have is expressed.” Kate nods in response and turns to her left to write something down on her paper. 141 Following the completion of the fishbowl, the moderator led the TAs in discussion. For example, Michael asked TAS, “What was the goal of asking for students to explain what they mean by ‘competes for resources'? Where was Kelsey hoping to guide students' thinking by doing so?" These questions prompted discussion among TAs and guided them to think about their teaching and anticipating student responses. In addition to using the moderated fishbowl as a way to learn about pedagogy, TAs were involved actively in the professional development meetings by completing the lab for the following week, developing rubrics used for assessment, and practicing pedagogies they wished to implement in the classroom. A8 Conclusions There is a growing consensus in the research literature of the need for TA professional development that better prepares TAs to teach than the models currently used in practice. While some new models exist in practice in biology programs across the country (e.g., McManus 2002), these models are context- specific and not easily generalized to other programs. In addition, many of these programs do not consider the evaluation of the effectiveness of professional development during the design phase. In this chapter, I described a new professional development model that is based on theories of constructivism (see von Glasersfeld 1989) and cooperative learning (see johnson and johnson 1994) and influenced by the population of TAs for which it is designed. 142 The new professional development model I present here is a unique contribution to the research literature because it offers a new, broadly applicable approach to TA professional development that contrasts preceding (traditional) models in three key ways (Table A.l) Table A.l. Points of comparison between professional development models (Part 2). Traditional professional development views TAs as students who receive information from the leader. In contrast, the reformed model views TAs as teachers and students who construct understanding. Point of Comparison Traditional Professional Reformed Professional Development Development Theories of Learning Behaviorism Constructivist model (transmission model) Role of the TA Students, consumers Teachers and students, learners Role of the Leader Source of knowledge Facilitator First, traditional and reformed professional development operated under distinct models of how people learn. Traditionally, TA professional development embraces the transmission model of instruction (Smith et al. 2005), where TAs are passive observers of a lecture about content they are expected to convey to their students. In contrast, reformed professional development views Ieaming from a constructivist lens (see van Glasersfeld 1989), recognizing that TAs build and construct knowledge just as students do - through interaction and involvement with the material and one another. Therefore, these two contrasting models of professional development look very different in practice. TAs in traditional professional development observed and listen to a lecture, then try out the lab to make sure they know how to use the 143 equipment. In reformed professional development, TAs contributed to the discussion about teaching and Ieaming through active participation. TAs engage in discussion with their peers, and put into practice what they learn by developing learning objectives, assessments and practicing teaching pedagogies in groups. TAs have conversations about laboratory activities, how they might guide. students' thinking, where their students might confront challenges, and what pedagogies would enable them to help their students. Second, traditional and reformed professional development differed in the role assigned to the TA. In traditional professional development, TAs were students and consumers of the information that is provided to them in the professional development Reformed professional development acknowledges that TAs play dual roles as learners and as teachers, and the design of the professional development reflects this consideration of TA roles. Third, traditional and reformed professional development had distinct views of the role of the professional development leader. In traditional professional development, the leader was the source of knowledge. The leader had authority about what is taught and how the content is conveyed, and there was little room for deviation. In reformed professional development, leaders assumed the role of a guide. While they were still viewed as a source of information, the leaders did not exercise their role by imparting content to the TAs, rather, they allowed the TAs to construct their understanding through interactions and provided their input and guidance along the way. 144 Throughout the design and implementation process, we were reminded of the delicate balance required when working with TAs. On the one hand, we need instructional strategies that help the TAs achieve the goals of professional development (see step 1). On the other, TAs need a voice in guiding the direction and flow of professional development A key strategy to achieving this, we found, is iterative cycles of trying something new and responding to TA feedback and trying again. In our case, it took roughly one quarter of the second semester of reformed professional development to converge on the moderated fishbowl approach. We tried other pedagogical approaches, like describing and walking through the lesson plans, allowing TAs to lead the discussions about the labs, and treating everyone like students in the class. However, the feedback we consistently received from the TAs was that they did not “get it”. The TAs were uncertain of what it looked like to teach in learner-centered ways since many of them had never seen this in practice. We tried describing and practicing the desired learner-centered pedagogies, but the TAs struggled to gain a mental image of what this type of reformed teaching looked like in practice. Therefore, we tried the moderated fishbowl which gave TAS an opportunity to see what the pedagogy looked like in practice and to see how an instructor responded to student questions. When TAs saw what it was they were expected to do, it made sense for them. We discovered that actually teaching TAs in the very same manner we hope they will teach was an effective strategy for these TAs. Thus, we changed 145 the negative connotation associated with the phrase “teach as you are taught” to a positive one - and we hope that TAS will teach as they were taught to teach! 146 Appendix B This appendix provides copies of the surveys developed and administered as part of this research. In order, the surveys are: (1) TA Background Survey, (2) Views about Teaching, Learning and Research Survey, (3) Final Survey on Teaching and Learning, and (4) TA Preparation Effectiveness Survey. TA Background Survey As part of a project to improve Biol, we would appreciate your time to complete this survey. Thank you for your participation. 1. Name: 2. Department: 3. Current status: a. Masters year in program b. PhD year in program , are you a PhD candidate? 4. Gender: M __ F __ 5. Year of birth: _ 6. Describe your undergraduate experience: a. Degree granting school: b. Date degree was received: c. Undergraduate major: 7. I chose to enter the science field because... 8. Have you ever taken a course that focused on teaching/education? _ If so, how many courses have you taken? _ 9. Have you ever participated in a seminar or workshop focused on teaching/ education? _ If yes, please list the seminars/workshops. 10. How many semesters have you taught Biol? 11. How many semesters have you been teaching at this institution? 12. a. How many semesters of teaching experience do you have outside of this institution? b. Circle any/all applicable levels of teaching: -Elementary -Secondary -Undergraduate -Other: 13. What is the relationship, if any, between research and teaching? 14. What are your career goals? Does teaching fit into your goals? 15. Are you interested in gaining knowledge about how to teach and how students learn? Why or why not? 16. What do you hope to gain from your Biol TA experience? 147 Views about Teaching, Learning and Research Survey Please take a few minutes to indicate your responses to the questions below. Thank you for your feedback. 1. The qualities of an effective teacher are... 2. My teaching is strongly influenced by... 3. Students learn science best by... 4. I learn science best by... 5. What types of evidence provide feedback about your students' learning? 6. The qualities of an effective researcher are... 7. My research is strongly influenced by... Final Survey on Teaching and Learning 1) In your view, what is the role of the learner in the classroom? 2) In your view, what is your role as a teacher? 3) In your view, what is the purpose of teaching? 4) In your view, what does it mean to “know" biology? 5) Have your beliefs about teaching and student learning changed over the course of the semester? If so, how? If not, why? 6) You have just been given the opportunity to teach a course of your choosing, and the university is allowing you to pick your students. Please write a “WANT-AD" for your course, and the students you would like enrolled in your course below. 7) Now, imagine that you have the opportunity to interview and choose your professors for a course you will be taking. Please write a “WANT-AD" for faculty applicants. 8) Describe your model of an ideal classroom. Include details about the teacher and the students, as well as the classroom environment. 148 TA Preparation Effectiveness Survey Please take a few minutes to complete this questionnaire. 1. Estimate the proportion of time spent in the following areas during a weekly lab prep meeting: Activity or Action Percentage of Time (%) TAs are doing the next weeks’ lab activity TAs are practicing the recommended teaching methods Lab coordinator/leader models the desired teaching methods (not just talking about how to implement, but actually modeling it) Lab coordinator/leader talks about the desired teaching methods (i.e. “I recommend doing this here, or I would do this in this fashion..fl Administrational details Other (please explain) Total 100% 2. Describe how you use the provided TA Lesson Plans in preparing and or teaching the lab. 3. I usually leave the weekly prep meetings feeling... 4. In the future, I think TA prep meeting should... 149 Mons; For each question below, please indicate your answer by marking a location on the line that best describes your answer to the question. EQLexamrLle: Vanilla ice cream is the best flavor of ice cream. Agree ----- X Disagree 1. The Biol TA orientation meeting gave me an overview of my role as a Biol TA. Agree Disagree Please explain. 2. The weekly lab meetings helped prepare me to teach the next weeks’ laboratory exercise. Agree Disagree Please explain. 3. The lab meetings helped me to gain confidence in teaching the next weeks’ lab exercise. Agree Disagree Please explain. 4. The lab meetings helped me to improve my teaching skills. Agree Disagree Please explain. 5. The lab coordinators prepared me to deal with difficult aspects of the laboratory exercises. Agree Disagree Please explain. 150 6. The lab coordinators prepared me to know how to handle difficult situations with students that may arise in my classes. Agree Disagree Please explain. 7. The lesson plans helped me to gain confidence in teaching the labs. Agree Disagree Please explain. 8. Feedback from peers during lab prep meetings has increased my self- confidence as a teacher. Agree Disagree Please explain. 9. The “mock classroom" set up during prep meetings helped me to see how I could implement new pedagogies in my classroom. Agree Disagree Please explain. 10. I tried new teaching methods that were discussed in the prep meetings during the semester. Agree --- Disagree Please explain. 151 Appendix C - Coding Categories for Qualitative Data This appendix contains the coding categories and descriptions of categories used in this research. Questions from the surveys (see Appendix A) are bold face and categories are underlined. Examples of responses coded into each category are provided below each category. Teaching and Learning Survey QZ: Characteristics of an effective teacher are... Coding categories for characteristics of an effective instructor: E l' I . [I] -Affective traits including interest in teaching, cares about students, approachable, interested in students and their learning, etc. - Teacher is organized, prepared, punctual, logical, etc. mammal -Teachers know their content/science E' 'E .[3] -Teachers have an ability to pass on information to students (e.g., great lecturer) -Clearly communicate expectations -Teacher creates a positive classroom environment Holosthestudonfilfl -Acts as a helper to the students WEI -Perceptive to the needs /strengths/weaknesses of the student Engagingthostuoonrsfll -Successfully engages students, maybe through active learning -Motivating 152 Teaching and Learning Survey Q3: My teaching is strongly influenced by... Coding categories for influences on teaching: Boliofsfll -Beliefs about effective teaching and learning (and biases) and what’s important W -Experiences as a student (or in some cases as a teacher), both positive and negative Eonsonallntonostm -Interest in and enjoyment of teaching Iimofll -Constraints on TA's time WW -Courses/seminars taken on teaching -Books/literature read on teaching W -The students in the courses they are teaching Teaching and Learning Survey Q4: Students learn science best by Coding categories for students Ieaming science: Re ' ' r 1 -Students learn science by reading textbooks, papers, etc. W21 -Students learn science by listening to a lecture and taking notes, or seeing a demonstration 153 “1.1!!!!“ .[3] -Students learn science best through multiple means (e.g., each student learns differently); there is not one right way to learn science. N u -Students learn science by making connections among course content and real life. W51 -Students learn science though experiments, doing science, practicing the scientific method/ process, discussing it, and teaching it to others. Final Survey Q1: In your view, what is the role of the learner? Coding categories for role of the learner: I . . l l l [l] -A students’ role is to learn/ understand/ gain knowledge. lobotosnonsihleloatnorsfll -A student is expected to be a responsible learner; to do the work required of them, to be punctual and prepared, to participate, etc. I I C I O I . 'e'l.,'..l'.‘ .2!!!’ 11'1lel ' 0'Il' 'eIIO' -A student is expected to take an active role in their Ieaming by building connections/understanding, and engaging with the content/ material, etc. Final Survey Q2: In your View, what is the role of the teacher? Coding categories for role of the teacher: Mansmitknofllodgolll -A teacher needs to pass on knowledge of science to the next generation 154 Iosnnooctsoidontloamingfll -A teacher needs to engage students, create a successful learning environment, be fair, and be timely in grading. W -A teacher needs to provide support/help/guidance to students. Final Survey Q3: In your view, what is the purpose of teaching? Coding categories for purpose of teaching: Inansmitknondodgofll -Pass on the knowledge of science -lnspire or prepare/train future scientists I' [i .. ”.1 [3] -Educate non-scientists so they can be informed citizens 155 Coding schemes for additional survey questions Additional coding categories are provided here for questions not related to TA beliefs about teaching and student Ieaming. Coding formatting is identical to prior codes in this appendix. Demographic Survey Q10: I chose to enter the science field because... Coding categories for reasons for going in to science: Emotional -Reason for entering science is based on a love, enjoyment, passion or interest in science -Reason for entering science is based on TA “ownership” or value - that they can either help, make a difference, impact in some way, or contribute to knowledge / making things better. E lCl .. -Reason for entering science is based on personality traits TAs identify in themselves that they believe lend themselves towards a career in science (e.g., smart, curious, good at science). I 'l [S . -Reason for entering science is based on the inherent nature of science (e.g., discovery, interconnectedness, experimentation) Demographic Survey Q17: What is the relationship, if any, between research and teaching? Coding categories for reasons for relationship between research and teaching: I . . [I l l -Teaching provides a venue to communicate/transmit/pass on knowledge gained through research 156 Bl II" -Research provides “real-life” examples (i.e., context) for teaching students about science Il"'li .. ~Teaching can spark interest in pursuing science Il' .. -Teaching can pass on skills and knowledge of the scientific process to help foster scientists I l' . I'll . . -Teaching helps me understand/practice/ communicate science better B l l I 'II I l E l . -Increases my content knowledge -Increases my understanding of the scientific method/process, which helps me teach it II “1.1. -Not clear on relationship, or even if there is one or TA may never have taught before II B l . l . -No stated relationship between research and teaching Demographic Survey Q18: What are your career goals? Does teaching fit into your goals? Coding categories for career goals: NO YES Wang 157 Demographic Survey Q19: Are you interested in gaining knowledge about how to teach and how students learn? Why/Why not? Coding categories for interest in teaching/Ieaming: LLQ 1E5 WW -Personal interest in increasing teaching ability -More effective educator mmm -Teaching is a skill for which the TA would like to be trained in -Having this tool will help them meet their personal goals C .. r II' I 158 Appendix D - TA Lesson Plan for Cellular Reproduction Lab (Fall 2008) This appendix provides one example of the lesson plan provided to TAs during reformed professional development. For labs that TAs re-designed, TAs created these lesson plans, however, the one included here represents a lesson plan created by prep meeting leaders. Goal: The goal of this week’s lab is to allow students to practice the scientific method to investigate a question about cellular reproduction specific to mitosis. [ll . l' . As a result of participating in this laboratory activity, you will: 1. Develop hypotheses (null and alternative) 2. Design and conduct experiments 3. Analyze data using statistics 4. Draw conclusions based on gathered data 5. Use appropriate terms when discussing mitosis and integrate your knowledge of mitosis into the cell cycle Model the process of mitosis 7. Make stained slides to observe mitosis and see phases of mitosis 9‘ Materials: -Cellular Reproduction Lab -Course website -Grading sheet checklist for the pre lab (in our ANGEL TA group) -Rubric and checklist for guidebook entry ErioLtsLClass: Email students...about (l) Pre-Iab completed for cell reproduction lab — they may need to type in the URLs (the link may not work). Lossonflan: 1. As students arrive, continue established protocol: a. EAs can check off pre-lab using the grading sheet for the pre—lab 2. Announcements a. Announcements - Starting the “genetics" module (cell reproduction - mitosis and meiosis as a foundation for understanding where variation comes from, and how variation gets expressed in a population) b. Goals/ Objectives of Cellular Reproduction Lab i. Ask students what these are - and what they hope to learn from the lab 159 ii. Ask students what they know about mitosis (elicit prior knowledge) and the scientific method (as that is what they will do today) . Cell Reproduction Lab a. Common student preconceptions/difficulties with cell reproduction: i. All cells spend equal time in all parts of the cell cycle ii. Cell cycle IS only mitosis/meiosis (M- phase)- —all cells do is divide. iii. Confusion among concepts: sister chromatids, homologous pair, n, ploidy b. Engagement - Seeing mitosis (Part 1) — 15-20 minutes i. Students first use stained slides of mitosis to view cells at various stages ii. Remind students about microscope use tips, and to use appendix A as well as posters up in the lab to help identify which stage they are looking at iii. Students create their own stained slide using procedure in their guidebooks and identify the stage of any 5 cells they choose 1. Each pair of students creates their own slide c. Exploration - Testing hypotheses about cell cycle (Pt 2) -- 1 hour or so i. Ask students - what did you notice when looking at cells in the root tip? Were all 5 of your cells in the same stage? Did you have a hard time finding some of the stages? What might this indicate about the length of time cells spend in each part of the cell cycle? ii. After seeing cells and observing onion root tips, students now propose null and alternative hypotheses. 1. Students will likely have broad conceptions of hypotheses at this point, and you may wish to briefly discuss with your students the differences between a null/ alternative hypothesis, and the necessary components of a hypothesis. 2. For this lab, students may test either their null or alternative hypothesis (that is, some students will want to test the hypothesis that cells spend equal amounts of time in all the phases of the cell cycle - this is the null hypothesis... there is no difference between the amt. of time a cell spends in each phase) 160 iii. iv. vii. a. Testing the null hypothesis is fine, because it helps students in a real way gather data and test a preconception they may have. Translating hypotheses into predictions 1. In this case, we’re looking for a numerical prediction - how many cells would you expect to find in a given phase (out of 100 total) if the hypothesis you are testing is true. 2. Students will struggle with this - but let students use their groups to talk about this because they can figure it out. Developing a Method 1. Students in their groups of 4 decide on a method that their group will use to test their hypothesis. (hints: ask students to think about the variables they need to control, as well as validity and replication in their design) Students collect data for the hypothesis they are testing Students analyze their data using Chi-Square 1. Chi-Square will be new for your students 2. You might consider doing a class example of Chi-Square (following Appendix) with students as many of them reach this stage. 3. Students may choose to test their null or alternative in the Chi-Square... hence why the table says “reject your hypothesis” NOT “reject the null hypothesis if..." Students draw conclusions - you may, if time permits, choose to discuss conclusions as a class. (I. Explanation - Student Reflections Questions (Part 3) — 30 minutes i. ii. Students get a chance to express their understanding of cell reproduction focusing on the bigger picture of mitosis — that is, why is mitosis useful? What would happen if our cells didn't undergo mitosis? If they choose, students may use tablet PCs in the lab to complete their drawings for mitosis 1. Students should email these to themselves when complete and then include this in their post-lab write up. 161 2. EAs should help students get PCs if they are interested in using them (sign out and make sure that all PCs are returned) e. Expansion — Through a class or small group discussion - 15 minutes i. What are the limitations of mitosis? (“identical cells"...) ii. In what situations would having identical cell reproduction be undesirable? 1. Get at lack of variation... implications for disease outbreaks, etc. iii. Predict what might happen if a random mutation occurs in the DNA of the organism 1. Are there situations that you can think of where this may occur? (Cancer cells...) iv. Preview next lab - meiosis and Mendel (here, cell reproduction does not result in identical cells, but rather in different cells) 4. Clean up a. Make sure students have followed aseptic protocol (i.e. cleaned up their lab space, returned all equipment to where it belongs, returned microscope to starting position, and have bleached down their lab tables) 5. For next week: a. Finish post-lab for cell reproduction b. Complete pre-lab for Mendelian genetics 162 Appendix E - Interview Questions This appendix provides a copy of the interview protocol and questions I used to interview jack (see Chapter 5). Questions are divided into those about lack’s teaching, his preparation in two models of TA professional development and some synthesizing/reflection questions. Introduction -Introduce myself, the purpose of our interview, and why I’m videotaping -Thank TA for participating IEALHIALG 1) Can you describe and explain your all time favorite teacher? Why did this individual make such an impression on you? On your learning? What did this teacher teach you about your own learning? 2) What characteristics do you believe define good teaching? 3) Can you tell me something about the kind of teacher you see yourself as being? What would I see if I were to observe you teaching a typical lesson? EREEABAIIQN 4) Can you describe what a prep meeting is like from your perspective? 5) Can you explain how you use or don’t use the TA lesson plans provided in the prep meetings as you prepare your teaching? 6) After you leave the prep meeting, can you describe the process you go through to get ready to teach the upcoming lab? 7) What is most challenging to you as a TA? 8) What kinds of supports would help you improve your work as a TA? 9) Can you describe what a typical class meeting looks like for Kelsey’s course meeting? 10)Through attending these lectures, what has been most helpful to you as at teacher? W 11)Since you began teaching in 131, what factors (people, resources, experiences, etc) have influenced your teaching? Have they changed during your time in [31? Can you describe this change? (Or lack of change - that is, what would it take for you to change?) 12)ln what areas do you aim to grow in your teaching? What else would you like to learn about teaching or student learning that you wish IBl would address? Conclusion -Thank TA for his time; ask if he has any questions. 163 REFERENCES Abbott, RD., D.H. Wulff, and CK. Szego. 1989. Review of research on TA training. In: Nyquist, ].D., RD. Abbot, D.H. Wulff, and l. Sprague (Eds). Preparing the professoriate of tomorrow to teach: Selected readings in TA training. Dubuque, IA: Kendall/Hunt. p. 111-124. Adams, K.A. 2002. What colleges and universities want in new faculty. Preparing Future Faculty Occasional Paper Series. Washington, DC: Association of American Colleges and Universities. Alberts B. 2008. Considering science education. Science 319:1589. Alberts, B. 2010. Science education websites. Science 327:405. Allen, RR. and T. Rueter. 1990. Teaching assistant strategies: An introduction to college teaching. Dubuque, IA: Kendall/ Hunt. American Association for the Advancement of Science (AAAS). 2009. Vision and change in undergraduate biology: A call to action. Washington, DC: AAAS. Angelo, TA. and KP. Cross. 1993. Classroom assessment techniques: A handbook for college teachers. 2Ind Edition. San Francisco, CA: lossey-Bass Publishers. Anning, A. 1988. Teachers' theories about children's learning. In: Calderhead, 1. (Ed). Teachers' professional learning. New York: Falmer. p 128-145. Ball, D.L. and H. Bass. 2000. Interweaving content and pedagogy in teaching and Ieaming to teach: Knowing and using mathematics. In: Boaler 1 (Ed). Multiple perspectives on mathematics teaching and learning. Westport, CT: Ablex Publishing. p 83-104. Ballou, D. and M. Podgursky. 1999. Teacher training and Iicensure: A layman's guide. In: Kanstroom, M. and CE. Finn (Eds). Better teachers, better schools. Washington, DC: Thomas B. Fordham Foundation. Bandura, A. 1986. Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, N]: Prentice Hall. Barkastas-Tasos, A. and ]. Malone. 2005. A typology of mathematics teachers' beliefs about teaching and learning mathematics and instructional practices. Mathematics Education Research journal 17(2):69-90. Bates, R. 2004. A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence. Evaluation and Program Planning 27(3):341- 347. 164 Baumgartner, E. 2007. A professional development teaching course for science graduate students. journal of College Science Teaching 36(6):16-21. Baviskar, S. and P. Beardsley. 2006. Survey report on graduate teaching assistants at ISU. ISU. Beswick, K. 2007. Teachers' beliefs that matter in secondary mathematics classrooms. Educational Studies in Mathematics 65(1):95-120. Black, B, and M. Kaplan. 1998. Evaluating TAs’ teaching. In: Marincovich, M., j. Prostko, and F. Stout (Eds). The professional development of graduate teaching assistants. Bolton, MA: Anker Publishing Company. p 213-2 34. Bloom, 85. and DR. Krathwohl. 1956. Taxonomy of educational objectives: The classification of educational goals. New York: Longmans, Green. Bogdan, RC. and SK. Biklen. 1998. Qualitative research for education: An introduction to theory and methods. Boston: Allyn and Bacon. Borko, H. 2004. Professional development and teacher learning: Mapping the terrain. Educational Researcher 33(8):3-15. Borko, H. and RT. Putnam. 1996. Learning to teach. In: Berliner, DO and RC. Calfee (Eds). Handbook of educational psychology. New York: Macmillian Library Reference. p 673-708. Boulmetis, j. and P. Dutwin. 2000. The ABCs of evaluation: Timeless techniques for program and project managers. San Francisco: jossey-Bass. Boyer, E. 1991. Preparing tomorrow's professoriate. In: Nyquist, j.D., R.D. Abbott, D.H. Wulff, and j. Sprague (Eds). Preparing the professoriate of tomorrow to teach: Selected readings in TA training. Dubuque, IA: Kendall / Hunt. p 3-11. Boyer, E. 1998. Reinventing undergraduate education: A blueprint for America's research universities. Boyer Commission on Educating Undergraduates in the Research University. 302, N. and Y. Boz. 2008. A qualitative case study of prospective chemistry teachers' knowledge about instructional strategies: Introducing particulate theory. journal of Science Teacher Education 19(2):135-156. Brainard, j. 2007. The tough road to better science teaching: Proponents of new methods encounter resistance, especially at research universities. The Chronicle of Higher Education, Special Report 53(48). 165 Bransford, j.D, A.L. Brown, and RR. Cocking. (Eds). 2000. How people learn: Brain, mind, experience and school. Washington, D.C.: National Academy Press. Bramald, R, F. Hardman, and D. Leat. 1995. Initial teacher trainees and their views of teaching and Ieaming. Teacher and Teacher Education 11(1):2 1-32. Brown, CA. and H. Borko. 1992. Becoming a mathematics teacher. In: Grouws, D. (Ed). Handbook of research on mathematics teaching and learning. New York: Macmillan. p 209-239. Bruner, LS. 1966. The process of education. Cambridge: Harvard University Press. Cady, A.j., and K. Reardon. 2007. Pre-service teachers’ beliefs about knowledge, mathematics and science. School Science and Mathematics. 107:237-245. Calderhead, j. 1996. Teachers: Beliefs and knowledge. In: Berliner, D.C., and RC. Calfee (Eds). Handbook of educational psychology. New York: Macmillian Library Reference. p 709-725. Caris, N.L. and HF. Merchant. 1991. Teaching enhancement training program for graduate student instructors in biology. In: Nyquist, j.D., R.D. Abbott, D.H. Wulff, and j. Sprague (Eds). Preparing the professoriate of tomorrow to teach: Selected readings in TA training. Dubuque, IA: Kendall/ Hunt p 215- 222. Carnegie Foundation for the Advancement of Teaching. 2005. A classification of - institutions of higher education, 2005 Edition. Carroll, j.G. 1977. Assessing effectiveness of a training-program for university teaching assistants. Teaching of Psychology 4(3):135-138. Carroll, j.G. 1998. Effects of training programs for university teaching assistants - Review of empirical research. journal of Higher Education 51(2):167-183. Chism, N.V. 1998. Preparing graduate students to teach: Past, present and future. In: Marincovich, M., j. Prostko, and F. Stout (Eds). The professional development of graduate teaching assistants. Bolton, MA: Anker Publishing Company. p.1-18. Collier, GP. 1972. Prospective elementary teachers’ intensity and ambivalence of beliefs about mathematics and mathematics instruction. journal for Research in Mathematics Education 3:155-163. DeFord, D. 1985. Validating the construct of theoretical orientation in reading instruction. Reading Research Quarterly 20:351-367. 166 DeFranco T.C., McGivney-Burelle j. 2001. The beliefs and instructional practices of mathematics teaching assistants participating in a mathematics pedagogy course. Proceedings of the Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education. 23rd meeting. DeHaan, R.L. 2004. Issues in undergraduate science education: A national perspective. Developmental Biology 271(2):553-553. Desimone, L.M., A.C. Porter, M.S. Garet, K.S. Yoon, and B.F.Birman. 2002. Effects of professional development on teachers' instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis 24(2):81-112. Desimone, L.M. 2009. Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher 38(3):181-199. Dewey, l. 1916. Democracy and education: An introduction to the philosophy of education. New York: The Macmillan Company. Druger, M. 1997. Preparing the next generation of college science teaching: Offering pedagogical training to graduate teaching assistants as part of the college reform agenda. journal of College Science Teaching 26(6):424-427. Ebert-May, D., T. Derting, j.L. Momsen, and TM. Long. Submitted. The impact of professional development on undergraduate biology teaching. Ebert-May, D., C. Brewer, and S. Allred. 1997. Innovation in large lectures-teaching for active Ieaming. BioScience 47:601-607. Elliott, R., C.T. Fischer, D.L. Rennie. 1999. Evolving guidelines for publication of qualitative research studies in psychology and related fields. British journal of Clinical Psychology 38(3):215-229. Etkina, E. 1999. How to help post grads to teach. Physics World 12(8):15-16. Etkina, E. 2000. Helping graduate assistants teach physics: Problems and solutions. journal of Graduate Teaching Assistant Development 7(3):123-137. Fajet, W., M. Bello, S.A. Leftwich, j.L. Mesler, and AN. Shaver. 2005. Pre-service teachers' perceptions in beginning education classes. Teaching and Teacher Education 21(6):717-727. Fink, LB. 1977. Developing temporary faculty: The challenge posed by teaching assistants. Educational Horizons 55:56. 167 Fiszer, E.P. 2004. How teachers learn best: An ongoing professional development model. Lanham, MD: Scarecrow Education. Freeman, 8., E. O’Connor, j.W. Parks, M. Cunningham, D. Hurley, D. Haak, C. Dirks, and MP. Wenderoth. 2007. Prescribed active learning increases performance in introductory biology. CBE - Life Science Education 6:132-139. French, D. and C. Russell. 2002. Do graduate teaching assistants benefit from teaching inquiry-based laboratories? Bioscience 52(11):1036-1041. Gerber, A. and V. Hoffmann. 1998. Diffusion of eco-farming in Germany. In: Roling, N.G. and MAE. Wagemakers (Eds). Facilitating sustainable agriculture: Participatory Ieaming and adaptive management in time of environmental uncertainty. New York: Cambridge University Press. Gibbs, G. and M. Coffey. 2004. The impact of training university teachers on their teaching skills, their approach to teaching, and the approach to learning of their students. Active Learning in Higher Education 5(1):87-100. Gray, P.L., and N.L. Buerkel-Rothfuss. 1991. Teaching assistant training: The view from the trenches. In: Nyquist, j.D., R.D. Abbott, D.H. Wulff, and j. Sprague (Eds). Preparing the professoriate of tomorrow to teach: Selected readings in TA training. Dubuque, IA Kendall / Hunt. p 40-51. Green, T. 1971. The activities of teaching. New York: McGraw-Hill. Guskey, TR. 1986. Staff development and the process of teacher change. Educational Researcher 15(5):5-12. Guskey, TR. 2002. Does it make a difference? Evaluating professional development. Educational Leadership 59:45-51. Hake, RR. 1998. Interactive-engagement verses traditional methods: A six- thousand-student survey of mechanics test data for introductory physics courses. American journal of Physics 66(64-74). Hammrich, PL 2001. Preparing graduate teaching assistants to assist biology faculty. journal of Science Teacher Education 12:67-82. Handelsman, j., D. Ebert-May, R. Beichner, P. Bruns, A. Chang, R. DeHaan, j. Gentile, S. Lauffer, j. Stewart, S.M. Tilghman and others. 2004. Scientific teaching. Science 304(5670):521-522. Handelsman, j., S. Miller and C. Pfund. Scientific Teaching. 2007. New York, NY: W.H. Freeman and Company. 168 Haney, H. and j. McArthur. 2002. Four case studies of prospective science teachers' beliefs concerning constructivist teaching practices. Science Education 86(6):783-802. Hiiemae, K., L. Lambert, and D. Hayes. 1991. How to establish and run a comprehensive teaching assistant training program. In: Nyquist j.D., R.D. Abbott, D.H. Wulff, and j. Sprague (Eds). Preparing the professoriate of tomorrow to teach: Selected readings in TA training. Dubuque, IA: Kendall/Hunt p 123-134. Hollingsworth, S. 1989. Prior beliefs and cognitive change in learning to teach. American Educational Research journal 26(2):l60-189. Holton III, ER 1996. The flawed four-level evaluation model. Human Resource Development Quarterly 7(1):5-21. Holt-Reynolds, D. 1992. Personal history-based beliefs as relevant prior knowledge in course work. American Educational Research journal 29(2):325-349. johnson, D.W. 1984. Circles of learning: Cooperation in the classroom. Alexandria, Va.: Association for Supervision and Curriculum Development johnson, D.W. and RT. johnson. 1994. An overview of cooperative learning. In: Thousand, j., A. Villa, and A Nevin (Eds). Creativity and Collaborative Learning. Baltimore: Brookes Press. johnson, D.W., R.T. johnson, and MB. Stanne. 2000. Cooperative learning methods: A meta-analysis. Tablelearning.com. Kane, R., S. Sandretto, and C. Heath. 2002. Telling half the story: A critical review of research on the teaching beliefs and practices of university academics. Review of Educational Research 72(2):177-228. Kember, D. and K. Kwan. 2000. Lecturers' approaches to teaching and their relationship to conceptions of good teaching. Instructional Science 28:469- 490. Kettle, B. and N. Sellars. 1996. The development of student teachers' practical theory of teaching. Teaching and Teacher education 12(1):1-24. Kirkpatrick, D.L. 1976. Evaluation of training. In: Craig, RL (Ed). Training and development handbook. New York: McGraw-Hill. Kirkpatrick, D.L. 1994. Evaluating training programs: The four levels. San Francisco: Berrett-Koehler. 169 Latulippe, CL. 2007. Environments that encourage mathematics graduate teaching assistants: The effects of institution type and availability of training. Bozeman: Montana State University. Lawrence, F., P. Heller, R. Keith, and K. Heller. 1992. Training the teaching assistant. journal of College Science Teaching Nov.:106-109. Lawson, A.E. 2002. Reforming and evaluating college science and mathematics instruction: Reformed teaching improves student achievement journal of College Science Teaching 31:388—393. Lawson, A.E., M.R. Abraham, and j. Renner. 1989. A theory of instruction: Using the learning cycle to teach science concepts and thinking skills. Monographl. Cincinnati, OH: National Association for Research in Science Teaching. Lortie, DC. 1975. Schoolteacher: A sociological study of teaching. Chicago: University of Chicago Press. Loucks-Horsley, S. KE. Stiles, S. Mundry, N. Love, and P.W. Hewson. 2009. Designing professional development for teachers of science and mathematics. Thousand Oaks: Cowrin. Luft, j.A., j.P. Kurdziel, G.H. Roehrig, and j. Turner. 2004. Growing a garden without water: Graduate teaching assistants in introductory science laboratories at a doctoral / research university. journal of Research in Science Teaching 41(3):211-233. Lumsden, AS. 1993. Training graduate students to teach. American Biology Teacher 55(4):233-236. Maclsaac, D. and K. Falconer. 2002. Reforming physics instruction via RTOP. The Physics Teacher 40(8):479-485. Marder, E. 1997. The laws of choice: Predicting customer behavior. Free Press: New York. Marincovich, M. j. Prostko, and F. Stout (Eds). 1998. The professional development of graduate teaching assistants. Bolton, MA: Anker Publishing Company. McManus, DA. 2002. Developing a teaching assistant preparation program in the School of Oceanography, University of Washington. journal of Geoscience Education 50(2):158-168. Michael, 1. 2006. Where's the evidence that active learning works? Advanced Physiological Education 30:159-167. 170 Miller, S., C. Pfund, C.M. Pribbenow, and j. Handelsman. 2008. The pipeline: Scientific teaching in practice. Science 322(5906):1329-1330. National Research Council (NRC). 1999. Transforming undergraduate education in science, mathematics engineering and technology. Washington DC: National Academy Press. National Research Council (NRC). 2003. Improving undergraduate education in science, mathematics, engineering and technology. Washington DC: National Academy Press. Nespor, l. 1987. The role of beliefs in the practice of teaching. journal of Curriculum Studies. 19:317-328. Nyquist, j.D. R.D. Abbott, and DH. Wulff. 1989. The challenge of TA training in the 19905. New Directions for Teaching and Learning 39:7-14. Nyquist, j.D., R.D. Abbott, D.H. Wulff, and j. Sprague. 1991. Preparing the professoriate of tomorrow to teach: Selected readings in TA training. Dubuque, IA: Kendall/ Hunt. Pajares, MP. 1992. Teachers' beliefs and educational research: Clearing up a messy construct. Review of Educational Research 62 (3):307-332. Pfund, C., S. Miller, K. Brenner, P. Bruns, A. Chang, and D. Ebert-May. 2009. Summer institute to improve university science teaching. Science 324(5926):470-471. Piaget, j. 1932. The moral judgment of the child. New York: Harcourt Brace jovanovich. Piaget, j. and B.Inhelder. 1969. The psychology of the child. New York: Basic Books. Piaget, 1. 1972. Development and learning. In: Lavattelly, CS, and F. Stendler (Eds). Reading in child behavior and development. New York: Hartcourt. Pillar, G.D., K. Karnok, and 5.]. Thein. 2008. Perceptions, utilization, and training of graduate student teaching assistants in introductory soil science courses: Survey results. North American Colleges and Teachers of Agriculture journal 52(3). Posner, G.j., K.A. Strike, P.W. Hewson, and WA. Gertzog. 1982. Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education 66(21 1-217). 171 Prieto, L.R., C.A. Yamokoski, and SA Meyers. 2007. Teaching assistant training and supervision: An examination of optimal delivery modes and skill emphases. journal of Faculty Development 21(1). Prince, M. 2004. Does active learning work? A review of the research. journal of Engineering Education 93(3):223-231. R Core Development Team. 2008. R: A language and environment for statistical computing. R Foundation for Statistical Computing, W protectors- Raymond, A. 1997. Inconsistency between a beginning elementary school teacher's mathematics beliefs and teaching practices. journal for Research in Mathematics Education 28(3):550-576. Richardson, V. and M. Hamilton. 1994. The practice argument staff development process. In: Richardson, V. (Ed). Teacher change and the staff development process: A case in reading instruction. New York: Teachers College Press. p 90-108. Richardson, V. 1995. The consideration of beliefs in staff development In: Richardson, V. (Ed). A theory of teacher change and the practice of staff development: A case in reading instruction. New York: Teachers College Press. Richardson, V. 1996. The role of attitudes and beliefs in learning to teach. In: Sikula, j. (Ed). The handbook of research on teacher development (2nd Edition). New York: Macmillian. p 102-119. Rogers, EM. 1995. Diffusion of innovation (4th Edition). The Free Press: New York. Roling, N.G., and M.A.E. Wagemakers (Eds).l998. Facilitating sustainable agriculture: Participatory learning and adaptive management in times of environmental uncertainty. Cambridge, UK: Cambridge University Press. Rossman, G.B., and SF. Rallis. 1998. Learning in the field: An introduction to qualitative research. Thousand Oaks, CA: Sage Publications. Rushin, j.W., j. DeSaix, A. Lumsden, D.P. Streubel, G. Summers, and C. Bernson. 1997. Graduate teaching assistant training - A basis for improvement of college biology teaching 8: faculty development? American Biology Teacher 59(2):86-90. Sawada, D. 1999. Psychometric properties of RTOP. Tempe, AZ: Arizona Collaboration for Excellence in the Preparation of Teachers. 172 Sawada, D., M. Pilbum, K. Falconer, j. Turley, R. Benford, and I. Bloom. 2000. Reformed teaching observation protocol (RTOP). Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers. Report nr INOO- 1. Schram, P.j., S.G. Wilcox, P.E. Lanier, and G. Lappan. 1988. Changing mathematical conceptions of pre-service teachers a content and pedagogical intervention [microform]. East Lansing, Mich.: National Center for Research on Teacher Education. Seaman, C.E., j.E. Szydlik, S.D. Szydlik, and j.E. Beam. 2005. A comparison of pre- service elementary teachers' beliefs about mathematics and teaching mathematics: 1968 and 1998. School Science and Mathematics 105(4):197- 211. Shirk, GB. 1972. An examination of conceptual frameworks of beginning mathematics teachers'. Unpublished doctoral dissertation: University of Illinois, Urbana-Champaign. Short, KL, and PM. Short. 1989. Teachers' beliefs, perceptions of behavior problems and intervention strategies. journal of Social Studies Research, 12:28-33. Shulman, LS. 1986. Those who understand: Knowledge grth in teaching. Educational Researcher 15(2):4-14. Smith, D. and D.C. Neale. 1989. The construction of subject matter knowledge in primary science teaching. Teaching and Teacher Education 5:1-20. Smith, K.A., S.D. Sheppard, D.W. johnson, and RT. johnson. 2005. Pedagogies of engagement: Classroom-based practices. journal of Engineering Education 94:87-102. Speer, N.M., T. Gutmann, and T]. Murphy. 2005. Mathematics teaching assistant preparation and development College Teaching 53(2):75-80. Speer, NM. 2008. Connecting beliefs and practices: A fine-grained analysis of a college mathematics teacher's collection of beliefs and their relationship to his instructional practice. Cognition and Instruction 26:218-267. Sprague, j., and j.D. Nyquist 1991. A developmental perspective on the TA role. In: Nyquist, j.D., R.D. Abbot, D.H. Wulff, and j. Sprague (Eds). Preparing the professoriate of tomorrow to teach: Selected readings in TA training. Dubuque, IA: Kendall/ Hunt p. 295-312. 173 Stamp, N., and A. Pagano. 2002. Evaluation of a workshop series in university science education. Bioscience 52(4):366-3 72. Steinert, Y., K. Mann, A. Centeno, D. Dolman, j. Spencer, M. Gelula and D. Prideaux. 2006. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME guide no. 8. Medical Teacher 28(6):497-526. Thackwray, B. 1997. Effective evaluation of training and development in higher education. London: Kogan and Page. Thompson, A. 1992. Teachers' beliefs and conceptions: A synthesis of the research. In: Grouws, D. (Ed). Handbook of research on mathematics teaching and learning. New York: Macmillan. p 127-146. Trigwell, K. and S. Shale. 2004. Student learning and the scholarship of university teaching. Studies in Higher Education 29(4):523-536. van Driel, j.H., D. Beijaard, and N. Verloop. 2001. Professional development and reform in science education: The role of teachers' practical knowledge. journal of Research in Science Teaching 38(2):137-158. von Glasersfeld, E. 1984. An introduction to radical constructivism. In: Watzlawick, P. (Ed). The invented reality. New York: Norton. p 17-40. von Glasersfeld, E. 1989. Cognition, construction of knowledge and teaching. Syntheses 80(1):121-141. von Glasersfeld, E. 1991. A constructivist's view of Ieaming and teaching. In: Duit, R., F. Goldberg, and H. Niedderer (Eds). Research in physics learning: Theoretical issues and empirical studies. Kiel, Germany. P 29-39 Wayne, A.j., K.S. Yoon, P. Zhu, S. Cronen, and MS. Garet 2008. Experimenting with teacher professional development: Motives and methods. Educational Researcher 37(8):469-479. Whitcomb, j.A. 2003. Learning and pedagogy in initial teacher preparation. In: Weiner, I.B., W.M. Reynolds, D.K. Freedheim, and GE. Miller (Eds). Handbook of educational psychology. New York: john Wiley and Sons. p 535-552. Wiggins, G. and j. McTighe. 1998. Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development. 174 Wilson, S.M., R.E. Floden, and j. Ferrini-Mundy. 2002. Teacher preparation research: An insider's view from the outside. journal of Teacher Education 53(3):190- 204. Winternitz, T. and W.B. Davis. 2000. Lessons learned during five years of the UC Davis program in college teaching. journal of Graduate Teaching Assistant Development 7(1):69-75. Wood, W.B. 2003. Inquiry-based undergraduate teaching in the life sciences at large research universities: A perspective on the Boyer Commission Report Cell Biology Education 2(Summer):112-116. Worthen, T.K. 1992. The frustrated GTA: A qualitative investigation identifying the needs within the graduate teaching assistant experience 78th Annual Meeting of the Speech Communication Association. Chicago, IL. Yelon, S., L. Sheppard, D. Sleight, and j.K. Ford. 2004. Intention to transfer: How do autonomous professionals become motivated to use new ideas? Performance Improvement Quarterly 17(2):82-103. Yin, RK 2008. Case study research: Design and methods. Thousand Oaks: Sage. Zheng A.Y., j.K. Lawhorn, T. Lumley, and S. Freeman. 2008. Assessment — Application of Bloom's taxonomy debunks the "MCAT myth". Science 319: 414-415. 175 ' 31293 03083 4