COLLABORATIVE LEARNING ENVIRONMENTS IN

INTRODUCTORY PHYSICS

By

Alanna Shannon Pawlak

A DISSERTATION

Submitted to

Michigan State University

in partial fulfillment of the requirements

for the degree of

Physics – Doctor of Philosophy

2018

COLLABORATIVE LEARNING ENVIRONMENTS IN INTRODUCTORY PHYSICS

ABSTRACT

By

Alanna Shannon Pawlak

Introductory physics courses are being transformed in many ways as physics education

strives to better prepare students for continuing studies and the workforce. One particu-

lar transformation being adopted is the implementation of “collaborative learning environ-

ments”, wherein students work together in small groups, and instructors teach by guiding

the students as they solve problems, rather than disseminating information as in a tradi-

tional lecture. Collaborative learning environments are complex and rich with opportunities

to study the teaching and learning of physics, and this dissertation was motivated by the

need to understand how both students and instructors engage in these environments. When

examining a system as complex as collaborative learning environments, it is necessary to

conduct research from several perspectives. The research presented in this dissertation ac-

complishes this by investigating the perspectives of both instructors and a students, and by

considering expanding scopes of analysis.

The first study presented investigates collaborative learning environments from the in-

structor perspective with a narrow scope. Through the analysis of interviews with undergrad-

uate teaching assistants (known as learning assistants) working in a collaborative learning

environment in introductory physics, it identifies the ways they approach teaching prob-

lems that require their students to model physical phenomena computationally. The results

indicate varying levels of sophistication in how learning assistants perceive and utilize the

computational problems in their teaching, with implications for how learning assistants are

trained and supported.

The second study presented shifts from the instructor perspective to the student per-

spective, with a relatively narrow scope of analysis. It examines specifically how students

reason conceptually in collaborative learning environments in introductory physics, and uses

the construct of epistemic games, which are emergent, structured problem-solving strategies

that students may be observed to employ. The results describe the identification of a new

group-level epistemic game comprised of both individual and collective actions that a group

of students may take when reasoning through conceptual problems.

The third and final study presented continues to focus on the student perspective, but

applies the most expansive scope by considering the ways that students engage in collabora-

tive learning environments in introductory physics that go beyond conceptual reasoning. It

presents the development of a new framework for understanding student engagement with

collaborative learning environments that also attends to the tone of students’ interactions

and the structure of their discussions. The results indicate that the framework is flexible

enough to be productively applied in diverse types of collaborative learning environments,

and offers instructors a practical tool to understand and develop their classrooms.

Together, the three studies presented in this dissertation provide a multifaceted view of

collaborative learning environments in introductory physics courses that offers new insight

into these complex environments, with practical utility for informing instructional choices

and best serving students.

ACKNOWLEDGMENTS

I would like to thank my advisor, Danny Caballero, for all of his help throughout my

graduate career.

I was lucky enough to have Danny arrive at Michigan State University

right as I was deciding to shift from nuclear physics to physics education research, and he

immediately took me on as a graduate student, despite my complete lack of experience with

education research. Through his advising, I’ve gained confidence as a physics education

researcher and become a member of the greater physics education research community, and

this work would not have been possible without him.

I would also like to acknowledge my advising committee: Paul Irving, Vashti Sawtelle,

Stuart Tessmer, Melanie Cooper, and Sekhar Chivukula. Our regular meetings were essential

in strengthening my abilities as a researcher and my results, and I am very thankful for their

guidance in carrying out this work.

The Physics Education Research Laboratory at MSU also played a huge role in the

completion of this work. PERL is a very supportive group that has always been available

to provide feedback on my work, to help me grow as a physics education researcher, and to

simply provide moral support while I’ve traversed the graduate program. I would especially

like to thank Paul Irving, who in addition to agreeing to join my committee late in my work,

played an essential role in mentoring me through my final research project. His expertise

and his commitment to my success on the project and in general were invaluable, and I am

so appreciative of all the time he invested in my development.

In addition to all of the academic support I’ve received, the support of my friends and

family for the past six years has been incredible, and I would not have been able to persevere

through the trial that grad school can be without them. To my friends in Chicago and

iv

my friends at MSU in both the wider physics department and within PERL, all of our

commiserating, celebrating, and just wasting time together has been an amazing part of my

grad school experience, and it was all truly so helpful in getting here. To Mike Bennett, I

can’t thank you enough for your constant support and confidence in me for the past four

years. Your encouragement through everything has meant so much to me. To my parents,

I never would have been able to accomplish any of this without you. From driving me to

and from science summer schools when I was in elementary school, to helping me keep going

when grad school seemed hopeless, I’m so thankful for your support.

v

TABLE OF CONTENTS

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

xi

1

5
Chapter 2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.2 What is collaborative learning? . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.2.1 Formats of collaborative learning . . . . . . . . . . . . . . . . . . . .
8
2.2.2 Activities in collaborative learning
. . . . . . . . . . . . . . . . . . .
10
Instructors in collaborative learning environments . . . . . . . . . . . . . . .
11
2.3.1 The roles of instructors in collaborative learning . . . . . . . . . . . .
12
2.3.2
Instructor practices in collaborative learning environments . . . . . .
Instructor perceptions and beliefs in collaborative learning environments 14
2.3.3
15
16

2.4 Student reasoning in collaborative learning environments . . . . . . . . . . .
2.4.1 Frameworks for understanding student reasoning . . . . . . . . . . . .
2.4.2 Quantitative approaches to understanding student reasoning in collab-
orative learning environments . . . . . . . . . . . . . . . . . . . . . .
2.4.3 Qualitative approaches to understanding student reasoning in collab-
orative learning environments . . . . . . . . . . . . . . . . . . . . . .
2.5 Beyond student reasoning in collaborative learning environments . . . . . . .
Student perceptions of collaborative learning environments . . . . . .
Student interactions in collaborative learning environments . . . . . .
2.6 Physics learning environments at Michigan State University . . . . . . . . .
2.6.1 Lyman Briggs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6.2 Department of Physics and Astronomy . . . . . . . . . . . . . . . . .
2.6.2.1 Traditional courses . . . . . . . . . . . . . . . . . . . . . . .
2.6.2.2 P Cubed . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7 Closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.3

2.5.1
2.5.2

17

18
20
21
22
23
24
25
25
26
28

29
29
31
32
35
37
41
42
44

Chapter 3

Learning assistant approaches to teaching computational prob-
lems in a problem-based learning course . . . . . . . . . . . . . .
3.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1 Problem-based learning . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.2 Learning assistants . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.3 Computation in physics education . . . . . . . . . . . . . . . . . . . .
3.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Phenomenography . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Study design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

vi

3.4.1
Study context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.2 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.2.1
Interview protocol development . . . . . . . . . . . . . . . .
3.4.2.2 Conducting interviews . . . . . . . . . . . . . . . . . . . . .
3.5 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.1 Data reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.2
Identification of preliminary themes . . . . . . . . . . . . . . . . . . .
3.5.3 Analysis of preliminary themes
. . . . . . . . . . . . . . . . . . . . .
3.5.4 Development of categories of description . . . . . . . . . . . . . . . .
3.5.5 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.1 Categories of description . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
3.6.1.1 Narrow programming focus
3.6.1.2
. . . . . . . . . . .
3.6.1.3 Computation as a tool focus . . . . . . . . . . . . . . . . . .
Shifting perceptions of learning focus . . . . . . . . . . . . .
3.6.1.4
3.6.2 Outcome space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Learning physics via computation focus

Chapter 4

Identification of a shared answer-making epistemic game in a
group context
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Study context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 Shared answer-making epistemic game
. . . . . . . . . . . . . . . . . . . . .
4.4.1 Entry condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Individual AMEGs . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.2
4.4.3 Group negotiation of their responses
. . . . . . . . . . . . . . . . . .
4.4.4 Disruption and repetition of the individual AMEGs . . . . . . . . . .
4.4.5 Exit condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.6 Ontology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Discussion and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . .

44
46
48
48
49
49
50
52
54
58
59
59
61
64
68
72
76
79

88
88
89
90
91
92
93
95
96
97
98
98

Chapter 5 Development of a modes of collaboration framework

. . . . . 100
5.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.2
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.3 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Individual content understanding . . . . . . . . . . . . . . . . . . . . 103
5.3.1
Student perceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.3.2
5.3.3 Optimizing group work . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.3.4 Categorizing student engagement with group work . . . . . . . . . . . 107
5.4 Study context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.5 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.6 The Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Social dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

5.6.1

vii

5.7.2

5.6.2 Discursive dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.6.3 Disciplinary content dimension . . . . . . . . . . . . . . . . . . . . . 114
5.7 The Modes of Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.7.1 Debate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.7.1.1 Episode A1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.7.1.2 Episode A2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.7.1.3
Social dimension . . . . . . . . . . . . . . . . . . . . . . . . 117
5.7.1.4 Discursive dimension . . . . . . . . . . . . . . . . . . . . . . 118
5.7.1.5 Disciplinary content dimension . . . . . . . . . . . . . . . . 118
Informing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.7.2.1 Episode B . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.7.2.2
Social dimension . . . . . . . . . . . . . . . . . . . . . . . . 120
5.7.2.3 Discursive dimension . . . . . . . . . . . . . . . . . . . . . . 120
5.7.2.4 Disciplinary content dimension . . . . . . . . . . . . . . . . 121
5.7.3 Co-construction of an Answer . . . . . . . . . . . . . . . . . . . . . . 121
5.7.3.1 Episode C . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.7.3.2
Social dimension . . . . . . . . . . . . . . . . . . . . . . . . 123
5.7.3.3 Discursive dimension . . . . . . . . . . . . . . . . . . . . . . 123
5.7.3.4 Disciplinary content dimension . . . . . . . . . . . . . . . . 124
. . . . . . . . . . . . . . 125
5.7.4.1 Episode D . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.7.4.2
Social dimension . . . . . . . . . . . . . . . . . . . . . . . . 126
5.7.4.3 Discursive dimension . . . . . . . . . . . . . . . . . . . . . . 127
5.7.4.4 Disciplinary content dimension . . . . . . . . . . . . . . . . 127
5.7.5 Relationships Among Modes . . . . . . . . . . . . . . . . . . . . . . . 128
5.7.5.1 Episode E . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

5.7.4 Building Understanding towards an Answer

Chapter 6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.1 Summary of findings
6.2
Implications and future directions . . . . . . . . . . . . . . . . . . . . . . . . 138
6.3 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

APPENDICES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Appendix A Expanded description of phenomenographic analysis
. . . . . . . . . 143
Appendix B Sample computational problem . . . . . . . . . . . . . . . . . . . . . 181
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Appendix C Interview protocol

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

viii

LIST OF TABLES

Table 3.1: Majors of the learning assistants interviewed . . . . . . . . . . . . . . . .

46

Table 3.2: Class ranks of the learning assistants interviewed . . . . . . . . . . . . . .

47

Table 3.3: Genders of the learning assistants interviewed . . . . . . . . . . . . . . . .

47

Table 3.4: Semesters of experience teaching in P Cubed of the learning assistants
interviewed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47

Table 3.5: List of preliminary emergent themes identified in the data . . . . . . . . .

51

Table 3.6: Variations within the teaching outcome theme, describing learning assis-
tants’ goals for their students . . . . . . . . . . . . . . . . . . . . . . . . .

Table 3.7: Variations within the teaching strategy theme, describing the techniques
. . . . . . . . . . . . . . . . . . . .

learning assistants use while teaching

Table 3.8: Variations within the characteristic to moderate theme, describing what
aspects of a group’s experience a learning assistant feels they must monitor
and influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 3.9: Variations within the utility of computation theme, describing what learn-
ing assistants perceive the usefulness of computation to be . . . . . . . . .

Table 3.10: Final categories of description. The narrow programming focus treats the
computational problems as a way to develop students’ abilities to write
code. The learning physics via computation focus treats the computational
problems primarily as a means for learning physics concepts. The com-
putation as a tool focus treats the computational problems as a way to
help students understand the value of computation in modeling and solv-
ing problems. The shifting perceptions of learning focus treats the compu-
tational problems as opportunities to develop students’ understandings of
their own learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 3.11: Number of learning assistants with each teaching approach and their expe-
rience levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

52

53

53

55

60

81

Table 5.1: The three dimensions and the categorizations possible within each dimension115

Table 5.2: The Modes of Collaboration identified in this data and their characteriza-

tions along each of the three dimensions . . . . . . . . . . . . . . . . . . . 116

ix

Table A.1: List of preliminary emergent themes identified in the data . . . . . . . . . 146

Table A.2: List of preliminary variations within the teaching technique theme

. . . . 147

Table A.3: List of preliminary variations within the teaching goal theme

. . . . . . . 147

Table A.4: List of preliminary variations within the teaching outcome theme . . . . . 148

Table A.5: First refinement of variations within the teaching outcome theme . . . . . 149

Table A.6: List of variations within the teaching strategy theme . . . . . . . . . . . . 149

Table A.7: List of variations within the characteristic to moderate theme . . . . . . . 149

Table A.8: List of preliminary variations within the purpose of computation theme . . 153

Table A.9: Variations within the utility of computation theme . . . . . . . . . . . . . 154

Table A.10:Variations within the benefit of computation in P Cubed theme . . . . . . 154

Table A.11:Preliminary categories of description . . . . . . . . . . . . . . . . . . . . . 156

Table A.12:Variations within the reorganized utility of computation theme

. . . . . . 158

Table A.13:Final categories of description. The narrow programming focus treats the
computational problems as a way to develop students’ abilities to write
code. The learning physics via computation focus treats the computational
problems primarily as a means for learning physics concepts. The com-
putation as a tool focus treats the computational problems as a way to
help students understand the value of computation in modeling and solv-
ing problems. The shifting perceptions of learning focus treats the compu-
tational problems as opportunities to develop students’ understandings of
their own learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

x

LIST OF FIGURES

Figure 3.1: Outcome space indicating the hierarchical structure relating the identified
learning assistant approaches to teaching computational problems . . . .

Figure 4.1: A schematic of the shared answer-making epistemic game showing how
students progress through the game . . . . . . . . . . . . . . . . . . . . .

77

91

xi

Chapter 1

Introduction

A growing collection of reports and recommendations developed by federal agencies, profes-

sional scientific organizations, and other interested groups have called for an increase in the

number of well-prepared STEM majors entering the workforce and higher education [1–5]. In

order to achieve this, STEM disciplines have had to examine the instruction and education

that they provide to their undergraduate students [6–9]. Discipline-based education research

(DBER) is a field of study that has developed that is particularly well-suited to conducting

the research necessary to guide and inform the changes that STEM disciplines make to their

undergraduate experiences [10]. DBER researchers study the teaching and learning of a par-

ticular discipline from the perspective of being experts in that discipline and being situated

in that discipline. Physics education research (PER) is one area of DBER that has developed

specifically around the teaching and learning of undergraduate physics. PER has studied nu-

merous aspects of undergraduate physics, including students’ conceptual reasoning [11–30],

instructional methods [19–22, 31–43], the impact of implementing reforms [19–25, 31, 44–46],

student attitudes and beliefs [44–46], and issues of equity and inclusion[47–49].

From the results of studies on many of these aspects, and from knowledge taken from

other fields of research, there has been an increasing push to make introductory physics

courses more interactive [8, 10, 19, 50, 51]. “Interactive engagement” is not always a clearly

defined term in physics education and physics education research, and can refer to things

1

as varied as asking clicker-questions in a lecture course, conducting traditional laboratory

experiments in groups, allowing students discussion time in class, and facilitating a problem-

based learning course [19, 50, 52, 53]. If physics as a field is promoting interactive engagement

methods as a way to improve their undergraduate education and to ultimately produce more

and better-qualified graduates, then it is crucial that we understand how these methods are

being implemented and the nuances of the many effects they can have on student learning and

engagement. Because interactive engagement is such an all-inclusive term, it is very difficult

to study it as a whole productively. Instead, it is often more useful to focus on particular

forms of interactive engagement. The form of interactive engagement that I examine in this

dissertation is “collaborative learning environments”. For the purposes of this dissertation,

these are formal physics learning environments in which there is little to no lecturing by

an instructor, and instead small groups of students work together with the guidance of an

instructor.

In this dissertation, I will present three studies that investigate some of the many research

questions surrounding collaborative learning environments in introductory physics courses.

A topic as complex as collaborative learning environments can be, and perhaps must be,

studied from multiple perspectives in order to develop a robust understanding of it. One

way to capture different perspectives is to consider varying scopes; that is, attending to

different scales of focus in one’s research. For example, using a narrow scope to focus

specifically on students’ use of equations in groups versus using a more expansive scope

to consider student engagement in collaborative learning environments holistically when

conducting one’s analysis. Another way to capture different perspectives is to study the

experiences of people inhabiting different roles. For example, observing the behavior of

students versus the behavior of instructors. The studies I will present here use both of these

2

methods of capturing multiple perspectives in order to examine collaborative physics learning

environments in a multifaceted way. Starting from a very narrow scope and broadening to the

largest, and starting from the instructor perspective then shifting to the student perspective,

these studies together present results that provide a rich understanding of these learning

environments.

The first study, presented in Chapter 3, is “Learning Assistant Approaches to Teaching

Computational Problems in a Problem-Based Learning Course”. It presents a phenomeno-

graphic analysis [54, 55] of interviews with undergraduate learning assistants regarding their

experiences teaching computational problems in a problem-based physics course. This study

takes place at narrowest scope, probing the learning assistants’ experiences teaching a spe-

cific type of problem, and it examines the unique role that instructors have through the

experiences of individual learning assistants. The second study, presented in Chapter 4,

shifts from the instructor perspective to the broader and more varied student perspective,

but still maintains a relatively narrow scope by focusing on students’ conceptual reasoning

specifically. Titled “Identification of a Shared Answer-Making Epistemic Game in a Group

Context”, it describes a particular method of problem-solving that groups of students were

observed to engage in when working on conceptual physics problems in a collaborative lab-

oratory class session [56]. The third study, presented in Chapter 5, has the most expansive

scope. The “Development of a Modes of Collaboration Framework” presents a framework for

describing the distinct ways in which groups of students interact when working in collabora-

tive physics learning environments [57]. Like Chapter 4, it considers the student perspective,

but broadens its scope to consider aspects of students’ engagement with collaborative learn-

ing environments beyond their conceptual reasoning, such as their social interactions and

the structure of their discussions. As a result of studying collaborative physics learning en-

3

vironments from these three increasing scopes, and from the perspective of both students

and instructors, I will present in this dissertation a rich picture of this complex topic.

Before the presentation of the aforementioned studies, Chapter 2 will provide a broad

overview of collaborative learning environments, research on collaborative learning environ-

ments, and the collaborative physics learning environments at Michigan State University,

where the studies in Chapters 3, 4, and 5 took place. After the presentation of the studies,

Chapter 6 will synthesize the results of these studies and provide some closing thoughts. We

will proceed now to an overview of collaborative learning environments.

4

Chapter 2

Background

2.1

Introduction

Physics education research, discipline based education research, and education research

broadly have studied collaborative learning from many perspectives and with many methods

over the past decades [50, 58–61]. Having such a multitude of approaches is essential to

studying the complex system that collaborative learning is. To gain a deep understanding

of such a multifaceted phenomenon, researchers must investigate it with varying scopes and

from the diverse perspectives of different participants. This chapter will give an overview of

some ways the education research community has done this. First, a review of what “col-

laborative learning” can refer to will be provided, in order to give an understanding of the

landscape of these contexts. Next, a more detailed discussion of research focused specifi-

cally on instructor perspectives on collaborative learning will be given. Then, shifting to the

student perspective, work focused specifically on student reasoning in collaborative learning

environments will be discussed. Finally, broadening to the most expansive focus, a review

of research on student experiences beyond conceptual reasoning in collaborative learning

environments will be presented. Like the literature reviewed in this chapter, the three novel

studies presented in Chapters 3 through 5 of this dissertation examine collaborative learning

from both the instructor and student perspective, and from a narrow focus to a broad focus.

5

The review in this chapter provides the basis of the novel research in this dissertation, with

a more detailed discussion of the particular literature relevant to each study and its methods

appearing in each chapter. This chapter will also provide background on the context in

which these studies were carried out, physics courses at Michigan State University.

2.2 What is collaborative learning?

Before examining the different ways in which collaborative learning environments in physics

have been studied, it is worth unpacking what “collaborative learning” means. Like “inter-

active engagement” and “active learning”, there is no single agreed upon definition regarding

what constitutes collaborative learning [19, 50–53], and correspondingly, it can take on vastly

different forms. Here we will review several formats of collaborative learning and various col-

laborative activities that have been implemented in physics education, in order to come to

an understanding of the landscape of collaborative learning.

2.2.1 Formats of collaborative learning

Perhaps the most common form of collaborative learning in university physics courses is the

interactive lecture. Like a traditional lecture, these environments tend to have a primary

faculty instructor positioned at the front of the room giving lectures, but also incorporate

pauses for small group discussions or demonstrations [20, 33–36]. Some particularly common

forms of interactive lectures include the use of Clicker questions and discussions [35] and

Peer Instruction [20, 36]. Clicker questions are typically brief, multiple-choice questions

given in lecture, which students respond to in real time using an electronic remote.

In

the collaborative implementation of Clicker questions, students are given the opportunity

6

to discuss their thoughts and answers with their neighbors [35]. Peer Instruction is a well

defined implementation of Clicker-style questions, wherein students are asked questions after

each portion of new material, which they respond to individually, then discuss with their

neighbors, then respond to again. This type of lecture that incorporates time for student

discussions is just one form of collaborative learning, however.

Another well established form of collaborative learning in physics is seen in laboratory

courses. Laboratory exercises are often done in pairs or small groups, in which students

are expected to work together to carry out and analyze an experiment [19, 34, 62]. Even

within this context, however, the appearance of collaborative learning can vary. In some

laboratory courses, students may work relatively independently even within their pairs, while

in a more intentionally facilitated courses, they may be explicitly encouraged to interact in

particular ways [34, 62]. Chapters 4 and 5 of this dissertation present studies carried out

in a collaborative laboratory context that required students to work in groups, but did not

closely monitor or facilitate their interactions. Chapter 4 examines how students reason

about physics in this environment [56], and Chapter 5 examines how students interact with

each other in this environment [57].

In addition to interactive lectures and collaborative laboratories, many research-based

pedagogies have been developed and implemented in university physics. SCALE-UP, studio

physics, Modeling Instruction, ISLE, and problem-based learning (PBL) all take carefully

designed approaches to transforming the physics lecture and/or laboratory experience [22,

32, 37, 39–43, 63, 64]. The aforementioned pedagogies are all centered fundamentally around

group work. Students spend the vast majority of their class time working in small groups

on conceptual or experimental tasks. Some pedagogies include large group synthesis and

discussions, as well. All of these pedagogies tend to leverage several instructors in order to

7

facilitate student interactions and learning in the complex environment that arises. Chapter

3 of this dissertation presents a study that took place in a carefully facilitated problem-based

learning physics course, and investigates instructors’ approaches to teaching computational

problems in this environment.

2.2.2 Activities in collaborative learning

In addition to the format of collaborative learning environments varying, the tasks on which

students work in these environments can vary greatly. Given that the laboratory context is

often collaborative, it naturally follows that many times, students in collaborative learning

environments work on experiments or other measurement and observation based exercises

[19, 34, 41, 42, 62]. These sorts of activities do not only appear in laboratory courses,

however. Pedagogies like SCALE-UP and Modeling Instruction incorporate experiments

into their curriculum outside of a purely laboratory context [41, 42].

Collaborative learning also can incorporate conceptual questions regarding physics con-

tent. In the interactive lecture environments, the tasks that students collaborate on are often

short, conceptual questions [20, 33, 35, 36]. Conceptual questions also play an important

role in the SCALE-UP pedagogy, where they form the basis of short “ponderable” tasks,

which students consider together [41]. Chapters 4 and 5 of this dissertation present studies in

which students worked in groups on a series of brief, conceptual physics problems. Chapter

4 investigates their reasoning, and Chapter 5 investigates their interactions.

Some collaborative learning incorporates complex activities, which students spend sig-

nificant time solving together [22, 32, 38, 42, 43, 65, 66]. In ISLE and Modeling Instruction,

students work together to develop models and representations of physical phenomena over

the course of a class session or sessions [22, 42, 43]. In problem-based learning, students

8

work together to develop a solution to a contextualized problem where the goal and infor-

mation given are not always straightforward [32, 37, 63, 64]. Chapter 3 of this dissertation

presents a study that took place in a problem-based learning style course, where students

worked on complex problems, including some which were computational, and investigates

how instructors approach teaching the computational problems in this environment.

Aligned with the push to incorporate authentic physics practices such as programming

[67, 68] into physics curricula, computational work is also becoming increasingly incorporated

into collaborative learning environments [32, 54, 69–75]. We take computational work here

to mean activities which involve the use of a computer to numerically model a physical

situation, often via students writing and modifying code in a programming language. Such

computational work has been incorporated into collaborative learning environments through

the development of dedicated courses on scientific computing which incorporate collaboration

[69], the incorporation of computation throughout the physics major experience [70], and

through the incorporation of collaborative computational problems into individual physics

courses [32, 71–74]. Chapter 3 of this dissertation presents a study in students collaborated

on computational problems in a particular course that had been transformed to incorporate

both collaboration and computation [32].

As the preceding review demonstrates, collaborative learning environments can take on

many different forms and can require students to work on many different types of tasks. From

interactive lectures to problem-based learning, and from traditional experiments to compu-

tational modeling problems, the landscape of collaborative learning is rich and diverse. This

variation and the complexity inherent to environments in which students interact with not

only the physics concepts, but also each other, makes it essential to study collaborative learn-

ing environments from multiple perspectives. Only through investigating the experiences of

9

both students and instructors, and through implementing different scopes of analyses can

the most complete understanding be achieved. The remaining sections of this Chapter will

review the research that has been done on collaborative learning environments in physics,

beginning with a narrow scope, focused on the unique position of instructors. We will then

shift to research that has been conducted investigating the more diverse student perspective,

focusing first on students’ reasoning within these environments. Finally, taking the most

expansive view, we will review the research that has been done on how students interact

with each another in collaborative learning environments. This expanding focus mirrors the

expanding of focus of the studies that will be presented in the Chapters 3 through 5 of this

dissertation. Chapter 3 will present a study of individual instructors’ approaches to teaching

(specifically computational problems) in a collaborative learning environment, Chapter 4 will

present a study of students’ reasoning in a collaborative learning environment, and Chapter

5 will present a study of how students interact in a collaborative learning environment.

2.3

Instructors in collaborative learning environments

There has been a great deal of research done to understand instructors’ actions, perceptions,

and beliefs in collaborative learning environments. In any classroom, instructors inhabit a

unique position, and in collaborative learning environments, that position can become even

more demanding and varied. Studies on the instructor side of collaborative learning environ-

ments have a relatively narrow research focus, centering specifically on the unique instructor

position. This narrowest focus is what the study presented in Chapter 3 of this dissertation

investigates, examining how undergraduate learning assistants teach computational prob-

lems in a problem-based learning course. These learning assistants are individuals who were

10

successful as students in the course who are invited to join the teaching team. While a

more detailed review of relevant literature appears in Chapter 3, an overview of research

on instructors in collaborative learning environments will be presented here. Our review

of research on instructors will begin with an investigation of who they are and how they

function in collaborative learning environments.

2.3.1 The roles of instructors in collaborative learning

In collaborative learning environments, instructors can take on many different roles. In some

contexts, they function similarly to the way they would in a traditional lecture course. For

example, in Peer Instruction and other interactive lectures, while the instructor takes on

the new role of instigating and sometimes moderating small discussions, they still lecture to

some degree from the front of the classroom and transmit information to the class as a whole

[20, 34–36]. Instructors in collaborative learning environments can also act as synthesizers,

bringing together diverse discussions and viewpoints shared in collaboration to a cohesive

takeaway [36, 38, 41, 42, 76]. Instructors may also act as floating facilitators, giving groups

of students individualized attention and instruction [32, 38, 41, 42, 76, 77]. Chapter 3 of

this dissertation presents a study of instructor approaches to teaching in an environment in

which the instructors function as floating facilitators, and do not engage in lecture or whole

class synthesis at all. Finally, many collaborative learning environments are treated in a

relatively “hands-off” fashion. Students work in groups, but are largely left to their own

devices. Such an approach is commonly seen in traditional laboratory courses, for example.

Chapters 4 and 5 of this dissertation present studies that took place in a largely unfacilitated

collaborative laboratory context.

In addition to instructors functioning differently in different types of collaborative learn-

11

ing environments, instructors can come from different populations. Like traditional college

learning environments, the majority of instructors in collaborative environments are faculty

members, but they also often incorporate graduate teaching assistants and undergraduate

learning assistants. Graduate teaching assistants in collaborative physics learning environ-

ments (and in physics courses in general) are typically graduate students of physics. They

are therefore expected to have relatively strong disciplinary expertise, but the pedagogical

training they undergo can vary greatly [78–81]. It should be noted that this is not vastly

different from faculty members, who often also do not receive extensive pedagogical training

[82, 83]. Perhaps the most unique and interesting population of instructors to consider, how-

ever, is undergraduate learning assistants [80, 84–87]. Since they are undergraduates, they

naturally have not often developed the content expertise of faculty or graduate students.

However, they have strengths that the faculty and graduates often do not. Learning assis-

tants are not always physics majors, so they have expertise in other disciplines that physics

graduate students and faculty members may not have. Additionally, learning assistants are

often selected from students who have taken a course themselves in the past, and so they

have a unique perspective, having experienced the course as a student. Chapter 3 of this

dissertation presents a study of the approaches that learning assistants take to teaching in

a collaborative learning environment. Having considered the roles that instructors may play

and who instructors may be, our review will now turn to studies of how instructors enact

their teaching in collaborative learning environments.

2.3.2 Instructor practices in collaborative learning environments

A natural place to begin the investigation of the instructor side of collaborative learning

environments is to characterize what it is instructors do while teaching in these environments;

12

their practices. Some work of this nature has been done from the perspective of defining

what instructors in particular pedagogies are meant to be doing [36, 37, 41, 42]. Other

research on instructor practices is more descriptive in nature; analyzing what instructors do

while teaching based on observations in the classroom. Such work has been done in various

collaborative learning contexts. For example, researchers have investigated how instructors

teach in laboratories [62], interactive lectures [52], collaborative problem-solving sessions [88],

and collaborative classrooms [89]. While there is some work on graduate teaching assistant

and undergraduate learning assistant teaching practices [90, 91], the majority of this work

has investigated how faculty members teach in collaborative learning environments, leaving

a gap in our understanding of how teaching and learning assistants teach.

Some research regarding instructors in collaborative learning environments has focused on

their practices when teaching particular types of activities. As outlined in Section 2.2.2 of this

chapter, there has been a move to incorporate computation into collaborative learning, and

there has been research conducted on how instructors teach in these situations [74, 92, 93].

This has included work done at the pre-college level, investigating teachers incorporating

technology-dependent activities in general [92, 93], and also work done at the college level,

examining how instructors teach computational problems [74]. However, there has not been

a great deal of work done in this area, and so there is a need to better understand how

instructors teach computation in collaborative physics learning environments.

Given an understanding of what instructors do when teaching in these environments,

many researchers have gone on to investigate instructors’ perceptions and beliefs about these

environments, that is, their opinions about, attitudes towards, and conceptions of collabo-

rative learning environments. The last portion of this review of literature on instructors in

collaborative learning environments will investigate this area of research.

13

2.3.3 Instructor perceptions and beliefs in collaborative learning

environments

Bridging the gap between research on instructor practices and research on instructor be-

liefs, work has been done on relating instructors’ actions to their beliefs about collaborative

learning environments. Some of this work has aimed to use both beliefs and practices to

come to a more cohesive view of an instructor’s teaching [88, 89, 94], while other work has

investigated how instructors’ beliefs impact their practices [90, 91, 95, 96].

Some research has focused more completely on instructors’ beliefs. This type of work

is typically conducted via interviews, and aims to understand how instructors perceive col-

laborative learning environments and what beliefs they have about them, without trying to

directly tie it to classroom practices. Research has been conducted investigating what beliefs

instructors have about teaching and learning in collaborative problem-solving [85, 87, 97].

There have also been studies of what instructors believe good teaching is, and how they

believe collaborative learning should be taught [55, 76, 98]. It is worth noting that again,

despite there being some research on teaching and learning assistant beliefs [85, 87, 90, 91],

much more research has been conducted on faculty beliefs.

Again zooming in on collaborative learning environments that incorporate computation,

research has been conducted to investigate what beliefs instructors have about computation

in these environments. Researchers have examined the beliefs of teachers incorporating

interactive technology-based activities in the K-12 level, ranging from leveraging the use of

the internet and computers in any way to incorporating computational modeling exercises

[92, 93, 99]. There has been less work done investigating instructor beliefs about collaborative

computational exercises at the college level [100–102].

14

Chapter 3 of this dissertation presents a study that fills two of the gaps in the litera-

ture on the instructor side of collaborative learning environments outlined here: a lack of

research on learning assistant beliefs, and a lack of research about beliefs regarding collab-

orative computational exercises. The study takes a phenomenographic approach [54, 55] to

analyze interviews conducted with learning assistants regarding their experiences and be-

liefs about teaching computational problems in a problem-based style introductory physics

course. The study has a narrow scope, looking at the unique instructor perspective, and

analyzing individual perspectives on teaching specific types of problems. This focus alone,

however, provides only a small window in the complex system that collaborative learning

environments present. In order to best understand them, it is essential to consider other par-

ticipants’ perspectives and to consider more expansive scopes. Chapter 4 of this dissertation

presents a study that shifts to the student perspective of collaborative learning, and specifi-

cally investigates how they reason in these environments. The following section presents an

overview of research on student reasoning in collaborative learning environments.

2.4 Student reasoning in collaborative learning envi-

ronments

To best understand collaborative learning environments, they must be studied from multiple

perspectives. Having considered the unique position of instructors, we will now turn our

attention to the perspectives of students. Specifically, we will begin with a narrow focus on

how students reason in collaborative learning environments. Chapter 4 of this dissertation

presents a study investigating students’ reasoning during a collaborative activity, specifically

identifying a new epistemic game describing their problem-solving approach [56]. Our review

15

of the literature on student reasoning in collaborative learning environments will begin with

a background on frameworks that have been used to understand student reasoning.

2.4.1 Frameworks for understanding student reasoning

Several frameworks have been developed in order to understand student reasoning both in

collaborative learning environments and outside of them. In physics education research, one

perspective that was used often in early research was centered around the construct of student

“misconceptions” [103–109]. In this perspective, students are considered to have specific,

discrete, and relatively stable conceptual misunderstandings, and instruction is the process

of replacing these “misconceptions” with the correct understanding of the concepts. This

perspective can provide useful insights into student reasoning, but it is limited in its ability to

describe the nuances of student ideas. Students’ misunderstandings are not static and are not

disconnected from their understandings of other concepts [11–13]. They may demonstrate

a common “misconception” regarding a particular concept in one context, but be able to

use that same concept correctly in a new context. This sort of contradiction indicates

that students’ ideas are not simple misconceptions that can be replaced by the instructor’s

expert-like conceptions. Instead, students ideas are highly contextual and dynamic, and it is

productive to acknowledge this when building frameworks to understand student reasoning

[12].

The resources framework accounts for this by taking a different approach. In the resources

framework, students’ understandings are imagined to be a network of different concepts and

ideas about concepts, referred to as resources. When presented with a problem or put in a

certain context, different resources are activated for a student, and these activation patterns

are what lead to the understandings that a student expresses. With this framework, learning

16

is then the process of providing students with new resources and helping them to form

different activation patterns among their pre-existing resources [14, 15]. This framework

not only attends more to the complexity of student ideas and their contextual nature, but

also acknowledges the valuable ideas that they already have, rather than treating them as

something to replace.

The “misconceptions” and resources frameworks provide two perspectives on the struc-

ture of student knowledge. Physics education research has also leveraged several perspectives

on learning. The cognitive perspective treats learning primarily as the acquisition of new

ideas. In the constructivist perspective, learning is the process by which knowledge is built

through the learner’s interactions with the world. In sociocultural perspectives, an emphasis

is placed on the central role of social interactions in learning. The majority of the literature

presented in this review of literature on student reasoning will take the cognitive or con-

structivist approach, however many studies presented on student interactions in Section 2.5

will provide examples of the sociocultural approach.

2.4.2 Quantitative approaches to understanding student reason-

ing in collaborative learning environments

A great deal of research has investigated student reasoning in collaborative learning environ-

ments through the use of quantitative measures. One common method of doing so is through

the use of concept inventories. Concept inventories are typically multiple-choice tests probing

students’ understanding of some topic [16–18]. Concept inventories have been used widely

in the assessment of student understanding in collaborative learning environments, often

by measuring student performance before and after instruction or an intervention. For ex-

17

ample, several studies have compared collaborative learning environments with traditional

lecture environments, very often finding that collaborative learning leads to larger pre to

post improvement on concept inventory scores [19–22, 110]. This type of measurement has

also been used to assess the impact of specific types of collaborative interventions [111].

Concept inventories have also been used to investigate issues of equity, seeking to determine

if collaborative learning environments can reduce the gap in scores that is typically observed

between students from majority and minority populations [47–49]. Concept inventories are

not the only measures used in quantitative studies of student reasoning, with others using

problem sets, exams, or other assessments [23–25, 112, 113].

This type of work has natural limitations, however. By focusing on pre- and post-

measurements of student understanding, these studies do not provide very much insight into

the mechanisms by which students learn. They do not allow for observations of the ways

that students reason together when working in groups. Furthermore, there are systematic

issues with concept inventories in particular, as it is not always clear to what degree they

really measure students’ understanding of the content they purport to assess [114–116] and

they may unintentionally support deficit views of underrepresented minorities [117]. In order

to get a better understanding of how students reason as they collaborate, it can therefore be

productive to take a more qualitative approach.

2.4.3 Qualitative approaches to understanding student reasoning

in collaborative learning environments

Many studies have been conducted seeking to qualitatively examine how students reason and

work through problems in collaborative learning environments. Some of these have taken ex-

18

planatory or causal stances, attempting to explain the mechanisms by which students learn

and reason while collaborating. For example, some work has proposed various processes by

which learning may occur in collaborative environments [26, 118], or ways that effective col-

laborative reasoning must be structured [119]. Other qualitative work on student reasoning

has been primarily descriptive in nature, offering ways to describe observations of student

reasoning and problem-solving as they collaborate. One notable example of this is the con-

struct of epistemic games. Epistemic games are emergent problem-solving strategies with

which students are observed to engage [27–29]. They are productive in that they provide a

way to understand the seemingly well-defined, but not explicitly stated or imposed, problem-

solving methods that students may use. While many epistemic games have been identified

in individual and collaborative learning environments, because students’ engagement with

collaborative learning environments, and the environments themselves, are so varied, the

identification of new epistemic games is always insightful.

Chapter 4 of this dissertation takes a qualitative approach to investigating student rea-

soning in collaborative learning environments. Specifically, it identifies a new epistemic

game, the shared answer-making epistemic game [56], a variation on an epistemic previ-

ously observed in individual students [29]. The shared answer-making epistemic game adds

to the literature by offering a new lens through which to understand collaborative student

reasoning, and also providing an example of how an individual epistemic game may be ob-

served in a collaborative context. Student reasoning is only part of the complex system of

collaborative learning environments, however. These environments must be studied from

multiple perspectives and with different scopes of analysis in order to be best understood.

The focus of Chapter 4 on student reasoning represents a focus on the student perspective,

and a relatively narrow scope, attending specifically to students’ interaction with the physics

19

concepts. This represents an expansion from the even more narrow focus of Chapter 3, which

will present a study on the unique instructor perspective, examined through individual in-

terviews. Chapter 5 will continue the trend of expanding focus, by taking the largest scope

and examining how students interact with each other in collaborative learning environments.

An overview of the literature on the student interactions and other elements of the student

experience beyond reasoning in collaborative learning environments will be provided in the

following section.

2.5 Beyond student reasoning in collaborative learning

environments

In order to come to the richest understanding of collaborative learning environments, they

must be studied from multiple angles. Having considered the unique instructor perspective,

then shifting to the broader student perspective, but limiting our scope to student reasoning,

we will now shift to the most expansive focus; the elements of student behavior and expe-

rience in collaborative learning environments that go beyond the physics content. Chapter

5 of this dissertation presents a study taking this most expansive approach, describing the

development of a framework for understanding student interactions in collaborative learning

environments that attends to not only the physics they discuss, but also the way in which the

structure their conversations, and the tenor of their interactions [57]. A more detailed review

of the relevant literature will be presented in Chapter 5, but an overview of research on stu-

dent experiences and behavior beyond physics content in collaborative learning environments

will be presented here.

20

2.5.1 Student perceptions of collaborative learning environments

One area of student experiences beyond reasoning that has been examined in collaborative

learning environments is how they perceive the collaboration itself and collaborative tasks.

This has produced a variety results with a variety of areas of interest, including descriptions

of what leads students to feel successful in when collaborating [120] and what students

perceive the value of their peers to be when collaborating [121]. Students’ perceptions in

collaborative learning environments have also been investigated by examining the impact

the learning environments have on students’ attitudes and beliefs about science and learning

[44–46, 64, 110, 122].

Much research on student perceptions of collaborative learning has made use of the

constructs of framing and epistemological framing. Framing is the process by which a person

or group interprets their surroundings, including the context and any task at hand [15, 30,

123], and epistemological framing describes this specifically with respect to students’ framing

of activities related to knowledge and learning [30]. Epistemological framing has been used

productively to describe the different ways that students engage with each other and tasks in

collaborative learning environments [30, 124, 125]. Such studies have resulted in both discrete

[30, 124] and continuous [125] frameworks describing students’ framing of collaborative tasks.

Students’ perceptions and framing of their environment is only one of many facets of

their experiences beyond reasoning in collaborative learning environments. Another highly

salient aspect of their experiences is the interactions they have with each other, and work

on this aspect will be reviewed next.

21

2.5.2 Student interactions in collaborative learning environments

Given that student interactions are a unique and crucial part of collaborative learning en-

vironments, it is no surprise that these interactions have been studied in many ways. The

ways that students interact with each other can have a huge impact on how they learn and so

it’s essential to understand these interactions [126, 127, 133]. Some research has specifically

been aimed at understanding what can account for the differences in how successful differ-

ent groups of students are when factors such as their individual understandings and their

demographics are unable to, and they have found that it is the types of interactions present

in the groups that explain those differences [126, 127]. In order to better understand these

different types of interactions, some work has sought to develop schemes for categorizing

different types of interactions [128–131]. Some work has focused specifically on how groups

of students navigate the conflicts that arise when they need to collaborate to complete an

activity [31, 129, 130, 132]. There have also been studies aimed at understanding what kinds

of interactions are most productive [133], and what kinds of interventions lead to the most

productive student interactions [31]. Research such as this on how students interact in col-

laborative learning environments and the effects of those interactions provides an important

piece in building our understanding of these learning environments.

Chapter 5 of this dissertation presents a framework developed in order to better un-

derstand how students engage in collaborative learning environments [57]. The Modes of

Collaboration framework attends to three dimensions of their experiences: the tone of their

social interactions, the structure of their discussions, and the physics content on which they

focus. By attending to all three of these elements, the framework provides a rich picture of

how students engage with collaborative learning environments that goes beyond their con-

22

ceptual reasoning. This is still only one part of the complex system of collaborative learning

environments, however. This most expansive view must be combined with other perspectives

to develop the most robust understanding of these environments. This dissertation combines

this expansive focus on student engagement with a more narrow focus on student reasoning

in Chapter 4, and a narrower focus still on individual instructor perspectives in Chapter 3

in order to provide a multifaceted view of collaborative learning environments.

All three of these studies were carried out in physics courses at Michigan State University.

An overview of this context will be provided in the following section.

2.6 Physics learning environments at Michigan State

University

The studies presented in this dissertation were conducted at Michigan State University

(MSU), a large public research university. MSU has a total undergraduate enrollment of

over 35,000 students, with approximately 350 physics majors, and graduating approximately

35 physics majors each year. The physics major within MSU is unique in that a student may

pursue it in two different colleges: the College of Natural Science or Lyman Briggs College.

In the College of Natural Science, a physics major is part of the Department of Physics and

Astronomy, which grants degrees in both physics and astrophysics. This conforms to a more

traditional physics major experience, and all physics courses are taken through the Depart-

ment of Physics and Astronomy. The Lyman Briggs College is a residential college for the

sciences, in which most students live in the same dormitory, which also houses their intro-

ductory courses. A student pursuing a physics degree from this college will obtain the same

degree as a student in the College of Natural Science, but may take their introductory level

23

science and math classes through Lyman Briggs, which offers its own introductory series of

courses. Students in the Lyman Briggs College also take several courses on the history, phi-

losophy, and sociology of science. It is worth noting that students in either college may take

either college’s introductory physics courses. Chapter 3 of this dissertation presents a study

that was conducted in the Department of Physics and Astronomy introductory mechanics

course. Chapters 4 and 5 of this dissertation present studies that were conducted in the

Lyman Briggs introductory electricity and magnetism course. A more detailed description

of the different physics courses at MSU will now be provided.

2.6.1 Lyman Briggs

At the time of this research, the Lyman Briggs physics courses were taught primarily via

interactive lectures and the required co-enrollment in a laboratory component, with the

lecture and laboratory components typically having different instructors. It was a calculus-

based physics sequence, which required one semester of calculus as a pre-requisite for the

introductory mechanics course, and recommended a second semester of calculus for students

taking the introductory electricity and magnetism course. In practice, however, both courses

made relatively little use of calculus. The lecture component of the course met three times

per week for one hour and had an enrollment of approximately 100 students. The students

attended one three hour laboratory section per week in sections of approximately 20 students

each. The students in the course were typically life science majors and sophomores. The

course demographics were majority female with respect to gender, and majority white with

respect to race.

In the lecture meetings, the class was taught in a highly interactive way. A great deal of

time was spent soliciting student questions, encouraging student discussions, and engaging

24

with clicker questions, and lecturing time was kept at a minimum. In the laboratory sessions,

students worked in groups of approximately four on a variety of collaborative activities,

including traditional experiments, tutorials, and conceptual and numeric problems. The

laboratory sections were taught by pairs of undergraduate learning assistants, who received

weekly training from the laboratory instructor. The learning assistants were students who

had been successful in taking the course themselves previously. The course also employed a

graduate teaching assistant who worked in both the lecture and laboratory components.

The studies presented in Chapters 4 and 5 of this dissertation were carried out in the

laboratory component of the Lyman Briggs electricity and magnetism course while the author

was serving as its graduate teaching assistant. Video data and written artifacts were collected

from every section for one week as they worked on a series of conceptual questions regarding

electric field and potential. This data provided insight into both how students reasoned

about the concepts [56] and also how they interacted with each other [57] while working in

this collaborative environment.

2.6.2 Department of Physics and Astronomy

2.6.2.1 Traditional courses

The majority of the introductory physics courses in the Department of Physics and Astron-

omy at the time of this research were taught in a traditional lecture style. There was no

required co-enrollment in a laboratory section for the Department of Physics and Astron-

omy introductory physics courses. The Department of Physics and Astronomy offers both

algebra and calculus based introductory sequences, with the former being taken primarily by

life science majors, and the latter being taken primarily by physics and engineering majors.

25

The remainder of this overview of introductory physics at MSU will focus on the calculus

based sequence. The demographics of the calculus based sequence were majority male with

respect to gender, and majority white with respect to race.

The lectures meetings were taught in sections of approximately 300 students in large

auditorium-style lecture halls, meeting three times a week for one hour. Instructors typically

taught from the front of the room using projected slides. There was some variation among

instructors with respect to fostering an interactive lecture through the use of clicker questions

or peer discussions, but many instructors relied primarily on traditional lecturing. These

sections typically did not employ graduate teaching assistants or undergraduate learning

assistants, outside of grading assistance.

In addition to the traditional lecture style, the Department of Physics and Astronomy

now also offers a problem-based learning style of the calculus based introductory sequence.

This is the context in which the study presented in Chapter 3 was carried out, and it will

be described in greater detail in the following section.

2.6.2.2 P Cubed

Shortly before the research presented in this dissertation was conducted, a problem-based

learning version of the Department of Physics and Astronomy introductory mechanics course

was developed, known colloquially as Projects and Practices in Physics, or “P Cubed” [32].

P Cubed has the same course number and satisfies the same requirements as the traditional

lecture sections; it is simply another section of the same course that students may choose

to enroll in. P Cubed is open to any student and does not require special permission to

enroll, and it is explained explicitly as being taught in a problem-based learning style on

the online course registration platform. P Cubed was piloted with a 40 student section, but

26

now is taught to a 100 student section. The students in P Cubed are primarily engineering

majors and sophomores. The demographics of P Cubed are majority male with respect to

gender, and majority white with respect to race, however the gender demographics are closer

to parity than those of the traditional sections.

In P Cubed, there is no lecture whatsoever, and instead students spend every class ses-

sion working on a complex problem in groups of four. The course meets for two hours twice

a week, and there are four three hour long evening exams over the course of the semester.

The problems that students work on in P Cubed are designed to be complicated and require

significant collaboration, planning, and investigation. Students often must seek out addi-

tional information online, or must ask an instructor for additional information. P Cubed

also incorporates computational problems. While most of the problems are analytic, that is,

solved by hand, approximately five problems throughout the semester are computational. In

the computational problems, students are given a prewritten portion of code that will run

and render a visualization of a physical scenario, but will not correctly model the physics of

the situation. Their task on computational problems is to modify and add to the code to

make it properly reflect the physics. P Cubed is taught by a team comprised typically of

one or two faculty members, a graduate teaching assistant, and 12 undergraduate learning

assistants. The undergraduate learning assistants are selected from students who were suc-

cessful as students in P Cubed. In the classroom, all members of the teaching team function

identically regardless of their “rank”, with each being responsible for facilitating two groups

of students and ensuring that they develop an understanding of the concepts.

Chapter 3 of this dissertation presents a study regarding the P Cubed section of the

Department of Physics and Astronomy introductory mechanics course after it had scaled up

to a 100 student enrollment. Specifically, interviews were conducted with learning assistants

27

regarding their experiences teaching the computational problems. These interviews resulted

in the development of a set of categories that describe different approaches learning assistants

may take to teaching computational problems in problem-based learning environments.

2.7 Closing

This concludes our overview of the literature on collaborative learning environments and

the collaborative learning environments at Michigan State University. Like the three studies

presented in Chapters 3, 4, and 5, our review of the literature began with a narrow focus on

the instructor perspective in collaborative learning environments, then shifted to the broader

student perspective, first with a targeted focus on student reasoning, then with a more

expansive focus on student engagement with collaborative learning environments beyond

conceptual reasoning and understanding. While a more detailed review of the relevant

literature and methodologies will be provided in the chapters devoted to each study, this

background gives a broad perspective on collaborative learning environments and those at

Michigan State University.

We will proceed now to the first of the three novel studies presented in this dissertation,

and the one with the finest focus: learning assistant approaches to teaching computational

problems in a problem-based learning course.

28

Chapter 3

Learning assistant approaches to

teaching computational problems in a

problem-based learning course

3.1

Introduction

In order to meet the repeatedly identified need for well-prepared STEM majors in both

modern academia and industry [1–5], undergraduate physics education has adapted in many

ways [6–10]. One strategy has been the increased incorporation of collaborative learning

activities and the development of collaborative learning environments.

In these contexts,

students work primarily with each other in groups, with instructors facilitating their work

rather than lecturing [22, 32, 37–43]. Problem-based learning, for example, has students work

in groups to solve complex problems framed in realistic contexts, often without explicitly

stated questions [32, 37, 63, 64]. Problem-based learning environments can require careful

facilitation to ensure that students are having productive learning experiences. Instructors

may need to moderate group dynamics and guide students’ learning in a way that isn’t

demanded of them in a traditional lecture course [37, 76, 77, 134]. This context offers a rich

environment in which to study instructors’ enacted teaching methods, that is, their practices,

29

and their opinions and perceptions of teaching and learning, that is, their beliefs.

To give students this level of attention, collaborative learning environments often employ

teaching assistants in addition to faculty members [77, 79, 80, 84, 135]. These teaching

assistants may be graduate students, or undergraduate students, with the latter often referred

to as learning assistants. Learning assistants are a unique population of instructors, as they

may be best suited to understand their students’ perspectives, and also may not have a great

deal more experience with the content than their students [80, 84]. The teaching practices

and beliefs of learning assistants have not been greatly studied however [85–87], and so there

is a need for further research on this group.

In tandem with transforming the structure and instruction of learning environments in

these ways, there has been a push to incorporate authentic practices into physics education

in order to better prepare students for academia and industry [67, 68]. One particular

practice that has had an increasing emphasis is computation [32, 68–74]. The practice of

modern science relies on the use of computers to model and simulate phenomena that are

impossible or impractical to study analytically, but historically, students have received little

explicit instruction in these skills [68]. Computational exercises, that is, exercises which

require students to model or simulate physical scenarios numerically, are beginning to be

more incorporated into undergraduate physics courses. Studying the ways in which these

are being implemented and taught is important for making them as productive as possible

for students.

We conducted a study taking place at the intersection of these three unique develop-

ments in physics education: problem-based learning, undergraduate learning assistants, and

computational problems. We interviewed learning assistants who had taught in a section

of an introductory mechanics course that is taught in a problem-based learning style and

30

that incorporates several computational problems. Our aim was to examine how they ap-

proached teaching computational problems in this environment, because understanding how

these uniquely positioned and not often studied members of the teaching team work in such a

complex environment provides valuable insight into how we can best design and implement

computational work for students, and gaining such insight will be increasingly important

as computation becomes more prevalent in physics curricula. In particular, better under-

standing how learning assistants approach teaching computational problems could allow for

the development of training and supports that encourage them to teach in ways that are

most in line with the course goals surrounding computation. To examine the learning assis-

tants’ approaches to teaching computational problems, we conducted a phenomenographic

analysis [54, 55, 136, 137] of interviews with 12 physics learning assistants, resulting in the

development of four approaches they may take to teaching computational problems in a

problem-based learning course. This chapter presents these approaches, the process of their

development, and their implications, and Appendix A provides an expanded description of

the analysis process.

3.2 Background

The study presented here took place in a very unique context. The participants were un-

dergraduate learning assistants, a relatively little studied population [84–87], the course was

taught in a problem-based learning style, a context rich with complex interactions among

students and instructors [37, 63, 64, 76], and the course made use of computational prob-

lems, a unique type of activity at the center of the push to incorporating authentic practices

into physics education [32, 68–74]. A review of the literature on these three subjects will be

31

provided here.

3.2.1 Problem-based learning

Problem-based learning was developed in the field of medical education in order to give

medical students better opportunities to apply their knowledge in realistic scenarios [138,

139]. Since then, it has been adopted in many contexts, across different education levels

and disciplines [37, 63, 64, 134, 140]. Problem-based learning is founded upon a shift from

the instructor-centered approach of traditional lecture environments, to a student-centered

approach, wherein students must navigate complex problems together [37, 63, 64, 77, 134].

There are a variety of definitions and implementations of problem-based learning, but these

learning environments are often characterized by students working in collaborative groups,

instructors functioning as guides rather than lecturers, and problems that require students

to leverage decision-making, planning, and problem-solving skills. The goal of these learning

environments is that students develop conceptual understandings and discipline-relevant

skills through the process of their largely self-directed work through the problems [37, 63,

64, 134, 138, 139]. The dynamic structure and complex tasks in problem-based learning

environments make them a rich environment for educational research, and such research has

been done on the perspectives of both instructors and students.

The student perspective of problem-based learning has been studied in several ways.

Many quantitative studies have measured the conceptual learning of students in these envi-

ronments and found that they are more effective than traditional pedagogies [37, 110, 113,

122, 141]. Dochy et al. conducted a meta-analysis of such studies and found that they over-

whelmingly indicated that problem-based learning has positive impacts on students’ learning

[113]. Barron et al. found that middle school students’ learning improved by several metrics

32

when they engaged in problem-based learning activities [37]. In post-secondary physics edu-

cation specifically, multiple studies have echoed these results. Van Kampen et al. found that

students’ exam performance improved after the development of a problem-based learning

module [141]. Selcuk et al. investigated the impact of problem-based learning on pre-service

teachers in a physics course and found that it improved their achievement on physics activ-

ities [122]. Using the Force Concept Inventory, Sahin et al.

found that students who had

engaged in problem-based learning had greater learning gains than those in traditionally

taught courses [110].

Several studies have measured the impact of problem-based learning on student engage-

ment and perceptions. In the previously mentioned study by Van Kampen et al., results also

indicated that the problem-based learning module they implemented had a positive effect

on how interesting and relevant students found the physics content [141]. Ahfeldt et al.

conducted a quantitative study to assess the impact of problem-based learning on student

engagement, and found that it resulted in greater engagement than traditional courses [142].

Selcuk et al. similarly found that problem-based learning increased students’ interest [122].

Results have not been universally positive, however, with Sahin el al. finding that problem-

based learning did not appear to have any benefits over traditional courses when comparing

student attitudes and beliefs as measured by the Colorado Learning Attitudes about Science

Survey [110].

There has also been research examining other aspects of students’ experiences in problem-

based learning environments. Raine et al.

found that students perceive problem-based

learning to effectively model how work is done outside of classrooms. They also found

that students are cognizant of and appreciate the approach to learning that problem-based

learning encourages [63]. Selcuk et al.

similarly found that problem-based learning was

33

effective at encouraging students to take up a deep approach to learning [122]. Through

interviews with students, Cockrell et al.

identified “ownership of knowledge” as a salient

characteristic of problem-based learning to students, with it being discussed in three contexts:

group dynamics, tutor feedback, and metacognitive awareness [64]. Chiriac developed a

framework to identify and describe the group processes that students engage in when working

in problem-based learning environments [131].

Less work has been done on the instructor perspective of problem-based learning en-

vironments, despite the fact that these environments require just as radical of a shift in

approach of instructors as they do of students [37, 76, 77, 134]. While instructors in lecture-

based environments may attend primarily to the transmission of content, problem-based

learning environments require instructors to interact with their students, foster productive

interactions among their students, and allow students to approach the problems in their

own ways, while still guiding them towards conceptual understanding. These considerations

are complex and result in a myriad of approaches and beliefs that instructors may bring to

problem-based learning. Barron et al.

identified several design principles that instructors

should consider when developing problem-based learning activities [37], while Maudsley dis-

cussed the ways that instructors must function in problem-based learning environments [77].

Abrandt et al. interviewed instructors who implemented problem-based learning and found

notable variations in their perceptions. They found that teachers considered problem-based

learning from either the teacher perspective or from the learner perspective, and that they

viewed their role to be either supportive or directive [76].

It is important to understand the perspective of instructors and how they approach

their teaching in problem-based learning environments, as their practices will greatly affect

students’ learning and experiences. In particular, the way that more junior instructors in

34

problem-based learning environments, such as undergraduate learning assistants, enact and

perceive their teaching merits further study. Learning assistants can often comprise the

majority of the teaching team in a problem-based course, and so understanding how they

approach their teaching is essential to understanding these classrooms.

3.2.2 Learning assistants

The complex and collaborative nature of problem-based learning often requires greater fa-

cilitation than traditional lecture courses. In order to ensure that students are developing

conceptual understandings of the physics content as they work through the complex prob-

lems, instructors must carefully guide their progress and be aware of their conversations and

work [37, 76, 77, 134]. To do this effectively, problem-based learning courses may employ

teaching and learning assistants. Learning assistants provide a unique perspective, often

being closer in experience with the content to their students than faculty members and

graduate teaching assistants.

A particularly robust learning assistant program is in place at the University of Colorado

Boulder, where the the first official learning assistant program was developed, and where

the term “learning assistant” was coined. In the model developed at University of Colorado

Boulder, the learning assistant program is framed as not only a means to provide greater

facilitation in their courses, but also as an opportunity to build the next generation of

science teachers and make students more aware of the possibilities for teaching careers in

STEM [84, 143]. Much of the currently available research on learning assistants in physics

has taken place in this context. There have been studies by Pollock et al. and Otero et

al. conducted on the effects of employing learning assistants on student learning and course

transformation sustainability, indicating that learning assistants have a valuable and positive

35

impact on students and courses [84, 135]. There has also been work done by Spike et al.

examining how to best prepare learning assistants for their teaching assignments, resulting

in the development of particular pedagogical models for learning assistant training [80].

Researchers have also investigated the long-term effects that participating in the learning

assistant program has on individuals who go on to become teachers. In a study based on

teaching observations, Gray et al.

found that in-service teachers who had been learning

assistants scored higher on measures of interactive teaching methods than peers who had

not been learning assistants [86]. In a similar comparative study based on interviews with

in-service teachers, Gray et al. found that former learning assistants were more comfortable

implementing group work in their classrooms and were more likely to state that building a

learning community was a goal in their implementation of group work [85]. Gray et al. also

examined how current learning assistants described their use of questions when facilitating

recitation sessions of introductory physics courses, finding that their comments suggested

several different goals in asking questions, and several different perceptions of their role as a

learning assistant, including facilitating physics content, facilitating students’ sense-making,

shaping the learning environment, and professional growth [87]. Learning assistant programs

have been implemented at other institutions as well, such as Florida International University

[144] and Texas State University [145], but research is relatively limited currently.

These studies provide valuable insight into learning assistants, however they have limi-

tations. The learning assistants studied in the aforementioned studies worked in a variety

of types of learning environments. For example, some worked in recitation sections where

students worked on physics tutorials, which are relatively linear activities, with well-defined

questions for students to answer sequentially [146]. This sort of activity is very different than

the complex problems in problem-based learning, which can require students to determine

36

their own goals and questions and may have multiple possible solutions [32]. This difference

could lead to differences in how learning assistants approach and perceive teaching, and it

is the teaching approaches of learning assistants in a problem-based learning course that we

examine in this study. In particular, we focus on how they perceive teaching problems that

require students to engage in computational modeling, and we will review the research on

computational activities in physics education next.

3.2.3 Computation in physics education

For the past several decades, STEM education has increasingly incorporated computer-

based activities and computational activities into the classroom [74, 101, 147–149]. While

computer-based activities can be as simple as leveraging the internet for instruction and

homework, “computational activities” here will refer to tasks which require students to en-

gage in the numerical modeling, analysis, or simulation of phenomena. Physics education has

pushed to incorporate these sorts of computational activities in the curriculum, as they pro-

vide students with valuable practice engaging in an authentic aspect of modern science, and

also can aid students’ understanding of physics concepts [68]. Early examples of computa-

tional activities in physics education include the Berkeley BOXER project and the Maryland

University Project in Physics and Educational Technology (MUPPET) [74, 148, 149]. These

environments were very open, engaging students with the act of programming and giving

them a great degree of control. In contrast, in another early example, the Physics Educa-

tional Technology simulations (PhETs) are relatively closed, giving students the ability to

vary some parameters of a physical situation, but not making the underlying programming

visible to them [147].

Some implementations of computational problems take elements of both styles, making

37

some code visible and modifiable, but not expecting students to write programs from the

ground up. In the field of biology, Wilensky et al. had students model biological phenom-

ena in a computational platform that required them to write logical “rules” controlling the

simulation, but did not engage them with more typical programming syntax [75]. In several

different contexts, Caballero et al. developed computational physics problems in VPython, a

variation of the Python language containing elements making it well suited to visual model-

ing. In these problems, students are given code that will run, but that does not yet correctly

model the physical situation, known as “minimally working programs”. Students’ task is

to then modify and add to the code so that it correctly reflects the physics of the situation

[71, 72]. This type of problem design is similar to the approach that Chabay et al. had

taken previously to incorporating computational problems [73]. In addition to these various

designs, computational problems can be incorporated into curricula in different ways. At

Oregon State, computational problems like these have been intentionally woven into courses

across the curriculum [70], while Kaplan describes the development of a stand-alone course

on scientific computation at Macalester College [69]. Often, however, computational prob-

lems are incorporated into courses on an individual basis at the discretion of the instructor.

Given this wide variation in how computational problems are implemented and their in-

creasing incorporation into STEM education, research has been conducted in order to better

understand how students and instructors experience and engage with these problems.

Research on the student experience of computational problems often focuses on assess-

ing students’ abilities to engage with the problems, and what identifying difficulties they

encounter [71–73, 75, 150]. Chabay et al. found that when working on computational prob-

lems, students had more opportunities to use creativity in their problem-solving and were

better able to see connections across physical phenomena, however they sometimes struggled

38

due to their lack of programming experience [73]. When students engaged in computational

modeling using the logical rule-based format in the context of Wilensky et al., they were

able to complete more complex problems, and engaged in more spontaneous question-asking

and exploration [75]. Caballero et al. found that the majority of students who had engaged

in their style of computational problem were able to successfully solve new computational

problems. They also observed that the students who were unsuccessful in the new problems

had errors that suggested that a misunderstanding of the physics was limiting them, rather

than indicating that the programming was a barrier to their success [71, 72].

Some research has taken a broader approach when investigating the experiences of stu-

dents working on computational problems. Lewis and Shah observed students working on

pair programming exercises using an equity perspective, noting what inequities arose and sug-

gesting methods to mitigate them [151, 152]. Other studies have taken a phenomenographic

approach to identify the different ways that students experience computational exercises.

Booth et al. found four different approaches that college students in a programming class

would take to their programming: structural, operational, constructual, and expedient [54].

In a similar phenomenographic study of introductory programming students, Bruce et al.

identified five experiences that students may have with respect to programming: following,

coding, understanding and integrating, problem-solving, and participating, with each rep-

resenting a student taking on a more expansive view of programming [153]. Studies such

as these, and those on student achievement and difficulties regarding computational prob-

lems provide just one perspective, however, and it is important to also consider the teaching

beliefs and practices of instructors who implement such problems.

Some research regarding instructors who incorporate computation into their classrooms

has simply sought to identify the obstacles they perceive and to develop methods to help

39

instructors overcome those barriers to implementation, such as that by Niess et al. [93]. With

a similar motivation, Koehler et al.

investigated the experiences of faculty and graduate

students who took part in a workshop on incorporating technology into their courses, and

found that having the experience embedded in their scientific disciplines had a significant

impact on their perceptions [102]. Yerrick et al. examined the effect that perceived obstacles

have on the different ways that teachers implement computational activities [99]. Hennessy

also investigated variations in teachers’ implementation of computational activities, finding

several salient aspects of the teachers’ experiences, such as leveraging the dynamic nature

of simulations, focusing students’ attention on key concepts, and often employing a teacher-

directed facilitation style [92]. Phenomenography has also been applied to the instructor

perspective of computational problems, as it was to the student perspective by Booth and

Bruce. Magana et al.

interviewed faculty teaching undergraduate and graduate STEM

courses that implemented computational simulation exercises. They found eight different

intended outcomes that instructors may have in incorporating computation into their courses:

to become aware of the role of computation in science, to make measurements, to find causal

relationships in models, to test models, to validate results, to use computational techniques,

to predict behavior, and to discriminate among possible models. They further argued that

these eight intended outcomes represent increasingly higher order levels of thinking [100].

This sort of research on the perceptions of instructors is essential to understanding the

implementation of computational problems, because instructors’ beliefs and goals greatly

impact how computational activities are carried out in practice [99].

The research on computational activities in physics (and STEM) education reviewed

here gives a broad picture of ways computation is being implemented, how students per-

form on computational activities and what they struggle with, what instructors struggle

40

with when teaching them, and some of the perceptions instructors have regarding them.

While there has been some work done to qualitatively understand students’ experiences of

computational problems [54, 153], there has been less done investigating the experiences of

instructors [100]. Computational problems require instructors to attend to not only science

content, but also computational practices in their teaching. Depending on the structure

of the course, instructors might additionally need to carefully facilitate group interactions

and problem-solving processes, as in problem-based learning. The research presented here

takes place in a problem-based learning course that incorporated computational problems

and that employed undergraduate learning assistants [32]. Teaching computational problems

in this environment can be complex and challenging, and it is important to understand how

instructors approach that challenge. In particular, better understanding the approaches and

perceptions of learning assistants can allow us to better support their teaching and their

development as teachers.

In order to investigate the learning assistants’ approaches, we

employed phenomenography as our method, which will be described next.

3.3 Methodology

The research question at the heart of this study was: what are the different approaches that

learning assistants may take to teaching computational problems in a problem-based learning

physics course? In investigating this question, our goal was not to describe what methods

they use in practice based on observations of their teaching, but rather to understand the

approaches they identify with based on their opinions and self-described experiences. Phe-

nomenography is a method uniquely suited to this type of study, wherein one seeks to

identify different categories describing the ways that a group of individuals may experience

41

a phenomenon.

3.3.1 Phenomenography

Phenomenography developed as a research methodology in the 1970s through the work of

Marton et al. in the context of education research as a means to understand student expe-

riences [154, 155]. Marton defined it as the study of the variations in the ways individuals

perceive and understand phenomena, taking a relational stance, that is, arguing that the

phenomena are not separate and exterior to the individual, but experienced through a rela-

tionship [156]. Phenomenography rests on the assumption that although individuals expe-

rience a given phenomenon in unique ways, by probing a group of individuals’ experiences,

one can uncover the essential variations in how they experience the phenomenon. These

variations represent the underlying differences in experience that emerge when considering

the group as a collective, and not differences observed by directly comparing individuals

[157]. As a result, variations may appear not only across individuals, but also within indi-

viduals. Indeed, a phenomenographic study does not seek to account for the entirety of a

given individual’s experience of the phenomenon, nor does it seek to identify variations that

allow for each individual to unambiguously exhibit only one variation of experience. Instead,

the objective is to use the individuals’ experiences to identify meaningful variations that give

insight into how the collective may experience the phenomenon [137, 157, 158].

The goal of a phenomenographic study is to find a set of categories that capture the

variations in individuals’ experiences of the phenomenon [54, 137, 156, 158, 159]. These

“categories of description” should satisfy several criteria. Each category should describe a

distinct way of experiencing the phenomenon of interest, the categories should have a logical

relationship to one another, and the set of categories should be as concise as possible while

42

still capturing the essential variations. The logical structure relating the categories is know as

an “outcome space”, and the outcome space is often hierarchical. In a hierarchical outcome

space, categories represent increasingly sophisticated and detailed ways of experiencing the

phenomenon. The hierarchical nature of many outcome spaces can provide a great deal

of insight into individuals’ experiences of the phenomenon. For example, since a given

individual may exhibit several variations, if these variations represent different levels of

sophistication of experience, we can gain insight into how individuals progress through the

hierarchy to develop more sophisticated perceptions [160, 161].

A great deal of phenomenographic work has focused on approaches to teaching and learn-

ing. A common result from these studies are such hierarchical outcome spaces, describing the

identification of increasingly sophisticated levels of teaching and learning [54, 100, 153, 162–

166]. These results have great practical value in understanding learning environments, be-

cause they provide insight into how students adopt more sophisticated approaches to learn-

ing, and how teachers develop more sophisticated approaches to teaching. In this study, we

investigated the teaching approaches of learning assistants, and specifically, their approaches

when teaching computational problems in a problem-based learning course. By employing

phenomenography, we were able to identify the underlying variations in how they approach

teaching in this context, gaining insight into their beliefs as teachers and how we might

support their development as teachers. In the next section, we describe the design of our

phenomenographic study.

43

3.4 Study design

3.4.1 Study context

This study investigated the experiences of undergraduate learning assistants who had taught

in an introductory calculus-based mechanics course. The course was a transformed section

of the traditional lecture style course that is taught in problem-based learning style, known

as “Projects and Practices in Physics” or “P Cubed” [32]. In its implementation of problem-

based learning, P Cubed makes no use of lecture, and instead, students spend all time in the

classroom working in groups of four on complex problems. These problems are written in

such a way that they require that students work together and think intentionally about their

problem-solving process. Over the course of the semester, approximately 5 of the problems

are computational. In the computational problems, students are given a physical situation

that highlights the physics concepts being taught that week, and they are given a portion

of VPython code. The code they are given functions in that it will run without error, but

it does not yet correctly model the physics relevant to the situation. The students’ task for

the class period is to modify the code such that it does correctly model the situation. The

correctly modified code will render a visualization of the situation and/or graphs of physical

quantities present in the situation. An example of a computational problem assigned in the

course appears in Appendix B.

P Cubed meets for two hours twice a week, and has an enrollment of 100 students, most

of whom are sophomore engineering majors. The course is open to all students, with en-

rollment occurring on a first come, first serve basis. Additionally, the course description

on the university registration website makes it clear that it is taught in a problem-based

learning style. P Cubed is taught by a teaching team composed of one or two physics faculty

44

members (changing each semester), a graduate teaching assistant, and approximately 12 un-

dergraduate learning assistants. The learning assistants are individuals who were successful

as students in the course who are then invited to become instructors in the course. Cor-

responding to the student demographics, the learning assistants are primarily engineering

majors. Learning assistants are invited to continue teaching the course as long as they are

interested and available, and most choose to teach for several semesters.

In the classroom, all members of the teaching team function identically. Each instructor

is responsible for two groups of students, guiding their progress through the concepts and

problem of each day. Outside of the classroom, the teaching team has two pre-class meetings

every week (the day before each class session), and a post-class meeting at the end of each

week. In the pre-class meetings, the learning assistants work through the next day’s problem

in small groups, facilitated by the faculty members or graduate teaching assistant. In these

meetings, the learning assistants have the opportunity to make sure they understand the

concepts and solution themselves, and are also encouraged to think about how they will help

their students navigate the problem and concepts. At the post-class meeting, all members

of the teaching team summarize how their teaching went that week, and they can solicit

help for any issues they are facing. Every member of the teaching team is responsible for

writing individualized feedback for each of their students regarding their work that week. In

addition to the aforementioned on-going responsibilities, the learning assistants also attend

a day long training workshop at the beginning of each semester, which goes over in class

teaching strategies, writing feedback, and course logistics.

45

Major

Number

Mechanical engineering
Biosystems engineering

Physics

Chemical engineering

Biochemistry

Computer science

Chemistry and Criminal justice

Biochemistry and Molecular biology

3
1
3
1
1
1
1
1

Table 3.1: Majors of the learning assistants interviewed

3.4.2 Data collection

The participants in this study were 12 learning assistants from P Cubed, who had taught

for at least one semester over the 7 semesters of the course’s existence at the time of this

study. They varied in major, class rank, gender, and semesters of experience teaching in P

Cubed, shown in Tables 3.1, 3.2, 3.3, and 3.4. They also had a wide variance in personal

experiences with teaching and computation outside of P Cubed. These differences led us to

believe that this would be a productive population for conducting a phenomenographic study,

which depends on identifying variations as described previously [54, 137, 158, 159]. In order

to conduct a phenomenographic analysis of the learning assistants’ approaches to teaching

computational problems, we investigated their experiences via interviews. It is worth noting

that the interviewer had a personal familiarity with the participants, as she had been a

graduate teaching assistant in P Cubed and had worked with each of the participants. The

interviewer strove to minimize any bias this might result in during the interviews and ensuing

analysis.

The development of the protocol for these interviews and the interview process will be

described in the following sections.

46

Class rank Number
Freshman
Sophomore

Junior
Senior

0
2
4
6

Table 3.2: Class ranks of the learning assistants interviewed

Gender Number
Female
Male

7
5

Table 3.3: Genders of the learning assistants interviewed

Semesters of experience Number

1
2
3
4
5
6

2
3
2
2
2
1

Table 3.4: Semesters of experience teaching in P Cubed of the learning assistants interviewed

47

3.4.2.1 Interview protocol development

In line with the goals of a phenomenographic study and with our phenomenon of interest,

approaches to teaching computational problems, we developed a semi-structured interview

protocol aimed at probing the learning assistants’ experiences teaching computational prob-

lems [159, 167–169]. The semi-structured nature of the protocol allowed for the interviewer

to follow emergent threads that were salient to the participant, while also being sure to get as

complete a picture of the individual’s experience as possible. The questions on the protocol

were developed based on several considerations. Of course, the majority of the questions

directly probed their experiences and opinions on teaching computational problems in P

Cubed. In addition to these questions, however, questions on the participants’ backgrounds,

opinions on the course structures (feedback, collaboration, et cetera), and general opinions

on teaching and learning were included. These were included in order to better contextu-

alize their responses regarding the computational problems, and to also provide them with

additional spaces to discuss computation and computational problems if they chose. The

protocol underwent two rounds of revisions based on two pilot interviews. These revisions

primarily involved restructuring the protocol in order to better follow conversational flow,

and adding the questions regarding general opinions on teaching and learning, which had

not been present in the initial version of the protocol. The interview protocol is presented

in Appendix C.

3.4.2.2 Conducting interviews

The interviews were conducted over the course of approximately one year, based on the

availability of the interviewer and participants. Participants were solicited by contacting

current and former P Cubed learning assistants, inviting them to participate and explaining

48

the goals of the study, the nature of the interviews, and the compensation offered (a $10 gift

card). The interviews were conducted one-on-one between the interviewer and each learning

assistant, and lasted approximately one hour and fifteen minutes. The interviews were open

to following topics that the participants brought up spontaneously, and the questions on the

interview protocol were not all explicitly asked in each interview or asked in the same order.

The goal of the interviews was to develop as rich an understanding of the learning assistants’

experiences as possible, so the primary goal throughout each interview was to provide the

participants with as many opportunities to share their experiences in the ways that were

most salient to them [159, 167–169]. The interviews were audio and video recorded, and the

audio recordings were sent to an external company for transcription. Once the transcripts

were received and reviewed for accuracy, analysis began.

3.5 Analysis

The following section describes the analysis procedures carried out in developing the cat-

egories of description identified in this study. Phenomenographic analyses require several

iterations and modifications [54, 137, 158, 159], and so an abridged accounting of the pro-

cess, highlighting key steps will be presented here. For a complete description of the analysis,

see Appendix A.

3.5.1 Data reduction

Analysis began with reviewing all the transcripts several times in order to become familiar

with the data set as a whole. After gaining familiarity with the full data set, the data

was reduced based on relevance to the phenomenon of interest. Because the interviews

49

were open to following tangents, and because several background questions were asked (see

Appendix C), there were portions of the transcripts that were not relevant to the participants’

experiences with teaching computational problems. For example, participants may have

discussed their motivations for becoming learning assistants, or described their teaching

explicitly with respect to analytic problems.

In an example of the latter, one learning

assistant spoke about teaching the analytic problems, which occur on Tuesdays,

On Tuesday, the first question I’ll ask them is, “What do you ultimately need
to get? Where is your end goal?” They say, “Oh, this is our end goal.” “Okay,
what are the steps you need to take to get that?” That’s on a Tuesday.

Portions of transcript such as these were neglected for the thematic analysis described next

since they were not directly relevant to our phenomenon of interest, but they were retained

in order to provide fuller context for participants’ comments if necessary.

3.5.2 Identification of preliminary themes

Once identified, the reduced data set was analyzed for emergent themes in participants’ dis-

cussion of teaching computational problems. These themes represented trends and common

threads in what the participants spoke about over the course of the interview. Some of these

themes corresponded to questions that were directly asked by the interviewer, while others

did not.

In either case, the themes were identified based solely on the comments of the

participants in the transcripts, and the researchers strove to analyze the transcript without

bias towards any particular themes emerging. This process resulted in a preliminary set of

seventeen emergent themes shown in Table 3.5. These themes do not account for all of the

reduced data set. Some relevant portions of transcript were too unique or specific to be

50

Preliminary Themes

Differences between analytic and computational teaching

Written feedback

Varying levels of student programming experience

Different student approaches to computation

Differences in student behavior solving computation versus analytic problems

What students should do when solving computation problems

Acceptable places to end computation days

Minimally working programs

Solution manuals

Why code in groups
Student challenges
Teaching challenges

Teaching physics versus teaching coding on computation days

In-class teaching strategies

Important things for students to take away

Purpose of computation

Student frustration

Table 3.5: List of preliminary emergent themes identified in the data

sorted into a theme. Examples include a learning assistant speaking on very specific role

that he plays as computation expert on computation days, or extra preparation that a learn-

ing assistant engages in before she teaches computational problems. While these comments

were interesting, they were not further analyzed. Since the goal of this study, and of any

phenomenographic study, is to find the most concise way to capture the different ways that

individuals may experience a phenomenon [54, 137, 158, 159], it is expected that not every

relevant comment from the participants will be identified as belonging to a theme.

It is also worth noting that these themes are not necessarily mutually exclusive, and

that there are portions of transcript that were coded as belonging to multiple themes. This

is because in a given comment, a participant often discusses several dimensions of their

experience at once. Thus one would not expect, and it is not a requirement that the themes

be wholly orthogonal.

51

Teaching Outcome Variations

Variation

Programming skill

Physics-code connection

A new approach to learning

Capabilities of computation

Description

a familiarity with the syntax and

structures of code

an understanding of the relationship
between the computational problems

and the physics concepts

new perceptions and beliefs about

learning

an appreciation of the usefulness of

computation

Table 3.6: Variations within the teaching outcome theme, describing learning assistants’
goals for their students

3.5.3 Analysis of preliminary themes

Once preliminary themes had been identified, individual themes were selected and exam-

ined in more detail to identify the variations in how the learning assistants discussed them.

This analysis began with the investigation of the themes that were most richly populated,

and which had the strongest apparent tie to our phenomenon of interest. These themes

were in-class teaching strategies, differences between analytic and computational teaching,

and teaching physics versus teaching coding on computation days since all three had explicit

references to how learning assistants teach in the classroom, along with important things for

students to take away since it appeared to be related to learning assistants’ motivations for

their teaching. Through the process of identifying emergent variations within these prelim-

inary themes, they were ultimately reorganized into three new themes: teaching outcome,

teaching strategy, and characteristic to moderate. These three themes and their respective

variations are shown in Tables 3.6, 3.7, and 3.8.

Teaching outcome refers to what a learning assistant hopes their students gain from

working on the computational problems as a result of their teaching. Teaching strategy

52

Teaching Strategy Variations

Variation

Focus on navigating
programming errors

Encourage reflection on coding

Leverage collaboration
Leverage affordances of
computational problems

Description

identify and help students overcome

programming errors

promote deeper thought on the

function of computation

stimulate group discussion of ideas

utilize elements unique to

computational problems in teaching

Table 3.7: Variations within the teaching strategy theme, describing the techniques learning
assistants use while teaching

Characteristic to Moderate Variations

Variation

Description

Student attitudes

Student attention to
programming details
Student work pace

Impact of course design

prevent students from becoming too

unhappy

avoid students focusing on coding

idiosyncracies

ensure students complete the activity
mitigate the effect of the limitations of

course structures

Table 3.8: Variations within the characteristic to moderate theme, describing what aspects
of a group’s experience a learning assistant feels they must monitor and influence

53

refers to the techniques that a learning assistant may employ when teaching. Characteristic

to moderate refers to what on-going element of students’ experiences during class the learn-

ing assistant feels they must attend to and keep within certain bounds. The preliminary

themes were restructured in this way because these themes better described the underly-

ing features that emerged from the data. While the preliminary themes attended to the

surface commonalities among the learning assistants’ comments, focusing on the content of

their statements, the themes shown in Tables 3.6, 3.7, and 3.8 are better grounded in the

substantive differences in the levels of considerations the learning assistants’ discussed.

The next preliminary themes analyzed were the purpose of computation theme, because

it was hypothesized that a learning assistant’s perception of the usefulness of computation

would have a notable effect on their teaching, and the student frustration theme because it

was suspected that it may be accounted for already by the student attitudes variation of the

characteristic to moderate theme. This was found to be largely correct, and the majority of

the portions of transcript in the student frustration theme not identified as student attitudes

were nonetheless accounted for by variations within the already identified themes. The

variations ultimately identified within the purpose of computation theme appear in Table

3.9, retitled as utility of computation.

3.5.4 Development of categories of description

The final steps of developing the four themes presented thus far were informed by developing

preliminary categories of description. The goal of a phenomenographic study is to develop

categories of description that capture the different ways individuals may experience the phe-

nomenon of interest [54, 137, 158, 159]. With this goal in mind, we considered the variations

seen in the themes analyzed thus far. This was done at this point in analysis because

54

Utility of Computation Variations

Variation

Description

Programming is an important

programming is a significant part of

skill

modern society

Computation aids content

computation helps students better

learning

understand physics concepts

Computation makes difficult

computation is valuable in modeling

problems easier

and solving complex problems

Computation offers space for

broader skills

computation provides opportunities to

develop skills valuable beyond the

classroom and computation

Table 3.9: Variations within the utility of computation theme, describing what learning
assistants perceive the usefulness of computation to be

the preliminary themes analyzed thus far indicated a rich variation in experiences teaching

computational problems and provided a great deal of information with which to develop

preliminary categories of description. Furthermore, the data analyzed thus far represented

a great majority of the reduced data set to be considered. Based on these considerations, it

was decided that it was an appropriate point at which to begin developing categories of de-

scription by looking across themes. This stage of analysis constituted a secondary thematic

analysis within the bounds of the already identified themes, where we now analyzed the vari-

ations for emergent threads that described coherent approaches to teaching computational

problems, and a set of preliminary categories were developed. The development of these

categories aided in the finalization of the themes and variations in Tables 3.6, 3.7, 3.8, and

3.9, which in turn informed the refinement of the categories of description. This mutually

informative process is described in detail in Appendix A, and resulted in the categories of

description presented in the Results section of this chapter.

Having arrived at a set of coherent categories of description, the final step in analysis

was to review the remaining preliminary themes in Table 3.5 and confirm that they were

55

accounted for within the four themes identified thus far, or that they could justifiably be

neglected. Several of the remaining preliminary themes were already accounted for within

the themes in Tables 3.6, 3.7, 3.8, and 3.9 upon closer analysis. Specifically, it became

evident that some preliminary themes had emerged based on characteristics that were su-

perficial in light of the deeper themes and categories that had developed. The written

feedback, varying levels of student programming experience, acceptable places to end compu-

tation days,minimally working programs, solution manuals, and teaching challenges themes

were largely already described by the themes in Tables 3.6, 3.7, 3.8, and 3.9. A majority

of the portions of transcript that had initially been sorted into these preliminary themes

were determined to be simply context-specific discussions of learning assistants’ teaching.

An example appears in the quote below,

[Using minimally working programs] impacts my teaching a lot because I feel
like I only really focus on the things I teach them how to code, like understand
the code, or more like the while loops. I teach them about what’s already there
and it’s not like I’m starting like this is what code is, and this is how you code
things, like a coding class would teach you. And so you can focus specifically on
while loops and if statements, which really limits the amount of coding you’re
necessarily teaching them and you can focus more on the physics side of it.

While this learning assistant discusses minimally working programs in this excerpt, he

ultimately is discussing that he seeks to avoid programming becoming a distraction. He

emphasizes how he teaches computational elements like while loops and if statements, but

does not get into the details of programming “like a coding class would”. In this way, he

expresses an attention to ensuring that those details do not become the students’ focus. Thus,

this excerpt is accounted for within the characteristic to moderate theme, specifically the

student attention to programming details variation. The majority of the transcript sorted into

the aforementioned preliminary themes was similarly better captured by the final themes.

56

Other preliminary themes were eliminated because they were found to not be central

to our phenomenon of interest:

learning assistants’ approaches to teaching computational

problems. The different student approaches to computation, differences in student behavior

solving computation versus analytic problems, what students should do when solving compu-

tation problems, why code in groups, and student challenges themes were neglected because

upon closer analysis, it was observed that while they did relate to learning assistants’ per-

ceptions of the computational problems, they did not provide insight into their approaches

to teaching. In the student challenges, different student approaches to computation, and dif-

ferences in student behavior solving computation versus analytic problems themes, learning

assistants simply shared their observations of students, but did not connect these observa-

tions to their teaching. In the why code in groups theme (and the portions of the written

feedback, varying levels of student programming experience, minimally working programs, and

solution manuals themes that were not already accounted for by the final themes), learning

assistants shared their opinions on the design of P Cubed, but again, did not connect these

opinions to their teaching and did not elaborate sufficiently for their comments to give insight

into their approach to teaching. In the what students should do when solving computation

problems theme, learning assistants discussed the ideal problem-solving approach to compu-

tational problems, but did not describe why they held that belief or how it impacted their

teaching. In all of these cases, interesting information about the learning assistants’ percep-

tions of computational problems was revealed, but this information was not productive in

describing their teaching approach.

For example, in the student challenges theme, one learning assistant discussed a problem

that requires the students to model the motion of a rotating disk, in which the given code

defines the y-axis to be the axis of rotation. She notes that this causes confusion for the

57

students, because in analytic problems, the z-axis is often taken to be the axis of rotation.

I think, I mean like, I feel like they are going to get confused about anything
new, but that one, I know there was a lot of confusion with the different things.
There was an axis and the origin, and so I know I had discussions with both my
groups about what those meant and think it’s also confusing because it’s like in
the z-axis, but I think on the code it’s on the y-axis, something like that. So, I
don’t know, all of those changes, kinda I think make those a little bit difficult to
understand.

The learning assistant’s discussion of how students struggle with coordinate systems in

the computational problems gives some insight into her opinions on computational problems

and the students, but it does not give insight into how she approaches her teaching overall.

While she does state that she “had discussions” with her groups about this particular issue,

it doesn’t reveal her teaching strategy or teaching outcome, and it doesn’t give information

that is helpful in identifying the variations in learning assistants’ teaching approaches.

Having thus confirmed that the themes and variations in Tables 3.6, 3.7, 3.8, and 3.9

were sufficient to meaningfully describe the learning assistants’ experiences as they related

to their teaching approaches, and that cohesive categories of description based on groupings

of variations within them could be developed (described in the Results section), analysis was

concluded.

3.5.5 Validation

In addition to the processes described here, results were subject to validation throughout

analysis and upon completion. The entire analysis process was conducted in regular con-

sultation with a researcher with expertise in phenomenography. All analysis decisions were

subject to scrutiny and discussion, and had to be adequately justified through evidence

58

from the transcript and defensible arguments. This included decisions to merge themes or

variations, eliminate themes or variations, and especially decisions on the final groupings of

variations defining each category of description. In addition to this on-going validation, at

several intermediate points in analysis, preliminary results were presented to external groups

for critique and feedback. Finally, upon the conclusion of analysis, the final results, the in-

terview transcripts, and examples of coded transcript were provided to another researcher

for external validation of the results. Based on the evidence and arguments provided in this

Chapter and in Appendix A, and the validation of these external researchers, we argue that

the results presented in the following section constitute a novel and legitimate phenomeno-

graphic result that gives meaningful and useful insight into learning assistants’ approaches

to teaching computational problems.

3.6 Results

3.6.1 Categories of description

In phenomenographic studies, the categories of description are formed by looking across the

themes identified in the data, seeking to identify groupings of variations that describe a co-

hesive way to experience the phenomenon of interest [54, 137, 158, 159]. Four categories of

description capturing the differences in learning assistants’ approaches to teaching computa-

tional problems were developed based on our data. They are summarized in Table 3.10 based

on the variations that compose them, and are described in detail in the following section.

59

Category of
Description

Utility of

Computation

Teaching
Outcome

Characteristic
to Moderate

Narrow

Programming

programming

is an

focus

important skill

Learning
physics via
computation

focus

Computation
as a tool focus

Computation
aids content

learning

Computation
makes difficult

problems

easier

Shifting

perceptions of
learning focus

Computation
offers space for
broader skills

A new

approach to

learning

Programming

Student work

skills

pace

programming

errors

Leverage

Physics-code
connection

Impact of

course design

affordances of
computational

Student

Capabilities of
computation

attention to
programming

details

Student
attitudes

Teaching
Strategy
Focus on
navigating

problems

Encourage
reflection on

coding

Leverage

collaboration

Table 3.10: Final categories of description. The narrow programming focus treats the com-
putational problems as a way to develop students’ abilities to write code. The learning
physics via computation focus treats the computational problems primarily as a means for
learning physics concepts. The computation as a tool focus treats the computational prob-
lems as a way to help students understand the value of computation in modeling and solving
problems. The shifting perceptions of learning focus treats the computational problems as
opportunities to develop students’ understandings of their own learning.

60

3.6.1.1 Narrow programming focus

The narrow programming focus category is based around an emphasis on computation as the

act of entering code. There is little reflection on how computation may be used in teaching

or science, rather it is treated as a valuable but unconnected skill.

In the utility of computation theme, it is characterized by the belief that programming is

an important skill to have in today’s society, with very little elaboration or explicit reflection

on why this is. For example, one learning assistant said,

[Programming] has been really helpful in a lot of my different classes, and
then also in my internship this last summer, just like as an engineering person in
general. That’s one thing that I try and stress to the students, is it’s like, “Okay,
you might think that you’re never going to use this, but it’s actually so helpful
if you do know how to use even just a small portion of it.”

In this quote, the learning assistant describes how she tries to convey to her students

that programming is helpful, noting that she has used it in multiple contexts. She discusses

how she has used it inside and outside of the classroom, and references her background as

an engineer. She does not elaborate on its importance in her learning or work, however, and

she does not describe how or why it is helpful. Instead, she only states that students are

likely to encounter it in other situations and that it can help them. This lack of reflection on

how computation is helpful and relevant is an example of a learning assistant expressing a

perception that the utility of computation is simply that programming is an important skill.

In the teaching outcome theme, the narrow programming focus is characterized by an

emphasis on programming skills. That is, the learning assistant desires their students to gain

an understanding of how to carry out programming tasks. One learning assistant said,

61

If it’s something where they added an object, or they were talking about an
object, or coded in an object in their calculation loop, but never defined that
object, then that’s a place where it’s like, “Okay, let’s kind of go through this
more line by line. What is this line telling us? Okay, where else does that object
show up in the code? Wait, it doesn’t show up anywhere else? We might need
to address that.” Because that is an understanding thing where they need to
understand that they have to define something in the code before they can do
anything to that object. You can’t do something to an object that you don’t
have.

In this quote, the learning assistant discusses students facing an error resulting from

referencing an object that they had not yet defined in their code, and states that is an error

that she will intentionally target with her teaching. She goes on to say that she does this

because the students “need to understand” the process of defining and calling objects in

programming. Expressing a belief that her teaching should result in students knowing how

to carry out elements of programming such as this indicates that programming skills are her

desired teaching outcome.

In the characteristic to moderate theme, the narrow programming focus is characterized

by student work pace. When moderating this characteristic, learning assistants attend to

their students’ rate of progress, seeking to ensure that students reach the end of the problem

by the end of class. An example of this is given by a learning assistant who said,

I might let them just mess with it for awhile. But it’s sort of a time balance
thing. So I’ve done these problems so many times, I know where they should be
at certain times to make sure they finish, and so I usually let them mess with
it until I think, “Okay, it’s halfway through, they should be at this point. I’m
gonna try to redirect them back to where they should be.”

With this quote, the learning assistant indicates that regardless of the exploration she’ll

sometimes allow her students to do, she will take steps to ensure that they are able to finish

the problem in the allotted time. She explicitly comments on using particular time markers

62

to check on where students should be at different points in the class session. This emphasis

on making sure students complete the problem indicates that she moderates students’ work

pace.

In the teaching strategy theme, the narrow programming focus is characterized by a focus

on navigating programming errors. With this strategy, learning assistants focus their teaching

on helping students identify and overcome programming errors in the traditional sense. That

is, mistakes in their code that cause it not to function. One learning assistant employing

this strategy stated,

Most of the time, I just teach them how to do it because it’s usually when
they’ve just like edited like one line of code, and then it’s like, “Oh, we have the
tabbing error.” I’ll just be like, “Here’s how you solve that: Highlight, and then
do the thing, and then, yay, it’s good.” Then they’ll be like, “Okay. Cool. Now
I know how to do this in the future.”

In this quote, the learning assistant describes a common error the students encounter:

problems with their indentation. She explains that she tailors her teaching to this fact by

preemptively instructing students on how to resolve that error. This emphasis on helping

students handle errors of this nature demonstrates a teaching strategy that is focused on

navigating programming errors.

When considered together, one can see how these variations come together to form a

cohesive approach to teaching computational problems, and one that is focused on a narrow

view of programming. With the belief that computation is essentially entering programming

syntax, it follows that a learning assistant’s desired teaching outcome would be those sorts

of programming skills. Similarly, it follows that a learning assistant with this approach

would employ a teaching strategy focused on navigating programming errors, because those

errors represent the fundamental obstacles of computation when one views computation to

63

be correctly entering code. The narrow programming focus corresponds to a belief that

the utility of computation is simply that programming is an important skill. This aligns

with a narrow understanding of what programming is, because without understanding there

to be deeper connections between computation and science or learning, the usefulness of

computation in P Cubed would simply be that students learn how to program. Finally, this

approach is completed by a learning assistant moderating the pace at which their students

progress through the problem. This corresponds to a narrow view of programming focus

because again, without a reflection on computation as a pedagogical tool or a problem-

solving tool, and with programming being the act of entering code, the “game” that students

must play on computational days becomes merely to get properly working code. With this

belief in mind, it follows that a learning assistant would want to ensure that their students

complete the problem and arrive at a correct solution. All four of these variations therefore

come together to illustrate an approach to teaching computational problems characterized

by a narrow programming focus.

3.6.1.2 Learning physics via computation focus

The learning physics via computation focus is characterized by an emphasis on the compu-

tational problems providing a pedagogical tool that learning assistants may leverage in their

teaching. Computation is perceived primarily as useful in the classroom to help students

learn concepts.

In the utility of computation theme, the learning physics via computation focus is char-

acterized by the belief that the value of computational problems is that they aid students’

content learning. One learning assistant expressing this belief said,

64

The students get to see it represented so it’s kind of like a reward almost for
writing out the code. They think if I do this it changes this and it’s beneficial
to see how ... For example, we did a problem with drag. So, it’s good to see
how drag affects this and if we increase the coefficient it’ll do this and this. It’s
a good way they can input information and immediately see the results of how
the physics is applied.

In this quote, the learning assistant expresses that he believes the usefulness of the

computational problems to be their ability to help students better understand the concepts.

Specifically, he discusses how students can use the immediate feedback that computation can

provide as a means to see the impact of the different physics concepts they are learning, in

this case, drag. This type of emphasis indicates a belief in the utility of computation being

that it can aid students’ content learning.

In the teaching outcome theme, the learning physics via computation focus is character-

ized by an emphasis on the physics-code connection. This refers to a desire for students

to see and understand how the code and visualizations involved in computation relate to

the physics concepts highlighted in the course. In one example of this teaching outcome, a

learning assistant said,

Actually understanding what you’re coding and making sure you can tie it
back to the physics. Like I said when you do the position update formula, you
should be able to understand that that comes from this kinematic equation. You
should understand how it relates and how it should affect it.

In this quote, the learning assistant describes what she wants her students to learning from

the computational problems. She discusses that the students should be able to understand

how particular lines of codes, like the position update formula, relate back to the physics

equations they are familiar with in algebraic form. With this, she expresses that she wishes

her students to gain an understanding of the physics-code connection.

65

In the characteristic to moderate theme, the learning physics via computation focus is

characterized by a focus on the impact of course design. Moderating this characteristic means

attending to the particularities of the way P Cubed is taught and incorporates computation,

and seeking to minimize the negative impact of their limitations. Examples include the fact

that students must share one laptop, that not all necessary information is always provided

in the problem statements, and the availability of the four quadrants whiteboard for work

organization. A learning assistant discussed this, saying,

We have to consider who’s taking over the group, like typing in by themselves,
or maybe if somebody ... Like, if people have taken turns, but somebody hasn’t
typed in yet, then we do have to take into account that the code is something
they should work on specifically, like working together with the code.

In this quote, she discusses the impact of students needing to share a single computer

while solving the computational problems, which require code to be entered into the com-

puter. She notes how individuals will sometimes dominate a group and do all the coding,

and how other individuals may never get a turn to enter the code themselves, and how that

dynamic is something learning assistants must monitor and address in their teaching on the

computation days. Thus in this quote, she expresses that she moderates the impact of course

design on computational days.

In the teaching strategy theme, the learning physics via computation focus is characterized

by leveraging the affordances of computational problems. This refers to a learning assistant

utilizing aspects unique to computation days in their teaching. An example of leveraging

such an aspect is discussed by a learning assistant who said,

I’ll be like, “it’s acting this way, why is it acting that way?” And when the
students realize that, that’s not possible, I’ll start questioning things that they’re

66

missing. And if they see what they’re missing, then they try to add that into
the problem. This, once again, goes into pseudo code, because if they see what
they’re missing, I usually tell them to write it out first, so that they don’t make
those errors.

In this quote, the learning assistant describes how he will encourage his students to

write pseudocode when they encounter difficulties on computation days. By pseudocode,

he is referring to an approximation of what the desired code is, not yet worrying about

entirely correct syntax. He explains that he will have students write this almost-code out

on their whiteboard when they are having trouble getting it to work in the computer. As

pseudocode is an attempt at writing information in a way the computer will ultimately be

able to understand, it is a construct uniquely relevant on computation days. In this way, this

learning assistant discusses employing a teaching strategy which leverages the affordances of

the computational problems.

Considering these four variations together, there is a coherent approach to teaching com-

putational problems that emerges that is centered around a focus on computation as a means

of learning physics. With this emphasis on computational problems as a pedagogical tool, it

is consistent that a learning assistant would view the utility of computation to be its ability

to aid students’ content learning. It also follows that a learning assistant with this approach

would employ teaching strategies which leverage the capabilities of computation, given that

they focus on computational problems as a means for students to learn physics concepts. The

desired teaching outcome in the learning physics via computation focus is an understanding

of the physics-code connection, that is, an understanding of exactly how the computational

work relates to the physics concepts being taught. It is unsurprising that a learning assistant

with this approach would want their students to understand the physics-code connection,

because they view the computational problems as fundamentally being a venue for students

67

to learn the physics concepts. With that view, it is understandable that they would want

their students to explicitly understand how the code and visualizations present in the compu-

tational problems are connected to the physics. Finally, the characteristic moderated in this

approach is the impact of course design based. Because the learning physics via computation

focus as a whole is very focused on computation as a pedagogical tool and is grounded in the

context of education, it is consistent that a learning assistant with this approach would pay

attention to the classroom in which they are teaching, particularly the limitations and affor-

dances that it presents, resulting in a focus on moderating the impact of the course’s design

in their teaching. This combination of four variations together characterize an approach to

teaching computational problems that focuses on students learning physics via computation.

3.6.1.3 Computation as a tool focus

The computation as a tool focus is an approach to teaching computational problems that em-

phasizes the usefulness of computation as a method of approaching problems. Computation

is treated as a tool that is valuable in mathematical and scientific problem-solving.

In this utility of computation theme, the computation as a tool focus is characterized

by the belief that computation makes difficult problems easier. It is a perception that com-

putation provides affordances that are an important part of modeling or solving complex

problems. One learning assistant expressing this belief said,

What makes the computer good is that it involves ... Like a big thing is the
time step. The smaller you make that time step, the more accurate everything
is. If you wanted to write and draw that out, you could, but you’d have to use
a grid system. You could. I don’t know how much that would actually help or
prove.

In this quote, the learning assistant discusses that computers can be used to carry out

68

iterative numerical calculations, and that it is simple to control how many iterations the

computer calculates by modifying the time step (the interval between each calculation). He

contrasts that with how difficult it would be to do this analytically. He explicitly identifies

the ability to do an increasing number of calculations easily as “what makes the computer

good”. By articulating and emphasizing how computation can be utilized to carry out

difficult calculations, he indicates a belief that the utility of computation is that it makes

difficult problems easier.

In the teaching outcome theme, the computation as a tool focus is characterized by an

emphasis on students appreciating the capabilities of computation. With this outcome, a

learning assistant wants their students to understand the benefits that computation provides.

This is closely tied to the aforementioned belief that the utility of computation is that it

makes difficult problems easier. While computation making difficult problems easier describes

what the learning assistant perceives the usefulness of computation to be, the capabilities

of computation describes what they want their students to takeaway from their experience

with computation. An example of this appears from a learning assistant who said,

The computation is important, it’s an important skill, but understanding how
the computation is helping them solve physics is always what I think about the
most. The computer is solving all these equations and updating all this stuff,
so what does that mean on pen and paper, and what does that mean for the
physics?

In this quote, the learning assistant discusses how she focuses on how computation helps

students solve the physics problems. She explicitly states how this is the important part of

computation, and how it is more important than just the skill of being able to program. She

expresses that what she wants students to understand how exactly it is that the computer

69

is helping them. In this way, she suggests that the desired teaching outcome she has for her

students is that they gain an understanding of the capabilities of computation.

In the characteristic to moderate theme, the computation as a tool focus is characterized

by student attention to programming details. This means an attention to preventing students

from becoming too caught up in the details of syntax and coding, to the detriment of their

understanding the larger picture of computation. One learning assistant moderating this

characteristic said,

When it’s just a spacing error that they’ve been working on for ten minutes,
that’s not super productive. And hopefully they just call me over when that
happens, but I’m sure there are times when they’re just messing with an error
for 15 minutes, or ... probably not that long. Hopefully not to that point, but
where they still have no idea how to fix it, and it’s not helping them learn the
physics, and it’s not really helping them learn coding, ’cause they’re just trying
to find this tiny error.

In this quote, the learning assistant describes a difficulty students often have during the

computational problems: getting their code properly spaced and tabbed. She discusses how

spending too much time on this sort of error is not a good use of the students’ time, explicitly

noting that it does not help them learn the physics concepts or coding. She also expresses

that she hopes she is able to intervene before the students waste too much time on a task

like this. By treating the navigation of this sort of error as not aligned with learning coding

and by indicating that she does not want students to spend time on these types of issues,

she indicates a desire for students to not focus on unimportant programming details to the

neglect of other computational skills. In this way, she expresses that student attention to

programming details is a characteristic she moderates.

In the teaching strategy theme, the computation as a tool focus is characterized by en-

couraging reflection on coding. This means prompting students to think more deeply about

70

how their code functions and how they can best utilize the program. A learning assistant

discussing this strategy said,

I would get them to focus on talking about how the code is supposed to be
iterative. It’s supposed to do the calculations for you. You shouldn’t be doing the
calculations outside of it and inputting numbers because then you’re constraining
the code to specific parameters, [but] the code is very versatile. You’re supposed
to be able to just hit run, change things to see how changing the variables affects
the system.

In this quote, the learning assistant discusses how he will encourage his students to talk

about what the code is able to do, such as carrying out calculations and allowing one to keep

parameters variable. He emphasizes that the code is “very versatile” and that this should be

leveraged, and he describes how he encourages his students to discuss these matters. This

focus indicates a teaching strategy that is based on encouraging students to reflect on coding.

These four variations considered at once illustrate a coherent approach to teaching com-

putational problems that is based around a focus on computation as a tool. Given a focus on

computation as a tool, it follows that this teaching approach is characterized by a belief that

the utility of computation is its ability to make difficult problems easier when leveraged, as

this represents an example of its application as a tool. It also follows that a learning assistant

with this approach would want their students to understand the capabilities of computation

that make it such an important tool. In order to achieve this desired teaching outcome, it

is consistent that a teaching strategy focused on encouraging reflection on coding would be

employed. Prompting such reflection would facilitate students developing an appreciation

for the affordances that computation provides them, and this emphasis makes sense given a

focus on computation as a tool. The characteristic to moderate in this approach is student

attention to programming details, which is also consistent with a belief that computation is

71

a valuable tool. A learning assistant with this approach perceives computation to be more

than simply entering syntax, instead treating computation as an important tool that can be

utilized in modeling and solving difficult problems. Correspondingly, they do not wish their

students to become bogged down in the details of syntax and leave with the impression that

that is all that computation is. Rather, they want students to gain an appreciation of the

greater value of computation. In this way, these four variations come together to indicate

an approach to teaching computational problems that is focused on computation as a tool.

3.6.1.4 Shifting perceptions of learning focus

The shifting perceptions of learning focus is characterized by an emphasis on computational

problems as an opportunity to affect students’ beliefs about and approaches to learning.

Computation is viewed as more than a pedagogical or problem-solving tool, and is instead

treated as a means to impact students’ ongoing ideas about how they learn and approach

problems.

In the utility of computation theme, the shifting perceptions of learning focus is charac-

terized by the belief that computation offers a space for broader skills to be developed. These

skills could include how to collaborate effectively, how to think in new ways, and how to

persevere through challenging problems. One learning assistant with such a belief said,

That’s [the computation problems] definitely where I see them learn a lot of
effective ways to work in a group or in a team, which that’s a big part of this
class. Going forward, they definitely will need to know how to do that.

In this quote, the learning assistant describes how she has observed that the computation

days are where the students most learn how to collaborate. She goes on to state that this

is an important skill for them to know beyond the course. With this, she indicates a belief

72

that the computational problems’ utility is their ability to offer students a space to develop

broader skills that go beyond physics and computation.

In the teaching outcome theme, the shifting perceptions of learning focus is characterized

by a new approach to learning. This refers to a desire for students to develop a new approach

to their learning through the computational problems. A learning assistant with this goal

for their teaching said,

I was trying to get them to see that, “Okay, if these two conditions are
conflicting, then that statement will never be true, so that segment of code will
never run.” [...]
If it’s high up, then that just stops the code. Trying to get
them to think through it logically rather than just like, “Okay, well let’s just
type something in and see if that changes anything,” so stepping back and going,
“Okay, well we know that this is true, we know this, and let’s just look at this
from a logical physics perspective rather than a coding perspective.”

In this quote, the learning assistant describes a situation in which her students’ code

was not running because they had entered conflicting conditions. She explains how she

wanted her students to think “logically” through the code and error, rather than simply

making changes arbitrarily, demonstrating a desire she had for her students to take a more

intentional and methodical approach to their problem-solving. This sort of focus on changing

how students approach their learning and problem-solving is indicative of a new approach

to learning to being her intended teaching outcome.

In the characteristic to moderate theme, the shifting perceptions of learning focus is

characterized by student attitudes. When moderating this characteristic, a learning assistant

focuses on ensuring that students do not become too frustrated or unhappy while working

on the computational problems. A learning assistant moderating this characteristic with her

teaching said,

73

If they say something like “this is so frustrating, like I just want to give up,”
then you’ll see them kind of start to get disengaged I guess. Like while you’re
doing the trial and error, usually the whole group is kind of looking at what’s
going on and is involved and wants to fix it and make it work. When you see
them start to sort of you know, fall back and give up essentially then that’s kind
of where you want to make sure that you get them on the right track I guess.

In this quote, the learning assistant describes how she identifies when students have

become frustrated to the point of no longer participating. She notes that at the beginning of

problems, when groups are typically testing what different changes to the code do, students

are generally engaged, but that as the problem progresses, frustration can lead to students

becoming disengaged. She highlights that it is important to make sure that students do

not get to that point of frustration. This focus on preventing students from becoming too

demoralized indicates that she perceives student attitudes to be an important characteristic

to moderate.

In the teaching strategy theme, the shifting perceptions of learning focus is characterized

by leveraging collaboration. When employing this strategy, learning assistants capitalize on

the fact that the course is a collaborative learning environment, and that students work in

groups. One learning assistant discussing this teaching strategy said,

I might say something like you know, ask somebody, ask a group what they
are doing and if someone responds and it looks like the other two aren’t paying
any attention, I might ask, “Oh, are you guys good with that?” or like “Are you
guys on the same page?” or “Do these guys understand that?” or something like
that to sort of let them know that they should be conversing.

In this quote, the learning assistant describes how she will ask questions of a group that

are targeted at assessing if there is agreement amongst the students and if all the students

understand the group’s work. She explains that she does this to convey to them that they

should all be discussing the problem and concepts with each other as they work. This

74

focus on encouraging students to take advantage of the fact that they are working in groups

by having more discussions is an example of employing a teaching strategy that leverages

collaboration.

In combination, these four variations describe an approach to teaching computational

problems that is focused on computation as a way to shift students’ perceptions of learning.

With this view of computation in mind, it follows that the desired outcome of a learning

assistant’s teaching would be for students to adopt a new approach to their learning from

the avenue for this that computation provides. It also is consistent that a learning assistant

focused on computation as a way to shift learning perceptions would believe the utility of

computation to be that it provides a space for students to develop skills that are beyond

content. Skills like collaboration and intentionality when problem-solving are tied to how

students’ approach and perceive their learning, and so a learning assistant with a focus on

shifting perceptions of learning would emphasis computation’s utility in developing those

skills. The shifting perceptions of learning focus is characterized by a teaching strategy that

leverages collaboration. This strategy is also aligned with this focus because by leveraging

collaboration, a learning assistant utilizes an aspect of the learning environment that is

specifically designed to alter how students approach their learning and problem-solving.

This strategy thus implicitly addresses how students approach their learning, and continues

the focus on computation as a way to shift perceptions of learning. The characteristic to

moderate corresponding to this approach is student attitudes. In order to attempt to shift

students’ perceptions of learning through the computational problems, it is unsurprising that

a learning assistant would attend to student attitudes, and in particular, make sure students

did not become frustrated to the point of disengagement. If a learning assistant is focused

on shifting students’ perceptions of learning, it is understandable that they would believe

75

that engagement and a positive attitude are necessities to being able to influence students

in this way. Thus these four variations when considered together comprise an approach to

teaching computational problems that focuses on their ability to shift students’ perceptions

of learning.

These four categories of description capture the variation in how the learning assistants

described their approaches to teaching computational problems in this environment. As

described previously, however, categories of description are only part of the results of a

phenomenographic study. The categories must also be connected via some logical structure

known as an outcome space [54, 137, 158, 159]. The outcome space relating the categories

presented in this study is described next.

3.6.2 Outcome space

The four approaches to teaching computational problems identified in this study are con-

nected in a hierarchical structure, shown in Figure 3.1. Going from the narrow programming

focus to the shifting perceptions of learning focus, each approach represents an increasing

level of sophistication. While sophistication may be defined in many ways, here we take a

sophisticated approach to teaching to be one that indicates greater reflection on the purpose

of one’s teaching and students’ learning.

With the narrow programming focus, computation is treated primarily as a means to learn

programming skills, such as syntax and elements like loops. These skills are not framed as

being connected to students’ learning or their perceptions of computation or learning, and

instead are treated as things that are simply good to know with little reflection on why. By

neglecting to reflect in this way on what computation is and what it offers students, such an

approach indicates a relatively unsophisticated understanding of the purpose and usefulness

76

Figure 3.1: Outcome space indicating the hierarchical structure relating the identified learn-
ing assistant approaches to teaching computational problems

of teaching computational problems in the course, and a very narrow goal for one’s teaching.

In the learning physics via computation focus, the computational problems are treated as

pedagogical tools. They are framed as contexts that offer particular opportunities to develop

students’ understandings of the physics concepts. This approach is more sophisticated than

the narrow programming focus because it indicates some reflection on the ways that the

computational problems can aid students’ learning beyond simply gaining programming

proficiency. It demonstrates an attention to the fact that the computational problems are

about physics concepts and are embedded in a physics course, and an intentionality about

leveraging that in one’s teaching. This approach does not expand beyond the classroom or the

course’s subject, however, to consider the broader value and opportunities the computational

problems provide.

77

The computation as a tool focus represents such an expanded focus, by treating the

computational problems as an opportunity for students to better understand the value of

computation outside of the classroom. Rather than seeing computation as a mechanism to

help students understand the physics concepts that they are meant to learn in the course,

the computational problems are framed as a space for students to gain an appreciation

of the power of computation in solving complex problems. This approach indicates an

increased level of sophistication from the learning physics via computation focus because it

goes beyond the physics content the course explicitly addresses, and attends to the impact

that the computational problems can have on students’ perceptions of computation itself. As

the context of this study was a physics course, it is to some degree a given that developing

students’ understanding of physics is a central part of teaching the course. Because of

this, taking an approach to teaching that is centered around the learning of physics is not

as indicative of additional personal reflection as taking an approach that goes beyond the

course’s explicit subject matter. The computation as a tool focus represents an approach

that indicates such additional reflection.

The shifting perceptions of learning focus is more sophisticated still, as it treats the com-

putational problems as a venue in which to impact students’ perceptions of and approaches

to their own thinking, learning, and problem-solving. Rather than focusing on the classroom-

style learning of physics concepts or even students’ general perceptions of computation, this

approach frames the computational problems as a chance to alter how students go about

learning new things regardless of the context. This represents the most sophisticated ap-

proach of the four, as it aims to make a lasting change in how students approach problems

beyond the bounds of the classroom and computation.

In this way, the four approaches to teaching computational problems identified through

78

the learning assistants in this study represent an increasing level of sophistication of ap-

proach. Progressing up Figure 3.1, each approach indicates learning assistants engaging in

deeper reflection on and possessing a broader understanding of how they might utilize the

computational problems in their teaching. In the next section, we will discuss the meaning

and implications of the approaches identified in this study and the hierarchical structure

relating them.

3.7 Discussion

There were four themes that were salient when considering the teaching approaches the

learning assistants interviewed for this study took to teaching computational problems: their

perceptions of the utility of computation, their intended outcome for their students, what

characteristics they moderated while teaching, and the strategies they employed while teach-

ing. Based on learning assistants’ beliefs about each of these themes, we identified four

approaches to teaching computational problems: a narrow focus on the act of programming,

a focus on learning concepts through computation, a focus on computation’s position as a

valuable tool, and a focus on shifting students’ perceptions of their own learning. Each of

these approaches represents an increasing level of sophistication in approach to teaching,

and furthermore, indicates varying degrees of alignment with goals of the course design of P

Cubed. In its intended design, P Cubed does not emphasize the acquisition of simple pro-

gramming skills as a goal, and thus the narrow programming focus is not very aligned with

course goals. Naturally as a physics course, developing students’ conceptual understanding

of physics is an intended outcome of P Cubed, and the learning physics via computation

focus is in line with this. P Cubed has intended outcomes for students beyond conceptual

79

understanding, however. It is a goal of the course to frame computation as a valuable skill

and tool in approaching problems, and to encourage students to reflect on their learning,

problem-solving, and collaboration [32]. These goals appear in the two most sophisticated

approaches identified: the computation as a tool focus and shifting perceptions of learning

focus. The approaches to teaching computational problems that we identified are therefore

not entirely surprising, as they suggest learning assistants are taking up different goals of

the course to different degrees.

As described previously, the learning assistants interviewed for this study came from

diverse majors and backgrounds with respect to computation and teaching, and also had

varying levels of experience teaching in P Cubed. This led us to believe we would uncover

meaningful variations in their approaches, and the development of the categories of descrip-

tion and outcome space presented here confirms that this was the case. After developing

these results, the interview of each learning assistant was considered individually, in order

to identify with which approach to teaching each was most aligned. An individual’s tran-

script was examined for comments regarding each theme, and for which variation within

each theme these comments reflected. Based the variations an individual most reflected in

each theme, they were assigned to the teaching approach that was most aligned with their

combination of variations. Because the teaching approaches identified in this study were

not developed with the aim of unambiguously categorizing these learning assistants, most

individuals did not align perfectly with a single teaching approach, and instead expressed

elements of multiple teaching approaches. Learning assistants were therefore assigned to a

teaching approach based on the one that best accounted for the variations they expressed

in each theme. Table 3.11 shows the number of learning assistants aligned most with each

approach, and the number of semesters that each had taught in P Cubed.

80

Teaching approach

Number

Experience levels

Narrow programming focus

Learning physics via computation focus

Computation as a tool focus

Shifting perceptions of learning focus

1
4
1
6

5 semesters

1, 2, 3, and 3 semesters

6 semesters

1, 2, 2, 4, 4, and 5 semesters

Table 3.11: Number of learning assistants with each teaching approach and their experience
levels

The results of this sorting are interesting for several reasons. First, the majority of the

learning assistants take either the learning physics via computation focus or the shifting per-

ceptions of learning focus. These two approaches have less explicit focus on computation

than the narrow programming focus and computation as a tool focus themes, perhaps indi-

cating that learning assistants are likely to view the computation as secondary to learning,

whether it be conceptual learning or reflection on learning. Another notable result is that

there does not appear to be a correlation between amount of experience and teaching ap-

proach. The two most populated approaches had comparable distributions of experience,

and the only learning assistant with the least sophisticated approach was one of the most

experienced, with five semesters of experience. This result suggests that it is unclear what

effect experience in the classroom has on the teaching approaches that learning assistants

take. Investigating the teaching approaches that learning assistants discuss over the course

of their tenure, or before and after training could shed light on what causes shifts in their

approaches. Using the teaching approaches identified here as a lens to consider the changes

or lack of changes in their approaches could inform for how we train and support the develop-

ment of our learning assistants as teachers. For example, the pre-semester training currently

does not discuss the computational problems or their purpose explicitly, and modifying the

training to address this could encourage learning assistants to adopt the more sophisticated

approaches. Doing this could ensure that the intended course goals and messages regarding

81

computation are reaching the students. We also investigated if there were correlations be-

tween the approaches learning assistants took and their major, class rank, or gender, and

found no correlations.

The increasingly sophisticated approaches to teaching presented here have connections to

previous research on approaches to learning, and previous research on instructors’ beliefs and

practices in teaching [87, 98, 100, 153, 170–172]. Several studies in particular have notable

connections to our results.

In their work on learning assistants’ beliefs about teaching and learning, Gray et al.

examined a context that did not involve computational problems, but their results have

a relationship with ours nonetheless. They found that learning assistants may perceive

themselves as occupying four different roles: facilitating physics content, facilitating students’

sense-making, shaping the learning environment, and professional growth [87]. The first two

roles identified in their work are aligned with the learning physics via computation focus, in

that they frame physics content in some way as the primary goal. Their identification of

two versions of a focus on physics content could be related their context, in which students

work primarily on tutorials. Tutorials are structured and ask direct questions, which could

prompt learning assistants to engage in simple content transmission so that students arrive at

those answers. The problem-based learning style of P Cubed might make such an approach

difficult to implement, since students must navigate complex problems that may not have

clearly defined questions. As a result, it’s possible the learning physics via computation

focus identified here is more comparable to their facilitating students’ sense-making role.

The shaping the learning environment role they identified may be related to the shifting

perceptions of learning focus, as both contain a focus on the collaborative nature of the

learning environments.

In the shaping the learning environment role, however, it is not

82

clear if the learning assistants want their students to adopt a new learning style as in the

shifting perceptions of learning focus, or if they simply see themselves as a moderator of group

dynamics. The professional growth role identified by Gray et al. did not appear in our data.

This could be because we did not probe this area of their experiences in our interviews, and it

could also be because the P Cubed learning assistant model does not explicitly emphasize its

role in the development of new teachers in the way that the University of Colorado Boulder

(where Gray et al. conducted their work) learning assistant model does [84].

Our results are also related to the work by Bruce et al. on students’ experiences learning

programming. Although they focused on student experiences, and the work presented here

focuses on instructor approaches, parallels can be seen between the results of each. In their

phenomenographic study, Bruce et al.

identified five experiences that students may have

with respect to programming: following, coding, understanding and integrating, problem-

solving, and participating, with each representing a student taking on a more expansive

view of programming[153]. Both the “following” and “coding” categories are related to the

narrow programming focus, as they all focus on simply accomplishing the computational

task and gaining some degree of proficiency with writing code. The “problem solving”

category is aligned with the computation as a tool focus, as both treat programming as

a way to approach problems. Because their work was carried out in a context explicitly

teaching programming, while ours was carried out in a physics course which incorporated

programming, other connections between the results are less clear. The “understanding

and integrating” category may be related to the learning physics via computation focus as

both focus on gaining a conceptual understanding of the course’s content, whether it be

programming itself or physics. The “participating” category may be related to the shifting

perceptions of learning focus, as both describe a broader change in how students perceive

83

their work, gaining an understanding of programming as a community in the case of the

former, and gaining an understanding of a new approach to learning in the latter. While

the relationship between our results and those of Bruce et al.

is not entirely clear, these

connections could provide some insight into how learning assistants come to their teaching

approaches. Because learning assistants experience the class first as students themselves,

and because little time passes between their experiences as students and their experiences

as instructors, it could be that their student experiences are having a significant impact on

their teaching approaches.

The results presented here are closely related to the phenomenographic study by Mag-

ana et al. on instructors’ intended outcomes when teaching computation. They found eight

different intended outcomes that instructors may have in incorporating computation into

their courses: to become aware of the role of computation in science, to make measurements,

to find causal relationships in models, to test models, to validate results, to use computa-

tional techniques, to predict behavior, and to discriminate among possible models. They

further argued that these eight intended outcomes represent increasingly higher order levels

of thinking [100]. It is difficult to match these to our results precisely, since their results

describe desired outcomes, which is just one of the four themes that comprise each of the

teaching approaches identified in our work. Despite this, there are relationships that can be

uncovered. The “become aware of the role of computation in science” outcome is aligned

with the computation as a tool focus, as both emphasize students gaining an understanding

of the usefulness of computation. The “use computational techniques” outcome is aligned

with the narrow programming focus in that both emphasize students gaining programming

skills. It is difficult to determine how their remaining identified outcomes relate to the teach-

ing approaches we identified. It could be that the instructors they interviewed have these

84

intended outcomes towards the goal of students better understanding the concepts of their

scientific discipline, which could align with the learning physics via computation focus. It

could also be that they have these intended outcomes towards the goal of showing students

specific computational capabilities, which could align with the computation as a tool fo-

cus. None of the intended outcomes identified by Magana et al. appear to have an explicit

relationship to the shifting perceptions of learning focus. It’s also worth noting that their

hierarchy of instructors’ intended outcomes based on higher orders of thinking does not align

with our hierarchy based on increasing levels of sophistication in approach. For example,

while they position “become aware of the role of computation in science” as the lowest order

level of thinking, we position the computation as a tool focus as one of the more sophisti-

cated approaches to teaching. These seemingly contradictory rankings may be a result of

our teaching approaches being described by elements beyond the learning assistants’ desired

outcomes that when considered together, provide a picture of an approach that has more

sophistication.

There are limitations to our results. The approaches to teaching computation identified

here are based on learning assistants teaching in a problem-based course, and one that puts

significant time and effort into their training. Additionally, the course implements computa-

tional problems in an intentional way and has particular goals for students’ engagement with

those problems. This likely affected the experiences the learning assistants discussed and the

approaches to teaching that they adopted. Indeed, as previously described, the approaches

identified here are closely related to the course’s intended goals, and it is possible that in a

course emphasizing different goals or implementing computation differently, different teach-

ing approaches may be identified. A different set of learning assistants could discuss new

themes that were not present in these interviews, and they could also discuss the themes

85

identified here in different ways, resulting in new variations. Because of this, the themes and

variations upon which the approaches to teaching identified here were built do not account

for all learning assistant experiences, though our results still provide a framework that would

be useful when examining the experiences of new populations. For example, Michigan State

University now offers a degree in computational mathematics, science, and engineering, and

it is possible that a learning assistant from that program would have unique ideas about the

purpose of computation instruction in a science course. We have had no learning assistants

yet from this major, but if any do join the teaching team, it would be interesting to compare

their comments to the approaches found here. The themes and variations identified here also

do not fully describe the experiences of the learning assistants in our context. As a method,

phenomenography seeks to build the most concise categories that capture the variations un-

covered in the experiences of the participants. This means that not every element of each

participant’s experience is captured in the categories, because it is the crucial variations in

experiences that are the focus of analysis. While this does result in the loss of some of the

nuance in each individual’s experience, it is also what lends phenomenography its strength

as a method for finding commonalities and variations in the experiences of a collective. It is

worth noting that phenomenographic studies typically use sample sizes of 15-20 participants

[169], and that 12 learning assistants were interviewed for this study. While we do not be-

lieve that interviewing more learning assistants from P Cubed would drastically change our

results, it is possible that such additional data could provide some refinement.

Despite these limitations, the work presented here provides valuable insight into learning

assistants’ experiences teaching computation. The identification of the increasingly sophis-

ticated teaching approaches identified here provides a framework that can be used to un-

derstand what course goals our learning assistants are taking up, and could also be applied

86

to investigate how their approaches develop over time or to determine if similar teaching

approaches emerge in other contexts. As more physics courses incorporate computational

problems into their curricula, and as more courses shift towards collaborative learning envi-

ronments that may require teaching to be delegated to learning assistants, it will be impor-

tant to understand the approaches that instructors take to teaching in these environments.

Better understanding these approaches and how to influence them can allow us to best

support instructors as they facilitate students’ learning in these complex environments.

87

Chapter 4

Identification of a shared

answer-making epistemic game in a

group context

The following chapter was published in the proceedings of the 2015 Physics Education Re-

search Conference [56], and is presented here with minor modifications from its appearance

in publication. It was published with second and third authors Paul W. Irving and Marcos

D. Caballero.

4.1 Abstract

When physics students engage in collaborative exercises, they must negotiate their different

problem-solving strategies in order to work together effectively. One lens through which to

understand these interactions is the construct of “epistemic games”. These constructs have

been used to describe particular methods of problem solving with which students are observed

to engage. In prior work, an “answer-making epistemic game” has been observed, wherein

the student perceives the objective of the activity as producing an answer, and reasons until

they arrive at an answer or intuits an answer and then tries to justify this answer. This game

88

was observed in the context of individual students working independently on multiple-choice

questions. We present the identification of a shared answer-making epistemic game when a

group of students worked collaboratively on conceptual problems.

4.2

Introduction

The use of collaborative work is becoming increasingly prevalent in introductory physics

classrooms. In such environments, students must engage with not only physics content, but

also with their peers and their peers’ understanding of physics content and problem-solving

strategies. In order to best provide learning opportunities, it is essential to understand how

students interact in such environments. To this end, the construct of epistemic games has

been used as a productive lens through which to analyze students’ behavior when problem-

solving [27–29]. An epistemic game is defined by Tuminaro and Redish as “a coherent activity

that uses particular kinds of knowledge and processes associated with that knowledge to

create knowledge or solve a problem” [28].

The answer-making epistemic game (AMEG) is relevant to the work presented here [29].

In this game, a student works under the belief that the objective of the activity is to produce

a solution. The AMEG begins with students engaging in the entry condition of attempting to

remember the answer to the question on which they are working. If they cannot remember

an answer, they will attempt to intuit an answer. At this point, students will enter one

of two possible paths consisting of different series of moves.

If the student can intuit an

answer, they will accept this answer, and then attempt to use conceptual reasoning or do

math in order to arrive at a justification for the answer (the “answer justify” path, AJ).

If a student cannot intuit an answer, they will reason conceptually or do math until they

89

arrive at an answer (the “reason answer” path, RA). Both paths are defined by the same

exit condition: a solution that the student finds to be sufficient. These series of moves and

entry and exit conditions describe the structure of the AMEG. The AMEG is further defined

by an ontology comprised of its epistemic form, a truth table, and its knowledge base, the

physics and mathematical resources that students activate as they engage with the game.

The AMEG was studied in the context of think-aloud interview protocols carried out

with individual students working on multiple-choice hydrostatics questions. In this paper,

we identify via a case study a shared answer-making epistemic game (SAMEG) in a group

context in which four students worked together on a conceptual electricity and magnetism

(E&M) problem. The students in this group were observed to engage with the activity in

ways that appeared similar to the AMEG, but that could not be fully accounted for by that

epistemic game. The model of the SAMEG was developed in order to attempt to describe

more fully the behavior of this group. As such, we do not propose the prevalence of the

SAMEG, and instead only present its existence through the case study of this group.

4.3 Study context

The data presented in this paper was collected from an introductory E&M course at a large

university. There were approximately 120 students in the course, and most were sophomore

life-science majors. The students all attended lecture three times a week, and a laboratory

session once a week in sections of approximately 20 students. In these laboratory sessions,

students worked in groups of three or four collaboratively on a variety of activities depending

on the week (labs, tutorials, conceptual and calculational problems, etc.). The data presented

here are transcripts from video recorded of one group of four students working during their

90

Figure 4.1: A schematic of the shared answer-making epistemic game showing how students
progress through the game

weekly laboratory section. In the episode, the students are attempting to identify if there

is a point of equilibrium in the space surrounding an electric quadrupole, and if so, where

that point is. They do so by examining an image of the space surrounding the charges as

represented by electric field vectors. It is worth noting that the students are not told that

such a configuration of charges is called a quadrupole, and that they had not seen such a

configuration previously in class. The students in question have been given the pseudonyms

Buster, George, Lindsay, and Michael. The data will specifically regard George, Lindsay,

and Michael, as Buster quickly disengages with the activity. He leans back from the table,

does not speak, and does not often look at the students who are speaking.

4.4 Shared answer-making epistemic game

As illustrated in Figure 4.1, three students participated in the SAMEG, while one was not

observed to participate. In these observations, the SAMEG was characterized first by an

entry condition, general group discussion. After entering the SAMEG, the participating stu-

91

dents played the original AMEG individually. One student played RA, while the other two

students played AJ. After the participating students exited their individual AMEGs, they

engaged in a group negotiation of their responses. The first time they attempted this, they

were unsuccessful, and so the their next moves were to return to their individual AMEGs

(dashed lines in Figure 4.1). The participating students played their individual AMEGs a

second time, exited them, and then again attempted a group negotiation of their responses.

This time they were successful, and so they exited the game. The exit condition was thus all

participating students being simultaneously satisfied with their individual solutions and also

having successfully navigated a group negotiation of their responses. We detail the compo-

nents of the SAMEG below, providing evidence from video data of the four aforementioned

students working on this problem.

4.4.1 Entry condition

The participating students began playing the SAMEG through the entry condition of a

general group discussion. All four students’ first actions were to discuss together what the

problem was asking of them and to begin proposing ideas. The students were observed to

lean over the image and speak in approximately equal amounts about the image. No answers

were put forth, and no student offered reasoning directly leading to producing an answer.

This stands in contrast to the original AMEG, which is characterized by the entry condition

of remembering or intuiting an answer [29]. By the nature of the problems these students

were asked (i.e., the students had never seen the given charge distributions), remembering

was unlikely to occur and was not observed. Intuiting answers was also not observed to be

any of these students’ first actions.

As they begin working, all four students leaned in towards the image at the same time,

92

and began gesturing simultaneously. They took turns identifying charges (00:02-00:18),

Michael: So clearly, right here is a negative.

George: [Pointing at another charge] Positive.

Lindsay: [Pointing at different charge] This is positive.

By the student responses, we can interpret that no student remembers or intuits an answer

to the problem, and no student reasons conceptually in a way that is directly related to

answering the problem.

Instead, the students are speaking in equal amounts about the

problem; trying to understand the image they have received – a general group discussion.

4.4.2 Individual AMEGs

After the students entered the SAMEG by engaging in the entry condition, they entered a

sequence of particular moves. In the first of these moves, the students were observed to play

the original AMEG on an individual basis (thin solid lines in Figure 4.1). During this time,

the students still spoke and responded to each other while they engaged independently in

the AMEG. The intuit, answer, conceptual reasoning, and sufficient solution moves were all

present. These moves were observed in the context of individual students playing the RA or

AJ paths.

It is at this point in the episode that one student, Buster, disengaged from the group.

He leaned back in his seat, and did not speak until the very end of the group’s work on

the problem. Two students, Lindsay and George, played the AJ game by first intuiting

answers, then using conceptual reasoning until they found their solutions sufficient. Having

not provided any explicit reasoning first, Lindsay points at the image (00:23-00:29) and says,

93

Lindsay: So this is the place right where you could put the charge. Because these
[electric field vectors] cancel.

Similarly, at (00:25), George agrees with Lindsay’s answer without explicitly demonstrating

conceptual reasoning first. Instead, it is after affirming an answer that he reasons (00:36-

00:38),

George: It’s like tic-tac-toe.

where it is interpreted that he is referring to the electric field vectors. The third student,

Michael, played the RA game by using conceptual reasoning first until arriving at an answer

comprising a solution he found sufficient. While leaning over the image (00:30-00:36) he

says,

Michael: [Pointing at electric field vectors] One in, one out, one in, one out. [Now
pointing at center of quadrupole] Yeah.

He then leans away from the image and begins to write on his worksheet.

The presence of both the RA and AJ paths suggests this portion of the students’ behavior

was in fact an individual playing of the original AMEG. They approached the problem in

different ways, independent of one another’s choice of pathway. Furthermore, the students

did not progress through the moves of the original AMEG at the same rate and did not

arrive at sufficient solutions simultaneously. Lindsay found her solution sufficient first at

(00:31), followed by Michael at (00:36), followed by George at (00:39), as determined by the

point at which students stopped speaking and began writing on their worksheets. This also

points to the students playing the AMEG as an individual activity, because they determined

the sufficiency of their answers on an individual basis.

94

4.4.3 Group negotiation of their responses

In the SAMEG, once all participating students in the group completed playing their indi-

vidual AMEGs, they engaged in a group negotiation of their responses. In this negotiation,

students discussed their individual solutions in an attempt to agree upon a final response

to the problem that all group members found acceptable. They did not seek to agree upon

the specific wording of a response. Instead they assessed first whether each student had a

solution that she personally found sufficient, and then that no individuals’ solutions were

contradictory. The students in this group were observed to carry out a group negotiation of

their responses twice (described further in the next section). The data in this section will

regard the group’s second, and successful, attempt at this negotiation (thick dashed line in

Figure 4.1).

After the students (except Buster, who continues to not engage with the group) complete

their second passes through the original AMEG, Lindsay asks Michael (04:10-04:15),

Lindsay: [addressing Michael] So you’re saying it’s going to be between two. . .

Michael: Either two negatives or two positives.

At this point, Lindsay begins writing – an action that we have used to indicate that she finds

her solution sufficient. Michael and George continue discussion (05:30-05:48),

Michael: It’s two like charges will cancel each other out.

George: You have a positive here and then it [the test charge] wants to go here
and here, and it doesn’t know where to go.

Michael: Yeah, you’re correct.

95

At this point, George and Michael begin writing, indicating that they find their solutions

sufficient, and Lindsay begins reading the next problem on the worksheet.

In this exchange, the students determined that they all had solutions that they individ-

ually found sufficient. They further assessed whether their solutions align. In this way, they

engaged in a group negotiation of their responses.

4.4.4 Disruption and repetition of the individual AMEGs

The data presented in the previous section describes the students’ second, and successful,

attempt at a group negotiation of their responses. The group negotiation of their responses

is not trivial, however, and so a disruption caused students to struggle with this negotiation

on their first attempt. The data presented in this section will regard the students’ first

attempt at a group negotiation of their responses (thick solid line in Figure 4.1) and their

following moves. The disruption was a series of statements made by a student that caused

all group members participating in the SAMEG to become dissatisfied with the individual

solutions they had arrived at following their individual AMEGs. When the disruption oc-

curred, students sought to again find solutions they personally found sufficient, and did so

by returning to their individual AMEGs (thin dashed lines in Figure 4.1).

The students (with the exception of Buster who still was not engaged) were observed

to move through the original AMEG a first time individually and arrive at solutions they

personally found sufficient, seen in all three students quietly writing on their worksheets.

After a few seconds of this silence, however, Michael stops writing (00:52-00:57),

Michael: Oh actually, will a charge remain where we placed it if it’s. . . oh yeah
but. . .

96

At this point, George and Lindsay stop writing, and begin watching Michael. The three

students now return individually to the original AMEG path they had played before. For

example, after his speech and discontinuing his writing, Michael, who played RA path in his

first pass, reasons conceptually for the duration of (02:06-05:08), saying things like (03:56-

04:06),

Michael: So it’s [a charge] pushing away from it [the test charge]. . . and this
negative is going to be attracting it [the test charge], so it’s going to get pushed
in that direction.

before proposing another answer at (05:08), suggesting that he again followed the RA path.

Similarly, Lindsay and George were observed to follow the AJ path as they had on their first

individual passes through the original AMEG.

By this series of events, we can interpret that Lindsay, Michael, and George became

dissatisfied with their individual solutions as a result of Michael’s comment, then returned

to their individual AMEGs. The presence of both the RA and AJ paths suggests that this

is in fact a repeated playing of the original AMEG on an individual basis. Additionally,

the three students did not find their new solutions sufficient simultaneously, as determined

by being observed to write quietly. Lindsay found her solution sufficient first at (04:16),

followed by Michael at (05:08), followed by George at (05:22). This further suggests that

they were playing the original AMEG individually following the disruption.

4.4.5 Exit condition

The exit condition for the SAMEG is, like the AMEG, a sufficient solution [29]. Due to

the group context of the SAMEG, this manifested as all students having solutions that they

97

individually found sufficient. The group context of the SAMEG further adds the necessity

of the group negotiation of their responses. As described previously, there was a point at

which all three students found their solutions sufficient, but did not exit the SAMEG due

to the disruption caused by Michael. Instead, the group only began working on the next

problem, evidenced by Lindsay reading it aloud for the group, after they had completed

the group negotiation of their responses successfully. Thus, in order for the students to

exit the SAMEG, it was necessary for them to all have sufficient solutions and also to have

successfully completed the group negotiation of their responses.

4.4.6 Ontology

In the previous sections, we discussed the structure of the SAMEG. The ontology of the

SAMEG is described by its knowledge base and its epistemic form [28]. The knowledge base

of the SAMEG is the set of physics resources that the students activated as they worked on

the problem. The epistemic form of the SAMEG is a group negotiated worksheet response.

The students’ actions throughout the SAMEG were directed by the need to complete the

worksheet that they had been assigned together. This negotiated worksheet response pro-

vided external structure that guided the students as they played the SAMEG [27, 28].

4.5 Discussion and conclusions

The context of this study undoubtedly played a role in the identification of the SAMEG. Due

to the collaborative nature of the environment, the students needed to navigate their differ-

ent and possibly opposing problem-solving strategies in order to successfully complete the

assignment, and what we observed was the emergence of a shared epistemic game, comprised

98

of both individual and group moves.

The particular students analyzed in this case may have had an impact on the appearance

of the SAMEG. In this group, Michael disrupted the first attempt at a group negotiation of

their responses. It is possible that another group may not face such a disruption, and there-

fore not repeat their individual AMEGs. The particular students may have also impacted

the appearance of the SAMEG via Buster’s low level of engagement. The group of four

students functioned largely as a group of three. It is unclear what impact this may have had

on the actions of the three participating students and how the SAMEG then manifested.

As epistemic frames and games are related, it is worth noting that the SAMEG appears

to occur within the “discussion frame” [30]. Throughout the episode, the students spoke

in clear voices towards each other, maintained eye contact, and responded to each other’s

comments, indicating that they all worked under the assumption that their attention should

be on one another. The discussion frame is broad, however, and does not attend to group

members’ individual frames [125]. The SAMEG can allow for a description of individuals’

different frames within a shared, group-level discussion frame. Namely, it can be inferred

that students engaging in the RA path while participating in the SAMEG are not framing

the activity in quite the same way that students engaging in the AJ path are. While it is not

entirely clear what causes these alternative frames, the SAMEG nonetheless provides a lens

wherein individual students’ frames may be differentiated within a group-level discussion

frame, and such a lens can be productive in the analysis of the complexities of student

reasoning in collaborative learning environments.

99

Chapter 5

Development of a modes of

collaboration framework

The following chapter was published in Physical Review Physics Education Research [57],

and is presented here with minor modifications from its appearance in publication. It was

published with second and third authors Paul W. Irving and Marcos D. Caballero.

5.1 Abstract

Group work is becoming increasingly common in introductory physics classrooms. Under-

standing how students engage in these group learning environments is important for design-

ing and facilitating productive learning opportunities for students. We conducted a study

in which we collected video of groups of students working on conceptual electricity and

magnetism problems in an introductory physics course. In this setting, students needed to

negotiate a common understanding and coordinate group decisions in order to complete the

activity successfully. We observed students interacting in several distinct ways while solv-

ing these problems. Analysis of these observations focused on identifying the different ways

students interacted and articulating what defines and distinguishes them, resulting in the

development of the Modes of Collaboration framework. The Modes of Collaboration frame-

work defines student interactions along three dimensions: social, discursive, and disciplinary

100

content. This multi-dimensional approach offers a unique lens through which to consider

group work and provides a flexibility that could allow the framework to be adapted for a

variety of contexts. We present the framework and several examples of its application here.

5.2

Introduction

Interactive instruction is becoming increasingly common in introductory physics classes, with

more instructors implementing techniques such as small group discussions, group problem

solving, and team-based projects. These techniques have been found to be more effective by

some metrics [19, 21, 25, 48, 51], but a number of aspects remain ill understood [47, 52]. Much

work remains to be completed to better understand these interactive learning environments

and the effects they have on student learning [126, 127, 130].

Several lenses aimed at understanding various aspects of how students engage in such

work have been developed. Some seek to assess the impact of collaborative work on individual

student learning [19, 47, 48, 112]. Others attempt to understand the different ways students

may perceive group work [30, 120, 121, 125]. Still others aim to identify ways to optimize

group work [31, 129, 133]. Another broad area of investigation has endeavored to categorize

the ways that students engage with group work. Such work has approached this goal in a

variety of ways, including a particular focus on social aspects of group work [21, 119, 132],

discursive aspects of group work [118, 128, 129, 173], and framing aspects of group work [15,

112, 124, 130]. In this paper, we present a new framework called the Modes of Collaboration

that attends to three dimensions: social, discursive, and disciplinary content. We did not

make use of framing as a dimension directly, as we found that considering the ways in which

students discussed physics content was better able to capture our observations, however, this

101

dimension does have a relationship to framing (see Framework section).

By attending to all three of these dimensions simultaneously and independently, the

Modes of Collaboration is a framework that is simple to apply, but that still provides multi-

faceted insight into students’ engagement with group work. In addition to presenting the

framework here, we identify four specific Modes that students engaged in within our context

based on observation of a small set of video data from one day of class work. The Modes

of Collaboration framework presented here acts as a proof that student group work can be

described along the three dimensions, and we propose that it is flexible enough to be used

in other contexts beyond the one analyzed here.

5.3 Background

Group work has long been an area of investigation in physics education and education more

generally, and has been studied from many different perspectives. Some work focuses on

individual content understanding, typically using pre and post measures of individual learn-

ing to measure the impact of group work [19, 47, 48, 112]. Other studies have attended to

student perceptions of group work, usually through the observation of student behavior in

groups or through interviews with students, aiming to more qualitatively understand stu-

dents’ experiences [30, 120, 121, 125]. Research has also been conducted on ways that group

work might be optimized to achieve the best outcomes for students, for example, considering

which types of activities or which group compositions result in the greatest learning gains

for students [31, 129, 133]. Finally, a great deal of work has been done with the aim of

developing ways to categorize student engagement with group work. This has been done in

a variety of ways, including attending primarily to social dynamics [21, 119, 132], student

102

discourse [118, 128, 129, 173], or student perceptions and framing [112, 130]. Below, we offer

additional background on the work conducted along each of these lines to frame and situate

the Modes of Collaboration framework. While much of this previous work is not specific to

undergraduate students or to physics, it is nonetheless valuable in informing and providing

context for our work. Elementary school students and college students are of course different

in many ways, and even college math students and college physics students are different, but

themes found in one of these populations can still be productive when considering the other.

We describe some of these themes below, and demonstrate their utility in our context in the

Framework and Modes of Collaboration sections.

5.3.1 Individual content understanding

Much early research surrounding group work focused on identifying or assessing the impact

of collaborative work on individual students’ understanding of the content. In a study of high

school science students, Amigues compared the individual post-test performance of students

who worked on a preceding activity alone to students who worked on the preceding activity

in dyads, and found that the students who worked in dyads were more successful on the

post-test [112]. In a later study specific to college physics students, Heller et al. tracked how

students’ individual problem solving abilities developed over the duration of a semester-long

course that implemented collaborative group problem solving sessions, and found that their

individual problem solving abilities improved [25].

In efforts to similarly study the impact that group work has on individual student out-

comes, many studies have made use of concept inventories as a metric. These concept

inventories [16–18] are multiple-choice exams centered on a particular content area, which

students complete individually. While they are unable to capture many aspects of a student’s

103

experience, they have historically been used as an indicator of a student’s understanding of

the concepts probed. Lumpe et al. used the Photosynthesis Concept Test (PCT) to com-

pare the learning outcomes of high school students who completed a task on photosynthesis

individually to students who completed the same task in a group. They found that students

who had worked in groups were more successful on the PCT, but that not every member of

a given group experienced the same degree of improvement [21].

Studies such as these have been important in providing evidence of group work’s efficacy,

albeit as measured by metrics that have limitations [49, 115, 116]. What they cannot provide,

however, is an understanding of what happens during group work. By focusing on compar-

isons between students’ pre- and post- instruction understanding, they neglect to consider

the ways in which students interact and speak while engaging in group work; the social and

discursive dimensions. Furthermore, a strict focus on individual content understanding does

not account for the aspects of learning that may occur beyond purely cognitive models [174].

5.3.2 Student perceptions

In addition to identifying the ways in which group work impacts individual student content

understanding, efforts have been made to examine how students perceive group work. In

a study of elementary students working on a science task in groups, Anderson et al. con-

ducted observations and interviews to understand students’ goals and feelings of success or

lack thereof during the activity [120]. They defined three areas on which students may focus

their attention when working in groups: task structure and accountability systems, inter-

personal relationships, and scientific activity. Based on Anderson et al.’s observations and

interviews with students after the activity, they found that the students focused primarily

on interpersonal relationships and task structure goals, and that students did not appear

104

to focus as much on the scientific activity component. Additionally, Anderson et al. found

that students felt that they were successful at the activity overall. Grindstaff et al. also

examined students’ perceptions of collaborative work by interviewing students working on

short-term research projects with peers [121]. They found that students discussed several

types of support that peers may provide when working collaboratively: emotional, technical,

and cognitive, and that there was a great variance in which ones students reported as being

most relevant or important.

Student perceptions of group work have also been examined through the construct of

epistemological framing [15, 30, 123–125]. Scherr et al. developed this construct by building

upon previous work on framing in general, which they characterized as “how an individual

or group forms a sense of ‘What is it that’s going on here?’” From this, they narrowed to

examining how students frame activities specifically related to knowledge, and called this

epistemological framing [30]. In analyzing the discussion of undergraduate physics students

working on tutorials, they observed four behavioral clusters that they then mapped to differ-

ent epistemological framings. These frames were: discussion, worksheet, TA, and joking. In

another study of students working collaboratively, Irving et al. proposed an alternative way

to understand students’ epistemological framing [125]. In their work, they define two axes

with which to characterize students’ framing, rather than the discrete categories presented

by Scherr et al. One axis describes the degree to which a student’s statements are serious

or silly, and the other describes the degree to which a student’s discussion is narrow versus

expansive.

This type of work on students’ perceptions of the tasks and goals in group work and

their perceptions of group work itself provides a complementary perspective to the insights

gained by studies of individual student content understanding. While the latter attends to

105

outcomes, the former attends to the process and experiences of students. Studies of student

perceptions of group work frequently focus on social aspects, sometimes discuss discursive

aspects, and less frequently consider disciplinary content-related aspects of group work, but

rarely attend to all three simultaneously.

5.3.3 Optimizing group work

There have also been efforts made to identify the ways to best design and implement group

work to maximize its benefit to students. Heller et al. examined the effect that group makeup

had on the success of groups [31]. Their results indicated that heterogeneity with respect to

incoming ability produced the highest rate of success for groups. They also found that groups

that were homogenous with respect to gender, or groups in which there were more women

than men were most successful. In addition to examining group makeup, they investigated

the impact of giving students explicit guidance on how to work in groups. Students were

given one of three roles: manager, skeptic, and checker/recorder. They found that assigning

these roles reduced issues of individuals becoming too dominant in groups, or groups being

conflict-avoidant. They also found that giving students time to engage in explicit discussion

of their group’s interactions was beneficial. Van Boxtel et al. examined the impact that

having students complete individual preparatory work before working in groups had on their

success in those groups [129]. Their results indicated that the individual preparatory work

led to students asking each other more questions while working together, and improved

individual learning gains measured after the group work. Webb et al. sought to identify

the conditions that must be met in order for students working in peer-directed groups to

give and receive help in a productive way [133]. They propose that in order for received

help to be effective it must be relevant, timely, correct, and sufficiently elaborated. They

106

also identify three more conditions necessary for received help to be effective: the recipient

must understand the help, the recipient must have a chance to make use of the help, and the

recipient must act on that chance to use the help. The authors go on to use these conditions

to identify the responsibilities of the help-seeker, help-giver, and teacher in making these

conditions possible.

Studies seeking to optimize group work are essential to providing students with the

best learning experiences in collaborative environments. Their results can directly inform

instructional choices. Similar to the studies on individual content understanding, however,

they frequently compare students’ pre- and post- performance, and do not attend to the

processes that occur during group work. In doing so, they typically do not attend to the

discursive and social dimensions of group work.

5.3.4 Categorizing student engagement with group work

Another area of investigation has endeavored to categorize the various ways that students

engage with group work. Some of this work has focused primarily on the social dimen-

sion when developing categories. In the work by Lumpe et al. previously discussed, they

identified two general interaction styles that students may experience when collaborating:

consonant, or generally agreeable, and dissonant, or generally negative [21]. Roth et al.

also attended to social factors, and categorized the ways in which students may navigate a

disagreement, observing that they would proceed in several distinct ways: collaborative con-

struction, adversarial exchanges, and the formation of temporary alliances [132]. Richmond

et al.

identified the different social roles that students took on when working in groups.

They observed four social roles:

leader, helper, active non-contributor, and passive non-

contributor [119]. They further found that students taking on a leadership role would lead

107

in one of three styles: inclusive, persuasive, or alienating.

Others seeking to categorize students’ engagement in group work have done so by attend-

ing to the discursive dimension. Hogan et al. examined the ways that students speak to one

another when working in peer guided group discussions and teacher guided group discussions

[128]. They identified four modes of discussion: peer knowledge construction, teacher guided

knowledge construction, logistical conversation, and off task conversation. They found that

the relative occurrence of these modes varied greatly across groups. One hypothesis they

offered for this result was the teacher spending more or less time with a group depending

on their level of prior knowledge.

In the Van Boxtel work described earlier, the authors

identified several styles of student interactions based on their discourse: question, conflict,

and reasoning [129]. They found that questioning episodes occurred most frequently, and

that reasoning episodes were most likely to lead to elaboration of ideas. Haussman et al.

also made use a categorization scheme attending to discourse [118]. They analyzed three

proposed mechanisms of collaborative learning: other directed explaining, self directed ex-

plaining, and co-construction. The authors found that co-construction led to high individual

learning gains for both participants as measured by an individual post test, and self directed

and other directed explaining led to higher individual learning gains for the students giving

the explanations. Other directed explaining was more effective for the listener than self

directed explaining was for the listener.

Students’ engagement with group work has also been categorized by attending to their

perceptions of collaboration and the activity on which they are working. As described earlier,

both the work of Scherr et al. and Irving et al. set forward frameworks with which to classify

students’ behavior in groups based on their epistemological framing [30, 125].

All of these methods of categorization provide valuable frameworks to understand stu-

108

dents’ engagement with group work. Applying such categories can offer a way to make sense

of what students find important, how they perceive each other, and how they speak to one

another. As outlined though, these frameworks typically attend to only one dimension of

group work. A framework that categorizes students’ engagement with group work while si-

multaneously attending to discursive, social, and disciplinary content dimensions can provide

insights that focusing on only one may be unable to provide. The Modes of Collaboration

framework attempts to do just this, outlining several distinct Modes of student interaction

that are characterized by their discourse, their social interactions, and their engagement with

the disciplinary content.

5.4 Study context

The data presented in this paper were collected from an introductory electricity and mag-

netism course at a large university. There were approximately 120 students in the course,

and most were sophomore life-science majors. The students all attended lecture three times

a week, and a laboratory session once a week in sections of approximately 20 students. In

these laboratory sessions, students worked collaboratively in groups of three or four on a

variety of activities depending on the week (traditional labs, tutorials, conceptual and calcu-

lational problems, etc.). For this study, nine unique groups of students were video recorded

as they worked during their weekly laboratory sections, resulting in a total of 2 hours and

29 minutes of video.

The activity they are working on is a series of three conceptual questions about the

electric field and electric potential energy in the area around different distributions of charge.

Students would first read the problem statement without knowledge of the charge distribution

109

in question. They would then select as a group a representation of the charge distribution

(electric field lines, electric field vectors, or electric potential lines) to view, based on what

they thought would be most helpful in answering the problem. An instructor would then

bring them their requested representation, and the students would use it in order to answer

the problem. Each question had a well-defined correct answer, but could be solved using

multiple methods. For example, one might use the representation to deduce where the

source charges are located, then use this information to answer the question, or one might

use the representation directly to answer the question, without considering the location of

the source charges. This activity was similar to those they had experienced in previous

laboratory sessions in that it required cooperative group work on conceptual problems, but

was unique in that it required a level of explicit planning and strategy (in the selection of a

representation) that was not typically necessary.

5.5 Methods

Analysis began with multiple coarse viewings of the video data, attending to instances of

explicit interaction among students where their discussion related to the activity. An explicit

interaction is one that involves two or more students speaking, and an interaction that is

related to the activity is one wherein the content of the students’ speech was related to

the physics content present in the activity. Focusing on these segments reduced our data

to 1 hour and 11 minutes of video. Once these segments were identified, they were viewed

successively, seeking emergent trends in the student interactions and behavior observed. In

these emergent trends, distinct patterns were found that related to the three dimensions:

social, discursive, and disciplinary content. The data was then split into “episodes”, which

110

were defined by a shift in a group’s behavior along any of the three dimensions. For example,

in a given segment of video, if a group’s interaction with the disciplinary content appeared

to change, this would be considered the end of one episode, and the beginning of a new

episode. For each episode, the students’ interactions were characterized along each dimension

independently, and then episodes having all three dimensions in common were grouped and

termed Modes. Preliminary definitions for each Mode were crafted based on exemplars, then

refined through multiple viewings of every illustrative episode.

Similar to the work of Hogan et al., our procedure of analysis was not predetermined,

instead emerging from our observations, and quantitative inter-rater reliability in the identi-

fication of Modes or dimensions was not our goal [128]. Our goal was to craft definitions for

the Modes emerging from this data set that qualitatively described the speech and behavior

seen in each Mode. The first author’s knowledge of the data made her best suited to making

such identification and analysis decisions, and the co-authors provided critical feedback on

these decisions throughout the development of the framework. Through iterating on this

process, we arrived at robust Mode definitions that were able to capture the commonalities

seen in each instance of each Mode, and also accounted for the differences seen among Modes.

5.6 The Framework

The Modes of Collaboration are defined along three dimensions: social, discursive, and

disciplinary content. Each individual Mode is characterized in a particular way within each

of these dimensions.

111

5.6.1 Social dimension

The social dimension addresses the overall tenor of the students’ interactions with one an-

other. It accounts for the atmosphere in which the students’ conversation takes place. To

describe the social dimension, we make use of the interaction styles identified by Lumpe and

Staver [21]. In their work, they observed that groups of students would interact in ways that

were consonant or dissonant. Consonant interactions were characterized by agreement among

students and the validation of peers’ ideas, while dissonant interactions were characterized

by conflict (taken here to mean explicit disagreement), a lack of recognition, or criticism of

peers’ ideas. It is important to note that consonant interactions are not necessarily better

than dissonant interactions. For example, respectful critiques of one another’s reasoning

can lead students to a more robust understanding. Furthermore, in selecting this scheme of

categorization for the social dimension, we do not claim that it fully describes the richness of

students’ social interactions. Rather, the assessment of consonant versus dissonant provides

a simple and productive way to determine the general tone of a group’s discussion, which is

what we define the social dimension of the Modes of Collaboration framework to be.

5.6.2 Discursive dimension

The discursive dimension deals with the way in which students communicate with each other.

It describes the ways that students present their ideas and the structure of their conversa-

tion. The discursive dimension is grounded in Hogan’s work on knowledge construction and

Toulmin’s work on argumentation [128, 175]. In their work, Hogan et al.

identified three

modes that described the interactions of their students: knowledge construction (peer or

teacher-guided), logistical, and off task. Knowledge construction refers to when the con-

112

versation is related to scientific phenomena, logistical refers to when students discuss tasks

necessary to complete the activity but not related to scientific content, and off task refers

to when students discuss things unrelated to the task entirely. Since we sought to develop a

framework that describes students’ interactions when they are discussing physics, the Modes

of Collaboration all occur within the knowledge construction mode identified by Hogan et

al. We therefore could not use their three modes as a scheme with which to further analyze

the student interactions we were interested in, but we did make use of an element of their

analysis. In developing their three modes, Hogan et al. identified three interaction patterns.

These interaction patterns were consensual, responsive, and elaborative.

Consensual interactions are those in which only one student makes substantive contri-

butions, while other students simply agree, accept (explicitly or passively), or repeat the

contributions of that student. Responsive interactions are those in which multiple students

make substantive statements. Elaborative interactions are those in which multiple students

make substantive statements, and those statements build off preceding statements by mak-

ing connections between ideas, correcting someone’s idea, or disagreeing with someone’s idea

and providing a counterargument. In addition to Hogan et al.’s interaction patterns, we used

argumentation as a way to characterize student discourse. As conceptualized by Toulmin,

argumentation is composed of evidence, a claim made based on that evidence, and warrants

justifying how the evidence supports the claim. If students were observed to make use of

these elements when presenting their ideas, their discourse was characterized as argumenta-

tion. It is worth noting that argumentation falls under the elaborative interaction pattern

described by Hogan et al. Nonetheless, the choice to supplement Hogan et al.’s interaction

patterns with Toulmin’s argumentation was made because separating discourse that had

formal argumentation from that which had a “non-argumentative” elaborative interaction

113

pattern was productive. The distinction allows the Modes of Collaboration framework to

attend to the difference between a series of unsubstantiated ideas shared by students, and

a sequence of explicitly supported claims. These two interactions may suggest different mo-

tives for the students and may have different results for their further interactions, and we

therefore wanted to capture this difference in the discursive dimension.

5.6.3 Disciplinary content dimension

The disciplinary content dimension addresses the ways that the students discuss physics. It

describes the types of physics content on which their conversations focus. In the previously

mentioned work by Irving et al. on epistemological framing, they identified an axis that

described the scope of students’ framing, with narrow at one end and expansive at the other

end [125]. We do not make direct use of this construct, as our work does not attempt

to identify the ways in which students frame the activity.

Instead, we define students’

discussion to be related to specific physics content or abstract physics content. These terms

were selected in order to convey the degree to which the content being referenced is tied

concretely to the situation the students are analyzing. When students discuss specific physics

content, they focus on physics content applied specifically to the question at hand, such

as discussing the orientation of the electric field vectors in the diagram presented in the

question. Abstract physics content discussion refers to discussion that is not directly related

to producing an answer to the activity question. Instead, it centers on concepts in general,

such as a discussion of the meaning of electric potential energy. Thus, based on the scope of

the physics content that the students discussed, the disciplinary content of their discussion

was characterized as either abstract or specific.

A Mode is defined by its classifications in each of the three dimensions. It is worth noting

114

Dimension

Categorizations

Social

Discursive

Disciplinary
content

Consonant
Dissonant
Consensual
Responsive
Elaborative

Argumentation

Specific
Abstract

Table 5.1: The three dimensions and the categorizations possible within each dimension

that the dimensions are treated independently. Each dimension is assessed solely based on the

discourse and behaviors observed, without reference to categorizations made along the other

dimensions. It could be that some combinations do not occur (consonant social dimension

and argumentation discursive dimension appear contradictory, for example), but by coding

across each dimension individually, we do not make any assumptions about such connections.

It is also important to recognize the grain size that the Modes of Collaboration framework

considers. When applying the framework, data is broken into episodes defined by apparent

shifts in a group’s interaction along any of the three dimensions.

It is these demarcated

episodes that are then analyzed along the three dimensions. A summary of the dimensions

and the categorizations possible within each appears in Table 5.1.

5.7 The Modes of Collaboration

In this study, we identified four distinct ways in which students interacted: Debate, Inform-

ing, Co-construction of an Answer, and Building Understanding Towards an Answer, which

are described below in detail. Table 5.2 provides an overview. The following subsections

each begin with a description of how the Mode manifests based on student behavior, then

outlines how it is defined using the three dimensions.

115

Mode

Social

dimension

Discursive
dimension

Debate

Dissonant

Argumentation

Informing

Consonant

Consensual

Disciplinary

content

dimension
Specific or
abstract
Specific

Co-construction

of an Answer

Building

Understanding

towards an

Answer

Consonant

Elaborative

Specific

Consonant

Elaborative

Abstract

Table 5.2: The Modes of Collaboration identified in this data and their characterizations
along each of the three dimensions

5.7.1 Debate

In the Debate Mode, two students engage in a dialogue, arguing their conflicting understand-

ings of the concepts or responses to the activity prompts, while the remaining students in the

group do not speak. It will continue until one student ultimately capitulates and accepts the

other’s reasoning, or at least ceases to argue their own. The Debate Mode is characterized

by dissonant interactions in the social dimension, argumentation in the discursive dimension,

and can be characterized by specific or abstract content in the disciplinary content dimen-

sion. The two episodes presented here happen in immediate succession, and demonstrate an

example of specific Debate and an example of abstract Debate. In the episodes, Lindsay and

Michael work on a problem that asks them to determine if there are any points of equilibrium

in the area surrounding an electric quadrupole using an image of the electric field vectors.

Immediately before the transcript begins, Lindsay has argued that the electric field vectors

cancel.

116

5.7.1.1 Episode A1

Michael: Yeah, they’re [electric field vectors] not in opposite directions. They all
are going in the same direction, so they’re not cancelling.

Lindsay: Well no no no, they are. These are, [gesturing at the electric field
vectors on the image because this one’s going this way, and this one’s going this
way. And then these two are going in opposite directions, as well. So, they are
in opposite directions.

5.7.1.2 Episode A2

Michael: Well, you remember the tutorial homework we did? And it had equal
magnitude charges, but one was negative and one was positive, and they added
together.

Lindsay: Yeah. . .

[no longer pointing at the image]

Michael: Ones that are of the same sign and equal magnitudes cancel out.

Lindsay: Why would they cancel if they’re the same sign, when you add them
together?

1

2

3

4

5

6

7

8

9

10

11

12

13

5.7.1.3 Social dimension

When engaged in the Debate Mode, the two active students interact in a dissonant way. For

the duration of the Mode, the two Debating students explicitly disagree with one another.

Rather than validating each others’ ideas, they put forth criticism. In the examples, there are

multiple instances of explicit disagreement. In lines 1-2, Michael’s first response to Lindsay’s

argument that the field vectors cancel, he directly disputes her claim. Following this, in lines

3-6, Lindsay reiterates her belief, contradicting Michael’s ideas. In addition to this overt

conflict, Lindsay and Michael explicitly critique each other’s ideas.

In lines 1-2, Michael

not only disagrees with Lindsay, but also explains the flaw he finds in her ideas. Similarly,

117

in lines 3-6, Lindsay explains why she believes Michael is incorrect. Finally, in lines 12-13,

Lindsay pushes back against a perceived weakness in Michael’s reasoning. These critiques,

along with the conflict present in these episodes, indicate that they both take place with a

dissonant interaction style.

5.7.1.4 Discursive dimension

Along the discursive dimension, the Debate Mode is characterized by argumentation. The

two Debating students making use of evidence, claims, and warrants as they present their

ideas to one another. In the examples, we see Michael and Lindsay make use of these elements

of argumentation in their discussion. In lines 1-2, Michael presents both his evidence, the

electric field vectors are pointing in the same direction, and his claim, that they do not cancel.

Later, in lines 7-9, he provides his warrant. He refers to a previous assignment to provide

further information on why vectors going in the same direction do not indicate cancelling.

In lines 3-6, Lindsay presents her competing claim, that the vectors do cancel, and evidence,

that the image shows the vectors pointing in opposite directions. Such implementation of

claims, evidence, and warrants indicates that the students are engaging in argumentation in

these episodes.

5.7.1.5 Disciplinary content dimension

The Debate Mode may be specific or abstract in the disciplinary content dimension. Students

may discuss physics content directly related to the question at hand, or they may focus

their conversation on general cases and abstract concepts.

In the examples, we see both

kinds of disciplinary content present.

In lines 1-6, Michael and Lindsay both attempt to

directly answer the question. Each refers to the vectors on the image representing the charge

118

distribution the question requires them to consider. Thus they consider specific physics

content. In lines 7-13, the focus of their discussion shifts. Michael no longer refers to the

question in the activity. He expands the conversation to discuss charges and “cancelling”

more generally.

In this way, the conversation now focuses on abstract physics content,

indicating the beginning of a new episode. The students’ behavior has not shifted along

the social or discursive dimensions, however, and so Episode A2 is still Debate, but now a

Debate of abstract content.

5.7.2 Informing

In the Informing Mode, one student, the “Informer”, explains his or her ideas about the

question at hand to one or more other group members. The Informer consistently offers

his or her thoughts while other students do not; instead they only listen to or ask ques-

tions of the Informer. This continues until all participating group members begin writing

the results of the discussion on their worksheets. The Informing mode is characterized by

consonant interactions in the social dimension, consensual interaction patterns in the discur-

sive dimension, and specific content in the disciplinary content dimension. In the example

presented here, Jim, Erin, and Angela engage in the Informing Mode, with Erin acting as

the Informer, as they attempt to determine how to maximize electric potential energy when

placing a test charge near a given charge distribution, using a diagram of the equipotential

lines surrounding the distribution.

5.7.2.1 Episode B

1

2

Erin: So all those rings [pointing at equipotential lines] show the same potential
energy – one right. So the one that is the smallest ring has the most potential

119

3

4

5

6

7

8

9

10

11

energy.

Angela: Ohh.

Jim: Okay, so we just put it [the test charge] right in the middle? [looking at
Erin]

Erin: On the smallest ring.

Jim: On the smallest ring.

Angela: On the smallest ring.

Erin: Yeah.

[All three begin quietly writing on their worksheets.]

5.7.2.2 Social dimension

In the Informing Mode, students interact in a consonant way. There is no explicit conflict

among group members or criticism of peers’ ideas. Instead, the ideas presented are recognized

and validated without resistance.

In the example, Erin presents her thoughts about the

answer to the question in lines 1-3. After this statement, the conversation contains implicit

validation of Erin’s statement by Angela in line 4, and explicit validation of Erin’s idea by

both Jim and Angela in lines 8 and 9, respectively. There is no criticism or rejection of Erin’s

ideas at any point, and no other students present ideas that could be subject to criticism or

rejection. Thus we see that this episode is characterized by a consonant interaction style.

5.7.2.3 Discursive dimension

During the Informing Mode, students’ discourse is characterized by consensual interactions.

The Informer is the only student who makes substantive contributions to the conversation,

while the other students either explicitly agree and accept the Informer’s statements, or ask

short clarifying questions. In the example, Erin is the only student who makes a substantive

120

contribution with her statements about the equipotential lines in the diagram in lines 1-3.

In contrast, Angela responds with an implicit acceptance of Erin’s statement in line 4, and

in line 9, a direct repetition of a statement made by Erin. Jim’s contributions consist of a

question in lines 5-6 clarifying Erin’s initial statement, and in line 8, a direct repetition of

Erin’s statement. In this way, we see that this episode has a consensual interaction pattern,

with Erin acting as the substantive contributor.

5.7.2.4 Disciplinary content dimension

In the Informing Mode, the disciplinary content of the students’ conversation is characterized

by specific content. The students discuss physics content as it relates directly to producing

an answer to the question at hand. They do not discuss physics concepts in the abstract

or expand their conversation to general cases. In lines 1-3 of the example, Erin presents an

answer to the question the group is discussing. She does not discuss the meaning of electric

potential energy or the function of equipotential lines in general. The only other statement in

the episode that is not a simple assent or repetition is Jim’s question in lines 5-6. With this

question, he confirms the answer that Erin is proposing, still limiting the scope of the physics

content being discussed to the question at hand, and not abstract concepts. Accordingly, we

see that the conversation in this interaction centers on specific content.

5.7.3 Co-construction of an Answer

In the Co-construction of an Answer Mode, two or more students work towards creating an

answer to the question on which they are working. As the students work towards this answer,

nearly every contributed statement is acknowledged and built upon. A student is considered

to be participating in the Co-construction of an Answer if he or she makes statements relevant

121

to the conversation during the episode. Non-participating students may or may not appear

to be paying attention, but are not considered a part of the Co-construction of an Answer

regardless, as they are not aiding in the construction of the answer. The Co-construction of

an Answer Mode is characterized by a consonant interaction style in the social dimension,

elaborative interaction patterns in the discursive dimension, and specific physics content

in the disciplinary content dimension. In the example here, all four students in the group

engage in the Co-construction of an Answer Mode as they consider which way a test charge

would move when placed near an electric dipole using an image of electric field vectors.

5.7.3.1 Episode C

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Lindsay: So, if we put a negative charge here at this X, it’s asking you which
way it would move after it’s released.

Michael: It would move towards the outside middle.

Lindsay: What do you mean by outside middle? [looking at Michael]

Michael: Well, it’s moving. . .
hesitating]

[gestures hand over the image, then pulls back,

Lindsay: [looking at Michael, then speaking] This [the image] is the electric field
at each point. . .

George: Wouldn’t this be, this [gesturing at a field vector] is showing where a
positive test charge would go, so wouldn’t the electric charge move opposite?

Lindsay: Well it’s a negative, so wouldn’t it just move. . . [takes the image and
starts to draw on it]

Oscar: This is negative though, right here [indicating a point on the image],
isn’t it?

Lindsay: Oh, it is negative; you’re right, ok [erases what she’s drawn] so it would
move. . . opposite.

122

5.7.3.2 Social dimension

The Co-construction of an Answer Mode is characterized by consonant interactions in the

social dimension. There is no explicit conflict in the group’s discussion, and the ideas that

group members put forth are acknowledged and validated.

In the example, nearly every

statement made is acknowledged by the following statement. For example, Michael proposes

an answer in line 3, and in her clarifying question in line 4, Lindsay makes direct reference to

Michael’s statement. In lines 7-8, while Lindsay does not explicitly acknowledge Michael’s

attempt to answer her clarifying question, the fact that she waits for Michael to trail off, then

begins her statement by looking at him indicates that she is attempting to aid him in his

hesitation. In the episode, we also see the explicit validation of peer ideas. In line 15, Lindsay

specifically says, “you’re right” in response to the idea that Oscar has presented in lines 13-

14. Beyond the recognition and validation of peer ideas, there is also no explicit conflict

present in the group’s discussion. At no point do any of the students outright disagree with

something another student has said. The closest statement to a disagreement in the episode

comes from Oscar in lines 13-14, where he points out to Lindsay that a point charge she had

been considering positive is in fact negative. Even in this statement, however, Oscar does not

present his correction as a disagreement. Instead, he simply offers a new idea, phrasing it as

a question, and not a rejection of Lindsay’s understanding. This lack of explicit conflict, and

the recognition and validation of peers’ ideas indicate that the interactions in this episode

are consonant.

5.7.3.3 Discursive dimension

The discursive dimension of the Co-construction of an Answer Mode is characterized by

elaborative interaction patterns. All participating students not only make substantive con-

123

tributions to the discussion, but also explicitly connect those contributions to those of the

other students. In the example, all four students in the group provide statements that are

relevant to their discussion of the question and also explicitly relate their statements to each

other’s. In lines 7-8, Lindsay provides her understanding of what the image is showing them,

thus offering a substantive contribution to the conversation. In lines 9-10, George responds

by building off of this, offering a more specific understanding of what the electric field vectors

show. With this response, he contributes substantively to the conversation, and also explic-

itly connects his ideas to what Lindsay has contributed. Earlier in the conversation, Michael

provides a substantive contribution in line 3 when he proposes an answer to the question,

and in lines 5-6, provides another substantive contribution that is directly connected to the

question Lindsay asks him in line 4. In lines 13-14, Oscar also contributes a substantive and

explicitly connected statement when he offers his correction of Lindsay’s thoughts in lines

11-12 . These substantive contributions and the explicit connections that the students make

among them are what show this to be an elaborative interaction pattern.

5.7.3.4 Disciplinary content dimension

The disciplinary content dimension of the Co-construction of an Answer Mode is described

by specific physics content. The students focus their conversation on physics as it relates

directly to the question they are working on, and do not attempt to expand their conversation

to general cases or abstract concepts. In the example, the whole of the discussion is centered

on producing an answer to the question, “which way will the charge move?”.

In line 3,

Michael provides a possible answer. In lines 7-10, Lindsay and George discuss the image of

the charge distribution that the question asks them to consider, with Lindsay describing what

the image shows, and George describing what the individual vectors on the image show. In

124

neither case do they make claims about what electric fields or field vectors show in general,

instead they refer specifically to the image they have. In lines 7-10, Lindsay and George

both use the image to propose answers to the question. In response to Lindsay’s answer,

Oscar brings the group’s attention to an element of the image in lines 13-14. Finally, Lindsay

incorporates this and presents an answer again in lines 15-16. Throughout this episode, the

students’ conversation is focused on producing an answer to the question using information

from the image they have been provided. This attention to the question and the absence of

discussion of the meaning of physics concepts in the abstract indicate that this conversation

is characterized by specific physics content.

5.7.4 Building Understanding towards an Answer

In the Building Understanding towards an Answer Mode, two or more students discuss

physics concepts in a way that is not directly related to answering a component of the

activity, instead seeking to develop an understanding of the underlying concepts. Due to

the nature of the activity the students in our data completed, this understanding was ul-

timately be aimed at answering a question in the activity, but nonetheless, the Building

Understanding towards an Answer Mode focuses first on developing an understanding. Sim-

ilar to Co-construction of an Answer, during Building Understanding towards an Answer,

nearly every statement contributed is recognized and built upon. Also as in the case of

Co-construction of an Answer, a student is only considered to be participating in the Build-

ing Understanding towards an Answer Mode if they verbally contribute to the discussion,

as they otherwise are not contributing to the building of the group’s understanding. The

Building Understanding towards an Answer Mode is characterized by consonant interactions

along the social dimension, elaborative interaction patterns along the discursive dimension,

125

and abstract physics content along the disciplinary content dimension. In the example here,

Leslie, Ben, and Ron engage in the Building Understanding towards an Answer Mode as

they discuss the meaning of electric potential energy and its relationship to electric field

lines while they work on a question asking them to determine how to maximize electric po-

tential energy when placing a test charge near a given charge distribution, using a diagram

of the electric field lines surrounding the distribution.

5.7.4.1 Episode D

1

2

3

4

5

6

7

Leslie: What does it mean though to have electric potential energy?

Ben: Remember here we did that question? [flipping to a previous page in
the activity] It was a question where you compared the electric potential energy
between like A and B, and the answer was A, here, has the greater electric
potential energy. So I think it’s how close you are to the actual. . .

Ron: [looking at Leslie] In other words, how much energy you need to put in to
like move it.

5.7.4.2 Social dimension

The social dimension of the Building Understanding towards an Answer Mode is described

by consonant interactions. The participating students recognize and validate one another’s

ideas, and there is no explicit conflict present in the discussion.

In the example, Leslie

begins the conversation by asking the group a question in line 1. Ben acknowledges Leslie’s

question and engages with it when he answers her in lines 2-5. After this, in lines 6-7,

Ron acknowledges both Leslie’s question and Ben’s answer by providing another answer

to Leslie’s question, and by framing it as “in other words” to Ben’s answer.

In addition

to this consistent recognition of each other’s statements, the episode also demonstrates a

lack of conflict. In lines 2-7, neither Ben nor Ron disagree with one another’s answers to

126

Leslie’s question. This lack of conflict and the students’ acknowledgement of their peers’

contributions show this to be a consonant interaction style.

5.7.4.3 Discursive dimension

The discursive dimension of the Building Understanding towards an Answer Mode is char-

acterized by elaborative interaction patterns. All students participating contribute substan-

tively to the conversation, and they explicitly connect their ideas to those put forth by other

students. In the example, Leslie, Ben, and Ron all make substantive statements. In line 1,

Leslie asks a question to the group about a relevant physics concept. In lines 2-5, Ben pro-

vides his thoughts on this concept, making use of a previous example and his understanding

of it. In lines 6-7, Ron shares his own reasoning regarding the topic of Leslie’s question. Not

only do all three students contribute substantively to the discussion in this way; they also

make explicit connections across their contributions. In lines 2-5, Ben directly relates his

answer to Leslie’s preceding question, and in lines 6-7, Ron explicitly connects his answer

to Ben’s by stating that it is “in other words”. The substantive contributions the students

make and the way that they connect them indicate that this conversation has an elaborative

interaction pattern.

5.7.4.4 Disciplinary content dimension

The disciplinary content dimension of the Building Understanding towards an Answer Mode

is described by abstract physics content. While the students in this context ultimately aim to

produce an answer to a question, when they engage in the Building Understanding towards

an Answer Mode, they do not focus on this goal directly.

Instead, they discuss physics

concepts in the abstract or general cases, first establishing an understanding of these before

127

attempting to apply them in the creation of an answer to a particular question.

In the

example, the students attempt to arrive at an understanding electric potential energy in

general and how it relates to electric field lines. In line 1, Leslie begins the episode with a

question about the meaning of electric potential energy. Her question isn’t directly related to

determining an answer to the current question of the activity. When Ben responds in lines

2-5, he makes reference to a previous problem the students had completed, but does not

attempt to connect it to producing an answer to the current question. In Ron’s response in

lines 6-7, he provides his understanding of electric potential energy, again, not connecting it

to answering the current question in the activity. As the focus of their conversation was not

the production of an answer, but instead the meaning and relationships among the concepts

involved, their conversation is characterized by abstract physics content.

5.7.5 Relationships Among Modes

The four Modes of Collaboration defined here can be distinguished from one another along

the three dimensions (see Table 5.2).

In addition to considering each Mode independently as above, it can be illustrative to

consider the similarities and differences among them. One element to consider is the number

of participants each Mode tends to have. The Debate Mode is the only Mode which has a

specific number of participants – two. Informing, Co-constructing an Answer, and Building

Understanding towards an Answer all may have any number of participants (greater than

one). The Debate Mode is also unique from the other three Modes in that it is the only

one to take place with a dissonant interaction style in the social dimension, while the others

have a consonant interaction style.

Co-constructing an Answer and Building Understanding towards an Answer are perhaps

128

the most similar Modes; they both have elaborative interaction patterns in the discursive

dimension and consonant interaction styles in the social dimension. Even in the disciplinary

content dimension, both Modes ultimately seek to produce answers to the questions in the

activity. The crucial distinction between them is the way in which the students go about

producing those answers. To illustrate this difference, a continuation of the transcript in

Episode D is presented here, in which we see a switch from Building Understanding towards

an Answer to Co-constructing an Answer. The fourth member of the group, Donna, who

did not participate in Episode D, begins participating here.

5.7.5.1 Episode E

Leslie: Okay, but then looking too, [gesturing at the image] so this has a large
radius away from this section, and it also has like a similar density of lines and
like really close to here. So since they the same density of lines and this is a
bigger radius, would this be more potential energy?

Donna: Well, okay, if you look at like the original lines [gesturing at the image]
and the arc length between those, as they get farther out, that shows like. . . but
they added lines in here. I don’t know why they did that. But, like that line’s
kind of added in there. It didn’t start from the beginning.

Ben: [speaking while Donna continues] Yeah.

Donna: This line goes from the beginning.

Ron: [speaking while Donna continues] Mmmhmm.

Donna: That line goes from the beginning.

Leslie: Ohhh. That’s confusing, cause then that’s not like. . .

Donna: [inaudible] so maybe that’s not the same thing.

Ben: I don’t think that’s related to [energy]. . .

is it?

Leslie: Related? What do you mean? [looks at Ben]

129

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

Ben: I was thinking like if I were to like just go this distance [gesturing at
the image], like anything. . . anything in this area would have a greater electric
potential energy than like something [gesturing at another area on the image]

Donna: Maybe the equipotential lines would have been better.

Ben: Yeah.

Ron: [inaudible] So where is that? So it would be over the center? Not dead
center. [miming being away from the center with his hand]

Donna: We could just say the origin of the arrows.

Ron: Yeah, the origin of the lines.

Leslie: Yeah.

[all begin writing on their worksheets]

In contrast to the focus on abstract content in lines 1-7 (discussed in the Building Un-

derstanding towards an Answer: Disciplinary Content Dimension section), in lines 8-11, we

see Leslie shift the focus of the discussion to the image of the charge distribution from the

activity question. Furthermore, her question makes a comparison of the magnitude of the

electric potential energy at two points in the image. This is directly related to answering

the activity question, which asks the students where the maximum magnitude of electric

potential energy in the image occurs. When Donna responds in lines 12-19, she also refers

to the image from the activity, and in lines 24-26, Ben discusses relative amounts of electric

potential energy at different points in the image, again related directly to answering the

activity question. Finally, in lines 29-34, the students explicitly discuss what to write as

their final answers on their worksheets, and then do so.

The explicit focus on answering the activity question seen in Episode E indicates that the

students are engaging in the Co-constructing an Answer Mode. This is notably different than

the Building Understanding towards an Answer seen in Episode D, in which the students

130

instead focused on establishing an understanding of the relevant physics concepts. Episodes

D and E also demonstrate the relationship between Building Understanding towards an

Answer and the production of an answer to an activity question. In Episode D, the students

did not focus directly on producing an answer, but the results of the understanding they had

built were later applied to answering an activity question.

Finally, episodes D and E indicate that shifts between Modes might be readily identifiable.

Between lines 7 and 8, there was a clear change in the focus of the conversation, which showed

a change in the Mode. That the four Modes of Collaboration observed in this context can be

defined using the social, discursive, and disciplinary content dimensions, and meaningfully

distinguished from one another makes them a framework that could be productively applied

to acquire insights not offered by other frameworks, which will be explored in the Discussion.

5.8 Discussion

The Modes of Collaboration can offer a perspective not provided by existing frameworks,

but shares elements with and is greatly informed by previous work regarding student group

dynamics in physics and other disciplines. In particular, the three dimensions that are used

to define the Modes are closely tied to previous work. The social dimension rests on the

interaction styles identified by Lumpe and Staver [21], the discursive dimension makes use

of the interaction patterns identified by Hogan et al. and argumentation as conceived of by

Toulmin [128, 175], and the disciplinary content dimension bears similarity to the work on

epistemic framing by Irving et al. [125].

The relationship of the Modes of Collaboration framework with the work on epistemic

framing by Irving et al. merits particular discussion, as this relationship may be the least

131

clear. The expansive vs narrow axis defined by Irving et al. clearly has a relationship to

the abstract and specific content distinguished in the disciplinary content dimension of the

Modes; both are related to the degree to which student talk is directly tied to the task

on which they are working. While the Modes solely attend to whether or not students

are focusing on producing an answer to make a distinction between abstract and specific

content, however, that is only one piece of evidence that Irving et al. make use of in

their characterization of student discussion on the expansive vs narrow axis. For example,

they also consider aspects such as the use of multiple representations to indicate a more

expansive framing, and an extended focus on answering an instructor’s questions to indicate

a narrow framing. In this way, the narrow vs expansive axis is more broadly defined and

inclusive of more elements than the abstract and specific physics content distinguished in the

disciplinary content dimension. It is also worth noting that the Modes are fundamentally

different than the framework for epistemic framing developed by Irving et al. in that they

have different goals. The Modes of Collaboration seek to holistically consider and distinguish

different types of student interactions in groups based on their behavior, while Irving et al.’s

epistemic framing aims to describe how students appear to frame the activity in which they

are engaged. To identify a group as engaged in a particular Mode is to describe how they

are interacting with each other. This is not necessarily the same as stating that all the

students are framing the activity in the same way. One could imagine different students

framing a Debate quite differently, for example. Some may see it as a positive manifestation

of academic rigor, while others may feel that it is undesirable conflict. In this way, analyzing

student behavior through the work on epistemic framing by Irving et al. and using the

Modes of Collaboration to identify how students are engaged in group work are separate

endeavors.

132

More generally, the Modes of Collaboration framework is set apart from the previous

work on which it builds in that it makes use of the constructs developed by the aforemen-

tioned authors simultaneously in the analysis of student interactions, and in a way that

distinguishes the social, discursive, and disciplinary content related elements. These three

dimensions each carry valuable information on student engagement, and by attending to all

three, none of that information is neglected. For example, if one only considered the so-

cial and disciplinary contentment dimensions, Informing and Co-construction of an Answer

would not be distinguishable, as they are both consonant and focused on specific content.

However, to an instructor or researcher, the difference between one student instructing their

group on the correct answer, and multiple students working as peers in the development of

an answer, can be of great importance.

In addition to retaining the information carried by all three dimensions by attending to

them simultaneously, the Modes of Collaboration framework attends to them independently.

The framework does not make any a priori assumptions about the relationship between the

dimensions, and so analysis along one dimension is not used to inform analysis along the

other dimensions. This allows for the possible identification of Modes that may have been

neglected had such relationships been assumed. For example, while we did not observe an

episode with consonant social interaction and argumentation as its discursive dimension,

and such a pairing may seem unlikely, the Modes of Collaboration does not discount its

possibility. Attending to the dimensions independently also makes the framework relatively

simple to apply, and suitable for multiple types of analyses. One does not need to consider

the convolution of the dimensions in identifying Modes, as episodes are categorized along

each dimension separately. Once the Modes present have been identified, one could use this

data to analyze a snapshot of a single group, compare the Modes present across multiple

133

groups, or consider the Modes a single group engages in over time.

This multi-dimensional approach has benefits when compared to frameworks that focus

on a single area. For example, Toulmin analysis focuses on the content and structure of

speakers’ discourse. Applied in our context, it would be very successful in describing how

students constructed their arguments. One thing it would not attend to, however, would be

the type of physics the students discussed. A Debate on specific content and a Debate on

abstract content would be treated the same. The multi-dimensional approach used in our

framework captures these differences, which can be crucial for understanding the variety of

ways that students experience group work.

While the Modes of Collaboration framework is able to provide a unique and productive

description of student engagement with group work, it is not an all-encompassing description.

Of the video analyzed once moments with off topic conversation or no explicit interaction

had been neglected, 68% was identified as being an instance of a Mode. Thus the Modes

defined here do not completely describe the possibilities of student interactions. Common

interactions not classified as Modes in our data included students confirming how to phrase

their answers on their worksheets, and sequences of disconnected statements.

The small number of students and relatively short amount of time analyzed in our data

naturally limits the degree to which the Modes might be generalizable. Because the students

in our data worked on a single activity, the Modes we observed may have been influenced by

that activity. Specifically, the fact that the task required the students to produce answers to

turn in on a worksheet may have fostered an environment which favored Modes that focus

on the construction of an answer. It is also possible that there is a relationship between the

particular physics content in the activity and the Modes in which the students engaged. The

highly abstract nature of electric field and potential energy may have prompted students

134

to engage in different Modes than they might if the activity focused on a more familiar or

concrete area, such as projectile motion. Additionally, the classroom in which our data was

collected was facilitated with very minimal instructor interaction. It could be that with more

active facilitation, or with certain types of facilitation, the Modes observed may vary.

There are other facets of the Modes that may exist, but that we simply did not observe

due to our context or the small size of our data. For example, the Co-Construction Mode

as observed in this context was specifically the Co-Construction of an Answer. It is possible

that other “products” than an answer may be possible in the Co-Construction Mode, such as

calculations or responses to instructor questions. The Debate mode may appear differently

in other data, as well. In our context, we only observed Debates between two students, but it

is conceivable that a Debate could have more than two students, all arguing unique perspec-

tives, or even that “teams” could form, with multiple students arguing for one perspective

and multiple students arguing for another.

Another manifestation of the Modes of Collaboration that may exist, but that we did

not observe, is the co-occurrence of Modes. As all of the Modes may have as few as two

participants, it is possible that in a group of four, dyads could form and simultaneously

engage in different Modes. In our data, when only two students in a group of four participated

in a particular Mode, the other two students were not identified as participating in any Mode.

They were either quietly observing the two active students, or it was unclear on what their

attention was focused. One could imagine, however, that two students in a group could

engage in one Mode, while the other two students simultaneously engaged in another. For

example, two students could engage in a Debate while the other two students in the group

engage in the Co-Construction of an Answer. We did not observe such an episode in our

data, but the Modes of Collaboration framework does allow for such an event.

135

Though it of course cannot describe all of student interactions, the Modes of Collaboration

framework shows promise for being a useful tool for instructors. A simple implementation

could be identifying which students are participants in episodes of Co-construction of an

Answer or Building Understanding towards an Answer and using this as a way to consider

how engaged different students are with their work. The Modes may also offer one way to

identify if and how students alter their interactions with each other after an intervention by an

instructor. Finally, the Modes of Collaboration framework may be useful in comparing how

students engage with different facilitation techniques and different activities. Considering

the Modes students engage in across such different contexts could provide better insight into

the impact that different instructional choices have on student engagement with group work.

As interactive instruction continues to gain popularity, it is essential that instructors and

researchers understand how students engage in such activities. The Modes of Collaboration

provide a framework allowing insight in this area. The social, discursive, and disciplinary

content dimensions on which the framework is based allow it to capture elements of student

engagement that other frameworks may not, and also give it the flexibility to be adapted

for cross-contextual analysis of different learning environments. This kind of analysis will be

crucial for understanding the inherently complex and varied nature of collaborative learning

environments and for ultimately providing students with the best opportunities for learning.

136

Chapter 6

Conclusions

6.1 Summary of findings

The motivation of this dissertation was to better understand the complexity of collabora-

tive learning environments in introductory physics. We accomplished this by examining

these learning environments from the perspectives of both instructors and students, and by

considering expanding scopes of analysis.

In the most narrowly focused study, we considered the instructor perspective, examin-

ing the teaching approaches of learning assistants in a problem-based learning course, and

specifically their approaches to teaching computational problems. We found four different

approaches that learning assistants took, with the approaches representing increasing levels

of sophistication in their perceptions and treatment of the computational problems. This

result has implications for the way we train learning assistants and what factors may impact

their teaching practices and beliefs.

Broadening slightly to consider the student perspective, but maintaining a relatively

narrow focus by limiting its scope to conceptual reasoning, the second study investigated

how groups of students reason in collaborative learning environments in physics courses.

Using the construct of epistemic games, we identified a new group epistemic game, the

shared answer-making epistemic game. Based on a previously identified individual epistemic

137

game, the shared answer-making epistemic game is comprised of both individual and group

level moves describing the way students were observed to reason through or intuit their way

through conceptual problems. This result gives insight into how students may engage with

activities depending on their design, and indicates that individual epistemic games may be

expanded to describe similar observations of groups.

With the broadest scope, the third study considered the student perspective of collabo-

rative learning environments, but expanded to consider elements of their experiences beyond

their conceptual reasoning about the physics content. We developed a new framework, the

Modes of Collaboration, for understanding student engagement with collaborative learning

environments that attends to not only the disciplinary content of their discussions, but also

the structure of their discussions, and the tone of their interactions. The Modes of Collab-

oration framework gives insight into multiple aspects of students experiences when working

in groups and is flexible enough to be applied across collaborative learning environments,

making it useful for practical applications in assessing and designing instruction and activi-

ties.

6.2

Implications and future directions

All three of the studies presented in this dissertation have practical applications for improving

introductory physics courses and also suggest future areas of research.

The learning assistant approaches to teaching computational problems identified here can

be used to make informed changes to the training and on-going support they receive. It’s

possible that explicitly discussing the intended purpose of the computational problems dur-

ing training or during the weekly pre-class meetings could encourage learning assistants to

138

adopt more sophisticated approaches. Supporting learning assistants in this way could have

benefits for not only their development as teachers, but could also ensure that their students

are receiving the computational experiences that the course intends they have. There are

several open areas that would benefit from further research regarding this project. Given

that the learning assistants’ approaches in this study were largely uncorrelated with their

levels of experience, it would be interesting to track their approaches longitudinally, to see

what affects an individual’s progression or lack thereof through the different approaches.

For example, it is possible that the faculty members a learning assistant works with are im-

portant to the approach they take, or that their own ensuing experiences with computation

affect their approach. It would also be informative to investigate if learning assistants teach-

ing in a problem-based learning course on a different topic presented similar approaches to

teaching computational problems, or if learning assistants teaching in a laboratory or more

traditionally taught course presented similar approaches, to investigate to what degree the

approaches identified here are contextual. One could also analyze in-class observations of

learning assistants’ enacted teaching practices to see if they align with their self-described

approaches. Additionally, as a part of this study, interviews with several faculty members

and a graduate teaching assistant were also conducted. Comparing learning assistants’ ap-

proaches to teaching computational problems to those of graduate teaching assistants and

faculty members could give more insight into how and why these different populations teach

the computational problems differently or similarly.

In particular, one interviewee was a

developer of the course, and it would be interesting to compare his beliefs regarding the

purpose of computation in the course to the learning assistants’ approaches, since the intent

is that the learning assistants teach in ways that are aligned with the goals of the course’s

developers.

139

The shared answer-making epistemic game can give insight into what factors might lead

students to engage in more reasoning versus more intuition, and instructors could use this to

inform how they design their activities depending on the type of conceptual engagement they

wish to encourage. It also could give instructors insight into how their students navigate

conflicting understandings when collaborating, and how they might be able to make such

conflicts productive for students’ learning. Related to this, one opportunity for further re-

search on this project would be investigating what happens if no student induces a disruption

to a group’s progression through the game. It could be that such disruptions are important

in encouraging students to fully articulate their ideas to each other, and analyzing the work

of a group that experiences no disruption could provide insight into this issue. It would also

be informative to examine if the shared answer-making epistemic game appears in the same

way or at all when students work on a less abstract concept than electric field. Student

confidence or familiarity with the concepts being probed could affect the degree to which

they rely on their intuition as opposed to conceptual reasoning, and investigating students’

work on a different subject could shed light on this issue. A study such as this, or a study

examining the work of students in a more closely facilitated learning environment could also

indicate what elements of the shared answer-making game were tied to our context.

The Modes of Collaboration framework can be applied across learning contexts and activ-

ities, and provides instructors with a simple tool to identify how their students are engaging

with one another while collaborating. By assessing how their students collaborate in dif-

ferent situations, the framework could allow instructors to make choices regarding activity

design, interventions, and group formation that encourage the types of interactions most

aligned with their goals. The flexibility of the Modes of Collaboration framework also opens

up several new areas of investigation. Examining which Modes students use in a more care-

140

fully facilitated collaborative learning environment could give insight into what impact the

different facilitation styles have on their engagement. In fact, the Modes of Collaboration

framework was applied to a small amount of data from a problem-based learning course, and

preliminary analysis suggested that students did not engage in the same Modes at the same

rates, and that there were likely entirely new Modes present. In addition to comparative

studies like this, a longitudinal study examining which Modes of Collaboration a group of

students uses over the course of a semester could help us understand how students’ learn to

collaborate and what affects that process.

6.3 Concluding remarks

As undergraduate physics education continues to transform to better meet the needs of stu-

dents and society, collaborative learning environments will play an important part. As more

instructors and students find themselves in these learning environments, we will need to be

able to make informed decisions about their design and implementation to best provide stu-

dents with experiences and skills that will serve them in their continuing studies and careers.

By understanding the many facets of these environments, we can create learning environ-

ments in introductory physics that will help our students develop as learners, scientists, and

scientifically literate members of society.

141

APPENDICES

142

Appendix A

Expanded description of

phenomenographic analysis

This appendix presents a more detailed description of the analysis conducted for the study

presented in Chapter 3. As described in Chapter 3, the study examined learning assis-

tant approaches to teaching computational problems in a problem based course, specifically

employing phenomenography as the method.

Goals of a phenomenographic study

Phenomenography rests on the assumption that although individuals experience a given phe-

nomenon in unique ways, by probing a group of individuals’ experiences of the phenomenon,

a set of categories can be developed that describe the different ways people experience the

phenomenon. The goal then of a phenomenographic study is to find the most concise set

of categories that capture the essential variations in individuals’ experiences of the phe-

nomenon. A phenomenographic study does not seek to account for the entirety of a given

individual’s experience of the phenomenon, nor does it seek to develop categories that allow

for each individual to be unambiguously sorted. Instead, the objective is to use the individu-

als’ experiences to identify themes in what people experience regarding the phenomenon, and

variations in how they discuss those themes. It is these variations that allow meaningful cat-

143

egories to be developed that capture the essential differences in how people may experience

the phenomenon [54, 137, 154, 155, 158, 159]. In order to draw a rich description of indi-

viduals’ experiences with the phenomenon of interest, phenomenographic studies often use

interviews as their data source. Phenomenographic interviews are typically semi-structured,

with the interviewer having a set of topics or questions they wish to address, but allowing

for the conversation to follow tangents that arise [159, 167–169].

Phenomenographic methods applied to learning assis-

tant approaches to teaching computational problems

Data collection and reduction

Such interviews were conducted for the study presented in Chapter 3, surrounding the phe-

nomenon of interest: approaches to teaching computational problems. The interviews were

audio and video recorded, and the audio recordings were sent to an external company for

transcription. Once the interview transcripts were received, analysis began with reviewing

all the transcripts several times in order to become familiar with the data set as a whole.

After gaining familiarity with the full data set, the data was reduced based on relevance to

the phenomenon of interest. Because the interviews were open to following tangents, and

because several background questions were asked (see Appendix C), there were portions of

the transcripts that were not relevant to the participants’ experiences with teaching com-

putational problems. For example, participants may have discussed their motivations for

becoming learning assistants, or described their teaching explicitly with respect to analytic

problems.

In an example of the latter, one learning assistant spoke about teaching the

144

analytic problems, which occur on Tuesdays,

On Tuesday, the first question I’ll ask them is, “What do you ultimately need
to get? Where is your end goal?” They say, “Oh, this is our end goal.” “Okay,
what are the steps you need to take to get that?” That’s on a Tuesday.

Portions of transcript such as these were neglected for the thematic analysis described next

since they were not directly relevant to our phenomenon of interest, but they were retained

in order to provide fuller context for participants’ comments if necessary.

Identification of preliminary themes

Once identified, the reduced data set was analyzed for emergent themes in participants’ dis-

cussion of teaching computational problems. These themes represented trends and common

threads in what the participants spoke about over the course of the interview. Some of these

themes corresponded to questions that were directly asked by the interviewer, while others

did not.

In either case, the themes were identified based solely on the comments of the

participants in the transcripts, and the researchers strove to analyze the transcript without

bias towards any particular themes emerging. This process resulted in a preliminary set of

seventeen emergent themes shown in Table A.1. These themes do not account for all of the

reduced data set. Some relevant portions of transcript were too unique or specific to be

sorted into a theme. Examples include a learning assistant speaking on very specific role

that he plays as computation expert on computation days, or extra preparation that a learn-

ing assistant engages in before she teaches computational problems. While these comments

were interesting, they were not further analyzed. Since the goal of this study, and of any

phenomenographic study, is to find the most concise way to capture the different ways that

145

Preliminary Themes

Differences between analytic and computational teaching

Written feedback

Varying levels of student programming experience

Different student approaches to computation

Differences in student behavior solving computation versus analytic problems

What students should do when solving computation problems

Acceptable places to end computation days

Minimally working programs

Solution manuals

Why code in groups
Student challenges
Teaching challenges

Teaching physics versus teaching coding on computation days

In-class teaching strategies

Important things for students to take away

Purpose of computation

Student frustration

Table A.1: List of preliminary emergent themes identified in the data

individuals may experience a phenomenon [54, 137, 158, 159], it is expected that not every

relevant comment from the participants will be identified as belonging to a theme.

It is also worth noting that these themes are not necessarily mutually exclusive, and

that there are portions of transcript that were coded as belonging to multiple themes. This

is because in a given comment, a participant often discusses several dimensions of their

experience at once. Thus one would not expect, and it is not a requirement that the themes

be wholly orthogonal.

Analysis of preliminary themes

Once preliminary themes had been identified, individual themes were selected and exam-

ined in more detail to identify the variations in how the learning assistants discussed them.

This analysis began with the investigation of the themes that were most richly populated,

146

Preliminary Teaching Technique Variations

Focus on navigating programming errors

Encourage reflection on coding

Use the same techniques on analytic and computation days
Attend to course design considerations on computation days

Don’t let programming details become a distraction

Leverage affordances of computational problems

Table A.2: List of preliminary variations within the teaching technique theme

Preliminary Teaching Goal Variations

Building student independence

Encouraging computational norms

Supporting content learning

Encouraging reflection on learning and process

Ensuring group has consensus

Maintaining positive student attitudes
Getting students through the problem

Table A.3: List of preliminary variations within the teaching goal theme

and which had the strongest apparent tie to our phenomenon of interest. These themes

were in-class teaching strategies, differences between analytic and computational teaching,

and teaching physics versus teaching coding on computation days as all three had explicit

references to how learning assistants teach in the classroom. Through the process of identi-

fying emergent variations within these preliminary themes, the themes were reorganized into

two themes: teaching technique and teaching goal. These two themes and their variations

are shown in Table A.2 and Table A.3.

The identified variations were split into these two themes because it was observed that

those in Table A.2 represent specific teaching moves, while those in Table A.3 relate to

broader motivations the learning assistants had regarding their teaching. The relationship

between these two themes will be addressed in greater detail later in this appendix.

The next preliminary theme to be analyzed was important things for students to take

147

Preliminary Teaching Outcome Variations

Programming skills

Physics understanding

Translating physics to code

Seeing physics concepts in code

Translating physics to visualizations

Capabilities of computation

Table A.4: List of preliminary variations within the teaching outcome theme

away, which was found to have the variations listed in Table A.4. Based on the variations

identified, the theme was retitled teaching outcome, since it was observed that the variations

all related to the outcome the learning assistants desired for their students.

At this point in the analysis, these three preliminary themes and their variations were

presented to a team of researchers for feedback and critique. A major finding from this

was that it was difficult to articulate what the difference between the teaching outcome

and teaching goal themes was. While it was initially argued that the former referred to

tangible outcomes for students, and the latter referred to more general motivations of learning

assistants, it became clear that based on the relevant portions of transcripts and discussion

amongst researchers, such a distinction was not always observable.

First refinement of themes

Based on the aforementioned feedback, the variations in Tables A.2, A.3, and A.4 were

considered holistically, with an eye for themes that would better capture the commonalities

and differences among them. This resulted in a new organization comprised of three themes

and their respective variations indicated in Tables A.5, A.6, and A.7. This restructuring

retained elements of the original organization, while also making several important changes.

The teaching outcome theme underwent the least change, as seen in Table A.5. Three of

148

Refined Teaching Outcome Variations

Programming skills

Physics understanding
Physics-code connection

A new approach to learning
Capabilities of computation

Table A.5: First refinement of variations within the teaching outcome theme

Teaching Strategy Variations

Focus on navigating programming errors

Encourage reflection on coding

Use the same techniques on analytic and computation days

Leverage collaboration

Leverage affordances of computational problems in teaching

Table A.6: List of variations within the teaching strategy theme

Characteristic to Moderate Variations

Student attitudes

Student attention to programming details

Student work pace

Impact of course design

Table A.7: List of variations within the characteristic to moderate theme

149

its initial variations (programming skills, physics understanding, and capabilities of computa-

tion) were unchanged. The other three initial variations (translating physics to code, seeing

physics concepts in code, and translating physics to visualization) were merged into a single

new variation, called physics-code connection. Because our goal was to ultimately develop

a set of categories of description that captured the essential differences in how learning as-

sistants approach teaching computational problems, we argue it is not essential to separate

these three initial themes. All three are related to students seeing relationships between

the physics concepts and the computational representation of them, be it lines of code or

resulting visualizations. Furthermore, the difference across these three initial variations is

very small in magnitude when compared to their collective differences from the other vari-

ations. That is, there is a greater difference between a student gaining an understanding

of the capabilities of computation and a student learning to translate physics to code than

there is between a student learning to translate physics to code and a student being able

to see physics concepts in code. For these reasons, the three initial variations were merged

into the single physics-code connection variation. The final change to the teaching outcome

theme in this stage of the analysis was the addition of a new variation: a new approach to

learning. This variation was comprised of two variations that were formerly in the teaching

goal theme, building student independence and encouraging reflection on learning and pro-

cess. These were merged because they both address elements beyond content understanding

or even reflection on physics or computation. Instead, they discuss learning strategies and

approaches that are broadly applicable, not tied directly to the context or content of the

course. This new variation was then moved to the teaching outcome theme because the

relevant transcript indicated a focus on what the learning assistants wanted their students

to gain or take away from the experience, and thus constituted an outcome.

150

The changes to the teaching technique and teaching goal themes were more extensive.

As stated previously, a major motivation for the refinement carried out in this stage of

analysis was a difficulty in arguing the delineation between these two themes. It became clear

that it was not possible to distinguish between the two themes as they had been defined.

Instead, two new themes emerged that were better able to capture differences among their

collective variations. One of these new themes was named teaching strategy. This theme

refers to the ways learning assistants describe their teaching moves.

It is similar to the

former teaching technique, however its definition does not depend to a particular scale of

teaching act. Instead, the teaching strategy theme accounts for both very specific moves and

also larger scale teaching methods. The variations within the teaching strategy are shown

in Table A.6. As indicated in the table, four of the variations are those formerly found in

the teaching technique theme, and one is a new variation called leverage collaboration. This

variation was formerly titled ensuring group has consensus, and belonged to the teaching

goal theme. Upon closer inspection of the transcript, it was determined that the variation

was not always specifically aimed at simply making sure a group of students agreed with

each other, but also included other methods of utilizing the group-based nature of the course

in teaching. Because of this, it was retitled leverage collaboration, and it was moved to the

teaching strategy theme, because it represented a method of teaching in the course.

The final element of this stage of analysis was the identification of a wholly new theme:

characteristic to moderate. This theme was comprised of variations from the former teaching

technique and teaching goal themes that were identified as not really being about teaching

methods or outcomes, but were instead related to what on-going aspects of their students’

work learning assistants feel they must monitor and influence. These variations (with slight

name changes) are indicated in Table A.7, and demonstrate this focus. In particular, when

151

considering Tables A.6 and A.7 together, one can see that the variations within the former

represent methods of teaching, while those in the former represent on-going elements of a

group’s work to moderate.

With this new thematic organization developed, analysis continued on the remaining

preliminary themes in Table A.1.

Continuing analysis of preliminary themes

The next preliminary theme to be analyzed was purpose of computation, because it was

hypothesized that a learning assistant’s perception of the usefulness of computation would

have a notable effect on their teaching. Table A.8 shows the variations identified within the

purpose of computation theme. After this, the student frustration theme was analyzed for

variations, because it was suspected that it may be accounted for already by the student

attitudes variation of the characteristic to moderate theme. This was found to be largely

correct, and the majority of the portions of transcript in the student frustration theme

not identified as student attitudes were nonetheless accounted for by variations within the

already identified themes. For example, learning assistants would discuss student frustration

in the context of how it leads them to encourage reflection on coding, a variation from the

teaching strategy theme. The only type of comments in the student frustration theme not

accounted for by existing variations within other analyzed themes related to the benefit

that the challenge presented by coding problems provided students by giving them practice

with overcoming struggles. This class of comments was titled students learn to overcome

struggles, and was categorized as an additional variation within the purpose of computation

theme, since similar to several other variations seen in Table A.8, it represents a perceived

utility of the computational problems.

152

Preliminary Purpose of Computation Variations

Visualization helps students check solutions

Computation days require students to interact with concepts multiple times

Computation shows the effects of making changes easily

Programming is an important skill
Computation helps solve problems

Visualization helps show physics concepts
Students learn to work together effectively

Computation teaches one to think differently

Table A.8: List of preliminary variations within the purpose of computation theme

At this point in the analysis, the purpose of computation theme was split into two more

specific themes: utility of computation and benefit of computation in P Cubed based on

whether the described purposes related to the general usefulness of computation, or to the

advantages it provided students in the course. This separation is show in Tables A.9 and

A.10. The variations indicated in Table A.9 represent learning assistants’ perceptions of the

general applicability or purpose of coding, not simply in the classroom. For example, they

regard programming’s general importance in modern working society, or its usefulness in

carrying out science by helping to solve difficult physics problems. On the other hand, the

variations shown in Table A.10 are related to the positive aspects of incorporating computa-

tion into the classroom specifically. For example, noting that students learn to work together

better on computational days, or discussing how computational days require students to use

the concepts from analytic days an additional time. One additional modification made at

this stage in analysis is that visualization helps students check their solutions and compu-

tation shows the effects of making changes easily variations were combined into a single

variation called visualization provides students immediate feedback. This was done because

both initial variations referred to ways in which the computational problems allowed students

to get information quickly regarding their solution, whether it be checking correctness, or

153

Utility of Computation Variations
Programming is an important skill
Computation helps solve problems

Visualization helps show physics concepts

Computation teaches one to think differently

Table A.9: Variations within the utility of computation theme

Benefit of Computation in P Cubed Variations

Visualization provides students immediate feedback

Computation days require students to interact with concepts multiple times

Students learn to work together effectively

Students learn to overcome struggles

Table A.10: Variations within the benefit of computation in P Cubed theme

investigating the effects of making various changes. The difference between these two initial

variations was small compared to their differences from the other variations, and so they

were merged to provide a more meaningful set of variations.

We will note here that not every variation was easily identified as belonging in utility of

computation versus benefit of computation in P Cubed, and will return to this point later in

this appendix. However, the five themes identified thus far (utility of computation, benefit of

computation in P Cubed, teaching outcome, teaching strategy, and characteristic to moderate

were used for the next stage in analysis.

Development of preliminary categories of description

The five themes defined thus far were then considered holistically. The goal of a phenomeno-

graphic study is to develop categories of description which capture the different ways indi-

viduals may experience the phenomenon of interest. This is done by looking across themes

and identifying which variations when considered together describe a logically coherent way

to experience the phenomenon. It is important to note that categories are not developed

154

on the basis of examining which variations cooccur in individuals. Instead, the goal is de-

velop categories that represent holistic experiences of the phenomenon that highlight the

essential differences uncovered in the data [54, 137, 158, 159]. With this goal in mind, we

considered the variations seen in the five themes analyzed thus far. This was done at this

point in analysis because the five themes indicated a rich variation in experiences teaching

computational problems and provided a great deal of information with which to develop

preliminary categories of description. Furthermore, the data analyzed thus far represented

a great majority of the reduced data set to be considered. Based on these considerations,

it was decided that it was an appropriate point at which to begin developing categories of

description by looking across the themes and variations defined thus far.

The first decision made in this stage of analysis was to remove the use the same tech-

niques on analytic and computation days variation from the teaching strategy theme, and

to remove the physics understanding variation from the teaching outcome theme. In both

cases, this was done because the variations were not useful in developing categories that de-

scribed different ways that learning assistants approached teaching computational problems.

As it was a physics course, it is not surprising that every learning assistant discussed physics

understanding as their desired teaching outcome to some degree. Thus this variation did not

provide useful information. Instead, the remaining variations in the teaching outcome theme

were more meaningful in identifying differences in the learning assistants’ experiences. Sim-

ilarly, every learning assistant discussed using at least some of the same teaching strategies

on both analytic and computational days. This is also not surprising, as the computational

days are not meant to be completely different experiences for the students. The learning as-

sistants are aware that P Cubed encourages certain types of teaching and learning regardless

of the type of problem being solved that day, and so there were none who claimed to not use

155

Category of
Description

Utility of

Computation

Narrow

programming

focus

Programming

is an

important

skill

Learning
physics via
computation

focus

Visualization
helps show

physics
concepts

Computation
as a scientific

tool focus

Computation

helps solve
problems

Shifting

perceptions
of learning

focus

Teaches one

to think
differently

Benefit of

Computation
in P Cubed
Visualization

provides
students
immediate
feedback

Students are
required to
interact with

concepts
multiple

times

Students
learn to
overcome
struggles
Students

learn to work

together
effectively

Teaching
Outcome

Characteristic
to Moderate

Teaching
Strategy

Programming

Student work

Focus on
navigating

skills

pace

programming

Physics-code
connection

Impact of

course design

errors

Leverage

affordances
of computa-

tional

problems

Encourage
reflection on

coding

Student

attention to
programming

details

Capabilities

of

computation

A new

approach to

learning

Student
attitudes

Leverage

collaboration

Table A.11: Preliminary categories of description

any of the same strategies on analytic and computational days. This made this variation

unproductive when looking for differences across learning assistants’ experiences, and so it

was removed from the teaching strategy theme. Having made these two cuts, the first set of

categories of descriptions were developed, shown in Table A.11. The second through sixth

columns of the table represent the five themes considered at this stage in analysis, with their

respective variations appearing as one looks down each column. The first column indicates

the name of each preliminary category of description. Each category of description is de-

fined by one variation within each theme, thus the narrow programming focus category is

described by the variations indicated by looking across the rest of the second row.

In phenomenographic studies, the categories of description are formed by looking across

the themes identified in the data, seeking to group them in ways that indicate a cohesive

156

way to experience the phenomenon of interest [54, 137, 158, 159]. In our case, we sought to

identify coherent categories that would describe a learning assistant’s approach to teaching

computational problems. As such, this stage of analysis constituted a sort of secondary

thematic analysis, where we now analyzed the variations for emergent threads that describe

a coherent approach to teaching computational problems. As indicated in Table A.11, four

preliminary categories were developed. As these categories were only slightly modified be-

fore being finalized, they will be described briefly here, with a fuller explanation of their

development appearing in the “Final categories of description” section of this appendix.

The narrow programming focus category was developed around an emergent thread found

in the identified variations, and this thread related to a learning assistant having a relatively

limited view of computation’s purpose and what computation is. With this focus, computa-

tion is viewed largely to be the act of using programming syntax, and there are few explicit

connections drawn between computation and learning or computation and science.

The learning physics via computation focus was developed around an emergent thread

wherein a learning assistant treats computation as primarily a pedagogical tool. Computa-

tion is seen as a useful way to learn physics concepts in a classroom setting, however there

are few connections made to its usefulness in practicing physics or science outside of the

classroom.

The computation as a scientific tool focus was developed around variations that indicated

an emphasis on computation as a tool used in the practice of science. With this focus,

computation is treated as an important part of carrying out science outside of the classroom,

and is not viewed as only a learning aid.

The shifting perceptions of learning focus was developed around variations that focused

on computation as a means to achieve goals beyond the scope of the classroom or even

157

Utility of Computation Variations
Programming is an important skill
Computation aids content learning

Computation makes difficult problems easier
Computation offers space for broader skills

Table A.12: Variations within the reorganized utility of computation theme

of science specifically.

Instead, computational problems were treated as opportunities to

engage students with broader skills and reflection on their own learning and problem-solving

processes.

Based on the organization shown in Table A.11, several issues were identified with the

preliminary categories of description. First, it was observed that a few variations did not

have clear connections to their categories. In particular, in the computation as a scientific

tool focus category, it was difficult to articulate the connection of the students learn to

overcome struggles variation. As this category centers around an emphasis on the real-life

usage of computation in science, the relationship that students learning to persevere in the

face of difficulty had to this was not unambiguously clear. Second, as alluded to earlier

in this appendix, it was not readily apparent which variations belonged in the utility of

computation theme versus the benefit of computation in P Cubed theme. For example, the

visualization helps shows physics concepts was at times used in reference to scientists using

visualizations to predict difficult to observe phenomena, but at other times was used to refer

to demonstrating difficult to understand concepts to students. Based on these two issues, the

decision was made to remerge these two themes, retaining the name utility of computation,

but no longer with a limitation on whether those utilities were classroom specific or more

general. This new organization of these variations is shown in Table A.12.

To arrive at this organization of the theme, several variations were merged. Visualization

158

provides students immediate feedback, students are required to interact with concepts multiple

times, and the examples of visualization helps show physics concepts that were contextually

bound to the classroom were merged to form the new variation: computation aids in con-

tent learning. This was done because all of those variations centered around computation’s

usefulness within the classroom, helping students learn physics concepts. Computation helps

solve problems and the examples of visualization helps show physics concepts that referred to

their use outside the classroom were merged to form the new variation: computation makes

difficult problems easier. This was done because these variations both related to computa-

tion’s utility as a tool for modeling and solving difficult problems outside of the classroom.

Finally, computation teaches one to think differently, students learn to overcome struggles,

and students learn to work together effectively were merged to form the new variation: com-

putation offers space for broader skills. This decision was made because these variations all

related to skills that students may develop through the computational problems that are not

directly related to physics, science, or programming.

As described with respect to analysis decisions made earlier in the process, these deci-

sions were all ultimately made in order to develop categories which captured the essential

variations in how learning assistants approach teaching computational problems. Based on

the initial categories of description identified, and the similarity of the themes and variations

that were merged to create Table A.12, it was determined that, for example, the particular-

ities of how learning assistants viewed computation aiding classroom learning were not as

important as the difference between viewing computation as a pedagogical tool in general

and viewing computation as a tool for problem-solving outside the classroom.

The final modification made to the preliminary categories of description was based on

the determination that not all portions of transcript coded as belonging to the computation

159

as a scientific tool focus had an explicit connection to the practice of science.

Instead,

they often focused more generally on computation’s usefulness as a tool in approaching

problems. To better reflect this, the category was retitled the computation as a tool focus,

emphasizing computation as a way to approach difficult problems, rather than computation

as a specifically scientific practice.

Having made these modifications, the categories were finalized, and will be described in

the following section.

Final categories of description

Based on the final changes described, the final categories of description were developed,

which appear in Table A.13. A full explanation of the development of the categories will

now be provided.

Narrow programming focus

The narrow programming focus category is based around an emphasis on computation as the

act of entering code. There is little reflection on how computation may be used in teaching

or science, rather it is treated as a valuable but unconnected skill.

In the utility of computation theme, it is characterized by the belief that programming is

an important skill to have in today’s society, with very little elaboration or explicit reflection

on why this is. For example, one learning assistant said,

[Programming] has been really helpful in a lot of my different classes, and
then also in my internship this last summer, just like as an engineering person in
general. That’s one thing that I try and stress to the students, is it’s like, “Okay,
you might think that you’re never going to use this, but it’s actually so helpful
if you do know how to use even just a small portion of it.”

160

Category of
Description

Utility of

Computation

Teaching
Outcome

Characteristic
to Moderate

Narrow

Programming

programming

is an

focus

important skill

Learning
physics via
computation

focus

Computation
as a tool focus

Computation
aids content

learning

Computation
makes difficult

problems

easier

Shifting

perceptions of
learning focus

Computation
offers space for
broader skills

A new

approach to

learning

Programming

Student work

skills

pace

programming

errors

Leverage

Physics-code
connection

Impact of

course design

affordances of
computational

Student

Capabilities of
computation

attention to
programming

details

Student
attitudes

Teaching
Strategy
Focus on
navigating

problems

Encourage
reflection on

coding

Leverage

collaboration

Table A.13: Final categories of description. The narrow programming focus treats the com-
putational problems as a way to develop students’ abilities to write code. The learning
physics via computation focus treats the computational problems primarily as a means for
learning physics concepts. The computation as a tool focus treats the computational prob-
lems as a way to help students understand the value of computation in modeling and solving
problems. The shifting perceptions of learning focus treats the computational problems as
opportunities to develop students’ understandings of their own learning.

161

In this quote, the learning assistant describes how she tries to convey to her students

that programming is helpful, noting that she has used it in multiple contexts. She discusses

how she has used it inside and outside of the classroom, and references her background as

an engineer. She does not elaborate on its importance in her learning or work, however, and

she does not describe how or why it is helpful. Instead, she only states that students are

likely to encounter it in other situations and that it can help them. This lack of reflection on

how computation is helpful and relevant is an example of a learning assistant expressing a

perception that the utility of computation is simply that programming is an important skill.

In the teaching outcome theme, the narrow programming focus is characterized by an

emphasis on programming skills. That is, the learning assistant desires their students to gain

an understanding of how to carry out programming tasks. One learning assistant said,

If it’s something where they added an object, or they were talking about an
object, or coded in an object in their calculation loop, but never defined that
object, then that’s a place where it’s like, “Okay, let’s kind of go through this
more line by line. What is this line telling us? Okay, where else does that object
show up in the code? Wait, it doesn’t show up anywhere else? We might need
to address that.” Because that is an understanding thing where they need to
understand that they have to define something in the code before they can do
anything to that object. You can’t do something to an object that you don’t
have.

In this quote, the learning assistant discusses students facing an error resulting from

referencing an object that they had not yet defined in their code, and states that is an error

that she will intentionally target with her teaching. She goes on to say that she does this

because the students “need to understand” the process of defining and calling objects in

programming. Expressing a belief that her teaching should result in students knowing how

to carry out elements of programming such as this indicates that programming skills are her

desired teaching outcome.

162

In the characteristic to moderate theme, the narrow programming focus is characterized

by student work pace. When moderating this characteristic, learning assistants attend to

their students’ rate of progress, seeking to ensure that students reach the end of the problem

by the end of class. An example of this is given by a learning assistant who said,

I might let them just mess with it for awhile. But it’s sort of a time balance
thing. So I’ve done these problems so many times, I know where they should be
at certain times to make sure they finish, and so I usually let them mess with
it until I think, “Okay, it’s halfway through, they should be at this point. I’m
gonna try to redirect them back to where they should be.”

With this quote, the learning assistant indicates that regardless of the exploration she’ll

sometimes allow her students to do, she will take steps to ensure that they are able to finish

the problem in the allotted time. She explicitly comments on using particular time markers

to check on where students should be at different points in the class session. This emphasis

on making sure students complete the problem indicates that she moderates students’ work

pace.

In the teaching strategy theme, the narrow programming focus is characterized by a focus

on navigating programming errors. With this strategy, learning assistants focus their teaching

on helping students identify and overcome programming errors in the traditional sense. That

is, mistakes in their code that cause it not to function. One learning assistant employing

this strategy stated,

Most of the time, I just teach them how to do it because it’s usually when
they’ve just like edited like one line of code, and then it’s like, “Oh, we have the
tabbing error.” I’ll just be like, “Here’s how you solve that: Highlight, and then
do the thing, and then, yay, it’s good.” Then they’ll be like, “Okay. Cool. Now
I know how to do this in the future.”

163

In this quote, the learning assistant describes a common error the students encounter:

problems with their indentation. She explains that she tailors her teaching to this fact by

preemptively instructing students on how to resolve that error. This emphasis on helping

students handle errors of this nature demonstrates a teaching strategy that is focused on

navigating programming errors.

When considered together, one can see how these variations come together to form a

cohesive approach to teaching computational problems, and one that is focused on a narrow

view of programming. With the belief that computation is essentially entering programming

syntax, it follows that a learning assistant’s desired teaching outcome would be those sorts

of programming skills. Similarly, it follows that a learning assistant with this approach

would employ a teaching strategy focused on navigating programming errors, because those

errors represent the fundamental obstacles of computation when one views computation to

be correctly entering code. The narrow programming focus corresponds to a belief that

the utility of computation is simply that programming is an important skill. This aligns

with a narrow understanding of what programming is, because without understanding there

to be deeper connections between computation and science or learning, the usefulness of

computation in P Cubed would simply be that students learn how to program. Finally, this

approach is completed by a learning assistant moderating the pace at which their students

progress through the problem. This corresponds to a narrow view of programming focus

because again, without a reflection on computation as a pedagogical tool or a problem-

solving tool, and with programming being the act of entering code, the “game” that students

must play on computational days becomes merely to get properly working code. With this

belief in mind, it follows that a learning assistant would want to ensure that their students

complete the problem and arrive at a correct solution. All four of these variations therefore

164

come together to illustrate an approach to teaching computational problems characterized

by a narrow programming focus.

Learning physics via computation focus

The learning physics via computation focus is characterized by an emphasis on the compu-

tational problems providing a pedagogical tool that learning assistants may leverage in their

teaching. Computation is perceived primarily as useful in the classroom to help students

learn concepts.

In the utility of computation theme, the learning physics via computation focus is char-

acterized by the belief that the value of computational problems is that they aid students’

content learning. One learning assistant expressing this belief said,

The students get to see it represented so it’s kind of like a reward almost for
writing out the code. They think if I do this it changes this and it’s beneficial
to see how ... For example, we did a problem with drag. So, it’s good to see
how drag affects this and if we increase the coefficient it’ll do this and this. It’s
a good way they can input information and immediately see the results of how
the physics is applied.

In this quote, the learning assistant expresses that he believes the usefulness of the

computational problems to be their ability to help students better understand the concepts.

Specifically, he discusses how students can use the immediate feedback that computation can

provide as a means to see the impact of the different physics concepts they are learning, in

this case, drag. This type of emphasis indicates a belief in the utility of computation being

that it can aid students’ content learning.

In the teaching outcome theme, the learning physics via computation focus is character-

ized by an emphasis on the physics-code connection. This refers to a desire for students

165

to see and understand how the code and visualizations involved in computation relate to

the physics concepts highlighted in the course. In one example of this teaching outcome, a

learning assistant said,

Actually understanding what you’re coding and making sure you can tie it
back to the physics. Like I said when you do the position update formula, you
should be able to understand that that comes from this kinematic equation. You
should understand how it relates and how it should affect it.

In this quote, the learning assistant describes what she wants her students to learning from

the computational problems. She discusses that the students should be able to understand

how particular lines of codes, like the position update formula, relate back to the physics

equations they are familiar with in algebraic form. With this, she expresses that she wishes

her students to gain an understanding of the physics-code connection.

In the characteristic to moderate theme, the learning physics via computation focus is

characterized by a focus on the impact of course design. Moderating this characteristic means

attending to the particularities of the way P Cubed is taught and incorporates computation,

and seeking to minimize the negative impact of their limitations. Examples include the fact

that students must share one laptop, that not all necessary information is always provided

in the problem statements, and the availability of the four quadrants whiteboard for work

organization. A learning assistant discussed this, saying,

We have to consider who’s taking over the group, like typing in by themselves,
or maybe if somebody ... Like, if people have taken turns, but somebody hasn’t
typed in yet, then we do have to take into account that the code is something
they should work on specifically, like working together with the code.

In this quote, she discusses the impact of students needing to share a single computer

while solving the computational problems, which require code to be entered into the com-

166

puter. She notes how individuals will sometimes dominate a group and do all the coding,

and how other individuals may never get a turn to enter the code themselves, and how that

dynamic is something learning assistants must monitor and address in their teaching on the

computation days. Thus in this quote, she expresses that she moderates the impact of course

design on computational days.

In the teaching strategy theme, the learning physics via computation focus is characterized

by leveraging the affordances of computational problems. This refers to a learning assistant

utilizing aspects unique to computation days in their teaching. An example of leveraging

such an aspect is discussed by a learning assistant who said,

I’ll be like, “it’s acting this way, why is it acting that way?” And when the
students realize that, that’s not possible, I’ll start questioning things that they’re
missing. And if they see what they’re missing, then they try to add that into
the problem. This, once again, goes into pseudo code, because if they see what
they’re missing, I usually tell them to write it out first, so that they don’t make
those errors.

In this quote, the learning assistant describes how he will encourage his students to

write pseudocode when they encounter difficulties on computation days. By pseudocode,

he is referring to an approximation of what the desired code is, not yet worrying about

entirely correct syntax. He explains that he will have students write this almost-code out

on their whiteboard when they are having trouble getting it to work in the computer. As

pseudocode is an attempt at writing information in a way the computer will ultimately be

able to understand, it is a construct uniquely relevant on computation days. In this way, this

learning assistant discusses employing a teaching strategy which leverages the affordances of

the computational problems.

Considering these four variations together, there is a coherent approach to teaching com-

167

putational problems that emerges that is centered around a focus on computation as a means

of learning physics. With this emphasis on computational problems as a pedagogical tool, it

is consistent that a learning assistant would view the utility of computation to be its ability

to aid students’ content learning. It also follows that a learning assistant with this approach

would employ teaching strategies which leverage the capabilities of computation, given that

they focus on computational problems as a means for students to learn physics concepts. The

desired teaching outcome in the learning physics via computation focus is an understanding

of the physics-code connection, that is, an understanding of exactly how the computational

work relates to the physics concepts being taught. It is unsurprising that a learning assistant

with this approach would want their students to understand the physics-code connection,

because they view the computational problems as fundamentally being a venue for students

to learn the physics concepts. With that view, it is understandable that they would want

their students to explicitly understand how the code and visualizations present in the compu-

tational problems are connected to the physics. Finally, the characteristic moderated in this

approach is the impact of course design based. Because the learning physics via computation

focus as a whole is very focused on computation as a pedagogical tool and is grounded in the

context of education, it is consistent that a learning assistant with this approach would pay

attention to the classroom in which they are teaching, particularly the limitations and affor-

dances that it presents, resulting in a focus on moderating the impact of the course’s design

in their teaching. This combination of four variations together characterize an approach to

teaching computational problems that focuses on students learning physics via computation.

168

Computation as a tool focus

The computation as a tool focus is an approach to teaching computational problems that em-

phasizes the usefulness of computation as a method of approaching problems. Computation

is treated as a tool that is valuable in mathematical and scientific problem-solving.

In this utility of computation theme, the computation as a tool focus is characterized

by the belief that computation makes difficult problems easier. It is a perception that com-

putation provides affordances that are an important part of modeling or solving complex

problems. One learning assistant expressing this belief said,

What makes the computer good is that it involves ... Like a big thing is the
time step. The smaller you make that time step, the more accurate everything
is. If you wanted to write and draw that out, you could, but you’d have to use
a grid system. You could. I don’t know how much that would actually help or
prove.

In this quote, the learning assistant discusses that computers can be used to carry out

iterative numerical calculations, and that it is simple to control how many iterations the

computer calculates by modifying the time step (the interval between each calculation). He

contrasts that with how difficult it would be to do this analytically. He explicitly identifies

the ability to do an increasing number of calculations easily as “what makes the computer

good”. By articulating and emphasizing how computation can be utilized to carry out

difficult calculations, he indicates a belief that the utility of computation is that it makes

difficult problems easier.

In the teaching outcome theme, the computation as a tool focus is characterized by an

emphasis on students appreciating the capabilities of computation. With this outcome, a

learning assistant wants their students to understand the benefits that computation provides.

169

This is closely tied to the aforementioned belief that the utility of computation is that it

makes difficult problems easier. While computation making difficult problems easier describes

what the learning assistant perceives the usefulness of computation to be, the capabilities

of computation describes what they want their students to takeaway from their experience

with computation. An example of this appears from a learning assistant who said,

The computation is important, it’s an important skill, but understanding how
the computation is helping them solve physics is always what I think about the
most. The computer is solving all these equations and updating all this stuff,
so what does that mean on pen and paper, and what does that mean for the
physics?

In this quote, the learning assistant discusses how she focuses on how computation helps

students solve the physics problems. She explicitly states how this is the important part of

computation, and how it is more important than just the skill of being able to program. She

expresses that what she wants students to understand how exactly it is that the computer

is helping them. In this way, she suggests that the desired teaching outcome she has for her

students is that they gain an understanding of the capabilities of computation.

In the characteristic to moderate theme, the computation as a tool focus is characterized

by student attention to programming details. This means an attention to preventing students

from becoming too caught up in the details of syntax and coding, to the detriment of their

understanding the larger picture of computation. One learning assistant moderating this

characteristic said,

When it’s just a spacing error that they’ve been working on for ten minutes,
that’s not super productive. And hopefully they just call me over when that
happens, but I’m sure there are times when they’re just messing with an error
for 15 minutes, or ... probably not that long. Hopefully not to that point, but

170

where they still have no idea how to fix it, and it’s not helping them learn the
physics, and it’s not really helping them learn coding, ’cause they’re just trying
to find this tiny error.

In this quote, the learning assistant describes a difficulty students often have during the

computational problems: getting their code properly spaced and tabbed. She discusses how

spending too much time on this sort of error is not a good use of the students’ time, explicitly

noting that it does not help them learn the physics concepts or coding. She also expresses

that she hopes she is able to intervene before the students waste too much time on a task

like this. By treating the navigation of this sort of error as not aligned with learning coding

and by indicating that she does not want students to spend time on these types of issues,

she indicates a desire for students to not focus on unimportant programming details to the

neglect of other computational skills. In this way, she expresses that student attention to

programming details is a characteristic she moderates.

In the teaching strategy theme, the computation as a tool focus is characterized by en-

couraging reflection on coding. This means prompting students to think more deeply about

how their code functions and how they can best utilize the program. A learning assistant

discussing this strategy said,

I would get them to focus on talking about how the code is supposed to be
iterative. It’s supposed to do the calculations for you. You shouldn’t be doing the
calculations outside of it and inputting numbers because then you’re constraining
the code to specific parameters, [but] the code is very versatile. You’re supposed
to be able to just hit run, change things to see how changing the variables affects
the system.

In this quote, the learning assistant discusses how he will encourage his students to talk

about what the code is able to do, such as carrying out calculations and allowing one to keep

parameters variable. He emphasizes that the code is “very versatile” and that this should be

171

leveraged, and he describes how he encourages his students to discuss these matters. This

focus indicates a teaching strategy that is based on encouraging students to reflect on coding.

These four variations considered at once illustrate a coherent approach to teaching com-

putational problems that is based around a focus on computation as a tool. Given a focus on

computation as a tool, it follows that this teaching approach is characterized by a belief that

the utility of computation is its ability to make difficult problems easier when leveraged, as

this represents an example of its application as a tool. It also follows that a learning assistant

with this approach would want their students to understand the capabilities of computation

that make it such an important tool. In order to achieve this desired teaching outcome, it

is consistent that a teaching strategy focused on encouraging reflection on coding would be

employed. Prompting such reflection would facilitate students developing an appreciation

for the affordances that computation provides them, and this emphasis makes sense given a

focus on computation as a tool. The characteristic to moderate in this approach is student

attention to programming details, which is also consistent with a belief that computation is

a valuable tool. A learning assistant with this approach perceives computation to be more

than simply entering syntax, instead treating computation as an important tool that can be

utilized in modeling and solving difficult problems. Correspondingly, they do not wish their

students to become bogged down in the details of syntax and leave with the impression that

that is all that computation is. Rather, they want students to gain an appreciation of the

greater value of computation. In this way, these four variations come together to indicate

an approach to teaching computational problems that is focused on computation as a tool.

172

Shifting perceptions of learning focus

The shifting perceptions of learning focus is characterized by an emphasis on computational

problems as an opportunity to affect students’ beliefs about and approaches to learning.

Computation is viewed as more than a pedagogical or problem-solving tool, and is instead

treated as a means to impact students’ ongoing ideas about how they learn and approach

problems.

In the utility of computation theme, the shifting perceptions of learning focus is charac-

terized by the belief that computation offers a space for broader skills to be developed. These

skills could include how to collaborate effectively, how to think in new ways, and how to

persevere through challenging problems. One learning assistant with such a belief said,

That’s [the computation problems] definitely where I see them learn a lot of
effective ways to work in a group or in a team, which that’s a big part of this
class. Going forward, they definitely will need to know how to do that.

In this quote, the learning assistant describes how she has observed that the computation

days are where the students most learn how to collaborate. She goes on to state that this

is an important skill for them to know beyond the course. With this, she indicates a belief

that the computational problems’ utility is their ability to offer students a space to develop

broader skills that go beyond physics and computation.

In the teaching outcome theme, the shifting perceptions of learning focus is characterized

by a new approach to learning. This refers to a desire for students to develop a new approach

to their learning through the computational problems. A learning assistant with this goal

for their teaching said,

173

I was trying to get them to see that, “Okay, if these two conditions are
conflicting, then that statement will never be true, so that segment of code will
never run.” [...]
If it’s high up, then that just stops the code. Trying to get
them to think through it logically rather than just like, “Okay, well let’s just
type something in and see if that changes anything,” so stepping back and going,
“Okay, well we know that this is true, we know this, and let’s just look at this
from a logical physics perspective rather than a coding perspective.”

In this quote, the learning assistant describes a situation in which her students’ code

was not running because they had entered conflicting conditions. She explains how she

wanted her students to think “logically” through the code and error, rather than simply

making changes arbitrarily, demonstrating a desire she had for her students to take a more

intentional and methodical approach to their problem-solving. This sort of focus on changing

how students approach their learning and problem-solving is indicative of a new approach

to learning to being her intended teaching outcome.

In the characteristic to moderate theme, the shifting perceptions of learning focus is

characterized by student attitudes. When moderating this characteristic, a learning assistant

focuses on ensuring that students do not become too frustrated or unhappy while working

on the computational problems. A learning assistant moderating this characteristic with her

teaching said,

If they say something like “this is so frustrating, like I just want to give up,”
then you’ll see them kind of start to get disengaged I guess. Like while you’re
doing the trial and error, usually the whole group is kind of looking at what’s
going on and is involved and wants to fix it and make it work. When you see
them start to sort of you know, fall back and give up essentially then that’s kind
of where you want to make sure that you get them on the right track I guess.

In this quote, the learning assistant describes how she identifies when students have

become frustrated to the point of no longer participating. She notes that at the beginning of

174

problems, when groups are typically testing what different changes to the code do, students

are generally engaged, but that as the problem progresses, frustration can lead to students

becoming disengaged. She highlights that it is important to make sure that students do

not get to that point of frustration. This focus on preventing students from becoming too

demoralized indicates that she perceives student attitudes to be an important characteristic

to moderate.

In the teaching strategy theme, the shifting perceptions of learning focus is characterized

by leveraging collaboration. When employing this strategy, learning assistants capitalize on

the fact that the course is a collaborative learning environment, and that students work in

groups. One learning assistant discussing this teaching strategy said,

I might say something like you know, ask somebody, ask a group what they
are doing and if someone responds and it looks like the other two aren’t paying
any attention, I might ask, “Oh, are you guys good with that?” or like “Are you
guys on the same page?” or “Do these guys understand that?” or something like
that to sort of let them know that they should be conversing.

In this quote, the learning assistant describes how she will ask questions of a group that

are targeted at assessing if there is agreement amongst the students and if all the students

understand the group’s work. She explains that she does this to convey to them that they

should all be discussing the problem and concepts with each other as they work. This

focus on encouraging students to take advantage of the fact that they are working in groups

by having more discussions is an example of employing a teaching strategy that leverages

collaboration.

In combination, these four variations describe an approach to teaching computational

problems that is focused on computation as a way to shift students’ perceptions of learning.

With this view of computation in mind, it follows that the desired outcome of a learning

175

assistant’s teaching would be for students to adopt a new approach to their learning from

the avenue for this that computation provides. It also is consistent that a learning assistant

focused on computation as a way to shift learning perceptions would believe the utility of

computation to be that it provides a space for students to develop skills that are beyond

content. Skills like collaboration and intentionality when problem-solving are tied to how

students’ approach and perceive their learning, and so a learning assistant with a focus on

shifting perceptions of learning would emphasis computation’s utility in developing those

skills. The shifting perceptions of learning focus is characterized by a teaching strategy that

leverages collaboration. This strategy is also aligned with this focus because by leveraging

collaboration, a learning assistant utilizes an aspect of the learning environment that is

specifically designed to alter how students approach their learning and problem-solving.

This strategy thus implicitly addresses how students approach their learning, and continues

the focus on computation as a way to shift perceptions of learning. The characteristic to

moderate corresponding to this approach is student attitudes. In order to attempt to shift

students’ perceptions of learning through the computational problems, it is unsurprising that

a learning assistant would attend to student attitudes, and in particular, make sure students

did not become frustrated to the point of disengagement. If a learning assistant is focused

on shifting students’ perceptions of learning, it is understandable that they would believe

that engagement and a positive attitude are necessities to being able to influence students

in this way. Thus these four variations when considered together comprise an approach to

teaching computational problems that focuses on their ability to shift students’ perceptions

of learning.

176

Confirming final categories of description

Having defined and articulated the reasoning behind the categories of description in Table

A.13, the final step in analysis was to review the remaining preliminary themes in Table A.1

and confirm that they were accounted for within the final four themes, or that they could

justifiably be neglected.

Several preliminary themes were already accounted for within the final themes upon

closer analysis.

It became evident that some preliminary themes had emerged based on

characteristics that were superficial in light of the deeper themes and categories that had

developed. In specific, the written feedback, varying levels of student programming experience,

acceptable places to end computation days,minimally working programs, solution manuals,

and teaching challenges themes were largely already described by the final four themes in

Table A.13. This is because many of the portions of transcript that had initially been sorted

into these preliminary themes were simply context-specific discussions of learning assistants’

teaching strategies. An example appears in the quote below,

[Using minimally working programs] impacts my teaching a lot because I feel
like I only really focus on the things I teach them how to code, like understand
the code, or more like the while loops. I teach them about what’s already there
and it’s not like I’m starting like this is what code is, and this is how you code
things, like a coding class would teach you. And so you can focus specifically on
while loops and if statements, which really limits the amount of coding you’re
necessarily teaching them and you can focus more on the physics side of it.

While this learning assistant discusses minimally working programs in this excerpt, he

ultimately is discussing that he seeks to avoid programming becoming a distraction. He

emphasizes how he teaches computational elements like while loops and if statements, but

does not get into the details of programming “like a coding class would”. In this way, he

177

expresses an attention to ensuring that those details do not become the students’ focus. Thus,

this excerpt is accounted for within the characteristic to moderate theme, specifically the

student attention to programming details variation. The majority of the transcript sorted into

the aforementioned preliminary themes was similarly better captured by the final themes.

Other preliminary themes were eliminated because they were found to not be central

to our phenomenon of interest:

learning assistants’ approaches to teaching computational

problems. The different student approaches to computation, differences in student behavior

solving computation versus analytic problems, what students should do when solving compu-

tation problems, why code in groups, and student challenges themes were neglected because

upon closer analysis, it was observed that while they did relate to learning assistants’ per-

ceptions of the computational problems, they did not provide insight into their approaches

to teaching. In the student challenges, different student approaches to computation, and dif-

ferences in student behavior solving computation versus analytic problems themes, learning

assistants simply shared their observations of students, but did not connect these observa-

tions to their teaching. In the why code in groups theme (and the portions of the written

feedback, varying levels of student programming experience, minimally working programs, and

solution manuals themes that were not already accounted for by the final themes), learning

assistants shared their opinions on the design of P Cubed, but again, did not connect these

opinions to their teaching and did not elaborate sufficiently for their comments to give insight

into their approach to teaching. In the what students should do when solving computation

problems theme, learning assistants discussed the ideal problem-solving approach to compu-

tational problems, but did not describe why they held that belief or how it impacted their

teaching. In all of these cases, interesting information about the learning assistants’ percep-

tions of computational problems was revealed, but this information was not productive in

178

describing their teaching approach.

For example, in the student challenges theme, one learning assistant discussed a problem

that requires the students to model the motion of a rotating disk, in which the given code

defines the y-axis to be the axis of rotation. She notes that this causes confusion for the

students, because in analytic problems, the z-axis is often taken to be the axis of rotation.

I think, I mean like, I feel like they are going to get confused about anything
new, but that one, I know there was a lot of confusion with the different things.
There was an axis and the origin, and so I know I had discussions with both my
groups about what those meant and think it’s also confusing because it’s like in
the z-axis, but I think on the code it’s on the y-axis, something like that. So, I
don’t know, all of those changes, kinda I think make those a little bit difficult to
understand.

The learning assistant’s discussion of how students struggle with coordinate systems in

the computational problems gives some insight into her opinions on computational problems

and the students, but it does not give insight into how she approaches her teaching overall.

While she does state that she “had discussions” with her groups about this particular issue,

it doesn’t reveal her teaching strategy or teaching outcome, and it doesn’t give information

that is helpful in identifying the variations in learning assistants’ teaching approaches.

Having thus confirmed that the final themes and categories in Table A.13 were sufficient

to meaningfully describe the learning assistants’ experiences as they related to their teaching

approaches, analysis was concluded.

Validation

In addition to the processes described in this appendix, the results were subject to validation

throughout analysis and upon completion. The entire analysis process was conducted in

179

regular consultation with a researcher with expertise in phenomenography. All analysis

decisions were subject to scrutiny and discussion, and had to be adequately justified through

evidence from the transcript and defensible arguments. This included decisions to merge

themes or variations, eliminate themes or variations, and especially decisions on the final

groupings of variations defining each category of description. In addition to this on-going

validation, at several intermediate points in analysis, preliminary results were presented to

external groups for critique and feedback. Finally, upon the conclusion of analysis, the final

results, the interview transcripts, and examples of coded transcript were provided to another

researcher for external validation of the results. Based on the evidence and arguments

provided throughout this appendix and the validation of these external researchers, we argue

that the results presented here constitute a novel and legitimate phenomenographic result

that gives meaningful and useful insight into learning assistants’ approaches to teaching

computational problems.

180

Appendix B

Sample computational problem

The following appendix provides an example of a computational problem that is taught in

the course in which the study presented in Chapter 3 took place. In the class session be-

fore the students solve the problem here, they solve analytically for the radius and speed

of geostationary orbit. This problem requests that they now model their solution computa-

tionally. The problem statement, the VPython code students are provided, and an example

of correctly modified code are presented here.

Problem statement

Carver is impressed with your work, but remains unconvinced by your predictions. He has

asked you to write a simulation that models the orbit of the satellite. To truly convince

Carver, the simulation should include representations of the net force acting on the spacecraft,

which has a mass of 15 x 103 kg. Your simulation should be generalized enough to model

other types of orbits including elliptical ones.

181

Given code

import d i v i s i o n

from f u t u r e
from v i s u a l
from v i s u a l . graph import ∗
from p h y s u t i l

import ∗

import ∗

#Window s e t u p
s c e n e . range=7e7
s c e n e . width = 1024
s c e n e . h e i g h t = 760

#O b j e c t s
Earth = s p h e r e ( pos=v e c t o r ( 0 , 0 , 0 ) ,
S a t e l l i t e = s p h e r e ( pos=v e c t o r ( 7∗ Earth . r a d i u s , 0 , 0 ) ,

BlueMarble )

r a d i u s =6.4 e6 , m a t e r i a l=m a t e r i a l s .

r a d i u s =1e6 ,

c o l o r=

c o l o r . red , m a k e t r a i l=True )

#Parameters and I n i t i a l C o n d i t i o n s
m S a t e l l i t e = 1
p S a t e l l i t e = v e c t o r ( 0 , 5 0 0 0 , 0 )

#Time and time s t e p
t = 0
t f = 60∗60∗24
dt = 1

#MotionMap/Graph
S a t e l l i t e M o t i o n M a p = MotionMap ( S a t e l l i t e ,

l a b e l M a r k e r O r d e r=F a l s e )

t f , 2 0 , m a r k e r S c a l e =2000 ,

#C a l c u l a t i o n Loop
while t < t f :

r a t e ( 1 0 0 0 0 )

Fnet = v e c t o r ( 0 , 0 , 0 )
p S a t e l l i t e = p S a t e l l i t e + Fnet∗ dt
S a t e l l i t e . pos = S a t e l l i t e . pos + ( p S a t e l l i t e / m S a t e l l i t e )∗ dt

S a t e l l i t e M o t i o n M a p . update ( t , p S a t e l l i t e / m S a t e l l i t e )

t = t + dt

#Earth R o t a t i o n (IGNORE)
t h e t a = 7 . 2 9 e−5∗dt

182

Earth . r o t a t e ( a n g l e=t h e t a , a x i s=v e c t o r ( 0 , 0 , 1 ) , o r i g i n=Earth . pos )

Example of correctly modified code

import d i v i s i o n

import ∗

from f u t u r e
from v i s u a l
from v i s u a l . graph import ∗
from p h y s u t i l

import ∗

#Window s e t u p
s c e n e . range = 7 e7
s c e n e . width = 1024
s c e n e . h e i g h t = 760

#O b j e c t s
Earth = s p h e r e ( pos=v e c t o r ( 0 , 0 , 0 ) ,

BlueMarble )

r a d i u s =6.4 e6 , m a t e r i a l=m a t e r i a l s .

S a t e l l i t e = s p h e r e ( pos=v e c t o r ( 4 . 2 e7 , 0 , 0 ) ,

r a d i u s =1e6 ,

c o l o r=c o l o r . red ,

m a k e t r a i l=True )

#Parameters and I n i t i a l C o n d i t i o n s
m S a t e l l i t e = 15 e3
p S a t e l l i t e = m S a t e l l i t e ∗ v e c t o r ( 0 , 3 0 7 3 , 0 )
G = 6 . 6 7 e−11
mEarth = 5 . 9 7 e24

#Time and time s t e p
t = 0
t f = 60∗60∗24
dt = 1

#MotionMap/Graph
S a t e l l i t e M o t i o n M a p = MotionMap ( S a t e l l i t e ,

l a b e l M a r k e r O r d e r=F a l s e )

t f , 2 0 , m a r k e r S c a l e =2000 ,

FnetMotionMap = MotionMap ( S a t e l l i t e ,

t f , 2 0 , m a r k e r S c a l e =2000 ,

l a b e l M a r k e r O r d e r=F a l s e )

s e p a r a t i o n G r a p h = PhysGraph ( numPlots =1)

#C a l c u l a t i o n Loop
while t < t f :

r a t e ( 1 0 0 0 0 )
Fgrav = −G∗ m S a t e l l i t e ∗mEarth∗ S a t e l l i t e . pos / (mag( S a t e l l i t e . pos )

∗∗ 3 )

183

Fnet = Fgrav
p S a t e l l i t e = p S a t e l l i t e + Fnet∗ dt
S a t e l l i t e . pos = S a t e l l i t e . pos + ( p S a t e l l i t e / m S a t e l l i t e )∗ dt

S a t e l l i t e M o t i o n M a p . update ( t , p S a t e l l i t e / m S a t e l l i t e )
FnetMotionMap . update ( t , Fnet )

s e p a r a t i o n G r a p h . p l o t ( t , mag( S a t e l l i t e . pos ) )

t = t + dt

#Earth R o t a t i o n (IGNORE)
t h e t a = 7 . 2 9 e−5∗dt
Earth . r o t a t e ( a n g l e=t h e t a , a x i s=v e c t o r ( 0 , 0 , 1 ) , o r i g i n=Earth . pos )

184

Appendix C

Interview protocol

The following appendix presents the protocol used for the interviews carried out for the

study presented in Chapter 3. As described in Chapter 3, the protocol does not represent a

strict order of required questions. Instead, it served as a guide for the interviewer to ensure

that as complete a description as possible of the participants’ experiences was obtained.

Background

(cid:4) What is it like to teach in P-Cubed?
(cid:4) How long have you been teaching in P-Cubed?
(cid:4) Did you have any teaching experience prior to P-Cubed? Could you describe it for

me?

(cid:4) Have you had any other teaching experiences since beginning in p-cubed? Can you

describe them for me?

(cid:4) What sort of computational experiences did you have before P Cubed?
(cid:4) Since teaching in P Cubed have you gained any further computational experiences?

P Cubed context

(cid:4) Describe what computation looks like in P-Cubed.
(cid:4) Describe what a typical computational problem looks like.

(cid:3) Can you talk about a specific problem?
(cid:3) What kinds of tasks are students asked to do?

(cid:4) What is a model? What is modeling?

185

(cid:4) Is there a difference between models on analytic days and models on computation days?
(cid:4) Why do you think the computational problems are designed the way they are?
(cid:4) What do you think of using VPython?
(cid:3) How does it impact your teaching?

(cid:4) What do you think of minimally working programs?

(cid:3) How do they impact your teaching?

(cid:4) Why do you think we pick the topics we do for the computational problems?
(cid:4) Are there any differences among the computational problems?
(cid:4) What are the learning goals for students are on computation days?
(cid:4) We give the students some other tools to use, such as the four quadrants and written

feedback. Can you describe your experience with those?

(cid:3) Why do you think we have them?
(cid:3) How did you use them in your teaching?

(cid:4) Why do we have students work in groups? Why do computation in groups?
(cid:4) Why do we have 15 instructors in the room?
(cid:4) At the beginning of the semester, we have you go through some training about being

an LA. Can you tell me your experience of it?

(cid:3) How did it affect your teaching?
(cid:3) What do you think the instructors’ goals for your teaching are?

(cid:4) During the semester, we have pre-class and post-class meetings every week. What was

your experience of those?

(cid:3) Why do you think we have them?
(cid:3) How did they affect your teaching?
(cid:3) Do you do anything else to prepare for teaching?

(cid:4) We give you solutions including tutor questions. Can you tell me your experience with

them?

(cid:3) What do you think their purpose is?
(cid:3) How do you use them?

186

Teaching in P Cubed

(cid:4) Moving into the classroom specifically, can you describe your teaching on a computa-

tional week, starting with Tuesday and then moving onto Thursday?

(cid:4) Are there specific things you do and say? Why?
(cid:4) Are there specific student actions/comments you watch out for?

(cid:3) What do you do when you see them happen?

(cid:4) Are there things you avoid doing? Why?
(cid:4) How do you decide when to go talk to a group?
(cid:4) How do you decide when to give information versus give hints versus ask questions?
(cid:3) Is it affected by the issue being a syntax error versus a conceptual misunderstand-

ing versus something else?

(cid:4) What kinds of questions do you ask? Why?
(cid:4) How do you decide whom to ask questions to?
(cid:4) Where do you want a group to be after you interact with them?
(cid:4) Do you see groups take different approaches to computational problems?

(cid:3) How do you deal with those approaches?

(cid:4) Students often struggle with the geostationary orbit problem. Can you describe what

you’ve experienced and how you handle it?

(cid:4) What do the students struggle with on computation days?

(cid:3) Why do you think this is challenging for them?
(cid:3) How do you help students resolve the issues you’ve mentioned?
(cid:3) Do you think students learn from these situations?
(cid:3) How do you decide when struggling or frustration is okay?

(cid:4) How does teaching different students affect your teaching? Different groups?
(cid:4) How do you balance having two groups?
(cid:4) Do you change how you work with a given group over time?
(cid:4) What do you see as your role as a teacher in P Cubed?
(cid:4) What is your role on the teaching team?

187

General thoughts on teaching

(cid:4) How would you describe your general approach to teaching?

(cid:3) How did you develop that approach?
(cid:3) How does that apply to computational problems?

(cid:4) When do you know you have done a good job as a teacher?

(cid:3) What kind of progress do you look for?

(cid:4) When do you know that your students have learned something?
(cid:4) What does understanding mean?
(cid:3) What shows a student has it?

(cid:4) What’s the most important thing for students to do in class to learn?

(cid:3) How does that thing lead to understanding?
(cid:3) How do you make sure students do that thing?

(cid:4) Is it important for students to like you? Why?
(cid:4) By the end of the semester, what is your goal for the students you work with?
(cid:4) Why did you want to be an LA?
(cid:4) Why do you think you were chosen to be an LA?
(cid:4) What do you think you personally bring to the classroom as a teacher?

Closing Question

(cid:4) Is there anything you’d change about the way P Cubed uses computational problems?

188

BIBLIOGRAPHY

189

BIBLIOGRAPHY

[1] Norman R Augustine et al. Rising above the gathering storm: Energizing and employ-

ing america for a brighter economic future. Retrieved March, 19:2008, 2005.

[2] Robert Atkinson and Merrilea Mayo. Refueling the us innovation economy: Fresh ap-
proaches to science, technology, engineering and mathematics (stem) education. 2010.

[3] National Research Council et al. Education for life and work: Developing transferable

knowledge and skills in the 21st century. National Academies Press, 2013.

[4] US National Academy of Engineering. The engineer of 2020: visions of engineering in

the new century. National Academies Press Washington, DC, 2004.

[5] National Research Council et al. The mathematical sciences in 2025. National

Academies Press, 2013.

[6] National Research Council et al.

STEM integration in K-12 education: Status,

prospects, and an agenda for research. National Academies Press, 2014.

[7] National Research Council et al. A framework for K-12 science education: Practices,

crosscutting concepts, and core ideas. National Academies Press, 2012.

[8] National Research Council et al. Adapting to a changing world: Challenges and op-

portunities in undergraduate physics education. National Academies Press, 2013.

[9] National Research Council et al. Engineering in K-12 education: Understanding the

status and improving the prospects. National Academies Press, 2009.

[10] Susan Singer and Karl A Smith. Discipline-based education research: Understanding
and improving learning in undergraduate science and engineering. Journal of Engi-
neering Education, 102(4):468–471, 2013.

[11] David Hammer. Misconceptions or p-prims: How may alternative perspectives of
cognitive structure influence instructional perceptions and intentions. The Journal of
the Learning Sciences, 5(2):97–127, 1996.

[12] David Hammer. More than misconceptions: Multiple perspectives on student knowl-
edge and reasoning, and an appropriate role for education research. American Journal
of Physics, 64(10):1316–1325, 1996.

[13] John P Smith III, Andrea A Disessa, and Jeremy Roschelle. Misconceptions recon-
ceived: A constructivist analysis of knowledge in transition. The journal of the learning
sciences, 3(2):115–163, 1994.

[14] David Hammer. Student resources for learning introductory physics. American Journal

of Physics, 68(S1):S52–S59, 2000.

190

[15] David Hammer, Andrew Elby, Rachel E Scherr, and Edward F Redish. Resources,
framing, and transfer. Transfer of learning from a modern multidisciplinary perspective,
89, 2005.

[16] Julie Libarkin. Concept inventories in higher education science. In BOSE Conf, 2008.

[17] David Hestenes, Malcolm Wells, and Gregg Swackhamer. Force concept inventory. The

physics teacher, 30(3):141–158, 1992.

[18] Antti Savinainen and Phillip Scott. The force concept inventory: A tool for monitoring

student learning. Phys. Educ., 37(1):45–52, 2002.

[19] Richard R Hake. Interactive-engagement versus traditional methods: A six-thousand-
student survey of mechanics test data for introductory physics courses. American
journal of Physics, 66(1):64–74, 1998.

[20] Adam P Fagen, Catherine H Crouch, and Eric Mazur. Peer instruction: Results from

a range of classrooms. The physics teacher, 40(4):206–209, 2002.

[21] Andrew T Lumpe and John R Staver. Peer collaboration and concept development:
Learning about photosynthesis. Journal of Research in Science Teaching, 32(1):71–98,
1995.

[22] Eugenia Etkina and AV Heuvelen. Investigative science learning environment: Using

the processes of science and cognitive strategies to learn physics. 2001.

[23] Shaona Zhou, Jing Han, Kathleen Koenig, Amy Raplinger, Yuan Pi, Dan Li, Hua
Xiao, Zhao Fu, and Lei Bao. Assessment of scientific reasoning: The effects of task
context, data, and design on student reasoning in control of variables. Thinking skills
and creativity, 19:175–187, 2016.

[24] Anita M Schuchardt and Christian D Schunn. Modeling scientific processes with math-
ematics equations enhances student qualitative conceptual understanding and quanti-
tative problem solving. Science Education, 100(2):290–320, 2016.

[25] Patricia Heller, Ronald Keith, and Scott Anderson. Teaching problem solving through
cooperative grouping. part 1: Group versus individual problem solving. American
journal of physics, 60(7):627–636, 1992.

[26] Brian W Frank and Rachel E Scherr. Interactional processes for stabilizing conceptual
coherences in physics. Physical Review Special Topics-Physics Education Research,
8(2):020101, 2012.

[27] Allan Collins and William Ferguson. Epistemic forms and epistemic games: Structures

and strategies to guide inquiry. Educational psychologist, 28(1):25–42, 1993.

[28] Jonathan Tuminaro and Edward F Redish. Elements of a cognitive model of physics
problem solving: Epistemic games. Physical Review Special Topics-Physics Education
Research, 3(2):020101, 2007.

191

[29] Ying Chen, Paul W Irving, and Eleanor C Sayre. Epistemic game for answer mak-
ing in learning about hydrostatics. Physical Review Special Topics-Physics Education
Research, 9(1):010108, 2013.

[30] Rachel E Scherr and David Hammer. Student behavior and epistemological fram-
ing: Examples from collaborative active-learning activities in physics. Cognition and
Instruction, 27(2):147–174, 2009.

[31] Patricia Heller and Mark Hollabaugh. Teaching problem solving through cooperative
grouping. part 2: Designing problems and structuring groups. American Journal of
Physics, 60(7):637–644, 1992.

[32] Paul W Irving, Michael J Obsniuk, and Marcos D Caballero. P3: a practice focused

learning environment. European Journal of Physics, 38(5):055701, 2017.

[33] David E Meltzer and Kandiah Manivannan. Transforming the lecture-hall environ-
ment: The fully interactive physics lecture. American Journal of Physics, 70(6):639–
654, 2002.

[34] David R Sokoloff and Ronald K Thornton. Using interactive lecture demonstrations
to create an active learning environment. In AIP Conference Proceedings, volume 399,
pages 1061–1074. AIP, 1997.

[35] Christopher Keller, N Finkelstein, K Perkins, S Pollock, C Turpen, and M Dubson.
Research-based practices for effective clicker use. In AIP Conference Proceedings, vol-
ume 951, pages 128–131. AIP, 2007.

[36] Catherine H Crouch and Eric Mazur. Peer instruction: Ten years of experience and

results. American journal of physics, 69(9):970–977, 2001.

[37] Brigid JS Barron, Daniel L Schwartz, Nancy J Vye, Allison Moore, Anthony Petrosino,
Linda Zech, and John D Bransford. Doing with understanding: Lessons from research
on problem-and project-based learning. Journal of the Learning Sciences, 7(3-4):271–
311, 1998.

[38] Joseph S Krajcik and Phyllis C Blumenfeld. Project-based learning. na, 2006.

[39] Karen Cummings. The rensselaer studio model for learning and teaching: What have
we learned. In Proceedings of the National Research Council’s Workshop Linking Evi-
dence to Promising Practices in STEM Undergraduate Education, 2008.

[40] Patrick B Kohl and H Vincent Kuo. Chronicling a successful secondary implementation

of studio physics. American Journal of Physics, 80(9):832–839, 2012.

[41] Robert Beichner. The scale-up project: a student-centered active learning environment
for undergraduate programs. An invited white paper for the National Academy of
Sciences, 2008.

[42] Eric Brewe. Modeling theory applied: Modeling instruction in introductory physics.

American Journal of Physics, 76(12):1155–1160, 2008.

192

[43] Eugenia Etkina and Alan Van Heuvelen. Investigative science learning environment–
a science process approach to learning physics. Research-based reform of university
physics, 1, 2007.

[44] KK Perkins, WK Adams, SJ Pollock, ND Finkelstein, and CE Wieman. Correlating
student beliefs with student learning using the colorado learning attitudes about science
survey. In AIP Conference Proceedings, volume 790, pages 61–64. AIP, 2005.

[45] Wendy K Adams, Katherine K Perkins, Noah S Podolefsky, Michael Dubson, Noah D
Finkelstein, and Carl E Wieman. New instrument for measuring student beliefs about
physics and learning physics: The colorado learning attitudes about science survey.
Physical review special topics-physics education research, 2(1):010101, 2006.

[46] Eric Brewe, Laird Kramer, and George O’Brien. Modeling instruction: Positive at-
titudinal shifts in introductory physics measured with class. Physical Review Special
Topics-Physics Education Research, 5(1):013102, 2009.

[47] Steven J Pollock, Noah D Finkelstein, and Lauren E Kost. Reducing the gender gap
in the physics classroom: How sufficient is interactive engagement? Physical Review
Special Topics-Physics Education Research, 3(1):010107, 2007.

[48] Mercedes Lorenzo, Catherine H Crouch, and Eric Mazur. Reducing the gender gap in

the physics classroom. American Journal of Physics, 74(2):118–122, 2006.

[49] Adrian Madsen, Sarah B. McKagan, and Eleanor C. Sayre. Gender gap on concept
inventories in physics: What is consistent, what is inconsistent, and what factors
influence the gap? Phys. Rev. ST Phys. Educ. Res., 9:020121, 2013.

[50] Michael Prince. Does active learning work? a review of the research. Journal of

engineering education, 93(3):223–231, 2004.

[51] Michelene TH Chi. Active-constructive-interactive: A conceptual framework for dif-

ferentiating learning activities. Topics in cognitive science, 1(1):73–105, 2009.

[52] Chandra Turpen and Noah D Finkelstein. The construction of different classroom
norms during peer instruction: Students perceive differences. Physical Review Special
Topics-Physics Education Research, 6(2):020123, 2010.

[53] Chandra Turpen and Noah D. Finkelstein. Not all interactive engagement is the same:
Variations in physics professors’ implementation of peer instruction. Phys. Rev. ST
Phys. Educ. Res., 5:020101, 2009.

[54] Shirley Booth. On phenomenography, learning and teaching. Higher education research

& development, 16(2):135–158, 1997.

[55] Keith Trigwell and Michael Prosser. Towards an understanding of individual acts
of teaching and learning. Higher Education Research & Development, 16(2):241–252,
1997.

193

[56] Alanna Pawlak, Paul Irving, and Marcos Caballero. Identification of a shared answer-
making epistemic game in a group context. In Physics Education Research Conference
2015, PER Conference, pages 255–258, College Park, MD, July 29-30 2015.

[57] Alanna Pawlak, Paul W. Irving, and Marcos D. Caballero. Development of the modes

of collaboration framework. Phys. Rev. Phys. Educ. Res., 14:010101, Jan 2018.

[58] David P Maloney. Research on problem solving: Physics. Handbook of research on

science teaching and learning, pages 327–354, 1994.

[59] Edward F Redish and Richard N Steinberg. Teaching physics: Figuring out what

works. 1999.

[60] Lillian C McDermott. Research on conceptual understanding in mechanics. Physics

Today, 37:24–32, 1984.

[61] Lillian C McDermott and Edward F Redish. Resource letter: Per-1: Physics education

research. American journal of physics, 67(9):755–767, 1999.

[62] Anita Roychoudhury and Wolff-Michael Roth. Interactions in an open-inquiry physics

laboratory. International Journal of Science Education, 18(4):423–445, 1996.

[63] Derek Raine and Sarah Symons. Experiences of pbl in physics in uk higher education.

PBL in Context–Bridging Work and Education, page 67, 2005.

[64] Karen Sunday Cockrell, Julie A Hughes Caplow, and Joe F Donaldson. A context
for learning: Collaborative groups in the problem-based learning environment. The
Review of Higher Education, 23(3):347–363, 2000.

[65] John W Thomas, JR Margendoller, and A Michaelson. Project-based learning.

Overview. Novato, CA: Buck Institute for Education, 1998.

[66] Renata Holubova. Effective teaching methods–project-based learning in physics. On-

line Submission, 5(12):27–36, 2008.

[67] NGSS Lead States. Next Generation Science Standards: For States, By States. The

National Academies Press, Washington, DC, 2013.

[68] Ernest Behringer and Larry Engelhardt. Guest editorial: Aapt recommendations for
computational physics in the undergraduate physics curriculum, and the partnership
for integrating computation into undergraduate physics. American Journal of Physics,
85(5):325–326, 2017.

[69] Daniel T Kaplan. Teaching computation to undergraduate scientists. In ACM SIGCSE

Bulletin, volume 36, pages 358–362. ACM, 2004.

[70] David H McIntyre, Janet Tate, and Corinne A Manogue. Integrating computational
activities into the upper-level paradigms in physics curriculum at oregon state univer-
sity. American Journal of Physics, 76(4):340–346, 2008.

194

[71] Marcos D Caballero, Matthew A Kohlmyer, and Michael F Schatz. Implementing and
assessing computational modeling in introductory mechanics. Physical Review Special
Topics-Physics Education Research, 8(2):020106, 2012.

[72] Marcos D Caballero, John B Burk, John M Aiken, Brian D Thoms, Scott S Douglas,
Erin M Scanlon, and Michael F Schatz. Integrating numerical computation into the
modeling instruction curriculum. The Physics Teacher, 52(1):38–42, 2014.

[73] Ruth Chabay and Bruce Sherwood. Computational physics in the introductory

calculus-based course. American Journal of Physics, 76(4):307–313, 2008.

[74] Bruce Sherin, Andrea A diSessa, and David Hammer. Dynaturtle revisited: Learn-
ing physics through collaborative design of a computer model. Interactive Learning
Environments, 3(2):91–118, 1993.

[75] Uri Wilensky and Kenneth Reisman. Thinking like a wolf, a sheep, or a firefly: Learn-
ing biology through constructing and testing computational theories—an embodied
modeling approach. Cognition and instruction, 24(2):171–209, 2006.

[76] M Abrandt, Reinholdt Castensson, and Lars-Ove Dahlgren. Pbl from the teacher’s
perspective. conceptions of the tutor’s role within problem based learning. Higher
education, 36:437–447, 1998.

[77] Gillian Maudsley. Roles and responsibilities of the problem based learning tutor in
the undergraduate medical curriculum. BMJ: British Medical Journal, 318(7184):657,
1999.

[78] Eugenia Etkina. Helping graduate assistants teach physics: Problems and solutions.

Journal of Graduate Teaching Assistant Development, 7(3):123–37, 2000.

[79] Grant E Gardner and M Gail Jones. Pedagogical preparation of the science graduate

teaching assistant: Challenges and implications. Science Educator, 20(2):31, 2011.

[80] Benjamin T Spike and Noah D Finkelstein. Preparing tutorial and recitation instruc-
tors: A pedagogical approach to focusing attention on content and student reasoning.
American Journal of Physics, 80(11):1020–1026, 2012.

[81] Jacquelyn J Chini, Carrie L Straub, and Kevin H Thomas. Learning from avatars:
Learning assistants practice physics pedagogy in a classroom simulator. Physical Re-
view Physics Education Research, 12(1):010117, 2016.

[82] Dennis W Sunal, Jeanelle Hodges, Cynthia S Sunal, Kevin W Whitaker, L Michael
Freeman, Leo Edwards, Ronald A Johnston, and Michael Odell. Teaching science in
higher education: Faculty professional development and barriers to change. School
Science and mathematics, 101(5):246–257, 2001.

[83] Charles Henderson and Melissa H Dancy. Impact of physics education research on the
teaching of introductory quantitative physics in the united states. Physical Review
Special Topics-Physics Education Research, 5(2):020107, 2009.

195

[84] Valerie Otero, Steven Pollock, and Noah Finkelstein. A physics department’s role in
preparing physics teachers: The colorado learning assistant model. American Journal
of Physics, 78(11):1218–1224, 2010.

[85] Kara E Gray and Valerie K Otero. Analysis of former learning assistants’ views on
In AIP Conference Proceedings, volume 1179, pages 149–152.

cooperative learning.
AIP, 2009.

[86] Kara E. Gray, David C. Webb, and Valerie K. Otero. Effects of the learning assistant

model on teacher practice. Phys. Rev. Phys. Educ. Res., 12:020126, Sep 2016.

[87] Kara Gray and Valerie Otero. Analysis of learning assistants’ views of teaching and
learning. In Physics Education Research Conference 2008, volume 1064 of PER Con-
ference, pages 123–126, Edmonton, Canada, July 23-24 2008.

[88] M Prosser, K Trigwell, F Marton, and U Runesson. Relations between epistemolog-
ical beliefs and educational practice (as revealed in problem solving). Research and
Development in Higher Education, 20:611–619, 1997.

[89] David Stroupe. Examining classroom science practice communities: How teachers and
students negotiate epistemic agency and learn science-as-practice. Science Education,
98(3):487–516, 2014.

[90] Benjamin T Spike and Noah D Finkelstein. Examining the beliefs and practice of
teaching assistants: Two case studies. In AIP Conference Proceedings, volume 1289,
pages 309–312. AIP, 2010.

[91] Benjamin T Spike and Noah D Finkelstein. Design and application of a framework
for examining the beliefs and practices of physics teaching assistants. Physical Review
Physics Education Research, 12(1):010114, 2016.

[92] Sara Hennessy, Jocelyn Wishart, Denise Whitelock, Rosemary Deaney, Richard Brawn,
Linda La Velle, Angela McFarlane, Kenneth Ruthven, and Mark Winterbottom. Peda-
gogical approaches for technology-integrated science teaching. Computers & Education,
48(1):137–152, 2007.

[93] Maggie L Niess. Preparing teachers to teach science and mathematics with technol-
ogy: Developing a technology pedagogical content knowledge. Teaching and teacher
education, 21(5):509–523, 2005.

[94] Renee Michelle Goertzen, Rachel E Scherr, and Andrew Elby. Tutorial teaching as-
sistants in the classroom: Similar teaching behaviors are supported by varied beliefs
about teaching and learning. Physical Review Special Topics-Physics Education Re-
search, 6(1):010105, 2010.

[95] Anne M Raymond.

Inconsistency between a beginning elementary school teacher’s
mathematics beliefs and teaching practice. Journal for research in mathematics edu-
cation, pages 550–576, 1997.

196

[96] Matthew T Hora. Exploring faculty beliefs about student learning and their role in

instructional decision-making. The Review of Higher Education, 38(1):37–70, 2014.

[97] Charles Henderson, Edit Yerushalmi, Vince H. Kuo, Kenneth Heller, and Patricia
Heller. Physics faculty beliefs and values about the teaching and learning of problem
solving. ii. procedures for measurement and analysis. Phys. Rev. ST Phys. Educ. Res.,
3:020110, 2007.

[98] Noel Entwistle, Don Skinner, Dorothy Entwistle, and Sandra Orr. Conceptions and
beliefs about “good teaching”: An integration of contrasting research areas. Higher
Education Research & Development, 19(1):5–26, 2000.

[99] Randy Yerrick and Timothy Hoving. Obstacles confronting technology initiatives as
seen through the experience of science teachers: A comparative study of science teach-
ers’ beliefs, planning, and practice. Journal of Science Education and Technology,
8(4):291–307, 1999.

[100] Alejandra J Magana, Sean P Brophy, and George M Bodner. Instructors’ intended
learning outcomes for using computational simulations as learning tools. Journal of
Engineering Education, 101(2):220–243, 2012.

[101] Christine S Bruce and Camille A McMahon. Contemporary developments in teaching
and learning introductory programming: Towards a research proposal. Faculty of
Information Technology, QUT Teaching & Learning Report, (2/2002), 2002.

[102] Matthew J Koehler and Punya Mishra. What happens when teachers design educa-
tional technology? the development of technological pedagogical content knowledge.
Journal of educational computing research, 32(2):131–152, 2005.

[103] Lillian C McDermott, Mark L Rosenquist, and Emily H Van Zee. Student difficulties
in connecting graphs and physics: Examples from kinematics. American Journal of
Physics, 55(6):503–513, 1987.

[104] David E Trowbridge and Lillian C McDermott. Investigation of student understanding
of the concept of velocity in one dimension. American journal of Physics, 48(12):1020–
1028, 1980.

[105] Fred M Goldberg and John H Anderson. Student difficulties with graphical represen-

tations of negative values of velocity. The Physics Teacher, 27(4):254–260, 1989.

[106] Michael McCloskey, Alfonso Caramazza, and Bert Green. Curvilinear motion in
the absence of external forces: Naive beliefs about the motion of objects. Science,
210(4474):1139–1141, 1980.

[107] Igal Galili and Varda Bar. Motion implies force: where to expect vestiges of the

misconception? International Journal of Science Education, 14(1):63–81, 1992.

[108] John Clement. Students’ preconceptions in introductory mechanics. American Journal

of physics, 50(1):66–71, 1982.

197

[109] Alfonso Caramazza, Michael McCloskey, and Bert Green. Naive beliefs in “sophisti-
cated” subjects: Misconceptions about trajectories of objects. Cognition, 9(2):117–123,
1981.

[110] Mehmet Sahin. Effects of problem-based learning on university students’ epistemo-
logical beliefs about physics and physics learning and conceptual understanding of
newtonian mechanics. Journal of Science Education and Technology, 19(3):266–275,
2010.

[111] Lin Ding. Verification of causal influences of reasoning skills and epistemology on
physics conceptual learning. Physical Review Special Topics-Physics Education Re-
search, 10(2):023101, 2014.

[112] Rene Amigues. Peer interaction in solving physics problems: Sociocognitive confronta-
tion and metacognitive aspects. Journal of Experimental Child Psychology, 45(1):141–
158, 1988.

[113] Filip Dochy, Mien Segers, Piet Van den Bossche, and David Gijbels. Effects of problem-

based learning: A meta-analysis. Learning and instruction, 13(5):533–568, 2003.

[114] Richard N Steinberg and Mel S Sabella. Performance on multiple-choice diagnostics

and complementary exam problems. The Physics Teacher, 35(3):150–155, 1997.

[115] Douglas Huffman and Patricia Heller. What does the force concept inventory actually

measure? The Physics Teacher, 33(3):138–143, 1995.

[116] Rebecca S Lindell, Elizabeth Peak, and Thomas M Foster. Are they all created equal?
a comparison of different concept inventory development methodologies. In AIP Con-
ference Proceedings, volume 883, pages 14–17. AIP, 2007.

[117] Rochelle Guti´errez and Ezekiel Dixon-Rom´an. Beyond gap gazing: How can thinking
In

about education comprehensively help us (re) envision mathematics education?
Mapping equity and quality in mathematics education, pages 21–34. Springer, 2010.

[118] Robert GM Hausmann, Michelene TH Chi, and Marguerite Roy. Learning from collab-
orative problem solving: An analysis of three hypothesized mechanisms. In Proceedings
of the Annual Meeting of the Cognitive Science Society, volume 26, 2004.

[119] Gail Richmond and Joanne Striley. Making meaning in classrooms: Social processes
in small-group discourse and scientific knowledge building. Journal of Research in
Science Teaching: The Official Journal of the National Association for Research in
Science Teaching, 33(8):839–858, 1996.

[120] Charles W Anderson, J David Holland, and Annemarie S Palincsar. Canonical and
sociocultural approaches to research and reform in science education: The story of
juan and his group. The Elementary School Journal, 97(4):359–383, 1997.

[121] Kelly Grindstaff and Gail Richmond. Learners’ perceptions of the role of peers in a
research experience: Implications for the apprenticeship process, scientific inquiry, and
collaborative work. Journal of Research in Science Teaching, 45(2):251–271, 2008.

198

[122] Gamze Sezgin Selccedil et al. The effects of problem-based learning on pre-service
teachers achievement, approaches and attitudes towards learning physics. International
Journal of Physical Sciences, 5(6):711–723, 2010.

[123] Erving Goffman. Forms of talk. University of Pennsylvania Press, 1981.

[124] Luke D Conlin, Ayush Gupta, Rachel E Scherr, and David Hammer. The dynamics
of students’ behaviors and reasoning during collaborative physics tutorial sessions. In
AIP Conference Proceedings, volume 951, pages 69–72. AIP, 2007.

[125] Paul W Irving, Mathew Sandy Martinuk, and Eleanor C Sayre. Transitions in students’
epistemic framing along two axes. Physical Review Special Topics-Physics Education
Research, 9(1):010111, 2013.

[126] Brigid Barron. Achieving coordination in collaborative problem-solving groups. The

journal of the learning sciences, 9(4):403–436, 2000.

[127] Brigid Barron. When smart groups fail. The journal of the learning sciences, 12(3):307–

359, 2003.

[128] Kathleen Hogan, Bonnie K Nastasi, and Michael Pressley. Discourse patterns and
collaborative scientific reasoning in peer and teacher-guided discussions. Cognition
and instruction, 17(4):379–432, 1999.

[129] Carla Van Boxtel, Jos Van der Linden, and Gellof Kanselaar. Collaborative learn-
ing tasks and the elaboration of conceptual knowledge. Learning and instruction,
10(4):311–330, 2000.

[130] Alandeom W Oliveira and Troy D Sadler. Interactive patterns and conceptual con-
vergence during student collaborations in science. Journal of Research in Science
Teaching, 45(5):634–658, 2008.

[131] Eva Hammar Chiriac. A scheme for understanding group processes in problem-based

learning. Higher Education, 55(5):505–518, 2008.

[132] Wolff-Michael Roth and Anita Roychoudhury. The social construction of scientific
concepts or the concept map as device and tool thinking in high conscription for social
school science. Science education, 76(5):531–557, 1992.

[133] Noreen M Webb, Sydney H Farivar, and Ann M Mastergeorge. Productive helping in

cooperative groups. cse technical report. 2001.

[134] Diana HJM Dolmans, Sofie MM Loyens, H´el`ene Marcq, and David Gijbels. Deep and
surface learning in problem-based learning: a review of the literature. Advances in
Health Sciences Education, 21(5):1087–1112, 2016.

[135] Steven J Pollock and Noah D Finkelstein. Sustaining educational reforms in introduc-
tory physics. Physical Review Special Topics-Physics Education Research, 4(1):010110,
2008.

199

[136] P¨aivi Kinnunen and Beth Simon. Phenomenography and grounded theory as re-
search methods in computing education research field. Computer Science Education,
22(2):199–218, 2012.

[137] Malcolm Tight. Phenomenography: The development and application of an innovative
research design in higher education research. International Journal of Social Research
Methodology, 19(3):319–338, 2016.

[138] David Boud and Grahame Feletti. The challenge of problem-based learning. Psychology

Press, 1997.

[139] Howard S Barrows, Robyn M Tamblyn, et al. Problem-based learning: An approach to

medical education. Springer Publishing Company, 1980.

[140] Monica McParland, Lorraine M Noble, and Gill Livingston. The effectiveness of
problem-based learning compared to traditional teaching in undergraduate psychia-
try. Medical education, 38(8):859–867, 2004.

[141] Paul van Kampen, Caroline Banahan, Michael Kelly, Eilish McLoughlin, and Eoin
O’Leary. Teaching a single physics module through problem based learning in a lecture-
based curriculum. American Journal of Physics, 72(6):829–834, 2004.

[142] Stephanie Ahlfeldt*, Sudhir Mehta, and Timothy Sellnow. Measurement and analysis
of student engagement in university classes where varying levels of pbl methods of
instruction are in use. Higher Education Research & Development, 24(1):5–20, 2005.

[143] Valerie Otero, Steven Pollock, Richard McCray, and Noah Finkelstein. Who is respon-

sible for preparing science teachers? Science, 313(5786):445–446, 2006.

[144] Renee Michelle Goertzen, Eric Brewe, Laird H Kramer, Leanne Wells, and David
Jones. Moving toward change: Institutionalizing reform through implementation of
the learning assistant model and open source tutorials. Physical Review Special Topics-
Physics Education Research, 7(2):020105, 2011.

[145] Eleanor W Close, Jessica Conn, and Hunter G Close. Becoming physics people: De-
velopment of integrated physics identity through the learning assistant experience.
Physical Review Physics Education Research, 12(1):010109, 2016.

[146] Lillian C McDermott and Peter S Shaffer. the physics education group at the university

of washington, tutorials in introductory physics, 2002.

[147] Katherine Perkins, Wendy Adams, Michael Dubson, Noah Finkelstein, Sam Reid, Carl
Wieman, and Ron LeMaster. Phet: Interactive simulations for teaching and learning
physics. The Physics Teacher, 44(1):18–23, 2006.

[148] Andrea A. diSessa and Harold Abelson. Boxer: A reconstructible computational

medium. Communications of the ACM, 29(9):859–868, 1986.

[149] Edward F Redish and Jack M Wilson. Student programming in the introductory

physics course: Muppet. 2000.

200

[150] Renee McCauley, Sue Fitzgerald, Gary Lewandowski, Laurie Murphy, Beth Simon,
Lynda Thomas, and Carol Zander. Debugging: a review of the literature from an
educational perspective. Computer Science Education, 18(2):67–92, 2008.

[151] Niral Shah, Colleen M Lewis, Roxane Caires, Nasar Khan, Amirah Qureshi, Danielle
Ehsanipour, and Noopur Gupta. Building equitable computer science classrooms:
elements of a teaching approach. In Proceeding of the 44th ACM technical symposium
on Computer science education, pages 263–268. ACM, 2013.

[152] Colleen M Lewis and Niral Shah. How equity and inequity can emerge in pair program-
ming. In Proceedings of the eleventh annual International Conference on International
Computing Education Research, pages 41–50. ACM, 2015.

[153] Christine Bruce, Lawrence Buckingham, John Hynd, Camille McMahon, Mike
Roggenkamp, and Ian Stoodley. Ways of experiencing the act of learning to program: A
phenomenographic study of introductory programming students at university. Journal
of Information Technology Education: Research, 3:145–160, 2004.

[154] Ference Marton and Lennart Svensson. Conceptions of research in student learning.

Higher education, 8(4):471–486, 1979.

[155] Roger S¨alj¨o. Learning about learning. Higher education, 8(4):443–451, 1979.

[156] Ference Marton. Phenomenography—describing conceptions of the world around us.

Instructional science, 10(2):177–200, 1981.

[157] Shirley Booth. Learning computer science and engineering in context. Computer

Science Education, 11(3):169–188, 2001.

[158] John A Bowden. The nature of phenomenographic research. Phenomenography, pages

1–18, 2000.

[159] Ference Marton. Phenomenography—a research approach to investigating different

understandings of reality. Journal of thought, pages 28–49, 1986.

[160] F Marton and S Booth. Learning and awareness. 1997. Hillstdale: Lawrence Earlbaum

Google Scholar, 1997.

[161] Ference Marton. The idea of phenomenography.

In Conference Proceedings, Phe-

nomenography: philosophy and practice QUT, Brisbane, pages 7–9, 1994.

[162] Michael Prosser, Keith Trigwell, and Philip Taylor. A phenomenographic study of
academics’ conceptions of science learning and teaching. Learning and instruction,
4(3):217–231, 1994.

[163] Dawne Bell. The reality of stem education, design and technology teachers’ percep-
tions: A phenomenographic study. International Journal of Technology and Design
Education, 26(1):61–79, 2016.

201

[164] Keith Trigwell, Michael Prosser, and Philip Taylor. Qualitative differences in ap-
proaches to teaching first year university science. Higher education, 27(1):75–84, 1994.

[165] Claire Englund, Anders D Olofsson, and Linda Price. Teaching with technology
in higher education: understanding conceptual change and development in practice.
Higher Education Research & Development, 36(1):73–87, 2017.

[166] Rintaro Imafuku, Takuya Saiki, Chihiro Kawakami, and Yasuyuki Suzuki. How do
students’ perceptions of research and approaches to learning change in undergraduate
research? International journal of medical education, 6:47, 2015.

[167] Peter Ashworth and Ursula Lucas. Achieving empathy and engagement: A practical
approach to the design, conduct and reporting of phenomenographic research. Studies
in higher Education, 25(3):295–308, 2000.

[168] Christine S Bruce. Reflections on the experience of the phenomenographic interview.

Phenomenography: Philosophy and practice, pages 47–56, 1994.

[169] Keith Trigwell. A phenomenographic interview on phenomenography. Phenomenogra-

phy, 1:62–82, 2000.

[170] Despina Varnava-Marouchou. Teaching and learning in an undergraduate business con-
text: an inquiry into lecturers’ conceptions of teaching and their students’ conceptions
of learning. PhD thesis, University of Nottingham, 2008.

[171] Jennifer Case* and Delia Marshall. Between deep and surface: procedural approaches
to learning in engineering education contexts. Studies in higher education, 29(5):605–
615, 2004.

[172] Keith Trigwell, Michael Prosser, and Fiona Waterhouse. Relations between teach-
ers’ approaches to teaching and students’ approaches to learning. Higher education,
37(1):57–70, 1999.

[173] Clark A Chinn, Angela M O’donnell, and Theresa S Jinks. The structure of discourse
in collaborative learning. The Journal of Experimental Education, 69(1):77–97, 2000.

[174] Jean Lave, Etienne Wenger, and Etienne Wenger. Situated learning: Legitimate periph-

eral participation, volume 521423740. Cambridge university press Cambridge, 1991.

[175] Stephen E Toulmin. The uses of argument. Cambridge university press, 2003.

202