EXAMINING PHYSICS IDENTITY IN LABORATORY SETTINGS THROUGH SURVEY

DEVELOPMENT

By

Kelsey Marie Funkhouser

A DISSERTATION

Michigan State University

in partial fulﬁllment of the requirements

Submitted to

for the degree of

Astrophysics and Astronomy – Doctor of Philosophy

2019

ABSTRACT

EXAMINING PHYSICS IDENTITY IN LABORATORY SETTINGS THROUGH SURVEY

DEVELOPMENT

By

Kelsey Marie Funkhouser

Labs have been closely tied with physics education for well over a century. In that time practitioners
have struggled to take advantage of the unique opportunities provided by lab learning environments.
Recently, there have been renewed calls to emphasize the practices and skills of ‘doing physics’ in
lab courses. Physics education has struggled to utilize labs to their full potential and to assess the
impacts labs have on the students. This dissertation will describe the design and implementation
of an algebra-based sequence of lab courses at Michigan State University (MSU). In the course
design, lab skills and practices are the central learning goals instead of content speciﬁc knowledge.
We utilize the context of the newly redesigned lab courses to develop an assessment tool to measure
the impact that labs are having on our students, speciﬁcally on their physics identity development.
An essential aspect of developing a physics identity is the opportunity to engage in the practices
that are authentic to the ﬁeld of physics. From a robust understanding of students’ ideas about
these practices, we can get information about how they situate themselves with respect to the
practices as an indicator of their physics identity. We assert that the survey development process we
have undergone to produce a practice-based identity survey, has ensured that the survey accurately
represents how students interpret these practices and how they identify with them. We have
taken a mixed methods approach to reduce the items and overall dimensions of the survey into
a fully closed-responses short form survey of 24 practice and 5 identity focused questions. This
dissertation describes the process of developing the practice-based identity survey from the context
and theoretical model through each iteration of the survey.

Copyright by
KELSEY MARIE FUNKHOUSER
2019

This thesis is dedicated to Scott Funkhouser and The Duchess of Fluﬃngton.

I could not have done this without the both of you.

iv

ACKNOWLEDGEMENTS

I want to start by thanking the Department of Physics and Astronomy for allowing me to be the
ﬁrst Astrophysics graduate student to complete a thesis in physics education research here at MSU.
I also want to thank my committee for supporting me in this process.
I want to thank my advisors, Vashti Sawtelle and Danny Caballero, for ﬁrst accepting an astro-
physicist into their physics education research group and then nurturing me into the researcher I
am today. To Kim Crosslan who has always had my back and is the best physics mom there is. We
are so lucky to have you.
I want to thank the astronomy and PER graduate students who have been beside me through my
time here. I will always remember and value the support, solidarity, distractions, and everything
else your friendships have provided. I want to thank PERL@MSU, the many iterations of the group
in my time here. It has been a place where I could grow as a researcher and a physicist and I will
be forever grateful for that.
I want to thank everyone who has been involved in the DATA Labs over the years, from faculty to
TAs and LAs to students. Working with all of you has been a gift. Although it was not always
smooth sailing, every bit was worth it.
I want to thank a few teachers without whom I would not be here. First, my 9th grade physical
science teacher Mr. Fields, who boosted my love of science at just the right time. To my high
school AP Physics teacher, Dr. Kim Moore, thanks for pushing me to jump into AP. Finally, my
college astronomy teacher, Dr. Chris Laws, who shared with us the wonders of the universe.
I want to thank my family for their love and support. I especially want to thank my parents, who
have no college experience to speak of but have produced three college educated daughters. Thank
you for instilling in me a passion for learning, for trusting me in my pursuits, and for your continued
pride in my accomplishments. I love you both very much. Finally, I want to thank my husband, Dr.
Scott Funkhouser. I am so grateful that you are in my life and that we have grown in directions that
complement and support each other. I am excited for our next adventure. I love you.

v

TABLE OF CONTENTS

.
.

.
.

.
.

.
.

.
.

.
.

ix

LIST OF TABLES .
LIST OF FIGURES .
CHAPTER 1

.
.
.
.
INTRODUCTION .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
CHAPTER 2 LITERATURE REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
2.1 Physics Labs Historically .
7
2.2 Lab Practices and The New Standards of Science Education . . . . . . . . . . . . .
8
2.3 Assessing Labs
.
2.4 Perspectives on Physics Identity . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
2.5 Classes as Communities of Practice . . . . . . . . . . . . . . . . . . . . . . . . . . 11
. 12
2.6 Decisions and Motivations for this Study . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xi

1

.

.

.

.

.

.
.
.

.
.
.

.
.
.

.
.
.

. .
.

Final Projects . .
3.4 Overview of Key Supports

.
3.1
Introduction .
3.2 Learning Goals
.
3.3 Course Structures .

CHAPTER 3 DESIGN ANALYSIS TOOLS AND APPRENTICESHIP (DATA) LAB . . . 14
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3.1 Laboratory Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.2 Communication Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3.3
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4.1 Typical Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4.2 Additional Supports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
In Course Assessments .
. 29
Formative Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.5.1
Summative Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.5.2
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.6 Example Experiment
3.7 Redesign Eﬃcacy .
.
3.8 Conclusion .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.5

. .

.
.
.

.
.
.

.
.
.

.
.

.

CHAPTER 4

.
.

.
.

Introduction .

IDENTITY .
. .
.

.
4.1
.
4.2 Theoretical Background .
.
Physics Identity .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2.1 Lab Practices .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2.2
4.2.3 Labs as Communities of Practice . . . . . . . . . . . . . . . . . . . . . . . 47
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3.1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3.2
. . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3.3 Open-Response Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3.4
Survey Analysis Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

.
Interviews .
Interview Analysis Methods

4.3 Methods .

.
.

.
.

.
.

.

.

.

.

vi

.

.

.

4.5 Discussion . .

4.4 Results and Analysis

.

.

.

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4.1 What Makes an Identity Statement . . . . . . . . . . . . . . . . . . . . . . 51
In the Data
4.4.2
. 62
4.4.2.1
Learning Trajectory (LT) . . . . . . . . . . . . . . . . . . . . . . 62
4.4.2.2
Interest
. 62
4.4.2.3 Negotiated Experience (NE) . . . . . . . . . . . . . . . . . . . . 63
. . . . . . . . . . . . . . . . . . 64
4.4.2.4 Community Membership (CM)
4.4.2.5 Unclassiﬁed . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 64
. 65
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 65
4.4.3.1 Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4.3.2 Competence . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 66
4.4.3.3 Not Combined . . . . . . . . . . . . . . . . . . . . . . . . . .
. 66
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Summary .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
.
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 68
Practices and Identity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

.
.
4.5.1 Labs and Aﬀect .
4.5.2

4.4.3 Not in the Data .

4.4.4

. .

.
.

.
.

CHAPTER 5 WHAT COUNTS IN LABORATORIES: TOWARD A PRACTICE-BASED

.

.

.

. .

Introduction .

5.1
.
5.2 Theoretical Background
5.3 Survey Development
.
.
Interviews .
.
.
Pilot Survey .
.
.

IDENTITY SURVEY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

5.4 Discussion .

5.3.1
5.3.2

. .

. .

.
.
.
.

.
.

.
.
.
.

. .

6.1
6.2

6.3.1 Version 1 Analysis

Introduction .
.
Interview Analysis .
6.2.1
6.2.2
6.2.3
6.2.4

CHAPTER 6 SURVEY DEVELOPMENT . . . . . . . . . . . . . . . . . . . . . . . . . . 82
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Practice Space .
Identity . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
. .
Survey Development Overview . . . . . . . . . . . . . . . . . . . . . . . . 85
Similar Distributions of Responses . . . . . . . . . . . . . . . . . . . . . . 86
6.3 Version 1 - Open-Response Survey . . . . . . . . . . . . . . . . . . . . . . . . . . 86
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3.1.1 Reduction of Practices . . . . . . . . . . . . . . . . . . . . . . . 88
6.3.1.2 Developing Identity Items . . . . . . . . . . . . . . . . . . . . . 88
. . . . . . . . . . . . . . . . . . . . . . . . 92
6.3.1.3 A Note on Practices
. 92
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.4.1.1 Reduction of Practices . . . . . . . . . . . . . . . . . . . . . . . 94
Identity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.4.1.2
. 102
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Practices - Final Reduction . . . . . . . . . . . . . . . . . . . . . 102

6.4 Version 2 - Closed-Response Survey . . . . . . . . . . . . . . . . . . . . . . . .

6.5 Version 3 - Closed-Response Validation . . . . . . . . . . . . . . . . . . . . . .

6.4.1 Version 2 Analysis

6.5.1 Version 3 Analysis

6.5.1.1

vii

6.6 Discussion .

6.5.1.2
.

.

Identity .
.
.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. 103
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

. .

CHAPTER 7 CONCLUSION .
.
.
.
.

7.1 Future Directions .
. .
7.2 Big Picture .

. .

.
.
.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 107
. 108
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

.

.

APPENDICES .
APPENDIX A
APPENDIX B
APPENDIX C
APPENDIX D
APPENDIX E
APPENDIX F
APPENDIX G
APPENDIX H
APPENDIX I
APPENDIX J
APPENDIX K
APPENDIX L
APPENDIX M

BIBLIOGRAPHY .

.

.

.

.

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
.
DATA LAB EXPERIMENT DESCRIPTIONS . . . . . . . . . . . . 111
DATA LAB RUBRICS . . . . . . . . . . . . . . . . . . . . . . . . 113
INTERVIEW PROTOCOL . . . . . . . . . . . . . . . . . . . . . . 137
CODEBOOK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
DEVELOPMENT OF PRACTICE QUESTIONS . . . . . . . . . . 154
STATISTICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
SURVEY QUESTIONS REFERENCE TABLE . . . . . . . . . . . 162
SURVEY VERSION 1 . . . . . . . . . . . . . . . . . . . . . . . . 170
SURVEY VERSION 2 . . . . . . . . . . . . . . . . . . . . . . . . 267
SURVEY VERSION 3 . . . . . . . . . . . . . . . . . . . . . . . . 287
DISTRIBUTION OF VERSION 2 RESPONSES . . . . . . . . . . . 306
DISTRIBUTION OF VERSION 3 RESPONSES . . . . . . . . . . . 313
FINAL SURVEY VERSION . . . . . . . . . . . . . . . . . . . . . 320
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
.

.

.

.

.

.

viii

LIST OF TABLES

Table 3.1: Finalized learning goals for DATA Lab . . . . . . . . . . . . . . . . . . . . . . 16

Table 4.1: What makes an identity statement? These examples are used to aid in answer-
ing that questions. Ex 1-3 come from the interviews and Ex 4-7 come from
the open-response survey. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Table 4.2: Examples of LT responses from the data. LT1 is from the interviews and LT2-4

are from the open-response survey. . . . . . . . . . . . . . . . . . . . . . . . . . 57

Table 4.3: Examples of Interest statements from the data. Int 1 is from the interviews and

Int 2-3 are from the open-response survey . . . . . . . . . . . . . . . . . . . . . 58

Table 4.4: Examples of NE from the data. NE 1 is from the interviews and NE 2-5 are

from the open-response survey. . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Table 4.5: Example of a CM statement from the data. This example is from the open-

response survey. .

. .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Table 4.6: Unclassiﬁed examples. All six comes from the open-response survey.

. . . . . . 61

Table 4.7: The number of statements from students coded into each of the identity components. 67

Table 5.1: Overview of Practice Space.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Table 6.1: Summary of the outcome of the interview analysis, showing the number of

sub-bins and questions for each practice. . . . . . . . . . . . . . . . . . . . . . . 84

Table 6.2: Group Work questions that were removed at this stage and the reason for removal. 88

Table 6.3: Summary of the outcome of the interview analysis, showing the number of

sub-bins and questions for each practice. . . . . . . . . . . . . . . . . . . . . . . 90

Table 6.4: The following are the four LT statements used in the closed-response pilot

survey (version 2).

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Table 6.5: Summary of the outcome of the interview analysis, showing the number of

sub-bins and questions for each practice. . . . . . . . . . . . . . . . . . . . . . . 91

Table 6.6: The ﬁnal NE statements from analysis of Version 1 . . . . . . . . . . . . . . . . 92

ix

Table 6.7: Summary of the outcome of the interview analysis, showing the number of

sub-bins and questions for each practice. . . . . . . . . . . . . . . . . . . . . . . 93

Table 6.8: Question statement additionally evaluated. . . . . . . . . . . . . . . . . . . . .

. 95

Table 6.9: Reasoning statements used in Version 2 of the survey. . . . . . . . . . . . . . .

. 101

Table 6.10: The updated reasoning questions with their origins as described in this section.

. 101

Table 6.11: The ﬁnal ﬁve identity reasoning questions. . . . . . . . . . . . . . . . . . . . .

. 104

Table A.1: DL1 Experiments .

Table A.2: DL2 Experiments .

.

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Table E.1: Review interviews for where students talk about speciﬁc practice, 1-3 word
summary of their interpretation of the practice. The practice here is under-
standing equipment.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Table E.2: Summaries turned into common sub-bins . . . . . . . . . . . . . . . . . . . .

. 155

Table E.3: Student responses sorted into bins and summarized . . . . . . . . . . . . . . . . 155

Table E.4: Statements that reﬂect all of the ‘How it works’ examples . . . . . . . . . . . . . 155

Table F.1: Observed distributions of responses to two practice questions.

. . . . . . . . . . 157

Table F.2: Expected distributions of responses to two practice questions . . . . . . . . . .

. 157

Table F.3: χ

2 Table

.

.

.

.

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Table F.4: How each researcher coded the set of responses into the ﬁve diﬀerent categories. 160

Table F.5: nk1nk2 for each category.
Table G.1: Survey questions from every version and the reason for removal.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

. . . . . . . . . 162

Table M.1: Questions for the ﬁnal version of the survey . . . . . . . . . . . . . . . . . . . . 321

x

LIST OF FIGURES

Figure 3.1: Week-by-week schedule of DATA Lab I & II. . . . . . . . . . . . . . . . . . .

. 18

Figure 3.2: A snapshot of an experiment from pre-class homework through the commu-

nication project.

.

.

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Figure 3.3: Pre-class research prompts for the Snell’s Law lab. . . . . . . . . . . . . . . . . 33

Figure 3.4: Snell’s Law: Rainbows Lab Handout. Top: Exploring refraction, ﬁrst day of
Snell’s Law. Bottom: Beginning model evaluation, main Snell’s Law activity.

. 34

Figure 3.5: Sample of a student’s Communication Project for DL2. Blue: graph with
sine of the angle of incidence plotted against sine of the angle of refraction
for each wavelength of light. Green: The slope for each wavelength, which
is the index of refraction of the block. Red: Results and conclusions where
they discuss the diﬀerences in the indices of refraction and how that is related
to rainbows.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

.

.

.

.

.

.

Figure 3.6: Comparing student E-CLASS outcomes between the traditional course and
the transformed course: (a) Fraction of statements with expert-like responses
as presented in Zwickl et al. [1] and (b) Shifts in student expert-like thinking
from pretest to post-test as presented in Wilcox and Lewandowski [2].

. . . . . 39

Figure 4.1: Example question from the open-response survey . . . . . . . . . . . . . . . . 50

Figure 4.2: Figure showing process of determining what makes an identity statement . . . . 52

Figure 4.3: A visualization of the initial model of identity from Close et al. 2016 and the
revised model based on the data and analysis presented here. The overlap in
the intial model is meant to illustrate the combination of components.

. . . . . 68

Figure 5.1: The components of our theoretical framework, with the interaction between
the physics lab practices and the components of the identity framework form-
ing the physics identity [3, 4, 5].

. . . . . . . . . . . . . . . . . . . . . . . .

. 74

Figure 5.2: The survey development process. The theoretical framework as described in

Sec. II. .

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

.

.

.

.

.

.

.

.

.

.

.

.

Figure 5.3: This is an example of the interview analysis process for the practice of ana-

lyzing data. .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Figure 5.4: An example question from the pilot survey. . . . . . . . . . . . . . . . . . . . . 80

xi

Figure 6.1: The survey development process from establishing the theoretical framework

to describe identity development to the newest version of the survey.

. . . . . . 82

Figure 6.2: Visual representation of the ‘practice space’

. . . . . . . . . . . . . . . . . .

. 84

Figure 6.3: A subset of the practice space for analyzing data shown with speciﬁc examples.

85

Figure 6.4: Overview of the diﬀerent versions of the survey, described in detail in the

following sections. .

.

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Figure 6.5: Example item from version 1 of the survey (Ref. DA5).

. . . . . . . . . . . . . 87

Figure 6.6: Example of an item from version 2 of the survey (Ref C4). The ﬁrst part is the
practice question, each student saw 10 of the 79. Then there are the identity
reasoning questions, this part remained the same for all of the practice questions. 94

Figure 6.7: Six diﬀerent GW questions to look for similar distributions of responses.
Notice the similarities between GW5, 6, and 17. The responses are normalized
to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat agree, 2 is Niether
agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. (See
Appendix G for the questions, and Appendix K for additional plots)

. . . . . . 96

Figure 6.8:

Illustration of the removal of questions based on similar distributions of
responses. Step 1 shows the ﬂagging of questions with similar distributions of
answers. Step 2 shows the question statements being examined for similarities,
the three blue questions were determined to be similar enough to be considered
for collapse. Step 3 provides a closer look at the three related questions, the
light blue (Q3) is the questions chosen to remain. (Reference numbers: Q1 is
GW6, Q2 is GW7, Q3 is GW5, Q4 is GW19, and Q5 is GW17). . . . . . . . . . 97

Figure 6.9: Similar questions being considered for removal. Here Q1 and 2 talk about
groups doing things together, which is why they were considered for collapse.
Their response distributions were similar enough to combine the questions.
(Reference numbers Q1 is GW19 and Q2 is GW20).

. . . . . . . . . . . . . . . 98

Figure 6.10: The initial step of analysis of reasoning questions. We had to conﬁrm that it
was reasonable to collapse the responses to each practice question by doing
pairwise tests to make sure responses to R1 (This will be important when
working with patients) were the same for each practice question.

. . . . . . . . 99

Figure 6.11: In the process of analyzing responses to the identity-speciﬁc reasoning ques-
tions, we did pairwise tests on the questions grouped into NE or LT. The
analysis showed that R5, 6, and 8 and R7, 9, and 10 had statistically similar
distributions, i.e., each group should be considered for collapse. . . . . . . . . . 100

xii

Figure 6.12: An example item from version 3 of the survey (Ref. C12). . . . . . . . . . . . . 102

Figure 6.13: Showing the process for conﬁrming that the practice questions that had unique

reasoning responses were kept (Ref. GW9).

. . . . . . . . . . . . . . . . . . . 104

Figure K.1: Distributions of responses to Communicating questions in Version 2 of the
survey. The distributions are all normalized to one. On the x-axis: 0 is
Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 307

Figure K.2: Distributions of responses to Data Analysis questions in Version 2 of the
survey. The distributions are all normalized to one. On the x-axis: 0 is
Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 308

Figure K.3: Distributions of responses to Executing Experiments questions in Version 2
of the survey. The distributions are all normalized to one. On the x-axis: 0
is Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 309

Figure K.4: Distributions of responses to Group Work questions in Version 2 of the survey.
The distributions are all normalized to one. On the x-axis: 0 is Strongly agree,
1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree,
and 4 is Strongly disagree. The reference number of the question is in the
title (see Appendix G for questions).

. . . . . . . . . . . . . . . . . . . . . . . 310

Figure K.5: Distributions of responses to Planning Experiments questions in Version 2
of the survey. The distributions are all normalized to one. On the x-axis: 0
is Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 311

Figure K.6: Distributions of responses to Understanding Equipment questions in Version
2 of the survey. The distributions are all normalized to one. On the x-axis: 0
is Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 312

xiii

Figure L.1: Distributions of responses to Communicating questions in Version 3 of the
survey. The distributions are all normalized to one. On the x-axis: 0 is
Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 314

Figure L.2: Distributions of responses to Data Analysis questions in Version 3 of the
survey. The distributions are all normalized to one. On the x-axis: 0 is
Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 315

Figure L.3: Distributions of responses to Executing Experiments questions in Version 3
of the survey. The distributions are all normalized to one. On the x-axis: 0
is Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 316

Figure L.4: Distributions of responses to Group Work questions in Version 3 of the survey.
The distributions are all normalized to one. On the x-axis: 0 is Strongly agree,
1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree,
and 4 is Strongly disagree. The reference number of the question is in the
title (see Appendix G for questions).

. . . . . . . . . . . . . . . . . . . . . . . 317

Figure L.5: Distributions of responses to Planning Experiments questions in Version 3
of the survey. The distributions are all normalized to one. On the x-axis: 0
is Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree. The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 318

Figure L.6: Distributions of responses to Understanding Equipment questions in Version
3 of the survey. The distributions are all normalized to one. On the x-axis: 0
is Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is
Somewhat disagree, and 4 is Strongly disagree.The reference number of the
question is in the title (see Appendix G for questions).

. . . . . . . . . . . . . . 319

xiv

CHAPTER 1

INTRODUCTION

Lab experiences have been a central aspect of physics and science education for well over a century
as evidenced by a quote from Griﬃn in 1892, “the laboratory has won its place in school; its
introduction has proved successful. [...] Pupils will go out from our laboratories able to see and
do” (Griﬃn 1892, cited by [6, 7]). Lab activities represent the hands-on and practical process
of ‘doing science.’ Since their establishment as essential to science education there has been
a struggle to appropriately and eﬀectively assess the impacts they are having on our students’
learning and development [8, 9, 7, 10, 11]. Most recently there have been national calls to
focus on practices, from the National Research Council in the Next Generation Science Standards
and in the American Association of Physics Teachers (AAPT) Committee on Laboratories and
their recommendations for undergraduate physics labs [12, 5]. Both push for reforms in science
education that emphasis the practices and skills fundamental to the process of doing science. As it
is understood, practices are the ways of knowing, doing, and participating in a speciﬁc community.
Today, as historically true, the goals of labs in education are to provide students with opportunities
to engage in the authentic practices necessary in doing physics (or science more generally). Lab
courses are uniquely structured to produce these opportunities and as such it is important that the
design of these courses reﬂect that structure by emphasizing practices in design and assessments1
in the course. As we are meant to be preparing the next generation of physicists we need to use
the opportunities we have to train them in what it means to be a part of the physics community.
This connection makes the motivation for instructing physics majors this way but they are not the
only populations that we teach and not the only populations who will beneﬁt from experience with
physics practices. The skills and practices valued in physics are relevant for all STEM ﬁelds, just
as the subject of physics itself is. Labs are spaces for this authentic engagement in practices but,
1Measurement tools developed to assess speciﬁc areas of interest (content knowledge, beliefs,

etc.). Often in the form of multiple choice tests or surveys.

1

these authentic experiences also positively impact students’ attitudes and beliefs about physics,
as well as other aﬀective measures [13, 14, 15, 16, 17]. Engagement with the practices of a
particular community are fundamental to forming a community-speciﬁc identity [4]. Identity is a
multidimensional construct that describes how someone perceives and positions themselves with
respect to a speciﬁc community. The strength of one’s identity has been found to correlate with
persistence and belonging [18, 19, 20].
It has also been posited that identity development (or
struggles with it) can be linked to underrepresentation of white women and people of color in
some STEM programs [20, 19]. Seeing as physics labs are in need of large scale assessments,
engagement with practices inﬂuences identity, and identity can be linked to persistence, belonging,
and underrepresentation in physics, it is not such a stretch to focus work on this. Motivation also
comes from the experience of working in the transformed course described in Chapter 3. The
emphasis on practices and other adjustments to the course resulted in completely diﬀerent attitudes
and levels of engagement from the students (see Sec. 3.6 for some evidence of this).

With the motivation for this speciﬁc focus laid out it is also important to include justiﬁcation
for the development of a survey as opposed to a fully qualitative investigation. The ﬁrst reason
has already been discussed: the community of physics lab education is in need of large scale and
broadly applicable assessments of the impacts of these courses. There is not quantitative work
being done on identity in physics labs, and the identity speciﬁc qualitative work has been focused
on upper division students and physics majors. The lab transformation described in Ch 3, which
has been the main data source for the survey development, is an introductory algebra-based physics
lab for non-physics majors. The mode, context, and focus of this work are unique and therefore
relevant to both the study of physics identity and physics labs.

This dissertation presents the development of a survey to measure the connection between
physics labs experiences and the physics identity of students. In Chapter 2, we provide an overview
of the history of labs in science and physics education in order to illustrate the goals related to
practices. We also discuss the lack of large scale assessments for evaluating the impacts of these
courses. We propose that a survey to measure physics identity in lab classes is a valuable addition

2

to the ﬁeld. The diﬀerent perspectives on identity in the literature are summarized, and our focus
on communities of practice is justiﬁed.

Chapter 3 describes a large-scale course transformation we conducted here at Michigan State
University in the introductory lab courses for non-physics majors. This chapter establishes the
context where the majority of the data discussed in the dissertation were obtained. The new course
was designed to center lab practices as called for in the AAPT Lab guidelines [5]. This chapter
outlines which practices the transformation centers on and how that translates to the classroom. It
also includes the only large-N assessment available to evaluate the lab transformation, the E-CLASS
[1].

In Chapter 4, we focus on the identity-speciﬁc work done in the process of developing the
survey. The main focus is on working to operationalize2 a model3 of identity used in Close et
al. (2016) [21] into laboratory contexts, including the context described in Chapter 3. This involved
the development of a codebook4 from interviews with students in lab classes and an open-response
survey. After successfully operationalizing identity in this context, we evaluated the data’s ﬁt to
the Close et al. model of identity from the literature. The model in question was a combination of
two diﬀerent frameworks on identity [4, 19]. Through the analysis of both data sources we found
that the individual components of the identity model could be applied to our context, but we saw
no evidence of the components being combined.

Chapter 5 is a bridge from the end of the identity analysis of Chapter 4 to the survey speciﬁc
work in Chapter 6. This chapter provides additional insight into the development of Version 1 of
the survey and the open-response questions. It is also a snapshot of our perspective on identity at
the beginning of piloting diﬀerent versions of the survey.

Chapter 6 provides an overview of the entire survey development process from theoretical
framework to the current version of the survey. There is a special focus on the development

2To turn a theoretical construct into concrete parts that can be observed and/or assessed.
3A detailed description (or theoretical framework) developed to represent an abstract behavior

or concept.

deﬁnitions and explanations.

4A document listing the catagories generated in close analysis of student data along with detailed

3

and then reduction of survey questions over three diﬀerent distributions (from 108 to 24 items).
The chapter describes the qualitative and quantitative analysis performed at the diﬀerent stages of
development in order to produce the current, fully closed-response 24 item version of the survey.
Finally in Chapter 7, we discuss the dissertation as a whole, any implications of the work, and
proposed next steps for the survey. We also include some of our own perspectives on identity, lab
classes, and practices.

This survey is limited in scope: it is designed in the speciﬁc context of undergraduate physics
labs and it should not be assumed to apply elsewhere, e.g., other science labs or combined lecture
and labs, without additional validation. Along the same lines, the perspectives on identity presented
here can not be simply moved into a physics lecture course or an informal learning space and be
expected to represent identity development in those environments. Even in the speciﬁc learning
environment of physics labs, the survey has not been validated with non-STEM majors, although
there is no reason it could not be. We have been able to reduce the survey to a fully closed-response,
relatively short version but complex analytics have not been completed and must be before any
broad claims can be made from responses. This version of the survey needs to be distributed to a
large number of students and analytics need to be done in order to determine what claims can be
made about the quantitative results. This additional work was not possible within the constraints
of the dissertation.

4

CHAPTER 2

LITERATURE REVIEW

As motivation for this work, I present a look at the establishment and study of physics labs in the
literature, both historic and modern. The review highlights that the goals of lab instruction have
always reﬂected the skills and practices necessary in ‘doing science.’
It describes the struggle
to eﬀectively evaluate the impacts of labs from their conception in physics education to today.
As described below, lab experiences and engagement with authentic practices have been shown
to impact students on attitudinal and aﬀective measures. These impacts were observed in small
scale studies, while large scale, broadly applicable assessments have been lacking. This struggle
of assessment motivated my work to study how the focus on practices in lab courses impact the
development of physics identity. The perspectives of identity development are discussed below
along with the model for identity that I have selected and as further justiﬁcation for this work. The
conception of studying lab classes as communities of practice is introduced as justiﬁcation for our
model choice and this work on the impact lab experiences have on physics identity.

2.1 Physics Labs Historically

Prior to the Civil War, hands-on laboratories were not a part of the high school physics
curriculum. The introduction of land grant colleges inﬂuenced the curriculum of high schools
and in the 1870s, the lab method of teaching physical science, with a focus on the process of
‘doing science’, began popping up in high schools but was still rare until the 1890s [6]. The slow
introduction was attributed to the cost of equipment and a lack of teachers with the necessary
expertise. The Harvard List, a set of forty experiments of required experiments for admission in
1887, likely inﬂuenced the large jump in schools with laboratory physics from the 1880s to the early
1900s. In 1915 Edwin H. Hall described the inﬂuence of the Harvard List on high school physics,
“it established the use of the laboratory as a most important element in the teaching of physics in
secondary schools” (Edwin H. Hall, France in 1915 as cited by Rosen, 1953 [6]). The experiments

5

indicated on the List were prescriptive and lead to a focus on measurement for measurement’s sake,
at least partially in opposition to the author Hall’s own views [22]. In 1897 the List additionally
reﬂected the suggestions of the Committee of Ten, a group of educators working to standardize
the curriculum of American high schools. It was updated to emphasize a mixed course containing
laboratories, lectures, and textbooks. Even with the updates and suggestions, labs took on the
behavior of a fad becoming the dominant means of physics education, in some cases usurping
lecture and textbooks entirely. By 1910, labs were a prominent part of physics education in the high
schools even as Mann, whose perspective focused on the necessity of developing habits of mind,
warned “we don’t know how to use labs most eﬀectively” [8]. While some questioned the beneﬁt
of labs in science education others, such as the founder of the progressive education movement,
John Dewey, provided a perspective on what the goals of labs should be: “Science is more than a
body of knowledge to be learned, there is a process or method to learn as well” [23].

After WWI labs were largely conﬁrmatory and that was how things remained until the new
science curricula of the 1960s, “which stress the process of science” [24].
In and around that
time, educators were leaning on the unique opportunities provided by the labs for investigation and
inquiry [25, 26, 27]. But even with the new push in science education in the 70s, the value of labs
in science education were in question, in part because we lacked the tools to assess the speciﬁc
beneﬁts of labs. “Teachers who believe that the laboratory accomplishes something special for
their students would do well to consider carefully what those outcomes might be, and then to ﬁnd
ways to measure them” [9]. Although the review by Bates highlighted evidence for labs improving
skills in working with equipment, and positive attitudes. Bates also suggested that new assessments
could and should be developed to measure the outcomes of labs because they “appear to represent
a signiﬁcantly diﬀerent area of science learning than content acquisition.”

This point was restated by Hofstein and Lunetta in their 1982 review of labs in science teaching
[7]. They asserted that there had not been a comprehensive examination of the eﬀects of lab
instruction. They highlighted works listing the goals of laboratory work in science education. The
goals of both inquiry and laboratory education at this time are reﬂective of what we now refer to

6

as science practices. The goals include communicating [28], analyzing data [29], planning, and
executing experiments [28, 24, 29], and understanding equipment [28, 30]. In addition to inquiry
skills [31] and the scientiﬁc method [24].

2.2 Lab Practices and The New Standards of Science Education

Inquiry in science education came into focus again in the late 1990s with the National Science
Education Standards (NSES) from the National Research Council and the goals for all students to
achieve scientiﬁc literacy through inquiry [32, 33]. In the early 2000s came the development of
the Investigative Science Learning Environment (ISLE), an introductory physics curricula with the
goal of better preparing students for the modern workforce by helping them to develop abilities
used in the doing of science [34, 35]. They deﬁne scientiﬁc abilities as “the most important
procedures, processes, and methods that scientists use when constructing knowledge and when
solving experimental problems” [35].

Recently there have been renewed pushes for an emphasis on practices in science and physics
education. The Next Generation Science Standards (NGSS) name scientiﬁc and engineering prac-
tices as one of their central goals of science education, along with cross-cutting concepts, and core
ideas [12]. In 2014 the American Association of Physics Teachers Committee on Laboratories came
out with their Recommendations for Undergraduate Physics Laboratory Curriculum that centers
the practices of science in six focus areas, modeling, designing experiments, developing techni-
cal and practical skills, analyzing and visualizing data, communicating physics, and constructing
knowledge [5]. These national calls to better integrate practices into science and physics education
have been especially inﬂuential in the laboratory physics community, where laboratory physics
courses are being redesigned from the introductory [36, 37] to the advanced level [38, 39]. As an
example, our large scale transformation of the introductory algebra-based physics lab sequence to
center practices is described in Funkhouser et al. (2019) [40], which is reproduced in Ch 3.

7

2.3 Assessing Labs

The timeline here is relevant because it shows several themes of laboratory education that have
been repeated for over a century. First, the stated goals of laboratory education in the sciences have
always included practical skills related to the methods of doing science, even if at times they went
under diﬀerent names, for example inquiry/scientiﬁc skills and abilities. Though we now refer to
them as scientiﬁc or physics practices, they have always been at least part of the stated goals of
laboratory instruction. Second, although these have been the goals, we have been struggling to
properly integrate and assess them for equally as long. Mann, in 1910, worried about our inability
to use labs eﬀectively [8]. In 1978, Bates pleaded with instructors to ﬁnd ways to measure the
eﬀects they saw labs having on their students [9]. Most recently, in 2017, Wieman et al., in their
review of the literature, determined that labs had no measurable impact on conceptual knowledge as
measured by standard content assessments [41]. If the assessments are not measuring the impacts
labs have on our students, either in their skills and practices, or in other not content related ways,
then the clear path forward is to develop ones that do.

Some of these assessments do exist. There is the Colorado Learning About Science Survey in
Experimental Physics (the E-CLASS), where attitudes and expectations about experimental physics
are measured [1]. There is also the Physical Measurement Questionnaire (PMQ), which is used to
assess students’ procedural understanding in experimental physics [42, 43]. Walsh et al. recently
produced the Physics Lab Inventory of Critical thinking (PLIC) [44]. However, examples of unique,
large scale assessments of the impact of physics labs are scarce and more work is needed on how
lab practices can inﬂuence the aﬀective1 dimension of learning.

There are many more small-scale studies providing evidence of the impacts of labs in science
and physics education. It has been shown many times that laboratory and authentic experimental
experiences have a positive impact on students’ attitudes about science [13, 14, 15, 16, 17]. Several
studies have shown the impact of ISLE on diﬀerent scientiﬁc abilities, such as critical thinking,
experimental design, scientiﬁc habits of mind, and formative assessment tasks to acquire and assess
1Related to emotions, beliefs, and attitudes of the learners, as opposed to conceptual learning.

8

scientiﬁc process abilities [34, 34, 45, 46, 47]. O’Neill and Polman (2004) found that authentic
experimental experiences improved scientiﬁc literacy [48]. Holmes et al. (2015) saw sustained
improvements of critical thinking skills in students who were initially instructed on speciﬁc aspects
of said skills [49]. Even with these studies, the impacts of laboratories are a clearly understudied
in the ﬁeld of physics education. This is in spite of the fact that lab courses have been a hallmark
of both science and physics education for over a century.

2.4 Perspectives on Physics Identity

There is no one true model of identity in the literature; instead the chosen model must be tested
and justiﬁed within the goals and context of interest. Narrative identities as introduced by Sfard
& Prusak (2005) deﬁne identity as the story someone tells about themselves, which changes based
on context and who they are talking to [50]. Gee (2000) describes identity as being recognized as
a certain ‘kind of person.’ This discourse identity is deﬁned such that discourse is a speciﬁc way
of acting, speaking, doing, etc. that would get you recognized as a certain ‘kind of person’ [51].
In this work we have chosen to utilize models of identity that center practices, models that will be
further detailed below.

The models from Hazari (2010) [19] and Carlone & Johnson (2007) [20] have similar perspec-
tives with a focus on competence, performance, recognition, and interest (as explicitly added to
the model in Hazari (2010) [19]). Competence: How good a students is at skills that are valued in
science, as deﬁned by the broader culture or by the individual. Performance: the process of acting
out the role of a scientist in the form of larger scientiﬁc presentations or day-to-day routine and
how the student presents themselves as aligned with professional scientists. Recognition: both self
recognition as a science person, and recognition from ‘meaningful others’. Meaningful others are
deﬁned by community norms but the individual can also redeﬁne whose recognition is meaningful
to them. One cannot pull oﬀ being a particular kind of person (enacting a particular identity) unless
one makes visible to (performs for) others one’s competence in relevant practices, and, in response,
others recognize one’s performance as credible. Interest is self-evident, but is included because of

9

its impact on persistence [18].

Another model of identity comes from the social theory of learning and communities of practice
(COP) [52, 4]. Wenger (1999) states that identity is produced by experiences and participation
in speciﬁc communities and describes identity in practice as engaging in practices that deﬁne a
COP. Identity is lived, negotiated, social, a learning process, a nexus, and a local-global interplay.
Lived because it involves participation and reiﬁcation. Negotiated because it is always a work
in progress and never permanent. Social because it is formed through participation in a COP. A
learning process is a trajectory that incorporates the past and future to establish meaning in the
present and creates coherence through time. A nexus in that people are members of multiple COPs
and the diﬀerences must be continually reconciled. A local-global interplay due to the fact that one
participates in a local COP but must also negotiate their connection to the broader world.

Irving and Sayre use the COP model of identity in their small-scale qualitative study of the
process that students go through from identifying as a physics student to a physicist [53, 54, 55,
56]. They studied physics majors in upper division physics labs and how the students positioned
themselves within the physics community in connection with the COP model of identity. They
motivate their work on identity with a focus on the retention of people within the physics major in
order to quell the dwindling of the physics community. They reference the inﬂuence that a subject-
speciﬁc identity has on retention in that discipline [18] and the connection between identiﬁcation
with being a physicist and physics-speciﬁc career choices.

In their qualitative study of the process of identity transformation in undergraduate learning
assistants in their undergraduate Learning Assistant (ULA) program, Close et al.(2016) [21] in-
troduce an adaptation of the two previously presented models of identity, COP and the one used
by Carlone & Johnson [20] and Hazari et al. [19]. Their goals in that small-scale study are to
understand the experience of the ULAs, using identity as a lens. Close et al. deﬁne identity as
making sense of how others see and react to us, including recognition as competent and valued.
In their data this manifests for the ULAs in the ways in which they can impact and inﬂuence the
students they are working with. The next part of the model is community membership combined

10

with competence. Close describes this as competence developed and valued by members of the
community. Community membership and competence showed up in the form of competence in
teaching and understanding physics as well as the development of a community of physics ULAs
and a connection with the broader physics community. Finally there is learning trajectory combined
with interest. According to Close this incorporates past and potential future identities to make sense
of the present. For the ULAs, participating in the program increased their interest in teaching and
they viewed the experience as valuable for their future.

2.5 Classes as Communities of Practice

Irving and Sayre (2016) [56] argue that positive experiences with practices that are seen to be
more legitimate and authentic can produce a change in identity status. This supports our goals to
examine experiences with practices as a means to understand identity. In order to utilize a COP
related model of identity in physics labs, it needs to be established that lab classes are COPs.
In Irving (2014) [57] they describe the ways in which a lab course can become a community of
practice. The concept of a classroom community as a community of practice is not a new one
[58, 59]. But it is an eﬀective way to examine the impacts an individual course can have. In Irving
(2014) [57] they motivate the relevance of a lab class as a COP with the fact that “A community
that has many similarities to that of a professional research environment provides the opportunity
for students to recognize each other as more authentic physicists than before.” This work further
motivates the value of participating in authentic practices for students in developing a physics
identity.

Although not all (maybe not even most) lab classes form communities of practices that eﬀectively
progress students toward central membership in the community of practicing physicists, they all
have the potential to impact a student’s relationship with physics [57]. Common practices are not
the only things that make a community of practice, but they are a distinct feature and one that is
often made more apparent in lab classes. The experiences the students have with those physics
practices, in participation or in integrating them with those from other communities, provide us

11

with information about their physics identity.

2.6 Decisions and Motivations for this Study

Grounding the study of identity in the COP model can allow for direct analysis of the impacts
of a course or program has on an individual’s subject-speciﬁc identity. For lab classes in particular
it is relevant to look at the practices and skills valued in the course and how students position
themselves with practices they perceive as important to the physics community, as indicated by
their treatment in the course. For this reason physics lab classes with their emphasis on practices
(both historically and more recently, including in our context) lend nicely to a COP-based model
of physics identity.

In their work on identity, Close et al. [21] produced operationalized deﬁnitions of the diﬀerent
components of identity in the context of the ULA program, including a variety of examples from
their data, which are utilized in the work described in this thesis. We chose this model because it
led to clear and usable deﬁnitions of the components of identity allowing for adaptation into our
context of interest.

As laid out previously, since their inception as tools of science education, labs have held skills
and practices as a central goal. Most recently this goal has been reaﬃrmed (NRC and AAPT),
which makes now the ideal time to both redesign labs with practices as a focus and assess the
impacts of doing so. Additionally, an emphasis on practices has been found to impact students’
attitudes [13, 14, 15, 16, 17] and development of a subject-speciﬁc identity [4, 54, 56].

The education community has always struggled with properly assessing the impacts of labs on
our students, especially with large scale assessment tools [8, 9, 7, 41]. It makes sense to combine
the study of how labs and practices impact identity with the development of a large scale assessment
tool, meeting many of the aforementioned needs of the science education laboratory community in
one assessment.

It is one thing to justify the use of an assessment, but it is another to justify the development of
one. In the work done by Hazari et al. on identity, their analysis relies on the Persistence Research

12

in Science and Engineering (PRiSE) survey [19]. The survey focuses on experiences students have
in their high school physics and science courses, which does not allow for a focus on speciﬁc lab
courses. Additionally, the framework used by Hazari et al. does not have the speciﬁc focus on
practices, which is fundamental in the motivation of this work. There also exists an assessment of
attitudes and expectations about experimental physics, the E-CLASS [1], and although the majority
of the questions center on lab practices, only a handful of them relate in anyway to identity. In
order to reach the goal of understanding the impact practices have on the identity development of
students in physics lab classes inline with assessing the impacts labs themselves are having, a new
assessment tool is required.

13

CHAPTER 3

DESIGN ANALYSIS TOOLS AND APPRENTICESHIP (DATA) LAB

The following chapter has been published in the European Journal of Physics [40] with minor
modiﬁcations.
It is published with co-authors William M. Martinez, Rachel Henderson, and
Marcos D. Caballero.

3.1 Introduction

New knowledge in physics is driven by the observation of phenomena, the design of experiments
to probe these phenomena, and the communication of and debate around the resulting measurements
in public fora. Laboratory courses in physics are thus unique spaces where students can engage in
these central aspects of studying physical systems. Greater emphasis on these aspects in laboratory
spaces is needed to accurately represent the physics discipline and to engage students in the universal
scientiﬁc endeavor that is driven by observation, measurement, and communication.

Recently in the United States, national calls have been made to design laboratory instruction such
that it emphasizes students’ engagement in experimental scientiﬁc practices rather than simply re-
enforcing content learning [60, 11]. Such experiences would be better aligned with discovery-based
learning [61], which is more representative of the enterprise of experimental physics. This focus on
science practices is articulated in the American Association of Physics Teachers’ Recommendations
for the Undergraduate Physics Laboratory Curriculum [60]. These recommendations call for all
laboratories in undergraduate physics to better represent experimental physics by constructing
laboratory curriculum around science practices such as designing experiments, analyzing and
visualizing data, and communicating physics. Arguably, middle-division and advanced laboratory
courses for physics and astronomy majors – with their more complex experiments and equipment
as well as their focus on the professional development of future physicists – tend to engage students
with these practices.

By contrast, introductory physics laboratory courses tend to have more prescriptive and direct

14

approaches to instruction. In these courses, students often follow a well-documented procedure
and do not typically have opportunities to explore the observed phenomenon and the associated
experimental work. At larger universities in the United States, these introductory laboratory courses
are taught to thousands of students per semester, which makes these more direct approaches to in-
struction attractive as they are quite eﬃcient. At many US schools, engineering students, physical
science majors, and biological science students must pass these laboratory courses to complete
their degree program. The scale of these course oﬀerings provides an additional challenge to
incorporating science practices. There are unique examples in the literature where students of
introductory physics are engaged with scientiﬁc practices such as the Investigative Science Learn-
ing Environment (ISLE) [62], Student-Centered Activities for Large Enrollment Undergraduate
Programs (SCALE-UP) [63] and Studio Physics [64]. However, these courses have the advantage
of being taught to smaller population of students than most introductory laboratory courses, in the
case of ISLE, or having an integrated “lecture” and a modiﬁed instructional space, in the case of
Studio Physics and SCALE-UP, and thus can make use of greater instructional resources.

In this paper, we describe a stand-alone, introductory physics laboratory course sequence for
biological science majors at Michigan State University (MSU) that was designed speciﬁcally to
engage students in scientiﬁc practices through the work of experimental physics. Students learn to
design experiments, analyze and visualize their data, and communicate their results to their peers
and instructors. Design, Analysis, Tools, and Apprenticeship (DATA) Lab is unique in that it is was
explicitly designed with the AAPT Lab Recommendations in mind. The sequence is a stand-alone
mechanics laboratory (DL1) and a separate E&M and optics laboratory (DL2), which is taught
to more than 2000 students per year. Furthermore, the process of developing and launching this
pair of courses required that we confront and overcome several well-documented challenges such
as departmental norms for the course, expectations of content coverage, and the lack of instructor
time [65].

We begin this paper by describing how the learning goals for the lab sequence were constructed
through a consensus-driven process (Sec. 3.2). In Sec. 3.3, we provide an overview of the course

15

Table 3.1: Finalized learning goals for DATA Lab

Description

Learning Goal
LG1 - Experimental Process Planning and executing an experiment to eﬀectively explore
how diﬀerent parameters of a physical system interact with
each other. Generally taking the form of model evaluation or
determination.
Knowing how to turn raw data into an interpretable result
(through plots, calculations, error analysis, comparison to an
expectation, etc.) that can be connected to the bigger physics
concepts.
Working eﬀectively as a group. Communicating your ideas
and understanding. Coming to a consensus and making de-
cisions as a group.
Communicating understanding – of the physics, the experi-
mental process, the results – in a variety of authentic ways –
to your peers, in a lab notebook, in a presentation or proposal.

LG4 - Communication

LG2 - Data Analysis

LG3 - Collaboration

structure – diving deeper into the details of the course materials later (Sec. 3.4). We describe the
assessments for this course in Sec. 3.5 as they are somewhat non-traditional for a course of this level
and scale. To make our discussion concrete, we highlight a particular example in Sec. 3.6. Finally,
we oﬀer a measure of eﬃcacy using student responses to the Colorado Learning Attitudes about
Science Survey for Experimental Physics [1] (Sec. 3.6) and some concluding remarks (Sec. 3.8)

3.2 Learning Goals

As this laboratory course serves the largest population of students enrolled in introductory
physics at MSU, it was critical to develop a transformed course that reﬂected faculty voice in the
design. While physics faculty are not often steeped in formal aspects of curriculum development,
sustained eﬀorts to transform physics courses take an approach where faculty are engaged in the
process to develop a consensus design [66, 67, 41]. In this process, interested faculty are invited
to participate in discussions around curriculum design, but experts in curriculum and instruction
synthesize those discussions to develop course structures, materials, and pedagogy. These eﬀorts
are then reﬂected out to faculty to iterate on the process. Our design process followed the approach
developed by the University of Colorado’s Science Education Initiative [66, 67, 41]. In this process,

16

faculty are engaged in broad discussions about learning goals, the necessary evidence to achieve
the expected learning, and the teaching practices and course activities that provide evidence that
students are meeting these goals. Below, we discuss the approach to developing learning goals for
the course as well as present the ﬁnalized set of learning goals from which the course was designed.
We refer readers to [41] for a comprehensive discussion of setting about transforming courses at
this scale.

Prior to engaging in curriculum and pedagogical design, an interview protocol was developed by
WMM to talk with faculty about what they wanted students to get out of this laboratory course once
students had completed the two semester sequence. The interview focused discussion on what made
an introductory laboratory course in physics important for these students and what role it should
play as a distinct course since, at MSU, students do not need to enroll in the laboratory course at the
same time as the associated lecture course. A wide variety of faculty members were interviewed
including those who had previously taught the course, those who had taught other physics laboratory
courses, and those who conduct experimental research. In total, 15 interviews were conducted with
faculty. This number represents more than half of the total number of experimental faculty who
teach at MSU.

The discussion of faculty learning goals was wide-ranging and covered a variety of important
aspects of laboratory work including many of the aspects highlighted in the AAPT Laboratory
Guidelines [60]. Interviews were coded by WMM for general themes of faculty goals and the initial
list included: developing skepticism in their own work, in science, and the media; understanding
that measurements have uncertainty; developing agency over their own learning; communicating
their results to a wider variety of audiences; learning how to use multiple sources of informa-
tion to develop their understanding; demonstrating the ability to use and understand equipment;
documenting their work eﬀectively; and becoming reﬂective of their own experimental work.

With the intent of resolving the faculty’s expressed goals with the AAPT Lab Guidelines,
the goals were synthesized under larger headings, which aimed to combine and/or to connect
seemingly disconnected goals. In addition, through a series of informational meetings that roughly

17

Figure 3.1: Week-by-week schedule of DATA Lab I & II.

10-12 faculty attended regularly, how these goals were being combined and connected to interested
faculty were reﬂected upon. Additional critiques and reﬁnements of these goals were collected
through notes taken during these meetings. Through several revisions, a set of four broad goals that
faculty agreed reﬂected their views on the purpose of this part of laboratory courses was ﬁnalized.
Additionally, these goals were also represented in the AAPT Lab Guidelines. The ﬁnalized goals
are listed in Table 3.1 along with short description of each; they are enumerated (LGX) in order to
refer to them in later sections.

The learning goals formed the basis for the design of course structures including materials and
pedagogy. To construct these course structures, constructive alignment [68] was leveraged, which
helped ensure that the designed materials and enacted pedagogy were aligned with the overall
learning goals for the course. These structures are described in the next section where we have
included a direct reference to each learning goal that a particular course structure is supporting.

3.3 Course Structures

Each laboratory section consists of twenty students and two instructors – one graduate teaching
assistant (GTA) and one undergraduate learning assistant (ULA) [69]. The students are separated
into ﬁve groups of four, which they remain in for 4 to 6 weeks – 4 to 6 class meetings. This time

18

frame works well because it gives the students time to grow and improve as a group as well as
individuals within a consistent group. In addition, when the groups are switched it requires the
students to adapt to a new group of peers. The groups complete 6 (DL1) or 5 (DL2) experiments
during the semester, most of them spanning two weeks – two class meetings. Fig. 3.1 provides
an overview of the two-semester sequence and will be unpacked further below. We indicate the
laboratories that students complete with light green squares (introductory experiments) and dark
green squares (two week labs). The students keep a written lab notebook, which they turn in to be
graded at the end of each experiment.
In this laboratory course, each group conducts a diﬀerent
experiment. This is possible because, in general, students tend to follow a similar path with respect
to the learning goals and there is no set endpoint for any individual experiment. As long as students
continue to work through the experimental process and complete analysis of their data, they are
working towards the learning goals and can be evaluated using the aligned assessments (Sec. 3.5).
This approach also emphasizes that there is not one way to complete an experiment; this has added
beneﬁts for students’ ownership and agency of the work as they must decide how to proceed through
the experiment. In addition, having no set endpoint and two weeks to complete most experiments
takes away the time pressure to reach a speciﬁc point in a given time. All of these aspects allow
students to more fully engage with the work they are doing and, in turn, make progress toward the
learning goals. Having each group conduct a diﬀerent experiment addressed a signiﬁcant point of
discussion among the faculty; speciﬁcally, not covering the same breadth of content was a major
concern. Although, through this design, students do not complete all of the experiments, they are
introduced to all of the concepts through the peer evaluation of the communication projects (red
squares in Fig. 3.1, addressed in detail below).

3.3.1 Laboratory Activities

The laboratory activities were designed around the learning goals. As such, the experiments follow
a similar path from the beginning of the experimental process through analysis, with communica-
tion and collaboration as central components throughout. The course structures in relation to each

19

of the learning goals are highlighted below. The core component (i.e. lab activities) of the course
sequence is outlined in Fig. 3.2.
LG1 - Experimental Process: The students begin each experiment by broadly exploring the rele-
vant parameters and their relationships. Typically, students investigate how changing one parameter
aﬀects another by making predictions and connecting their observations to physics ideas (quali-
tative exploration in Fig. 3.2). From these initial investigations, students work toward designing
an experiment by determining what to measure, change, and keep the same. This often requires
grounding decisions on some known model or an observed relationship (quantitative exploration,
experimental design, and investigation in Fig. 3.2).
LG2 - Data Analysis: After additional formal investigations in which data has been collected,
students summarize the raw data into an interpretable result. This typically includes some form of
data analysis; for example, constructing a plot to evaluate a model or determining a quantitative
relationship between the diﬀerent variables in the data. In this work, the students are expected to
make claims that are supported by their results. This often involves the students ﬁnding the slope
and/or intercept in a plot and interpreting those results with respect to their expectations (discussion
and analysis in Fig. 3.2).
LG3 - Collaboration: Throughout the experimental work and analysis, students discuss and make
decisions with their peers in their lab group. Students are encouraged to develop a consensus
approach to their work – deciding collectively where to take their experiment and analysis. Fur-
thermore, students are expected to make these decisions by grounding their discussions in their
experiment, data, and analysis.
LG4 - Communication: Overall, the entire process requires that students communicate with their
group and instructors. Additionally, students communicate their experimental approach and the
results of their work including their analysis in their lab notebook. Later, students provide a more
formal presentation of their work in the form of the communication projects.

It be should emphasized that this process is not content dependent; each laboratory activity
conducted by a student group follows this process. This generalization enables the core components

20

Figure 3.2: A snapshot of an experiment from pre-class homework through the communication
project.

21

of the course to be repeated (see Fig. 3.1) to help address external constraints, such as limited
equipment and time to work on experiments.

3.3.2 Communication Projects

DATA Lab is also deﬁned by the focus on authentic scientiﬁc communication through the com-
munication projects (CPs). The CPs are a formal way for the students to present their work and
they are one of the assessments of the course in which the work done by the students is completed
individually. CPs replace the lab practical from the traditional version of the course where students
would conduct a smaller portion of a laboratory by themselves. CPs occur in the middle and at the
end of the semester (red squares in Fig.3.1). In DL1, the CP is a written proposal that summarizes
the work the students conducted in one of their previous experiments and proposes an additional
investigation. In DL2, the students create and present a research poster on one of (or a portion
of one of) their experiments. In both courses, the projects are shared with and reviewed by their
primary instructor and their peers in the class.

Through the CPs, students continue to engage with the faculty consensus learning goals

(Sec. 3.2) as described below:
LG1 - Experimental Process: Students are expected to reﬂect on and summarize the process
through which they went to complete the experiment. In so doing, they must communicate their
rationale and reasoning for following that process.
LG2 - Data Analysis: The students must show that they can turn their raw data into an interpretable
result. Again, this is often and, ideally, done in the form of a plot of their data with the emphasize
of a model, including a ﬁt, is needed. Students also present and explain what the results mean in
the context of the experiment and a physical model.
LG3 - Collaboration: While the experiment was completed with the student’s group where they
may have consulted with their group mates, the CPs themselves are not inherently collaborative.
However, in DL1, the reviews that students perform on each other’s projects are done collaboratively
in their groups.

22

LG4 - Communication: The CPs are the formal communication of a student’s experimental work.
In both courses, a student’s CP is reviewed by their peers and feedback is provided describing
successes and shortcomings along with suggestions for improvements.

3.3.3 Final Projects

The course structure was designed with the intent to provide students with a variety of ways to engage
in the experimental physics practices. The ﬁnal projects are an additional form of communication
including an analysis and interpretation of experimental results through critiquing other scientiﬁc
results (DL1–Critique Project) and describing a new experimental design (DL2–Design Project).
Critique Project: For the ﬁnal project in DL1, students critique two sides of a popular science
topic. In the prior week, students are arranged into new groups and before the class meeting, they
must choose, as a group, from a list of possible topics such as climate change and alternative energy.
In class, students collectively write up a summary and critique both sides of the scientiﬁc argument.
Design Project: For the ﬁnal project in DL2, students choose an experiment that was conducted
previously and design a new experiment for a future semester of DATA Lab. Similarly to DL1, the
students are sorted into new groups and they must decide, as a group, which experiment they will be
working on before the class meeting. Due to the structure of the course, speciﬁcally everyone doing
diﬀerent experiments throughout the semester, this choice may be an experiment that individual
members of the group did not complete; negotiating this decision is part of the process of the Design
Project. In class, students construct two documents: (1) a document that explains the design of
the new experiment and (2) a document that would aide a future DATA Lab instructor to teach the
experiment. Through this ﬁnal project, DL2 students can design a project covering material that
they may not have had the chance to explore during the course.

For both ﬁnal projects, students turn in one assignment per group and they receive a single grade
(as a group) for the assignment. Students also assess their own in-class participation, providing
themselves a participation score (on a 4.0 scale) for the day. This score is submitted to their
instructor along with their rationale for assigning themselves the grade.

23

These projects oﬀer the ﬁnal opportunity for DATA Lab students to engage with the faculty-

consensus learning goals:
LG1 - Experimental Process: In DL1, students evaluate and summarize both sides of the chosen
argument by reviewing the relevant data and experiments. Although students are not conducting
an experiment, they are still asked to be critical of the experimental process in each side of the
argument. In DL2, students must create a clear procedure for their proposed experiment. Here,
they must consider the available equipment as well as how the data would be collected and why.
LG2 - Data Analysis: In DL1, the students must evaluate the evidence provided in each article.
They must decide if there are obvious ﬂaws in the way the analysis was conducted and if the analysis
is compelling; that is, if the overall claims made in article align with the data and analysis. In DL2,
students must consider the kind of analysis that would ﬁt with their experiment and the data that
they would collect. In addition, students are also expected to reﬂect on their analysis in light of the
models that are available to explain the data they would collect.
LG3 - Collaboration: In both courses, students continue to work as a group and are graded
accordingly. In addition, the students have been put into new groups, which they must adjust to.
LG4 - Communication: In both courses, students continue to communicate with their group as part
of the collaboration. In DL1 speciﬁcally, the ﬁnal project provides an opportunity to communicate
their own evaluation and critique of a scientiﬁc arguments. Students in DL2 are expected to
communicate to diﬀerent audiences, including future DATA Lab students and instructors, about
their newly planned experiment.

3.4 Overview of Key Supports

As the students’ work in this course is suﬃciently open-ended, speciﬁc supports to ensure
they feel capable of conducting the lab activities have been designed. Since the CPs are the main
assessments in the DATA Lab course sequence and are a large portion of their overall grade for the
course, the goals of the key supports are intended to provide students with the tools to help them
succeed in the projects. Each of the supports designed for DATA Lab will be discussed in detail

24

below (Secs. 3.4.1 & 3.4.2). Assessments will be discussed in Sec. 3.5 (also in Appendix B).

Broadly, the key supports for the students are outlined in Fig. 3.2. Before each class day,
students complete a pre-class homework assignment (vertical green lines). Students also have three
communication project homework (CPHW) assignments during the semester (vertical pink line)
to help them complete their CPs. These supports, in addition to feedback on students’ in-class
participation and lab notebooks, apply for any of the regular two week experiments (green squares
Fig. 3.1).
In the following section, these will be described in detail along with the additional
supports that were designed for the courses.

3.4.1 Typical Experiment

Each two-week experiment follows a similar path, highlighted in Fig. 3.2 and described, in part,
in Sec. 3.3. In this section, details of the general course components necessary to maintain the
ﬂexibility of the path students take through each experiment will be described.

Pre-Class Homework: At the beginning of an experiment, students are expected to complete the
pre-class homework assignment which includes reading through the lab handout and investigating
the suggested research. This assignment is usually 2-4 questions designed to have students prepare
for the upcoming experiment. For example, before the ﬁrst day of a new lab, students are asked
what they learned during their pre-class research and if they have any questions or concerns about
the lab handout. Between the ﬁrst and second class meeting of the two-week experiment, students
are expected to reﬂect on what they have already done and prepare for what they plan to do next.
Typically, the 2-4 questions include reﬂections from the prior week, such as any issues their group
ran into on the ﬁrst day, and what they intend on doing during the second day of the experiment.
Answers to the pre-class homework serve as additional information that the instructors can draw
on during the class; knowing what questions and confusions that their students might have can help
instructors be more responsive during class. Overall, the goal of the pre-class homework is for the
students to come into class prepared to conduct their experiment and this assignment is used to
hold them accountable for that preparation.

25

In-class Participation: With the overall intent of improving students’ speciﬁc laboratory skills
and practices that are outlined in the course learning goals (Sec. 3.2), students receive in-class
participation grades and feedback after every lab section (green squares in Figs 3.1 & 3.2) on their
engagement with respect to these practices. As the lab handouts do not provide students with
speciﬁc steps that they must take to complete the experiment, students are expected to make most
of the decisions together as a group. Generally, students have control over how their investigation
proceeds; however, this control varies between experiments (i.e. students choose how to set up the
equipment, what to measure, how to take measurements, etc.). The in-class participation grades
and feedback are where students are assessed most frequently and where they have the quickest
turnaround to implement the changes. See Sec. 3.5.1 for the details of how in-class participation is
assessed.

Lab Notebooks: For each experiment that the students engage in, they are expected to document
their work in a lab notebook. In comparison to formal lab reports, lab notebooks are considered a
more authentic approach to documenting experimental work. Furthermore, lab notebooks provide
students with space to decide what is important and how to present it. The lab notebooks are the
primary source that the students use to create their CPs. Like in-class participation, students receive
lab notebook feedback much more regularly than CP feedback, so they have greater opportunity
to reﬂect and make improvements. The speciﬁc details of the assessment of lab notebooks will be
explained in Sec. 3.5.1.

CP Homeworks: Three times during the semester the students complete CPHW assignments
in addition to that week’s pre-class homework. Each CPHW focuses on a relevant portion of the
CPs (e.g., making a ﬁgure and a caption). Through the CPHWs, the aim is for students to develop
experience with more of the CP components.
In addition, students receive feedback on these
diﬀerent aspects (see Sec. 3.5.1) , which they can act upon before they have to complete their ﬁnal
CPs.

Communication Projects: Throughout each semester, the students complete two CPs, the
ﬁrst of which is a smaller portion of their overall course grade. With the goal of providing the

26

students with a second opportunity to conduct a CP after receiving initial feedback, this course
design feature intends to create less pressure on students during their ﬁrst CP assignment. Students
are expected to reﬂect on the process, their grade, and the feedback before they have to complete
another CP. The CP assessment details will be discuss further in Sec. 3.5.2.

3.4.2 Additional Supports

Along with the support structures for the core components of the course sequence, additional
supports have been designed to ease students into the more authentic features of DATA Lab such as
designing experiments and documenting progress in lab notebooks. DL1 begins with three weeks
of workshops (purple squares in Fig. 3.1), followed by the introductory experiment (light green
squares in Fig. 3.1) that all of the students complete. DL2 begins with an introductory experiment
as well, under the assumption that the students already went through DL1. The workshops and
introductory experiments are designed to assist the students in navigating the diﬀerent requirements
and expectations of the overall course sequence, and of a typical experiment within each course.
The additional support structures are described in detail below.

DL1 Workshops: The ﬁrst workshop focuses on measurement and uncertainty with a push
for the students to discuss and share their ideas (LG1,3). The students perform several diﬀerent
measurements – length of a metal block, diameter of a bouncy ball, length of a string, mass of a
weight, and the angle of a board. Each group discusses the potential uncertainty associated with one
of the measurements. Then, students perform one additional measurement and assign uncertainty
to it. The second workshop also focuses on uncertainty but in relation to data analysis and
evaluating models (LG2,4) using the concept of a spring constant. Students collect the necessary
measurements, while addressing the associated uncertainty and plot the measurements to analyze
how the plot relates to the model of a spring. The ﬁnal workshop focuses on proper documentation.
The lab handouts do not contain their own procedure, so each student is expected to document
the steps they take and their reasoning (LG4) in their lab notebook. In preparation for the third
workshop, as a pre-class homework, students submit a procedure for making a peanut butter and

27

jelly sandwich, which they discuss and evaluate in class. Students are then tasked with developing
a procedure to determine the relationship between diﬀerent parameters (length of a spring and mass
added, angle of metal strip and the magnets placed on it, or time for a ball to roll down a chute
and how many blocks are under the chute). At the end of each workshop the students turn in their
notebooks, just as they would at the end of any experiment.

Introductory Experiments: In DL1, the introductory experiment occurs after the three work-
shops. All students conduct a free-fall experiment where they must determine the acceleration due
to gravity and the terminal velocity for a falling object. In DL2, the introductory experiment is the
ﬁrst activity in the course. This is because students will have already completed DL1 prior to taking
DL2; rather than being slowly introduced to what DATA Lab focuses on, students can be reminded
in a single experiment. The introductory experiment for DL2 involves Ohm’s Law; students must
determine the resistance of a given resistor.

As these are the ﬁrst DATA Lab experiments for either course, the instructors take a more
hands-on and guiding approach than they will later in the semester. In DL1, these instructional
changes represent a dramatic shift from the guidance students had during the workshops where
instructors are often quite involved. In DL2, the one week lab is intended to be simple enough
that students can be reminded of the expectations with respect to the overall learning goals of the
course.

CP Prep Day: As discussed in the prior section, the CPs comprise a large portion of the
students’ total grade in the course.
In addition to the supports that were already mentioned –
in class grades, notebooks, CPHW, and a lower stakes CP1 – in the spring semester, the MSU
academic calendar oﬀers time for a communication project prep day (pink squares in Fig. 3.1).
This gives the students an extra day where they have time to work on their CPs in class. They
can take additional measurements, seek help from their group or instructor, or work on the project
itself. This prep day allows for a gentler transition into the CPs with a bit more guidance. It also
reduces the amount of work that the students have to do outside of class.

28

3.5 In Course Assessments

The DATA Lab activities described above were designed around the overall learning goals
outlined in Sec. 3.2. As such, the course assessments were also aligned with these overall course
goals. There are two types of assessments used in DATA Lab – formative (to help the students
improve upon their work) and summative (to evaluate the students’ output); these are separated
for clarity. In this section, the various assessment tools are discussed with respect to the overall
learning goals of the course. (All rubrics are included in Appendix B).

3.5.1 Formative Assessments

In DATA Lab the formative assessments are comprised of students’ work on their in-class activities,
lab notebooks, and CPHWs. Other than the pre-class homework, which is graded on completion,
there is a rubric for each activity for which students receive a score. Each is structured to ensure
that any improvements students make carry over to their CPs.

In-class Participation: In-class participation feedback is broken into group, which covers the
general things everyone in the group or the group as a whole needs to work on, and individual,
which is speciﬁc to the student and not seen by other group members. The general structure of the
feedback follows an evaluation rubric used in other introductory courses and focuses on something
they did well, something they need to work on, and advice on how to improve [70]. It is expected
that students will work on the aspects mentioned in their prior week’s feedback during the next
week’s class. Students are graded based on their response to that feedback. Any improvements
they make with respect to the learning goals in class will also likely impact how well they complete
their CPs.

Students’ in-class participation is assessed with respect to two components, group function and
experimental design. Speciﬁcally, group function covers their work in communication, collabora-
tion, and discussion (LG3,4). For communication they are expected to contribute to and engage in
group discussions. To do well in collaboration, students should come to class prepared and actively

29

participate in the group’s activities. Discussion means working as a group to understand the results
of their experiment. Experimental design evaluates the process that students take through the
experiment and their engagement in experimental physics practices (LG1,2). They are expected to
engage with and show competence in use of equipment, employ good experimental practices (i.e.,
work systematically, make predictions, record observations, and set goals) and take into account
where uncertainty plays into the experimental process (i.e., reduce, record, and discuss it).

Speciﬁcally for the DL1 Workshops, instructors grade students diﬀerently than they would for a
typical experiment. The emphasis for the workshops is on the group function aspect of the rubrics,
communication and participation. This is because the students are being eased into the expectations
that the instructors have around experimental work.

Lab Notebooks: Feedback and grades for lab notebooks are only provided after the experiment
is completed (the two week block in Figs 3.1 & 3.2). Students receive individual feedback on their
notebook, although members of a group may receive feedback on some of the same things simply
because they conducted the experiment together. Like for in-class participation, it is expected that
the students will work on the aspects mentioned in their feedback for the next lab notebook and the
instructor can remind them of these things in class during the experiment.

Lab notebooks are also graded over two components, experimental design and discussion.
Experimental design focuses on the experimental process and how students communicate it (LG1,4).
Here, instructors typically look for clearly recorded steps and results, and intentional progression
through the experiment. Discussion covers uncertainty in the measurements and the models, as
well as the results, with respect to any plots and conclusions (LG2,4). These evaluation rubrics for
the lab notebooks were designed to be aligned with the those for the CPs, so that when students
work toward improving their notebooks they are also making improvements that will beneﬁt their
CPs. For example, if a student is getting better at analyzing data and communicating their results
within their notebooks, instructors should expect the same improvement to transfer to their CPs.

For the DL1 Workshops, the lab notebooks are graded on the same components but the grades
and feedback are speciﬁcally focused on the parts of the rubric that the students should have

30

addressed in each of the previous workshops. For example, as documentation is emphasized in the
last workshop, the students are not heavily penalized on poorly documented procedures in the ﬁrst
two workshops.
CPHW: The goal of this CPHW is to have students think about creating a more complete CP
that connects their in-class work to the bigger picture. Students are evaluated on the quality and
relevance of their sources, including the background and real-life connections (LG2,4). Each
CPHW has a diﬀerent rubric because each one addresses a diﬀerent aspect of the CPs. Figure and
caption: The students create a ﬁgure with a robust caption based on the data from one of the labs
they completed. Both the ﬁgure and the caption are evaluated on communication and uncertainty
(LG2,4). For the plot, the students are expected to visualize the data clearly with error bars and it
should provide insight into the various parameters within the experiment. For the caption, students
need to discuss what is being plotted, make comparisons to the model including deviations, and
draw conclusions that include uncertainty.
Abstract: For a given experiment, students write a research abstract that covers the main sections
of their project including introduction, methods, results, and conclusion. These are assessed on
experimental process (motivation and clarity of the experiment) (LG1,4), and discussion (results
and conclusions) (LG2,4).
Critique (DL1 only): Students are given an example proposal that they must read, critique, and
grade. This assignment plays two roles. First, students must examine a proposal, which should help
to produce their own. Second, students must critique the proposal, which should help them provide
better critiques to their peers. Students’ performance is evaluated based on their identiﬁcation of
the diﬀerent components of a proposal, and the quality of the feedback they provide (LG4).
Background (DL2 only): Students are tasked with ﬁnding three out-of-class sources related to one
of their optics experiments, which they must summarize and connect back to the experiment.

31

3.5.2 Summative Assessment

The CPs form the sole summative assessment of student learning in DATA Lab. As described
above, each of the formative assessments are designed to align with the goals of the CPs.

CPs: As mentioned above, although students conduct the experiments together, the CPs are
completed individually. In DL1, students’ CP is a proposal that emphasizes their prior work and
discusses a proposed piece of future work. As a result, the CP rubric is divided into two sections,
prior and future work. Within those sections, there is a focus on experimental design and discussion.
This rubric was iterated on after piloting the course for two semesters as it was found that students
would often neglect either their future work or prior work when they were not directly addressed
in the rubric; the rubrics were reorganized in order to account for this. Experimental design,
which covers methods and uncertainty, focuses on the experimental methods and the uncertainty
in measurements, models, and results when students discuss their prior work (LG1,2,4). In future
work, experimental design refers to the proposed experimental methodology and the reasoning
behind their choices (LG1,4). For the student’s discussion of prior work, the rubric emphasizes
how the they communicate their results (LG2,4). When students discuss their future work, the
rubric emphasizes the novelty of the proposed experiment and the arguments made on the value of
the project (LG1,4).

In DL2, students’ CP is a poster that they present to their classmates for peer review. The rubric
includes an additional component on the presentation itself, but the rubric still emphasizes the ex-
perimental design and discussion. Experimental design covers communication of the experimental
process including students’ reasoning and motivation. Discussion focuses on the discussion of
uncertainty (i.e., in the measurements and models) and the discussion of results (i.e., in the plot and
conclusions). The additional component focusing on presentation is divided into speciﬁcs about
the poster (i.e., its structure, ﬁgures, layout) and the student’s presentation of the project (i.e., clear
ﬂow of discussion, ability to answer questions).

32

3.6 Example Experiment

Overall, the course structures, supports, and assessments of DATA Lab have been discussed.
In this section, the key supports will be grounded in examples from a speciﬁc experiment. The
details of a speciﬁc two-week experiment will be described to better contextualize the features of
the course. Additional experiments are listed in Tables A.1 and A.2 in Appendix A. The chosen
experiment is from DL2 and is called “Snell’s Law: Rainbows.” In this experiment, students
explore the index of refraction for diﬀerent media and diﬀerent wavelengths of light.

Before attending the ﬁrst day of the laboratory activity, students are expected to conduct the
pre-class homework assignment, including the recommended research in Fig. 3.3. In addition, the
homework questions for the ﬁrst day of a new experiment address the pre-class research, as follows:

Research Concepts
To do this lab, it will help to do some research on the concepts underlying the bending of light at
interfaces including:

• Snell’s Law (get more details than presented here)
• Refraction and how it diﬀers from reﬂection
• Index of refraction of materials
• Fiber optics
• Using this simulation might be helpful: http://goo.gl/HEﬂDI
• How to obtain estimates for ﬁts in your data (e.g., the LINEST function in Excel -

http://goo.gl/wiZH3p)

Figure 3.3: Pre-class research prompts for the Snell’s Law lab.

1. Describe something you found interesting in your pre-class research.

2. From reading your procedure, where do you think you may encounter challenges in this lab?

What can you do to prepare for these?

3. Considering your assigned lab, is there anything speciﬁc about the lab handout that is unclear

or confusing?

33

Part 1 - Observing Light in Water
At your table, you have a tank of water and a green laser. Turn on the green laser and point it at the
water’s surface.

• What do you notice about the beam of light in the water?
• What about the path the light takes from the source to the bottom of the tank?

Let’s get a little quantitative with this set up. Can you measure the index of refraction of the water?
You have a whiteboard marker, a ruler, and a protractor to help you. Don’t worry about making
many measurements, just see if you can get a rough estimate by taking a single measurement.

• What does your setup and procedure look like for this experiment?

• What part(s) of your setup/procedure is(are) the main source of uncertainty for this

measurement?

• Can you gain a sense of the uncertainty in this measurement?

• How close is your predicted value to the “true value” of the index of refraction of mater?

On the optical rail you have a half circle shape of acrylic that is positioned on a rotating stage,
with angular measurements. You also have a piece of paper with a grid attached to a black panel
(i.e., a “beam stop”). Using this setup, you will test Snell’s Law for the green laser. Your group
will need to decide how to set up your experiments and what measurements you will make. You
should sketch the setup in your lab notebook and it would be good to be able to explain how your
measurements relate to Snell’s Law (i.e., how will the laser beam travel and be bent by the acrylic
block?). In conducting this experiment, consider,
• What measurements do you need to make?
• What is the path of the laser beam and how does it correspond to measurements that you are

making?

• What is a good experimental procedure for testing Snell’s Law?
• What kind of plot is a useful one to convey how the model (Snell’s Law) and your

measurements match up?

• Where is the greatest source of uncertainty in your experimental setup? What does that

mean about the uncertainty in your measurements?

Figure 3.4: Snell’s Law: Rainbows Lab Handout. Top: Exploring refraction, ﬁrst day of Snell’s
Law. Bottom: Beginning model evaluation, main Snell’s Law activity.

The ﬁrst day of the lab begins with exploring refraction in a water tank. Students are asked
to qualitatively explore the index of refraction of the water using a simple setup (Fig. 3.4). The
exploration is fully student led; they investigate the laser and tank, discussing what they see with

34

their group as they go and recording their observations in their notebooks. Students observe that
the path of the light changes once the laser crosses the air-water boundary. Students are then lead
to a quantitative exploration by determining the index of refraction of the water; instructors expect
the students to have an idea of how to do this after their pre-class research. If students are not sure
how to start, they are encouraged to search for Snell’s Law online where they can quickly ﬁnd a
relevant example. The instructors check in with the students toward the end of this work. Typically,
instructors will ask about the questions outlined in the lab handout.

The next part of the experiment is where students work to gain precision in their measurements
and evaluate the model of the system. This part is most similar to a traditional laboratory course.
The diﬀerence is that the students are told the goal but not how to proceed (see Fig. 3.4). There
are a number of decisions they must make as a group as they progress. Students record and explain
their decisions in their lab notebooks; they might also discuss them with their instructor.

Typically by the end of the ﬁrst day students know how to set up their experiment and have
documented that in their lab notebooks. They are unlikely to have taken more than one measurement
(the design and investigation phase in Fig. 3.2). They will return the following week to complete
their experiment. The homework questions between the ﬁrst week and the week that they return
emphasize students’ reﬂections on the previous week. Students also are asked think about the
experiment outside of class. The typical homework questions prior to Week 2 are the following:

1. Because you will be working on the same lab this week, it is useful to be reﬂective on your
current progress and plans. Describe where your group ended up in your current lab, and
what you plan to do next.

2. Now that you are halfway through your current lab and are more familiar with the experiment,

what have you done to prepare for this upcoming class?

3. Describe something that you found interesting in your current lab and what you would do to

investigate it further.

35

Figure 3.5: Sample of a student’s Communication Project for DL2. Blue: graph with sine of the
angle of incidence plotted against sine of the angle of refraction for each wavelength of light. Green:
The slope for each wavelength, which is the index of refraction of the block. Red: Results and
conclusions where they discuss the diﬀerences in the indices of refraction and how that is related
to rainbows.

The second week starts with setting up the experiment again and beginning the process of
taking multiple measurements. At this point, students often break up into diﬀerent roles: someone
manipulating the equipment, one or two people taking measurements, and someone recording the
data and/or doing calculations. These roles are what students appear to fall into naturally, and are
not assigned to them. Although, if one student is always working in excel or always taking the
measurements, instructors will address it in their feedback where they encourage the students to
switch roles.

The next step depends on the amount of time that students have left in the class. If there is not

36

much time, students focus on the data from one wavelength of light. If they have more time, they
can make the same measurements with lasers of diﬀerent wavelengths. In both cases, students can
determine the index of refraction of the acrylic block. With multiple wavelengths, students are able
to see that the index of refraction depends on wavelength. This leads to a conversation with the
instructor about how this relates to rainbows and a critique of the model of refraction – Snell’s Law.
Most of the analysis that students conduct in this example experiment is the same regardless of
how many lasers they collected data (discussion and analysis in Fig. 3.2). While considering the
diﬀerent variables in their experiment, students are expected to make a plot where the slope tells
them something about the physical system. In this case, the design is intended for the students to
plot the sine of the angle of incidence on the x-axis and the sine of the angle of refraction on the
y-axis, which makes the slope the index of refraction of the acrylic block. The optics experiments
occur in the second half of the semester after the students have become familiar with constructing
linear plots from nonlinear functions. For this lab, students usually do not have much diﬃculty
determining what they should plot. After they obtain the slope and the error in the slope, students
will typically compare it to the known index of refraction of the acrylic block. They must research
this online as it is not provided anywhere for them in the lab handout.

The second day of the experiment ends with a discussion of their plot. Students construct a
conclusion in their notebooks that summarizes the results, what they found, what they expected,
reasons for any diﬀerences, and an explanation of what it all means in the larger physics context.

After the experiment, the students may have their third and ﬁnal CPHW, background/literature
review. In the case of Snell’s Law, students would be asked to ﬁnd three additional sources where
these concepts are used in some other form of research, often in the ﬁeld of medicine but also
in physics or other sciences. Students then summarize what they did in class and connect their
experimental work to the sources that they found.

The student can choose to do their second CP on this experiment. An example of a poster can
be seen in Fig. 3.5. In the ﬁgure, three key features are highlighted. First, in the blue box, is the
graph where students plotted all three wavelengths of light. In the green box, is the slope for each

37

color, which is the index of refraction of the acrylic for each laser. Finally, in the red boxes, are
their results and conclusion. In the top box, students explained why their indices are diﬀerent, that
is, because of the assumption that Snell’s Law is wavelength independent. In the bottom box, they
make the connection to rainbows. The student would present this poster during the in-class poster
session, to their peers and their instructor.

3.7 Redesign Eﬃcacy

To measure the eﬃcacy of the DATA Lab course transformation, the Colorado Learning At-
titudes about Science Survey for Experimental Physics (E-CLASS) [1] was implemented in the
traditional laboratory course as well as the transformed courses. The E-CLASS is a research-based
assessment tool used to measure students’ epistemological beliefs, speciﬁcally students’ views
about learning experimental physics, and their overall expectations about a course designed around
experimental physics [1, 71, 72, 73]. This instrument was designed to probe various aspects of
experimental physics and although it has been recommended that ‘instructors should examine their
students’ responses to the questions individually with a particular focus on those questions that are
most aligned with their learning goals for that course’ [71], here, we will report student outcomes
on the overall instrument. Future research will include analyzing student responses to the E-CLASS
items in connection to the speciﬁc learning goals described in Sec. 3.2.

The well-validated survey consists of 30 items (5-point Likert scale) where students are asked
to rate their level of agreement with each statement. The scoring method of this assessment was
adapted from previous studies [74]. First, the 5-point Likert scale is compressed into a 3-point scale;
“(dis)agree” and “strongly (dis)agree” are combined into one category. Then, student responses
are compared to the expert-like response; a response that is aligned with the expert-like view is
assigned a +1 and a response that is opposite to the expert-like view is assigned a −1. All neutral
responses are assigned a 0.

In DL1 and DL2, the E-CLASS was administered as an online extra credit assignment both
pre- and post-instruction. Throughout the course transformation, DL1 and DL2 collected a total

38

Figure 3.6: Comparing student E-CLASS outcomes between the traditional course and the trans-
formed course: (a) Fraction of statements with expert-like responses as presented in Zwickl et al.
[1] and (b) Shifts in student expert-like thinking from pretest to post-test as presented in Wilcox
and Lewandowski [2].

number of responses from 1,377 and 925 students, respectively, with matched (both pretest and
post-test) E-CLASS scores. Fig. 3.6 presents two of the most common ways that E-CLASS scores
are reported. Here, the overall results between the traditional course and the transformed course
will be compared. Fig. 3.6(a) illustrates the fraction of statements with expert-like responses from
pretest to post-test, which was introduced in the initial E-CLASS manuscript [1]. Fig. 3.6(b)
illustrates the overall shift in students’ expert-like thinking from pretest to post-test which has been
a more recent way of representing overall E-CLASS results [71, 2].

As shown in Fig. 3.6(a), the fraction of E-CLASS statements with expert-like responses in the
traditional courses signiﬁcantly decreased (ps (cid:28) 0.001) by 3% in DL1 and by 1% in DL2 from
pre- to post-instruction. However, in the transformed courses, the fraction of expert-like statements
increased by 4% and by 6% in DL1 and DL2, respectively (ps (cid:28) 0.001). In reference to the analysis
method presented by [1], the DATA Lab course transformation showed a signiﬁcant positive impact
on the fraction of statements on the E-CLASS with expert-like responses.

Fig. 3.6(b) shows that, in both of the traditional courses, students’ overall expert-like thinking

39

decreases signiﬁcantly (ps (cid:28) 0.001) from pre-instruction to post-instruction, speciﬁcally by 2.1
points (pre = 14.4 points, post = 12.3 points) in DL1 and 1.1 points (pre = 12.5 points,
post = 11.4 points) in DL2. However, in both DL1 and DL2, there was a signiﬁcant positive
shift (ps (cid:28) 0.001) in students’ expert-like thinking, 1.6 points (pre = 14.9 points, post = 16.6
points) and 2.8 points pre = 14.3 points, post = 16.1 points), respectively. Overall, the DATA
Lab course transformation showed a signiﬁcant positive impact on students’ expert-like thinking
toward experimental physics.

3.8 Conclusion

In this paper, the large scale transformation of the MSU algebra-based physics labs for life
science students was described. The design was divorced from the speciﬁc physics content because
the learning goals developed from a faculty consensus design did not include speciﬁc content. This
design means that the individual lab activities do not matter per se, but instead the structure of the
course and how students work through the lab are what is important. Theoretically, one could adapt
this design to a chemistry or biology lab by making adjustments to the kinds of lab activities, and
relevant changes to the learning goals. That being said, there are still key structures to ensure the
functioning of the course which will be covered in detail in a subsequent paper (e.g. a leadership
team of four instructors, two GTAs and two ULAs, tasked with maintaining consistent grading and
instruction across the sections).

The transformation was centered to emphasize experimental physics practices. The overall
eﬀorts were focused on the two course series because the majority of the students that are taking
courses in the physics department at MSU are enrolled in the introductory algebra based series,
speciﬁcally 2,000 students per year.
In addition, the majority of the student instructors in the
MSU physics and astronomy department, nearly 80 graduate teaching assistants and undergraduate
learning assistants, teach in these labs. Because of its scale, special attention was given to the
voice of the physics faculty in the development of the learning goals for DATA Lab [41]. The
entire course was designed around the faculty-consensus learning goals, which are all based around

40

physics laboratory practices (Sec. 3.2). From course structures to assessments, everything was
intentionally aligned with the overall learning goals. Each component of the course builds upon
another through the two semester sequence. Each individual lab activity builds upon skills that will
be valuable for each subsequent activity, from lab handouts to pre-class homework assignments.
Such an eﬀort was put into designing this course sequence in large part because of the number of
MSU undergraduate students they are serving. The value in physics labs for these non-majors lies
in the scientiﬁc practices on which the redesign was centered. Those skills and practices are what
they will take with them into their future careers.

41

CHAPTER 4

IDENTITY

4.1 Introduction

Laboratories are a hallmark of science education and have been for over a century [23]. These
tools of science education provide students with opportunities to engage in the process of ’doing
science,’ i.e., in the skills and practices relevant to applying the scientiﬁc method. Physics education,
along with other ﬁelds of science education, has not always been successful at taking advantage of
this opportunity. Introductory labs, for example, are often still prescribed step-by-step experiments
with goals limited to understanding content. Lab courses have been found to add little to no
beneﬁt in content and conceptual goals [10, 11]. Instead, designing lab courses to emphasize the
unique opportunity provided for engagement in skills and practices necessary to do science has
recently become a focus thanks to the Next Generation Science Standards (NGSS) and the American
Association of Physics Teachers (AAPT) committee on laboratories [12, 5]. Both have pushed for
a focus in science and physics laboratory education to better represent experimental science and
physics. Upper level physics labs are more often structured this way than introductory ones, but
the skills and practices are just as relevant in both stages. This is true for physics majors as well as
other STEM majors. In either case, we want students to come out of our physics courses knowing
what it means to do physics.

Engagement with the practices of the physics community is necessary in the process of identi-
fying with the community [4, 53, 55, 56]. The fact that lab classes, especially recently, are a space
for this type of engagement implies that they are also spaces where identity development is likely
happening. Field speciﬁc identity has been found to impact persistence and retention in the major
[18], choice of major and future career [75], and to be linked to underrepresentation in the ﬁeld
[20, 19]. It could be argued that the value in physics identity development only applies to students
within the physics major but at many large institutions the dominant population of students served

42

are non-physics STEM majors. There are two main reasons why it is necessary to care about the
impact on their identity in this case. First, it is important to do well by the largest population of
students that our departments serve. Doing right by them in the case of lab courses is providing
them with authentic experiences in doing experimental physics. Second, the skills and practices
that are valued in physics are also relevant in other STEM ﬁelds.

In this study we provide a starting point for investigating identity development in lab classes
as connected to the practices emphasized in those courses at a large scale. This is part of our
work on developing a survey to measure physics identity in lab classes. Physics labs and authentic
experiences are relevant for all STEM students, from upper division physics majors to introductory
non-physics majors. The push to include authentic experiences with practices in physics laboratory
education solidiﬁes these practices as a relevant and impactful aspect of these courses and in
turn motivates us to ground our study of physics identity in the speciﬁc valued practices of the
ﬁeld. In this paper, we introduce a theoretical framework that ties identity and laboratory practices
together, and then demonstrate the applicability of this framework through the coding analysis of
15 interviews and 140 open-ended written responses. We will demonstrate through this data and
analysis that we can use a practice based identity framework in laboratory classes. We also call for
additional work on the components of identity that are assessable in these settings.

In the following sections we ﬁrst justify the relationship between labs, practices, and identity
from the existing literature (Sec. 4.2). We then describe the data sources and methods used in our
investigation (Sec. 4.3). Following that, in Sec. 4.4, we describe the analysis process and present
the results of that analysis. Finally we discuss the outcome of the adaptation of the combined
framework in our context and the implications of this work (Sec. 4.5).

4.2 Theoretical Background

We start out by grounding the motivation of our work in the literature. This section includes
a discussion of evidence for the prevalence of practices as central goals in science laboratory
education for over a century. There is a look at the work done showing how experiences in labs

43

and with lab practices work to produce attitudinal changes. Those changes can be connected to
some of the literature on physics identity. We highlight how practices may impact identity and
the framework being adapted is introduced. The work presented here is focused on adapting the
combined identity framework as introduced in Close et al, 2016 [21] (and described below).

4.2.1 Lab Practices

Historically laboratories have been an integral piece of science education. “The laboratory has won
its place in school; its introduction has proved successful. It is designed to revolutionize education.
Pupils will go out from our laboratories able to see and do” (Griﬃn 1892, cited by [6, 7]). The
process and practices that go along with it have also been a valued part of scientiﬁc education.
In the 1960s the ‘discovery-based learning’ movement lead to inquiry as a way to learn scientiﬁc
content [27, 25, 26]. The goals of both inquiry and laboratory education at the time are reﬂective
of what we now refer to as science practices. The goals include: communicating [28], analyzing
data [29], planning and executing experiments [28, 24, 29], and understanding equipment [28, 30],
in addition to inquiry skills [31] and the scientiﬁc method [24].

In the late 1990s a focus was again placed on inquiry in science and physics education through
the publication of the National Science Education Standards [32]. After this came the development
of ISLE (Investigative Science Learning Environment) with their emphasis on scientiﬁc abilities
[34, 35, 45]. They deﬁne scientiﬁc abilities as “the most important procedures, processes, and
methods that scientists use when constructing knowledge and when solving experimental problems”
[35].

Most recently the American Association of Physics Teachers released their recommendations
for undergraduate physics labs: six focus areas all emphasizing practices [5]. In addition to that
the Next Generation Science Standards address experimental and lab practices [12]. In response to
this, the ﬁeld has shifted the focus of lab courses to make practices and skills, not just a beneﬁt of
labs, but a central focus of the design of these courses [40, 38, 76, 77, 78, 39, 37, 79, 80, 81].

The literature shows that this emphasis on practices also translates to attitudinal changes.

44

Bechman [17] and Ergül [15] found that experiences with lab and research practices improved
attitudes about science. Others have found that authentic experiences with practices increase
interest in science and in science careers, and that students ﬁnd practical activities more enjoyable
[82, 16]. Although there has been work showing these positive eﬀects of experiences with lab
practices, there have not been large scale studies. In addition, the assessment tools to do that are
limited. As of this writing, there is one such tool that measures attitudes and expectations in physics
laboratories, the Colorado Learning about Sciences Survey in Experimental Physics (E-CLASS)
[1]. There is still a void in the literature, especially when it comes to speciﬁc attitudinal and aﬀective
measures, which is why we have been working to develop an assessment that measures the impact
experiences with lab practices have on a students’ physics identity. The full survey development
is not the focus of this paper, instead we will focus on the analysis speciﬁc to understanding and
analyzing physics identity in lab classes.

4.2.2 Physics Identity

As part of introducing the identity framework used in this research, several perspectives on identity
are deﬁned and discussed. One way to look at physics identity is highlighted in Hazari et al., 2010
[19]. In that work they analyzed responses to the Persistence Research in Science and Engineering
(PRiSE) survey. The elements they focused on for their analysis broadly focused on high school
physics experience and were correlated with an item that stated “I see myself as a physics person.”
Hazari deﬁnes physics identity with four components: competence, performance, recognition,
and interest. Recognition is mainly perceived recognition from parents or peers. Interest refers
speciﬁcally to interest in physics, coming from the fact that aﬀect has a large impact on persistence.
Competence and performance are connected: competence is belief in one’s ability to understand
the science content and performance is belief in one’s ability to perform tasks. In the study Hazari
et al. found recognition had the largest eﬀect on physics identity.

According to Wenger [4] identity is solidiﬁed through participation in speciﬁc communities
and it is produced by lived experiences in those communities. Identity is lived, negotiated, social,

45

a learning process, a nexus, and a local-global interplay. Lived because it involves participation
and reiﬁcation. Negotiated because it is always a work in progress and never permanent. Social
because it is formed through participation in a COP. A learning process because it is a trajectory
that incorporates the past and future to establish meaning in the present and creates coherence
through time. A nexus in that people are members of multiple COPs and the diﬀerences must be
continually reconciled. A local-global interplay due to the fact that one participates in a local COP
but must also negotiate their connection to the broader world.

Close et al. [21] combine the four components from Hazari et al. with multiple aspects of
COP and identity formation in a study of how identity development is impacted by participation in
an undergraduate learning assistant (ULA) program. Close et al. condensed the above mentioned
deﬁnitions of identity into four combined components. The ﬁrst is negotiated experience from
COP combined with interest from Hazari et al.. They deﬁne this as making sense of how others
see and react to us, including recognition as competent and valued. In their data this manifests for
the ULAs in the ways in which they can impact and inﬂuence the students they are working with.
The next is community membership combined with competence. This is described as competence
developed and valued by members of the community. This showed up in the form of competence
in teaching and understanding physics as well as the development of a community of physics
ULAs and a connection with the broader physics community. Then is learning trajectory combined
with interest. According to Close et al. this incorporates past and potential future identities to
make sense of the present. For the ULAs, participating in the program increased their interest in
teaching and they viewed the experience as valuable for their future. Finally there is the nexus of
multimembership combined with an integrated physics identity, such as incorporating practices and
values from diﬀerent communities. This is evidenced by the ULAs having increased competence in
one community from experience in another, the ULA community and the larger physics community.

46

4.2.3 Labs as Communities of Practice

As stated in Wenger [4] identity development happens through authentic participation in a speciﬁc
community. This refers speciﬁcally to participation in practices that are valued by a community. In
Irving [57] they describe the ways in which a lab course can become a community of practice. The
concept of a classroom community as a community of practice is not a new one [58, 59]. But it is
an eﬀective way to examine the impacts an individual course can have. Irving et al. [57] motivate
the relevance of a lab class as a COP with the fact that “A community that has many similarities to
that of a professional research environment provides the opportunity for students to recognize each
other as more authentic physicists than before.” This motivates further the value of participating in
authentic practices for students in developing a physics identity.

Although not all (maybe not even most) lab classes form communities of practices that eﬀectively
progress students toward central membership in the community of practicing physicists, they all
have the potential to impact a student’s relationship with physics [57]. Common practices are not
the only things that make a community of practice, but they are a distinct feature and one that is
often made more apparent in lab classes. The experiences the students have with those physics
practices, in participation or in integrating them with those from other communities, provide us
with information about their physics identity.

Irving and Sayre [56] argue that positive experiences with practices that are seen to be more
legitimate and authentic can produce a change in identity status. This supports our goals to examine
experiences with practices as a means to understand identity. We may be able to catch a snapshot
of the exploration described in Irving [56].

Grounding identity in communities of practice provides tangible elements that can inﬂuence a
student’s identity. One can determine if they are given the opportunity to perform practices, if they
know the practices at all, if they are recognized by the community, if the accepted practices are
at odds with some other part of their identity, which may be particularly relevant for non-physics
majors in physics lab classes. With this as a motivator and lab classes as the COP of interest, we
worked to apply the combined Close framework as an operationalized deﬁnition of physics identity.

47

We only utilized three of the four components described above. We left out the component focused
on the nexus of multimembership and an integrated physics identity not because it is incorrect, but
because the component of learning trajectory and interest covers this to the extent that it is relevant
to this stage of the work, as we will describe below.

4.3 Methods

Before it can be determined whether or not the Close framework can be applied to this context
the following question must be answered: “How is identity development connected to the valued
practices within the laboratory physics community?” The literature provides support for the claim
that these two things are connected but evidence of how they are connected is the goal of this study
through the development of a detailed codebook. First interviews were conducted with students
in introductory and upper division physics labs. Then an open-response survey was distributed in
order to determine if the results from the interviews were representative of the broader population.

4.3.1

Interviews

The ﬁrst step in operationalizing the connection between identity and lab practices was to interview
students about their experiences in physics lab classes in order to better understand how they
interacted with and interpreted the practices that were emphasized in the lab. Fifteen students were
interviewed about their experiences in the physics labs they were currently enrolled in. Six students
were physics majors or minors in upper level optics or the advanced lab. Nine students were in the
introductory algebra-based labs for non-majors, four in the transformed version of the EM course
(Ch 3) and ﬁve in the traditional version of the mechanics course.

The ﬁfteen semi-structured interviews followed the same general structure (see Appendix C
for interview protocol). The students were asked about their past physics lab experience and their
current lab course. Once the background was established, they were asked to deﬁne several diﬀerent
lab practices in the context of their physics lab class. Most participants were also asked to compare
this to any other lab experiences they may have had (e.g., earlier physics labs, chemistry and biology

48

labs). After they provided their interpretation of the practice in their physics labs, they were asked
whether or not they found that application of the practice valuable. They were speciﬁcally asked
about how valuable they found these practices, while keeping ‘valuable’ open to interpretation,
because of the connection to the practices valued in a speciﬁc community and the potential impacts
on identity development, especially with respect to integrating overlapping identities.

4.3.2

Interview Analysis Methods

The interviews were analyzed through open coding1 of recordings of the interviews. The majority
of the coding was done using the qualitative analysis software, MAXQDA, directly on the video.
The primary focus of the analysis was developing a representative understanding of the diﬀerent
interpretations of the physics practices. Through the analysis, the diﬀerent ways the practices
were described by the students were determined and were used in the open-response survey. The
secondary analysis of these data focused on operationalizing the connection of these practices to
identity statements and ﬁnding examples of how students make identity statements in connection
to the practices.

4.3.3 Open-Response Survey

In order to determine if the working deﬁnition of identity from the interviews was broadly applicable,
an open-response survey was distributed to get more student responses. Each question for the open-
response survey had four parts, two closed- and two open-response. The closed response questions
were Likert scale with the same structure ‘This practice is important...’ they then have two responses
to this ‘For ME’ and ‘For an EXPERIMENTAL PHYSICIST.’ (See Fig. 4.1 with survey example)
We asked for both responses because, like the ECLASS [1], during beta tests of the survey the
participants were uncertain if they should answer based on their own personal thinking or based on
the ‘correct’ answer. This structure enabled us to pay speciﬁc attention to the students’ personal
1Sorting responses into speciﬁc categories that come out of the analysis and are not based on a

speciﬁc theoretical framework.

49

Figure 4.1: Example question from the open-response survey

responses with potential diﬀerent avenues of analysis available with the responses speciﬁc to the
experimental physicist (or ‘expert’) questions. The practice speciﬁc questions, 108 in total, were
also developed from the interviews. The process is described in Chapter 5. The next two questions
were free-response, and developed with the goal that the students would supply an explanation or
justiﬁcation as part of identity statements that could be coded. Those questions were, ‘For ME, this
practice is important (not important) because...’ and the same thing but ‘For an EXPERIMENTAL
PHYSICIST.’ An example question is shown in Fig. 4.1.

The open-response survey was distributed via Qualtrics2 at the end of the spring semester
2018, to a transformed introductory algebra-based physics lab for non-physics majors with 640
students. Responses from approximately 500 students were collected, a 78% response rate. Each
student received a randomly selected set of ten of the 108 practice questions, in the format shown
and described above. As compensation for completing the survey, the students were given a
small amount of extra credit added to their in-class participation scores. The high response rate
is attributed in part to the extra credit but also to in-class advertising that the graduate teaching

2Qualtrics, 2019, Qualtrics Labs, Inc. Provo, UT, USA

50

assistants and undergraduate learning assistants that teach the course conducted in response to a
request from the developer, and the lead teaching assistant at that time.

4.3.4 Survey Analysis Methods

The primary analysis was focused on the questions that referenced what the students personally
thought and not what they thought an ‘expert’ would think (e.g.
the responses to ‘for ME’
and not ‘for an EXPERIMENTAL PHYSICIST’). Based on the initial coding of identity in the
interview analysis, the results from the open-response survey were analyzed for identity in parallel
with work that focused on practice-speciﬁc interpretations. From that subset and the interviews,
initial deﬁnitions for the components of the identity framework were established and they were
sorted according to emergent themes. Then the emergent codes were compared to the Close
et al. framework. The codes were reduced to parallel those used by Close et al. and then the
operationalized deﬁnition was validated with an external researcher. The researcher was provided
with a codebook and an initial teaching set of twenty responses to code in the diﬀerent identity
components, followed by forty additional training responses. The coding of the ﬁnal set of responses
(representing 25% of the total data) was used in the calculation of interrater reliability. The
researcher was given responses that were already determined to be identity, and was instructed to
code each response in whole into one of the identity components. This additional validation was
required to conﬁrm that the diﬀerent responses could be reliably coded (with a Cohen’s Kappa
reliability measure of 0.92 (near perfect agreement) [83, 84]) and as added conﬁrmation of the
interpretation of evidence of the components in the data.

4.4 Results and Analysis

4.4.1 What Makes an Identity Statement

Before any work could be done utilizing the Close model of identity, a deﬁnition of identity in
this context was needed. Each example of an identity statement discussed has a connection to
a speciﬁc lab practice as aligned with the goal of connecting experiences with lab practices to

51

Figure 4.2: Figure showing process of determining what makes an identity statement

physics identity. This deﬁnition was developed through the diﬀerent stages of the analysis. The
ﬁrst requirement is that the statement must connect to a lab practice. This requirement is in
connection with the claim that practices can impact identity and with the goals to examine that
connection in lab classes through the development of a survey. We cannot claim the origin of any
statement regarding identity unless it is clearly grounded within the context of interest, lab classes
and practices. The next requirement is based on the literature that an identity statement would be a
personalized statement of the value of the practice. With those two pieces of the deﬁnition in mind,
the interviews were analyzed for examples of these personalized value statements connected to the
lab practices. Several potential identity statements from the interviews were discussed with experts
on identity and other qualitative research methods [85]. From the consultation with experts, it was
further established that in order for a statement to be counted as identity it needs to be personalized.
It was also determined that in addition to the personalization, there needs to be an explanation or
justiﬁcation for the personalized value statement, e.g. ‘this practice is important to me because of
this reason.’ The following examples highlight the criteria at this stage and work to provide insight
into the development of three speciﬁc criteria described at the end of this section (see Fig. 4.2).

52

Table 4.1: What makes an identity statement? These examples are used to aid in answering that questions. Ex 1-3 come from the
interviews and Ex 4-7 come from the open-response survey.

Label Response

Data

Ex 1

Ex 2

Ex 3

Ex 4

Ex 5

Ex 6

Ex 7

Blake (discussing uncertainty): “[...] I do understand its importance,
and see it as a necessary thing. It’s just, I personally will never really
need to take uncertainty as seriously as it is taken in these upper-level
lab classes and...I don’t look at it the same way as someone who’s
thinking ‘I want to do experimental physics for the rest of my life”’.
Adam: I really enjoy it, I enjoy working on something and feeling like
you’re making progress. It’s not like following the steps and you end
up where you’re supposed to end up, right? So um I guess it gives
a sense of possession and a sense of accomplishment to actually do
something because you weren’t just gifted it.
Beth (describing something she found exciting): “It was good to, plug
it all into the calculator and [see] this is a reasonable answer. So, getting
something that...could be right and doing that without a procedure is
more self-satisfying because...you relied more on yourself”
Q: Determining patterns from my data is important. A: it is not so
important to look for trends in data because overall it is only a small
part of what I am studying for my degree and will not necessarily help
me in my overall career.
Q: Communicating my results by answering speciﬁc questions is im-
portant A: It helps me keep track of what I’m doing but sometimes the
organization demanded in class isn’t realistic for real life.

Q: Letting each group member have hands-on time with the equipment
is important. A: It can help everyone and if they need to use it in the
future.

Q: Doing calculations with data is important. A: data collection is
your main source of information in an experiment and you should use
your personal data to complete calculations.

1. Connected to a prac-
tice
Analyzing data with a
focus on uncertainty

Analysis
Contains a value

2.
statement
I do understand its im-
portance, and see it as
a necessary thing.

3. Has an explanation or justiﬁcation
for value statement
personally will never need to take un-
certainty as seriously and don’t look at
it the same way as someone who plans
a future in experimental physics.

Executing an experi-
ment

‘I really enjoy it, I enjoy
working on something

Analyzing data through
calculations

–

–

–

Analyzing data by look-
ing for trends

it is not so important.

small part of what I am studying. . . in
my overall career.

Communicating results

it helps me...

organization demanded...not realistic

Working in a group

It can help everyone.

if they need to use it in the future.

Analyzing data

–

–

53

The ﬁrst example discussed is Ex 1 from the “What makes an identity statement” table (Ta-
ble 4.1). Here Blake is talking about the practice of analyzing data with a focus on uncertainty. He
mentions the importance and necessity of the practice, which are both statements about the value
of the practice. Blake then qualiﬁes the value statement by saying that due to his future outside of
experimental physics he will not need to use uncertainty and therefore he does not ﬁnd it personally
valuable.

In Ex 2, Adam is describing the process of executing an experiment. He highlights the value
of that practice by describing his preference for it with the use of “enjoy.” The rest of the quote,
provides many examples of positive aﬀect. It could be argued that tying in agency in the mention
of possession and accomplishment implies identity but agency does not explicitly ﬁt into the model
of identity being tested, although there is work that suggests a connection [86, 87]. In addition,
we are looking for self-contained identity statements, in order to make claims about the connection
between practices and identity. At this stage this example from Adam does not qualify as identity
within the established criteria.

Ex 3 is Beth describing analyzing data through calculations in her physics lab. She provides
examples of positive aﬀect, ‘more self-satisfying,’ that in and of itself is not an identity statement.
It could be qualiﬁed as a value statement if Beth were to include an explanation or justiﬁcation for
why she values things that are self-satisfying. Or, using another example from her, she could also
have made a similar qualiﬁcation to the value of self-reliance. As with Adam, this example from
Beth does not meet the requirements of an identity statement as established thus far.

This is not to say that these examples from Beth and Adam are not indicative of identity, they
likely are. It is also not the responsibility of the student to independently explain their reasoning
and provide us with perfect, self-contained examples of identity statements. But, in the process of
this work, we are only looking for self-contained statements in order to develop a codebook to test
the combined identity framework and to highlight the connection between practices and identity.
All three quotes are evidence of a connection between experience with lab practices and diﬀerent
measures of aﬀect (e.g., identity, self-reliance, and agency).

54

The survey questions were designed based on the initial criteria for an identity statement, where
the closed response questions contained the connection to the practice and the value statement
(see Fig. 4.1). The open-response portion was where students explained their reasoning for their
answer to the previous question with the goal of producing further examples of identity to solidify
the criteria. Responses to survey questions are assumed to be personal unless they explicitly state
otherwise (e.g. ‘This is important for everyone’). This is because the open-response questions are
already personalized within the question statement, ‘For ME this is (not) important because...’

Breaking down Ex 4, the practice is analyzing data by looking for trends, which is included
in the question statement and the response. The value statement could be taken from the question
but the student provides it in their response, when they say it is not important. Then they justify
their value statement by connecting to their future career. In Ex 5 the practice is communicating
results. They establish value by mentioning how it helps them. Then they qualify that adding that
the organization required by the course is unrealistic. Both of these are examples of identity, they
are connected to a practice, contain a personal value statement, which is justiﬁed.

In Ex 6, the practice is working in a group, which is pulled from the question statement. There is
a value statement that is not personalized, describing how it is valuable for everyone. Additionally,
the justiﬁcation is not personal to the student. This example is not an identity statement because
it is lacking personalization in both the discussion of the value and the justiﬁcation. In Ex 7 the
practice is analyzing data, they also mention data collection in executing experiments. This is
simply a description of the practice, it does not contain any personal connection to the practice or
any further justiﬁcation.

The above seven student quotes are used to highlight the determination of the requirements for
an identity statement in this work on labs, practices, and identity. Each example of an identity
statement discussed has a connection to a speciﬁc lab practice as aligned with the goal of connecting
experiences with lab practices to physics identity. After initial analysis of the open-response
questions for identity, and an additional examination the potential statements from the interviews was
done. Through this semi-iterative process, the operationalized deﬁnition of an identity statement

55

was ﬁnally articulated. 1. It must discuss a practice (From our goals and in connection to the
context). 2. It must contain a personalized value statement (From the literature and from expert
consultation). 3. It must include an explanation or justiﬁcation for the value statement (From the
analysis of interviews and open-responses as well as from expert consultation).

Once the criteria for an identity statement was established we worked to connect and reduce the

emergent codes in parallel to the work done by Close et al.

56

Table 4.2: Examples of LT responses from the data. LT1 is from the interviews and LT2-4 are from the open-response survey.

Learning Trajectory (LT) statements are deﬁned as when

the value they put on the practice is connected to their future or their planned career

Label Response

Data

LT 1

LT 2

LT 3

LT 4

Blake Yeah, So I think just based on wanting to be
a teacher I see that as more valuable than someone
who maybe just wants to understand physics for
themselves because if you just want to understand
physics for yourself you don’t really care about
you being able to communicate results to others, ...
whereas I aspire to ... allow my understanding to
feed understanding of others and to help develop
their own... and so I think it’s it’s even more im-
portant to me to be able to communicate my results
than others.
Q: Determining patterns from my data is important
A: it is not so important to look for trends in data
because overall it is only a small part of what I am
studying for my degree and will not necessarily
help me in my overall career.

Q Working with the equipment is valuable A: It
helps build conﬁdence in doing hands-on things
somewhat, but I don’t think working with this spe-
ciﬁc equipment is really helpful for me in the fu-
ture.

Q: Working with the equipment is valuable A: I
may never see this equipment again, and if I were
to go to a lab the equipment and how it is used
may be very diﬀerent.
learning the speciﬁcs of
the equipment used may not be helpful, but it is
valuable to understand what it does and why you
would use it for the study.

1. Connected to a prac-
tice

communicating, specif-
ically
communicating
results

Analysis

3. Has an explanation
or justiﬁcation for value
statement
based on wanting to be
a teacher.

Contains a value

2.
statement

I see that as more valu-
able . . .
I think it’s it’s
even more important to
me.

Explanation

he ﬁnds this practice
valuable because of the
goals he has for his fu-
ture as a teacher.

Analyzing data by de-
termining patterns from
data

not so important

small part ... not nec-
essarily help with my
overall career

This connects to their
future with the discus-
sion of their career

Executing experiments,
speciﬁcally
working
with the equipment

I don’t think ... is really
helpful for me

in the future

They justify their value
statement based on their
future

Executing experiments,
speciﬁcally
working
with the equipment

may never see ... again
and may not be helpful

I may never see this ...
again

This statement of the
practice not necessarily
being relevant for them
going forward is what
makes it an example of
LT.

57

Table 4.3: Examples of Interest statements from the data. Int 1 is from the interviews and Int 2-3 are from the open-response survey

They are either statements that speciﬁcally mention interest or enjoyment of something or they show a positive or negative preference

Interest statements are an indication of personal interest or preference for something.

Label Response

Data

1. Connected to
a practice

Analysis

2. Contains a value state-
ment
3. Has an explanation or
justiﬁcation for value state-
ment
what makes science enjoy-
able to me

not
physics...could care less

really

into

Explanation

The value she sees in this practice comes
from the enjoyment she gets out of it,
which connects to the idea of preference
in statements of interest.

This is an example of negative preference

Fern: especially if I don’t understand what the results mean. Cause
that’s the part where I’m like, this makes total sense, this is really cool
now, I can see this in my normal life. Connecting the dots is what
makes science enjoyable to me

Communicating
results

Q: Presenting what my results mean is important A: I am not really
into physics so I could care less if I present the data that I ﬁnd because
most the time I don’t even understand it.

Communicating
results

Int 1

Int 2

Int 3

Q: Presenting what my results mean is important A: I don’t love
physics, so I don’t love presenting the work, but it is important to
receive peer feedback.

Communicating
results

I don’t love physics...I don’t
love presenting work

Two negative preference statements

58

Table 4.4: Examples of NE from the data. NE 1 is from the interviews and NE 2-5 are from the open-response survey.

and values of the experimental physics community and negotiating how those values and expectations align (or not) with one’s own.

Negotiated Experience (NE) statements are about dealing with (or adapting to) the expectations

Label Response

Data

1. Connected to a prac-
tice

Analysis

Contains a value

2.
statement

Executing experiments

it is important...I value

3. Has an explanation
or justiﬁcation for value
statement
it is important in that I
need to do it I value both
of those ... because the
class values them

Understanding
equip-
ment, adapting it to new
situations

not as important might
be useful

I don’t think it is neces-
sary

Communicating results

maybe somewhat help-
ful

inconsequential ... not
the most important part

Analyzing data by inter-
preting graphs

not important

I feel obligated

NE 1

Blake: The process of executing the procedure
and then like trying ... It’s important in that I need
to do it.
I think because this class is so much
more based on the data, I think the data collection
takes precedent but I think also there are questions
within the lab that do drawback to those exploring
observations. So I think I value both of those. But
mostly because the class values them.

NE 2 Q: Knowing how to adapt equipment to new situ-
ations is useful A: As a student I feel like this is
not as important because most experiments I will
do in a lab class will be controlled. This might be
useful for the future but as of now I don’t think it
is necessary.

NE 3 Q: Presenting what my results mean is important
A: Typically, all of the results from DATA Lab ex-
periments are inconsequential in a larger physics
picture. Maybe it is somewhat helpful in under-
standing the lab, but it is not the most important
part of an entire process for students.

NE 4 Q: Interpreting graphs to understand relationships
between parameters is important A: Not important
because I don’t like graphs and I feel obligated to
make one even when I don’t understand the con-
cepts very well.

Explanation

The value that Blake
places on this practice
is based on the require-
ments of
the course.
The intrinsic value he
sees only goes as far
as his connection to the
course.

‘inconsequential’ indi-
cates tension,
‘maybe
somewhat helpful’ in-
dicates value vs seeing
value

They speciﬁcally de-
scribe feeling obligated

NE 5 Q: Communicating my results by answering spe-
ciﬁc questions is helpful A: It helps me keep track
of what I’m doing but sometimes the organization
demanded in class isn’t realistic for real life.

Communicating results

It helps me

organization demanded
... isn’t realistic

in

Obligation
‘de-
manded’ and tension in
‘not realistic’

59

Table 4.5: Example of a CM statement from the data. This example is from the open-response survey.

Community Membership (CM) statements are where the value of the practice comes from the community.

This can look like a distinction between communities or the value of something in one community versus another.

2. Contains a value
statement

Pulled from the Lik-
ert response

Analysis

3. Has an explana-
tion or justiﬁcation
for value statement
true in any science

Explanation

Implies that
talking about
ence community

they are
the sci-

Label

Response

Data

CM 1 Q: Explaining how I did what I did is important A: Be-
cause when performing an experiment of any sort you
need to be detailed in order to be able to recreate the
experiment exactly and reproduce the results. Or con-
versely to be able to go through and alter the procedure
and check for ﬂaws that could prohibit the results you hy-
pothesize. This is true in any science and good practice
even in physics.

1. Connected to a
practice

Communicating re-
sults by explaining
what I did

60

Table 4.6: Unclassiﬁed examples. All six comes from the open-response survey.

Unclassiﬁed Examples these are exmaples of identity that did not ﬁt into the identity components

1. Connected to a
practice

Communicating re-
sults

Not as helpful

Analysis

2. Contains a value
statement

3. Has an explana-
tion or justiﬁcation for
value statement
I am not a professional

Working with
group

a

Pulled from Likert
response

I am not a profes-
sional

Data
Label Response

Indicators

Not an expert - Something
is not valuable because they
are not experts or not
physicists

Un 1

Un 2

Q: Communicating my results by answering speciﬁc
questions is helpful A: Not as helpful because I am not a
professional and don’t necessarily understand everything
that the lab had to oﬀer, just the major concepts.

Q: Having help from my group if I have questions is
important A: I am not a professional and need a lot of
help when conducting experiments so having others to
lean on is helpful.

Purpose - This is important
because it shows me the pur-
pose of this

Un 3

Q: Communicating my results by answering speciﬁc
questions is helpful A: Explaining ﬁndings is the whole
purpose of doing an experiment

Communicating re-
sults

Pulled from Likert
response

whole purpose of do-
ing an experiment

Relevance - The practice or
activity is relevant (or not)
in some way

Un 4

Q: Working with the equipment is valuable A: Is not
really important because I don’t do many experiments or
measurements in my regular life.

Un 5

Q: Making connections to the bigger concepts is valu-
able A: it helps me to put forth more eﬀort if I can
understand the concept in a real life example.

Utility - Helps (or doesn’t)
to
learn/understand/do
something

Un 6

Q: Using math as a tool to explain and predict is helpful
A: As an undergraduate student, sometimes the underly-
ing concepts behind experiments go over my head. For
me, it’s sometimes most useful for me to relate a concept
to an equation and understand how diﬀerent variables in
the equation relate to one another, rather than more com-
plicated physical variables in an experiment.

Executing experi-
ments, speciﬁcally
working with the
equipment

Analyzing
data,
speciﬁcally making
connections
to
concepts

Is not really impor-
tant

I don’t do [this] in my
regular life

it helps me..

real life example

Analyzing
speciﬁcally
using math

data,
by

Pulled from Likert
response

most useful ... and un-
derstand

61

4.4.2

In the Data

This section highlights the places where the data connect with the model of identity. We use
examples from the interviews and the open-response survey to articulate the deﬁnition of the
diﬀerent components within the labs context.

4.4.2.1 Learning Trajectory (LT)

This work has been focused on students who are not physics majors but are STEM majors. The
practices that are important in physics labs are relevant to their current or projected ﬁeld. For
students who do not plan on a career in physics the ability to see how physics practices may connect
to their future goals is vital for them in identifying with physics. In LT 1 (Table 4.2) Blake says
that he ﬁnds the practice of communicating in the course valuable because it is something he will
need in his future as a teacher. A common theme in the open-response survey data is students
diminishing the value of a practice because they will not need it in the future. Examples LT 2-4 all
show this feature saying that the practice does not connect to their future, will not be helpful for
them, and will not be something they need to know after this physics lab. It can be posited that the
reason so many statements fell into this component is because the open-response data came from
non-majors, who use their future goals as the concrete connection for why they might identify with
some aspects of physics. It would not be surprising if this trend was not seen in physics majors,
where the connection between physics and their future is more clear.

4.4.2.2

Interest

An explicit interest in a community, or an aspect of that community, can be an indication of whether
or not someone personally identiﬁes with that community. In Int 1 (Table 4.3) Fern expresses that
the process of communicating results and “connecting the dots” is what she enjoys about doing
science, true in both her ﬁeld of biology and in physics. In Int 2 and 3 the students both express a
negative preference for physics, “I don’t love physics” and “I am not really into physics.” In all three

62

cases the preference statements, whether positive or negative, are used to explain their feelings
about a particular physics practice.
It is not surprising that many students mentioned negative
feelings and preferences toward physics, they are not physics majors for one thing, but the more
the students’ interests align with important physics practices the more likely the experience with
the practice will positively impact their identity. Int 2 and 3 would look very diﬀerent if after the
mention of their disinterest in physics students described a speciﬁc interest in a physics practice.

4.4.2.3 Negotiated Experience (NE)

A fundamental part of identifying with a community is the negotiation of how one ﬁts into that
community. This includes how the values and practices of the community align with or are in
opposition to those that they already identify with. This negotiation can have a strong inﬂuence
on how or if one comes to identify as a member of the community. This has played out in three
diﬀerent ways in the data. One way is through an expressed tension in the value the individual
puts on something versus the community. This can be seen in NE 3 (Table 4.4), where the student
describes the practice as “inconsequential” and “not the most important part.” Another way NE is
represented in the data is the distinction between personally valuing something and seeing the value
of it. In NE 1 and 2 this is evidenced by the fact that the student recognizes the value of the practice
for the course but it is not something they personally value. Along the same lines as the other two
show statements that are reﬂective of obligation, “I only care about this because I have to.” Several
of the NE examples show this: NE 5 with “organization demanded by the course” and NE 1 with
“important in that I need to do it.” These are similar to valuing versus seeing value, where the
student only cares about the practice because they are made to by the features or requirements of the
course. NE statements are some of the most common in the data: it is possible that the reason for
this partially falls on the fact that these students are non-physics STEM majors. As such, they have
opinions about the practices valued in physics since they often have experience with similar or the
same things in the courses they take in their major. When coming into their physics labs students
have to address their past experiences and perspectives in negotiation with how things are done in

63

the physics community, or at least their interpretation of that. Nearly all examples of NE statements
are negative in part or whole, which is likely because these diﬀerences in practices and values pop
up as internal conﬂicts that the students have to deal with in order to understand themselves and
their identities with respect to physics.

4.4.2.4 Community Membership (CM)

Part of that process of becoming a part of a community (and developing a community-speciﬁc
identity) means an alignment of the practices valued by the individual and by the community.
In order to feel a part of the community the individual needs to value and identify with (some
of) the same practices that the community does. This can be seen as the value of the practice
coming speciﬁcally from a community or a diﬀerence in value between two communities. In CM
1 (Table 4.5), they talk about the value of the practice coming from the fact that it is important in
all of science. Although there are not many examples of CM in the data, it is not hard to believe
that the distinctions between communities would come into focus during the process of developing
a subject-speciﬁc identity. CM could even be seen as a step that follows NE: that students need
to confront conﬂicts produced by straddling diﬀerent communities before they can form their own
opinions enough to evaluate diﬀerent perspectives. It’s possible that the students in are sample are
not at a stage where they can look beyond the conﬂict, and as such there are very few examples of
CM and many examples of NE.

4.4.2.5 Unclassiﬁed

In addition to the identity statements that align with the operationalized deﬁnitions, there are also
examples that do not ﬁt into any of the pre-established bins. These are examples of identity based
on the requirements for an identity statement they are focused on a practice, include a personal
value statement, and are justiﬁed but that justiﬁcation does not ﬁt with the identity components.
Examples of these statements are listed in Table 6. The ﬁrst two examples, Un 1 and 2 (Table 4.6),
can be described as references to the fact that the students are not experts and that is the reason they

64

do not care about the speciﬁc practice. This is diﬀerent than LT, where the students are projecting
to their perceived future. Un 3 establishes that the practice is valuable because it is the purpose of
doing experiments. This could be interpreted as a CM statement but lacks further connection to a
speciﬁc community. The relevance of the practice to real life is used as justiﬁcation in Un 4 and
5, which does not match up with any of the identity components. The ﬁnal example is one that
discusses the utility of the practice. Utility could connect to the identity model but in this example
there is no connection to the future (LT), no preference (interest), there is no focus on conﬂict (NE),
and it is not directly connected to a community (CM). Of all the identity statements, 20% were
unclassiﬁed, for the reasons discussed above.

4.4.3 Not in the Data

4.4.3.1 Recognition

Recognition was not discussed by the students in the interviews or in the open-response survey. It
could be predicted where statements of recognition might show up, recognition as a member of the
community from peers or instructors. Grades could be perceived as a form of recognition. Although
attempts were made in the interviews to probe for connections to recognition, it did not come out.
In the interviews for example, there were only examples of personal recognition, “I was able to
explain this to someone else, so I must really understand it.” One reason this might not have come
up in the data is that recognition is not closely tied with practices and related experiences, therefore
the focus on practices artiﬁcially suppressed examples of recognition. It could also be that in these
lab courses recognition from peers is not of the greatest importance to the students. The courses
where the majority of the student data comes from do not have any special emphasis on recognition
either by instructors or peers, while they do explicitly focus on physics practices and the values
of them (Ch 3). A last possibility is that impactful recognition comes not from large enrollment
courses but instead through departmental or cohort interactions. Since these interactions were not
addressed in interviews or the open-response survey they are not detected in the data.

65

4.4.3.2 Competence

Competence is the belief in one’s ability to perform speciﬁc practices relevant to the community.
It is not hard to establish a direct connection between a person’s competence in a speciﬁc practice
and its impact on their identity. Therefore, it was surprising that statements of competence in
relation to identity were not detected in the data. Competence was directly probed in some of the
interviews by asking if the students felt that they achieved their goals in levels of understanding
or skill with respect to a speciﬁc practice. Even with that direct line of questioning no examples
of identity related competence were found. Although it is possible to predict potential statements,
“this practice is important for physics but I am not good at it so...” it did not come up in the process
of this work.

4.4.3.3 Not Combined

One of the goals of this work was to adapt a speciﬁc identity framework to a diﬀerent context.
The components of the framework were each a combination of two perspectives on identity, COP
and the work done by Hazari et al.
[4, 19]. Of the examples found in interviews and the
open-response survey, none of them reﬂected the combination. During the initial coding of the
statements, there was no evidence of the framework components being combined as they are in
the Close framework. First of all, competence and recognition were combined with community
membership and negotiated experience respectively and no instances of competence or recognition
were detected in the data at all. Although it is possible to see how they could work as combined
components in this context. Community membership and competence for example, a student’s
identity could be impacted by whether or not they are competent in the skills and practices valued
in the labs community. Even examples of recognition combined with community membership
and competence are possible, the student’s competence in the valued practices is recognized by
their peers or instructor. An additional interpretation that combines recognition with negotiated
experience is that the student needs to ﬁgure out how recognition is utilized and given in this
community just as they have to ﬁgure out the practices of the community and integrate them with

66

their other community-speciﬁc identities. In the Close et al. framework, interest was combined
with learning trajectory and although there is evidence of both components there is not evidence
of them in combination with each other. An hypothesized example of this could be an adjustment
to LT 1, where it becomes “...makes science enjoyable to me and is why I will make science my
career.” None of these proposed examples were evident in the data. These outcomes lead to the
separation of the identity components as compared to Close et al.. The statements were coded from
there and another researcher was included to validate the coding of the identity statements.

4.4.4 Summary

In the analysis of both the interviews and the open-response survey the combined framework [21]
could not be applied. There was no evidence of any of the components being combined and when
broken into six distinct components evidence was found for only four of them (Fig. 4.3). Table 4.7
shows counts of the numbers of statements from the open-response survey that were coded into
each speciﬁc identity bin.

Table 4.7: The number of statements from students coded into each of the identity components.

Component
LT
Interest
NE
CM
Unclassiﬁed

Statements
38
19
42
15
24

4.5 Discussion

This paper has described the work done to characterize identity development in physics classes,
with a focus on lab practices. This was done in alignment with the operationalized model from
Close et al. [21]. Multiple examples from the literature, interviews, and an open-response survey
were provided as justiﬁcation for connecting labs to aﬀective measures and lab practices to identity
development. These examples will be brieﬂy summarized and discussed in this section.

67

Figure 4.3: A visualization of the initial model of identity from Close et al. 2016 and the revised
model based on the data and analysis presented here. The overlap in the intial model is meant to
illustrate the combination of components.

4.5.1 Labs and Aﬀect

Both the literature on labs [17, 15, 82, 16] and on physics identity [21, 54, 53, 56] show clear
connections between engagement with practices and changes to attitudes and other aﬀective mea-
sures. Similar impacts of practices can be seen in statements LT 1 and Int 1, where both express
positive views of two physics practices. Even Ex 2, which is not an identity statement based on the
criteria, still shows the positive attitudinal impact lab practices can have with Adam talking about
the “sense of accomplishment.” These results connecting practices to attitudes and aﬀect also come
with limitations, for one, not all lab courses center authentic skills and practices. Those that do still
require that the practices are implemented authentically and valued within the context of the course,
as evidenced in grading and learning goals (Ch 3). Large scale changes can also depend on whether
or not positive aﬀect and attitude were goals of the course design and implementation. Even with
these limitations, the connection between authentic engagement in physics practices and positive

68

attitudinal and aﬀective changes, including physics identity, should provide signiﬁcant evidence for
the value of these experiences for our students. The skills and practices of physics are relevant to all
science students, physics and non-physics majors alike. In many physics departments, the majority
of students served are non-physics STEM majors, so it should be ensured that the courses serving
the majority of the students going through our departments are eﬀective in the goals relevant to
these students.

4.5.2 Practices and Identity

Wenger [4], Irving [53, 54, 56] and Close [21] all indicate the connections between identity
development and the valued practices within a speciﬁc community. This connection is additionally
evident in all of the examples of identity statements discussed in this paper, every example provided
highlights this connection. Though this again comes with limitations, this work was done with a
communities of practice perspective and community of interest has been tentatively deﬁned as the
labs class community as a proxy for the larger physics community. This requires that the students
see their lab courses as an authentic stand-in for the physics community, Int 3 and 4 show evidence
that they do. Either way, if identity development, and the beneﬁts connected to that, are goals in
the course instruction than that implies that it is necessary to ensure that there are opportunities
for students to authentically engage in physics practices in order to aid in the development of
their physics identities. This paper has described the eﬀort to understand identity development
in lab classes in connection with physics practices and as part of that the work to operationalize
the framework used in Close [21] in the context of laboratory practices. In that framework the
components were combined from two diﬀerent perspectives on identity [4, 19]. This combination of
components was not reﬂected in our data from practice-focused lab environments, instead evidence
was found for four of the six individual components. Learning Trajectory highlighted the ways in
which students interpret the relevance and value of speciﬁc practices based on their projected future
plans. Interest showed statements of positive or negative preferences to diﬀerent lab practices, or
aspects of practices. Negotiated Experience covered the multitude of ways students must adjust

69

and adapt to the values, expectations, and practices of one community as compared to the other
communities they are members of. Community Membership provided examples where the speciﬁc
value a student has for a practice originates from their membership in a speciﬁc community. There
was no evidence of either competence or recognition, although it is possible to predict how they
could ﬁt into the context of physics labs. Recognition could have been represented in relation to
the grades received, or as recognition as a member of the community from peers or instructors. It
is easy to see how competence could ﬁt with a focus on practices, whether or not the student is
competent in speciﬁc physics practices could impact their identity. Although we found no evidence
of the components being combined we again can predict what that may have looked like. Learning
trajectory and interest could be represented as a positive preference for a practice that aligns with a
projected future path. Negotiated experience and recognition could highlight the social aspects of
adapting to a new community and the recognition from peers as an equal member in that process.
Community membership and competence might be examples of the student’s perceived competence
in practices that are valued in the physics or lab community. It is possible to hypothesize what
these may look like in this context but again, with the analysis of the described data sources, no
evidence was found. The analysis has shown that it is possible to use communities of practice
to study identity development in lab classes but not in the combined way that Close does. This
requires the assumption that lab classes represent a COP, as argued in Irving (2014) [57]. The
students also need to see the connection between their course and the larger physics community (Int
3 and 4). It has been found that COP ﬁts the student responses to both interview questions and to
the open-response survey. The open-response survey only went to intro non-physics STEM majors,
which suggests that this application of COP to identity development is conﬁrmed for these students
speciﬁcally. From this though, it can also be extrapolated that the same can be said for intro and
advanced physics majors, although evidence for this is not provided here, it will be highlighted in
an upcoming publication. Although COP ﬁts the context, the same combination of components
that Close [21] describes does not appear to, as there is no evidence of any of the six components
as combined with each other in either the interviews or open-response survey results. This may

70

be tied to the population, students within a lab course versus ULAs teaching other students. It
is possible that the process of identity development of the ULAs, who have a unique relationship
to physics and the physics community, is more complex than that of our students and therefore
results in these intertwined components. Building on the diﬀerence between students and ULAs,
the context of a classroom community, for the students, and the multilayered ULA and physics
teaching communities for the ULAs may also be responsible for the evidence (or lack) of combined
components. We do not believe Close was wrong to combine the components, as evidenced by
the results of their work, but our study implies that the combined components are not necessarily
broadly applicable. More work with this framework on identity, either in a fully qualitative or in a
mixed methods approach such as this, may show something entirely diﬀerent. Either way, at this
stage it should not be assumed that this framework would apply to any context without preliminary
work and potential adjustments.

71

CHAPTER 5

WHAT COUNTS IN LABORATORIES: TOWARD A PRACTICE-BASED IDENTITY

SURVEY

The following chapter was published in the proceedings of the 2018 Physics Education Research
Conference [88] with minor modiﬁcations from its appearance in publication. It was published
with co-authors Marcos D. Caballero, Paul W. Irving, and Vashti Sawtelle.

Some content in this chapter may be repeated in Chapter 6 in order to ensure the full survey

development story is told in Chapter 6.

5.1 Introduction

Laboratory courses in the undergraduate physics curriculum are intended to provide students
with opportunities to participate in authentic physics practices. There have been national calls to
develop courses that emphasize physics practices and focus on how science is done [61, 12, 5].
The American Association of Physics Teachers (AAPT) Committee on Laboratories released lab
guidelines, with focus areas that highlight the skills and practices necessary in doing physics [5].
The emphasis on practices in these lab courses makes them an ideal place to investigate the
impact the practices are having on our students. The opportunity to engage with authentic physics
practices is an essential step in the development of a physics identity [3]. We are working to
develop a survey to measure students’ physics identity in lab courses with a focus on the eﬀect of
lab practices. Identity is a construct that is nuanced in qualitative studies, so a survey designed to
probe identity must be tightly connected to a theoretical framework. In this paper we show how
we build on existing survey design methods by staying close to student understanding of physics
practices and add an intentional focus on the identity framework in every step in our survey design.
We show how asking students to reﬂect on scientiﬁc practices and values are critical steps in
designing a survey on practice-based identity.

72

5.2 Theoretical Background

Identity: Identity is a multidimensional construct that describes how an individual sees them-
selves and how they situate themselves with respect to a speciﬁc community. We are adapting
the identity framework presented in [3], which draws from Communities of Practice (CoP) [4]
and [19] frameworks. The framework has three main components: community membership &
competence (CMC), learning trajectory & interest (LTI), and negotiated experience & recognition
(NER) (Fig. 5.1). Due to limits in space we will break down these components further in future
publications. For the purposes of this paper we are adapting these components to our context with
the knowledge that they may not be what best describes identity in physics labs.

The context of physics labs and our goals of staying close to the theory and to the students’
ideas align to pull the majority of our focus onto the speciﬁc lab practices, how students interpret
them and how they identify with respect to those practices. This means examining the overlap
of the experimental physics community with the other communities the students are a part of,
and speciﬁcally, understanding how they interpret physics practices through the lens of those
communities.

Our study of identity is further complicated by the fact that we are working to produce a survey
that can be distributed on a large scale with closed response questions that combine to tell us about
the participant’s physics identity. This has meant that we have a carefully operationalized deﬁnition
for what we count as an identity statement. We are looking for three main components of an identity
statement: the student addresses the practice, applies a value statement to it, and then connects
it back to one of the identity components in Fig. 5.1. The deﬁnition here relies on the student’s
interpretation of the physics practice and a personal value statement about the practice in order to
meet the ﬁrst two criteria. The third component helps us to maintain our ties to the theory. We
recognize that this process of operationalizing a complicated construct into a single sentence loses
much of the nuance. Below, we argue that the intentional and focused process we are going through
to develop question statements respects the complexity of identity as a construct.

Physics Lab Practices: The CoP framework on identity deﬁnes a CoP based upon the shared

73

Figure 5.1: The components of our theoretical framework, with the interaction between the physics
lab practices and the components of the identity framework forming the physics identity [3, 4, 5].

practices, skills, and ideas within a speciﬁc community [4]. Physics lab practices are the skills
and activities valued and utilized within the experimental physics community. We focus our initial
survey design on practices from the AAPT lab guidelines and those emphasized in the physics lab
courses at our institution. These initial practices were adapted based on responses in the student
interviews, which are described in Sec. 5.3.1. We have thought carefully about how to represent the
practices in a way that they might resonate with even if they are not intending to become physicists.
The ﬁnal six broad lab practices on which we focus include: analyzing data, communicating,
working in a group, understanding equipment, designing experiments, and executing experiments
(Fig. 5.1). Starting with broader deﬁnitions we have iterated on this list based on student interviews
where we took in their interpretations. We acknowledge that our list of practices is limited and
may not be consistent across institutions, for example our lab courses do not currently emphasize
computation, while other institutions do [89]. We hope to expand our data collection to additional
institutions in order to account for this.
In this paper we also do not discuss the centrality of
particular practices as the CoP framework would call for, though we have structured our data
collection to allow for this analysis in the future.

74

5.3 Survey Development

We have been keeping the interpretations of the students and the components of the theoretical
framework at the center of our entire survey design process (as laid out in Fig. 5.2).
In the
interviews, we worked to ﬂesh out their understandings of the lab practices that are valued in the
physics community and to evaluate the ways they identify with those practices. The creation of the
survey has also centered on the idea that though students may identify with physics many of them
would not now, or maybe ever, identify themselves as physicists. This fact means that we need to
carefully consider the intersection of students’ communities with the physics community embodied
in the laboratory classroom. Therefore, in initial iterations of this survey we take into account
the many communities that these students are members of: their major, the science community,
sports/teams, racial/ethnicity groups etc. We acknowledge that students from life sciences are not
necessarily going to see physics practices the same way that a graduating physics major might, but
we assert that they both identify with physics in some way. To account for diﬀerent interpretations
and for future exploration of the centrality of these diﬀerent practices for students we have collected
data from introductory to upper division labs.

5.3.1

Interviews

Traditionally surveys are designed around “expert” ideas, but our students are not yet experts
[90, 74]. To make sure that our questions are representative of their ideas we need to understand
all the ways students are interpreting the practices. For example, one student’s interpretation of
“analyzing data” might diﬀer signiﬁcantly from another’s, which means we have to be speciﬁc
about the language and interpretations we are using in the survey design. To design a survey to be
applicable across a variety of students, we conducted semi-structured interviews with nine students
from an introductory algebra-based physics lab for non-majors and six students in upper-division
physics labs.

In the interviews we asked participants to deﬁne each of the lab practices and to describe what

75

Figure 5.2: The survey development process. The theoretical framework as described in Sec. II.

the practices looked like in their physics lab course (e.g., "What does data analysis mean in your
lab class?"). Then they were asked whether or not they found that practice valuable (e.g., "Do you
ﬁnd that useful?"). This enabled us to ﬁrst, explore a variety of interpretations of speciﬁc practices
and second, to explore how students may (or may not) identify with that practice.

We used the interviews to map the breadth of students’ interpretations of the practices, not
to make claims about how speciﬁc students understand the practices. The analysis focused on
covering all interpretations and levels of centrality in the students’ statements. Within each practice
we found interpretations fell into speciﬁc themes, which we refer to as sub-bins (Fig. 5.3). From
those sub-bins, we identiﬁed diﬀerent types of student responses with diﬀerent levels of centrality
– the spectrum of responses. We then turned those responses into question stems for the pilot

76

Figure 5.3: This is an example of the interview analysis process for the practice of analyzing data.

survey. For example, from the interview question: What does data analysis look like in your lab?,
we looked at student responses that fell into a similar theme, graphing in this case.

As soon as you have the graph and data in front of you, with a certain mindset, [you]
can interpret what it actually means. (Dean, pseudonym)

So interpreting the results, in this class, we use graphs and stuﬀ to do that. (Fern)

Looking across responses that fell into similar categories, we wrote pilot question statements
that would represent student interpretations of the practices that they could endorse in closed
response questions. For example, the above student quotes combine into the question stem for the
pilot survey: In my lab, using graphs to interpret results is important. An overview of the practice
space and the pilot survey questions produced can be seen in Table. 5.1.

In the analysis we also evaluated the ways students described how they identiﬁed with speciﬁc
practices. This lead us to our deﬁnition of an identity statement as discussed in Sec. 5.2. To

77

Table 5.1: Overview of Practice Space.

Practice
Analyzing Data
Communicating
Working in a Group
Understanding Equipment
Planning Experiments
Executing Experiments
Total

Sub-bins Pilot Survey Questions

7
3
9
7
3
2
31

19
18
25
25
14
7
108

highlight the steps we went through to form our deﬁnition we compare two statements where
students identify with lab practices.

Blake(discussing uncertainty): I do understand its importance, and see it as a necessary
thing. It’s just, I personally will never really need to take uncertainty as seriously as it
is taken in these upper level lab classes and...I don’t look at it the same way as someone
who’s thinking ‘I want to do experimental physics for the rest of my life’.

We see that Blake refers to a speciﬁc practice, uncertainty in analyzing data (which he deﬁnes

earlier in the interview). He makes a personal value statement about it:
I do understand its
importance, and see it as a necessary thing. He then connects it back to the identity framework:
I personally will never really need to take uncertainty as seriously. We see this statement ﬁtting
into the LTI component of the identity framework (Fig. 5.1) because Blake is projecting forward
to his future career (as a high school physics or math teacher), where he does not think he will
need to use uncertainty. These components of Blake’s statement address our main goals in the
survey development process, covering his interpretation of the practice, the way he does (or does
not) identify with it, and then connecting it back to our theoretical framework. We contrast this
example with one that does not make the connection back to the theory.

Beth (describing something she found exciting): It was good to, plug it all into the
calculator and [see] this is a reasonable answer. So, getting something that...could be

78

right and doing that without a procedure is more self-satisfying because...you relied
more on yourself.

Beth, too, refers to a practice, broadly analyzing data, and she applies a value statement to
it: more self-satisfying because...you relied more on yourself. But, she does not make the value
statement in a way that easily connects to the framework to mark it as an identity statement. If
she had talked about how important self-reliance is either for her future (LTI) or as an important
part of a community she is a part of (CMC), then we could classify it as an identity statement.
This comparison between statements from Blake and Beth exempliﬁes how we came to include the
third requirement to our identity statement deﬁnition, the students’ statement connects back to an
identity component in (Fig. 5.1).

The analysis of the interview data supported covering the breadth of students’ interpretations of
the six lab practices (Table 5.1). Working from this analysis, we built questions for the pilot survey
that explicitly asked students to reﬂect on the practice and the value statement, but left open how
participants’ statements connect the practice back to the identity framework.

5.3.2 Pilot Survey

The purpose of the pilot survey was not statistical validation that is traditionally part of a survey
development process, we were not yet at a point where such validation would be necessary or
useful. The overarching goal instead was to cover the ways that students identify with the physics
practices, similarly to how the interviews were used to determine how students were interpreting
the practices. This connects well with our intentions in the design process, we want to be certain of
our students’ ideas and interpretations, and we need more information to create questions to probe
identity reliably. In addition to the identity information we are using the pilot survey to conﬁrm
full coverage of the practice space.

We worked to elicit identity statements that met our criteria and we primed participants to think
about speciﬁc communities throughout the survey. We built the ﬁrst two components of the criteria
directly into the prompt, addressing a practice and applying a value statement to it. We then broke

79

Figure 5.4: An example question from the pilot survey.

each question statement into four pieces. The ﬁrst two were Likert-scale closed response items,
which gave the participants space to endorse (or not) the value statements. The other two were free
response questions where participants were asked to explain their reasoning, the aim being that
they would make that ﬁnal connection to the identity framework.

In each question group we asked the participants for both their own perspective and what they
thought an experimental physicist might say (see Fig. 5.4 for an example), following the strategy
from the Colorado Learning About Science Survey in Experimental Physics (ECLASS) [91]. We
did this as a way to prime them to think about speciﬁc communities when answering questions, in
addition to asking about the communities they fell into at the beginning and end of the survey. We
also found that in preliminary trials participants supplied the more “expert” response before the
more personal one, so this provided them space to do both in a transparent way.
To ensure that our survey development process remained aligned with student ideas we needed
to retain the level of variation in their interpretations of the practices. We structured the pilot
survey to target this alignment by ﬁrst, maintaining our large sample of question statements from
the interview analysis. We also needed large numbers in responses to justify removing redundant
questions and to conﬁrm that our interviews were representative of a much larger sample. This was

80

accomplished by distributing the survey to the students in the introductory algebra-based lab for
non-majors at the end of the spring semester, roughly 650 students. With nearly 500 responses to
the survey we received between 40-60 individual responses to each of the 108 questions.

5.4 Discussion

This paper has described the process we have taken in developing a practice based identity
survey, with our ideas about identity and practices grounded in the literature [3, 4, 5]. We described
our process of using interviews to map the breadth of students’ interpretations of the practices.
From that, we produced 108 questions for an atypical pilot survey, where we work to validate the
interpretations of practices and elicit identity statements.

We argue that in order to produce a robust survey that measures our intended construct it
is essential to make sure each component of the survey development process is focusing on the
students, derived from the information we get from them and is closely tied to the theoretical
framework on identity.
In doing so we intend to produce a survey that will tell us about our
students’ physics identities and the practices we emphasize in our lab courses. We also aim to
progress in our understanding of physics identity, especially for students who would not necessarily
call themselves physicists or those who do not intend to go into a physics related ﬁeld at all. We
posit that a process like ours is necessary to ensure a survey speaks to and for the community we
are studying, students in physics lab courses in our case. Especially, when studying something as
nuanced as identity, we need to be intentional every step of the way to understand our students, to
ensure that they understand us, and to retain the theory as the foundation.

81

CHAPTER 6

SURVEY DEVELOPMENT

6.1 Introduction

The overall direction of this thesis work has been to produce a survey that measures identity in
lab classes. This chapter will lay out the entire survey development process, as seen in Fig. 6.1, up
to the current stage. Anything discussed in previous chapters will only be brieﬂy mentioned here.
This chapter will describe the diﬀerent iterations of the survey from the initial interviews to the
three diﬀerent versions of the survey and related analysis. This chapter concludes with the current
state of the survey, which is made up of 24 closed-response practice questions and ﬁve identity
questions.

The theoretical framework being utilized to model identity was described in both the Literature
Review and Identity Chapters (CH 2 & 4) and will not be repeated here. In addition to a model
for understanding identity in the lab classes, the relevant practices and student understanding of
these practices needed to be determined. A combination of the AAPT lab guidelines and the
learning goals for the transformed introductory lab course (see Chapter 3) lead to the six practices

Figure 6.1: The survey development process from establishing the theoretical framework to describe
identity development to the newest version of the survey.

82

that this work focuses on communicating, analyzing data, executing experiments, working in a
group, planning experiments, and understanding equipment. After the framework for practices and
identity was established, interviews were conducted with students enrolled in a variety of physics
lab courses at Michigan State University (the second stage of the development process in Fig. 6.1).
Fifteen students were interviewed, six upper-division physics majors (or minors), four from the
transformed introductory labs [40], and ﬁve from the untransformed introductory labs. The goals
of the interviews were to determine how students interpret speciﬁc physics lab practices and to
characterize the kinds of identity statements they make around those practices. The interviews were
semi-structured, the students were asked to describe what the practice in question meant to them,
and then they were asked about whether or not they found the practice valuable (See Appendix C
for interview protocol).

6.2 Interview Analysis

6.2.1 Practice Space

We were trying to fully map the space of students’ interpretations of physics lab practices. The
interviews were coded for the diﬀerent practices of interest. Quotes and summaries were pulled
for each practice. Emergent coding was then done to get a picture of the main ways students talked
about the practices and produce sub-bins describing the central interpretations. All of the example
statements were coded into the sub-bins and then summarized into a few representative statements
describing the speciﬁc aspect of the practice. A visual representation of this process is in Fig. 6.2
below, and Fig. 6.3 shows speciﬁc examples (see Appendix E for full examples).

For example, from the interview question: "What does data analysis look like in your lab," we
looked at student responses that fell into a similar theme, graphing in this case. Dean: "As soon
as you have the graph and data in front of you, with a certain mindset, [you] can interpret what it
actually means." Fern: "So interpreting the results, in this class, we use graphs and stuﬀ to do that."
These quotes from Fern and Dean lead to the representative statement, "In my lab, using graphs to
interpret results is important." (Fig. 6.3 also provides examples of this)

83

Figure 6.2: Visual representation of the ‘practice space’

This process was completed for all of the practices and all of the interviews, resulting in 108
practice questions to be used in an open-response survey. A table summarizing the number of
sub-bins and survey questions produced from the analysis is shown in Table 6.1. Fewer sub-bins
just means that there were only a few ways that students were found to interpret the practice.

Table 6.1: Summary of the outcome of the interview analysis, showing the number of sub-bins and
questions for each practice.

Physics Lab Practices
Analyzing Data
Communicating
Working in a Group
Understainding Equipment
Planning Experiments
Executing Experiments
Total

Sub-bins Survey Questions
7
3
9
7
3
2
31

19
18
25
25
14
7
108

84

Figure 6.3: A subset of the practice space for analyzing data shown with speciﬁc examples.

6.2.2

Identity

There was not enough information from the interview analysis to create identity questions to mirror
the practice questions, or to determine if the identity model ﬁt our data and context (See Chapter 4
for further information). The identity statement criteria, described in the Identity Chapter, was
used to develop open-response questions used in the ﬁrst version of the survey.

6.2.3 Survey Development Overview

In the development of a closed-response survey, especially one that can be studied analytically, it
is essential to reduce it to a reasonable size, reasonable meaning that a student can be expected
to answer all of the questions in a moderate amount of time. The ﬁrst version of the survey
had 108 questions, which is clearly not reasonable, so along with conﬁrming that the practices
interpretations were representative, and developing closed-response identity-speciﬁc questions,
a goal of the analysis of this version was to cut down the number of questions in the survey
dramatically. Figure 6.4 outlines how this was done at each step. The methods used to eliminate
questions are brieﬂy described in this section, and the speciﬁc decisions are detailed in the rest of
the chapter.

85

Figure 6.4: Overview of the diﬀerent versions of the survey, described in detail in the following
sections.

6.2.4 Similar Distributions of Responses

In order to justify the inclusion of an item in a survey, it needs provides unique information. There
is value in conﬁrmatory information (i.e., a few questions here and there that cover the same thing)
but we take that into account with groups of questions asking about the same practice. If multiple
items are asking about the same thing, in the same way, the repeated items are providing no new
information. All versions of the survey after Version 1 were analyzed for redundancy in practice
questions. To do this initially, when the number of responses did not allow for robust statistical
analysis, items were considered for collapse or removal if they seemed to be asking the same or
a similar question and if the distributions of responses were visibly, if not statistically, similar. In
Version 3 we had the number of responses to each item necessary to perform statistics to determine
2 pairwise tests on items within each practice group, the
if the distributions were similar. We did χ
details of this analysis can be found in Appendix F. The methods used to reduce or add questions
at each stage are illustrated in Fig. 6.4 and detailed in the rest of this chapter.

6.3 Version 1 - Open-Response Survey

There were two primary goals for the open-response version of the survey. Concerning practices,
the goal was to conﬁrm we have covered the whole of the practice space (Fig. 6.2) and reduce the
number of questions. For identity, it was to generate more examples of identity statements in order to
understand identity in our context, as part of adapting Close’s model [21], and to produce questions

86

Figure 6.5: Example item from version 1 of the survey (Ref. DA5).

that reﬂect identity for a closed-response survey. Each item for the open-response survey had four
parts, two closed- and two open-response. The closed-response questions were Likert scale with
the same structure ‘This practice is important...’ they then have two responses to this ‘For ME’ and
‘For an EXPERIMENTAL PHYSICIST.’ We asked for both of these, like the ECLASS [1], because
during beta tests of the survey the participants were uncertain if they should answer based on their
own thinking or based on the ‘correct’ answer. The other two parts were open-response, where
they could explain their reasoning for how they answered the two Likert questions. A complete
example is shown in Fig. 6.5 (see Appendix H for the full survey).

The open-response survey was distributed via Qualtrics1 at the end of the spring semester 2018,
to a transformed introductory algebra-based physics lab for non-physics majors [40], a total of 650
students. Responses from approximately 500 students were collected. Each student received a
randomly selected ten of the 108 practice questions, in the format shown and described above. As
compensation for completing the survey, the students were given a small amount of extra credit
added to their in-class participation scores.

1Qualtrics, 2019, Qualtrics Labs, Inc. Provo, UT, USA

87

6.3.1 Version 1 Analysis

6.3.1.1 Reduction of Practices

At this stage, the goal was to reduce the number of practice items. Questions were removed for
several reasons, if they were being misinterpreted by students in the open-responses, the question
statement was confusing based on the open-responses or others (e.g., many responses that were ‘yes
and no’ so they would not translate well to a fully closed-response version), or they had no examples
of identity statements in the responses. Several examples of items from the practice ‘group work’
that were removed are listed in Table 6.2 below, along with the reason for their removal. The
analysis of Version 1 of the survey brought us from the initial 108 practice questions down to 79.

Table 6.2: Group Work questions that were removed at this stage and the reason for removal.

Question Statement
Being accountable to my group
(GW2)
Talking things out with my group
(GW8)
Figuring out how to work with my
group (GW4)

Removed Reason for Removal
Yes

No identity statements in the responses

Yes

Yes

Many of the responses contained a struc-
ture of ‘both yes and no’
Responses showed that students were ig-
noring ‘ﬁguring out’ in the question

6.3.1.2 Developing Identity Items

The identity analysis took on a diﬀerent direction; the goal was to characterize the types of
responses and turn them into questions for a closed-response survey. The open coding and analysis
are discussed in Chapter 4 and will not be repeated here. As described in Chapter 4, the majority
of the identity statements fell into two components, Learning Trajectory (LT), and Negotiated
Experience (NE). We decided to focus on just those two components moving forward. In order to
produce identity questions to be used in a closed-response survey, we needed to break down and
characterize the diﬀerent student responses. The ﬁrst step was to break down the responses into the
speciﬁc parts that made them identity statements.

88

An example survey question and response: Q: Being able to communicate my understanding
is important. A: Being able to communicate my understanding would be very important when
working with many patients a day to help explain their progress. In this response, the essential
parts (for being an identity statement) are ‘would be very important’ and ‘when working with [...]
patients.’ This is an example of an LT response.

An example of a survey question and a response that ﬁts into NE: Q: Interpreting graphs to
understand relationships between parameters is important. A: Not important because I don’t like
graphs, and I feel obligated to make one even when I don’t understand the concepts very well. Here
the central parts are ‘I don’t like graphs,’ ‘obligated,’ and ‘didn’t understand concepts.’

For the components, LT and NE, we did this with every response to the open-ended survey
(N = 80) and sorted the various ways of saying the same thing in order to produce statements for
the survey that matched the typical structures and meaning.

Learning Trajectory: From the analysis, LT statements primarily fall into two categories,
future and after. Future is any reference to the future concerning plans/goals/careers. As our
sample contains a lot of pre-medical school students, we also found that statements about future
‘patients’ appeared in this category. The other category is after, that is where they do not specify
a timeline or speciﬁc goal but instead say ‘I do (not) need [this practice] after this class.’ The LT
statements were sorted into four bins, three of those are future-focused, and one is after. The next
step was to look for distinct features of the statements in each category. The responses that fell
into the patients-future category were all positive and included phrases like ‘I will use this’ or ‘this
is something I would need.’ Career-future responses were mostly negative, ‘I will not need this
for my career.’ Future-future responses, those that use the word ‘future’ but do not specify things
further, were a mix of positive and negative. Although it seems like the after statements should
fall into the future category, the structure and types of sentences that ﬁt in that component were
diﬀerent from those that fell into one of the three future categories. All of the after responses were
negative. Representative examples for all types of responses are in Table 6.3.

Four representative examples, one for each of the four categories, were pulled from those in

89

Table 6.3: Summary of the outcome of the interview analysis, showing the number of sub-bins and
questions for each practice.

Components Category

Learning Trajectory - LT

Patient

Career

Future

Future

After

Examples
This is something I will need when work-
ing with patients in the future
This would be important when working
with patients
This is irrelevant to my future career
I won’t need to know this for my career
This is something I will use in my career
This won’t help me in the future
This will help me in the future
This is something I will use in the future
This is something I will need in the future
I won’t use this again after this class
I won’t need this again

Table 6.3 to be used in a closed-response version of the survey (See Table 6.4).

Table 6.4: The following are the four LT statements used in the closed-response pilot survey
(version 2).

Reasoning Number LT Statement
R1
R2
R3
R4

This will be important when working with patients
This is not something I need to know for my career
This will help me in the future
I will not use this again after this class

One thing to point out about the above statements is that they vary in whether they are positive –
‘I will need this’ – or negative – ‘I will never use this.’ Although we would prefer for the statements
to all be either positive or negative and not a mix, we also did not want to assume that either version
of the statement was equivalent to the students (i.e., would we get the same types of responses from
the positive version as the negative versions?). So, for the closed-response version, we decided only
to use the formats that the students had previously used (e.g., the patient statements were positive,

90

and the after statements were negative).

Negotiated Experience: Negotiated Experience statements broke down into two primary types:
obligation – they did it because they had to; and tension in expectations – they disagree with the
value put on something in the class. These are described in more detail in Chapter 4. Table 6.5 has
example statements that came from the detailed analysis of the NE statements. The NE statements
did not breakdown in the same way as the LT ones. Both sets of responses could be narrowed
down into a few categories with representative examples, but the NE examples could not be further
broken down into speciﬁc sentence structures. All of the representative examples of NE are a mix
of negative and positive, i.e., ‘it is useful for the future but not realistic for the class.’
Table 6.5: Summary of the outcome of the interview analysis, showing the number of sub-bins and
questions for each practice.

Negotiated Experience - NE

Components

Category

Less important

Tension in
Expectations

Somewhat
important

Obligation

Important but...

Personal
preference

Value

Understanding

Examples
It is not as important to me
It is not the most important thing for the students
It is important at times but there should not be so
much emphasis
It is somewhat important but it doesn’t apply to me
It is important to think about but it is not always
necessary
It is useful for the future but not realistic for the class
I only care about this because it is for a grade
I don’t care about this but I felt obligated to do it for
the class
It might be helpful but I felt forced to do it in the
class
It will not help my career but I felt like I had to do it
for the class
I did it for a grade even though I didn’t understand it
I felt forced to do it even though it didn’t mean any-
thing to me

Finally, Table 6.6 contains six statements that are representative of the typical NE responses
found in the analysis. We decided not to use the Obligation-Understanding questions because in our

91

experience ‘understanding’ is too broad and has too many possible interpretations (from interviews
and open-response analysis).

Table 6.6: The ﬁnal NE statements from analysis of Version 1

Reasoning Number NE Statement
R5
R6
R7
R8
R9
R10

It is not the most important thing to me
It is important at times but it does not apply to me
It is important to think about but not always necessary
I do not care about this but I felt like I had to do it for the class
It might be helpful but I felt obligated to do it in the class
This is important but I did it because I was expected to

6.3.1.3 A Note on Practices

It was unsurprising that the statements from diﬀerent identity components were not evenly dis-
tributed across the diﬀerent practices. The distribution coded in Version 1 of the survey can be
seen in Table 6.7. Communicating and data analysis had the most statements across the board, and
planning experiments had the fewest. Communicating was the largest source of NE statements,
while LT was spread across communicating, data analysis, and understanding equipment. We ended
up dropping the distribution of NE and LT across practices going forward for multiple reasons; the
most straightforward was the logistics of only having speciﬁc identity stems connected to speciﬁc
practice stems. Also, we want this survey to be broadly applicable outside our institution; we did
not want to assume that all students would respond in the same way as those in DATA Lab.

6.4 Version 2 - Closed-Response Survey

Version 2 was sent out to a new population of students from a diﬀerent institution, a highly
selective university in the northeast of the United States. The primary goal of this version of
the survey was to ensure that the interpretations of practices were representative of a diﬀerent
population. This version was also the ﬁrst time the identity-speciﬁc reasoning questions were
included; it was necessary to characterize how students interacted with those questions. In addition,

92

Table 6.7: Summary of the outcome of the interview analysis, showing the number of sub-bins and
questions for each practice.

Communicating
Data Analysis
Executing Experiments
Group Work
Planning Experiments
Understanding Equipment
Total

LT NE Other Total
9
10
7
2
1
9
38

47
41
17
13
3
17
138

16
8
6
2
2
8
42

22
23
4
9
0
0
58

as before, a goal was to ﬁnd items and questions that could be cut to reduce the survey into something
that would be reasonable for students to take in the whole. Each item on the survey had a closed-
response practice question (similar to Version 1) and then four reasoning questions (a random
selection of the 10 possible developed in the analysis of Version 1). The four reasoning questions
were consistent through the survey for each student, i.e., if student A had reasonings 4-7 for the ﬁrst
practice question, they had 4-7 for every practice question. Each student saw 10 items (meaning
one practice and 4 reasoning questions) of the remaining 79 practice questions (see Fig. 6.6 and
Appendix I). The survey was distributed through Qualtrics at the beginning of the spring semester
in 2019 to students in four introductory physics courses at this northeastern university. The majority
of the students were engineering or applied engineering physics majors. 353 responses in total. As
we distributed this version of the survey to a new population of students, not life science majors,
so we could not assume that ‘patients’ would mean anything toward their future. At the start of
the survey, we asked, ‘When thinking about your future career who do you imagine working with?
(Doctors work with patients, teachers work with students).’ We provided them with options to
select: Patients, Students, Clients, and a ﬁll in the blank. None of the other reasoning questions
were altered.

Since this was a new population of students, six validation interviews were conducted to ensure
that the students were interpreting questions as intended. This was also used as an opportunity

93

Figure 6.6: Example of an item from version 2 of the survey (Ref C4). The ﬁrst part is the practice
question, each student saw 10 of the 79. Then there are the identity reasoning questions, this part
remained the same for all of the practice questions.

to discuss the identity reasoning questions and see how students understood them. No signiﬁcant
diﬀerences were found for practice questions, but the interviews did inﬂuence decisions concerning
the identity-speciﬁc reasoning questions, which are described in the following section.

6.4.1 Version 2 Analysis

Reduction of survey responses was made with multiple cuts, as shown in Figure 6.4. The ﬁrst set
focuses on the practices and identifying similar wording and distributions of responses. The second
set focuses on the identity reasoning statements and uses statistical analysis.

6.4.1.1 Reduction of Practices

Since each student only saw 10 of the 79 practice questions, we were unable to do any substantial
statistical analysis as we received fewer than 70 responses to each practice question. Questions were
cut at this stage if they had similar distributions to similarly worded questions (e.g., two diﬀerent

94

group work focused questions about having help from their group were collapsed into one instead).
The process is highlighted with the ﬁgures below; the ﬁrst step was looking for similar distributions
in responses (Fig. 6.7 and Appendix K).

The questions that had similar distributions were examined to see if the question statements
were similar enough to combine, collapse, or remove. Once it was determined which questions in
the group were reasonably similar, the choice had to be made about what could be collapsed (see
Fig. 6.8 for an example). Questions were also considered for removal from the reverse direction.
Starting with asking if the questions were asking nearly the same things, and then checking to
see if the distribution of responses looked similar (Fig. 6.9). Finally, at this stage, questions were
considered for removal based on the question statement themselves and whether or not the question
was clear, concise, and speciﬁc; an example is shown in the Table 6.8 below. After the analysis at
this stage, we brought the number of practice questions from 79 to 45.

Table 6.8: Question statement additionally evaluated.

Question Statement
Agreeing as a group on how to com-
plete a task before moving forward
(ref GW22)

Removed Reason for Removal
Yes

Not a critical practice, reﬂected in
free-response statements, also am-
biguous in what a preferred re-
sponse would be

6.4.1.2

Identity

Since each student saw and responded to each of their four identity reasoning questions 10 times
(one time for each practice statement), we were able to do more substantial statistical analysis as
we had more than 1000 responses to each reasoning question. We used this statistical analysis to
collapse the reasoning questions across, and within practices, the process is illustrated in Fig. 10
(statistics are detailed in Appendix F). Once it was conﬁrmed that the reasonings could be combined
across practice questions, we did the same pairwise analysis, but this time grouped by framework
components, see Fig. 6.11 (LT is R1-4, and NE is R5-10).

95

Figure 6.7: Six diﬀerent GW questions to look for similar distributions of responses. Notice the
similarities between GW5, 6, and 17. The responses are normalized to one. On the x-axis: 0 is
Strongly agree, 1 is Somewhat agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree, and
4 is Strongly disagree. (See Appendix G for the questions, and Appendix K for additional plots)

96

Figure 6.8: Illustration of the removal of questions based on similar distributions of responses.
Step 1 shows the ﬂagging of questions with similar distributions of answers. Step 2 shows the
question statements being examined for similarities, the three blue questions were determined to
be similar enough to be considered for collapse. Step 3 provides a closer look at the three related
questions, the light blue (Q3) is the questions chosen to remain. (Reference numbers: Q1 is GW6,
Q2 is GW7, Q3 is GW5, Q4 is GW19, and Q5 is GW17).

For the NE questions, within all six of the practices, we ﬁnd signiﬁcant similarities in the
pairwise comparisons shown in Fig. 11, R5, 6, 8 and R7, 9, 10. Referring to the question
statements in Table 6.9, it is easy to see that R5 and R6 have to do with relative importance, so as
long as the resulting statement reﬂects the idea of relative importance, it should represent both of
these well. R8, on the other hand, is an obligation statement and does not clearly ﬁt with the other
two, so we will hold oﬀ on collapsing or combining it here.

Taking a closer look at R7, 9, & 10, the three questions start with similar statements of value,
‘important/helpful’. R9 and R10 both end with obligation statements, but R7 does not, it is more of
an example of value vs. seeing value (Ch 4). There is also an interesting similarity between R8-10,
they are all related to obligation in some way. Even with this similarity, there is an additional reason
that may explain why R8 is not signiﬁcantly similar to R9 and R10.

97

Figure 6.9: Similar questions being considered for removal. Here Q1 and 2 talk about groups doing
things together, which is why they were considered for collapse. Their response distributions were
similar enough to combine the questions. (Reference numbers Q1 is GW19 and Q2 is GW20).

98

Figure 6.10: The initial step of analysis of reasoning questions. We had to conﬁrm that it was
reasonable to collapse the responses to each practice question by doing pairwise tests to make
sure responses to R1 (This will be important when working with patients) were the same for each
practice question.

The structure of these two statements, R9 and R10, are:
‘positive value statement’ + but + ‘obligation statement’
Whereas R8 is:
‘negative value statement’ + but + ‘obligation statement’
This may explain why the distributions are diﬀerent. The obligation statements all originate from
the same part of the NE theory and open-response analysis. So, dropping that statement is justiﬁed.
At this point, we can collapse the original six NE statements into three (see Table 6.10). NE1
originates from the collapse of R5 and R6. It is just statement R5 because R6 could be interpreted
as asking two questions, and it is essential to reduce confusion in the respondents. NE2 is the
collapse of statements R9 and R10. A simple change was made to R10, we replaced ‘was expected’
with ‘had’ to leave ﬂexibility in where the obligation comes from. For example, maybe the teacher
expects it, or perhaps the student has personal ideas about doing physics, both are an obligation,

99

Figure 6.11: In the process of analyzing responses to the identity-speciﬁc reasoning questions, we
did pairwise tests on the questions grouped into NE or LT. The analysis showed that R5, 6, and 8
and R7, 9, and 10 had statistically similar distributions, i.e., each group should be considered for
collapse.

and neither is excluded in NE2. NE3 is just R7 since that statement did not appear to be collapsible
with R9 and R10. We are currently leaving out R8 because it is mainly an obligation statement,
which NE2 already is.

Two learning trajectory statements were signiﬁcantly similar in each practice group, R2, and
R4. We can examine them to determine if they should be collapsed. Comparing these statements
(Table 6.9), both are negative, and both imply not needing the physics practice in the future. One
of the validation interviews supports why these might collapse and not R3, which is ‘This will help
me in the future.’ Based on the interview it has to do with the fact that ‘future’ by itself leaves open
the context of what or when the future might be, whereas R2 and R4 tie to a speciﬁc context, even

100

Table 6.9: Reasoning statements used in Version 2 of the survey.

Reasoning Question Statement
R1
R2
R3
R4
R5
R6
R7
R8
R9
R10

This will be important when working with "BLANK"
This is not something I need to know for my career
This will help me in the future
I will not use this again after this class
It is not the most important thing to me
It is important at times but it does not apply to me
It is important to think about but not always necessary
I do not care about this but I felt like I had to do it for the class
It might be helpful but I felt obligated to do it in the class
This is important but I did it because I was expected to

if R4 does it unintentionally. Student 5, talked about this in his interview, he said that by saying
‘after this class’ R4 is more tied to the context of school, so that would have framed his response.
R2 clearly ties to a future career, as it is in the statement. Even though the context is diﬀerent
between the two statements, the fact that a solid context is provided might explain the similarity of
the distributions. We kept R2 because the school context could be missed in the use of ‘future’ in
R3.

The new LT reasoning questions are in Table 6.10. LT1 is R1 from the ﬁrst version of reasoning
questions. LT2 is the result of collapsed R2 and R4, it is the original R2, which includes a
connection to the context of school. Then LT3 is the original R3. This analysis of the identity
reasoning questions takes us from ten down to six, three for each component (NE and LT).

Table 6.10: The updated reasoning questions with their origins as described in this section.

Label Question Statement
LT1
LT2
LT3
NE1
NE2
NE3

This will be important when working with "BLANK" R1
I will not use this again after this class
This will help me in the future
It is not the most important thing to me
This is important but I did it because I had to
It is important to think about but not always necessary R7

Origin

R2 and R4
R3
R5 and R6
R9 and R10

101

Figure 6.12: An example item from version 3 of the survey (Ref. C12).

6.5 Version 3 - Closed-Response Validation

The third version of the survey was given to students in the second semester of the transformed
labs for non-physics majors at the end of the spring semester 2019. 360 students completed the
survey (a 60% response rate). The goal was to reduce questions to a number that would be
reasonable for a student to answer all of (fewer than 30, based on the E-CLASS [1]). Each student
saw 20 of the remaining 45 practice questions and all six reasoning questions. The structure was
otherwise the same as Version 2 (see Fig. 6.12 and Appendix J).

6.5.1 Version 3 Analysis

6.5.1.1 Practices - Final Reduction

Due to the reduction in the size of the survey, we were ﬁnally able to run pairwise tests of all of
the questions within each practice (greater than 100 responses to each question). The majority of
the distributions of responses within a speciﬁc practice are statistically similar; this implies that

102

students are responding to the questions in the same ways so they can be combined (for distributions
see Appendix L). Since the responses to these questions were statistically and visually similar, they
were all accomplishing the same thing, i.e., not highlighting anything unique about working in a
group, at least not that were showing up in student responses. This meant that any of them could be
removed or collapsed. The choice of removal or collapse was made based on several criteria. First,
if multiple questions were asking similar things, they should be collapsed, or all but one should be
removed (just as in Version 2). Then the goal was to cover most of the breadth of interpretations of
the practice with as few questions as possible.

The ﬁnal step was diﬀerent from the others because the goal was not to determine what questions
could be removed but instead, which should remain. The questions that should remain were those
that produced unique responses to the identity reasoning questions. To determine this, pairwise
tests were run on the responses to each set of reasoning questions within a practice group; Fig. 6.13
illustrates this. The pairs that showed signiﬁcant diﬀerences were examined (for details of the
statistics see Appendix F). Practice statements that produced unique identity reasoning responses
several times were deemed to be ones that must remain. This step was mainly a check to make sure
questions that produced unique reasoning responses were not being removed.

The analysis of the practice questions brought the total from 45 down to 24, which is a reasonable

number to expect students to answer as a whole instead of a subset.

6.5.1.2

Identity

Pairwise tests were run on the identity-speciﬁc reasoning questions both within practice groups
and across all responses (as in Version 2), in every case LT1 and LT3 were signiﬁcantly similar
(Table 6.10). Due to the statistically similar distributions, those two questions can be collapsed
into one. LT3 will be kept because LT1 is more complicated and requires a ﬁll-in-the-blank. None
of the NE reasoning questions were statistically similar, so they cannot be collapsed at this time.
Additional analysis will be necessary to determine if any can be collapsed or removed, but that is
not possible at this time. The ﬁnal ﬁve identity reasoning questions are in Table 6.11.

103

Figure 6.13: Showing the process for conﬁrming that the practice questions that had unique
reasoning responses were kept (Ref. GW9).

Table 6.11: The ﬁnal ﬁve identity reasoning questions.

Label Question Statement
LT2
LT3
NE1
NE2
NE3

I will not use this again after this class
This will help me in the future
It is not the most important thing to me
This is important but I did it because I had to
It is important to think about but not always necessary

6.6 Discussion

Each stage of the survey development process, including all three distribution and analysis
cycles, has lead to a fully-closed response survey with fewer than 30 questions, a reasonable
number for students to complete in entirety. After the establishment of a theoretical framework for
understanding labs, practices, and identity, the survey development process began with interviews
to aid in the understanding of the ways students interpret diﬀerent lab practices. Analysis of the
interviews resulted in 108 questions covering the space of students’ interpretations of the practices.
Those questions were tested in an open-response survey (Version 1), the analysis of which reduced
the number of practice questions to 79 and lead to the development of ten identity-speciﬁc reasoning

104

questions. Version 2 of the survey was in part a test of those identity reasonings, but also of the
survey in a new population of students. Through analysis of responses to Version 2, the reasoning
questions were reduced from ten to six, and the practices from 79 to 45. Analysis of the ﬁnal
survey distribution discussed here, Version 3, culminated in a remaining 24 practice and ﬁve
identity reasoning questions, a survey that can be reasonably completed by students in whole. (See
Appindix M for complete list of questions in Version 4).

The goal of a fully closed-response survey has been reached, but work is still necessary to
evaluate Version 4 quantitatively. A large number of responses are necessary to conduct analytics
in order to begin to make claims about the results of the survey. Although the entire survey has
been designed fully from student responses and interpretations and in alignment with a theoretical
framework on identity, it is not possible to claim what responses to the survey tell us analytically
about identity without additional quantitative work.

105

CHAPTER 7

CONCLUSION

This dissertation has laid out the picture of the work being done on physics labs and pointed to a
pressing need, large-scale assessment tools (Chapter 2). This has included a discussion of the large
scale lab transformation in the algebra-based physics labs at MSU (Chapter 3). The transformed lab
course set the context for the study of how physics identity is connected to practices in lab classes
and the development of a survey to measure that connection (Chapters 4, 5, and 6). The survey
development process has lead to a fully closed-response version that can be reasonably taken in
whole by students (see Appendix M). This ﬁnal version has yet to be validated analytically, which
is necessary before claims can be made about the responses.

If time allowed the next step would be to distribute the survey to a large number of students and
run analytics to evaluate any correlations between diﬀerent components, practices or identity. At this
time we have a survey that has been developed and validated in alignment with student interpretations
and the model of identity (as described in Chapter 4). Analytic validation is necessary before broad
claims can be made about responses as a whole but the process of developing the survey in and
of itself has produced signiﬁcant insights. We have provided a method for producing assessment
tools to measure other attitudinal and aﬀective changes. We have shown that identity can be tied to
experiences with practices and that the model used by Close et al. [21] cannot be directly applied
to our context without adjustment.

In experience with this work it has been apparent that the traditional question, ‘Do you see
yourself as a physics person,’ used as validation in other quantitative identity work, is an insuﬃcient
measure of identity in many cases. We are opposed to the dependence on this one question for
one thing a single question should not be the sole validation of and identity assessment. Also,
identity and identifying with physics is not nearly as simple as the ‘physics person’ question makes
it seem. The students in the DATA Lab were mainly life science majors, the majority of whom
would say in interviews that they do not see themselves as a physics person right before making a

106

statement that reﬂects physics identity. Using the ‘physics person’ question to validate studies of
physics identity greatly limits the scope of the work regarding who is being counted. It also narrows
the possible understanding of physics identity, it is not limited to only perspective physics majors,
anyone can identify with physics and ignoring that does a disservice to the broader population and
to physics. Working to make physics education better for everyone is hindered when we ignore
how the majority of people may identify with the ﬁeld. Additionally, the goals of physics identity
research do not need to be focused on getting and retaining more physics majors. Becoming a
physicist does not and should not be the end goal of developing a physics identity. We physicists
generally see physics as the foundation of most if not all studies of science, therefore we should see
value in making that foundation more accessible to all who study science.

7.1 Future Directions

There are several aspects of the work that we were unable to pursue, for example we focused
on only two components of the identity model in the survey development, learning trajectory and
negotiated experience. We found evidence of community membership, interest, and the unclassiﬁed
examples in the data but did not analyze them further. A more complete version of the survey
would include reasoning questions that address at least community membership and interest. The
‘physics person’ question was included on Versions 2 and 3 of the survey. It would be interesting
to see how that question correlates with other responses to the survey.

Most of the survey development was done in the context of the DATA Labs (Chapter 3), which
inevitably inﬂuenced the direction of the work. The practices central to the survey, although not
solely produced from responses of DATA Lab students, were undeniably aﬀected by the course. As
mentioned above, the DATA Labs and upper division labs at MSU did not emphasize the practice
of computation (at the time of the interviews), which is a relevant practice in lab courses at other
institutions.

The structure of the DATA Labs, both the course environment and the student population, also
likely impacted the work in regards to identity. As discussed in Chapter 4, there was no evidence of

107

competence or recognition in the student responses to the open-response survey and we hypothesize
that this is likely due to the features of the course and the goals of the students. As an example, the
course does not explicitly emphasize recognition from peers or instructors, and these students are
not physics majors so recognition with regards to their physics skills is likely to be less important
to them. We also expect that the community membership (CM) and negotiated experience (NE)
responses found were inﬂuenced by the course context, as one for non-physics STEM majors. The
NE responses were all negative predictably due to opposing perspectives about practices that the
students must resolve. Additionally, we see CM as a step to follow NE and as students getting
introduced to physics they have not had the time to move from the NE to the CM phase of identity
development. The potential inﬂuences of the course context both in practices and identity is a
relevant direction of additional future investigations.

During the analysis of Version 1 of the survey, we began looking at practice-speciﬁc responses
that showed diﬀerent levels of sophistication in application and understanding of the practice. For
example, one student might say that understanding the equipment means they know how to use it
for their purposes, while another student says they understand the equipment when they are able to
adapt it to new situations. In this case the second student has a more sophisticated interpretation
of the practice of understanding equipment. Like the other components of the identity model this
area of analysis was dropped due to limited time and resources.

Another unutilized direction from Version 1 of the survey is all of the answers to the ‘for an
experimental physicist’ open-response questions. We were again unable to do anything with that
area of analysis due to time constraints, even with the substantial amounts of data.

7.2 Big Picture

For practitioners, faculty in charge of physics labs, the hope is that this dissertation put into
focus the current state of undergraduate physics lab education. Labs need to move away from
conﬁrmatory and validation experiments, and away from the ﬁxation on content learning.
It is
time that we take full advantage of the speciﬁc opportunities aﬀorded to us through laboratory

108

instruction. As shown in this dissertation, the practices and process of ‘doing physics’ have been
goals of lab instruction since the beginning and yet we have struggled to realize those goals for just
as long. Once practices are set as the goal for lab courses the design and assessment must follow
(Funkhouser, 2019 [40] and Chapter 3). Part of the problem has been the disconnect between
having practices as a goal and still assessing based on content. What those assessments look like
is still an active area of research but there is enough out there to begin to turn the tide of labs in
physics education. The DATA Labs described in Chapter 3 provide examples of this process for
the learning goals speciﬁc to the course, but the steps can be followed with a variety of practices.
For example, the faculty at MSU did not discuss computation as a goal for non-physics majors in
the labs but there are plenty of situations where computation may become central to the course
design. No matter the direction preferred by the program, progress in physics labs education must
continue forward. Content learning has never been the strength of lab instruction and maintaining
it as a goal is and has always been a mistake.

Understanding physics identity outside of ﬁeld of physics is a valuable step in the direction of
reaching more people. Lab environments can and should be utilized as a space for students to take
agency over their learning. Since their inception in science education, the potential of labs have
not been fully realized. Researchers and practitioners alike need to lean into the opportunities for
engagement in the process of ‘doing science’ that are provided by lab environments. Let this be
a reminder of where labs are going and also how important they still very much are in physics
education. There is no time in the history of labs in science education that educators have used
them to their full potential and there is no time like the present to change that.

109

APPENDICES

110

APPENDIX A

DATA LAB EXPERIMENT DESCRIPTIONS

Descriptions of the DATA Lab experiments as discussed in Chapter 3.

Table A.1: DL1 Experiments

Main Idea
Taking a variety of measurements and determining the uncer-
tainty
Taking simple measurements, learning how to plot parame-
ters in useful ways, and incorporating uncertainty into analy-
sis and results
Using the example of making a peanut butter and jelly sand-
wich to evaluate procedures and discuss proper documenta-
tion
Exploring free fall and terminal velocity with coﬀee ﬁlters

Free Fall
(In)elastic Collisions Exploring elastic and inelastic collisions on an air track
Standing Waves
Impulse

Exploring waves on a string, nodes, and resonant frequencies
Exploring impulse with bouncing balls and relating that to
concussions
Exploring energy loss of a bouncing ball
Exploring a simple pendulum

Experiment
Workshop 1

Workshop 2

Workshop 3

Energy Transfer
Pendulum

111

Experiment
Ohm’s Law

Fields

Transformers

Resistivity

Charge-to-mass Ratio
Biomeasures

Snell’s Law: Rainbows

Table A.2: DL2 Experiments

Main Idea
Measuring current and voltage to determine resistance of a
resistor
Exploring electric ﬁelds, determining the number of electrons
transferred between two pieces of tape
Exploring solenoids and transformers, developing a formula
to describe the relationship between voltage and number of
turns
Exploring how resistance depends on length and width, de-
termining the resistivity of clay
Finding the charge-to-mass ratio of an electron
Exploring an oscilloscope, measuring a heartbeat on the
scope
Exploring refraction of light, investigating Snell’s Law and
diﬀerent wavelengths of light

Snell’s Law: Fiber Optics Exploring refraction of light, investigating Snell’s Law and

Lenses

Interference

Polarization

total internal reﬂection
Exploring thin lenses, investigating how corrective lenses
work with diﬀerent eye impairments
Exploring interference and diﬀraction, investigating single
and double slit diﬀraction
Exploring polarizers, investigating polarization to come up
with Malus’s Law

112

APPENDIX B

DATA LAB RUBRICS

These supplementary ﬁles provide insight into the assessments of the DATA Lab (Ch 3). What
follows are the rubrics used in diﬀerent aspects of the course to evaluate the students in relation to
the learning goals.

113

114

Design Analysis Tools and Apprenticeship (DATA) Lab Supplemental Materials Rubrics and Descriptions  --1--  DL 1/2 Notebook Rubric  --3-- DL 1/2 In-Class Rubric  --6-- DL 1 Communication Project   --9-- DL 2 Communication Project   --12-- DL 1/2 CPHW Figure/Caption  --14-- DL 1/2 CPHW Abstract  --16-- DL 1 CPHW Critique  --18-- DL 2 CPHW Background  --20-- DL 1 Critique Project  --22-- DL 2 Design Project  115

DATA Lab I/II  Notebook Rubric Learning Goals  Experimental Design Communication: Clearly record the steps and results of the experiment. Scientific Process: Proceed intentionally through the experiment. -Communication -Diagrams are labeled and clear -Procedure is clear and detailed enough to reproduce the experiment -Observations are noted and related back to the concepts -Predictions are listed and explained -Reasoning/motivation shows why the experiment was carried out in this manner -Scientific process -List goals for each section of the lab -Did they plan ahead? -Did they work systematically? And intentionally? -Multiple trials -Attempts to improve accuracy  Discussion Uncertainty: Discuss uncertainty in measurements, models, and results. Results: Present results in a clear way and support claims with evidence.  -Uncertainty -Measurements -Recorded with uncertainty -Discuss sources of uncertainty -Uncertainty is quantified and explained -Worked to reduce uncertainty -Models -Compared models to observations -Discussion of limitations/assumptions of model -Corrections to model based on experimental results -Results -Plot (if one exists) -They graph their data in a way that makes sense -The plot tells us something significant about the experiment -Not just, x and y are linearly related -There is a best fit line and a slope -There are error bars on the data points -Error bars that make sense -They graphed their data in an interesting way that tells us something about the experimental parameters -Conclusions -They discuss the motivation of the experiment  1116

DATA Lab I/II  Notebook Rubric -They summarize the process -They discuss the result -Slope or other significant result is given with uncertainty and compared to some expectation -The significance of the result is discussed  -It is tied to the bigger picture (implications)   General grading strategy -Combine -Did they do it at all? -Did they do it well? -Overall things to look for -Communication, scientific process, uncertainty, and results   General grading scale: 2 (65) - Very weak notebook, they turned something in but that’s about it -Most categories are nonexistent and what is present is done poorly 2.5 (68-74) - Not great, they left out very important pieces of information completely and what they have needs a lot of work -Many categories or aspects are left out and what is there needs major improvement 3 (78-84) - An alright notebook, they have some good stuff but it needs a lot of improvement -Most categories are present but are missing some aspects. Some aspects are good but others need a lot of improvement 3.5 (87-92) - A good notebook, could use some more work  -All categories are present but there are some missing aspects or significant improvements to be made 3.75 (93) - A very good notebook, they just have a couple of things they could improve on -All categories and most aspects are present but there are several aspects that have room for improvement 4 (96-100) - Perfect, I cannot think of anything else they could have done -All categories and aspects are present and no further improvement is necessary  2117

DATA Lab I/II  In-Class Rubric How, and why, is in​ class work assessed?  What’s the purpose of this in​ class grading?​  Everyone in this lab comes from a variety of backgrounds and have varying levels of experience with physics, lab equipment, and working in groups. The purpose of this In​ class grading is to allow for you to develop your abilities in engaging with lab equipment, describing the role of uncertainty in the experiment, communicating science to others, and collaborating effectively as a group member. As such, this rubric is not focused on if you are engaged in class, but rather on how you are engaged. This in ​class grade is not designed to make a big impact on your final grade in the course, it is to help develop abilities along these areas.  How are numerical scores given? ​ The goal is for every student to improve their abilities along each of the areas listed in the rubric over time. The expectation is that as you develop your abilities, the expectations in the course will rise. So what was considered a 3.5 early in the course may not be the same as what a 3.5 is by the end of the course. This is a growing experience, and as such, we do not intend anyone to be experts at the start. Truth be told, there is no way to be “perfect” at science. However, by leveraging what we do well, we can focus on the things in which we need improvement and continuously grow.  What’s the purpose of the written feedback from the tutors?​ Each day in the lab, the tutors will try to observe and notice how each student in their groups is doing in the areas on the rubric. After each day that you work on an experiment, you will receive feedback from the tutor on areas they would like to see you work as well as those they think you did well. The expectation is that, for future labs, you will work to improve in the areas the tutor noted.  How are the tutors going to notice everything in this rubric?​ It is not likely that every student will demonstrate everything in this rubric and that the tutor will notice everything. To help focus this, the tutors will approach each group with some questions related to the areas in the rubric to gauge understanding. The point isn’t to notice everything you have done, but rather to get an idea of where you are currently and give you feedback and support to develop as time goes on.  So how do I get a 4.0?​ Unfortunately, there are no concrete examples of how to get a 4.0 because there is not one correct way to engage in class. Instead, there are many ways to conduct an experiment ​ something we intend to value. In an attempt to remove some of the ambiguity, sample “ladders” of expertise are given below, as well as a graphical representation of the development tutors are valuing in this assessment. These should be taken as non​-exhaustive lists but rather simple examples of one way in which development could occur. The best way to increase or maintain your grade is to pay attention to the feedback your tutors provide.      3118

DATA Lab I/II  In-Class Rubric  Learning Goals  Group Function Communication: Contribute to and remain involved in group discussions. Collaboration: Come prepared and work well with your group. Discussion: Work together to understand the result. -Communication -Engaged in group discussions -Asking/answering questions -Discussing plan with group -Explaining equipment to each other  -Collaboration -Come prepared -Actively participate in group work -Make sure everyone is on the same page -Don’t move on too quickly -Try to come to a consensus  -Ask for and give input -Attempt to get everyone involved -Don’t dominate the discussion -Discussion -Work together to understand results -Discuss implications of different experimental results or observations -Contribute helpful ideas in data analysis  Experimental Design Equipment: Engage with the equipment. Show competence in use of equipment. Experimental Process: Employ good experimental practices. Uncertainty: Take into account where uncertainty plays into the experimental process. -Equipment -Engage with the equipment -Actively involved in data taking -Able to troubleshoot equipment -Show competence in use of equipment  -Experimental process -Work systematically -Don’t just randomly take measurements -Work with intention -Make predictions -Record observations and data -Set goals -Uncertainty 4119

DATA Lab I/II  In-Class Rubric -Work to reduce uncertainty in measurements -Discuss and quantify sources of uncertainty -Carry uncertainty through calculations -Discuss model assumptions and limitations -Compare results to expected   General grading scale: 2 (65) - Negatively impacted work of group and progress on experiment 2.5 (68-74) - Did not contribute in anyway to group’s progress 3 (78-82) - Weak participation, still worked with group but needs a lot of improvement  3.25 (85) - Sort of average, not outstanding but solid effort 3.5 (87-90) - They worked well with their group but there are definitely places where they can improve  3.75 (93) - They were vital to their group’s success but they could improve on some things 4 (96-100) - Perfect, I can’t think of anything they could have done better 5120

DATA Lab I  Communication Project Rubric Communication Project Rubric  This rubric is intended to assess student proposals during the communication project day. The communication project homework rubrics helped inform the topics, and therefore should help expedite grading. Each section is equally weighted. Information provided in a proposal should cover both the work previously completed and the work that might emerge from it. This rubric intends to assess both aspects.  A previous communication project homework tasked students with critiquing proposals, everyone should be aware of both what is expected in a proposal as well as how to assess them, using feedback provided there to inform critique decisions and their assessment of others. While a complete proposal is expected, by the first project day, students have only received feedback regarding previous work. Therefore, the first proposal is likely to be stronger in that area. To account for this, the first proposal is weighted significantly less than the second. This activity and future homework assignments are intended to build further skills, culminating in the second proposal. As such, please make sure the feedback you provide is productive and useful to the students receiving it. Student​-assigned feedback that does not meet this condition will not satisfy the “in ​class” portion of the project grade.  The grading breakdown for this day is as follows: ●70% of the complete grade - Tutor​-assigned feedback  ●20% of the complete grade - Student-​assigned feedback  ●10% of the complete grade - In ​Class Portion: Completion of five (4) peer reviews for other students   All feedback will be returned to the students but will be returned anonymously. Both strengths and areas of possible improvement should be noted, with suggestions to successfully improve.  There is no intent to assess on “beauty,” and no benefit should be awarded for proposals that are visually more “appealing” or “official.” If images are included, no benefit or penalty should be included outside of clarity.      6121

DATA Lab I  Communication Project Rubric Prior Work  [​4.0(100%)  4.0(96%)  3.5(88%)  3.0(80%)  2.5(72%)  2.0(64%)​] Experimental Design: It is clear how the past data was taken. Discussion: The document tells a complete story of the conducted experiment -Experimental Design -Method -The overall experimental method including pertinent equipment and critical procedural steps are included -Uncertainty -Discusses uncertainty in measurements, models, and results. -The quantitative data are presented with uncertainty -Sources of uncertainty are described -Deviations from the model are described -Discussion -Communication -Motivation of previous work is clear -Figures are complete (axes, titles, units, error bars, etc.) -Results  -Results from previous work are presented within experimental constraints -Reasonable conclusions and implications are drawn from the data presented  -Data are presented effectively Future Work  [​4.0(100%)  4.0(96%)  3.5(88%)  3.0(80%)  2.5(72%)  2.0(64%)​] Experimental design: It is clear how future data will be taken. Discussion: The document discusses compelling future work. -Experimental Design -Method -Necessary or additional equipment is discussed -New experimental methodology is summarized -Reasons for conducting experiment in this manner are clear -Uncertainty -Future work has considered possible issues -Acknowledges concerns and complications -Considers potential solutions -Discussion -Proposed investigation is original, but connected to and motivated from the previous work -There is a need for the future work -The proposal is compelling and well argued  Would you fund this project?  YesNoRequires further review  7122

DATA Lab I  Communication Project Rubric General grading strategy -Combine -Did they do it at all? -Did they do it well? -Overall things to look for -Communication, scientific process, uncertainty, and results  General grading scale: 2.0 (64%) - Very weak proposal, they turned something in but that’s about it 2.5 (72%) - Not great, they left out very important pieces of information completely and what they have needs a lot of work 3.0 (80%) - An alright proposal, they have some good stuff but it needs a lot of improvement 3.5 (88%) - A good proposal, could use some more work  4.0 (96%) - A very good proposal, they just have a couple of things they could improve on 4.0 (100%) - Perfect, I cannot think of anything else they could have done 8123

DATA Lab II  Communication Project Rubric Communication Project Rubric  This rubric is intended to assess student presentations during the communication project day. The communication project homework rubrics helped inform the topics, and therefore should help expedite grading. Each section is equally weighed. Information during a poster presentation is conveyed in two ways ​ the physical poster and the oral discussion. Neither one can be complete without the other, and therefore this rubric is intended to cover both aspects.  While a complete presentation is expected, there has been feedback so far on communicating figures and the contents of an abstract. Therefore, the first poster presentation is likely to be stronger in these areas. To account for this, the first presentation is weighed significantly less than the second. This presentation and future homework assignments are intended to build further skills, culminating in the second presentation. As such, please make sure the feedback given is productive and useful to the students receiving it. Student​-assigned feedback that does not meet this condition will not satisfy the “in​ class” portion of the project grade.  The grading breakdown for this day is as follows: ●70% of the complete grade - Tutor​-assigned feedback  ●20% of the complete grade - Student-​assigned feedback  ●10% of the complete grade - In ​Class Portion: Completion of five (5) peer reviews for other students   All feedback will be returned to the students, but will be returned anonymously. Both strengths and areas of possible improvement should be noted, with suggestions to successfully improve.  There is no intent or desire to have students print formal posters for this event. Therefore, “beauty” is not an assessable aspect and no benefit should be awarded for posters printed in this way. “Tiled printing,” as discussed in the project information, is desired to cut cost and remove “beauty” as an element. Black and white, taped together pages forming a complete poster is more than sufficient.      9124

DATA Lab II  Communication Project Rubric Experimental Design [​4.0(100%)  4.0(96%)  3.5(88%)  3.0(80%)  2.5(72%)  2.0(64%)​] Communication: It is clear how the data were taken. Scientific Process: Proceed intentionally through the experiment. -Communication -The overall experimental method is described -Pertinent or unique equipment is described -Critical procedural steps are explained -Scientific process -Reasoning/motivation - Is it clear why they conducted the experiment this way? -Quantitative data are presented with uncertainty -They worked to reduce the uncertainty  Discussion [​4.0(100%)  4.0(96%)  3.5(88%)  3.0(80%)  2.5(72%)  2.0(64%)​] Uncertainty: Discuss uncertainty in measurements, models, and results. Results: Present results in a clear way and support claims with evidence.  -Uncertainty -Measurements -Discuss sources of uncertainty -Uncertainty is quantified and explained -Models -Deviations from the model are described -Discussion of limitations/assumptions of model -Results -Plot -They graph their data in a way that makes sense -There is a best fit line and a slope -There are error bars on the data points -They graphed their data in an interesting way that tells us something about the experimental parameters -Conclusions -They discuss the results -Slope or other significant results are given with uncertainty and compared to some expectation -The significance of the results are discussed  -It is tied it to the bigger picture (implications)  Presentation [​4.0(100%)  4.0(96%)  3.5(88%)  3.0(80%)  2.5(72%)  2.0(64%)​] Poster: The poster tells a complete story in a way that is easy to follow. How they presented: The presentation tells a complete story that is engaging. -Poster 10125

DATA Lab II  Communication Project Rubric -The poster is structured in a clear way -The figures are appropriate for the data -Figures are complete (axes, titles, units, etc.) -Poster is easy to read with a mix of figures and text -Their presentation -The flow of the discussion is clear and connected -The presentation is engaging -The experiment is described so that those who did not conduct it can understand -They are able to answer questions related to the study    General grading strategy -Combine -Did they do it at all? -Did they do it well? -Overall things to look for -Communication, scientific process, uncertainty, and results   General grading scale: 2.0 (64%) - Very weak project, they turned something in but that’s about it 2.5 (72%) - Not great, they left out very important pieces of information completely and what they have needs a lot of work 3.0 (80%) - An alright project, they have some good stuff but it needs a lot of improvement 3.5 (88%) - A good project, could use some more work  4.0 (96%) - A very good project, they just have a couple of things they could improve on 4.0 (100%) - Perfect, I cannot think of anything else they could have done 11126

DATA Lab I/II  Communication Project HomeworkFigure/Caption CPHW Motivation  When thinking of conducting an experiment, there are some aspects that easily come to mind: set up the equipment, collect and analyze the data, and determine what can be interpreted from the results. What is often forgotten, but equally important, is presenting these results to the greater scientific community. Twice in the semester, you will be asked to develop a proposal that emerges as a result of your experiments.  Because proposals are potentially new to you, there will be a series of homework problems that are intended to help you develop your proposals as the semester goes. For each submission, your tutor will provide brief feedback on your successes as well as ways to improve. While proposals may seem foreign, being able to sell yourself or your ideas probably doesn’t.  Figure/Caption In this assignment, you will submit a figure and a caption that represents the data and results from your most recent experiment. More often than not, this figure will take the form of a graph. The rest of this description will use a graph as the example. However, the results of your specific experiment might not be best demonstrated using a graph. Consider your data and your experiment carefully as you prepare your figure.  For full credit, your figure should represent your results effectively and clearly. This should include a caption that describes your figure, discusses the uncertainty in your results, and discusses a conclusion or an implication that can be drawn from the analysis. Your figure will likely include a fit and comparison to a model, which should be discussed in the caption as well. Writing a good caption is not trivial ​ it is important to include the relevant information while maintaining brevity. Your caption should be concise ​ aiming for 2​-3 sentences and roughly 50 words.  Upload your CPHW on D2L.     12127

DATA Lab I/II  Communication Project HomeworkFigure/Caption Grading Rubric   Four grade options 0 - Nothing submitted 3 - Submitted but not of high quality 4 - Could use improvement 5 - High quality work submitted  Emphasis on written feedback  Graph - Out of 5 -Communication -Graph demonstrates the data clearly -Graph provides insight into the parameters beyond a linear relationship -Uncertainty -Error bars are included -Outliers are removed from fit   Caption - Out of 5 -Communication -Discussion of what is being plotted -Comparison to a model -Conclusions/implications are drawn from and supported by data -Uncertainty -Deviations from the model are addressed -Analysis includes uncertainties  13128

DATA Lab I/II  Communication Project Homework Abstract CPHW Motivation  When thinking of conducting an experiment, there are some aspects that easily come to mind: set up the equipment, collect and analyze the data, and determine what can be interpreted from the results. What is often forgotten, but equally important, is presenting these results to the greater scientific community. Twice in the semester, you will be asked to develop a proposal that emerges as a result of your experiments.  Because proposals are potentially new to you, there will be a series of homework problems that are intended to help you develop your proposals as the semester goes. For each submission, your tutor will provide brief feedback on your successes as well as ways to improve. While proposals may seem foreign, being able to sell yourself or your ideas probably doesn’t.  Abstract  During the last three weeks of class, you have participated in workshops that focused on a number of important experimental concepts. While participating and engaging with these activities introduced you to experimental concerns, being able to synthesize that information and determine what is relevant moving forward is important when trying to understand and implement experimental practices. Summarizing the work you have previously undertaken is a meaningful exercise in scientific communication, and scientists often write abstracts -​ single paragraph summaries -​ to do so. A successful abstract is one that not only presents the major results of the work, but also informs the reader of the experimental methods, motivation of the experiment, and what those results tell us about our world.  A main aspect of an abstract is how much it explains in so few words ​ there is very little that can be taken out of an abstract without changing the meaning. Most written scientific communication starts with an abstract because it informs the readers of the most important aspects of the work, allowing the reader to choose whether or not to continue reading. Therefore, the trick of writing an abstract is that, typically, all of this is contained in ​300 words or less.  For this Communication Project Homework, you are tasked with writing an abstract about your experiences in the workshops. It can either be written about the workshops as a whole or about a specific one, but it should tell a complete story of the experiment. A reader of your abstract should be able to understand what you are doing in your experiment and why, the way you conducted your experiment, what the results were, and conclusions that can be drawn from these results in the bigger “science picture.”  Upload your CPHW on D2L.    14129

DATA Lab I/II  Communication Project Homework Abstract Grading Rubric   Four grade options 0 - Nothing submitted 3 - Submitted but not of high quality 4 - Could use improvement 5 - High quality work submitted  Emphasis on written feedback  Experimental process - Out of 5 -Motivation -It is clear why the experiment was conducted -The goals and objectives are clear -Experiment -It is clear how the experiment was conducted -Critical pieces of equipment and critical steps of the procedure are addressed  Discussion - Out of 5 -Results -Quantitative results, including uncertainty, are presented clearly -Sources of uncertainty are presented and relevant -Conclusions -Implications that can be drawn from the data are clearly discussed -Deviations from model are discussed -Results are connected to the bigger picture 15130

DATA Lab I  Communication Project Homework Critique CPHW Motivation  When thinking of conducting an experiment, there are some aspects that easily come to mind: set up the equipment, collect and analyze the data, and determine what can be interpreted from the results. What is often forgotten, but equally important, is presenting these results to the greater scientific community. Twice in the semester, you will be asked to develop a proposal that emerges as a result of your experiments.  Because proposals are potentially new to you, there will be a series of homework problems that are intended to help you develop your proposals as the semester goes. For each submission, your tutor will provide brief feedback on your successes as well as ways to improve. While proposals may seem foreign, being able to sell yourself or your ideas probably doesn’t.  Critique For this Communication Project Homework, you are tasked with critiquing a proposal already completed. Giving and receiving critical yet constructive feedback is a skill that, with practice and reflection, can result in higher quality work. By thoroughly critiquing others work, you can find areas of your own work that may require additional attention and support. Meaningful critique usually contains two main points: recognition of strengths within the work as well as identification of opportunities to improve the work. Like writing about your own work, it is important to be clear and unambiguous ​ the receiver of your work should be able to understand your comments and improve the work from them. Therefore, it is important to identify specific examples that demonstrate the areas you are emphasizing.  A very strong critique will also suggest strategies intended to improve the work. Like many other aspects in this class, providing the necessary resources for growth is critical. Even though you will not resubmit proposals after receiving feedback, you will be writing two throughout the semester. Therefore, while specific improvements are useful, the author should be able to understand the reasoning behind the comments and scores so that they can incorporate them in their next proposal.  Samples of proposals are located on D2L. In these samples, the authors have written about work that goes beyond the content covered within Physics I. While the concepts covered may be different from what you’ve experienced in this class or in lecture, the document should still be written for scientists. Therefore, even if the science discussed is new or unknown, how the experiment was undertaken should be clear as should the results, conclusions, and rationality between the two. The rubric you will be using to critique the proposals, which is the same rubric by which your proposals will be assessed, is provided on D2L.  Upload your CPHW on D2L.  16131

DATA Lab I  Communication Project Homework Critique Grading Rubric   This is the rubric that will be used to grade ​your critique. ​See D2L for the rubric you should use to critique the proposals.  Four grade options 0 - Nothing submitted 3 - Submitted but not of high quality 4 - Could use improvement 5 - High quality work submitted  Emphasis on written feedback  Identification - Out of 5 -Areas of growth -Areas of success -Detailed examples  -Suggestions for improvement  Feedback - Out of 5 -Comments are reflected in the scores -Magnitude of success or need for improvement is signaled in the scores 17132

DATA Lab II  Communication Project Homework Background CPHW Motivation  When thinking of conducting an experiment, there are some aspects that easily come to mind: set up the equipment, collect and analyze the data, and determine what can be interpreted from the results. What is often forgotten, but equally important, is presenting these results to the greater scientific community. Twice in the semester, you will be asked to develop a poster and present out a result from your experiments ​ one from Electricity/Magnetism and one from Optics. Because presentations like these are potentially new to you, there will be a series of homework problems that are intended to help you develop your poster as the semester goes. Your instructor will provide brief feedback on ways to improve your submissions, helping develop your overall poster presentation.  Background  Through the process of creating and presenting a poster, many of you conducted your own background research to help you connect to your research. This process is very important and is something we hope to see more of on your second poster. The work we do in lab is rarely enough to fully understand the research ​ we need to investigate outside the lab, too.  Literature Review For this Communication Project Homework, you are tasked with undertaking a literature review ​ background research using credible and reliable sources ​ to motivate your work. Through expanding your knowledge base, you will be able to not only understand your work better but also to communicate it more effectively. We are asking you to find ​at least 3​ sources that you can reliably cite. These may provide background information into the experiment, support physics concepts, connect the work to “real ​life” situations, or provide applications that you may find interesting. These sources should help provide motivation for further study of this experiment.   Summary of Review Then, write a short (one​ page) summary that communicates the big take​aways from your review ​ how was that work relevant and related to the work you did in lab, what motivated the study, how is it connected to the real world, what are the limitations to these applications ​ citing your sources as appropriate. Finally, use this information, as well as the work you conducted in class, to define an interesting research question that you would pursue given more time in lab. Please note what equipment or type of methodology you would use to investigate this question. Please comment on the feasibility of conducting that experiment in this lab ​ would you be able to given our resources, or would you need more?  Upload your CPHW on D2L.    18133

DATA Lab II  Communication Project Homework Background Grading Rubric   Four grade options 0 - Nothing submitted 3 - Submitted but not of high quality 4 - Could use improvement 5 - High quality work submitted  Emphasis on written feedback  Sources - Out of 5 -Sources are credible and relevant  -Background -Connection to work done in the lab is clear -Relevant work conducted previously is explained -The motivation is clear -Real-life connections -Discusses connections between implications and fundamental physics  -Limitation of applicability  Future Research - Out of 5 -Communication -Related to work done in lab -Focused and clear -Equipment -Major equipment and methods are discussed -Feasibility of experiment is considered  19134

Critique Project DATA Lab PHY 251   1 Motivation((Scientists(strive(to(remain(objective(in(their(research,(drawing(logical(conclusions(from(their(data(and(analysis(rather(than(individual(views(or(beliefs.(Scientists(interpret(their(results(under(some(existing(framework((e.g.,(Newton’s(Laws).(These(interpretations(are(often(communicated(to(the(scientific(community(and(are(presented(using(that(framework(from(their(perspective(as(the(scientist(conducting(the(research.(These(findings(are(validated(and(vetted(by(the(research(community.((The(general(public(often(receives(their(scientific(news(from(media(outlets(and(social(media,(but(these(sources(are(typically(less(objective(as(the(science(they(present.(In(this(way,(bias(from(media(sources(or(journalists(can(influence(the(scientific(arguments(they(publish.(Therefore,(in(order(to(obtain(the(information(closest(to(what(was(reported,(it(is(incumbent(on(the(general(public(to(interpret(and(critique(the(media(they(consume.((For(this(project,(you(and(your(group(are(tasked(with(critiquing(two(sides(of(a(scientific(issue(that(has(been(presented(to(the(public.(The(topic(you(choose(is(up(to(you(but(should(be(discussed(and(agreed(upon(before(coming(to(class(for(the(day.(You(are(to(read(both(articles(and,(with(your(group,(determine(the(arguments(presented(and(how(they(are(being(supported.(((To(fully(critique(an(argument,(you(need(to(read(and(consider(both(sides(of(it(with(an(openHmind.(Analyze(it(from(your(perspective(as(a(budding(scientist(–(think(about(the(claims(that(each(article(makes,(what(supporting(evidence(is(provided,(and(what(rationale(is(used(to(connect(the(evidence(to(the(claims.(At(the(end(of(class(today,(you(will(hand(in(one(typed(copy(of(your(group’s(critiques,(which(should(consider(both(sides(of(the(topic(your(group(chose.((The(expectations(of(each(critique(are(detailed(below.((The(Critiques(and(Assessment((Below(is(a(breakdown(of(what(is(expected(in(each(critique(and(the(scores(carried.(You(and(your(group(should(turn(in(one(typed(copy(of(your(project,(which(includes(a(critique(of(both(sides(of(the(argument.((• Each(side(of(the(debate(will(count(for(45%(of(your(total(grade,(and(each(side(should(follow(the(structure(below:(o Review(of(the(argument((20%(total,(as(broken(below)(! What(are(the(claims(being(made?((5%)(! What(evidence(is(being(used(to(make(those(claims?((5%)(! How(is(the(claim(linked(to(the(evidence?((5%)(! What(assumptions(or(limitations(do(these(claims(work(under?((5%)(o Assessment(of(the(argument((25%(total,(as(broken(down(below)(! Address(the(validity(of(the(evidence((5%)(! Discuss(the(logical(reasonability(of(the(connections(between(evidence(and(claim((5%)(DATA Lab ICritique ProjectRubric20135

Critique Project DATA Lab PHY 251   2 ! Determine(the(applicability(of(the(assumptions(and(limitations((5%)(! What(are(the(strengths(of(the(article?((5%)(! What(are(the(weaknesses(of(the(article((5%)(• Overall(summary(of(the(debate((10%(of(the(total(grade,(broken(down(below)(o Summarize(the(issue(considering(all(sides,(including(why(there(are(multiple(conclusions(that(can(be(drawn((5%)(o What(does(your(group(conclude(about(this(scientific(debate?(Are(there(unresolved(disagreements(within(your(group?((5%)((Topics(and(Articles((Here(is(a(list(of(topics,(with(counterHpoint(articles,(that(you(may(choose(from.(However,(if(you(and(your(group(decide(to(discuss(a(different(topic(or(different(articles(within(a(topic,(please(discuss(this((and(which(articles(you(have(found)(with(your(tutors.((Alternative(Energy(Viewpoint(1(Viewpoint(2((Autism(and(Vaccinations(Viewpoint(1(Viewpoint(2((Climate(Change(Viewpoint(1(Viewpoint(2((Genetically(Modified(Organisms((GMOs)(Viewpoint(1(Viewpoint(2((Medical(Marijuana(Viewpoint(1(Viewpoint(2((Red(Meat(Viewpoint(1(Viewpoint(2(DATA Lab ICritique ProjectRubric21136

DATA Lab II  Design Project Rubric Motivation  Design a DATA Lab experiment -Goals​: Communication with peers and tutors, Experimental design -Output: -Overview​ of a new DATA lab experiment -Outline -Goals -Uniqueness of experiment -Tutor document -Explain the physics -Where may complications may arise/ how can you solve them  -Thought questions for the students   The Documents   Experiment Overview This is an outline that will be used to create the write up for the new experiment.  -Outline of the experiment -​ What equipment is necessary? What is involved in the experiment? What is the expected procedure? What observations are you hoping the students make? -Goals -​ What are the goals of the experiment? Why do it? What are they trying to find? How does the experiment ensure that the goals are met? What do you want students to learn from this? Provide predicted results. -Uniqueness -​ It builds off of previous experiments but does not replicate them. Differences from previous experiments are clear and highlighted. Why is this experiment uniquely qualified to achieve the learning goals?   Tutor Document In DATA lab we have tutors who help students progress through the experiments. It is important for them to have a document that gives them the information they need and helps them handle issues when they come up. -Physics concepts​ - What physics concepts are you trying to demonstrate? All necessary concepts should be detailed. Do not assume the specific physics knowledge of the tutor. Provide the information that they will need to have in order to help the students. Include the necessary physics equations. Connect the physics ideas to the bigger picture. -Complications​ - What complications may arise and how can you solve them? Describe any difficult parts of the experiment and how to deal with them. Include any tips and tricks for the experiment. 22APPENDIX C

INTERVIEW PROTOCOL

This appendix contains the protocol used for the interviews discussed in Chapters 4, 5, and 6. The
interviews were semi-structured therefore the protocol were only a guide and were not necessarily
strictly followed. Three diﬀerent protocol are included, the ﬁrst was used when interviewing
students from the DATA Labs. The second protocol was adapted from the ﬁrst and used for
interviews with upper division physics students and students in the untransformed introductory
algrabra-based physics labs. The third protocol comes from the survey validation interviews that
were conducted at the highly selective northeastern university, where Version 2 of the survey was
distributed (Sec. 6.4).

137

138

 Original​ (DATA Lab Students)  Tell me about you  -your major -why did you choose that -what’s your favorite class  How would you describe your physics lab to a friend who was thinking about taking it?  How do you think your instructor would describe it?  Can you walk me through a day in the lab? [How does class start, what do you do?]  What is challenging in the day-to-day of the lab?  What does it take to succeed in the class?  In the DATA labs you get written feedback right? Do you find it useful? Why do you think they do that?  Is there anything you struggled with in the course?  What was something you found really exciting?   I am going to list some things and for each one I would like to know what they mean to you and how important you think they are for the class.  -Group work -Uncertainty -Understanding the equipment -Planning your experiment -Communicating your results -Asking questions -Trying things  Is there anything that you would like the designers of the lab to know?   Is there anything else you would like to share about your lab experience?    139

 Final Version​ (Upper Division and Not DATA Lab Students)  Name:  Tell me about you,  -your major -why did you choose that -what are your future goals -what’s your favorite class  Can you tell me a bit about this physics lab? [big picture, how long is an experiment, what do you turn in, etc.]  Can you walk me through a day in the lab? [How does class start, what do you do?]  What does it take to succeed in the class?  Can you describe a time where it really felt like you were doing physics? [Do you feel like you’re doing science in the class?]  Do you feel like you are a physicist?  Would you call yourself a physicist?  I am going to list some things and for each one I would like to know: -What they mean to you (What you think of when I say it)? -How important is it for you? -Do you feel like you do this? Can you do this? -How important you think they are for the class? -Is this an important part of doing physics?  -Group work -Does your partner trust your input? -Uncertainty -Understanding the equipment -Do you usually feel like accomplish this? -Planning your experiment -Executing the experiment -Interpreting your results -Answering your research question -Communicating your results -Do you feel ownership over your lab work? -Asking questions -Trying things Is there anything else you would like to share about your lab experience? 140

  Do you have any questions for me? ————————————— Extras Tell me a bit about your experience in the physics department here.  How would you describe your physics lab to a friend who was thinking about taking it?  How do you think your instructor would describe it?  What is challenging in the day-to-day of the lab?  Is there anything you struggled with in the course? [Can you describe a time where you really struggled with something in the course?]  What was something you found really exciting?     141

 Survey Validation Interviews  (As transcribed from my notebook and memory) -Tell me about yourself -Physics classes -Other classes -Physics labs -What are your future career plans -Survey response:  -Thank you for taking the survey, I would like to ask you about some of your responses -Take a look at these, remind yourself of what you were thinking when you answered  -I would like to go through and talk about your response to each question -Practice question: -Why did you answer this way -How did you interpret this question -Reasoning question: -How did you interpret this reasoning question -Why did you answer this way for this practice question  -Did you feel like all of the relevant practices for the lab were represented in the survey questions -Did you feel like the reasonings provided were representative of your own thinking -Would something else have fit better -You actually received 4 of the 10 possible reasonings, please take a look at these and let’s talk about how you interpret them and if you would have answered them differently than what you already did  -Is there anything you would like to share  -about your experience of taking the survey -About your experience in the physics department here -Anything else  -Do you have any questions for me  APPENDIX D

CODEBOOK

This appendix contains the codebook disccussed in Chapter 4 and used by the external research
to test interrater reliability. The identity components come from the adaptation of the model from
Close et al. [21] as described in Chapter 4.

142

143

Identity Codebook  Introduction Notes on coding Identity Statements Requirements Experimental Physics Practices Identity Codes Learning Trajectory Indicators Interest Indicators Negotiated Experience Indicators Recognition Community Membership Indicators Competence Other Indicators  Introduction This codebook is for the process of developing survey questions to ask about identity.  -It is not for understanding how a particular student identifies with physics -It is meant to define the features of identity statements of different types, that can then be turned into question for a survey on identity All of the statements are tied to a specific practice, contained in the question statement -This codebook wouldn't capture pre statements before students have specific practices to tie their responses to (e.g. at the beginning of the semester) -This codebook will not be useful in any sort of pre/post analysis of responses -Although the outcome, a survey, should be   144

Notes on coding Each response should be coded in full -Although it is helpful to indicate the components of the response that make it fit into a specific code Responses may be coded as more than one thing -If this becomes an issue we will reassess     Identity Statements Requirements -Connected to an experimental physics practice 145

-This practice -Includes a personal value statement -This practice is valuable -Connects back to the identity framework -Generally speaking this should explain the value statement -This practice is valuable because of this reason  Note: The pieces of the requirements may also be pulled from the question statement -Q:​ This practice is valuable -R: ​Because of this reason   Experimental Physics Practices The names we use here are broad titles, the applications of the practices branch out from these as interpreted by the students in the interviews -Communicating -Working in a group -Analyzing data -Understanding equipment -Planning experiments -Executing experiments  Identity Codes -Learning Trajectory​: The value they put on the practice is connected to their future or their planned career -Interest​: Indication of personal interest or preference for something -Negotiated Experience​: Dealing with (or adapting to) the expectations and value of the experimental physics community. Negotiating how those values and expectations align (or not) with one's own -Recognition​: Receiving recognition for performing physics practices and showing competence in them -Community Membership​: The value of the practice comes from the community -Competence​: Perceived competence in practices valued in the community  -Other​: I am not sure where these fit but there are still identity statements     146

Learning Trajectory  The value they put on the practice is connected to their future or their planned career -i.e. I want to be a doctor, so I value things that will be helpful for me as a doctor   Indicators Future They say something about their future or how they may use this practice in the future -Referring to future career/profession or plans -Transferable or relevant to their future -Cues: future, career, profession, med school, patients   Q​: Determining patterns from my data R​: it is not so important to look for trends in data because overall it is only a small part of what I am studying for my degree and will not necessarily help me in ​my overall career​. Indicators​: Future  Q​: Using math to apply a pattern to what I saw R​: I will use it in​ everyday life in med school​ when thinking about medications, anesthesia and other factors. Indicators​: Future  Q​: Working with the equipment R​: It helps build confidence in doing hands on things somewhat, but I don't think working with this specific equipment is really helpful for ​me in the future Indicators​: Future   After/Later The utility of a practice after the course is over, without a specific future path or career indicator -I won’t use this again/anymore -This is the only time I will use this  Q​: Working with the equipment R​: I may ​never see this equipment ​again​, and if i were to go to a lab the equipment and how it is used may be very different. learning the specifics of the equipment used ​may not be helpful​, but it is valuable to understand what it does and why you would use it for the study Indicators​: After/later   147

Interest Indication of personal interest or preference for something  Indicators Enjoyable/interest Preference -Some discussion of like or dislike -Care/don’t care -Want/don’t want -Requirements: Preference statements require an additional connection to a practice -I like this thing because it helps me understand the purpose of the experiment -The connection does not need to be fully concrete but it does need to be more concrete than abstract -I like numbers because of how they work​ ​x -I like number because of how they work ​to prove the concepts​ ✅ Notable Feature -Interest statements frequently contain a sentence structure like 'thought-1 ​but​ thought-2.' -Statements like this are Interest statements when thought-1 contains a preference (like/hate/etc.).  -Generally in these cases thought-1 is a negative preference and thought-2 is a positive statement.  Note:  -Have not seen many examples of this in survey data -Have not seen examples that connect LT and I   Q​: Doing calculations with data  R​: is important because ​i like numbers​ and how they work out to prove your concepts. Indicators​: Preference, appreciation   Q​: Using math as a tool to explain and predict R​: I think it is helpful just because ​I enjoy math​ and ​it makes it easier for me to understand things. Indicators​: Preference, enjoyment  Q​: Presenting what my results mean R​: ​I am not really into physics​ so I ​could care less​ if I present the data that I find​ because most the time I don't even understand it. Indicators​: Preference, could [not] care less 148

Q​: Presenting what my results mean  R​: ​I don't love physics​, so ​i don't love presenting​ the work, ​but​ i​t is important​ to receive peer feedback Indicators​: Preference, I don’t love   Negotiated Experience Dealing with (or adapting to) the expectations and value of the experimental physics community. Negotiating how those values and expectations align (or not) with one's own  Indicators Tension in Expectations Disagreement in the necessity or value of a practice that is emphasized in the course or in physics generally -I don’t want to do this but I do it because I have to -Not as important, realistic, true, necessary -Frustrating  -Student -The response is based around the fact that they are a student. ​This isn’t important because I’m only a student. -Additional requirements to these statements -Statement about being a ​student -And a connecting ​value statement -ie: ​This isn’t important​ because ​I am only a student​ and I just need to get through this -Just a class -The value or lack of is based on the fact that it is just a class or lab -Additional requirements to these statements -Statement about it being a ​class/lab -And a connecting ​value statement -ie: ​This isn’t important​ because ​it is only a lab ​and I just need to get through this  Q​: Knowing how to adapt equipment to new situations R​: ​As a student​ I feel like this ​is not as important​ because most experiments I will do in a lab class will be controlled. This might be useful for the future but ​as of now I don't think it is necessary​. Indicators​: Tension in expectation - student  Q​: Presenting what my results mean 149

R​: for us ​in a lab like this​, we are all recreating the same experiments. While a lot of the time we get different results, we are all doing the same research and experiments. Due to this, it ​isn't the most important for us to convey our research. Indicators​: Tension in expectations - just a class  Valuing vs. seeing value They can recognize the value of the practice, with respect to the physics community or the course, but they do not personally value it -I understand the value in the course, but I don’t think it is valuable to me personally -(Ir)relevant, Inconsequential  -This is important in physics but it is not relevant to me   Q​: Presenting what my results mean R​: Typically, all of the results from DATA Lab experiments are​ inconsequential in a larger physics picture​. ​Maybe it is somewhat helpful in understanding the lab​, but ​it is not the most important part​ of an entire process ​for students​. Note​: Here the combination of the bold statements make this fit into the ‘just a class’ code Indicators​: Tension in expectations - student/just a class, also value vs seeing value Explanation​: There is a lot going on in this statement, first wrt tension we have the the first bolded part with ​“inconsequential​” and the underlined “​not the most important part​” both indicated a tension between the student’s expectations and that of the course. The already mentioned part plus the italicized “​maybe it is somewhat helpful​” indicates the value vs seeing value, they see why it could be a helpful thing but it isn’t what really matters to them  Obligation They only care about this thing because they have to for the course.  -I feel obligated to do this, even though I don’t think it matters so much -Obligated, Forced/pushed, Need/had to -Grade -Something done for a grade or something that is important because they are graded on it -Indicators:  -Need this for a good grade -You won’t get as good of a grade -It doesn’t matter for the grade   Q​: Using data to determine the significance of my results R​: It doesn't matter that much for me as long as I get the results. ​That is what most of the points for a lot of classes are​ [associated] with. Indicators​: Obligation - Grade 150

Explanation​: We get obligation from two things here, the first is “​it doesn’t matter … as long as I get the results​” and then they also show us that the value comes from the fact that they are graded on it “​that is what most of the points [are for]​”   Q​: Using graphs to interpret results R​: However, I felt that sometimes ​the "need" of graphs was a bit counter productive​. I felt like I was​ forced into​ creating a linear model of the data that we had taken, and that if I did not take data that could be graphed linearly then that data would be basically useless, ​which is not always true Indicators​: Obligation, Disagreement, tension in expectations Explanation​: We have a word that indicated obligation “forced” in addition to that we have two examples of tension -Counter productive -Not always true  Q​: Interpreting graphs to understand relationships between parameters R​: Not important because ​I don't like graphs​ and ​I feel obligated​ to make one even when I don't understand the concepts very well Indicators​: Obligation, Tension in expectations, Preference against Explanation​: Another example of obligation, where they use that word specifically. There is also tension in both the fact that they do not like graphs and they do not understand the concepts.  Notable Feature -These statements often have a structure of 'thought-1 but thought-2' where 1 and 2 are often contradictory -It is only NE when the first thought is not an interest statement (I like this but...) ​AND​ the second thought is an NE statement (any of the above indicators) -Often, thought-1 is positive and thought-2 is negative for NE, the opposite is true for Interest  Q​: Communicating my results by answering specific questions  R​: ​It helps me​ keep track of what I'm doing ​but​ sometimes the organization ​demanded in class isn't realistic​ for real life. Indicators​: Obligation, Tension in expectations Explanation​: Second half of the statement is Tension in Expectations   Recognition Receiving recognition for performing physics practices and showing competence in them  Notes: 151

-We have not seen examples of this in survey or interview responses -We do not have examples of R combined with NE    Community Membership The value of the practice comes from the community Indicators Distinction between multiple communities  Something is important in one community and not in another -Science vs physics -Class vs research lab  Q​: Explaining how I did what I did R​: Because when performing an experiment of any sort you need to be detailed in order to be able to recreate the experiment exactly and reproduce the results. Or conversely to be able to go through and alter the procedure and check for flaws that could prohibit the results you hypothesize. This is​ true in any science​ and good practice even in physics. Note​: Lower confidence Indicators​: Science, Comes from community, distinction between communities Explanation​: We are triggered on “true in any science” as our indicator that they are talking about the science community. The first two sentences are the description/explanation but don’t actually include any indicator for identity, that comes in the last sentence     Competence Perceived competence in practices valued in the community   Notes: -We have struggled to find examples of competency statements in the survey data  Other I am not sure where these fit but there are still identity statements 152

Indicators Not an expert  Q​: Communicating my results by answering specific questions R​: Not as helpful because I am​ not a professional ​and don't necessarily understand everything that the lab had to offer, just the major concepts. Indicators​: Not an expert  Q​: Having help from my group if I have questions R​: I am ​not a professional​ and need a lot of help when conducting experiments so having others to lean on is helpful. Indicators​: Not an expert  Purpose -The purpose of doing this  Q​: Communicating my results by answering specific questions R​: Explaining findings is the ​whole purpose of doing an experiment Indicators​: Purpose  Relevance -What I am doing is irrelevant/relevant -It has been done before -We know the answer -What we’re doing is relevant in some way  Q​: Making connections to the bigger concepts R​: it helps me to put forth more effort if I can understand the concept in a ​real life example​. Indicators​: Relevance  Q​: Working with the equipment R​: Is not really important because I don't do many experiments or measurements in my ​regular life​. Indicators​: Relevance  Utility -Helps (or doesn’t) me to  -To learn, understand, do something   Q​: Using math as a tool to explain and predict 153

R​: As an undergraduate student, sometimes the underlying concepts behind experiments go over my head. For me, it's sometimes ​most useful for me​ to relate a concept to an equation and understand how different variables in the equation relate to one another, rather than more complicated physical variables in an experiment. Indicators​: Utility      APPENDIX E

DEVELOPMENT OF PRACTICE QUESTIONS

This appendix provides a detailed examples of the process of producing the original questions for
Version 1 of the survey from the interview results (see Sec. 6.2.1).

E.1 Emergent Coding of Interviews

The interviews were coded for the diﬀerent ways students interpreted the physics practices. The
descriptions of the practices were used to create and code into speciﬁc sub-bins. Table E.1 shows
three interview quotes describing understanding equipment and the associated emergant coding
(the overview column). Table E.2 shows the sub-bins that were produced from analysis of all of
the responses to the question about understanding equipment.

Table E.1: Review interviews for where students talk about speciﬁc practice, 1-3 word summary of
their interpretation of the practice. The practice here is understanding equipment.

Student
Beth

Clara

Alice

Statement
We used diﬀerent solenoids, so like, do we want them really close together, do we
want them far apart, how does that aﬀect anything. We just kind of ﬁgured out
what works best for us in the experiment.
Be familiar with what it is and how it’s used, or like how it can be used because a
lot of things can be used in diﬀerent ways, um so, just being familiar with how it
works, what it can do, what it can measure.
Just seeing even just how the equipment works. I know that oscilloscopes are just
a whole bunch of buttons, two little screens, one of them is green with a whole
bunch of lines on it and you know its just, when you see that light going across in
that pattern, it’s i don’t know, I just think it’s cool.

Overview
Trying it out

What is it, How is it used,
Diﬀerent ways it can be
used, What it can measure
How the equipment works
What it allows you to do/see

E.2 Coding and Summarizing

Once the common sub-bins were deﬁned the student quotes were sorted into the associated bins.
The quotes were reduced to the main point. For example, Clara’s description can be simpliﬁed into
‘knowing how the equipment works’ (Table E.3).

154

Table E.2: Summaries turned into common sub-bins

Sub-bins
Adapting

Exploring

Description
Knowing how to ﬁt the equipment into new systems, or how to use it
in diﬀerent situations or to measure diﬀerent things
Exploring the equipment, trying things, ﬁddling with things, seeing
how adjustments change things

How it works Knowing how the equipment works
How to use it Knowing how to use it to get what you need
Limitations

Purpose

What are the limitations of the equipment, what can’t you do/measure
with it
What purpose does it serve in this experiment, what does sit let you
do/see, What does it measure, what are the diﬀerent ways it can be
used

Table E.3: Student responses sorted into bins and summarized

Sub-bin: How it works
Overall summary of each
interpretation
Main points

Alice
Understanding how it works, seeing how
it works
How it works
Seeing how it works

Clara
Familiar with how the equipment works

How it works

E.3 Representative Examples

After the coding was completed examples were created that were representative of the diﬀerent

student responses that fell into that sub-bin (Table E.4).

Table E.4: Statements that reﬂect all of the ‘How it works’ examples

See how the equipment works
Know how the equipment works
Know how the parts work
Know how the equipment works to adapt it to your needs

155

APPENDIX F

STATISTICS

In this appendix the statistics used in this dissertation are described. These tests were used in
Chapter 6.

F.1 Signiﬁcance Testing

In analzing survey data there is a challenge in generalizing the results from a sample to the
broader population. In most of our work we have been sampling the students in the DATA Lab, in
hopes of generalizing the broader population of students in physics labs.

F.1.1

2 Contingency Table Analysis

χ

2 contingency table analysis of the
The main means of signiﬁcance testing we did in this work was χ
distributions of responses to the survey questions. This is a way to determine if there is a statistically
signiﬁcant diﬀerence between two sets of data, or the responses to two diﬀerent questions in our
case. The formula to determine χ

2.

Here, observed in equation F.1 refers to eq. F.2, which comes from the data that have been collected.

Expected in F.1 refers to

E(cid:48)
ij =

(sum of row j)(sum of col i)

.

2 = (observed − expected)2

expected

χ

(sum of row j)(sum of col i)

sample size

Eij =

Observed =



E11 E21
 Eij

E12 E22

156

(F.1)

(F.2)

(F.3)

(F.4)

Using F.2 and F.2, χ

2 becomes

(cid:16)

Eij − E(cid:48)
ij
E(cid:48)
ij

(cid:17)2

.

2 =

χ

(F.5)

This can be illustrated with an example from the data. Say we want to determine whether or not
the responses to two practice questions are similar enough to collapse into one question. The
distributions to both questions are in Table F.1.

Table F.1: Observed distributions of responses to two practice questions.

Agree
Nuteral
Disagree

Practice 1 Practice 2
80
25
50

100
18
30

We can calculated the expected responses using equation F.4. The expected table becomes,

Table F.2: Expected distributions of responses to two practice questions

Agree
Nuteral
Disagree

Practice 1 Practice 2
92
22
41

88
21
39

Using we can calculate χ

2 for this sample.

2 = 1.6

χ

degrees of freedom, df = 2

Once χ

2 is calculated for the observations the p-value can be determined. The p-value is the
probability that the data would be at least as extreme as the observation, it is used to determine
whether or not to accept the null hypothesis.
In this case the null hypothesis is that the two
2 and the degrees of freedom (rows − 1)(cols − 1) the p-value
distributions are the same. With χ
can be determined from a table. Before that happens though it is important to correct the p-value

157

that determines signiﬁcance for the number of tests, the Bonferrinu Correction1 (Sec. F.1.2). In
2 = 1.6
this example there was only one test so we can refer to the table, for df = 2 the value of χ
is smaller than what is presented in the table, meaning our p-value > 0.1 and we must accept the
null hypothersis that these two samples are the same.

Table F.3: χ
0.05
p = 0.1
3.841
2.706
4.605
5.991
7.815
6.521

df
1
2
3

2 Table

0.01
6.635
9.210
11.345

F.1.2 Bonferroni Correction

When multiple statistical tests are completed on the same sample it is easy to keep reproducing an
error, so it is important to maintian stricter criteria. Instead of taking a p-value of 0.05 as signiﬁcant
a correction is necessary:

Corrected p-value =

(p-value of signiﬁcance, 0.05)

number of tests

.

(F.6)

In pairwise testing the number of tests can be determined by the formula below, where n = the
number of individual sources in the sample,

Number of tests =

n − 1.

(F.7)

This can again be illustrated with an example from the data. In the analysis for Version 3 of
the survey we compared all of the communicating questions against eachother in pairwise testing,
a total of 8 questions.

1Bonferroni, C. E. “Il calcolo delle assicurazioni su gruppi di teste.” In Studi in Onore del

Professore Salvatore Ortu Carboni. Rome: Italy, pp. 13-60, 1935.

158

Number of tests = n − 1 = 7 = 28

n = 8

( 0.05)

number of tests = 0.05

28 = 0.002

Corrected p-value =

Our new p-value of signiﬁcance is 0.002.

F.1.3 Eﬀect Size

2 are not suﬃcient to determine how reasonable it is to accept or reject the null
The p-value and χ
hypothesis. The p-value tells you whether or not the two samples are diﬀerent but not how diﬀerent
they are. It is necessary to also calculated the size of the diﬀerence between the two groups, or the
eﬀect size. The eﬀect size used here is called Cramér’s V:

(cid:115)

V =

2/n

χ

min(k − 1,r − 1),

(F.8)

Where k = number of columns r = number of rows n = number of observations. The size of the
eﬀect as calculated depends on the dimensions of the sample,

df ∗ = min(rows − 1,cols − 1)

small medium large
0.1
0.5
0.35
0.07
0.06
0.29

0.3
0.21
0.17

df ∗ = 1
df ∗ = 2
df ∗ = 3

(F.9)

(F.10)

F.2 Inter Rater Reliabiliy - Cohen’s κ

In qualitative studies it is common to bring in an outside researcher to help determine the
reliability of the work doone by the original researcher. For example, in Chapter 4 we describe the
coding of the identity responses. In order to verify the reliability of the codebook and the coding

159

done but the original researcher we brought in an external researcher to code a subset of responses.
It is important to quantify the inter rater reliability, we use Cohen’s κ to do this. The equation to
calculate Cohen’s κ is

where pe is deﬁned as

pe =

nk1n(cid:48)
k2

with nki = Number of times rater i picked catagory k. po is deﬁned by

,

κ ≡ po − pe
1 − pe


1
N2

k

(F.11)

(F.12)

(F.13)

Nagree
Ntotal
This can be illustrated by an example from our analysis.

po =

,

Table F.4: How each researcher coded the set of responses into the ﬁve diﬀerent categories.

Code
LT
Interest
CM
NE
Other
Total

Internal Researcher External Researcher
1
6
6
11
9
33

2
7
6
9
9
33

There are 5 possible categories k = 5 and 33 items Ntotal = 33
Table F.5: nk1nk2 for each category.

LT Interest CM NE Other
2

36

42

99

81

nk1nk2


k

pe =

1
N2

nk1nk2 =

260
332 = 0.24

160

(F.14)

po =

Nagree
Ntotal

31
33 = 0.94

=

κ ≡ po − pe
1 − pe

0.94 − 0.24
1 − 0.24 = 0.92

=

(F.15)

(F.16)

Our results is κ = 0.92 and a κ between 0.81-1.0 almost perfect agreement [84].

161

APPENDIX G

SURVEY QUESTIONS REFERENCE TABLE

G.1 Table of Survey Questions

Reference table of every survey question, whether it was kept or removed at each step and the

reason for removal. Speciﬁc questions are referenced in Chapter 6.

Table G.1: Survey questions from every version and the reason for removal.

Reference Question

Version 2 Version 3 Version 4

Reason for Removal

C1

C2

C3

C4

C5

C6

C7

C8
C9

C10

C11

Explaining what I found is impor-
tant
Explaining what I was trying to
do is important
Explaining what it means is im-
portant
Explaining my results is impor-
tant
Explaining my interpretation of
the results is important
Communicating my results by an-
swering speciﬁc questions is help-
ful
Presenting what my results mean
is important
Presenting what I did is important
Communicating my results by
writing in my lab notebook is
helpful
Communicating my results by
writing up a conclusion is help-
ful
Communicating my results with
data tables graphs and pictures is
important

Communicating

x

x

Removed

x

x

-

x

x

x

x

x
x

x

Removed

Removed

Removed

Removed

x
x

x

x

x

Removed

-

-

-

-

-

x
x

Statistically and visually similar
distributions
Too vague

Similar distributions

Similar distributions

Similar distributions

Similar distributions

x
x

Removed

Statistically and visually similar
distributions

Removed

-

-

Was not accomplishing intended
goal

162

Reference Question
C12

Table G.1 (cont’d)

Version 2 Version 3 Version 4
x

x

x

Reason for Removal
x

x

x

x

x

Removed

Being able to communicate my
understanding is important
Explaining what I did is important

Explaining why I did what I did is
important
Explaining how I did what I did is
important
Explaining what my results mean
is important
Communicating my results with
conﬁdence is important
Using uncertainty to express the
conﬁdence in my results is helpful

Comparing my results to the ex-
pected is important
Determining if my data matches
what I expected is important
Making connections to the bigger
concepts is valuable
Using the concepts to make my
data into something I can interpret
is important
Using graphs to interpret results
is important
Making graphs from collected
data is helpful
Interpreting graphs to understand
relationships between parameters
is important
Doing calculations with data is
important
Doing math with my data in order
to get my results is important

x

x

x

x

x

x

x

x

x

C13

C14

C15

C16

C17

C18

DA1

DA2

DA3

DA4

DA5

DA6

DA7

DA8

DA9

x

Removed

Removed

Removed

-

-

Statistically and visually similar
distributions
Similar distributions

Similar distributions

x

-

x

x

-

x

Data Analysis

Removed

Removed

-

Removed

-

x

x

-

Removed

Statistically and visually similar
distributions
Misinterpreting conﬁdence

x

x

No examples of identity

Covered by other questions in the
practice group
Similar distributions

Removed

Statistically similar distributions

x

x

Removed

Statistically and visually similar
distributions

-

Mistakenly dropped from 1 to 2

Removed

Statistically and visually similar
distributions

x

x

x

x

x

163

Reference Question
DA10

DA11

DA12

DA13

DA14

DA15

DA16

DA17

DA18

DA19

EE1
EE2

EE3

EE4

Using math as a tool to explain
and predict is helpful
Looking for trends in my data is
important
Using math to apply a pattern to
what I saw is valuable
Calculating the average value for
each variable tested/measured is
important

Using data to determine the sig-
niﬁcance of my results is impor-
tant
Observing trends and patterns
in the experiment and applying
equations to that is important
Determining patterns from my
data is important
Looking for trends or asking why
something happened mathemati-
cally is important
Testing the validity of my results
with uncertainty is important
Expressing conﬁdence in my re-
sults with uncertainty is important

Doing calculations is helpful
Working with the equipment is
valuable
Collecting data and taking mea-
surements is important
The fact that uncertainty means
measurements may be oﬀ inaccu-
rate or wrong is important

Reason for Removal
x

No examples of identity

Similar distributions

"They usually disagree with it be-
cause it is not actually an expert
practice it is not providing useful
information"
Signiﬁcance could be misinter-
preted

Similar distributions

x

Similar questions

Similar distributions

Nearly the same as a communi-
cating question

Too vague
x

x

Similar distributions

Table G.1 (cont’d)

Version 2 Version 3 Version 4
x

x

x

-

-

-

-

-

x

-

-

-

-
x

x

-

Removed

-

x

x

x

x

x

x

x

x

Removed

Removed

Removed

Removed

x

Removed

Removed

Removed

Executing Experiments
Removed
x

-
x

x

x

x

Removed

164

Reference Question
EE5

Knowing that the uncertainty my
have an impact on the experiment
is important
there is uncer-
Knowing that
tainty associated with our mea-
surements is important
Working to minimize the uncer-
tainty in my experiment is impor-
tant

Caring about how each member
of my group sees my work/eﬀort
is important
Being accountable to my group is
valuable
Figuring out how to communicate
with my group is important
Figuring out how to work with my
group is important
Communicating with my group
about the results is important
Communicating with my group
about how to do things is impor-
tant
Communicating with my group
about how things work is impor-
tant
Talking things out with my group
is helpful
Everyone
feeling comfortable
sharing their ideas in their group
is important
Everyone in the group contribut-
ing an equal amount is important
Giving everyone in the group a
chance to contribute is important

EE6

EE7

GW1

GW2

GW3

GW4

GW5

GW6

GW7

GW8

GW9

GW10

GW11

Table G.1 (cont’d)

Version 2 Version 3 Version 4
x
Removed

x

Reason for Removal
Statistically and visually similar
distributions

Removed

-

Similar distributions

x

x

x

x

Group Work

x

Removed

-

x

Removed

Removed

x

x

x

-

x

Removed

Removed

Removed

x

x

x

-

x

x

x

165

x

-

-

-

-

Removed

-

-

-

x

-

x

Mistakenly dropped from 1 to 2

No examples of identity

Similar questions

Disregarding ’ﬁguring out’

Statistically and visually similar
distributions
Similar distributions

Similar distributions

Answered ’yes and no’ a lot

x

Mistakenly dropped from 1 to 2

Removed

Statistically similar distributions

Reference Question
GW12

Table G.1 (cont’d)

Version 2 Version 3 Version 4
Removed

-

-

Everyone in the group agreeing
on the goal is important
Following one group member’s
interpretation of the goal is im-
portant
Having help from my group to
check my work is important
Having help from my group if I
have questions is important
Bringing many ideas together in a
group is important
Bouncing ideas oﬀ of my group
members is important
Working in a group is important
Working together as a group is im-
portant
Making decisions together as a
group is important
Letting each group member have
hands-on time with the equipment
is important
Agreeing as a group on how to
complete a task before moving
forward is important

Removed

-

x

x

x

x

Removed

x

Removed

x

Removed
x

-
Removed

x

x

x

x

Removed

Removed

GW13

GW14

GW15

GW16

GW17

GW18
GW19

GW20

GW21

GW22

GW23

GW24

GW25

PE1

PE2

Reason for Removal
No examples of identity

No examples of identity

Similar distributions

x

Similar distributions

x

-
Similar questions

x

Similar distributions

Not an important practice re-
ﬂected in free-response state-
ments also ambiguous in what a
preferred response would be
No examples of identity

No examples of identity

-

-

x

-

x

-
-

x

-

-

-

-

Everyone in the group doing their
own thing is important
Making decisions for the group by
myself is important
Valuing every group member’s
ideas is important

Removed

Removed

x

-

-

x

Removed

Statistically and visually similar
distributions

Having a plan to answer my ques-
tions is important
Having a plan to get to my goal is
important

Planning Experiments
x

x

Removed

x

Removed

-

Statistically and visually similar
distributions
Similar distributions

166

Reference Question
PE3

Knowing what equipment I need
for the experiment I am planning
is important
Knowing what design works best
for the experiment I am planning
is important
Having a procedure when design-
ing an experiment is important
Creating a procedure when de-
signing an experiment is impor-
tant
Planning out the steps when de-
signing an experiment is impor-
tant
Figuring out how to get the infor-
mation I need when desiging an
experiment is important
Figuring out how to set up the
equipment when planning an ex-
periment is important
Figuring out how to get the equip-
ment and measurement devices
into the system when designing
an experiment is important
Having an hypothesis when plan-
ning an experiment is important
Knowing what my goal is when
planning an experiment is impor-
tant
Knowing what I am trying to ﬁnd
out is important
Knowing how to get to my goal
when planning an experiment is
important

PE4

PE5

PE6

PE7

PE8

PE9

PE10

PE11

PE12

PE13

PE14

Table G.1 (cont’d)

Version 2 Version 3 Version 4
x
Removed

x

Reason for Removal
Statistically and visually similar
distributions

x

x

Removed

-

-

Statistically and visually similar
distributions
No examples of identity

No examples of identity

Removed

Statistically and visually similar
distributions

x

-

x

x

Some confusion on wording and
some misinterpretation

x

Removed

Statistically and visually similar
distributions

-

-

No examples of identity

Similar questions

x

x

Removed

Removed

x

x

x

x

-

-

x

x

Removed

-

x

x

x

x

Removed

-

x

Removed

Understanding Equipment

167

Reference Question
UE1

Table G.1 (cont’d)

Version 2 Version 3 Version 4
x

x

x

Reason for Removal
x

UE2

UE3

UE4

UE5

UE6

UE7

UE8

UE9

UE10

UE11

UE12

UE13

UE14

UE15

Knowing that the equipment may
have multiple uses is important
Knowing how to adapt equipment
to new situations is useful
Trying out the equipment and see-
ing what it does is valuable
Tinkering with the equipment is
useful
Seeing how changing one com-
ponent of the equipment aﬀects
something else is useful
Figuring out how the equipment
works best for me is important
Identifying the extreme settings
of the equipment is important
Testing the tolerance of the equip-
ment is important
Seeing how the equipment works
is helpful
Knowing how the equipment
works is important
Knowing what the equipment is
designed to measure is valuable
Knowing how the parts of the
equipment work is valuable
Knowing how to use the equip-
ment is important

Knowing how to get what I need
from the equipment is important
Knowing how to turn the equip-
ment on and plug it in is impor-
tant

x

x

x

x

Removed

Covered by other questions in the
practice group
Similar questions

Removed

-

Removed

Statistically and visually similar
distributions

-

Removed

-

-

x

Removed

-

-

-

-

Not an ideal question for intro stu-
dents, No examples of identity
Covered by other questions in the
practice group
Tolerance is too jargony, No ex-
amples of identity
Too vague, a lot of varity in re-
sponses No examples of identity
x

Statistically and visually similar
distributions
Similar distributions

A lot on correct/accurate exper-
iments/results, No examples of
identity
Similar distributions

No examples of identity

x

x

x

x

Removed

x

Removed

Removed

x

x

x

x

-

x

-

-

x

x

Removed

Removed

-

x

Removed

Removed

-

168

Reason for Removal
A lot of people seem to interpret
this as ’ﬁguring out how to use
the equipment’ and ignoring the
’I want’ part
Most people disagree with this be-
cause it is not a common practice
in lab classes so this question is
not useful
Similar distributions

x

Not everyone is connecting this to
uncertainty, No examples of iden-
tity
Apparatus is too jargony, No ex-
amples of identity
No examples of identity

No examples of identity

No examples of identity

x

Table G.1 (cont’d)

Reference Question
UE16

Figuring out how I want to use it
the equipment is important

Version 2 Version 3 Version 4
Removed

-

-

UE17

UE18

UE19

UE20

UE21

UE22

UE23

UE24

UE25

Making the equipment measure
something it was not originally in-
tended for is important

Knowing what not to do with the
equipment is useful
Knowing the limits/capabilities of
the equipment is useful
Understanding the limitations of
the measurements the equipment
can make is important
Understanding the limited preci-
sion of an apparatus is important
Knowing what the equipment is
used for is important
Knowing how the equipment can
be used is important
Knowing what the equipment can
do/measure is important
Knowing how to ﬁx the equip-
ment when it is broken is useful

x

x

x

Removed

Removed

Removed

Removed

Removed

x

Removed

-

Removed

x

-

-

-

-

-

x

-

x

-

-

-

-

-

x

169

APPENDIX H

SURVEY VERSION 1

This appendix has the entire survey from Version 1 (Sec. 6.3). This is how it would have been
viewed by each student on Qualtrics1. For each practice each student saw a random question from
the question group. Each item is idicated by:
‘Practice Abbriviation’ + ‘Question Name’ + ‘Question Group’
A group work item for example looks like:
GW + Help 1 + Q1
That is a group work question (the question name is irrelavant) in the ﬁrst group. Each student saw
one group work question in question group one. See Chapter 6 Sec. 6.3 for details on this version
of the survey.

1Qualtrics, 2019, Qualtrics Labs, Inc. Provo, UT, USA

170

171

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview3/104about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517­355­2180, Fax 517­432­4503, or e­mail irb@msu.edu or regular mail at Olds Hall, 408 West Circle Drive #207, MSU, East Lansing, MI 48824. 9.  DOCUMENTATION OF INFORMED CONSENT. Your affirmations below means that you voluntarily agree to participate in this research study. Do you agree to participate in the study? An answer is required to receive extra credit in your course.(If you choose 'No' you will be redirected to the end of the survey) Intro BlockWhat physics class(es) are you in currently?What is your current major?Why did you choose to major in science?What is your interest in physics?YesNoVery lowLow172

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview4/104What are your future plans?Community BlockAs an undergraduate student at Michigan State University taking science classes you arepart of many diﬀerent communities. These communities may inﬂuence your perspective asyou go through your undergraduate career. Below we have some examples of communitiesthat may be relevant to you, please choose all that apply and add on if we are missingsomething.Choose all that applyPLEASE READ BEFORE PROCEEDING The rest of the questions are focused on your experience in physics labs. A series of fourquestions will be addressing the same idea. There are no wrong answers, we are justlooking for your opinion on these things. We are trying to understand your reasoning, so it isimportant that you answer the free-response questions as well as the multiple-choiceones. Even if your reasoning seems obvious to you it may not be clear to us. This will look like:Do YOU agree/disagree with this statement?Would an EXPERIMENTAL PHYSICIST agree/disagree with this statement?ModerateHighVery high Science studentAn aware citizenPerson in my majorCurious personA person with a future in${q://QID4/ChoiceGroup/SelectedChoices}Other: 173

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview5/104Explain why YOU agree/disagree with the statementExplain why you think an EXPERIMENTAL PHYSICIST would agree/disagree with thisstatementGW NA 1 Q2 - Apr 19, 2018Working in a group is importantFor ME, working in a group is important (not important) because...For an EXPERIMENTAL PHYSICIST, working in a group is important (not important)because...GW Adapt 1 Q2 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  174

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview6/104Figuring out how to communicate with my group is importantFor ME, ﬁguring out how to communicate with my group is important (not important)because...For an EXPERIMENTAL PHYSICIST, ﬁguring out how to communicate with my group isimportant (not important) because...GW Accountable 1 Q2 - Apr 19, 2018Caring about how each member of my group sees my work/eﬀort is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  175

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview7/104For ME, caring about how each member of my group sees my work/eﬀort is important (notimportant) because...For an EXPERIMENTAL PHYSICIST, caring about how each member of my group sees mywork/eﬀort is important (not important) because...GW Accountable 2 Q2 - Apr 19, 2018Being accountable to my group is valuable   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  176

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview8/104For ME, being accountable to my group is valuable (not valuable) because...For an EXPERIMENTAL PHYSICIST, being accountable to my group is valuable (notvaluable) because...GW Help 1 Q2 - Apr 19, 2018Having help from my group to check my work is importantFor ME, having help from my group to check my work is important (not important)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  177

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview9/104For an EXPERIMENTAL PHYSICIST, having help from my group to check my work isimportant (not important) because...GW Help 2 Q2 - Apr 19, 2018Having help from my group if I have questions is importantFor ME, having help from my group if I have questions is important (not important)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  178

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview10/104For an EXPERIMENTAL PHYSICIST, having help from my group if I have questions isimportant (not important) because...GW Goal 1 Q2 - Apr 19, 2018Everyone in the group agreeing on the goal is importantFor ME, everyone in the group agreeing on the goal is important (not important) because...For an EXPERIMENTAL PHYSICIST, everyone in the group agreeing on the goal is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  179

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview11/104GW Goal 2 Q2 - Apr 19, 2018Following one group member's interpretation of the goal is importantFor ME, following one group member's interpretation of the goal is important (not important)because...For an EXPERIMENTAL PHYSICIST, following one group member's interpretation of thegoal is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  180

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview12/104GW Together 1 Q2 - Apr 19, 2018Working together as a group is importantFor ME, working together as a group is important (not important) because...For an EXPERIMENTAL PHYSICIST, working together as a group is important (notimportant) because...GW Together 2 Q2 - Apr 19, 2018Making decisions together as a group is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  181

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview13/104For ME, making decisions together as a group is important (not important) because...For an EXPERIMENTAL PHYSICIST, making decisions together as a group is important (notimportant) because...GW Adapt 2 Q2 - Apr 19, 2018Figuring out how to work with my group is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  182

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview14/104For ME, ﬁguring out how to work with my group is important (not important) because...For an EXPERIMENTAL PHYSICIST, ﬁguring out how to work with my group is important(not important) because...GW Ideas 1 Q1 - Apr 19, 2018Bringing many ideas together in a group is importantFor ME, bringing many ideas together in a group is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  183

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview15/104For an EXPERIMENTAL PHYSICIST, bringing many ideas together in a group is important(not important) because...GW Ideas 2 Q1 - Apr 19, 2018Bouncing ideas oﬀ of my group members is importantFor ME, bouncing ideas oﬀ of my group members is important (not important) because...For an EXPERIMENTAL PHYSICIST, bouncing ideas oﬀ of my group members is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  184

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview16/104GW Communication 1 Q1 - Apr 19, 2018Communicating with my group about the results is importantFor ME, communicating with my group about the results is important (not important)because...For an EXPERIMENTAL PHYSICIST, communicating with my group about the results isimportant (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  185

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview17/104GW Communication 2 Q1 - Apr 19, 2018Communicating with my group about how to do things is importantFor ME, communicating with my group about how to do things is important (not important)because...For an EXPERIMENTAL PHYSICIST, communicating with my group about how to do thingsis important (not important) because...GW Communication 3 Q1 - Apr 19, 2018Communicating with my group about how things work is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagree186

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview18/104For ME, communicating with my group about how things work is important (not important)because...For an EXPERIMENTAL PHYSICIST, communicating with my group about how things workis important (not important) because...GW Communication 4 Q1 - Apr 19, 2018Talking things out with my group is helpful   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  187

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview19/104For ME, talking things out with my group is helpful (not helpful) because...For an EXPERIMENTAL PHYSICIST, talking things out with my group is helpful (not helpful)because...GW Communication 5 Q1 - Apr 19, 2018Everyone feeling comfortable sharing their ideas in their group is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  188

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview20/104For ME, everyone feeling comfortable sharing their ideas in their group is important (notimportant) because...For an EXPERIMENTAL PHYSICIST, everyone feeling comfortable sharing their ideas in theirgroup is important (not important) because...GW Together 3 Q1 - Apr 19, 2018Letting each group member have hands-on time with the equipment is importantFor ME, letting each group member have hands-on time with the equipment is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  189

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview21/104For an EXPERIMENTAL PHYSICIST, letting each group member have hands-on time withthe equipment is important (not important) because...GW Together 4 Q1 - Apr 19, 2018Agreeing as a group on how to complete a task before moving forward is importantFor ME, agreeing as a group on how to complete a task before moving forward is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  190

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview22/104For an EXPERIMENTAL PHYSICIST, agreeing as a group on how to complete a task beforemoving forward is important (not important) because...GW Together 5 Q1 - Apr 19, 2018Everyone in the group doing their own thing is importantFor ME, everyone in the group doing their own thing is important (not important) because...For an EXPERIMENTAL PHYSICIST, everyone in the group doing their own thing isimportant (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  191

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview23/104GW Together 6 Q1 - Apr 19, 2018Making decisions for the group by myself is importantFor ME, making decisions for the group by myself is important (not important) because...For an EXPERIMENTAL PHYSICIST, making decisions for the group by myself is important(not important) because...GW Together 7 Q1 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  192

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview24/104Valuing every group member's ideas is importantFor ME, valuing every group member's ideas is important (not important) because...For an EXPERIMENTAL PHYSICIST, valuing every group member's ideas is important (notimportant) because...GW Contribution 1 Q1 - Apr 19, 2018Everyone in the group contributing an equal amount is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  193

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview25/104For ME, everyone in the group contributing an equal amount is important (not important)because...For an EXPERIMENTAL PHYSICIST, everyone in the group contributing an equal amount isimportant (not important) because...GW Contribution 2 Q1 - Apr 19, 2018Giving everyone in the group a chance to contribute is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  194

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview26/104For ME, giving everyone in the group a chance to contribute is important (not important)because...For an EXPERIMENTAL PHYSICIST, giving everyone in the group a chance to contribute isimportant (not important) because...UE How it works 1 Q1 - Apr 19, 2018Seeing how the equipment works is helpfulFor ME, seeing how the equipment works is helpful (not helpful) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  195

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview27/104For an EXPERIMENTAL PHYSICIST, seeing how the equipment works is helpful (not helpful)because...UE How it works 2 Q1 - Apr 19, 2018Knowing how the equipment works is importantFor ME, knowing how the equipment works is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  196

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview28/104For an EXPERIMENTAL PHYSICIST, knowing how the equipment works is important (notimportant) because...UE How it works 3 Q1 - Apr 19, 2018Knowing what the equipment is designed to measure is valuableFor ME, knowing what the equipment is designed to measure is valuable (not valuable)because...For an EXPERIMENTAL PHYSICIST, knowing what the equipment is designed to measure isvaluable (not valuable) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  197

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview29/104UE How it works 4 Q1 - Apr 19, 2018knowing how the parts of the equipment work is valuableFor ME, knowing how the parts of the equipment work is valuable (not valuable) because...For an EXPERIMENTAL PHYSICIST, knowing how the parts of the equipment work isvaluable (not valuable) because...UE How to use it 1 Q1 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  198

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview30/104Knowing how to use the equipment is importantFor ME, knowing how to use the equipment is important (not important) because...For an EXPERIMENTAL PHYSICIST, knowing how to use the equipment is important (notimportant) because...UE How to use it 2 Q1 - Apr 19, 2018Knowing how to get what I need from the equipment is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  199

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview31/104For ME, knowing how to get what I need from the equipment is important (not important)because...For an EXPERIMENTAL PHYSICIST, knowing how to get what I need from the equipment isimportant (not important) because...UE How to use it 3 Q1 - Apr 19, 2018Knowing how to turn the equipment on and plug it in is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  200

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview32/104For ME, knowing how to turn the equipment on and plug it in is important (not important)because...For an EXPERIMENTAL PHYSICIST, knowing how to turn the equipment on and plug it in isimportant (not important) because...UE Exploring 1 Q1 - Apr 19, 2018Trying out the equipment and seeing what it does is valuableFor ME, trying out the equipment and seeing what it does is valuable (not valuable)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  201

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview33/104For an EXPERIMENTAL PHYSICIST, trying out the equipment and seeing what it does isvaluable (not valuable) because...UE Exploring 2 Q1 - Apr 19, 2018Tinkering with the equipment is usefulFor ME, tinkering with the equipment is useful (not useful) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  202

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview34/104For an EXPERIMENTAL PHYSICIST, tinkering with the equipment is useful (not useful)because...UE Exploring 3 Q1 - Apr 19, 2018Seeing how changing one component of the equipment aﬀects something else is usefulFor ME, seeing how changing one component of the equipment aﬀects something else isuseful (not useful) because...For an EXPERIMENTAL PHYSICIST, seeing how changing one component of the equipmentaﬀects something else is useful (not useful) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  203

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview35/104UE Purpose 1 Q1 - Apr 19, 2018Knowing what the equipment is used for is importantFor ME, knowing what the equipment is used for is important (not important) because...For an EXPERIMENTAL PHYSICIST, knowing what the equipment is used for is important(not important) because...UE Purpose 2 Q1 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  204

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview36/104Knowing how the equipment can be used is importantFor ME, knowing how the equipment can be used is important (not important) because...For an EXPERIMENTAL PHYSICIST, knowing how the equipment can be used is important(not important) because...UE Purpose 3 Q1 - Apr 19, 2018Knowing what the equipment can do/measure is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  205

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview37/104For ME, knowing what the equipment can do/measure is important (not important)because...For an EXPERIMENTAL PHYSICIST, knowing what the equipment can do/measure isimportant (not important) because...UE How to use it 4 Q2 - Apr 19, 2018Figuring out how I want to use it the equipment is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  206

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview38/104For ME, ﬁguring out how I want to use it the equipment is important (not important)because...For an EXPERIMENTAL PHYSICIST, ﬁguring out how I want to use it the equipment isimportant (not important) because...UE How to use it 5 Q2 - Apr 19, 2018Making the equipment measure something it was not originally intended for is importantFor ME, making the equipment measure something it was not originally intended for isimportant (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  207

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview39/104For an EXPERIMENTAL PHYSICIST, making the equipment measure something it was notoriginally intended for is important (not important) because...UE Exploring 4 Q2 - Apr 19, 2018Figuring out how the equipment works best for me is importantFor ME, ﬁguring out how the equipment works best for me is important (not important)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  208

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview40/104For an EXPERIMENTAL PHYSICIST, ﬁguring out how the equipment works best for me isimportant (not important) because...UE Exploring 5 Q2 - Apr 19, 2018Identifying the extreme settings of the equipment is importantFor ME, identifying the extreme settings of the equipment is important (not important)because...For an EXPERIMENTAL PHYSICIST, identifying the extreme settings of the equipment isimportant (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  209

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview41/104UE Exploring 6 Q2 - Apr 19, 2018Testing the tolerance of the equipment is importantFor ME, testing the tolerance of the equipment is important (not important) because...For an EXPERIMENTAL PHYSICIST, testing the tolerance of the equipment is important (notimportant) because...UE Limitations 1 Q2 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  210

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview42/104Knowing what not to do with the equipment is usefulFor ME, knowing what not to do with the equipment is useful (not useful) because...For an EXPERIMENTAL PHYSICIST, knowing what not to do with the equipment is useful(not useful) because...UE Limitations 2 Q2 - Apr 19, 2018Knowing the limits/capabilities of the equipment is useful   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  211

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview43/104For ME, knowing the limits/capabilities of the equipment is useful (not useful) because...For an EXPERIMENTAL PHYSICIST, knowing the limits/capabilities of the equipment isuseful (not useful) because...UE Limitations 3 Q2 - Apr 19, 2018Understanding the limitations of the measurements the equipment can make is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  212

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview44/104For ME, understanding the limitations of the measurements the equipment can make isimportant (not important) because...For an EXPERIMENTAL PHYSICIST, understanding the limitations of the measurements theequipment can make is important (not important) because...UE Limitations 4 Q2 - Apr 19, 2018Understanding the limited precision of an apparatus is importantFor ME, understanding the limited precision of an apparatus is important (not important)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  213

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview45/104For an EXPERIMENTAL PHYSICIST, understanding the limited precision of an apparatus isimportant (not important) because...UE Adapting 1 Q2 - Apr 19, 2018Knowing that the equipment may have multiple uses is importantFor ME, knowing that the equipment may have multiple uses is important (not important)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  214

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview46/104For an EXPERIMENTAL PHYSICIST, knowing that the equipment may have multiple uses isimportant (not important) because...UE Adapting 2 Q2 - Apr 19, 2018Knowing how to adapt equipment to new situations is usefulFor ME, knowing how to adapt equipment to new situations is useful (not useful) because...For an EXPERIMENTAL PHYSICIST, knowing how to adapt equipment to new situations isuseful (not useful) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  215

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview47/104UE Troubleshooting 1 Q2 - Apr 19, 2018Knowing how to ﬁx the equipment when it is broken is usefulFor ME, knowing how to ﬁx the equipment when it is broken is useful (not useful) because...For an EXPERIMENTAL PHYSICIST, knowing how to ﬁx the equipment when it is broken isuseful (not useful) because...C Presenting 6 Q1 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  216

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview48/104Explaining what I did is importantFor ME, explaining what I did is important (not important) because...For an EXPERIMENTAL PHYSICIST, explaining what I did is important (not important)because...C Presenting 7 Q1 - Apr 19, 2018Explaining why I did what I did is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  217

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview49/104For ME, explaining why I did what I did is important (not important) because...For an EXPERIMENTAL PHYSICIST, explaining why I did what I did is important (notimportant) because...C Presenting 8 Q1 - Apr 19, 2018Explaining how I did what I did is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  218

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview50/104For ME, explaining how I did what I did is important (not important) because...For an EXPERIMENTAL PHYSICIST, explaining how I did what I did is important (notimportant) because...C Presenting 9 Q1 - Apr 19, 2018Explaining what my results mean is importantFor ME, explaining what my results mean is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  219

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview51/104For an EXPERIMENTAL PHYSICIST, explaining what my results mean is important (notimportant) because...C Presenting 10 Q1 - Apr 19, 2018Presenting what my results mean is importantFor ME, presenting what my results mean is important (not important) because...For an EXPERIMENTAL PHYSICIST, presenting what my results mean is important (notimportant) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  220

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview52/104C Presenting 11 Q1 - Apr 19, 2018Presenting what I did is importantFor ME, presenting what I did is important (not important) because...For an EXPERIMENTAL PHYSICIST, presenting what I did is important (not important)because...C Explaining 1 Q1 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  221

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview53/104Explaining what I found is importantFor ME, explaining what I found is important (not important) because...For an EXPERIMENTAL PHYSICIST, explaining what I found is important (not important)because...C Explaining 2 Q1 - Apr 19, 2018Explaining what I was trying to do is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  222

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview54/104For ME, explaining what I was trying to do is important (not important) because...For an EXPERIMENTAL PHYSICIST, explaining what I was trying to do is important (notimportant) because...C Explaining 3 Q1 - Apr 19, 2018Explaining what it means is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  223

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview55/104For ME, explaining what it means is important (not important) because...For an EXPERIMENTAL PHYSICIST, explaining what it means is important (not important)because...C Explaining 4 Q1 - Apr 19, 2018Explaining my results is importantFor ME, explaining my results is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  224

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview56/104For an EXPERIMENTAL PHYSICIST, explaining my results is important (not important)because...C Explaining 5 Q1 - Apr 19, 2018Explaining my interpretation of the results is importantFor ME, explaining my interpretation of the results is important (not important) because...For an EXPERIMENTAL PHYSICIST, explaining my interpretation of the results is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  225

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview57/104C Presenting 1 Q2 - Apr 19, 2018Communicating my results by answering speciﬁc questions is helpfulFor ME, communicating my results by answering speciﬁc questions is helpful (not helpful)because...For an EXPERIMENTAL PHYSICIST, communicating my results by answering speciﬁcquestions is helpful (not helpful) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  226

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview58/104C Presenting 2 Q2 - Apr 19, 2018Communicating my results by writing in my lab notebook is helpfulFor ME, communicating my results by writing in my lab notebook is helpful (not helpful)because...For an EXPERIMENTAL PHYSICIST, communicating my results by writing in my labnotebook is helpful (not helpful) because...C Presenting 3 Q2 - Apr 19, 2018Communicating my results by writing up a conclusion is helpful   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagree227

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview59/104For ME, communicating my results by writing up a conclusion is helpful (not helpful)because...For an EXPERIMENTAL PHYSICIST, communicating my results by writing up a conclusion ishelpful (not helpful) because...C Presenting 4 Q2 - Apr 19, 2018Communicating my results with data tables, graphs, and pictures is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  228

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview60/104For ME, communicating my results with data tables, graphs, and pictures is important (notimportant) because...For an EXPERIMENTAL PHYSICIST, communicating my results with data tables, graphs,and pictures is important (not important) because...C Presenting 5 Q2 - Apr 19, 2018Being able to communicate my understanding is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  229

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview61/104For ME, being able to communicate my understanding is important (not important)because...For an EXPERIMENTAL PHYSICIST, being able to communicate my understanding isimportant (not important) because...C Uncertainty 1 Q2 - Apr 19, 2018Communicating my results with conﬁdence is importantFor ME, communicating my results with conﬁdence is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  230

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview62/104For an EXPERIMENTAL PHYSICIST, communicating my results with conﬁdence is important(not important) because...C Uncertainty 2 Q2 - Apr 19, 2018Using uncertainty to express the conﬁdence in my results is helpfulFor ME, using uncertainty to express the conﬁdence in my results is helpful (not helpful)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  231

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview63/104For an EXPERIMENTAL PHYSICIST, using uncertainty to express the conﬁdence in myresults is helpful (not helpful) because...PE What 1 Q1 - Apr 19, 2018Having an hypothesis when planning an experiment is importantFor ME, having an hypothesis when planning an experiment is important (not important)because...For an EXPERIMENTAL PHYSICIST, having an hypothesis when planning an experiment isimportant (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  232

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview64/104PE What 2 Q1 - Apr 19, 2018Knowing what my goal is when planning an experiment is importantFor ME, knowing what my goal is when planning an experiment is important (not important)because...For an EXPERIMENTAL PHYSICIST, knowing what my goal is when planning an experimentis important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  233

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview65/104PE What 3 Q1 - Apr 19, 2018Knowing what I am trying to ﬁnd out is importantFor ME, knowing what I am trying to ﬁnd out is important (not important) because...For an EXPERIMENTAL PHYSICIST, knowing what I am trying to ﬁnd out is important (notimportant) because...PE What 4 Q1 - Apr 19, 2018Knowing how to get to my goal when planning an experiment is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  234

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview66/104For ME, knowing how to get to my goal when planning an experiment is important (notimportant) because...For an EXPERIMENTAL PHYSICIST, knowing how to get to my goal when planning anexperiment is important (not important) because...PE How 1 Q1 - Apr 19, 2018Having a plan to answer my questions is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  235

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview67/104For ME, having a plan to answer my questions is important (not important) because...For an EXPERIMENTAL PHYSICIST, having a plan to answer my questions is important (notimportant) because...PE How 2 Q1 - Apr 19, 2018Having a plan to get to my goal is importantFor ME, having a plan to get to my goal is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  236

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview68/104For an EXPERIMENTAL PHYSICIST, having a plan to get to my goal is important (notimportant) because...PE How 3 Q1 - Apr 19, 2018Knowing what equipment I need for the experiment I am planning is importantFor ME, knowing what equipment I need for the experiment I am planning is important (notimportant) because...For an EXPERIMENTAL PHYSICIST, knowing what equipment I need for the experiment Iam planning is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  237

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview69/104PE How 4 Q1 - Apr 19, 2018Knowing what design works best for the experiment I am planning is importantFor ME, knowing what design works best for the experiment I am planning is important (notimportant) because...For an EXPERIMENTAL PHYSICIST, knowing what design works best for the experiment Iam planning is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  238

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview70/104PE Procedure 1 Q1 - Apr 19, 2018Having a procedure when designing an experiment is importantFor ME, having a procedure when designing an experiment is important (not important)because...For an EXPERIMENTAL PHYSICIST, having a procedure when designing an experiment isimportant (not important) because...PE Procedure 2 Q1 - Apr 19, 2018Creating a procedure when designing an experiment is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagree239

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview71/104For ME, creating a procedure when designing an experiment is important (not important)because...For an EXPERIMENTAL PHYSICIST, creating a procedure when designing an experiment isimportant (not important) because...PE Procedure 3 Q1 - Apr 19, 2018Planning out the steps when designing an experiment is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  240

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview72/104For ME, planning out the steps when designing an experiment is important (not important)because...For an EXPERIMENTAL PHYSICIST, planning out the steps when designing an experimentis important (not important) because...PE Procedure 4 Q1 - Apr 19, 2018Figuring out how to get the information I need when desiging an experiment is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  241

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview73/104For ME, ﬁguring out how to get the information I need when desiging an experiment isimportant (not important) because...For an EXPERIMENTAL PHYSICIST, ﬁguring out how to get the information I need whendesiging an experiment is important (not important) because...PE Procedure 5 Q1 - Apr 19, 2018Figuring out how to set up the equipment when planning an experiment is importantFor ME, ﬁguring out how to set up the equipment when planning an experiment is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  242

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview74/104For an EXPERIMENTAL PHYSICIST, ﬁguring out how to set up the equipment whenplanning an experiment is important (not important) because...PE Procedure 6 Q1 - Apr 19, 2018Figuring out how to get the equipment and measurement devices into the system whendesigning an experiment is importantFor ME, ﬁguring out how to get the equipment and measurement devices into the systemwhen designing an experiment is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  243

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview75/104For an EXPERIMENTAL PHYSICIST, ﬁguring out how to get the equipment andmeasurement devices into the system when designing an experiment is important (notimportant) because...EE Data 1 Q1 - Apr 19, 2018Doing calculations is helpfulFor ME, doing calculations is helpful (not helpful) because...For an EXPERIMENTAL PHYSICIST, doing calculations is helpful (not helpful) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  244

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview76/104EE Data 2 Q1 - Apr 19, 2018Working with the equipment is valuableFor ME, working with the equipment is valuable (not valuable) because...For an EXPERIMENTAL PHYSICIST, working with the equipment is valuable (not valuable)because...EE Data 3 Q1 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  245

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview77/104Collecting data and taking measurements is importantFor ME, collecting data and taking measurements is important (not important) because...For an EXPERIMENTAL PHYSICIST, collecting data and taking measurements is important(not important) because...EE Uncertainty 1 Q1 - Apr 19, 2018The fact that uncertainty means measurements may be oﬀ, inaccurate, or wrong isimportant   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  246

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview78/104For ME, the fact that uncertainty means measurements may be oﬀ, inaccurate, or wrong isimportant (not important) because...For an EXPERIMENTAL PHYSICIST, the fact that uncertainty means measurements may beoﬀ, inaccurate, or wrong is important (not important) because...EE Uncertainty 2 Q1 - Apr 19, 2018Knowing that the uncertainty my have an impact on the experiment is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  247

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview79/104For ME, knowing that the uncertainty my have an impact on the experiment is important(not important) because...For an EXPERIMENTAL PHYSICIST, knowing that the uncertainty my have an impact on theexperiment is important (not important) because...EE Uncertainty 3 Q1 - Apr 19, 2018Knowing that there is uncertainty associated with our measurements is importantFor ME, knowing that there is uncertainty associated with our measurements is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  248

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview80/104For an EXPERIMENTAL PHYSICIST, knowing that there is uncertainty associated with ourmeasurements is important (not important) because...EE Uncertainty 4 Q1 - Apr 19, 2018Working to minimize the uncertainty in my experiment is importantFor ME, working to minimize the uncertainty in my experiment is important (not important)because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  249

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview81/104For an EXPERIMENTAL PHYSICIST, working to minimize the uncertainty in my experimentis important (not important) because...DA Math 1 Q1 - Apr 19, 2018Doing calculations with data is importantFor ME, doing calculations with data is important (not important) because...For an EXPERIMENTAL PHYSICIST, doing calculations with data is important (notimportant) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  250

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview82/104DA Math 2 Q1 - Apr 19, 2018Doing math with my data in order to get my results is importantFor ME, doing math with my data in order to get my results is important (not important)because...For an EXPERIMENTAL PHYSICIST, doing math with my data in order to get my results isimportant (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  251

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview83/104DA Math 3 Q1 - Apr 19, 2018Using math as a tool to explain and predict is helpfulFor ME, using math as a tool to explain and predict is helpful (not helpful) because...For an EXPERIMENTAL PHYSICIST, using math as a tool to explain and predict is helpful(not helpful) because...DA Math 5 Q1 - Apr 19, 2018Using math to apply a pattern to what I saw is valuable   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  252

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview84/104For ME, using math to apply a pattern to what I saw is valuable (not valuable) because...For an EXPERIMENTAL PHYSICIST, using math to apply a pattern to what I saw is valuable(not valuable) because...DA Math 6 Q1 - Apr 19, 2018Calculating the average value for each variable tested/measured is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  253

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview85/104For ME, calculating the average value for each variable tested/measured is important (notimportant) because...For an EXPERIMENTAL PHYSICIST, calculating the average value for each variabletested/measured is important (not important) because...DA Connections 1 Q1 - Apr 19, 2018Making connections to the bigger concepts is valuableFor ME, making connections to the bigger concepts is valuable (not valuable) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  254

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview86/104For an EXPERIMENTAL PHYSICIST, making connections to the bigger concepts is valuable(not valuable) because...DA Connections 2 Q1 - Apr 19, 2018Using the concepts to make my data into something I can interpret is importantFor ME, using the concepts to make my data into something I can interpret is important (notimportant) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  255

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview87/104For an EXPERIMENTAL PHYSICIST, using the concepts to make my data into something Ican interpret is important (not important) because...DA Trends/Patterns 1 Q1 - Apr 19, 2018Observing trends and patterns in the experiment and applying equations to that is importantFor ME, observing trends and patterns in the experiment and applying equations to that isimportant (not important) because...For an EXPERIMENTAL PHYSICIST, observing trends and patterns in the experiment andapplying equations to that is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  256

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview88/104DA Trends/Patterns 2 Q1 - Apr 19, 2018Determining patterns from my data is importantFor ME, determining patterns from my data is important (not important) because...For an EXPERIMENTAL PHYSICIST, determining patterns from my data is important (notimportant) because...DA Trends/Patterns 3 Q1 - Apr 19, 2018   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  257

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview89/104Looking for trends or asking why something happened mathematically is importantFor ME, looking for trends or asking why something happened mathematically is important(not important) because...For an EXPERIMENTAL PHYSICIST, looking for trends or asking why something happenedmathematically is important (not important) because...DA Graphing 1 Q2 - Apr 19, 2018Using graphs to interpret results is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  258

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview90/104For ME, using graphs to interpret results is important (not important) because...For an EXPERIMENTAL PHYSICIST, using graphs to interpret results is important (notimportant) because...DA Graphing 2 Q2 - Apr 19, 2018Making graphs from collected data is helpful   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  259

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview91/104For ME, making graphs from collected data is helpful (not helpful) because...For an EXPERIMENTAL PHYSICIST, making graphs from collected data is helpful (nothelpful) because...DA Graphing 3 Q2 - Apr 19, 2018Interpreting graphs to understand relationships between parameters is importantFor ME, interpreting graphs to understand relationships between parameters is important(not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  260

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview92/104For an EXPERIMENTAL PHYSICIST, interpreting graphs to understand relationshipsbetween parameters is important (not important) because...DA Comparing 1 Q2 - Apr 19, 2018Comparing my results to the expected is importantFor ME, comparing my results to the expected is important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  261

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview93/104For an EXPERIMENTAL PHYSICIST, comparing my results to the expected is important (notimportant) because...DA Comparing 2 Q2 - Apr 19, 2018Determining if my data matches what I expected is importantFor ME, determining if my data matches what I expected is important (not important)because...For an EXPERIMENTAL PHYSICIST, determining if my data matches what I expected isimportant (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  262

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview94/104DA Signiﬁcance 3 Q2 - Apr 19, 2018Using data to determine the signiﬁcance of my results is importantFor ME, using data to determine the signiﬁcance of my results is important (not important)because...For an EXPERIMENTAL PHYSICIST, using data to determine the signiﬁcance of my resultsis important (not important) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  263

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview95/104DA Uncertainty 1 Q2 - Apr 19, 2018Testing the validity of my results with uncertainty is importantFor ME, testing the validity of my results with uncertainty is important (not important)because...For an EXPERIMENTAL PHYSICIST, testing the validity of my results with uncertainty isimportant (not important) because...DA Uncertainty 2 Q2 - Apr 19, 2018Expressing conﬁdence in my results with uncertainty is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagree264

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview96/104For ME, expressing conﬁdence in my results with uncertainty is important (not important)because...For an EXPERIMENTAL PHYSICIST, expressing conﬁdence in my results with uncertainty isimportant (not important) because...DA Math 4 Q2 - Apr 19, 2018Looking for trends in my data is important   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch     Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  265

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview97/104For ME, looking for trends in my data is important (not important) because...For an EXPERIMENTAL PHYSICIST, looking for trends in my data is important (notimportant) because...End Community BlockWhile taking this survey I was most frequently thinking of myself as a member of thiscommunity (select all that apply)Is there anything you would like to tell us about the experience of taking this survey   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat would anEXPERIMENTALPHYSICIST sayabout theirresearch  Science studentAn aware citizenPerson in my majorCurious personA person with a future in${q://QID4/ChoiceGroup/SelectedChoices}${q://QID28/ChoiceTextEntryValue/4}266

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview98/104Example BlockDoing calculations is helpfulFor ME, doing calculations is helpful (not helpful) because...For an EXPERIMENTAL PHYSICIST, doing calculations is helpful (not helpful) because...   Strongly agreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeWhat do YOUthink  What would anEXPERIMENTALPHYSICIST sayabout theirresearch  APPENDIX I

SURVEY VERSION 2

This appendix has the entire survey from Version 2 (Sec. 6.4). This is how it would have been
viewed by each student on Qualtrics1. The students saw a random practice question and four of the
ten reasoning questions. The practice questions are not individually labeled in this appendix, the
main purpose of this is to provide a visual of what the students saw when taking the survey.

1Qualtrics, 2019, Qualtrics Labs, Inc. Provo, UT, USA

267

268

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview3/26about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517­355­2180, Fax 517­432­4503, or e­mail irb@msu.edu or regular mail at Olds Hall, 408 West Circle Drive #207, MSU, East Lansing, MI 48824. 9.  DOCUMENTATION OF INFORMED CONSENT. Your affirmations below means that you voluntarily agree to participate in this research study. Do you agree to participate in the study? An answer is required.What physics class(es) are you in currently?What is your current major?Why did you choose to major in science?What is your interest in physics?YesNoVery lowLowModerateHighVery high269

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview4/26What are your future plans?If you selected OTHER please brieﬂy describe your future plansAs an undergraduate student taking science classes you are part of many diﬀerentcommunities. These communities may inﬂuence your perspective as you go through yourundergraduate career. Below we have some examples of communities that may be relevantto you, please choose all that apply and add on if we are missing something.I think of myself as a ... (Choose all that apply)When you think of your future career who do you imagine working with? (For example:Teacher work with students, doctors work with patients)(choose the one that best ﬁts)PLEASE READ BEFORE PROCEEDING  Science studentAn aware citizenPerson in my majorCurious personA person with a future in${q://QID172/ChoiceGroup/SelectedChoices}Other: StudentsPatientsClientsOther270

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview5/26The rest of the questions are focused on your experience in physics labs. A set of twoquestions will be addressing the same idea. There are no wrong answers, we are justlooking for your opinion on these things.  This will look like:Do YOU agree/disagree with this statement? Did these reasons contribute to your answer to the previous question?REASON 1REASON 2REASON 3REASON 4 ------ The majority of this survey is multiple choice. There is space at the end for you to makeadditional comments.  If you ﬁnd that the reasons provided do not suﬃciently cover your own reasoning there willbe space at the end for you to add other reasons.C1Explaining what I found is importantExplaining what I was trying to do is importantExplaining what I did is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree271

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview6/26Explaining why I did what I did is importantExplaining how I did what I did is importantExplaining my interpretation of the results is importantExplaining my results is importantI answered the previous question the way I did becauseC2Explaining what my results mean is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeither agreenor disagreeSomewhatdisagreeStronglydisagreeIt might be helpful but Ifelt obligated to do it inthe class  This will help me in thefuture  It is important at timesbut it does not apply tome  I do not care about thisbut I felt like I had to doit for the class  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree272

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview7/26Presenting what my results mean is importantPresenting what I did is importantCommunicating my results by answering speciﬁc questions is helpfulCommunicating my results by writing in my lab notebook is helpfulCommunicating my results by writing up a conclusion is helpfulBeing able to communicate my understanding is importantUsing uncertainty to express the conﬁdence in my results is helpfulI answered the previous question the way I did becauseStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree273

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview8/26DA1Using math as a tool to explain and predict is helpfulUsing math to apply a pattern to what I saw is valuableCalculating the average value for each variable tested/measured is importantObserving trends and patterns in the experiment and applying equations to that is important   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree274

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview9/26Determining patterns from my data is importantLooking for trends or asking why something happened mathematically is importantDoing math with my data in order to get my results is importantI answered the previous question the way I did becauseDA2Making connections to the bigger concepts is valuableStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  275

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview10/26Using the concepts to make my data into something I can interpret is importantComparing my results to the expected is importantUsing graphs to interpret results is importantMaking graphs from collected data is helpfulInterpreting graphs to understand relationships between parameters is importantUsing data to determine the signiﬁcance of my results is importantTesting the validity of my results with uncertainty is importantExpressing conﬁdence in my results with uncertainty is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree276

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview11/26I answered the previous question the way I did becauseEE1Working with the equipment is valuableCollecting data and taking measurements is importantKnowing that there is uncertainty associated with our measurements is importantWorking to minimize the uncertainty in my experiment is important   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree277

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview12/26The fact that uncertainty means measurements may be oﬀ, inaccurate, or wrong isimportantKnowing that the uncertainty my have an impact on the experiment is importantI answered the previous question the way I did becauseGW1Bringing many ideas together in a group is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree278

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview13/26Bouncing ideas oﬀ of my group members is importantCommunicating with my group about how things work is importantEveryone feeling comfortable sharing their ideas in their group is importantLetting each group member have hands-on time with the equipment is importantAgreeing as a group on how to complete a task before moving forward is importantValuing every group member's ideas is importantCommunicating with my group about how to do things is importantCommunicating with my group about the results is importantI answered the previous question the way I did becauseStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree279

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview14/26GW2Everyone in the group contributing an equal amount is importantGiving everyone in the group a chance to contribute is importantFiguring out how to communicate with my group is importantCaring about how each member of my group sees my work/eﬀort is important   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree280

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview15/26Having help from my group to check my work is importantHaving help from my group if I have questions is importantWorking together as a group is importantMaking decisions together as a group is importantI answered the previous question the way I did becauseStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  281

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview16/26PE1Having an hypothesis when planning an experiment is importantHaving a plan to answer my questions is importantKnowing what my goal is when planning an experiment is importantHaving a plan to get to my goal is importantKnowing what equipment I need for the experiment I am planning is importantKnowing what design works best for the experiment I am planning is importantHaving a procedure when designing an experiment is importantFiguring out how to get the information I need when desiging an experiment is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree282

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview17/26Figuring out how to set up the equipment when planning an experiment is importantKnowing how to get to my goal when planning an experiment is importantI answered the previous question the way I did becauseUE1Trying out the equipment and seeing what it does is valuableTinkering with the equipment is usefulStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree283

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview18/26Knowing what the equipment is designed to measure is valuableKnowing how the parts of the equipment work is valuableKnowing how to get what I need from the equipment is importantKnowing how the equipment works is importantSeeing how changing one component of the equipment aﬀects something else is importantI answered the previous question the way I did becauseStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  284

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview19/26UE2Knowing that the equipment may have multiple uses is importantKnowing how to adapt equipment to new situations is usefulIdentifying the extreme settings of the equipment is importantMaking the equipment measure something it was not originally intended for is importantKnowing what not to do with the equipment is usefulKnowing the limits/capabilities of the equipment is useful   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree285

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview20/26Knowing how to ﬁx the equipment when it is broken is usefulI answered the previous question the way I did becauseEnd BlockDo you see yourself as a physics person?While taking this survey I was most frequently thinking of myself as a member of thiscommunity (select all that apply)Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagree» This is not something I need to know for my career  » This will help me in the future  » I will not use this again after this class  » It is not the most important thing to me  » It is important at times but it does not apply to me  » It is important to think about but not always necessary  » I do not care about this but I felt like I had to do it forthe class  » It might be helpful but I felt obligated to do it in theclass  » This is important but I did it because I was expected to  » This will be important when workingwith ${q://QID99/ChoiceGroup/SelectedChoicesTextEntry}  No, not at all 12345Yes, very much 6Science studentAn aware citizenPerson in my majorCurious personA person with a future in${q://QID226/ChoiceTextEntryValue/4}286

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview21/26Contact Information Privacy Statement Site AccessibilityCall MSU: (517) 355-1855 Visit: msu.eduMSU is an aﬃrmative-action, equal-opportunity employer. Notice of NondiscriminationSPARTANS WILL. © Michigan State UniversityPowered by QualtricsWhile taking the survey did you ﬁnd the options provided under 'I answered the previousquestion the way I did because' suﬃciently covered your own reasoning? If you believe some options were missing please include them belowIs there anything you would like to tell us about the experience of taking this survey?${q://QID223/ChoiceGroup/SelectedChoices}YesSometimesNoAPPENDIX J

SURVEY VERSION 3

This appendix has the entire survey from Version 3 (Sec. 6.5). This is how it would have been
viewed by each student on Qualtrics1. The students saw a random practice question and all six
reasoning questions. The practice questions are not individually labeled in this appendix, the main
purpose of this is to provide a visual of what the students saw when taking the survey. Note: the
identity reasoning questions have a choicce that is partially cutoﬀ on the right side. The cutoﬀ
option is ’Strongly disagree.’

1Qualtrics, 2019, Qualtrics Labs, Inc. Provo, UT, USA

287

288

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview3/23about your role and rights as a research participant, would like to obtain information or offer input, or would like to register a complaint about this study, you may contact, anonymously if you wish, the Michigan State University’s Human Research Protection Program at 517­355­2180, Fax 517­432­4503, or e­mail irb@msu.edu or regular mail at Olds Hall, 408 West Circle Drive #207, MSU, East Lansing, MI 48824. 9.  DOCUMENTATION OF INFORMED CONSENT. Your affirmations below means that you voluntarily agree to participate in this research study. Do you agree to participate in the study? An answer is required.What physics class(es) are you in currently?What is your current major?Why did you choose to major in science?What is your interest in physics?YesNoVery lowLowModerateHighVery high289

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview4/23What are your future plans?If you selected OTHER please brieﬂy describe your future plansAs an undergraduate student taking science classes you are part of many diﬀerentcommunities. These communities may inﬂuence your perspective as you go through yourundergraduate career. Below we have some examples of communities that may be relevantto you, please choose all that apply and add on if we are missing something.I think of myself as a ... (Choose all that apply)When you think of your future career who do you imagine working with? (For example:Teacher work with students, doctors work with patients)(choose the one that best ﬁts)PLEASE READ BEFORE PROCEEDING  Science studentAn aware citizenPerson in my majorCurious personA person with a future in${q://QID172/ChoiceGroup/SelectedChoices}Other: StudentsPatientsClientsPeers/ColleaguesOther290

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview5/23The rest of the questions are focused on your experience in physics labs. A set of twoquestions will be addressing the same idea. There are no wrong answers, we are justlooking for your opinion on these things.  This will look like:Do YOU agree/disagree with this statement? Did these reasons contribute to your answer to the previous question?REASON 1REASON 2REASON 3REASON 4REASON 5REASON 6 ------ The majority of this survey is multiple choice. There is space at the end for you to makeadditional comments.  If you ﬁnd that the reasons provided do not suﬃciently cover your own reasoning there willbe space at the end for you to add other reasons.C1Explaining what I was trying to do is importantExplaining what I did is importantPresenting what I did is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree norSomewhat disagreeStrongly disagree291

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview6/23I answered the previous question the way I did because ... (Consider the entire statement when choosing your response)C2Explaining what my results mean is importantCommunicating my results by writing in my lab notebook is helpfulCommunicating my results by writing up a conclusion is helpfulI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisagI will not use this again after this class  It is important to think about but not alwaysnecessary  This will help me in the future  This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  This is important but I did it because I had to  It is not the most important thing to me  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree292

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview7/23C3Being able to communicate my understanding is importantExplaining what I found is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  293

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview8/23DA1Using graphs to interpret results is importantMaking graphs from collected data is helpfulInterpreting graphs to understand relationships between parameters is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)DA2Using math as a tool to explain and predict is helpfulStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree norSomewhat disagreeStrongly disagree294

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview9/23Determining patterns from my data is importantExpressing conﬁdence in my results with uncertainty is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)DA3Comparing my results to the expected is importantMaking connections to the bigger concepts is valuabledisagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree295

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview10/23Doing math with my data in order to get my results is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)EE1Working with the equipment is valuableCollecting data and taking measurements is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag296

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview11/23EE2Knowing that the uncertainty my have an impact on the experiment is importantWorking to minimize the uncertainty in my experiment is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  297

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview12/23GW1Bouncing ideas oﬀ of my group members is importantEveryone feeling comfortable sharing their ideas in their group is importantValuing every group member's ideas is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)GW2Communicating with my group about the results is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree norSomewhat disagreeStrongly disagree298

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview13/23Making decisions together as a group is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)GW3Giving everyone in the group a chance to contribute is importantHaving help from my group if I have questions is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree299

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview14/23PE1Having an hypothesis when planning an experiment is importantHaving a procedure when designing an experiment is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  300

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview15/23PE2Having a plan to answer my questions is importantKnowing what my goal is when planning an experiment is importantKnowing what design works best for the experiment I am planning is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)PE3Knowing what equipment I need for the experiment I am planning is importantStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree norSomewhat disagreeStrongly disagree301

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview16/23Figuring out how to get the information I need when desiging an experiment is importantFiguring out how to set up the equipment when planning an experiment is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)UE1Trying out the equipment and seeing what it does is valuableKnowing what the equipment is designed to measure is valuabledisagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree302

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview17/23Knowing how to get what I need from the equipment is importantI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)UE2Seeing how changing one component of the equipment aﬀects something else is usefulKnowing that the equipment may have multiple uses is importantKnowing how to adapt equipment to new situations is usefulI answered the previous question the way I did because ... Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree303

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview18/23(Consider the entire statement when choosing your response)UE3Knowing how to ﬁx the equipment when it is broken is usefulIdentifying the extreme settings of the equipment is importantKnowing the limits/capabilities of the equipment is usefulI answered the previous question the way I did because ... (Consider the entire statement when choosing your response)   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  » I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  Strongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagreeStrongly agreeSomewhat agreeNeither agree nordisagreeSomewhat disagreeStrongly disagree   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» This will be important when working with${q://QID55/ChoiceGroup/SelectedChoicesTextEntry}  304

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview19/23End BlockDo you see yourself as a physics person?While taking this survey I was most frequently thinking of myself as a member of thiscommunity (select all that apply)While taking the survey did you ﬁnd the options provided under 'I answered the previousquestion the way I did because' suﬃciently covered your own reasoning? If you believe some options were missing please include them below   StronglyagreeSomewhatagreeNeitheragreenordisagreeSomewhatdisagreeStrodisag» I will not use this again after this class  » This will help me in the future  » It is not the most important thing to me  » This is important but I did it because I had to  » It is important to think about but not alwaysnecessary  No, not at all 12345Yes, very much 6Science studentAn aware citizenPerson in my majorCurious personA person with a future in${q://QID223/ChoiceGroup/SelectedChoices}${q://QID226/ChoiceTextEntryValue/4}YesSometimesNo305

6/3/2019Qualtrics Survey Softwarehttps://msu.co1.qualtrics.com/Q/EditSection/Blocks/Ajax/GetSurveyPrintPreview20/23Contact Information Privacy Statement Site AccessibilityCall MSU: (517) 355-1855 Visit: msu.eduMSU is an aﬃrmative-action, equal-opportunity employer. Notice of NondiscriminationSPARTANS WILL. © Michigan State UniversityPowered by QualtricsIs there anything you would like to tell us about the experience of taking this survey?APPENDIX K

DISTRIBUTION OF VERSION 2 RESPONSES

This appendix contains the distributions of responses to all questions from Version 2 of the survey
(Chapter 6 Sec. 6.4).

306

Figure K.1: Distributions of responses to Communicating questions in Version 2 of the survey. The
distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat agree, 2
is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The reference
number of the question is in the title (see Appendix G for questions).

307

Figure K.2: Distributions of responses to Data Analysis questions in Version 2 of the survey. The
distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat agree, 2
is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The reference
number of the question is in the title (see Appendix G for questions).

308

Figure K.3: Distributions of responses to Executing Experiments questions in Version 2 of the
survey. The distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is
Somewhat agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly
disagree. The reference number of the question is in the title (see Appendix G for questions).

309

Figure K.4: Distributions of responses to Group Work questions in Version 2 of the survey. The
distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat agree, 2
is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The reference
number of the question is in the title (see Appendix G for questions).

310

Figure K.5: Distributions of responses to Planning Experiments questions in Version 2 of the survey.
The distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat
agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The
reference number of the question is in the title (see Appendix G for questions).

311

Figure K.6: Distributions of responses to Understanding Equipment questions in Version 2 of
the survey. The distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1
is Somewhat agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly
disagree. The reference number of the question is in the title (see Appendix G for questions).

312

APPENDIX L

DISTRIBUTION OF VERSION 3 RESPONSES

This appendix contains the distributions of responses to all questions from Version 3 of the survey
(Chapter 6 Sec. 6.5).

313

Figure L.1: Distributions of responses to Communicating questions in Version 3 of the survey. The
distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat agree, 2
is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The reference
number of the question is in the title (see Appendix G for questions).

314

Figure L.2: Distributions of responses to Data Analysis questions in Version 3 of the survey. The
distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat agree, 2
is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The reference
number of the question is in the title (see Appendix G for questions).

315

Figure L.3: Distributions of responses to Executing Experiments questions in Version 3 of the
survey. The distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is
Somewhat agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly
disagree. The reference number of the question is in the title (see Appendix G for questions).

316

Figure L.4: Distributions of responses to Group Work questions in Version 3 of the survey. The
distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat agree, 2
is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The reference
number of the question is in the title (see Appendix G for questions).

317

Figure L.5: Distributions of responses to Planning Experiments questions in Version 3 of the survey.
The distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1 is Somewhat
agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly disagree. The
reference number of the question is in the title (see Appendix G for questions).

318

Figure L.6: Distributions of responses to Understanding Equipment questions in Version 3 of
the survey. The distributions are all normalized to one. On the x-axis: 0 is Strongly agree, 1
is Somewhat agree, 2 is Niether agree nor disagree, 3 is Somewhat disagree, and 4 is Strongly
disagree.The reference number of the question is in the title (see Appendix G for questions).

319

APPENDIX M

FINAL SURVEY VERSION

This appendix contains the ﬁnal version of the Practice-Based Identity Survey, see Table M.1.

320

Table M.1: Questions for the ﬁnal version of the survey

Practice Questions

PE11
UE1
UE2
UE10
UE19
UE25

Label
LT2
LT3
NE1
NE2
NE3

Reference Question
C1
C8
C9
C12
DA1
DA6
DA10
DA16
DA19
EE2
EE3
EE7
GW9
GW15
GW17
GW20
PE4
PE9

Explaining what I found is important
Presenting what I did is important
Communicating my results by writing in my lab notebook is helpful
Being able to communicate my understanding is important
Comparing my results to the expected is important
Making graphs from collected data is helpful
Using math as a tool to explain and predict is helpful
Determining patterns from my data is important
Expressing conﬁdence in my results with uncertainty is important
Working with the equipment is valuable
Collecting data and taking measurements is important
Working to minimize the uncertainty in my experiment is important
Everyone feeling comfortable sharing their ideas in their group is important
Having help from my group if I have questions is important
Bouncing ideas oﬀ of my group members is important
Making decisions together as a group is important
Knowing what design works best for the experiment I am planning is important
Figuring out how to set up the equipment when planning an experiment is
important
Having an hypothesis when planning an experiment is important
Knowing that the equipment may have multiple uses is important
Knowing how to adapt equipment to new situations is useful
Knowing how the equipment works is important
Knowing the limits/capabilities of the equipment is useful
Knowing how to ﬁx the equipment when it is broken is useful

Identity Reasoning Questions

Question
I will not use this again after this class
This will help me in the future
It is not the most important thing to me
This is important but I did it because I had to
It is important to think about but not always necessary

321

BIBLIOGRAPHY

322

BIBLIOGRAPHY

[1] Benjamin M. Zwickl, Takako Hirokawa, Noah Finkelstein, and H. J. Lewandowski. Episte-
mology and expectations survey about experimental physics: Development and initial results.
Physical Review Special Topics-Physics Education Research, 10(1):010120, 2014.

[2] B. R. Wilcox and H. J. Lewandowski. Developing skills versus reinforcing concepts in physics
labs: Insight from a survey of students’ beliefs about experimental physics. Physical Review
Physics Education Research, 13(1):010108, 2017.

[3] Eleanor W. Close, Jessica Conn, and Hunter G. Close. Learning Assistants’ Development of
Physics (Teacher) Identity. 2013 Physics Education Research Conference Proceedings, pages
89–92, 2013.

[4] Etienne Wenger. Identity in practice. In Communities of Practice Learning, Meaning, and

Identity, chapter 6, pages 149–163. Cambridge University Press, 1999.

[5] AAPT. AAPT Recommendations for the Undergraduate Physics Laboratory Curriculum.

Report, page 29, 2014.

[6] Sidney Rosen. A History of the Physics Laboratory in the American Public High School (to

1910). American Journal of Physics, 22(4):194–204, 1953.

[7] Avi Hofstein and Vincent N. Lunetta. The Role of the Laboratory in Science Teaching:

Neglected Aspects of Research. Review of Educational Research, 52(2):201–217, 1982.

[8] C. R. Mann. The Present Condition of Physics Teaching in the United States. In Broad lines

in science teaching, pages 227–238. 1910.

[9] G. R. Bates. The role of the laboratory in secondary school science programs. What research

says to the science teacher, 1:55–82, 1978.

[10] Carl Wieman and N. G. Holmes. Measuring the impact of an instructional laboratory on the

learning of introductory physics. American Journal of Physics, 83(11):972–978, 2015.

[11] N. G. Holmes, Jack Olsen, James L. Thomas, and Carl E. Wieman. Value added or misat-
tributed? a multi-institution study on the educational beneﬁt of labs for reinforcing physics
content. Physical Review Physics Education Research, 13(1):010129, 2017.
[12] National Research Council. A framework for K-12 Science Education. 2012.
[13] Michael P. Freedman. Relationship among laboratory instruction, attitude toward science, and
achievement in science knowledge. Journal of Research in Science Teaching: The Oﬃcial
Journal of the National Association for Research in Science Teaching, 34(4):343–357, 1997.
[14] Elaine Seymour, Anne-Barrie Hunter, Sandra L. Laursen, and Tracee DeAntoni. Establishing
the beneﬁts of research experiences for undergraduates in the sciences: First ﬁndings from a
three-year study. Science education, 88(4):493–534, 2004.

323

[15] Remziye Ergül, Yeter Şımşeklı, Sevgül Çaliş, Zehra Özdılek, Şirin Göçmençelebı, and Meral
Şanli. The Eﬀects of Inquiry-Based Science Teaching on Elementary School Students’ Science
Process Skills and Science Attitudes. Bulgarian Journal of Science and Education Policy,
5(1):48–68, 2011.

[16] Melinda Harrison, David Dunbar, Lisa Ratmansky, Kimberly Boyd, and David Lopatto.
Classroom-based science research at the introductory level: Changes in career choices and
attitude. CBE Life Sciences Education, 10(3):279–286, 2011.

[17] Anja Bechmann and Stine Lomborg. Relationship among laboratory instruction, attitude
toward science, and achievement in science knowledge. Journal of Research in Science
Teaching, 34(4):343–357, 1997.

[18] Olga Pierrakos, Thi Kay Beam, Jamie Constantz, Aditya Johri, and Robin Anderson. On
the development of a professional identity: Engineering persisters vs engineering switchers.
Proceedings - Frontiers in Education Conference, FIE, (2):1–6, 2009.

[19] Zahra Hazari, Gerhard Sonnert, Philip M. Sadler, and Marie-Claire Shanahan. Connecting
high school physics experiences, outcome expectations, physics identity, and physics career
choice: A gender study. Journal of Research in Science Teaching, 47(8):978–1003, 2010.

[20] H. B. Carlone and A. Johnson. Understanding the Science Experiences of Successful Women
of Color: Science Identity as an Analytic Lens. Journal of Research in Science Teaching,
44(8):1269–1291, 2007.

[21] Eleanor W. Close, Jessica Conn, and Hunter G. Close. Becoming physics people: Devel-
opment of integrated physics identity through the Learning Assistant experience. Physical
Review Physics Education Research, 12(1):010109, 2016.

[22] Alexander Smith and Edwin Herbert Hall. The teaching of chemistry and physics in the

secondary school. Longmans, Green, and Company, 1902.

[23] John Dewey. How We Think. D. C. Heath and Co. Publishers, 1910.
[24] Lee S Shulman and Pinchas Tamir. Research on teaching in the natural sciences. Second

handbook of research on teaching, pages 1098–1148, 1973.

[25] Paul DeHart Hurd. New directions in teaching secondary school science. Rand McNally,

1969.

[26] Vincent N. Lunetta and Pinchas Tamir. An analysis of laboratory activities in two modern

science curricula: Project physics and pssc. 1978.

[27] Joseph Jackson Schwab. The Teaching of Science. The Teaching of Science as Enquiry.[By]
JJ Schwab. Elements in a Strategy for Teaching Science in the Elementary School.[By] Paul
F. Brandwein. Harvard University Press, 1962.

[28] Jack Cassels Jeﬀrey. Identiﬁcation of objectives of the chemistry laboratory and development

of means for measuring student achievement of some of these objectives. 1968.

324

[29] R. F. Kempa and J. E. Ward. The eﬀect of diﬀerent modes of task orientation on observational
attainment in practical chemistry. Journal of Research in Science Teaching, 12(1):69–76,
1975.

[30] Pinchas Tamir. An Inquiry Oriented Laboratory Examination. Journal of Educational Mea-

surement, 11(1):25–33, 1974.

[31] O. Roger Anderson. The experience of science: A new perspective for laboratory teaching.

Teachers College Press, Teachers College, Columbia University, 1976.

[32] National Research Council. National Science Education Standards. 1996.
[33] National Research Council. Inquiry and the National Science Education Standards. 2000.
[34] Eugenia Etkina, Sahana Murthy, and Xueli Zou. Using introductory labs to engage students

in experimental design. American Journal of Physics, pages 979–986, 2006.

[35] Eugenia Etkina, Alan Van Heuvelen, Suzanne White-Brahmia, David T. Brookes, Michael
Gentile, Sahana Murthy, David Rosengrant, and Aaron Warren. Scientiﬁc abilities and their
assessment. Physical Review Special Topics - Physics Education Research, 2(2):1–15, 2006.
[36] Kirsty Dunnett, M. N. Gorman, and P. A. Bartlett. Assessing ﬁrst-year undergraduate physics
students’ laboratory practices: seeking to encourage research behaviours. European journal
of physics, 40(1):015702, 2018.

[37] H. J. Lewandowksi, Daniel Bolton, and Benjamin Pollard. Initial Impacts of the transformation
of a large Introductory lab course focused on developing experimental skills and expert
epistemology. Physics Education Research Conference Proceedings, page submitted, 2018.
[38] Benjamin M. Zwickl, Noah Finkelstein, and Heather J. Lewandowski. The process of trans-
forming an advanced lab course: Goals, curriculum, and assessments. American Journal of
Physics, 81(1):63–70, 2013.

[39] H. J. Lewandowski and Noah Finkelstein. Redesigning a junior-level electronics course to sup-
port engagement in scientiﬁc practices. Physics Education Research Conference Proceedings,
pages 191–194, 2015.

[40] Kelsey Marie Funkhouser, William Martinez, Rachel Henderson, and Marcos D. Caballero.

Design, analysis, tools, and apprenticeship (data) lab. European Journal of Physics, 2019.

[41] Carl Wieman. Improving how universities teach science: lessons from the science education

initiative. Harvard University Press, 2017.

[42] Saalih Allie, Andy Buﬄer, Bob Campbell, and Fred Lubben. First-year physics students’
perceptions of the quality of experimental measurements. International Journal of Science
Education, 20(4):447–459, 1998.

[43] Andy Buﬄer, Saalih Allie, and Fred Lubben. The development of ﬁrst year physics students’
ideas about measurement in terms of point and set paradigms. International Journal of Science
Education, 23(11):1137–1156, 2001.

325

[44] Cole Walsh, Katherine N. Quinn, C. Wieman, and N. G. Holmes. Quantifying critical thinking:
Development and validation of the physics lab inventory of critical thinking. Physical Review
Physics Education Research, 15(1):010135, 2019.

[45] Eugenia Etkina, Anna Karelina, Maria Ruibal-villasenor, David Rosengrant, Rebecca Jordan,
and Cindy E. Hmelo-silver. Design and Reﬂection Help Students Develop Scientiﬁc Abilities:
Learning in Introductory Physics Laboratories. The Journal of Learning Sciences, 2010.

[46] Eugenia Etkina and Gorazd Planinšič. Deﬁning and developing “critical thinking” through
devising and testing multiple explanations of the same phenomenon. The Physics Teacher,
53(7):432–437, 2015.

[47] Anna Karelina and Eugenia Etkina. Acting like a physicist: Student approach study to exper-
imental design. Physical Review Special Topics-Physics Education Research, 3(2):020106,
2007.

[48] Kevin D. O’Neill and Joseph L. Polman. Why educate “little scientists?” examining the poten-
tial of practice-based scientiﬁc literacy. Journal of research in Science Teaching, 41(3):234–
266, 2004.

[49] N. G. Holmes, Carl E. Wieman, and D. A. Bonn. Teaching critical thinking. Proceedings of

the National Academy of Sciences, 112(36):11199–11204, 2015.

[50] Anna Sfard and Anna Prusak. Telling identities: In search of an analytic tool for investigating

learning as a culturally shaped activity. Educational researcher, 34(4):14–22, 2005.

[51] James Paul Gee. Identity as an Analytic Lens for Research in Education. Review of Research

in Education, 25(1):99–125, 2000.

[52] Jean Lave and Etienne Wenger. Situated learning: Legitimate peripheral participation.

Cambridge university press, 1991.

[53] Paul W. Irving and Eleanor C. Sayre. Physics identity development: A snapshot of the stages
of development of upper-level physics students. Journal of the Scholarship of Teaching &
Learning, 13(4):68–84, 2013.

[54] Paul W. Irving and Eleanor C. Sayre. Upper-Level Physics Students ’ Perceptions of Physicists.
Proceedings of the International Conference of the Learning Sciences, pages 1513–1514,
2014.

[55] Paul W. Irving and Eleanor C. Sayre. Becoming a physicist: The roles of research, mindsets,
and milestones in upper-division student perceptions. Physical Review Special Topics - Physics
Education Research, 11(2):1–21, 2015.

[56] Paul W. Irving and Eleanor C. Sayre. Identity statuses in upper-division physics students.

Cultural Studies of Science Education, 11(4):1–46, 2016.

[57] Paul W. Irving and Eleanor C. Sayre. Conditions for building a community of practice in an
advanced physics laboratory. Physical Review Special Topics - Physics Education Research,
10(1):1–16, 2014.

326

[58] Dedra Demaree and Sissi Li. Promoting productive communities of practice: An instructor’s

perspective. AIP Conference Proceedings, 1179:125–128, 2009.

[59] Leema K. Berland. Explaining variation in how classroom communities adapt the practice of

scientiﬁc argumentation. Journal of the Learning Sciences, 20(4):625–664, 2011.

[60] Joseph Kozminski, Heather Lewandowski, Nancy Beverly, Steve Lindaas, Duane Deardorﬀ,
Ann Reagan, Richard Dietz, Randy Tagg, Jeremiah Williams, Robert Hobbs, et al. Aapt
recommendations for the undergraduate physics laboratory curriculum. American Association
of Physics Teachers, page 29, 2014.

[61] Steve Olson and Donna Gerardi Riordan. Engage to Excel: Producing One Million Additional
College Graduates with Degrees in Science, Technology, Engineering, and Mathematics.
Report to the President. Executive Oﬃce of the President, 2012.

[62] Eugenia Etkina and A. V. Heuvelen. Investigative Science Learning Environment: Using the

processes of science and cognitive strategies to learn physics. 2001.

[63] Robert Beichner. The student-centered activities for large enrollment undergraduate programs
(scale-up) project. In Research-Based Reform of University Physics, volume 1. April 2007.
[64] Jack M. Wilson. The CUPLE physics studio. The Physics Teacher, 32(9):518–523, 1994.
[65] Melissa Dancy and Charles Henderson. Barriers and promises in stem reform. In National

Academies of Science Promising Practices Workshop, 2008.

[66] Stephanie V. Chasteen, Katherine K. Perkins, Paul D. Beale, Steven J. Pollock, and Carl E.
Wieman. A thoughtful approach to instruction: Course transformation for the rest of us.
Journal of College Science Teaching, 40(4):24, 2011.

[67] Stephanie V. Chasteen, Steven J. Pollock, Rachel E. Pepper, and Katherine K. Perkins. Trans-
forming the junior level: Outcomes from instruction and research in E&M. Physical Review
Special Topics-Physics Education Research, 8(2):020107, 2012.

[68] John Biggs. Enhancing teaching through constructive alignment. Higher Education,

32(3):347–364, 1996.

[69] Valerie Otero, Steven Pollock, and Noah Finkelstein. A physics department’s role in preparing
physics teachers: The Colorado learning assistant model. American Journal of Physics,
78(11):1218–1224, 2010.

[70] Paul W. Irving, Michael J. Obsniuk, and Marcos D. Caballero. P3: a practice focused learning

environment. European Journal of Physics, 38(5):055701, 2017.

[71] Bethany R. Wilcox and H. J. Lewandowski. Students’ epistemologies about experimental
physics: Validating the colorado learning attitudes about science survey for experimental
physics. Physical Review Physics Education Research, 12(1):010123, 2016.

327

[72] Dehui Hu, Benjamin M. Zwickl, Bethany R. Wilcox, and H. J. Lewandowski. Qualitative in-
vestigation of students’ views about experimental physics. Physical Review Physics Education
Research, 13(2):020134, 2017.

[73] Bethany R. Wilcox and H. J. Lewandowski. A summary of research-based assessment of
students’ beliefs about the nature of experimental physics. American Journal of Physics,
86(3):212–219, 2018.

[74] Wendy K. Adams, Katherine K. Perkins, Noah S. Podolefsky, Michael Dubson, Noah D.
Finkelstein, and Carl E. Wieman. New instrument for measuring student beliefs about physics
and learning physics: The colorado learning attitudes about science survey. Physical review
special topics-physics education research, 2(1):010101, 2006.

[75] Angela Calabrese Barton and Kimberley Yang. The culture of power and science education :

Learning from Miguel. Journal of Research in Science Teaching, 2000.

[76] Benjamin M. Zwickl, Noah Finkelstein, and H. J. Lewandowski. Incorporating learning goals
about modeling into an upper-division physics laboratory experiment. American Journal of
Physics, 82(9):876–882, 2014.

[77] Natasha Grace Holmes. Structured quantitative inquiry labs : developing critical thinking in

the introductory physics laboratory. PhD Thesis, (December):1–127, 2014.

[78] N. G. Holmes, Jack Olsen, James L. Thomas, and Carl E. Wieman. Value added or misat-
tributed? A multi-institution study on the educational beneﬁt of labs for reinforcing physics
content. Physical Review Physics Education Research, 13(1):1–12, 2017.

[79] Jacob T. Stanley and H. J. Lewandowski. Lab notebooks as scientiﬁc communication: In-
vestigating development from undergraduate courses to graduate research. Physical Review
Physics Education Research, 12(2):1–11, 2016.

[80] Jacob T. Stanley, Dimitri R. Dounas-Frazer, Laura Kiepura, and H. J. Lewandowski.

In-
vestigating student ownership of projects in an upper-division physics lab course. Physics
Education Research Conference Proceedings, pages 1–4, 2016.

[81] D.H. Putri, E. Risdianto, S. Sutarno, and D. Hamdani. The development of cooperative
problem solving physics laboratory model on simple pendulum concept. Journal of Physics:
Conference Series, 1157:032005, 2019.

[82] Bobby Cerini, Ian Murray, and Michael Reiss. Student review of the science curriculum:

Major ﬁndings. 2003.

[83] Jacob Cohen. A coeﬃcient of agreement for nominal scales. Educational and psychological

measurement, 20(1):37–46, 1960.

[84] Mary L. McHugh. Interrater reliability: the kappa statistic. Biochemia medica: Biochemia

medica, 22(3):276–282, 2012.

[85] John W. Creswell and Dana L. Miller. Analyzing Qualitative Data. Theory into practice,

39(25 Jun 2010):124–130, 2000.

328

[86] Sreyashi Jhumki Basu, Angela Calabrese Barton, and Neil Clairmont. Developing a framework
for critical science agency through case study in a conceptual physics context. Cultural Studies
of Science Education, (May), 2008.

[87] Angela Calabrese Barton and Edna Tan. We be burnin’! agency, identity, and science learning.

Journal of the Learning Sciences, 19(2):187–229, 2010.

[88] Kelsey Funkhouser, Marcos D. Caballero, Paul W. Irving, and Vashti Sawtelle. What counts
In Physics Education Research

in laboratories: Toward a practice-based identity survey.
Conference 2018, PER Conference, Washington, DC, August 1-2 2018.

[89] Robert Beichner, Ruth Chabay, and Bruce Sherwood. Labs for the matter & interactions

curriculum. Am. J Phys., 78(5):456–460, 2010.

[90] Edward F. Redish, Jeﬀery M. Saul, and Richard N. Steinberg. Student expectations in

introductory physics. Am. J Phys., 66(3):212–224, 1998.

[91] Benjamin M. Zwickl, Takako Hirokawa, Noah Finkelstein, and H. J. Lewandowski. Episte-
mology and expectations survey about experimental physics : Development and initial results.
Physical Review Special Topics - Physics Education Research, 10(1):010120, 2014.

329