IDENTIFYING THE FOUNDATIONAL STRUCTURES OF INFORMAL PHYSICS
PROGRAMS TO SUPPORT PROGRAM LEADERS AND THEIR VOLUNTEERS

By

Bryan Stanley

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

Physics — Doctor of Philosophy

2024

ABSTRACT

Physicists, physics students, and community members engage with each other through a

variety of informal physics programs. These programs differ in format, such as after school

programs, public lectures, planetariums, summer camps, and other non-formal classroom

environments. These programs also differ in the physics topics they cover, activities, audience

demographics, frequency of events, and involved personnel. While studies have found that

these programs have positive impacts on audiences and volunteers, we do not fully understand

the organizational and programmatic structures of these programs or their impacts on those

involved with these programs.

In this work, I apply an organizational theory framework to understand and analyze

six functional aspects of informal physics programming: Assessment, Audience, Institution,

Personnel, Program, and Resources. One finding is that the personnel have a central role

in the functionality of these programs, however, program leaders often experience challenges

because most of the responsibilities are placed on a single person. Program leaders state they

want to do more assessment of their programs, but lack the time, resources, or experience to

do so. Building off of my findings, I developed an empirical framework of key components.

This framework identifies 12 key components of informal physics programming that program

leaders have control over and that are important to the program’s functionality. Identifying

these components lays the groundwork for developing tools and resources that program

leaders can use to support and improve their programs. Multiple of these components

are centered around the recruitment and support of volunteers. Since these are voluntary

positions, understanding the motivations of volunteers can help program leaders to better

support their personnel.

I interviewed alumni who had volunteered in informal physics

programs as university students. I applied a volunteerism framework to understand their

motivations for volunteering in the first place and the impacts that volunteering had on their

career pursuits. The findings of these studies help to identify the aspects of informal physics

programs that are crucial for program functionality and how some of these aspects impact

those who facilitate these learning spaces.

“THIS IS SO COOL! THERE IS SO MUCH SCIENCE IT MAKES ME GO AAAHHH”
-First grader who yelled at me when I visited their school with science experiments.

iv

There are so many people who helped make this dissertation possible.

I will attempt to

ACKNOWLEDGMENTS

thank everyone here.

First, I would like to thank my advisor Katie Hinko. I could not have done this without

her. She was always supportive of me professionally and emotionally, and always made sure

that I was doing what was best for me.

I would also like to thank the rest of my committee members: Danny Caballero, Diane

Doberneck, Rachel Henderson, and Stuart Tessmer. Not only did they provide guidance on

my doctoral work, they also helped me to grow professionally in the physics, education, and

community engagement communities.

Thank you to the broader PERL research group for the sense of community. I especially

appreciate the professional feedback and support I got from all the faculty and post-docs.

I am also thankful from all the friendships I made with the PERL graduate students.

I

always enjoyed our intelligent and “intelligent” conversations.

I will miss answering the

PERL whiteboard questions.

I am also grateful for all the IPER folks. I could not have done my research projects

without the help of Dena, Jessi, and Lily. Claudia Fracchiolla has also provided great

mentorship and support and always providing me with great professional opportunities.

Also, a big thank you to Brean, who is the biggest reason why I even applied to MSU for

graduate school, accepted their offer, and got involved WaMPS public engagement.

Thank you Kim and Jenn. The physics department, let alone me, could not operate

without all their hard work.

Thank you to my LSOP family, who always got me re-excited about physics through

public engagement. I appreciate the constant support of me, even after I left Colorado to

v

start this PhD program.

A special thank you to my amazing parents for their constant love and support of me.

And hugs.

A big thank you to my wonderful partner, Kaitlin, who was especially patient with me

as I finished this dissertation. Also, thank you Gizmo, the coolest cat of all, for all of the

snuggles.

vi

TABLE OF CONTENTS

Chapter 1.

Introducing Informal Physics . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Science Education Outside of the Classroom . . . . . . . . . . . . . . . . . .
1.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3
Informal Physics Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 Physics Education Research: A Discipline-Based Approach . . . . . . . . . .
1.5 Research Goals and Structure of this Dissertation . . . . . . . . . . . . . . .

Chapter 2. Approaches to Study Design . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1 Theoretical Frameworks
2.2 Methodological Considerations . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 How Site Visits can Help us Learn More . . . . . . . . . . . . . . . . . . . .

Chapter 3. Central Role of Personnel in Informal Physics Programming . . . . . . .
3.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
1
4
9
10
12

14
15
24
30

44
44
47
49
58
87

Chapter 4. An Empirical Framework for the Key Components of Informal Physics
95
Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
95
4.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
97
4.2 Developing a Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
99
4.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 104
4.4 Key Components of Informal Physics Programs
. . . . . . . . . . . . . . . . . . . . 106
4.5 Program Goals, Design, and Assessment
4.6 Personnel
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.7 Audience and Community Partners . . . . . . . . . . . . . . . . . . . . . . . 125
Institutional Connection and Resources . . . . . . . . . . . . . . . . . . . . . 133
4.8
Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.9

Chapter 5. Using a Volunteerism Framework to Understand the Motivations of

University Students who Facilitate Informal Physics Programs . . . . . 152
5.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5.2 Volunteerism Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
. . . . . . . . . . . . . . . . . . . . . . 157
5.4 Applying Framework to Three Cases
5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5.6 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Chapter 6. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.1 My Positionality as an Informal Physics Researcher and Practitioner . . . . . 169

vii

6.2 Using the Key Components Framework to Reflect on my Experiences as an

Informal Physics Program Leader . . . . . . . . . . . . . . . . . . . . . . . . 171
6.3 Considering Different Framework Representations . . . . . . . . . . . . . . . 185
6.4 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

APPENDIX A. CHAPTER 3 INTERVIEW PROTOCOL . . . . . . . . . . . . . . 203

APPENDIX B. CHAPTER 3 OVERLAP TABLE . . . . . . . . . . . . . . . . . . . 208

APPENDIX C. CHAPTER 4 PROGRAM DESCRIPTIONS . . . . . . . . . . . . . 209

APPENDIX D. CHAPTER 4 REFLECTION QUESTIONS . . . . . . . . . . . . . 210

APPENDIX E. CHAPTER 5 INTERVIEW PROTOCOL . . . . . . . . . . . . . . 216

viii

Chapter 1. Introducing Informal Physics

1.1 Science Education Outside of the Classroom

1.1.1 People Engage with Science Outside of the Classroom

There are many different ways to engage the community with science. One seemingly obvious

way is to teach science when people are in school so that they grow up to be scientifically

literate. However, learners spend the majority of their lives outside the classroom [1]. As

such, people continue to learn outside of their classroom experiences, leading to the creation

of out-of-school-time activities.

In the United States, out-of-school-time activities have existed since the 18th century,

providing academic and social experiences, and have evolved over time to include more

science learning opportunities [2]. Presently, the landscape of out-of-school-time learning

for the sciences has grown for children and adults through a variety of opportunities like

after-school programming, camps, museums, popular media, public lectures, citizen science,

and more [1, 2]. In the present day, young adults are often engaging science content online

through social media platforms [3]. While the methods for engaging the public with science

differ in format, content, intended audiences, and frequency of events [4], audiences can build

upon their scientific interest, identity, and knowledge [5, 6, 7, 8, 9].

There is a large decrease in interest in science among students around the time that they

are in middle school [10, 11]. There is often a disconnect for students between the science

they learn in school and its applicableness to their everyday lives [12]. This is partially

due to the stressful environment of classroom spaces, which includes strict timelines and

high-stakes assessments [1]. Students who struggle in the formal classroom structure have

1

demonstrated competence on the same subject material in out-of-school contexts, in part due

to their ability to explore the content in a way that was comfortable and interesting to them;

this is impactful for students who are from non-dominant cultural or lower socioeconomic

groups [1, 13, 14].

Children are not the only ones who benefit from participating in out-of-class activities.

Adults also attend science learning opportunities in their free time. For example, attendees

of science festivals get opportunities to have direct conversations with scientists about more

complex science topics [15]. It has also been shown that attending science centers, museums,

aquariums, etc. helps to promote scientific understanding and action [16, 17].

1.1.2 Scientists Facilitate Science Engagement Outside of the

Classroom

There is a widespread push for scientists and researchers to engage with the greater

community. Arguments for doing so vary. As Thomas and Durant note, the range

of arguments include benefiting individuals as society becomes more scientifically and

technologically advanced, recruiting members to participate in the scientific community,

inform governmental decision making, and lessen the divide between scientists and people

who are not experts within that specific field [18]. Thomas and Durant note that these

arguments imply different underlying motivations. In this dissertation, we will see some of

these different motivations manifest in how physicists engage with the public.

To further promote that researchers should be engaging with the broader public,

university mission statements often include points about commitment to outreach and

engagement. For example, Michigan State University’s mission statement declares that one of

2

the ways the university will “advance knowledge and transform lives” is through “advancing

outreach, engagement, and economic development activities that are innovative, research-

driven, and lead to a better quality of life for individuals and communities, at home and

around the world” [19]. The National Science Foundation (NSF) requires funding proposals

to include a Broader Impacts Statement detailing how the project has “the potential to

benefit society and contribute to the achievement of specific, desired societal outcomes” [20].

NSF lists public engagement as one of the ways broader impacts can manifest [20]. While

there is a push for researchers and scientists to get more involved in public engagement,

there is a need to understand the structures that are necessary for effective programming.

To do this, we should first look at understanding the fundamental components of public

engagement programs that currently exist.

While there has been a push for scientists to engage with the public, it is often presented

with a deficit-lens that engagement is something that scientists do to benefit everyone else,

and some scientists do not see the direct benefit for themselves [21, 22]. Scientists are often

concerned about the “co-production of knowledge” and disturbed by the idea of science being

“informed by the public” [21]. However, there are great benefits from having the scientists

and the generally public meeting and sharing ideas with each other. For example, there is

a push for reflexive engineering, where engineers consider engineering problems with social,

economic, and environmental barriers in mind [23]. An example that Robbins provides deals

with a team tasked with developing an irrigation system for a rural village. A traditional

engineering approach is for the engineers to say “this is the answer” without considering

the logistics of how the system will be monitored or maintained by the community once

the engineering team leaves. A more successful approach is to have the farmers and other

community members involved in the process, as they also have expertise on the soil, types of

3

crops they will be farming, etc. [23]. That, however, requires communication and pedagogical

skills, as well as experience working with and collaborating with community partners, which

are all skills that facilitators of informal physics programs gain and improve upon in informal

spaces [24, 25, 26, 27, 28].

There is also a long history of inequity in the sciences. For example, the physics

community is about 80% male and 70% white [29, 30]. Participating in public engagement

has helped many scientists, especially those from underrepresented groups, to persist in their

careers and build their sense of belonging [31, 32, 33].

1.2 Terminology

The words and terminology used when describing the various learning environments that take

place outside of the classroom vary across researchers and practitioners in different fields.

Even within certain academic fields and across disciplines, people will use different words to

describe the same thing, and the same word to describe different things [34, 35, 36, 37, 38].

Here, I will define some of the common terminology used, and how I will be using these

terms throughout the rest of this dissertation.

1.2.1 Formal Learning Spaces

These are more traditional classroom settings. Typically, in these spaces, students are given

a grade or course credit. In many instances, students are required to take the course and

can not opt-out.

4

1.2.2 Informal Learning Spaces

In many instances, informal is considered to be the opposite of formal, in that the learning

takes place outside of a traditional course setting. Sometimes these spaces are referred to

as “free-choice” because learners can choose to opt in or out of participating, and choose

the level of their participation, in these spaces [1, 17, 34]. Due to this and the absence

of aspects like grades, the stakes are considered low for participants compared to formal

learning situations.

When it comes to learning outside of formal spaces, there are often variations in

terminology. For example, in the report, Formal, Non-formal and Informal Learning in

the Sciences, non-formal learning is described as being organized in some way with no credit

awarded while informal learning is never organized and not guided by a curriculum [39]. This

report characterizes non-formal activities as being led by someone with more experience, with

an example including guided programs at a National Park, while informal learning is driven

more by the learner, such as learning from a casual conversation with a friend [39].

As a counter example, The International Handbook of Physics Education Research also

defines non-formal spaces to have structure,

like a personal development course, while

informal physics education as without formalized structures. However, the handbook breaks

informal physics down into four blocks [34]:

• Informal physics education (non-designed), where learning takes place in spaces that

were not purposefully designed for learning to take place

• Informal physics education (designed), where physicists design activities for learners

• Public engagement with physics, where physicists and members of the public have

beneficial interactions with each other

5

• Physics communication, where physicists provide activities and content through various

forms of media like podcasts, books, etc.

The second and third points especially conflict with the previous definitions that

informal is without structures. Designed informal physics education and public engagement

have structures. The activities are often designed by a physicist and the content was

predetermined in some way and the learner opt-ed into joining that organized activity. The

organization and structures certainly vary in rigidness across programming, but there are

structures nonetheless.

Another key point is that non-designed informal physics education and physics

communication often, but not always, lack the direct interactions between the physicist

and the learner. Designed informal physics education and public engagement often have

opportunities for direct interactions between the physicist and learner.

For the purpose of this dissertation, I am going to analyze not-for-credit physics learning

spaces where there is a direct interaction between the learner and the person facilitating the

learning space.

I will be referring to these as informal physics spaces. For example, one

type of informal physics space is a science festival where attendees are able to converse and

interact with the scientists who are facilitating science activities. This would be an informal

space because the scientist and the attendees are able to have direct interactions with each

other.

1.2.3 Communication and Popular Media

Other ways that scientists present scientific information with the public is through “science

communication” and popular media. Some examples of these approaches include popular

science books, podcasts, online videos, apps, and TED talks. In most of these examples,

6

knowledge flows one way from the scientist to the audience. For example, with a popular

science book, a scientist writes a book based on their expertise, and then a consumer reads

that book. In that case, the reader has no direct interaction with the scientist nor is the

scientist learning directly from the reader.

In some instances, there can be some interaction between scientists and members of the

broader public. For example, scientists may use X (formerly known as Twitter), YouTube,

TikTok, and other social media platforms to share scientific information [34]. Through these

mediums, people can leave comments and responses for the scientists, which can lead to a

form of online dialogue about science. The level of constructive online interaction varies

dramatically.

The examples that I have provided are typically led by a single scientist rather than an

organization of scientists. In this dissertation, I am primarily interested in programs where

physicists and audiences are directly interacting with each other in real time through some

form of a physics activity. For example, public lectures are ways that physicists present their

research to non-physics experts. In this dissertation, I focus on how speakers were recruited,

how venues were selected, what audience interactions look like, frequency of events, and

other organizational decisions.

I do not directly focus on how Physicist A gives talks on

their own or how Physicist B writes books. As such, this dissertation does not directly focus

on physicists who do science communication or popular media.

1.2.4 Outreach and Public Engagement

Outreach and engagement are terms that are often used interchangeably by some people

while others define them as being different ideas. Physicists may say that they do outreach

when in reality they actually do engagement while some physicists describe their work as

7

engagement when they actually do outreach.

In the community engagement field, outreach refers to a one-way flow of knowledge from

the university to the public [38]. In this physics context, this frames the physicist as the

sole contributor of knowledge that the public audience is learning from, which can be very

deficit-oriented in many cases.

Engagement implies that the community and the universities have relevant knowledge

that they share with each other [38]. In the informal physics context, this implies that there

is a back and forth collaboration between physicists, audiences, and community partners in

creating events and activities that are meaningful and relevant to each party.

The physics community is adopting the term public engagement [40], however, it is

important to reflect upon the use of the word public. The public is often, though not always,

considered to be people who are not acting in the role of the expert [18], i.e.

in physics

public engagement, the public would be people who are not the physicists or physics students

facilitating the activities. This form of framing in informal physics programs, however, pins

physicists as not a part of the greater public. There has also been a history of framing “the

public” as scientifically ignorant and in need of a scientific expert rather than as people

with their own expertise and knowledge that they can contribute [41, 42, 43, 44]. While the

“public”” aspect of public engagement may be deficit oriented, it is the direction that the

physics community, leaning heavily on the engagement aspect to assume that physicists and

non-physicists are both learners in these environments [34, 45].

8

1.3

Informal Physics Programs

For this dissertation, I will be focusing on programs that facilitate informal learning spaces

centered around physics content and activities. Examples of these types of programs include

public lectures, planetarium shows, open house events, science festivals, summer camps, and

after-school programs [4, 34]. While we know about different styles of programs, there is

no national database of programs, and details about these programs are difficult to find

and sometimes inaccurate [4, 46]. There was a survey conducted by the American Physical

Society’s (APS) Forum on Outreach and Engaging the Public (FOEP) that asked members

about their public engagement and outreach work [47]. Over 68% of APS members who took

the survey reported engaging with the public with the most popular formats being public

lectures, demonstrations, and educational programs [47]. While these are the most popular

ways of engaging with the public, there is still limited information on the structures and

other details of this type of programming (audience demographics, program size, number of

personnel, etc) [4], hence my focus on informal physics programs for this dissertation.

While outreach and engagement are sometimes thought of as two separate things,

the programs that I analyze fall in varying points of a spectrum between outreach and

engagement. Multiple programs that I analyze are public lecture series where physicists give

a talk about a physics topic, usually about their research. This type of program is more

on the outreach end of the spectrum; however, some of these lecture series may implement

aspects of engagement. For example, a public lecture program may provide opportunities

where the physicists and audience members are able to converse with one another and ask

each other questions. Also, feedback opportunities could allow audiences to have a voice

and influence on the types of presentations. This type of program can collaborate with

9

community partners, such as event venues, to develop programming.

An example of a program that is high engagement is an after-school program that

partners with some teachers and hosts activities where students can create or design their

own experiments. In this case, the physicists may have the content knowledge, the teachers

have knowledge on working with K-12 students, and the students have more autonomy in

their physics exploration.

Throughout this dissertation, I will primarily refer to these informal physics programs as

engaging in public engagement; however, many of the research subjects interviewed in this

work may be quoted as saying outreach. The reality is that each program analyzed in this

dissertation rests somewhere somewhere within this outreach-engagement spectrum.

1.4 Physics Education Research: A Discipline-Based

Approach

Even though informal physics programs differ from classroom environments, they are

still educational learning spaces. Therefore, educational research practices are useful in

understanding some of the structures and experiences within these programs.

Discipline-based education research (DBER) focuses on understanding and improving

science and engineering education [48]. While certain skills and practices may be applicable

across different science and engineering disciplines, there are also challenges and goals that

may be discipline specific. Therefore, DBER studies utilize expert knowledge to complement

and inform the ways of knowing in that context [48].

Physics education research (PER) is one of the original DBER fields, with its origins

dating back to the late 1800s/early 1900s to address concerns about the quality of physics

10

education [48]. The American Association of Physics Teachers (AAPT) was established in

1930 to address some of these concerns [48, 49]. However, PER as a field of research did

not structurize more in the 1970s and 1980s when the first PER PhDs were earned and

research groups doing PER began to form within some physics departments [50]. Around

the 1990s and the early 2000s are when PER publications and the research community

increased substantially [48, 50, 51]. While PER has primarily focused on formal learning

spaces like classrooms, there has been a growing amount of research in informal learning

spaces in physics.

1.4.1 Physics Education Research in Formal Spaces

The majority of PER work is centered around classroom instruction, which includes

courses in K-12, two-year college, and four-year college settings, with the latter being

the most studied [52]. PER studies on undergraduate physics courses typically focus

on conceptual understanding, problem solving, curriculum and instruction, assessment,

cognitive psychology, and attitudes and beliefs about teaching and learning [53].

PER studies have had pivotal impacts in classroom structures and learning outcomes. For

example, the implementation of undergraduate learning assistants in the classroom makes

undergraduate students more comfortable with sharing their ideas and getting feedback [54].

Students in courses where they are able to interact with learning assistants tend to score

higher on concept inventories than students in classes without learning assistants [55].

Classroom structures have also evolved. Physics classes are moving away from traditional

lecturing to design set-ups where students work together in small groups and the instructional

staff, which may include the instructor, graduate teaching assistants, and undergraduate

learning assistants, are able to more easily engage students on a personal level [56, 57].

11

1.4.2 Physics Education Research in Informal Spaces

A growing sub-section of PER is informal physics education research (IPER) [58], which

primarily looks at physics education outside of typical classroom settings [34].

Informal

physics studies have looked at the impacts on audiences, finding that the positive impacts

on children include building scientific interest and identity in addition to overall content

learning [5, 6, 7, 8, 9].

Many informal physics facilitators are university students [4].

Studies have shown

that student volunteers build a variety of professional skills such as being able to

communicate to different audiences and implementing varying types of pedagogical practices

[24, 25, 26, 27, 28, 32, 33]. As informal physics spaces tend to be less structured than formal

spaces, student volunteers tend to have more autonomy and agency in their work, which

can contribute to building their confidence, physics identity, and participation in physics

[32, 33, 59, 60, 61, 62, 63].

1.5 Research Goals and Structure of this Dissertation

While there are noted impacts on informal physics audiences and volunteers, there is still

limited understanding on the structures of informal physics programs and the long-term

impacts on volunteers. This dissertation contributes to the growing field of IPER by studying

the important structures of these types of programs, particularly on the importance of

personnel for the operation of these programs. My research questions include: What are

the important structures of informal physics programs? What do the roles of personnel

in informal physics programs look like? What are the important aspects of programs that

12

program leaders can control? How can program leaders help to support their personnel? How

do the structures of these programs impact their student personnel long term?

Chapter 2 provides the methodological and study design considerations made to answer

these research questions. I used a case study methodology to compare and contrast informal

physics programs that differ in format and content covered. I apply an organization theory

framework to analyze the structures of programs and use a volunteerism framework to

study the motivations of student volunteers in these programs. Chapter 3 looks at the

core structures of informal physics programs, identifying personnel as playing a central role

in these programs.

In this chapter, we also find that program leaders have a desire to

conduct assessment within their programs, but lack resources, experience, and time to carry

that out. Chapter 4 is the development of a practitioner-friendly framework identifying

the key components of informal physics programs. This framework is meant to serve as

the foundation for the future development of assessment tools and resources that program

leaders can use. Building off of the findings of Chapter 3, this key components framework

identifies multiple key aspects for personnel. Chapter 5 analyzes the impacts of volunteering

in these programs on student volunteers’ careers and professional motivations. This chapter is

centered around interviews with alumni who had volunteered in an informal physics program

as a university student.

13

Chapter 2. Approaches to Study Design

As stated in Chapter 1, my research questions aim to 1) understand the structures of informal

programs in a way to help better support program leaders, and 2) understand the impacts

of informal physics programs on their student volunteers.

In this chapter, I lay out the

multiple frameworks and methodological considerations that inform the studies in the rest

of the dissertation.

Chapters 3 and 4 focus on my research primarily on the program structures. In Section

2.1.1 below, I turn to the business literature and use an organizational theory framework,

which has been used to study organizations like non-profits. Chapter 5 examines the

motivations of volunteer alumni in informal physics programs.

In Section 2.1.2 below, I

describe a volunteerism framework that I use in the informal physics space to test its ability

to measure student motivations in volunteering.

In Section 2.2, I describe key methodological approaches used in Chapters 3, 4, and

5. In Section 2.2.1, I describe my use of case study, specifically comparative case study, in

order to better understand the similarities and differences between different types of informal

physics programs. In Section 2.2.2, I discuss the multiple processes used to recruit research

participants.

For the studies in Chapters 3, 4, and 5, I utilized interview, survey, and site visit data.

In Section 2.2.3, I discuss the pros and cons of each of these forms of data. In Section 2.3, I

present my published work on how these types of data can be used together to form a more

holistic view of informal physics programs.

14

2.1 Theoretical Frameworks

As mentioned above,

in order to answer my research questions, I applied a couple of

theoretical frameworks to guide my research. Organizational theory has been used to

study the structures of different types of organizations, including non-profit organizations.

As described in Section 2.1.1.2, I use an organizational theory framework that has been

contextualized to the informal physics context in order to better understand the key

structures of informal physics programs.

In addition to studying the structures of programs themselves, I am also studying the

impacts that programs have on their university student volunteers. To do this, I apply a

volunteerism framework in the informal physics context by interviewing university alumni

who had volunteered in informal physics programs. This provided some insight into the

motivations that volunteers had for volunteering and the impacts that volunteering had on

their careers.

2.1.1 Organizational Theory

According to Lawrence and Lorsch, “An organization is the coordination of different activities

of individual contributors to carry out planned transactions with the environment” [64].

An organization is a group of people trying to achieve a set of common goals through

collaboration and distribution of tasks, although each individual may also bring their own

set of personal goals to the situation [64, 65]. Examples of organizations are non-profits,

businesses, and government agencies [64].

Since organizations are ubiquitous in many societies, a field of research has developed

to understand how organizations work. As defined in the in Organizational Theory, Design,

15

and Change [66], “Organizational theory is the study of how organizations function and

how they affect and are affected by the environment in which they operate.” This theory

is used to examine the structures of how tasks and resources are distributed to meet the

organization’s goals, the cultural norms and values of the people inside and outside of the

organization, and the actions taken to achieve the goals of those inside and outside of the

organization [66].

There are two levels of analysis in organizational theory: intraorganizational, which looks

at the internal relationships and structures, and interorganizational, which looks at the

relationships and environments external of the organization [67]. It is important to analyze

both the intraorganizational and interorganizational levels as both influence each other and

impact the organization’s approaches and ability to achieve its goals [67].

To help understand what organization theory is used to analyze, consider a business as

an organization. Within this business, the organization has its own goals that it is trying

to achieve. Within the business organization, there are workers, managers, and so on who

work for the organization; however, these people also have their own personal goals and

relationships with the other people in this business. An action that a business takes to

achieve its goals is to have labor contracts with its employees. Outside of this business,

the organization can also be involved in a larger network of other business which has its

own structures. The organization also needs to consider the relationships it has with other

outside organizations, such as suppliers and distributors. An action a business can take is

to subcontract with other organizations for particular tasks. Organizational theory is used

to analyze these different types of structures, relationships, and operations because each of

these components influence how the business works to achieve its goals [67, 68]. Table 2.1,

originally developed by Jaffee [67], details the levels of analysis with this example.

16

Table 2.1: This table was originally published in Jaffee [67]. This table shows the people,
structure type, and transaction between the organization and those it has relationships with.

Intraorganizational

Interorganizational

Definition Within the organization

Between organizations

Structures

Bureaucracy

Network of interorganizational relations

Relationships Workers, managers, coworkers Suppliers, manufacturers, distributors, regulators

Transactions Employment contract

Market exchange, subcontracting

The goals of a business, however, are quite different from those of an informal physics

program. For example, while informal physics programs may be concerned about money

in terms of being able to buy supplies for their activities, their goal is not typically profit

oriented. A type of organization that the informal physics programs more closely resemble

is a non-profit.

2.1.1.1 Organization Theory to Describe Non-Profits

The list of types of nonprofits is long. Examples include, but are not limited to, research

institutions, schools and universities, health organizations, humanitarian organizations,

foundations, and advocacy groups [69]. A precise definition of non-profit organizations varies

across countries and even fields of study. However, some core aspects are that non-profit

organizations are self-governing, profits are not distributed to directors or other members

in the organization, they are institutionally separated from a government (however they

can still receive funds and other supports from the government), and there must be some

voluntary or non-compulsory aspect within the organization [69].

Non-profit organizations have their own goals, as well as they have to consider the internal

17

relationships and goals with their members as well as external relationships and goals with

their stakeholders in order to achieve their goals [70], making organizational theory a useful

tool in analyzing nonprofits. People who have an influence on nonprofit organizations include,

but are not limited to, volunteers, paid personnel, collaborating nonprofits, community

partners, public administrations, private companies, schools, universities, and the general

public [71]. Each of these groups bring their own goals, contributions, power, and influence

in what nonprofit organizations aim to achieve and how they work to achieve those goals.

In their meta study, Tay¸sir and Tay¸sir did a deep literature review on the dimensions

of nonprofit organization effectiveness and the different ways that effectiveness has been

measured [70]. They compiled a list of effectiveness indicators from their literature review

and asked nonprofit organization board members and leaders to identify which dimensions

were most important to them for measuring the performance of their organization. From the

responses of 69 nonprofit organizations, Tay¸sir and Tay¸sir identified six main dimensions of

nonprofit organization effectiveness: Board (corporate governance), Managerial (strategic

planning), Resources (fundraising and recruiting volunteers), Financial (budgeting and

audits), Environmental (sustainability of technology used), and Program (stakeholder and

client satisfaction of projects) [70]. While these six areas are presented as separate, they

are closely related to each other and can influence each other. For example, if a nonprofit

organization is unable to fundraise enough funds, their budget will be limited, which will

impact the number of people they can hire and recruit to carry out their project tasks, and

that can negatively impact their relationship with stakeholders and clients.

18

2.1.1.2 Organization Theory to Describe Informal Physics Programs

In Chapter 1, I discussed the breadth of informal physics learning activities. Informal physics

programs share many characteristics that are also found in organizations. The programs

have structures in the format of their activities, there are power structures among those

running the programs, there are pressures and influences from outside sources like community

partners, public members, and institutions —all of which impacts the goals and steps to

achieving those goals.

Fracchiolla et al. used a nonprofit organization theory framework to analyze informal

physics, finding that there are similarities between nonprofit organizations and informal

physics programs [72]. Notably, they found that informal physics programs are adaptable to

their changing funding sources, align their goals with their own mission statements and their

funding partners, and have access to tangible and intangible resources like physical space

locations and volunteers [72].

Some elements of a nonprofit organization framework do not completely make sense

in the context of informal physics programs. For example, “board members” are not really

applicable in most informal physics program spaces. In Izadi, et al., elements of the nonprofit

organization framework were contextualized to be more relevant to the informal physics space

[73]. In their contextualized framework, informal physics programs have six main dimensions:

Personnel, Program, Resources, Audiences, Institution, and Assessment, as shown in Fig.

2.1. Definitions of each dimension are listed in Table 2.2.

19

Table 2.2: Definitions of each of the six dimensions of the organizational theory framework
contextualized for informal physics programs. This table was originally published in Izadi,
et al. [4].

Categories Definitions

Personnel The people involved with the functionality of the program.

Examples: directors, presidents, volunteers, graduate students.

Program Logistical and programmatic details of the organization.

Example: content, structure, history, objectives.

Audience

The people that the organization is reaching.

Example: local communities, K-12 students, college students.

Resources The physical and financial assets that the organization utilizes.

Example: donations, physical spaces, grants.

Institution The umbrella establishment that the program is housed under.

Example: Universities, National labs, Physics centers.

Assessment Evaluations regarding the organization, (sometimes conducted to evaluate itself).

Example: Survey taken by the audience, and research done on the efficiency and/or effectiveness of the program.

20

Figure 2.1: This figure shows the six dimensions the organizational theory framework adapted
to the informal physics context. Definitions of each dimension are listed in Table 2.2. A
variation of this figure was originally published in Izadi, et al. [4].

In Chapter 3, I will use this contextualized framework to do a deeper analysis on the

structures of informal physics programs. I will analyze all six dimensions with a heightened

focus on the Personnel dimension to understand the roles of personnel in informal physics

programs. Chapter 4 will expand off of those findings and look at the development of a

framework that can be used by informal physics researchers and practitioners to evaluate

programs.

21

2.1.2 Volunteerism Framework

As will be discussed in Chapter 3, the most prominent organizational theory dimension is

Personnel. Given the central role that personnel have in informal physics programs, I am

interested in the motivations of people who volunteer in these spaces. In Chapter 5, I test

a volunteerism framework to understand why university students volunteered in informal

physics programs and how it impacted their career pursuits.

In the midst of the COVID-19 pandemic, 23% of Americans formally volunteered through

some sort of organization while over 50% of Americans engaged in some form of informal

helping [74]. Prior to the pandemic, those numbers were even higher [75]. Examples of formal

volunteering include public health efforts, support food banks, and mentoring students, while

examples of informal helping include doing favors for neighbors such as lending a hand and

watching each other’s kids, all of which can benefit the greater community [74].

There are also benefits for the volunteers themselves, for example, improved physical and

mental well-being, job and career opportunities [76]. In addition, volunteers gain leadership

and communication skills in addition to building upon other specialized skills that they

can use in their own work place [77]. These benefits also influence some of the volunteers

motivations for volunteering in the first place.

People may volunteer in the program or activity for different reasons. Clary and

Snyder present six motivational categories for volunteers as described in Figure 2.2: Values,

Understanding, Enhancement, Career, Social, and Protective [78, 79]. This framework

originates from work in psychology and was tested in a variety of volunteering contexts

like public health, business, and student work [78]. Listing six categories does not imply

that they only volunteer for one of these reasons. Volunteers may have multiple motivations

22

of varying weights that influence their involvement. In addition, if a volunteer stays with a

program or activity for a while, their motivations for doing so may evolve over time.

Figure 2.2: This figure shows adapted definitions of the six volunteerism categories
[78, 79, 80, 81, 82, 83]. This figure origninally appeared on a poster presentation at the
2023 Physics Education Research Conference in Sacramento, CA [84].

While this framework originally aimed to describe volunteerism more broadly, these

motivational categories may need to be adjusted or adapted to the specific context that

it is being applied to [78]. For example, this framework has been used in a variety of

scientific contexts, but there were changes to the category names or some additions based

on the situation. Wright, et al. applied this framework to a bird conservation citizen science

program to study the relationship between volunteers motivations and their satisfaction

within the program [80]. Similarly, Domroese and Johnson studied the motivations of

volunteers in the Great Pollinator Project and how well the program met those motivations,

ultimately making adjustments to their program protocols and training procedures [81].

Bruyere and Rappe found the same types of motivations in the framework to be present

23

in their study of volunteers in multiple natural resource and land management organizations,

but they also found another category unique to their specific context in which stated that

a motivation for them to volunteer was to “get outside” [82]. This is important to note

because going outside is a context-specific motivation that is not relevant or applicable to

other forms of volunteering.

Like in the Bruyere and Rappe report, it is important to also apply this framework to

other disciplines, like informal physics programs, to identify context-specific motivations. It

could be that some types of motivations are less relevant to informal physics programs, and

there could also be additional motivations that are unique to informal physics programs. In

Chapter 5, I apply a volunteerism framework to identify and analyze the motivations and

career pursuits of alumni who had volunteered in informal physics programs as university

students.

2.2 Methodological Considerations

To answer my research questions, I used qualitative research approaches to understand the

nuances and decision making that takes place within informal physics programming.

In

Section 2.2.1, I describe the methodological considerations I made in my research. I utilized

comparative case study to analyze programs of varying format, audiences, and physics

content to identify commonalities across different program types. To gather data for these

case studies, research participants were recruited through convenience and snowball sampling

approaches as described in 2.2.2. I collected and analyzed survey, interview, and site visit

data, with each form of data providing a different perspective of a program as described in

sections 2.2.3 and 2.3.

24

2.2.1 Case Study

Until around the 1970s and 1980s, education research was primarily quantitative [85].

Around this time, the use of case study in the USA and UK became more prominent as a

reaction to the research community’s and policy maker’s bias for statistical analysis [85, 86].

Case studies provide a valuable context a nuanced perspective of a studied situation [87].

For example, a quantitative study may show what someone did, but a qualitative case study

can provide context for intentions and motives [88].

What makes a case study is up for debate though as definitions vary across time and

researchers [85]. Yin, whose background is quantitative research, views case study as a

research method that can be used to explore, describe, or explain the item being studied

[89]. Yin also applies traditionally quantitative views of validity checking in case studies,

however, some critics argue that such validity checks are oversimplified for the complex nature

of education research, and thus require different definitions for quality research [85, 89]. As

Hamilton and Corbett-Whittier note, Merriams definition of case study has evolved from the

description of a phenomenon to defining the case as a clearly bounded unit [21]. As a third

example of a differing view on case studies, Stake views case study as a way to evaluate or

learn about a specific person or program [90, 91].

There continues to be debate on whether case study is a method, methodology, or research

design [85]. For my work, I apply the framing used by Hamilton & Corbett-Whittier and

Elliott & Lukeˇs who view case study as an “approach to research” [85] and a genre of

research [86] that uses a wide array of data types to “capture the complexity of relationships,

beliefs and attitudes within a bounded unit” [85]. Throughout my research, informal physics

programs as my cases. I am studying the complex nature of these programs, notably the

25

perspectives and relationships of the personnel facilitating within these programs.

The specific case study approach that I am implementing in my research is comparative

case study. A comparative case study allows for phenomenon to be tied to its specific

context while also exploring the similarities and differences across other cases [92]. My goal

with using this approach is to be able to compare the similarities of these different types of

informal physics programs and be able to make claims about these programs more broadly.

For example, in Chapters 3 and 4, I am analyzing the key structures of informal physics

programs. However, if I just analyzed a public lecture series, there are some phenomena that

may only be relevant to the context of public lecture series and not other types of programs,

for example after school programs. Therefore, I analyzed informal physics programs that

differed in format, content, audience demographics, and type of personnel. As discussed in

Chapter 1, I am only analyzing informal programs where physicists and physics students

are engaging with audiences. I am not analyzing science communication and popular media

work that individual physicists may do.

2.2.2 Recruiting Research Participants

This work could not have been done without the involvement of research participants. For

the studies in Chapters 3, 4, and 5, there were several methods for recruiting participants

as described respectively in each of those chapters. The main two methods used were

convenience sampling and snowball sampling.

2.2.2.1 Convenience Sampling

Convenience sampling what the name implies, selecting participants because it is convenient

[93]. A classic example of convenience sampling is in psychology and social science research,

26

where the majority of research participants are college students [94, 95, 96].

In these

fields, there have been arguments for and against using college students in these studies,

but ultimately, the appropriateness of using these populations is dependent on the claims

and research questions that the researchers are trying to make (for example, it may be

inappropriate to conveniently use college students in a study and try to make broader

claims about non-student populations as students may have differing cognitive skills or social

relationships than older adults) [95]. That is not to say that convenience sampling is bad,

the researcher just needs to clearly state the assumptions that they are making and the

potential limitations.

Izadi et al., which my work is a continuation of, implemented a convenience sampling

approach of surveying informal physics programs across the state of Michigan [4]. They

surveyed programs across Michigan because most of that research team was based in

Michigan and had some familiarity of the programs and could more easily travel to some of

them. My work in Chapter 3 also looks solely at programs in Michigan. There are some

limitations with only looking at Michigan programs. Programs in Michigan may not be

representative of programs across the country. The needs, goals, and resources for programs

could vary state to state. As such, additional participants were recruited around the country

based on personal connections with the research team. Based on these personal connections,

we were able to recruit participants running programs from across the country, which I

report on in Chapter 4. A limitation from that is potential bias from the research members

in analyzing the data. While I and the other researcher members attempt to minimize our

bias, our knowledge and familiarity with various programs can influence the type of questions

we ask and how we interpret participant responses. Our knowledge and experience can be

beneficial as we can ask more detailed and probing questions, but it can also influence our

27

interpretation.

2.2.2.2 Snowball Sampling

Often used in qualitative research, snowball sampling is when researchers ask some initial

number of research participants to recommend other people who may be interested and meet

the goals of the research project [97, 98]. A benefit of snowball sampling is that it is easier

to access otherwise difficult to reach populations as there may be a level of trust between

the research participants, however, there also comes a risk of selection bias as the sample is

dependent on the researchers initial sample of participants [97, 99].

A snowball sampling approach was used in Chapters 4 and 5 in order to reach programs

outside of Michigan. Some research participants were recruited through other means like

at national and regional conferences, however, such approaches may overrepresent program

leaders who were tenure-stream faculty with funds to be able to attend such conferences. As

found in Izadi et al., program personnel are mostly university students and faculty members

lower down in the academic hierarchy who may not have the resources to attend such

conferences [4]. Therefore, using a snowball sampling approach helps to reach programs

around the country that we may not have otherwise been able to reach.

2.2.3 Types of Data

For my research projects, I used a combination of survey, interview, and site visit data. Each

form of data provides its own set of information and perspective on a particular situation,

each with its own pros and cons.

Survey data is good for logistical information. As shown in Izadi et al.

[4], survey

data can provide information like the number of personnel in an informal physics program,

28

the main jobs of personnel members, the content and topics covered by the program,

program format, types of activities, how long the program has operated, number of audience

attendees, etc. This type of data provides insights of what informal physics programs look

like broadly. However, the survey information is limited to the knowledge and perspective

of the participant filling it out. In addition, survey data provide limited insights on decision

making within the program and do not easily allow for follow-up questions.

Interview data allow for more nuanced insights on decision making rationale. For

example, in Chapter 3, I analyze interview data from program leaders in which the program

leaders provide detailed explanations on how they choose program content and activities, the

rationale on how tasks are distributed among personnel, the relationship between community

partners, etc. While in an interview, we could ask the same questions that were in a survey,

interviews already take a lot of time to conduct. By implementing a survey first, we can

get logistical information more easily, and then use interview time more effectively to ask

follow-up and clarifying questions to participants’ responses. Similar to surveys, interview

data is limited to what the participant says and then the researcher’s interpretation of their

responses.

Site visits are when the researcher visits the program to observe the program in person. I

have visited and embedded with programs to witness how programs prep and organize their

events as well as interact with the members of the program. Data from site visits include

field notes, physical artifacts, and media (video, pictures, etc). Doing so provides context

for the scale and operation of the program being analyzed. Unlike surveys and interviews,

site visits are more so limited to the observations and perspectives of the researcher.

All three of these types of data have their strengths and weaknesses. Collecting and

analyzing all three types of data creates a more holistic view and understanding of informal

29

physics programs. For a more detailed description of this, see my published work in the

following section (2.3). The findings of that work can be found in section 2.3.5.

2.3 How Site Visits can Help us Learn More

I have used site visits in my work to gain further perspective and first-hand observation of

how some of the programs I am analyzing operate. I have found site visits to provide valuable

information that complements other forms of data like interviews and surveys. The rest of

this section was published as an article for the 2020 Physics Education Research Conference

Proceedings [100] and is provided here verbatim with only formatting changes. The author

order on this article is Bryan Stanley, Dena Izadi, Kathleen Hinko.

2.3.1 Introduction

Informal physics learning environments refer to learning spaces outside the traditional

classroom setting. This may include programs such as a science cafe, a physics open

house, or a demo show. Physics education research has demonstrated that the local

context and implementation of curricular materials in classroom settings critically affect

the impact of those materials on students. For example, a variety of factors such as faculty

buy-in, the presence of LAs, backgrounds of students, and if the chairs and tables in a

classroom are movable, all can affect the way students experience curricular transformations

[54, 55, 56, 57, 101]. Our project seeks to characterize the even more diverse variables

that are in play in the local context and implementation for informal physics environments.

Without this information, we cannot diagnose ways to improve the learning experience or

design different formats that more meaningfully engage audiences.

30

The landscape of informal physics learning environments is complex as programs can

vary drastically in their structure, activities, contents, and engagement [46, 72, 73]. Due to

the complexity and richness of the landscape, it is difficult to evaluate the functionality and

effectiveness of the programs. While informal efforts are often the public face of their host

institutions, there is limited understanding compared to the more studied formal learning

counterparts.

It is necessary to study informal learning environments more rigorously as

these programs and activities not only impact the public, but also those participating in the

programs. Without a clear understanding of the resources, structures, and environments of

the programs, we are limited in our abilities to critique and evaluate the programs and their

impacts.

This work is part of a broader project to study the landscape of informal physics in effort

to better understand the key factors that play into the effectiveness of informal programs.

The general goals of the project are to determine what aspects are important in an informal

program and how those factors are connected with each other. Understanding the challenges

that programs encounter, as well as the internal and external dynamics of the programs, can

provide insight on the functionality of these programs. To study some of these factors, we

have already implemented and analyzed surveys and interviews with program facilitators.

This study builds off our previous work that collected interviews/surveys about program

logistics from the lead program facilitators [73]. From that work, broad organizational themes

emerged as program, personnel, audience, resources, institution, and assessment, as well as

finer sub-codes for each category. These data, however, are limited to the perspective of the

lead interviewee.

Here, we aim to address the limitations of the interview/survey data by expanding our

methodology to include site visits to informal physics programs. We hypothesize that data

31

collected from site visits will complement the interview/survey data by revealing emergent

sub-themes that were otherwise filtered out. Thus, the analysis we present here will further

guide the methodology of the broader landscape project. We seek to characterize the factors

that are important to how an informal physics program functions by looking at a site visit

from an organizational theory lens. We draw from the business literature on organizational

theory, which breaks down the different operational aspects of organizations. Taking an

ethnographic approach, we collected field-notes and artifacts, and recorded other interactions

during a site visit to a large informal physics program in the midst of running its annual

community-wide event. From these data, we looked to see if new emergent themes came up by

using organizational theory as guideposts. Then, we examined the content of these emergent

themes, focusing on how these new data complement the already collected interview/survey

data. We find that site visits provided significant insight into how personnel, resources and

audience were important during informal physics program events in ways that were not

salient from the interviews and surveys. For the broader project, this research shows that

site visits will be critical into establishing full profiles of how informal physics programs

function.

2.3.2 Contextualizing Organizational Theory for Informal Physics

Programs

Programs may have multiple different goals and objectives they want to achieve,

for

example, wanting to focus on specific content or increasing engagement of certain age

groups. The individuals participating in these programs may share the same goals, but

their priorities or reasons for participating may also differ, for example, improving teaching

32

abilities or experimental skills [70]. We take an organizational theory perspective to

better understand the functionality and dynamics within these programs. Organizations

operate under their own structure, and their performance is influenced by a multitude

of components such as resources, the working environment, and the social relationships

both internally and externally of the organization, and power dynamics within these

relationships [70, 102]. Organizational theory looks at the connectedness and weighted

emphasis of these components’ impact on the organization [102]. Informal physics programs

have complex dimensions in terms of how the personnel and facilitators interact with each

other and their audience.

As established in previous work, we believe that informal physics programs have many

similarities to non-profit organizations [72]. Similarities include, but are not limited to,

having the key components of personnel, stakeholders, program, and management; however,

some of these terms need to be more appropriately reframed in the context of informal

programs [102]. We have adapted these broad categories to an informal context. The

six broad categories of factors that affect an organization’s performance are: personnel

(the people involved within the organization), programs (activities and logistical details),

audience (the people the organization is reaching),

institutional connections (the entity

that the program is housed under, like a university or national lab), resources (financial

and physical utilities), and assessment (self-evaluations by the organization) [102]. By

contextualizing organizational theory in this space, we aim to create a multidimensional

evaluation of informal physics programs.

33

2.3.3 Case Study

In this paper, we focus on a case study of an open house style event held by a program we

will refer to as “House Physics”, which is housed in a large western R1 university. Site visits

are more resource exhaustive for the researcher than interviews/surveys. We are taking a

case study approach to guide our site visit methodology for future site visits. Before the site

visit, the lead facilitator of House Physics, who is a physics faculty member, was interviewed

by a member of the research team about program goals, basic logistics, activities, history,

and recruitment and training of student volunteers. This interview was a little over an

hour in length and had been previously analyzed for themes with the organizational theory

constructs [72].

House Physics is a traveling physics outreach program that visits schools with hands-

on experiments that undergraduate interns built. We chose this program as a case study

for several reasons. First, two of the House Physics co-directors had participated in an

earlier phase of the broader study and contributed interviews about House Physics.

(In

subsequent data collection, we split the single long-form interview into a combination of a

survey and shorter interview) [73]. These interviews had been analyzed with a version of the

organizational theory framework and a preliminary list of sub-codes were developed for each

of the six main categories. From this analysis, it was clear that House Physics is a robust

organization: it has a large number of university students helping with the program, it has

long-term institutional support, and it undertakes many educational physics activities each

year. However, its large size and scope as an informal physics organization made it difficult

to determine a holistic picture of its organizational features from a few interviews alone.

We determined that a site visit would allow us to explore the nuances of how, for example,

34

volunteers interacted with audiences or with the program director, or the role of physical

resources, like equipment.

House Physics holds a free annual open house event on the university campus. We

decided to use this event and a subsequent planning meeting as the key events to collect data

from on the site visit. The event consists of several ballrooms filled with hands-on science

experiments, demo presentations, fundraising sales booths, and spaces for their science

partners. The open house is advertised to the general public and has an attendance of about

8,000 to 10,000 people. House Physics has three co-directors who are physics and astronomy

instructors at the university, two additional employees, and about 15 undergraduate interns.

Some of the undergraduate interns are physics majors, but most of the volunteers are from a

variety of other academic fields. For the open house event, there are about 50 volunteers who

were partners or former interns, as well as approximately 125 undergraduate volunteers who

were the current students of the directors’ physics courses. Another important consideration

is that the first author of this paper was recently a former House Physics undergraduate

intern with the program. His close acquaintance to the House Physics co-directors and

many of the House Physics volunteers allowed him to have access to the program events,

behind the scenes activity, and have rich conversations with the program personnel.

2.3.4 Methods

For the site visit, the first author embedded with the House Physics group for two full

days. The first day was the open house event and the second day was one of their weekly

preparation days. He observed and took detailed field-notes of the open house events,

including the opening meeting, volunteer training, and demo presentations. Similarly, he

documented the prep day activities, including meetings, experiment building, and video

35

production. Notes included observations of the social dynamics and interactions within the

program, descriptions of informal interactions with personnel members, and documentation

of programmatic and logistical details [103, 104, 105]. Photographs of the events and other

artifacts such as flyers and program maps were also collected [103, 104, 105].

He also conducted semi-structured interviews with two of the co-directors using an

interview protocol based on the broad organizational theory categories during the visit.

One of the directors, who identifies as a white female, also serves in an astronomy teaching

role at the university. The other director, who identifies as a white male, also serves in a

physics teaching role at the university. The interview with the latter director is a follow-up

to an interview originally conducted prior to the site visit.

Data analysis began with a single coder coding the field-notes and interview data in

MAXQDA, using the thematic categories developed for the overarching landscape project:

personnel, program, institution, resources, audience, and assessment [73]. In addition, the

coder explored emergent sub-codes and themes in just the field-note data, later discussing

with the research team to consolidate into more concise labels. The developed sub-codes

were compared to those developed for the pre-site visit interview data to provide insights on

the additional dimensions that a site visit brings to understanding a program.

2.3.5 Findings

From our analysis we have two main results: 1) we found over a dozen sub-codes describing

key aspects of this program that were otherwise undetected or unverified via interview

with the lead facilitator, largely in the category of personnel, 2) we observed a number of

instances where the theme of ‘personnel’ overlapped substantially with other categories, such

as ‘resources’ and ‘audience’, demonstrating the complex interwovenness of these categories.

36

The sub-codes generated from the interview and survey data tended to be more

programmatic, such as learning about a personnel member’s responsibilities, the physics

content topics, or funding. The site visit, however, revealed the ways in which informal

physics programs function as a social enterprise with respect to the main themes. For

example, sub-codes emerged as to how personnel interacted with each other and how the

program and personnel utilized available resources. We found that the majority of our

emergent sub-codes centered around ‘personnel’. Here we describe some of the key personnel

codes in more detail.

2.3.5.1 Emergent Themes in Personnel

One aspect we are interested in is the social dynamics within informal programs. In our

interview with the House Physics lead facilitator, he talked about how the team dynamic

plays a role in guiding the creative agenda and identity: “The other thing which we do is we

definitely are a team. When we meet we just meet around a table, and mostly we don’t have

presentations, mostly we have just brainstorming, like okay, here’s what we have in mind for

this year, what kind of projects would people like to see? What kind of focus would we like

to have?...So they’re really very much part of the team, and I think they get that. I want

them to really own it.”

In the existing codebook, that would be coded as a socio-cultural aspect of the personnel,

but there are limitations in what this tells us. From the site visit, we saw more specific details

that could not be extracted from the interview. We were able to witness program facilitators

repeatedly recognizing personnel members. For example, the facilitators mentioned several

times in multiple group meetings to the undergraduate volunteers that they are the ones who

make the program successful. Sometimes, recognition was given on a more personal level,

37

such as when an intern made a new physics demonstration, and the group gave praise toward

it, and the lead facilitator asked if they could use it for their event. There were repeated

actions like this observed that inspired a potential new code regarding the recognition of

personnel by the facilitators.

Another new type of code also emerged by observing personnel on site. By sitting in

on the weekly prep meeting (as described above by facilitator) and by observing personnel

interactions throughout the activities, we observed multiple group aspects such as socializing

and working in groups as emergent sub-codes that were otherwise absent or minimized in the

interview process. For example, there were moments where personnel members were sitting

around a table building a demo together while also laughing and sharing casual conversations

about non-work related topics.

When the researcher had direct interactions with the program personnel, the volunteers

were willing to converse in informal conversations, often with personnel members initiating

the interaction. These interactions provided different data points than what could be

extracted with a long, formal interview. They were brief and more conversational, which

brought in a broader range of perspectives from within the program and allowed for much of

the discussion to be driven by the members’ point of view. This led to additional emergent

sub-codes such as reflecting on past experiences they had in the program. From both outside

observation and one-on-one casual conversations, many members reflected on past events

they participated in, how those experiences stuck with them, and how that influenced their

future goals and aspirations. One former intern talked about how the skills they learned

from building demos transferred over to their current job. Another volunteer talked about

how they decided they wanted to pursue education after participating in the program.

38

2.3.5.2 Overlapping Codes with Personnel

We also observed multiple cases where emergent sub-codes overlapped personnel and another

of our themes. One of the most prominent that we saw was the relation between ‘personnel’

and ‘resources’. With the current interview codebook, there is a broad label for the

personnel’s role in the program, which refers to the member’s tasks and duties. However,

this code does not give insight into how the personnel complete these tasks. During the

site visit we found an abundance of personnel members interacting with physical resources

and were able to observe how these interactions take place. These interactions manifested

in many different forms. We observed the personnel members transporting, setting up, and

taking down event projects. Repairing projects was an activity that volunteers engaged in

during the open house and the general workday, as well as interns creating and building

their own projects. From the interview with the director prior to the site visit, we extracted

information on personnel roles and physical resources as separate categories.

Another crossover theme was between ‘personnel’ and ‘audience’. From casual one-on-

one conversations and sitting in on group meetings, we observed instances of personnel

members recounting interactions that they had with the audience. For instance, in the

weekly prep meeting that occurred after the open house, the personnel members reflected

on their experiences at the event, with several members recalling interactions with families

and other visiting groups. One volunteer talked about the positive conversations they had

with a senior citizen group that attended the open house. From an interview, the personnel

and the audience are looked at almost as separate contexts, but the site visit helps show the

overlap.

The information we can extract from the interviews on the interactions between these

39

two categories are broad and general, so the codebook is limited in identifying these overlaps.

Such limitations make it more difficult to fully describe the program characteristics. Direct

observation during the site visit allowed for codes across multiple themes to appear connected

in a larger context. From the site visit, we observed how the personnel interacted with the

audience, such as through casual one-on-one conversations. We also have an emergent sub-

code of the personnel members reflecting on their interactions with the audience and sharing

these stories with other personnel in both socializing and work meeting settings. We can

see the interaction of the personnel and resources, but also the social interactions between

members, where a lot of these emergent sub-codes seemed to blossom.

2.3.6 Discussion

2.3.6.1 Benefits of a Site Visit

In this paper, one goal was to test a methodology for conducting site visits to informal physics

programs. In our other work, we found that interviews with lead program facilitators can

provide long, detailed information of the program in about an hour time frame [73]. However,

since the operation of a program is complex and we are limited to the single perspective of

the facilitator, these nuances are not readily apparent and may serve as confounding factors

in the success and challenges of these programs. Taking an ethnographic approach for the

site visit allowed for a large amount of rich data to be collected. The facilitator interviews

present more of ‘what’ the program does. What are the key points that we know about the

program? We can learn what the top-level programmatic details are, what the main roles

of most of the personnel members are, who the majority of the audience is, and what the

funding sources are. Through a site visit, we are able to see more of ‘how’ the program

40

does all these things and how these themes connect with each other on a much finer scale.

We are able to have brief, informal conversations with other personnel members to get their

perspectives on the program, their role, and their experiences, in a more natural setting

than with a longer, formal interview. In future research, we can more carefully analyze the

importance of these codes and what role they might play in the success and sustainability

of programs, even seeing if this occurs in other programs.

The key takeaway in this work is that we were not able to get these types of data from

doing a singular, formal interview with the lead facilitator. Arranging longer, more formal

interviews with as many personnel members as we had casual interactions with during a

site visit is not easily feasible, especially for larger programs. The interview data allowed

for a deep look through a single perspective at each of the overarching themes that we have

established. The site visit allowed for many perspectives on just a few of the overarching

themes, in this case, primarily ‘personnel’ with previously unseen overlaps with themes such

as ‘audience’ and ‘resources’. This finding suggests that neither formal facilitator interviews

nor site visits should replace one another when studying informal physics programs, but

rather these two approaches are best suited to gather different types of data that complement

each other when studying these programs.

2.3.6.2 Limitations with Data Collection

There are some challenges that arise with conducting a site visit, the most prominent being

the actual data collection.

Informal physics programs vary considerably, so the ease of

information collection and the type of information can vary. For an all day event with over

100 personnel members and 10,000 visitors spread throughout an entire building, it is not

feasible for one person to document every detail.

41

With a large scale event like this program, there appeared to be minimal interruption

in the flow of the event as the research could sit in the background and blend in with the

crowd to observe. From the lead researcher’s prior experience with the program, the event

appeared to run naturally like past events, giving little indication that the program or the

personnel were acting more rigid while being observed. The invasiveness of the sight visit

should be handled with care. Site visits to smaller programs could potentially have higher

risk of disrupting the event. Another possible limitation could be with visiting programs

that we are less familiar with, which could impact data collection.

There needs to be a plan beforehand on key aspects to attempt to observe. Having

interview data before the visit helped to highlight key details to look out for that the

facilitator deems as important aspects of the program, such as how people share in a

group meeting. However, there also needs to be flexibility in the plan for the researcher

to appropriately adjust to the active surrounding environment to collect crucial data as they

appear. During the open house, there were several times that the researcher was moving

from one location to another to observe, and was stopped by personnel members who wanted

to discuss their experiences in the program. While the original goal in that moment was

to observe audience interactions, it was difficult to pass up the opportunity for a brief

conversation with a participant willing to reveal their rich perspectives on the program.

There needs to be a balance between the researcher following their plan to observe specific

components, as well as being flexible enough to react in the moment to what is happening

around them.

In future work, we intend to conduct more site visits to expand on these themes and

enhance the site visit methodology. We will also refine our emergent sub-codes to expand

our codebook through rigorous analysis and discussion. The more finely tuned codebook

42

will be used to compare more thoroughly with our interview data to aid in examining key

themes and subtle nuances within these programs. In the broader project scope, we aim to

find indicators that most impact the sustainability and success of informal physics programs.

We hope this methodological process helps those interested in examining their own learning

spaces consider how their data sources can work together for more holistic understanding.

This work is funded by NSF AISL #1423496.

43

Chapter 3. Central Role of Personnel

in

Informal Physics Programming

This chapter was published as a journal article in Physical Review Physics Education

Research in 2023 [106] and is provided here verbatim with only formatting changes. The

author order on this article is Bryan Stanley, Dena Izadi, Claudia Fracchiolla, Kathleen

Hinko.

3.1

Introduction

People can learn physics in many different spaces outside of traditional classrooms. Learning

that occurs outside of the classroom space is referred to as “informal,” “nonformal,” or “free

choice” learning [34, 107, 108, 109] in the field of education. In informal education settings,

whether and how to engage in learning is determined mostly by the individual, and learning

goals for the facilitators and/or the audience members can include excitement and identity

growth along with scientific content and practices. Examples of informal education activities

that are focused on physics content include, but are not limited to, afterschool programs,

summer camps, public lectures, online videos, podcasting, physics festivals and open house

events, planetarium shows, online games and apps, and demonstration shows [4, 34, 110].

Physics faculty, undergraduate students, graduate students, and postdoctoral researchers

often volunteer or act as lead facilitators in these spaces [111]. The physics community

more commonly uses the words “outreach,” “public engagement” and in some cases “science

communication” to describe how they interact with nonphysicists in informal educational

environments [34, 40]. In this paper, we use the term informal physics programs to describe

44

organized activities for public audiences that are led by physicists.

Studies in physics education and science education have looked at the impact of these

programs [112, 113]. Ours and others past studies have focused on audiences, especially

children and mixed-age groups, and have established positive impacts on building interest,

identity, and content learning [5, 6, 7, 8, 9]. There is also research looking at the

physics students and physicists who participate. Studies have shown that physics students

volunteering in informal physics programs are able to build communication skills and improve

their pedagogical practices with various demographics of audiences [24, 25, 26, 27, 28]. In

addition, these spaces often provide physics student educators with agency over activities,

which can increase their confidence and abilities to adapt their pedagogical practices quickly

[24]. Some of our previous studies have also looked at the impact that facilitating these

programs has on university students’ physics identity and sense of belonging in the physics

community [59, 60, 62, 114].

In particular, ours and others past studies have shown

that students who volunteer in informal programs can gain a sense of autonomy, build

relationships with their peers and faculty, connect their formal learning to the real world, and

see themselves in their fieldall of which can strengthen their participation in their respective

fields [32, 33, 60, 63, 114, 115].

It is clear from existing research that both audiences and physicists can be impacted by

their experiences participating in informal physics learning; however, we do not know how

the structural design, organization, and operation of informal physics programs affect these

experiences. Collectively, informal physics programs encompass a large range of audiences,

formats, and content; as such, programs are organized and function in a variety of different

ways.

In prior work, we characterized the structures and features important to informal

physics programs by adapting an organizational theory framework to the context of informal

45

physics spaces. This contextualized framework has six categories that encompass the main

functional elements of an informal physics program: Assessment, Audience, Institution,

Personnel, Program, and Resources (described in the following section) [73]. Previously, we

reported on logistical information for aspects of each categoryfor example, sizes of audience,

number of volunteers, and the type of physics contentfor a case study of a single state [4].

In this study, we go beyond program logistics and take the next step of framework

development to gain a deep understanding of how the different categories affect each other.

To investigate connections between framework categories, we collected extensive interviews

with informal physics program leaders about their program design and operation. We then

analyzed each interview for emergent program elements within each framework category and

looked for areas of overlap. As described in the Sec. 3.4, we observe the prominent role

that personnel, the physics students, staff, and faculty involved in the educational activities,

have in these programs. Based on this finding, we then shifted our analysis to focus on how

the personnel influence the content, activities, and audience interactions of informal physics

programs.

In this paper, we present an in-depth case study analysis of three programs selected from

a broader dataset we have collected. These programs have different formats, physics topics,

and audiences, and they showcase diverse and effective ways that personnel are integrated and

integral to the program. We present the cases of a physics and astronomy public lecture series

for adults held at a local bar, a weeklong summer camp for high school students at a research

laboratory, and an undergraduate student club doing astronomy open houses for general

public audiences. Our findings show the important role of the personnel of informal physics

programs and the influential connections they have with the other organizational components

of audience, resources, and program content and activities. Ultimately, understanding the

46

complexities between framework categories will allow researchers and practitioners to identify

areas to improve informal physics programs, especially in being inclusive, in being equitable

and accessible, in supporting physics students who participate, and in connecting more

strongly to the community and home institution.

3.2 Framework

This work is part of a bigger project to understand the landscape of informal physics,

identifying common themes, successes, and challenges that are present among the many

diverse styles of programs [58]. Some studies have looked at aspects of the landscape, such

as the 2015 American Physical Society (APS) Forum on Outreach and Engaging the Public

(FOEP) survey and the SySTEM 2020 project in Europe; however, those studies focus on

the content, types of activities, and the audiences that the physics community engages with,

and not at the physicist facilitators or the ways programs are structurally organized [47, 116].

In this study, to gain an understanding of how the different organizational structures of

informal physics programs are interrelated, we draw on our prior work contextualizing an

organizational theory framework to informal physics spaces [72, 73]. Organizational theory

originates from the business literature and describes the internal and external relationships

present in organizations. Organizations, including nonprofits, consist of various social groups,

each with their own goals, training, and contribution to organization [66, 69]. Aspects like

social and cultural relationships, power dynamics, monetary resources, physical resources,

and external partnerships all

influence the functionality of the organization [70, 102].

Nonprofit organizations and informal physics programs share some common characteristics:

they often have short-term or grant dependent funding, they depend heavily on volunteers,

47

and their planned activities are flexible to the needs in local communities. When adapted

for informal physics, the dimensions of the contextualized framework consist of six broad

categories [73]: Personnel, Program, Audience, Resources, Institution, and Assessment,

defined in Table 3.1.

Table 3.1: Defining the dimensions of the informal physics organizational theory framework.

Informal physics dimensions Definitions

Assessment

The form of evaluations that the program conducts.

Audience

The people engaging with the content and activities.

Institution

The larger organizational entity that is affiliated with the program.

Personnel

The people involved in the programs functionality.

Program

Logistical aspects of the content, activities, format, and other administrative details.

Resources

The financial and physical items of the program, as well as partners that program works with.

Using this framework, we identified parameters that differentiate informal physics

programs, such as the size and demographics of the audience, the number of volunteers,

pedagogical practices, physics topics, and connections with institutions [4]. We are able to

analyze the complex dimensions of informal physics programs as they pertain to how the

lead facilitators, volunteers, and audience members interact with each other, the supplies and

funds, and relationships with other organizations. We also found the goals and objectives

in informal physics spaces may differ among programs and the participants. Some programs

prioritize awareness or understanding of a certain topic while others aim to boost engagement

among specific age groups or demographics. Volunteers may share similar goals as the

program, but may also have their own priorities, such as building communication skills.

Similarly, audiences may have their own goals in attending.

Building off of the work we did to analyze the parameters within six framework categories,

48

we continue using organizational theory to study the interconnections between the categories.

The dimensions of any organization necessarily intersect and feedback into each other - in

this study, we seek to map key places of overlap for the specific context of informal physics

programs. As described in the Findings section, we find connections between all six main

categories present in our framework. Early on in our analysis, we observed the prominence

of the Personnel theme as well as the high level of connectedness between Personnel and the

other categories. This prompted further study into understanding the Personnel and their

connections to other organizational categories.

3.3 Methods

Building off of our prior work of adapting an organizational theory framework for informal

physics settings [73], we designed our study to identify how the six framework categories

affect each other. For our study design, we collected data from a wide variety of programs.

These programs differed in formats, personnel involved, and audience demographics. The

informal physics dimensions are broad and each program might have a different way of being

organized. Looking at multiple program variations can provide insight that can be useful

for not just one type of program (for example, summer camps), but be applicable across

different types of informal physics programs.

We interviewed lead facilitators to gain information about the program structures. Lead

facilitators are the people who make operational and design decisions for the program. As

a part of a larger research project, we have collected many forms of data about informal

physics programs, including surveys, interviews, and site visits [4, 100]; from this work,

we have found that interviewing lead facilitators provides insight into the rationale and

49

process for making these decisions. The decisions may involve the institution, partners, and

resources that other personnel members may not have knowledge about. For example, a

volunteer may not know how advertising works, how community partners are contacted,

or where funding is acquired. Lead facilitators work directly with volunteers, institutional

administrators, community partners, and event audiences. Formal documentation of these

practices is often limited, however, depending on the resources available and the support of

the host institution. Our prior studies have shown that knowledge of these practices is often

contained within the lead facilitators personal experience or documents, so speaking directly

with lead facilitators is the primary way to obtain information on these details.

As described in detail below, we designed a flexible interview protocol, tested and

validated it, and collected data from a national sample. Our final dataset included 31

interviews with lead program facilitators. Given the rich and detailed nature of these

interviews, we present a comparative case study analysis on a subset of these programs, which

allows us to describe the intricacies and nuances within individual programs. This approach

also helps us to compare similarities and differences between multiple types of programs in an

attempt to create a broader understanding of the structures and challenges that are present

across the cases [87, 89, 117]. Considering multiple programs helps to strengthen claims that

can be made across multiple forms of programming, while also guiding investigation into the

impacts of differing structures.

3.3.1 Interview Development

We developed a semistructured interview protocol that spans all six organizational categories.

The development process went through several iterations to best gather detailed information

about the program while also being aware of the time that the interviewee is volunteering to

50

be interviewed. The initial phase of interviews was with eight lead facilitators from different

programs with questions about the activities of their program, how the program began, how

the program evolved over time, and other details about formats and relationships within the

program. In this initial phase, lead facilitators were also asked for feedback on the interview

process and questions themselves.

Interviews with this first group were 1.5-3 hours in length, in part due to the interviewees

enjoying giving details about the history and events of their programs. We decided the

interview format needed to be more streamlined, as it would be difficult to recruit more

personnel to participate in long interviews; however, we still needed to collect logistical

information to provide context. Thus, after this initial testing phase, we revised our strategy

for interviews by (i) removing some questions from the interview and putting them into

a survey and (ii) revising some questions for conciseness and clarity. Program facilitators

took the survey prior to being scheduled for an interview. The survey took 15-20 min and

was a mix of multiple-choice questions with optional comment boxes and some open-ended

questions. Survey questions focused on logistical and numerical information, for example, the

number of volunteers involved in the program, the number of contact hours with audiences,

program location, and the physics content and activities. When a participant completed

the survey, they were contacted by a research member to schedule a follow-up interview.

Collecting survey responses in advance of the interview allowed the research team to prepare

follow-up questions that explored the particular program. Survey question preparation,

validation, and results were reported by Izadi et al. [4].

Second-round interviews with a condensed protocol were 45-60 min in length and followed

a semistructured format (see Appendix A). Interview questions asked about the history of

the program, the programs goals and objectives, the programs connection to its institution,

51

background information of the interviewee, the structure of the program, the personnel

involved in the program, the programs funding, the programs audience, activities and content

covered, assessment the program conducts, the future of the program, and challenges the

program faces. The interviewer would also ask follow-up questions based on the participants

responses. Questions were asked from each category, but based on the content of the response

and follow-up questions, not every single specific question from the protocol was asked of

each interview participant.

Since the protocol evolved iteratively with feedback from the interviewees, we coded

the full, condensed version to determine how much each framework category was focused

on. Even though not every single question was asked of each interview participant, the

percentage of questions that spanned each framework category was consistent across the

interviews and the condensed protocol. The percentage of questions that called out each

category is assessment 5%, audience 5%, institution 18%, personnel 27%, program 29%, and

resources 16%. The larger focus on personnel and program is not surprising because in this

study, we are interested in the operations and organization of these programs, and how all

of the pieces of the program fit together. Because the tasks required to run the program

are of course completed by humans, we necessarily asked lead facilitators to describe their

roles and the roles of their volunteers in doing that work. For example, one question asks,

“What assessment resources do you draw from?,” which is coded as both assessment and also

personnel, as it inquires about their involvement and rationale in carrying out assessments.

3.3.2 Data Collection

For data collection, we took multiple approaches to recruiting participants. We used a

convenience sampling strategy by contacting lead facilitators from informal physics programs

52

of which we had direct knowledge. We also gave talks on the broader landscape project

at conferences of the American Association of Physics Teachers (AAPT) and American

Physical Society (APS) and asked for participants. We also cold-emailed people who could

be identified as the lead facilitator of informal physics programs affiliated with physics-degree

granting institutions. A snowball sampling approach [46, 98] was used by asking participants

to name any additional informal physics programs they might be aware of in their institutions

or elsewhere.

In total, 31 interviews were conducted, spanning 7 states and 2 international programs.

Michigan was the most represented with 13 interviews—due to the residence of several

research team members, we had direct knowledge of higher education institutions and

national laboratories within the state. Most of the interviews (27 of the 31) were conducted

prior to the COVID-19 pandemic. The pandemic had many direct impacts on informal

physics programs, thus interviews conducted during the pandemic included the original

interview protocol as well as an additional COVID-focused interview protocol. Prior

interviewees were emailed to schedule a shortened, follow-up interview focused on the impacts

of COVID on their program. Analysis of the impacts of the COVID-19 pandemic on informal

physics programs has been discussed in other work and is not analyzed in this paper [118].

3.3.3 Case Study Analysis

From the dataset, five programs were selected and analyzed as described below. The five

programs were selected for their differing formats, content, activities, and program structures.

All five programs are based in Michigan. Of the five programs that were selected, three were

chosen as cases due to their detail-rich interviews.

Each interview was audio recorded and then transcribed through a web transcription

53

service [119]. Using the qualitative analysis software maxqda, we analyzed the interview

transcripts using the six main framework categories for coding. A single code unit was

defined as the full sentences in which a response met the definition of a code or subcode,

including a single sentence up to a couple of paragraphs, but not partial sentences.

Two researchers used the six framework categories as guideposts to explore emergent ideas

from the interviews [73]. They independently coded these emergent ideas, compared them

afterward, and discussed them with the research team. This process was done iteratively over

multiple interviews, modifying the codes and themes until there was agreement within the

team. As an additional validation measure, another researcher, who was not involved in the

development of the coding system, was trained to code the five interviews specifically with

those codes. They coded a subset of the interviews independently from the other researchers.

Afterward, the researchers discussed and compared their coding of the interviews and made

fine adjustments to the coding system. This was done iteratively across interviews until

there was consistent agreement among the researchers when coding the interviews.

Once there was consistency among the coders, we began analysis on the frequency and

overlaps of each code. An interviewees response often contained several codes simultaneously.

For example, a response may talk about the interactions that a volunteer has with an

audience member. That response would contain both personnel and audience codes. We

used the MAXQDA software to count and analyze when multiple codes were present in the

same statement.

3.3.4 Case Study Selection

Presenting a detailed analysis of a subset of our data allows us to describe the nuances of

our current framework and identify instances of overlap between the six main organization

54

theory categories. Specifically, this provides an opportunity to highlight the central role that

the personnel play in connecting all the organizational components together.

The three programs that we are analyzing here are among the most common informal

physics programs associated with physics departments and laboratories [1], but also provide

unique looks at the informal physics landscape, representing different format structures and

tackling different physics topics. The institutional support that each of these programs

receive varies. Each program targets different audiences, ranging from young children to

adults. These differences are beneficial for analysis because they highlight the important

connections between aspects of informal physics programs that may be common across

programs. Important to our findings, these three programs represent a variety of noticeable

challenges and successes regarding the program personnel. Analyzing these three programs

in more detail allows us to dissect the recurring personnel attributes that are present in all

three programs and consider the aspects that are only present in some of the programs.

3.3.4.1 Pub Physics

Pub Physics is a monthly public lecture series hosted at a local bar. Andrew, a faculty

member from a large, public university is the lead facilitator of Pub Physics. For each

event, two visiting guest presenters are invited to give a public talk about their research,

typically related to astronomy or astrophysics. The presentations are followed by a Q&A and

trivia session where audience members have the opportunity to ask questions and engage

with the speakers and content. While the program aims to reach an adult audience in

a downtown setting, away from the university, there is a small fraction of the audience

who are members from the facilitators department and university. Graduate student and

postdoctoral volunteers from the astronomy department assist Andrew with program logistics

55

and audience interactions.

3.3.4.2 Camp Physics

Camp Physics is a weeklong summer camp for high school students. Ben is the lead facilitator

of Camp Physics, which is held at a large research facility. During this program, the youth

participants engage in specialized physics-themed activities and presentations organized by

Ben and his team of faculty, postdoc, graduate student, and support staff volunteers. Youth

come to campus at the beginning of the week and stay overnight in the campus residence

halls until the end of the week.

3.3.4.3 Physics Club

Physics Club is an undergraduate student organization housed in the physics and astronomy

department at a public, research-focused university. Physics Club has many goals in

strengthening the physics and astronomy community within and beyond its department.

Its informal physics efforts are led by two elected undergraduate officers. The officers

recruit their undergraduate program members and department students to volunteer in

their events which include open house events, planetarium shows, and visits to schools.

These events involve demonstrations and astronomy-related activities that volunteers and

audiences engage with together.

3.3.5 Limitations

There are limitations to our dataset and analysis. For example, although the three

programs that we chose to describe here differ in format and content covered, they are

not representative of all forms of informal physics programs that exist.

In addition, the

56

data for each program are from the singular perspective of the lead facilitator. While

interviews provide a snapshot of the facilitators perceptions of programs, we do not have

direct knowledge of the experiences of the audience or other personnel members. For this

reason, when talking about the audience, we do so from the perspective of the personnel’s

interactions with them.

Considering the experiences and identities of the researchers is also important when

considering potential biases in the study design. Each researcher brings in their own

positionality which can and does influence multiple parts of the research process. The

interviews presented in this study were conducted by a White female immigrant. Other

authors who contributed to the analysis are a Latina woman, a White woman, and a White

man. For example, this can affect how comfortable participants may feel when speaking

with us in interviews. In addition, we as researchers also have our own biases which can

influence how we ask interview questions and how we analyze and interpret their responses.

As authors of this paper, we each have our own experience participating in or facilitating

informal physics programs. Our experiences in informal physics programs are beneficial

because we already have some understanding of how these spaces operate; for instance, our

past experiences also allow us to ask probing questions about specific aspects of informal

physics programming and structures. However, it could also influence our interpretation of

the data, such as overly focusing on certain aspects of running programs, we have personally

found a valuable or potential bias toward program leaders that we sympathize with around

challenges.

57

3.4 Findings

3.4.1 The Prominence of Personnel

In our initial analysis, lead facilitators did not discuss the six framework categories in equal

amounts. Figure 1 shows the percentage of how much each framework category was coded for

each program with respect to the programs total number of codes. The proportion of each

category that appears among the three programs follows similar patterns. The personnel

category was most discussed among the interviewed lead facilitators, though all six framework

categories are present in the data.

As expected, lead facilitators often discussed multiple categories in a single response.

Many responses included categories that were not explicitly asked about for their respective

question. For example, two program-related questions that we ask are, “Could you give us

more information on the physics content or concepts that your program aims to cover?,” and

“Has the content changed over time?” Lead facilitators gave responses that discussed how

the content of the program is selected, which often includes the academic background of the

personnel and the interests of the audience. In addition to discussing multiple categories

in responses, lead facilitators explanations varied in length for each question depending

on how many details they wanted to share. These factors explain the difference between

the percentage of codes in Fig. 3.1 and the percentage of codes that we asked about in

the interview protocol. While we asked questions about personnel and program the most,

lead facilitators talked about personnel more than we asked about it. Even in questions

explicitly asking about other categories, lead facilitators often talked about personnel in

their responses.

58

Figure 3.1: This chart represents the percentage of codes present for each framework category
across all three case study programs. For example, for all codes in the Camp Physics
interview, 3% of those were assessment, 17% were audience, etc. In comparison, for Physics
Club, 2% of its codes were assessment, 7% were audience, etc. We found personnel to be
the most prominent for all programs, accounting for roughly a third of all codes within each
interview. Camp Physics, Physics Club, and Pub Physics had 286, 149, and 344 total codes,
respectively.

While codes from all categories were found to overlap with each other, we found instances

of personnel overlap the most with the other categories. Figure 3.2 shows how much each

category overlapped with the other categories for each program. The width of the lines

connecting each circle is proportional to how frequently those categories overlapped; thus, a

thicker line represents more overlaps. For example, personnel-program overlaps accounted for

30% of all overlaps in the Pub Physics interview while audience-assessment only accounted

59

for 3%. This difference is represented in the figure by the personnel-program line being 10

times as thick as the audience-assessment line. Personnel is also one of the most mentioned in

conjunction with other categories alongside the audience and program categories. Overlaps

of categories with and between institution, resources, and assessment occur less frequently

in the data. A table with overlap frequencies can be found in Appendix B.

Figure 3.2: These charts show overlaps between framework categories in our interviews. Each
line connecting two framework categories means that there was at least one overlap between
those categories for that program. The thickness of those lines is scaled by the percentage
of overlaps coded in that particular interview between those two categories.

We find the personnel category to be foundational and central to the informal physics

programs in our dataset based on being the most coded and most overlapped framework

category. These initial quantitative results guided our qualitative analysis toward how people

spoke about personnel and the other categories. With a heightened focus on the personnel

of informal physics programs, our research questions shifted: Who are the people in the

program and what are their roles and tasks? How are roles and tasks distributed among the

personnel? How do the personnel interact with audiences, their institutions, and resources?

How do personnel engage with assessment? What are the challenges that personnel incur?

To answer these questions, we looked carefully at the comments surrounding personnel

60

and the associated overlapping codes. Looking across programs allowed us to identify key

themes, which we discuss in the following sections.

3.4.2 Personnel Roles and Tasks

We want to know more about the physics faculty and students who take on the

responsibility of organizing and facilitating these activities and programs. What are their

personal academic backgrounds, connections to physics, and prior experiences with public

engagement? What role do they currently have at their home institutions? Characterizing

the personnel is critical to understanding the training and prior experiences personnel bring

to their work in the informal programs, their interactions with the audience, and who

influences the content.

3.4.2.1 Who are the Personnel?

The lead facilitators of these three programs have varying academic ranks ranging from

undergraduate students to university faculty. The volunteers in the programs are also a

combination of undergraduate students, graduate students, postdocs, staff, and faculty;

however, each program has a different ratio of personnel from these groups. In addition,

the majority of these personnel members are dedicating their time to these programs on top

of their regular work and school responsibilities.

Pub Physics is led by Andrew, a university lecturer, who has a Ph.D. in astrophysics.

His main job is teaching multiple classes at the university each semester, and his role in Pub

Physics is not part of his regular duties for his lecturer position. He had been involved in

public engagement efforts when he was a graduate student and postdoctoral researcher and

wanted to continue with similar efforts when he started his current position. He did not find

61

other public engagement opportunities to be aimed at adults, so he built this program to

cater to that audience. Andrew recruits speakers who are typically faculty, postdocs, and

graduate students from the host university or from out-of-town institutions. The speakers

are not recurring volunteers each month. Recurring personnel members consist of about

three volunteers who are either graduate students or postdocs in the astronomy department.

They assist with program logistics and operations.

Physics Club’s outreach efforts are run by two yearly elected undergraduate co-chairs.

Carly is the undergraduate facilitator interviewed for this study. There are a couple of

faculty advisors who oversee the program and public engagement efforts, but it is the

public engagement co-chairs who handle most of the organizational work and planning.

Volunteers for the programs public engagement events are other undergraduates from the

institution. Throughout the academic year, the number of undergraduate volunteers for

events ranges from about 25-50 students. The volunteers are primarily involved in interacting

with audiences through physics activities and demonstrations.

Camp Physics is led by Ben, the public engagement program coordinator at a large

research laboratory. Ben differs from the other two program facilitators in that his paid

job is to organize informal physics events and activities. Ben earned his Ph.D. in physics

and currently organizes this program and works on other public engagement projects for the

laboratory; he does not have other teaching or research responsibilities. For the camp, there

are some business support staff from the department who help with program logistics. There

are also approximately 15 short-term volunteers who are all graduate students, postdocs, or

faculty members from the laboratory who assist with developing the content and activities.

There are several interesting outcomes to having programs be facilitated by personnel that

span academic ranks. Helpfully, activities can be informed by the perspectives and knowledge

62

of a considerable range. For example, Pub Physics has faculty speakers who give a talk about

their work on cutting-edge research topics, while graduate students who understand those

topics are speaking with audience members during the event and translating from their point

of view. However, personnel at different academic ranks vary in the amount of time that they

can dedicate to working in the program depending on other school and work commitments.

For example, Physics Clubs program leaders and volunteers are full-time students in classes,

some of whom have other jobs like working in research laboratories. For all personnel, except

dedicated staff hired only to do public engagement, these people have other commitments

as part of their professional physics work and connection to their institution, such as taking

classes, teaching classes, doing research, etc.

3.4.2.2 What are the roles and tasks, and how are they distributed?

We find that there are many different specific tasks that are necessary for programs to run.

The types of tasks that personnel members carry out span the other framework categories

that describe informal physics programs. The personnel are responsible for developing

content and other program-related activities. Part of the programmatic design that personnel

take into account is how to interact with the audience. The personnel members may

request support from the institution, sometimes requesting physical space, funding, or other

resources.

While programs differ in personnel and program formats, the types of roles and tasks

that the personnel carry out are often similar. In Table 3.2, we have grouped the specific

tasks according to common themes and documented the main tasks for each of our three

case study programs. These include both tasks that require physics content knowledge, like

designing hands-on activities, and more logistical and management tasks like making sure

63

the venue is available. Each program we looked at had several of each type. Tasks related

to physics content knowledge include giving presentations, designing physics activities, and

engaging with audiences with demonstrations and activities. How a personnel member carries

out these tasks may depend on factors like their academic rank. A program’s format and

structure can also dictate the tasks that a personnel member carries out. For example, the

Physics Club leader selects the demos that their undergraduate volunteers will do while the

Pub Physics leader allows their guest faculty speakers to give a presentation on a topic of

their choice. Here, the specific tasks that personnel carry out are different and the volunteer’s

level of agency varies based on the academic ranks of the personnel as well as the format

and structures of the program.

Logistical and management tasks may include scheduling events, advertising events, and

communicating with partners such as venues, sponsors, community groups, or research

specialists. These tasks tend to fall onto the lead facilitators; however, these tasks are

sometimes overlooked by administrators when recognizing the work of lead facilitators.

Coordinating with local venues for hosting events and designing promotional materials are

examples of time-consuming tasks that are necessary for holding events that audiences will

come to. As we will discuss later, tasks like communicating with partners make up a large

portion of the lead facilitator’s preparation time.

64

Table 3.2: Common program tasks and who in the program carries out these tasks.

Program tasks

Pub Physics

Camp Physics

Physics Club

Giving a

Guest speakers, typically faculty

Laboratory faculty presenting on

Undergraduate volunteers answer

presentation

at other institutions

their work

audience questions after

planetarium show

Designing

or

choosing

Grad student organizes trivia

Lead facilitator chooses demos

Grad students & postdocs design

experiments

activities

Engaging

audience

Lead facilitator,

grad student,

postdocs answer questions during

with demos and

activities

trivia

Lead

facilitator

sends

Recruiting

volunteers

emails through astro department;

Lead facilitator emails with guest

speakers

Grad

students

and

postdocs

Undergrad

volunteers

present

interact alongside experiments

with demos

Lead facilitator puts ad in weekly

Lead facilitator buys and set-

newsletter, and recruits via

up posters, sends messages in a

personal contact; Lead facilitator

group chat

talks with faculty member guest

speakers

Advertising

events

Lead facilitator designs flyers,

Lead facilitator shares info with

promotes on social media, sends

teachers and schools who pass

to audience email list

along info and promote the

Lead facilitator buys promotional

materials

Lead facilitator works with venue

program

Communicating

for space, organize drink tickets

with partners

for speakers, gift certificates for

raffles

Lead

facilitator

talks with

resident halls a year in advance;

Support staff organizes catering

Lead facilitator

communicates

with mega-organization for

organizing regional events

Maintaining

connection with

institution

Lead facilitator

communicates

with department newsletter

with

department

newsletter

Lead facilitator

communicates

organizers and direct interactions

with faculty advisor

organizers

with laboratory faculty

Lead facilitator

communicates

It is worth reiterating that these efforts are often in addition to the personnel’s other

responsibilities with their regular work and school commitments. For some programs, the

65

necessary time commitment for the personnel may be low, but there are more long-term

commitments for other programs, which can cause strain. As noted with Camp Physics,

volunteers may only need to dedicate part of one day in the summer to the program to carry

out mostly predetermined tasks. Contrast that with Pub Physics and Physics Club, which

host events year-round. Carly from Physics Club mentions that it can be difficult recruiting

volunteers during the summer or if there are conferences or department trips. Pub Physics

hosts an event every single month, so with the volunteers and guest speakers all working

in academia, their work commitments outside of the program will be heavily influenced by

the time of year that it is. Also as previously noted, many of the personnel are university

students and postdocs, so there is naturally a turnover rate every couple of years as personnel

graduate and move on to new positions.

We also find that these responsibilities tend to fall on one or a few people. As shown in

Table 3.2, many of the common tasks are taken on by the lead facilitator. Many of these

tasks, such as communicating with partners and advertising events can require a substantial

amount of time and resources. The overloading of responsibilities on a single person can

lead to challenges in carrying out certain tasks and hinder the program’s functionality and

sustainability. It is important for the program to consider how it is distributing its tasks

among its personnel, but also to take into consideration how the personnel are distributing

their own commitments within and outside of the program.

3.4.2.3 Recruitment and Retention

A program’s ability to recruit and retain volunteers is critical given that there are many

tasks and responsibilities that need to be completed. We find, however, that some programs

have challenges with recruiting and retaining volunteers, which leads to many tasks being

66

left to the lead facilitator. In some instances, the connections and level of support with the

institution contribute to the program’s ability to recruit volunteers like students and faculty

from the department.

For Pub Physics, Andrew primarily recruited personnel by asking faculty, postdocs, and

graduate students in the department. He was able to recruit a couple of volunteers to

assist with social media management, recording talks, coordinating with speakers, and other

logistical tasks; however, there were challenges in rotating tasks and responsibilities among

the small group of volunteers. As Andrew notes, “It would be nice if we comfortably had

upward of six people regularly involved or a larger group where the duties would rotate. It

would be nice if—for April I know that I have these two people that are in charge of finding

speakers and then for May it’s a different two people and there’s some sort of rotation. So

it’s not a monthly burden so maybe if I could get another 10 people to commit every other

month type of thing would be really nice. I’ve yet to find that level of support consistently.”

Without a full group of volunteers, most of the responsibilities have overwhelmingly fallen

onto Andrew, which can be difficult to complete on the side of his regular job. One of

the most difficult tasks to complete is recruiting guest speakers. He started by recruiting

members of the department to give talks, however, the department is relatively small. A

combination of limited responses to his requests and not wanting to repeat speakers led him

to quickly run out of speakers. Once the program became more established, Andrew was able

to recruit speakers from outside of his department, however, doing so was still challenging.

Andrew said, “I would say the thing that I spend most of my time doing is freaking out,

and not having a presenter, and trying to figure out what to do, and my event is a week

away, and I still don’t have anyone lined up. And so I don’t have—the advertisement says

to be announced, or whatever.” Without a speaker, there is no event. Even if the speaker

67

is recruited at the last minute, it hinders the ability to advertise the event or to organize

other activities like the trivia session that is related to the talk. Limited advertising can

limit the audience size, which can in turn impact the hosting venue that is selling food and

drinks. This recruitment challenge related to personnel has direct and indirect impacts on

the program, audience, and resources.

Physics Club reports similar challenges of volunteer shortages. Physics Club has other

types of activities than just public engagement, so to recruit volunteers for specific public

engagement events, Carly will send out emails to the program’s members through emails and

group messaging apps. Since the program’s members are primarily undergraduates from the

same department, there are common time conflicts among the volunteers. “We had an open

house, but the majority of our members were out of town on a department-sponsored trip. So

the people who are most active are gone. Everything went okay, but it was shaky for waiting

to hear—because I was also out of town—waiting to hear if everything went smoothly.”

The event ended up running okay, but as Carly notes, “knowing the people aspect of things

is something I don’t think we’ve quite handled yet.” This again signals the importance of

communication with partners as well as the institution. Something to consider is if program

facilitators and partners or institutions communicate with each other when planning their

events to make sure they are not losing personnel or audiences to events occurring at the

same time.

A notable difference between Physics Club and Pub Physics is that the Physics Club

lead facilitator role is elected every year, thus the leadership changes relatively quickly. Due

to this, there are some inconsistent program processes over the years, which can cause some

procedures to falter or have to be recreated with each leadership change. For example, one

of the challenges that Carly inherited was the volunteer sign-up. “There was very little

68

organized structure to knowing exactly before an event started, knowing exactly who was

going to be there.” A couple of days before an event, there would be a message sent out to a

volunteer list to recruit people. “What I would like to see our outreach cochairs change this

year is getting more of a signup sort of thing. I want to know exactly, going in, how many

people we have coming, and exactly where they’re going to be.” This allows Carly to plan

how to distribute tasks among the volunteers, how long event preparations may take, travel

logistics, and other arrangements that may need to be made.

Camp Physics did not report the same challenges with distributing tasks among its

personnel. Ben will recruit volunteers through the laboratory’s weekly newsletter while also

talking to faculty in the laboratory. The laboratory also has an outreach committee, so

once a year, Ben will contact the committee to recruit graduate student volunteers. “In the

end almost nobody has to do all that much. So it works out pretty well. I’m the only one

who’s there the whole time” says Ben. One reason for this is that Ben distributes smaller

and more discrete types of tasks to various groups of volunteers. For example, the support

staff are responsible for coordinating housing and catering for the youth participants. Other

volunteers are responsible for preparing the activity space by gathering name tags and other

supplies. Graduate students and postdocs are responsible for developing and facilitating

activities, which mostly entail carrying out or modifying activities from previous years. In

Camp Physics, each volunteers responsibility is a singular and well-defined task.

We can see from Camp Physics that task distribution can affect recruitment and also

that institutional support can play a role in this process. Since each personnel member has

their own dedicated task and the time commitment is relatively low, recruiting volunteers

has not been as challenging. While Ben advertises for volunteers in formal ways like the

laboratory newsletter, he comments that his success is, “...mostly, it’s personal contact. I

69

will talk to a faculty member myself, that kind of thing. We have an outreach committee

over there, and every year, I say, ‘You know what? We really could use some grad students

to help out, and we could probably use a postdoc.’” Here, there are fewer challenges with

the connections between personnel, resources, institution, and program.

3.4.3 Program Content and Activities

An important finding is that the personnel and their academic backgrounds directly impact

the type of content and activities that are part of the programs. Other important features

that can influence content and activities are the agency of volunteers, availability of space,

physical and financial resources,

interactions with audiences, and institutional support.

For example, Pub Physics personnel currently or have previously worked as researchers in

astronomy, astrophysics, or related fields. Similarly, Camp Physics volunteers are researchers

who utilize their laboratory space and resources to conduct related physics activities. Physics

Club utilizes a mixture of planetarium space and resources provided by the department as

well as undergraduate volunteer knowledge from their coursework and research experiences.

In some cases, volunteers’ academic backgrounds can be highly influential to the

programs’ goals because volunteers find it easier to go into the content more in-depth and

more passionately. Camp Physics focuses on physics topics related to the laboratory’s

primary research area. Program activities use the equipment that is available at the

laboratory. The recruited volunteers are faculty, postdocs, and graduate students who all

work at the laboratory, so they bring their expertise to their presentations and activities.

As Ben notes, “I kind of sketch out, ‘This is the array of things that are going to happen.’

But the individual experiments and talks and things, [the volunteers] pretty much have

free reign...they’re probably going to talk about their own experiments with their own

70

equipment.” Camp Physics leans into how the personnel bring their own expertise to the

activities and encourages them to relate the content to their own work.

However, sticking to a particular theme that the personnel may be comfortable with

can also challenge a program. As mentioned, Andrew from Pub Physics earned a Ph.D. in

astrophysics and therefore has the interest and expertise in that field. Pub Physics guest

speakers are free to choose the specific topic they present as long as it can loosely be tied

to astronomy. As Andrew notes, “the idea is basically the talks that we try to find have

some connection to astronomy research, even if thats a very tenuous and weak connection.”

Some connections may be made through computing, machine learning, and particle physics,

with some attempts or comments on how it can be useful for astronomy and astrophysics.

Other volunteers also bring their own physics content knowledge into the activity structure.

For instance, after each talk, there is a trivia session that one of the graduate students

or postdoctoral volunteers puts together based on the speaker’s presentation. Sometimes

volunteers bring other types of knowledge to program activities. For example, one of the Pub

Physics graduate student volunteers was interested in the live streaming of the event. While

she was a volunteer for the program, she used her technological knowledge and experiences

to record and stream events online.

Physics Club also has had to adapt to the varying content knowledge of its volunteers.

As an undergraduate organization, volunteers are sometimes majors outside of physics or

astronomy, so there is a larger variance in the depth of physics background among the

personnel. This can lead to varying degrees of autonomy among the personnel. For example,

Carly is the one who selects the activities from a database of available demos. Typically,

the volunteers are there to operate or assist with the demos. The volunteers have some

individual freedom in how they answer audience questions, but they do not have the same

71

level of creative or content input on the activities as the volunteers for the other two programs.

The institutional support also impacts Physics Club’s content as the department provides

the program with funding, a meeting workspace, and some physics demos, as well as access

to the planetarium where they can host planetarium shows and open house events. With

the institutional support and access to these resources, Physics Club is able to put together

astronomy-themed shows, events, and demonstrations.

Another factor in content and activity design is from feedback, communication, and

interactions with the audience. For example, Camp Physics used to run physics activities

with home-made detectors. Participants questioned why they were using a simplified version

of the same equipment that was available in a world-class laboratory. The program took

the audience feedback into consideration and switched over to using actual detectors and

laboratory space at the facility. For Pub Physics, Andrew says that roughly 80% of the topics

covered are astronomy or astrophysics related to astronomy because that is something that a

lot of the public is curious about. “It’s a great opportunity to get people in the door even if in

the end what they end up learning is more about computation or geology...If it’s astronomy

[laughter], it’s going to be pretty reliable [to attract audience members].” Similarly, Physics

Club has different themes to draw interest and excitement. Themes may include Star Wars,

planets, or black holes to encourage audience members to return again for a different event.

For these programs, personnel influences the program’s content and activities based on

their knowledge and academic background. In addition, the interactions that the personnel

have with the audience and the institution can also impact the types of activities and spaces

that the activities take place in.

72

3.4.4 Personnel Interactions with the Audience

For all

informal physics programs,

the way that personnel

interact with audiences

impacts the audience’s experience and engagement with physics content.

In our cases,

we find personnel and audience interactions in the three different programs happen as

presentations, demonstrations, Q&A, one-on-one interactions, social media, live streaming,

guided activities, and tours. These types of interactions are influenced by program structure

and design and impact different audiences in varying ways. For example, personnel members

may interact with children differently than adults. We are interested in how program

personnel connect with the audience based on program design and why lead facilitators

made those design decisions.

3.4.4.1 Personnel Recruit Audiences

Program leaders recruit and advertise their events to audiences through social media, flyers,

word of mouth, and contacting schools. Andrew promoted Pub Physics events via social

media, email lists, and word of mouth. Event turnout was large so he did not change that

approach for several months. As Andrew notes, “I would say, 90% of the crowd knows

us through [social media platform]. What I’ve been wanting to do is trying to figure out

better mechanisms for getting the people that aren’t on [that platform].” Andrew then added

another social media platform that he believed is more common in younger crowds, which

resulted in some new followers. However, if social media is the main form of advertising,

then only certain audiences are most likely to learn about the event. Andrew has considered

expanding to flier-based advertisements but is uncertain about their effectiveness. With

physical flyers, one needs to consider where they are posted. Andrew often does not promote

73

Pub Physics on campus. According to him, the university has plenty of opportunities to

learn science. While Andrew appreciates when a university faculty member does show up to

express support for the endeavor, he also wants to make sure that he is not catering primarily

to academics.

Ben has made connections with teachers in his local community through a teacher

program that he also runs. The teachers Ben has connected will spread information about

Camp Physics to other teachers and community members: “It used to be that [previous

program leaders] advertised solely by mailing posters to schools [around the state] and thats

how it got done before I got here. And then I was like, ‘Why are we mailing all the stuff?

It seems like a huge waste.’ So with these connections that we already have, basically,

[teachers we know] help get us people.” Camp Physics, however, is limited in the number of

students due to laboratory space. Camp Physics has an application process because there

are more interested students than available spaces. With an application process, one needs

to think about who they are accepting. Camp Physics aims to expose students who already

have an interest in science to some physics topics that they likely will not see in school. In

reviewing applications, Camp Physics tries to limit how much they are weighing the schools

and opportunities that are available to students. For example, looking at the number of

science classes or extracurriculars that students take may privilege students in certain schools

or districts. In recognition of this issue, the Camp Physics application considers activities

that the student pursues on their own, such as the types of science books they are reading

or science videos and shows that they are watching.

The personnel make decisions to attract certain audience demographics, such as kids,

adults, people unaffiliated with the institution, people of a particular socioeconomic status,

or gender. We find that some decisions that personnel make to attract their target audience

74

include the avenues in which they advertise, such as flyers versus social media or on campus

versus off campus. The personnel also takes into consideration how resources like location

and amount of physical space can impact the audience.

3.4.4.2 Opportunities for Audiences to Ask Personnel About Physics Content

We find multiple different opportunities for audience members to ask physics-related

questions and learn together with their peers. One opportunity for audiences to ask the

personnel questions is through a dedicated question and answer (Q&A) session. All three

programs have some form of a Q&A session built into their event, either during or after the

presentations.

Pub Physics dedicates some time after each speaker for the audience to ask questions.

In addition, after all the guest speakers have given their presentations, Andrew will open up

a broader Q&A session. It is not uncommon for physics and astronomy graduate students,

postdocs, and faculty to attend Pub Physics as audience members. Knowing this, Andrew

will say, “Can the physicists, astrophysicists, and astronomers come up to the front of the

room.” Those who are interested will go to the front of the room and answer astronomy-

related questions from the rest of the audience.

While audience members are able to ask physics-related questions to the personnel in

Q&A sessions, the formalism may make some audience members uncomfortable with sharing

their questions, much like a student asking an instructor a question in front of a large class.

To help mitigate the stress of asking questions, Camp Physics allows time after the faculty

member’s research presentation for the high school student audiences to form small groups

to develop questions. Each group then asks their questions together. The benefits of this

format, as Ben notes, is that the high school students get time to formulate their questions

75

and help each other understand concepts that they may not have figured out by themselves.

Also, “people who might not ordinarily raise their hand in class might get a chance to sort

of have their questions answered. So that’s kind of a signature aspect of our program.”

Pub Physics also creates opportunities for audience members to ask questions to the

personnel without having the entire audience watching. For example, a graduate student

volunteer will run trivia during the breaks between speaker presentations. During this time,

Andrew will walk around the room from table to table. “I come up, and they [audience

members] are talking about science. And, ‘Actually, we have a question for you. We can’t

figure out how thing X works that they’re talking about.’ I walk them through that and

explain it to them. So I get the opportunity [to] have informal science discussions in [that]

science education outreach [space] ... me-to-three people or me-to-six people.”

From these examples, we can see that the program’s activity design creates a connection

between the audience and personnel. For some audience members, the casual interaction

creates a space where they may feel more comfortable asking their physics questions to the

facilitator or their peers. The facilitator is also able to have more personable conversations

with the audience.

3.4.4.3 Finding Personal Connections Between Personnel and Audiences

In our cases, lead facilitators are aware of barriers to inclusion that can affect audience

participation,

including how audiences interact with personnel. Some factors that can

negatively affect audience-personnel interactions are perceived differences in background,

race, age, or academic degree. One way the programs try to mitigate barriers to these

interactions is by implementing structures that promote more personal connections between

the personnel and audiences.

76

When Andrew moves around the room after a speaker’s presentation at Pub Physics, he

is doing more than just answering physics questions from audience members. As he puts it,

“I try to ask people like, ‘Why are you here? Are you having a good time? What would

you like to see at future events?’ And make it feel like they’re not just a blind face in the

crowd that’s there for two hours. So I know a bunch of the regulars’ names at this point, the

ones that show up from time and time again. So I try to make it—I try to instill some level

of community.” With an audience size of about 100-200 people each month, and an email

list of about 180, there are many repeating audience members who Andrew and the other

personnel interact with each month.

Camp Physics runs for a week with the audience of high schoolers spending the nights in

the resident halls. With the physics activities running during the day, this allows for social

events for the youth in the evenings. The personnel are invited, but not required, to attend

the social events, and “a lot of them [the graduate and undergraduate students] just come

in the evening and hang out,” according to Ben. Also in the vein of being casual, Ben notes

that the evening activities are “an excuse to sit around with the students and our people

and just chat and ask anything you really want to at this point.” It is a low-stakes space

where personnel and audience members can casually interact with each other and discuss

anything, even if it is not physics related. Ben says, “[the high school students] being around

each other but also having all that time with faculty, staff and students here. That’s their

number one consistently rated aspect of the program.” Similar to Andrew at Pub physics,

Ben from Camp Physics makes claims about long-term connections and experiences that

their audiences take with them when they leave the program.

At the center of Physics Club is the idea of building an inclusive environment for anyone

interested and passionate about physics and astronomy. For example, Physics Club hosted

77

three separate events with the Girl Scouts, seeing 100-150 girls over three nights. Since this

was an audience of younger kids, Carly says, “I try to remember to bring stuff for them

because we have stickers and all sorts of posters, color pages, things like that here. So I

try to remember to bring that sort of stuff there so that theres something they’re taking

home. ...I know that being a woman in this department can be difficult, but if we can start

making that cultural change early or just fostering that value system, I think that’s really

important.” There is a gender inclusivity factor that Carly and Physics Club work to create

with their events.

For some sessions of Physics Club’s open house events, the audience may be “purely

undergraduates, and then I would say it’s pretty representative of what this university looks

like in the sense that it is predominately white...[with representation from] international

students a little bit as well.” Since Physics Club is composed of undergraduates as well,

there is an effort to connect with their peers through passion and excitement. Carly says, “If

it’s people that are really passionate about it, I think [the public engagement co-coordinator]

and I are, it comes across and we can encourage other people to do it.” Carly works to ‘tell

people that you don’t have to be an astronomy major to spend your Wednesday nights just

hanging out with us,” emphasizing that Club Physics events are spaces where casual and

scientific interactions can take place. Physics Club tries to consider the needs of its audience

to create an environment that fosters welcomeness and support on and around physics.

3.4.5 Resources and Institution Connections

The functionality of informal physics programs often requires some level of support from

their institutions and community partnerships. Partnerships and institutions may provide

resources such as financial support or physical items like supplies and space. The availability

78

of these resources plays a vital role in the program’s ability to run its events and activities.

For example, access to a venue can impact audience-related factors like number of attendees

and the audience demographics. Programmatic attributes such as the style of activities,

content, and planning can be influenced based on resources.

3.4.5.1 Partnerships Provide Resources

While programs are often viewed as single groups or entities, their event operations often

require collaboration and coordination with partners. Building partnerships is key for

programs because partners can provide necessary resources. Partners can include donors,

community organizations that the program works with, teachers or schools, and businesses

that host or sponsor events. For example, Pub Physics partners with local bars and venues

to host their monthly events. Pub Physics’ events depend on having these spaces available.

When one of the venues went out of business, Andrew had to find another venue that would

host them and could accommodate their audience of up to 200 people.

Building partnerships, however, takes time and effort. Personnel, often the program

leader, regularly work and communicate with partners to build relationships and trust. For

Pub Physics, Andrew is responsible for communicating with the venue about planning event

dates, providing drink tickets for the presenters, and raffle prizes for the audience. At

one point, one of the venues would help with marketing the event by creating promotional

images and advertising on social media. Transitioning to another venue, Andrew has taken

on more of the promotional roles, though the current venue does provide prizes and raffles

during the event. Andrew spends about a full work day each month dedicated to emailing

people, coordinating which month works for the speakers, coordinating with the venue, and

advertising the event.

79

For Camp Physics, Ben is responsible for coordinating with on-campus partners like the

residence and dining halls to house students for the week and to accommodate participants’

dietary needs for Camp Physics. As Ben notes, “I used to do almost everything myself. Now

we have some business support people that I can say, ‘We would like catering on this day

and this day. We’re going to eat in the residence hall the rest of the days. This is the kind of

thing we need. Here’s all the allergies we need to be aware of.’” This freed up Ben of some

necessary tasks such that he could focus on other responsibilities.

3.4.5.2 Choice of Program Location and Fees

An impact on audience attendance is program location and the cost of attendance. Pub

Physics events do not have an admission fee, making it more financially accessible for some

audience members. Since the events are hosted in a bar, Andrew encourages audience

members to purchase food and drinks to support the venue, but there is no purchase

requirement.

Transportation is a challenge the programs encounter. While program leaders take

into account transportation for audiences to attend the program’s events, there are some

limitations that occur. Pub Physics has been hosted at a couple of different bars over its

history. One location was in a downtown area, which led some people to attend because

they happened to be walking by at that moment. Other venue locations do not necessarily

have the same foot traffic and require different forms of transportation to attend. Camp

Physics does not fund transportation to attend the camp, and the camp is too far for some

students to walk to. Camp Physics is able to provide campus housing during the week

nearby the camp, but being able to arrive on the first day of the camp can be a challenge.

In some instances, some teachers have encouraged students to apply to the program, and if

80

they are accepted, there are some community resources that are provided to assist with the

transportation.

There are many factors that come into play when personnel determine the location and

cost of their program. Physical resources may influence the event location, but the location

can influence who is able to attend. Funding is a limitation for some programs, so personnel

have to make decisions that take into account the financial resources to keep the program

sustainable, while also considering the impacts that those decisions may have on the audience.

3.4.5.3 Institutional Connections and Challenges

The support and input from personnel and the institution can impact some of the events and

organizations that the lead facilitators need to plan for. Programs often serve as a public

face for departments, universities, and laboratories that they are housed in. While programs

may be representing their institution, the level of institutional support varies. Institutions

may provide funds, physical space for events and preparation, or volunteers, however, some

challenges may arise as well.

Camp Physics has reported strong support from the large research facility that it is housed

in the form of volunteers and funding. As discussed earlier, when Ben needs volunteers, he

will advertise through department newsletters and talk to faculty and students directly, with

many of them volunteering for multiple years. Among those repeat volunteers are some of

the facility’s directors. “My boss is involved almost every year. And the chair of our outreach

committee is involved almost every year...Our directors are pretty supportive of the outreach

program. They’re also pretty hands off. They’re like, ‘You do what you think needs to be

done.’ And people say yes when I ask them for things.” The institution also provides financial

resources. For one project, Camp Physics needed a sizable amount of funds. Ben went to

81

his director, who then committed some funds and wrote a letter to other departments asking

them to contribute as well. “And it completely worked. We had no trouble getting the

money that we needed. So again, as long as the laboratory exists and our directors are the

way they are, were not in trouble,” says Ben.

Some challenges that Physics Pub experiences are with the resources and support that

it receives from its institution. “Occasionally they’ll [department] advertise [Physics Pub].

Yeah, I didn’t—I never really got much in the way of actual support from the department

either advertisement or financially supported, or anything like that.” Andrew says that he

was once awarded a department outreach award, but that was the extent of acknowledgment

from the institution. “I don’t even know how many people in the physics department

knew I was doing it even though occasionally we would send out department-wide emails.”

Andrew asked a laboratory on campus to promote Pub Physics in their newsletter, but

they declined because it was not explicitly related to their laboratory. Andrew’s department

would occasionally mention Physics Pub’s event in the department’s weekly announcements,

but he had to constantly remind them to do so. This eventually led Andrew to say, “On

top of everything else I was juggling, it just wasn’t worth it.” He then stopped emphasizing

promotion within the department.

There are several instances where Physics Club and its home department are at odds

with each other, straining relationships. As described earlier, there were instances where

Physics Club events were scheduled during a departmental trip, which rendered several

undergraduate volunteers unavailable for the club’s event. In addition, Physics Club is a

due-paying member of a regional organization at the request of the department, which has

caused some tension with the program. As Carly states, Physics Club is not active in the

regional organization as it is more catered to nonstudent groups and has some internal

82

conflicts. However, part of the Physics Club’s budget has to be used to cover a membership

fee for the regional organization. “Personally, I disagree that we should have to donate to

that or build it into our budget.

I don’t think it’s something the department should be

sponsoring, but we do it.” It is worth noting the power dynamics that are at play in this

case. It may be more challenging for student-led programs to confront the faculty of their

institution compared to programs facilitated by other faculty.

The personnel and institution connections within informal physics programs can have

big impacts on programs’ functionality. Some programs are paying the salaries of the

lead facilitators, who are hired as professional physics educators for the institutions. Other

programs do not provide monetary support for the lead facilitator directly but may support

the program costs in other ways; thus the institution can rely on the interest and availability

of the lead facilitator to keep the program operating while maintaining an association with

the program that may be useful for cultural capital or grant agencies.

In some cases,

programs live on the fringe of institutional support, as seen with Pub Physics. Andrew

finds volunteers through his other work at the institution, though the institution itself

does not necessarily support the program.

It is also clear that the relationship to the

program impacts other framework components, such as resources. Impacts with resources

may include financial support, the types of partners the program works with, or supporting

and recognizing personnel members.

3.4.6 Assessment

In our findings, the assessment framework category was the least coded. We find that

within the interviews, the terms evaluation and assessment are being used interchangeably,

which is partially due to the researchers deliberately using both words to be inclusive of

83

the different ways that program leaders might think about these topics. We asked interview

and survey questions using both, for example, “Is there any evaluation associated with

your program, event, or activities?” and “What instruments and tools do you use for your

evaluation or assessment?” This approach lets the interviewees interpret and speak from

their understanding of evaluation and assessment. While evaluation and assessment are

related, there are some differences. Evaluation is focused on whether programs are meeting

their objectives; grants, similar to the ones that Ben earns for Camp Physics, require some

evaluative process about outcomes—for example, Ben tracks the number of students who

participate and then go on to be science, technology, engineering, and mathematics (STEM)

majors in college because that is a goal of his program. Assessment looks at impact as well

but may be used by facilitators to adapt their programs. For example, program leaders may

be interested in why certain people do or do not engage with their events or activities. At

the time of the interviews, assessments were interpreted by the interviewees and interviewers

as primarily formal. When answering questions about assessments, lead facilitators would

typically describe whether or not they facilitated something like pre and postsurveys or

written feedback forms.

A main finding is that all programs were not performing as much assessment as they

wanted partially because their personnel have limited time, resources, and/or experience

carrying out formal assessments like surveys and feedback forms. There were two key reasons

for this: (i) some aspects of the program are difficult to assess due to the voluntary, informal

nature of the interactions and (ii) program leaders do not necessarily have the knowledge and

resources to carry out intensive assessments. For example, Andrew would like to know more

about who the Pub Physics’ audience is and why they are attending but, “I just don’t know

what the right questions are, the right things to be asking...I would like to know some way

84

of figuring out why people return or why they don’t return. Are we alienating people?...I’m

thinking a lot of ideas. None of them are concrete because...I don’t feel like at the moment

I have any great resources to turn to for that.” While Andrew hosts Pub Physics away from

campus to avoid a crowd of predominantly academics, and the event is free, he lacks the

resources and time to assess his advertising and event practices in the way he would like.

While he does like casual conversations with the audience and observations, these informal

assessments do not provide as much information as he would like to inform his practices.

Camp Physics has some resources to carry out surveys and conversations with youth

participants, however, as Ben will admit, “I’m not really well equipped to do assessment.

I have a degree in physics. Educational assessment was never my thing.” In the past, Ben

partnered with an education researcher to study the impact of the program on students’

STEM pursuits after high school. Ben and the educational researcher analyzed past surveys

and designed new questions to ask youth alumni who participated in Camp Physics. The

results of the study were published, and Ben was able to present those to laboratory directors

and funders. However, the research partnership came to an end. Ben still uses the refined

survey questions and structures that were developed from the research partnership. The new

survey results that Ben collects are now just used for internal program improvement.

While formal assessment may not be commonly implemented in informal physics

programs, we noticed program leaders are still able to collect some amount of data

that informs their program decisions.

Examples include casual conversations with

audience members to better understand their experiences and needs, observations of

audience demographics, and noting audience engagement in activities. When it comes

to understanding the Pub Physics audience demographics, Andrew wonders, “What is

our demographic? I mean, I can look at the crowd and I know what our demographics

85

are. They’re not great. They’re not particularly diverse...But if we’re not getting diverse

individuals in the room, then there’s no way of assessing why they are or are not coming.”

Carly of the Physics Club will reflect on how she felt the program met some of its goals,

such as working with female and low-income families. “Low income is something that I don’t

think we really achieved this year. We did do some outreach events with students in [city],

but more could’ve been done with that.” Carly notes the location of the Physics Club may

impact the types of audiences they interact with since some of the locations that the Physics

Club could go already have other groups and institutions that do similar public engagement

events and activities.

Carly also uses formative feedback, particularly in how to effectively communicate

complex physics topics to younger audiences. Carly notes that this is a communication

challenge that many of her volunteers experience. “You know these complex physics concepts.

Explain them to an eight-year-old now because she wants to know what Hawking radiation

is or how black holes work from the conservation mass.” Reaching audiences can be difficult

in the first place, as well as actually engaging with them in a meaningful, supportive, and

informative way. Carly chooses topics that she thinks audiences may connect with and that

often requires adjusting how she and the volunteers explain that concept.

While program leaders encounter assessment challenges due to limited time, experience,

and resources, they still have an interest and desire to carry it out. Andrew from Pub Physics

says, “I’d like to develop an assessment to see if people are enjoying the events, learning things

from the events, understanding the messages we’re trying to send about the importance of

scientific literacy.” Andrew is interested in learning about already developed tools or at least

working with others to develop one. “I dont know much about the literature, and so are

there common assessments that are used for informal education? Or if not, maybe we could

86

develop one.” Ben also notes that collaborating with researchers in developing assessment

provides some professional benefits. “I had zero background in anything educational when

I got [this position]. So I had to pick it up...But a lot of what I’ve done is by finding

the expert...If you find somebody else who’s doing it [assessment]—because you can get a

publication out of it or—at the very least, it’s going to benefit them to some extent.”

3.5 Discussion

3.5.1 Personnel are a Central Part of Informal Physics Programs

Our findings show that personnel are instrumental

in developing program materials,

interacting with audiences, managing resources, connecting with the institution, and

conducting assessment. Figure 3.3 is a general visualization of how the framework categories

connect with each other. Personnel is the central category connecting to all others, and the

diagram provides an example of each interaction as informed by our data. In addition, the

double-sided arrows symbolize a two-way interaction between the categories. For the sake of

our visualization, resources and institution were combined as they are closely related to each

other. Assessment is not explicitly included in Fig. 3.3, however, we find that assessment

arises when personnel members (primarily program leaders) are evaluating or considering one

of these five categories. For example, assessment occurs when program leaders are observing

how personnel and audience interact or collect audience feedback.

Due to the connectedness of the framework categories, we find that if there are challenges

in one category, those can disrupt some of the other categories. This was discussed earlier

in the paper with Pub Physics where confirmation for volunteers (personnel) to give talks

87

occurred late, which made it difficult to have advertisements with full event information

(resources) to attract people (audience) to attend the event. Understanding the connections

between framework categories can be useful to identify areas to strengthen the program. In

the following sections, we will discuss in more detail how themes appeared in the data and

the implications that arise.

Figure 3.3: A visualization of how the framework categories connect with each other.
Connections with the personnel are accompanied by generalized examples extracted from
the interview data. As discussed, institution and resources were closely connected to each
other and not as prominent, so they were combined for this visualization. Assessment is not
explicitly included, however, we find that assessment arises when evaluating and observing
one or more of these five categories, and those observations are often conducted by the
personnel.

3.5.2 Issues of Access, Inclusion, and Equity

Thinking about how to improve access, inclusion, and equity of informal physics programs

in the context of personnel is critical because not only does it involve those who volunteer

88

in these spaces, the personnel are strongly connected to the program and audience. The

personnel and who makes up the personnel have the power to influence these issues in the

other categories.

The personnel and the programs they lead are a part of and inform the culture of physics.

The lack of more explicit recognition and support is already signaling issues of equity and

inclusion in physics as a field and the practices that physicists do. As we have shown,

many roles and tasks fall onto a few people. In addition, the majority of the personnel are

on the low end of the hierarchical chain in their institution. For example, undergraduate

students volunteer and sometimes run informal physics programs, but they have limited

time, resources, and power within their institutions. The personnel in these programs are

contributing to the physics community, so it is important to support the students and faculty

who are involved in these programs.

Addressing issues of access, inclusion, and equity with personnel can help address some

of those same issues with the audience. The decisions that program leaders make with

advertising, program design and location, fees, interacting with audiences, and other program

structures impact who attends the programs events and their experiences at the events.

Personnel need to consider how they connect with and attract audiences, including the

accessibility of the program to audiences.

Representation within personnel is important for considering aspects of inclusion with

both personnel and audiences. About 80% of those who earn a degree in physics are male, and

over 70% are white [30, 29]. The universities in which the programs we analyzed are housed

have demographics similar to the national averages. As program volunteers are recruited

from these departments, personnel demographics and backgrounds may not necessarily be the

same as the intended audience. There is sometimes difficulty in recruiting volunteers given

89

the many commitments students and faculty have, and there arises a tension in presenting

a more diverse face of physics without also overburdening members of marginalized groups

with participating in informal physics events alongside their other commitments. Making

sure that the personnel are supported in these programs and in their institutions can help

address representation, which in turn can impact the audience as that is who they directly

interact with.

Personal connections between the personnel and the audience are important as this

builds trust and interest between the two groups. This is especially true with getting and

maintaining an interest in physics among a younger audience. Carly specifically comments

on this based on her experience of being a woman in the department, and why she brings

stickers, color pages, and similar items to the Physics Club events. It creates a connection

with the younger kids and allows them to bring a part of the physics back with them after

the event.

Program leaders can also consider how they are creating spaces for interaction between

the personnel and the audience. For example, a stage in a physics lecture might automatically

create a separation between the physicist and the audience; additionally, if the stage is higher

up than the audience seats, this can reinforce a hierarchy of knowledge. However, if personnel

members are moving around the space and sitting with the audience, casual conversations and

social interactions among personnel and audience members can take place, which may create

a sense of community among the personnel and audiences as well as a more comfortable

physics learning environment for all participants to express themselves.

In these spaces,

audience members get exposure and personal connections with professionals and their work

in the physics field, which could impact the interest and academic and career pursuits of

younger audience groups.

90

Assessment is a tool that can help program leaders understand the experiences of all

participants and make decisions that affect those experiences. Assessments can help identify,

measure, and address challenges and successes that each program has in regard to its

audience and personnel. However, there is a need for resources to help program leaders

with assessments. Not all forms of assessment are the same and sometimes there needs

to be creative assessment methods due to the voluntary and sometimes short-term nature

of informal environments. For example, it can be difficult for a program leader to assess

why audiences are attending their program. A way to assess this issue can be from casual

conversations that personnel have with those in attendance, which can provide some form

of feedback on what the audience would like from the program. However, as noted earlier

by Andrew of Pub Physics, it is easier to get that feedback from the folks who are already

in attendance. It is difficult to assess those who are not present, which affects how much

programs can know about reasons people are not attending or stop attending events.

3.5.3 Considerations for Assessment

As noted by the lead facilitators of these programs, there is a need for assessment tools in

informal physics programs to evaluate and monitor many aspects of their work. Given the

importance of personnel members in these programs, checking in on their experiences and

satisfaction in the program can be crucial in retaining volunteers and helping develop their

physics identity and interest in participating in informal physics. There is also a need for an

assessment on connecting with audiences for many similar reasons as assessing the personnel:

attracting and retaining audiences, reviewing the intended audience versus the audience

that actually participates, adjusting practices as necessary, as well as building identity,

understanding, and interest in physics. We suggest that the broader physics education

91

community of researchers and practitioners work on this issue to produce more research on

assessment in informal physics programs.

Based on our study, we propose that lead facilitators might be able to self-assess their

programs across organizational categories. In our study, program leaders told us about what

they evaluated or wished they could evaluate in their programs; in Table 3.3, we rephrased

a selection of their responses into the forms of questions that other practitioners might ask.

For example, practitioners could consider what are the tasks that need to be completed

within the program, and how these tasks can be distributed among the volunteers such

that the tasks play to each person’s strengths and interests, while also not overburdening

any one person. This could include communication with the volunteers, asking them what

they perceive their roles to be, providing appropriate training and support mechanisms for

volunteers, and making sure that the team is in agreement on what needs to be done and

by whom.

The purpose of Table 3.3 is to inspire questions that practitioners could consider when

assessing aspects of their program; this is not meant to be an exhaustive list nor will

every example be relevant to all programs. Additionally, informal physics programs do

not implement assessments in the same way. A question that works for one program may

not work for others.

92

Table 3.3: Examples of questions that personnel members can consider when addressing
interconnecting aspects of their program. Each row corresponds to the double-sided arrows
in Fig. 3.3, representing the connections between the framework categories.

Connecting

framework

Example areas for assessment

categories

Do audience members feel comfortable asking questions or conversing with personnel?

Personnel-Audience

What is the audience feedback on the speakers?

Are volunteers equipped to address audiences questions?

Are tasks and responsibilities distributed among volunteers such that no one is overburdened?

Personnel-Program

Are volunteers adequately trained on building activities or communicating with audiences?

Do volunteers have agency in designing activities?

How much time are personnel able to dedicate to the program relative to other work

Personnel-Resources

and Institution

responsibilities?

How organized are activities and supplies for volunteers to locate?

Do the personnel report success and challenges to the institution?

Are the activities engaging for the audience?

Audience-Program

Are the physics topics presented at an appropriate level for the audience?

Audience-Resources

and Institution

Program-Resources

and Institution

Did the audience learn physics content or practices?

Is the physical space an appropriate size with enough seating?

Is the event cost or location a limitation for the audience?

Are youth audiences more likely to attend that institution for college?

How does content connect to the research of the department?

How much department funds are being spent? For example, Buying supplies, paying personnel

Is prep space and equipment provided and accessible at the institution?

93

3.5.3.1 Future work

There are still many understudied aspects of informal physics spaces. We suggest that

there could be room for collaborative partnerships between practitioners and researchers

to better understand the needs and functioning of informal physics spaces. There would

need to be clear communication between practitioners and researchers on what the goals of

these relationships would be, whether that is to better understand the landscape of informal

physics, understand certain aspects of the program or audiences, and/or build the skills

for practitioners to assess their own programs. We hope this work encourages partnerships

between researchers and practitioners and that our framework guides future studies.

Our overarching goal is to develop a model for the organization of informal physics

programs. This work helped us to understand facets related to the personnel and their

connections with program content and audiences. Throughout this paper, we found that

program leaders working with their institution administrators is important for financial

resources. Program leaders are in charge of program design and implementation. Volunteers

are also in charge of interacting with audiences. Going forward, we are working to identify

specific key components within each of the informal physics organizational theory categories

that practitioners can and should focus on to strengthen their programs. Identifying these

key components will allow us to craft more relevant and targeted assessment questions that

program leaders can use to reflect on and assess their own programs. The central role of

personnel in informal physics programs will be a significant influence on the development of

the upcoming key components model.

94

Chapter 4. An Empirical Framework for

the Key Components of Informal Physics

Programming

This chapter has been submitted and is currently under review as a journal article. The

author order on this article is Bryan Stanley, Dena Izadi, Jessi Randolph, Lily Boyd, Claudia

Fracchiolla, Kathleen Hinko.

4.1

Introduction

Informal education programs provide opportunities to teach and learn physics and astronomy

in creative ways outside of the constraints of formal classroom spaces. They can have a

positive impact on content learning, interest in physics and astronomy, and physics identity

development for both audiences [5, 6, 7, 8, 9] and facilitators of the programs themselves

[33, 32, 59, 60, 61, 62]. University students make up the majority of program volunteers

[5] and also benefit from building a sense of community and gaining professional skills like

communication and pedagogical practices [26, 27, 28, 32, 33, 32, 59, 60, 61, 62, 120].

The lead facilitators of these programs are often university students, staff, or faculty.

In prior work, we have found that most of the leaders of informal physics and astronomy

programs, whom we refer to as practitioners, want to be able to conduct assessment and

evaluation within their programs. However, program leaders often have limited time,

resources, and experience for such work [106]. The wealth of research from the physics

education research community over the past decades show the importance of understanding

95

the impacts of the learning environments that physicists create, not only to promote learning,

but also to prevent harm from being propagated to marginalized communities [121, 122]. As

the aim of a significant fraction of informal programs is tied to broadening participation

efforts,

it is even more important to engage in systematic and careful program design,

evaluation, and assessment [123, 124].

Partnerships between informal physics and astronomy program leaders and physics

education researchers have already created opportunities for assessment and evaluation

that can be used to better programs and their community impacts.

Initiatives like the

Joint Network for Informal Physics Education and Research (JNIPER) from the American

Physical Society (APS), with 160 members across 30 countries, demonstrates that there is

a movement to promote more collaboration between education researchers and practitioners

[125]. JNIPER hosts a variety of events to promote continuous learning and collaboration

among researchers and practitioners. Events include a series of workshops and monthly coffee

hours that cover topics such as evaluation and program sustainability. In these events, a

main theme has been the desire for tools that practitioners can use to assess their program’s

effectiveness [126]. However, there is a major challenge to developing assessment tools that

could be widely applicable because the logistical, management, and curricular aspects of

informal physics and astronomy programs have not yet been characterized systematically.

In contrast to the relatively standard operation of formal classroom environments, the

landscape of informal programs is diverse and complex [4, 106]. Some types of programs may

have similar organizational features, such as traveling physics demonstration presentations,

but this type of program could have significant differences from another type of program,

like a physics summer camp or an observing night. Therefore a necessary precursor to

developing instruments that could be used across informal physics and astronomy programs

96

is a categorization of the main organizational features of programs themselves.

The aim of this research study is to uncover and characterize the main aspects of informal

physics and astronomy programs that are relevant to their operation and success and to share

these results in a form that could be made useful to practitioners. To accomplish this, our

study design uses a qualitative approach to framework development. We interviewed lead

facilitators from a diverse array of informal physics and astronomy programs, and engaged

in iterative coding of fine-grained aspects of program operation. Then we coalesced these

fine-grained codes into key themes, and applied these themes to additional program data.

In this manuscript, we present the results of this study as an empirically-based framework

that identifies the key components of informal physics and astronomy programs. This

framework describes 12 components divided into 4 main categories that are a) important

to programs’ functionality and that b) program leaders have influence over. Our goal in this

presentation is to present the many pieces of the framework in a way that applies to the

multitude of informal programs while also being mindful of the quantity and grouping of the

key components to still be digestible and useful for practitioners and researchers. Identifying

these components sets a foundation for future development of assessment tools.

4.2 Developing a Framework

In the field of physics, frameworks, such as the Standard Model, serve as indispensable tools

for comprehending the intricacies of nature, enabling us to characterize and understand

fundamental phenomena based on empirical observations. In education research, frameworks

play a pivotal role in understanding and categorizing different forms of public engagement

initiatives within university settings. Frameworks provide a structured approach to assess

97

programs and enable nuanced analyses of objectives, methods, and outcomes. For example,

Doberneck, Glass, and Schweitzer work shows the lack of a common language for describing

publicly engaged scholarship makes it hard for faculty and administrators to understand each

other and for universities to demonstrate their social impact [37]. To address this issue they

developed a classification system based on how faculty members themselves describe their

work. This provides an understanding of what publicly engaged scholarship looks like in

practice and the importance of context in understanding and assessing engagement efforts.

Their framework facilitates nuanced analyses of objectives, methods, and outcomes, thereby

enabling researchers to evaluate the impact and effectiveness of these initiatives.

Furthermore, public engagement and informal STEM education programs have a crucial

role to play in fostering scientific literacy and interest in diverse communities. However,

achieving equity and inclusion requires more than just access. Recent research emphasizes

the importance of “meaningful engagement” that centers the voices and experiences of

historically marginalized groups [127, 128]. Frameworks can provide structures and guiding

principles for how to incorporate practices within informal learning programs to ensure

accessibility and equitable practices. For instance, the YESTEM program offers a notable

example by identifying four clear categories essential for cultivating equitable practices

and spaces within informal learning programs [129]. The Michigan Public Engagement

Framework describes the people, relationships, and contexts in public engagement spaces

more broadly, moving toward a shared language among researchers and practitioners in

different fields in order to identify crucial skills in public engagement work [130, 131].

Building upon this recognition of the need for robust frameworks, our research endeavors

have delved into the operational dynamics of informal physics and astronomy initiatives. In

our prior work, we leveraged Organizational Theory, particularly within the nonprofit sector,

98

to systematically categorize programs, analyze their organizational structures, and pinpoint

crucial components vital for their functionality [4, 72, 73, 106]. We developed a framework

consisting of six broad categories adapted from nonprofit organizational theory. This

framework serves as a foundational structure for understanding and categorizing informal

learning programs [4, 73, 106]. A main finding was that program personnel, including

program leaders and volunteers, play a central role in the functionality of informal physics

programs [106].

However, our intention with this work is to refine our analysis to a mid-grain level

—providing practical and actionable insights beneficial to program leaders, administrators,

and researchers. This refined approach aims to bridge the gap between overarching

organizational frameworks and context-specific details, offering valuable guidance to enhance

the effectiveness and sustainability of these programs. In this work, we draw inspiration from

an established framework utilized for assessing and enhancing physics teacher education

programs, The Physics Teacher Education Program Analysis (PTEPA) Rubric [132]. This

framework was designed as a self-assessment tool to empower physics teacher education

program teams to engage in reflective evaluation and improvement processes in collaboration

with key stakeholders. By leveraging this established framework, we aim to further refine

our analysis to ensure comprehensive assessment and continuous enhancement of informal

physics and astronomy initiatives.

4.3 Methods

For our approach to develop an empirical, practitioner-oriented framework, we decided to

interview program leaders about the structures and history of their programs. We found

99

from prior work that looking at online program materials or surveys did not capture the

complexities of the operation of these programs [46]. Interviewing program leaders provided

insights on the decisions, and the rationale of those decisions, on aspects of the program such

as the content covered, audiences the program works with, and how resources are allocated.

For this study, we limited our examination of informal physics programs to programs

where a team of physics faculty, staff, and/or students are engaging with audiences who

are not professionally involved in the field. Examples of programs include, but are not

limited to, summer camps, open house events, after-school programs, public lectures, and

planetarium shows. A key feature of these programs is that the audience is able to directly

interact with the program facilitators. For example, the physicists and audience members

can converse with each other, asking each other questions and bringing their own expertise

to the situation. Another example of engagement is working together through a physics

activity. The programs that we discuss in this paper have affiliations with a university or a

research laboratory. We did not include informal physics learning that might occur through

popular media, such as podcasts, books, websites, TV shows, or social media.

4.3.1 Data Collection

To recruit participants for this study, we took several approaches. First, we used a

convenience sampling method by contacting program leaders that we already had connections

with. We also recruited participants at American Association of Physics Teachers (AAPT)

and American Physical Society (APS) conferences. Finally, we cold-emailed program leaders

who were affiliated with degree-granting institutions. As reported in Stanley et al. [106], we

conducted 31 interviews, spanning 7 states and 2 international programs. Several research

members lived in Michigan, and thus had connections and knowledge of institutions and

100

research labs in the state. As a result, Michigan-based programs (13 total interviews) are the

most represented in our data set. A total of 27 out of the 31 interviews were conducted prior

to the COVID-19 pandemic. The pandemic had many direct impacts on programs. Prior

interviewees were contacted to participate in a follow-up interview to discuss the COVID-

impacts on their program. Analysis of the pandemic impacts on programs has already been

reported on [118] and is not analyzed in this paper.

Prior to being interviewed, program leaders took a 15-20 minute survey that consisted

of multiple choice questions with optional comment boxes and some open-ended questions.

In the survey, program leaders answered questions about logistical aspects of their program

such as audience sizes, number of volunteers, program format, etc. After completing the

survey, participants were contacted by a research member to schedule an interview. Further

information on the survey preparation and results are reported in Izadi et al. [4].

Interviews with program leaders typically lasted about 45-60 minutes and followed a semi-

structured format. Interview questions asked about aspects of the program such as format,

content, activities, institutional connections, funding, audiences, personnel, the program’s

use of assessment, and challenges the program experiences. The interviewer would ask follow

up questions based on the response. Further details on the preparation and validation of the

interview protocol are reported in Stanley et al. [106].

4.3.2 Data Set

We selected 15 programs from our dataset from which to build our framework. Our goal

was to select a diverse sample of programs so that themes identified would be applicable to

informal physics programs more broadly. The programs selected differ in format, physics

topics covered, frequency of events, audience demographics, and the type of host institution.

101

The research participants are a mixture of tenure and non-tenure/instructional faculty, paid

program coordinators, and university students. Each of these positions come with varying

degrees of power and relationship within the administration of their institution. In addition,

based on their job responsibilities, there are varying levels of support and expectations for

engaging in informal physics. For example, in our data set we spoke with paid staff who

are expected to entirely focus their program and also leaders of student organizations who

are voluntarily leading their program on top of their coursework and other commitments. A

table with details of each individual program is provided in Appendix C.

4.3.3 Key Components Analysis

A flowchart visualizing the steps of the analysis process is provided in Figure 4.1. Interviews

were audio-recorded and transcribed using a transcription service. Transcriptions of each

interview were qualitatively coded using the analysis software MAXQDA. Initially, five of

the interviews were coded for emergent themes within the six categories from the adapted

informal physics organizational theory framework. Further analysis of these five programs are

described in detail by Stanley et al. [106]. From that work, we found that personnel have a

central role in informal physics programs, as well as the importance of considering audiences,

resources, institution, programmatic aspects, and assessment. However, a limitation of that

study is the lack of actionable items for program leaders.

Two researchers examined the most prevalent emergent themes that arose from the

analysis of the initial five programs (Programs #1-5). We coalesced these themes into a set

of ten tentative components that we identified were a) crucial for the programs functionality,

b) applicable across different types of programs, and c) program facilitators have influence

over. Two different researchers then independently coded five new interviews (Programs

102

#6-10) for these tentative components. Each interview was coded one at a time and then

discussed by the whole research team, resulting in iterative modifications to the components.

Through this process, two additional key components were identified and defined. A new

research member, who was not involved in the original discussions of the components, was

trained to use the 12 components and coded an additional five interviews (Programs #11-

15) independent of the other researchers. After coding each interview, this research member

compared and discussed their coding with the rest of the research team, which resulted in

further refinement to the component names and definitions. One researcher then applied

this set of key components codes to Programs #1-5, so that all 15 programs had been coded.

In the next stage of analysis, we had three research team members, including a new

research team member who was not involved in the development of the framework, look

within each key component for themes. For each component, coded sections from all

interviews were compiled, and then major themes within that key component were identified

and discussed among the team. Descriptions of each component were constructed using data

that spans all programs.

103

Figure 4.1: A visualization of the key components analysis process.

4.4 Key Components of Informal Physics Programs

As shown in Figure 4.2, we have identified 12 Key Components of informal physics programs.

In the following sections, we describe the key themes and examples for each component.

Stemming from our initial adaptation of organizational theory for informal education

programs, we have grouped these 12 components into four groups that are thematically

related. For example, one group contains the three components that are centered around

program personnel. These groupings are identified in Figure 4.2. We will present each

grouping as its own section in this paper.

We recognize that this framework has many parts and, furthermore, that these parts are

interconnected. For example, one of the components is goals. Programs have many types

of goals that they aim to achieve, and some of these goals may be closely related to other

key components, such as wanting to support audience engagement within their events or

wanting to provide professional development opportunities for their student volunteers. In

104

this paper, we include details on all components and their interconnectedness to provide a

comprehensive framework that is useful for practitioners.

Figure 4.2: This is a visual representation of the 12 key components of informal physics
programs. The components are grouped by similar themes and designated by color/shape
icons (Program Goals, Design and Assessment in brown with a square icon; Personnel in
green with a diamond icon; Institutional Connection and Resources in yellow with a triangle
icon; Audience and Partners in purple with a circle icon). They are organized and connected
in a circle to show the interconnected nature of the components.

105

4.5 Program Goals, Design, and Assessment

While there are similarities across different programs, we find that each program has its

own set of goals that it is trying to achieve. Sometimes these goals guide the design of the

program; sometimes the design of the program guides its goals. For example, one program

may prioritize connecting with certain types of audiences such as kids. Another program

may prioritize getting university students to volunteer in community engagement work. Some

programs may put effort into creating activities that focus on a particular physics topic. We

also find that programs use varying forms of assessment to help programs determine if they

are meeting their goals and influence how they design their programs. We describe these

ideas with the following components: Goals, Program Content and Design, and Strategies

for Assessment.

4.5.1 Goals

Goals are what the program is working to achieve. Programs have many goals that are often

centered around audiences, personnel, and content. We find that programs may prioritize

their goals differently based on factors such as available resources, institutional connections,

and community partnerships.

Goals play an important role in influencing the program content and design of a program.

There are a variety of goals that may serve as the purpose and motivations of the program,

establishing a “big picture”. Practitioners leverage thoughtful design and adjustments to

the other 11 components to accomplish these goals. While there is a plethora of possibilities

for program goals, many programs are motivated by similar core ideas, such as curiosity,

engagement, professional development, and inclusivity. Some of the commonly reported

106

goals are described below.

4.5.1.1 Programs Seek Audience Curiosity, Exploration, and Engagement

Through Their Physics Activities

Many programs have goals centered around wanting to create a safe and inviting environment

for scientific exploration, curiosity, and discovery. Programs may offer opportunities to learn

experimentally or actively engage in activities or discussions that can help forge positive

experiences with and in science. They may also seek to share the importance of science

with their audience. For example, Program 7 hosts weekly physics classes on the weekends

for elementary and middle school students. To help address issues of imposter syndrome,

Program 7 utilizes team-based learning. As the program’s leader states, “By being able

to talk about it in a group, it solidifies their knowledge. They feel like they’re part of a

group, they feel like they contributed, and it grows their confidence. And their ability to

talk amongst their peers instead of just having the one-on-one dialog with the professor is a

big deal.”

Some programs want to offer opportunities for the general public to learn, experience,

and engage with science that they may not otherwise encounter in their daily lives. Put

broadly by Program 14, “That is what I see is the goal of what we do at the planetarium, is

to teach people about the universe.” Programs may also seek to incorporate and integrate

science into modern culture. Program 10, a summer camp, shares, “We’ve lost science as

part of our culture I think. It’s been turned into this esoteric study that isn’t part of our

culture anymore, and so trying to bring it back into this is part of what makes us American

or part of what makes us people is that we can make art, we can make music, we can do

science, we can learn, but it’s trying to think of it as more a part of their life than something

107

in school or a possible job. So trying to bring it out of that academic sphere and more into

their life.”

We also find that programs connect science to art, history, religion, music, or other

disciplines. This can offer audiences more entry points to engage with science and can

encourage the exploration of the intersections of science and other fields. For example,

Program 9 describes itself as “an eclectic mix of science, art and music. [There is] always a

mix of education and entertainment that’s pretty, pretty strong in our history and continues

through today. We’re definitely a very experimental facility. We have the freedom to explore

various different types of projects and partnerships . . . looking at those intersections of art

and science and, and other types of fields as well.”

4.5.1.2 Programs Aim to Provide Professional and Scientific Development for

Their Student Volunteers

In addition to supporting audiences, most programs in our data set seek to support

undergraduate and graduate students by providing learning opportunities that teach students

how to be professionals in the scientific field and effectively communicate science. As stated

by Program 9, a planetarium, “We host faculty talks frequently, we interact with graduate

students, undergraduates, [who] all make a core part of our team. And so we really see

ourselves kind of as a learning facility, a training facility, for our students in terms of how to

work at an institution.”

Program 7, which hosts weekend physics classes for elementary and middle school

students, has some of

its classes run by more senior undergraduate students. The

undergraduates are usually pre-service teachers or end up having careers in teaching. To

support the program’s goals of supporting student volunteers professionally, the program

108

leader will then work with the college students in developing lesson plans and course goals.

4.5.1.3 Programs Work to Provide Inclusive and Communal Learning Spaces

We find that most programs center their goals around the curation of inclusive and equitable

spaces in science. They may want to share physics with historically marginalized groups in

science or provide various forms of support for students in physics. Programs may prioritize

connecting to the surrounding community and supporting community development.

Programs may also combine a number of motivations and goals to formulate the broader

vision for their program. For example, with their planetarium programming, Program 14

considers ways to give audiences opportunities for exploration, which includes “giving people

the space to ask questions or to get their questions asked . . . because I think people are

naturally very curious, but you need to create an inviting space and safe space for people to

ask those questions in the first place.” Program 14’s leader also notes that just answering

people’s questions is not enough. “I think some level of exploration [is necessary] and I also

think having some ties to the real world . . . I think understanding why we should even care

in the first place is also a good key.” Here, Program 14’s goals are creating spaces where

people can explore for themselves and get their physics and astronomy questions answered.

In order to support that goal, the program considers how to do so in a way that is safe and

supportive of exploration, and ultimately finds relevance for the audience.

The graduate student organization Program 11 describes the goals of their public

engagement work as being two-fold. “I think, one, it’s to encourage grad students to give

back to their community because we don’t do that enough. And I think, two, it’s to share

something that we think is beautiful with younger students.” Here, Program 11 works to

improve overall involvement of community-based work, something that they note is lacking

109

among graduate students. The second goal is more focused on the impacts of that community

work, such as sharing physics with kids.

4.5.2 Program Content and Design

Program Content and Design is about the topics that a program covers as well as the

programmatic structures of their activities and events. Topics can range from broad to

specialized areas of physics and astronomy. With program types ranging from public lectures

to summer camps to science festivals, programs can engage with audiences with these topics

through varying levels of interactiveness.

4.5.2.1 The Topics Covered Depend on Personnel, Audiences, Partners, and

More

In prior work, we have found that the landscape of informal programs covers a multitude of

topics including general physics, particle physics, biophysics, astronomy, astrophysics, and

earth science [4]. As we discuss in the later components, how programs choose these topics

can depend on a combination of personnel knowledge and experience, the audiences’ interests,

funding, and the goals of community partners and/or the institution. For example, Program

9 does planetarium shows on a variety of topics, but repeats some topics like black holes

“because it always has high attendance.” However, if they get “funding from, for example,

[a specific funder] to do a show, then there might be constraints on what the show needs

to be about.” Program 8, which has guest speakers talk about their research, selects people

who they think will give a good talk, but ultimately the speaker who is the one deciding

what the presentation will be about.

110

4.5.2.2 Programs Structure Activities Around Physics Content Based on Their

Audiences and Resources

One facet that makes informal physics programs unique is that they exist in many different

formats with varying designs of activities and interactive opportunities. For example, if a

program is working primarily with an audience of children, the program may design their

curriculum to involve hands-on activities or demonstrations. For Program 15, “our design

is that it’s, it’s active, and that’s engagingwhatever the kid is doing is, is like tactile, right,

they can work with the thing, they can manipulate the thing . . . We like to have things where

they can take something home, that’s interesting. So like, making a rocket made out of a

balloon. So they have like, these rocket races that they have with balloons.”

Other programs design their activities around the space and resources that they have

at their disposal. For example, planetariums like Program 14 already have defined physical

spaces like a dome, but they can think about different ways to engage with audiences in these

spaces. “We’ve worked with people in art history and design on getting content in the dome

so people can be immersed in there . . . we have this really amazing, awesome technology that

allows us to teach about everything in the universe . . . so how can we share that technology

with other people? . . . We know we can hook people in different ways.

It might be the

history. It might be the art. It might be the science. But ultimately, our goal is science

education but it doesn’t necessarily have to be the entry point.”

4.5.2.3 Program Design is Influenced by Mobility and Accessibility Constraints

Programs also have to consider their audiences’ mobility and access to these programs.

Program 13 is able to travel to schools, so they have to consider activities and materials that

111

can be packed and moved around while also being appropriate and engaging for the varying

classroom and school spaces they are visiting. Other programs like museums, planetariums,

and summer camps are more restricted to their physical location. In which case, they do

not necessarily have to consider the transportability of their activities, but they do have to

consider more about how their audiences are able to visit them. Some audiences may not

have access to transportation or funds to travel to these locations, so programs have to take

this into consideration of their design.

There are many other logistical factors that programs take into consideration, including

the frequency and duration of their events, purchasing of materials, registration, and fees.

As mentioned above, programs take into account their audiences’ ages, needs, and the size

of the audience when designing their program; however, as we will discuss in the other key

components, the personnel have a large influence on the program design, as do partners, the

institution, and access to physical and financial resources.

4.5.3 Strategies for Assessment

Strategies for Assessment is about how programs gather information about their operation.

During the interview process, the researchers used the words assessment and evaluation

interchangeably to consider the different ways that program leaders think about these ideas

when gathering information about their programs. However, there are some key differences

between assessment and evaluation. Assessment is often used by program leaders to consider

how to evolve their program. For example, assessment may be used to understand people’s

interest in particular physics topics and thus inform how the program approaches future

events. Evaluation instead looks at if the program is meeting certain measurable outcomes,

such as determining how many youth participants go on to have a STEM-based career.

112

Research participants used assessment and evaluation interchangeably, so for the purpose of

this paper, we will refer to both terms as assessment. Programs may use formal assessments

like interviews and surveys with audiences to gather feedback. We find that some programs

also use informal methods like observations or casual interactions with others throughout an

event. Programs may collect this information to determine if they are meeting their goals,

to provide data to their institution or for funding purposes, and/or to make adjustments to

their program design.

4.5.3.1 Formal and Informal Methods of Assessment are Useful to Programs

We find that programs use feedback from the audience or personnel as a form of assessment.

This feedback can be gathered formally through surveys, scheduled meetings, or other

methods. For instance, Program 15 collected feedback by “having a board where the

board had the tables labeled, and we would ask [audience members] to put a sticker on

the board for the thing that they found most exciting or most enjoyable.” Feedback can also

be received informally through comments offered by the audience or personnel in person,

written online reviews, or views on online videos. Most programs used a combination of

formal and informal audience feedback, with surveys and in-person conversations being the

most frequently reported.

Another assessment method utilized by most programs was informal observations and

reflections, in which personnel take note of what seems to be working or not working, often

through observations of and interactions with the audience, and ask themselves questions

about how it can improve. For example, when Program 13 is hosting their demonstration

shows for kids, the program leader will watch the kids in the audience to see their reactions.

“It could be at the part of the show that’s really exciting and sometimes it’s the most

113

mundane thing. [laughter] Or at least from your perspective, it seems like the most mundane.

But there’s that one moment where you just happen to see that spark [in a student].”

These observations can inform the demonstrations and activities that they facilitate for

their upcoming shows. Other assessment methods include collecting data, keeping records,

conducting research, and receiving external recognition, such as awards. Attendance may

also be used as a passive form of assessment.

4.5.3.2 Programs Use Assessment to Evaluate Their Audience Related Goals

Programs assess a wide range of topics depending on the goals of the program or purpose of

the assessment. However, most programs reported assessing audience interest, excitement,

and enjoyment, as well as the diversity of their program, personnel, and audience. Many

programs also assessed logistics, such as the utilization of space and resources and scheduling

or timing of events, and the sustainability of their program. Programs may assess whether

or not audience and community needs or goals of the program are being met. For example,

Program 8 assesses if they are meeting their audiences needs and goals through the use of

surveys at the end of their events. Other types of assessment ask about reasons that audience

members attend and participate in events, what they value about the program, how they

heard about the program, and how easy it was for them to access the activities.

4.5.3.3 Programs Use Assessment to Inform How Their Program Should Evolve

Programs can operationalize the knowledge gained from assessments to make adjustments

that allow the program to be more successful. For several programs, assessment responses

were considered when hiring and supporting personnel, or when designing or planning future

events. Programs also adjusted their design or materials to be more accessible and inclusive,

114

or try to choose topics that would be interesting and engaging to the audience. Program 10

stated “I try to come up with really good activities. I think that’s kind of the crux. And I

try to make sure that whoever’s leading the activity is excited about it . . . So I just try to

choose things that you’re going to like, that’s I guess a lot of it.”

Programs also make adjustments to better align with their goals or to better fit the needs

of the audience and community. For instance, Program 12 is a science museum. When the

museum was originally being built, it was meant to be a children’s museum that was not

science-focused. The museum leaders had done a survey of teachers in the area. The survey

results found that while there were life-science museums in the area, there was not a museum

that focused on physical science. Based on requests from local teachers, the museum pivoted

to focus on physics content.

4.5.3.4 Assessment has Challenges but Overall is Beneficial to Programs

Some programs reported challenges finding the time, energy, or support to develop

assessments. Programs that were able to create formal assessments sometimes struggled to

get survey responses or engagement. It can also be difficult to create an assessment that is

accessible to all audience members, and program leaders do not necessarily have educational

research experience. Program 10 encountered this challenge when trying to develop a survey

for children: “I put a bunch of words on a page and said, ‘Circle the words that sound like

science, and put triangles around the words that sound like art’ And they can have both. But

apparently, when I said, ‘Sounds like,’ some of the kids took that literally and were trying

to rhyme words with art. And so I had to reword that because I have to use small words

because I didn’t want to say, ‘What words would represent science,’ so I think I changed it

to, ‘What words would you use to describe science or art?’ But yeah, and then I had five

115

years of data that I was really excited about until I learned that the youngest kids were not

understanding the directions.”

Despite the challenges to creating and utilizing assessments, programs described benefits

of assessment. For example, programs can use positive feedback and reviews to justify the

need for the program to their department or sources of funding, and asking for opinions can

help the audience feel heard and valued. The majority of programs describe how they use

informal forms of assessment like observations and conversations with audience members,

however, more formal forms of assessment, such as surveys and interviews, are less common

due to constraints such as lack of time, resources, or experience.

4.6 Personnel

Personnel are the students, staff, faculty and others that support the program’s operation

from the host institution. These individuals are central to informal physics programs as

they interact with audiences, they design and implement program events and activities, and

they work with community partners. We have looked deeply at the relationships between

Personnel and the other aspects of informal physics programs in related work; these are

described in detail in Stanley, et al. [106]. Based on the number of tasks and responsibilities

needed for these programs to operate, programs typically require a team of personnel. We

find that there are many considerations that program leaders have to make in order to

recruit and retain personnel in their programs. These ideas are described with the following

components: Recruitment and Onboarding of Personnel, Distribution of Roles and Tasks,

and Continued Support of Personnel.

116

4.6.1 Recruitment and Onboarding of Personnel

Recruitment and Onboarding of Personnel considers how program leaders can recruit

volunteers to participate in the program and events. This includes thinking about the types

of volunteers the program may want to recruit based on knowledge, skills, etc as well as

advertisement practices. Program leaders also have to design training experiences for their

personnel in order to carry out their tasks appropriately.

4.6.1.1 Program and Activity Design Influences the Type of Volunteer a

Program Recruits

In many cases, there is a program leader, often a single person, who is in charge of many

aspects of program logistics and management and also activity content and design. There

often can be strain in the program with the majority of tasks being assigned to a single person

(see additional examples in the Distribution of Roles and Tasks section). To overcome this,

programs will recruit personnel members to help carry out some of the many tasks that need

to be done to run the program. As we have found in past work, these volunteers are mostly

university students, but also include faculty and staff [4]. Students, staff, and faculty each

bring their own strengths and challenges, so programs make recruiting decisions on what

best fits their needs.

If the program is designed around research presentations and laboratory tours, then

a program will often need to recruit speakers. Programs may recruit faculty, post-docs,

or sometimes graduate students to present given their experience and expertise. Staff and

faculty may be more equipped to communicate with partners, requesting department funding,

etc. The schedules of student volunteers may be more flexible compared to faculty and staff

117

so that they can put more time into the program.

4.6.1.2 Programs use Multiple Methods to Recruit Volunteers

Graduate, undergraduate, and high school students make up a bulk of program volunteers.

In some instances, students will inquire about working or volunteering in the program. This

involves the student knowing about the program in the first place, which is usually by word of

mouth or having previously experienced the program (for example, the program visiting their

school when they were younger). In most instances, programs will still need to implement

some form of recruitment method. A common approach is to email students, whether

that is department wide or directed to a specific student organization. Program 9 will ask

professors teaching introductory astronomy classes to advertise their planetarium program

to recruit undergraduate volunteers. Programs may also create flyers. Programs attend

or host on-campus events to recruit student volunteers. Recruiting additional volunteers,

especially faculty and speakers, may include sending department emails and newsletters,

asking individuals directly, or contacting those at other institutions.

4.6.1.3 Programs Provide Varying Levels of Training

As a part of the onboarding process, programs often provide training to volunteers on aspects

like how to use equipment, explanation of content materials, event structures, pedagogical

practices, communicating with audiences of varying ages, and more. Beyond the initial

onboarding process, some programs have continuing training and development sessions for

their volunteers. However, organizing these additional training sessions can be difficult due

to scheduling conflicts. For example, Program 12, which has about 20 student volunteers,

had planned hosting pedagogical workshops like, “[If] you plan to volunteer, we really need

118

you to go to these, you know, we’ll, you know, two one hour sessions on informal science

pedagogy.” However, that did not work out because of “timing and and, you know, scheduling

all of these different people and so forth. But, it was something we had wanted to do.”

Programs vary in how they structure their training processes. Some programs have

meetings before events. Program 8 has presenters give a practice talk before their

presentation. Programs like 9 and 6 hold multi-day training sessions, oftentimes at the

beginning of the academic term.

In some instances,

it is the program leadership who facilitates the formal training

process.

In other instances, there may be more of a mentoring relationship between the

more experienced and less experienced volunteers. This can alleviate some of the labor

burden on the program leadership.

4.6.1.4 Student Volunteers have High Turnover Rates and Busy Schedules that

Affects Programming

An issue that programs’ face is that there is often a high turnover rate of student volunteers

because students graduate or become busy with other jobs or commitments. Therefore,

student volunteers typically are not involved with the program for more than a couple years,

in which case, programs may have to continually recruit and train new volunteers. Since

students are also busy with other commitments and interests, many programs think about

how to balance convincing students that it is worth dedicating time to the program while

also accommodating students’ busy lifestyles.

Program 11, which recruits primarily graduate student volunteers, works to convince

students in the department that volunteering can be another fun alternative to participate

in. “I’ve done a lot of thinking about, like, ‘Okay. What am I asking from these grad

119

students?’ Because most people are like, ‘I’m too busy.’ . . . We have a lot of events that are

on Friday nights, which is unfortunate because everyone’s like, ‘What? Why do I want to

do that on a Friday night? You’re crazy. I would never volunteer for that.’ And so it helps

to be able to explain a little bit further about why it would be fun to volunteer on a Friday

night.”

Students also have busy schedules with courses, jobs, and other activities that are rigid,

which can impact students’ ability to participate in program activities that take place during

the week. Program 6, a demonstration show, has to work around this when working with

other partners such as schools. “The only way we’re able to do [our program] is if we can get

a group of at least three [volunteers] that all have the same gap in their schedule and that it

works with an elementary school. And so if the [volunteers] are available, then we can send

them out to schools. But if we have eight [volunteers] that are willing to participate, but

they don’t have any overlaps in their schedule, well, we can’t send out any groups.”

4.6.2 Distribution of Roles and Tasks

Distribution of Roles and Tasks focuses on how responsibilities are split among the personnel.

We find that responsibilities often fall on a single person, which can strain the program. Some

of this strain can be reduced by considering effective and equitable ways to distribute tasks

among all of the personnel.

4.6.2.1 Task Distribution is Dependent on the Type and Number of Personnel

a Program has

Some programs host a single large event in a year while other programs have more frequent,

smaller events. For programs that host single events, the program may recruit a few

120

volunteers that work the entire event, or they may have many volunteers for smaller portions

of the event. For Program 15, some aspects of their science festival program could run with

four to five volunteers who worked all day, or they could “break it up, and you end up with

like, eight or ten people for a table because you all do half day shifts.” Working in shifts

alleviated the time commitment and workload for each volunteer, however, that requires

recruiting more volunteers for the event.

Some types of personnel have many different types of tasks while other volunteers have

one type of task. For example, in Program 10, the program leader has many different

responsibilities, particularly leading up to the event. “I create the registration forms, handle

all the logistics of parents paying, answering parents’ questions, getting the kids enrolled,

splitting up the kids into groups, hiring the counselors, filing all the paperwork for that,

doing the interviews, deciding which teams to hire, creating a schedule, deciding on all the

activities, finding all the activity leaders, getting the space, buying the materials, ordering

the food.” When the event takes place, then the other volunteers come in to take on a specific

role. For example, when 150 kids arrive at Program 10’s summer camp, the program will

“split them into 12 groups by age . . . [Counselors will] take them to six different . . . scientists,

usually grad students and postdocs, who will then have one activity that they’ll teach the

group . . . [grad students will] just teach the same activity to six different groups throughout

the day.”

As shown with Program 10, many different tasks, especially dealing with logistics, may

fall onto one person. Program 13, a traveling student program that is run by a paid staff

member, notes that “It’s hard for faculty to help with how often these events happen . . . It’s

not reasonable to ask a faculty member to try to have a schedule that’s opened out to schools

once or twice a week. That is literally a full-time job. That is why I was hired.” There are a

121

lot of tasks and responsibilities in running a program that can be overwhelming for someone

who is volunteering in addition to their main job responsibilities. Having a full-time staff

person can help alleviate some of that workload; however, that would require funding and

likely institutional support as discussed in the Connection to the Institution and the Funding

and Budget sections.

4.6.2.2 Leadership Roles are not Limited to Just the Program Leader

As discussed above, some volunteers are assigned a task such as running an activity multiple

times, like Program 10’s summer camp. Other responsibilities, which typically fall on the

program leader, require additional levels of decision making. This includes event scheduling,

designing activities, and hiring other personnel. Based on their role and power in the

program, program leaders are the ones doing these tasks, or at least assigning tasks among

the other personnel.

Other ways that leadership is built into a program is by allowing volunteers to have a role

in the program’s decision making process. For volunteers who are involved in the program

for longer periods of time, some program leaders will give them larger responsibilities that

involve decision making that impacts the larger functionality in the event. In Program 10,

volunteers “have to learn leadership experience in terms of how to guide the kids. How to

politely ask the kids to do something in a way that the kids will actually do it.” In this

case, volunteers have the ability to make decisions and learn from their decisions in running

their assigned responsibility. Another example is Program 13’s demonstration show, which

goes on a multi-day road trip over the university’s spring break to visit schools in rural

areas. The program was limited on how many volunteers they could take on this trip. The

program leader was busy with other responsibilities so they did not have time to pick which

122

volunteers could participate, so they gave that responsibility to more senior volunteers. “I

did relinquish choosing volunteers to [2 more-senior volunteers] who both have been on the

trip more than once at that point. And so I had confidence that they would be able to

choose good volunteers.”

4.6.3 Continued Support of Personnel

Continued Support of Personnel is centered on how personnel are retained and supported in

their roles in the program. This component is important as personnel participation is often

voluntary and in addition to their main job and responsibilities. Program leaders can shape

program culture, opportunities, and support for personnel members so that the personnel

also benefit from participating in these programs.

4.6.3.1 There are Different Compensation Options that Programs Offer

Personnel

Some programs financially compensate their personnel. For example, Program 8 is able to

use their grant money to pay for support staff. However, funding can be limited for some

programs. Program 7 can only pay some of their personnel members for their weekend class

programming. “I pay the project leaders and I pay the mentors. And then if I have a young

mentor . . . I usually have a volunteer mentor kind of role for them so that we can try them

out before we actually hire them. And I can’t hire that many mentors anyway because,

like I said, the budget is pretty tight.” In this case, the program financially compensates

more experienced personnel with more leadership responsibilities. Other positions are not

financially compensated and are more purely volunteer positions due to limited funding.

Program 6, a demonstration show, offers their student volunteers different compensation

123

options. “[Volunteers] have the option of either taking pay for this, or we also have a program

on campus [for students who] plan on becoming teachers, and they can get class credit for

it.” This particular option requires some support and buy-in from the institution.

4.6.3.2 Programs Mentor Student Volunteers to Build Professional Skills

Programs will check-in with their personnel and help them to improve their skills in the

program. In Program 10, “we provide, I think, honest feedback about where they need to

improve and what they’re good at and stuff like that.” Some programs, like Program 12,

rely on volunteers who may not be physics majors. “Most [volunteers] don’t have [a] physics

background, but some don’t have science backgrounds. And most of them aren’t educators

to begin with. And so we’re helping them develop this expertise in facilitation and a comfort

with physics.”

Some programs consider professional development of their student volunteers as one of

their goals beyond just engaging with the public. For Program 14, “We’re here to teach the

public and university students about the universe. But part of that is teaching students on

how to be professionals in science communication as well, whether or not they end up in

that career but that’s part of the learning experience.”

Part of the mentoring technique that we find programs utilizing is allowing students

to take ownership of their work and have the power to make some decisions. For example,

Program 7 allows more autonomy to their more senior students. “There’s the project leaders

who have been there for a while and know what they’re doing. They get full control over

what they want to run. I don’t tell them what to do.”

124

4.7 Audience and Community Partners

The goals and mission of programs include engaging with audiences, oftentimes engaging with

a specific type of audience such as kids, adults, low income, etc. To effectively meet and work

with these audiences, we find that programs take steps to understand their audiences and

learn how to support their needs. Oftentimes, this involves working with other partners in the

community. We describe these ideas with the following components: Understanding Audience

Motivations and Needs, Support of Audience Engagement and Learning, and Relationship

with Partners and Community.

4.7.1 Understanding Audience Motivations and Needs

Understanding Audience Motivations and Needs is centered around the initial steps that a

program takes to identify the audiences they want to work with. There are several types

of audiences that programs may cater their program to. For example, programs may be

designed for adults, teachers, families, or kids. Program leaders have to consider that

audiences have varying wants and needs from participating in a program. Understanding

what audiences want from a program can help with advertising and attracting people to

attend the program as well as designing appropriate activities.

4.7.1.1 Understanding Formal Environments Helps Create Appropriate

Informal K-12 Programs

Some audiences attend a program’s events because they want to learn physics knowledge or

skills that they may not learn elsewhere. For example, each year, Program 8 hosts an all-

day event for high school physics teachers from across the country on topics such as modern

125

physics. Each year is a different topic so that physics teachers are “getting exposed to physics

happening now . . . they’re having exposure to current research.” Skills and knowledge from

these types of events are often for teachers to apply to their current jobs, which may not

necessarily be relevant for other adult audiences.

Some programs collaborate with formal learning spaces for youth audiences. This includes

working with teachers and schools. There are local and national STEM education standards

that students need to meet, so some programs work with teachers to develop content that

meet those standards. Program 11 did this with their teacher partners by “look[ing] at

the standards and in the demo closet and figur[ing] out an activity that we [could] do.”

Program 9 is “trying to be more aware of the Next Generation Science Standards (NGSS)

and using those as at least, like launching off points.” Schools take field trips to the Program

12 museum, so the program designs their activities to connect with different NGSS content

with the goal that “teachers will be able to choose the field trip program right now.”

For some programs, learning goals extend beyond science. Program 9 leadership reports

that “we are learning more about the importance of social emotional learning, because lots

of elementary schools are starting to implement social emotional learning in there as part of

their curriculum, and so, you know, how can we help support that and more holistic learning

in an environment like what we have?”

4.7.1.2 Reducing Barriers Allows More Types of Audiences to Participate

Having an awareness of the community that a program is wanting to work with can help it

connect and engage with audiences. Some programs work with multilingual communities, so

programs consider their needs when developing activity and informational materials. Where

Program 12 is located, there is a push for more bilingual education in schools. Since this

126

science museum frequently works with schools, they are also translating their materials

for Spanish speaking audiences. Program 10 also takes language into consideration when

advertising their summer camp as the program leader will “deliberately send flyers in Spanish

and Chinese out to local communities because English is a second language in this area.”

Program 14 realized that some of their programming can be “sensorily intimidating,”

which led them to explore and partner with different groups to develop sensory-friendly

activities. “Our sensory-friendly programming came out of a desire to make the planetarium

more accessible. We know that [our program] is a very sensory-rich experience in terms of

loud sounds and the visuals and feeling like you’re surrounded.” One example is that some

of their planetarium shows adjust their presentation style using more calming sounds and

softer speaking voices to create a more relaxing show experience.

4.7.1.3 Physics Events can be Entertaining Endeavors for Audiences

Since some programs are designed for audiences to attend in their free time, programs

consider what the appeal of attending may be for their audiences. Appealing factors can

be the low price of attending and entertainment value. For example, college campuses have

college-aged students who are often looking for things to do on a budget. Program 14,

a planetarium, will put on public shows on campus and promote that “we’re a really good

cheap date which we emphasize to all of our college students.” Some audiences look for things

to do when the weather is not nice. Program 15 takes this into consideration when deciding

the time of year to hold their festival events. “One of the things that we’ve learned to do

is to hold it when the weather is crappy, right? Hold it in February, because everybody’s

inside.” Programs also consider if there are other events, holidays, and competing activities.

In some instances, parents may need something for their children to be involved with, in

127

part for the childcare aspect. In some cases, programs may serve in place of daycare. This

is the case for Program 10, where their camp was cheaper than daycare. This program also

altered the fee process to be more flexible for lower income families.

4.7.2 Support of Audience Engagement and Learning

Support of Audience Engagement and Learning focuses on audiences who are currently at the

program. Here, program leaders are interacting with audiences through the context of physics

activities. Program leaders may get feedback from audiences and use that information to

evaluate how to keep their events engaging, educational, and supportive for the audiences.

4.7.2.1 Social Interactions are Crucial for Audience Engagement

Programs interact with audiences in many different ways, but keeping audiences engaged

often requires some form of audience interaction. Q&As are a common way that audience

members can ask questions with physicists, especially programs that have presentation

components. Public lectures often have dedicated times where audiences can ask questions

to the speakers. Other interactions include assisting audience members with an activity or

sitting down and conversing with audiences around a table.

Many of these interactions are driven by the program leaders and volunteers. Programs

that have more hands-on activities tend to have a volunteer facilitator who can give direction,

feedback, and answers to questions, but often in a fashion of a multiple-way interaction. At

Program 15’s festival-style event, “I have definitely seen some of the undergrads [volunteers]

just like going through a 10 minute lecture on how craters are made. I’m just like, ‘maybe

you should take lunch now.’

. . . I think I did more of the Socratic dialogue . . . and I think

that’s what a lot of people do . . . good teaching is asking leading questions, regardless of the

128

environment.”

Sometimes the social interactions are more than just making connections between the

audience and the personnel. There is a community building that can happen in the

program space that comes from social interactions among the audience members themselves.

Programs do that by allowing breaks between activities or having audiences work together

on a task. As a result, some programs have repeat attendees to their events. According to

Program 8,“There are teachers who’ve been coming back to [our events] for a long time.”

4.7.2.2 Opportunities for Feedback Provide Insight on what Audiences Find

Engaging

Programs often collect feedback from their audiences in order to adapt their programming

to be more engaging. This can be in the form of surveys, feedback forms, or comments from

conversations. For Program 10’s demonstration show, the program leader asks kids in the

program which activities that they like, “‘And so that’s really useful for me in planning the

next year’s [activities].”

Audiences have different interests and background knowledge, so programs take that

into account in their presentations and activities. Programs encounter this especially when

working with kids of varying ages. Program 11 notes, “How do we talk about physics with

five-year-olds? Because people don’t think about that in their day-to-day life. Cultural

considerations are also important. For example, Program 10 explained how they work with

kids from other countries: “I have kids that don’t speak English sometimes because their

parents might be here for a research phase. So I had an Italian girl. Luckily, [we focused

on] hands-on activities. You don’t have to understand, you just watch and follow along.

She wrote a beautiful poem. We did poetry that year and so it was an Italian poem that

129

sounded great.”

4.7.3 Relationship with Partners and Community

Relationship with Partners and Community points to the important relationships that

programs have external to their home institution that allow them to communicate with

and understand their audiences. Additionally, many programs do not have the resources,

funding, or space to operate their events alone. Programs work with sponsors or donors to

help cover the expenses of their events. Local businesses or organizations help to provide

a location to hold events. Local community groups provide knowledge and insight about

working particular audiences, which can help with improving accessibility, relevance, and

trust with audiences.

We define partners as anyone outside of the program’s institution that a program works

with. While this broad, there are some commonalities in the types of partners that programs

work with. For example, in order to build a larger event that draws in bigger audiences,

some programs collaborate with festival organizers or student organizations to host an event

where multiple programs are participating.

Another type of partnership is one with an individual(s) who are serving in a liaison-style

role between the program and its intended audience. In the case of working with specific

cultural groups, this type of partnership is useful in understanding a group’s norms, needs,

and interests. For example, when Program 14 discussed how to make their program more

sensory friendly, “we did that through talking to [multiple resource groups] in the area just

to make sure we were doing it in a way that made sense for people.” Another example of a

liaison-style partnership is working with teachers. When working with schools, children are

the intended audience, but the teachers and administrators are often the ones that programs

130

work with on determining appropriate content and what works well for engaging students.

As discussed in the Understanding Audience Motivations and Needs section, some teachers

have certain science standards that they are required to reach, and these partnerships with

programs can help to meet those requirements.

A third type of partnership entails logistics such as venue hosting for events, funding

for the program, or sponsorships. These partners are often involved in operations, but not

necessarily content or programming (though in some cases they are). A program may partner

with a local business to host a physics event there and attract people to the business. For

example, Program 15 hosts their physics festival event at a local children’s museum. Program

1 hosts their public lecture series at local venues like pubs and restaurants.

4.7.3.1 Forming Partnerships Requires Time,

Communication,

and

Understanding Each Other’s Goals

A contributing factor to forming a partnership is understanding the program’s own goals

and the needs of potential partners. Programs may have a particular audience in mind,

such as school children, so they may partner with schools. For example, Program 11 is

a graduate student organization that visits schools, primarily working with K-6 students.

Because graduate students have limited time, most of the schools that they visit are local.

Building and maintaining partnerships takes time and work to form trust. Oftentimes,

partnerships stem from a relationship or connection with members of the community without

the initial

intention of forming an explicit partnership. For example, Program 13, a

demonstration show, partners with another informal physics program in hosting some annual

events. That partnership began “solely because I first made that personal connection with

the people who run the program, before learning about the program.” However, sometimes

131

programs may push to advertise themselves and what they have to offer in hopes that it is

of interest to other potential partners.

Advertisement occurs on social media, news and radio appearances, flyers, and emailing

potential partners. As Program 14, a planetarium, notes, “if the news calls us, we’ll be

on the news. If the radio calls us, we’re on the radio. If a school wants us to come out

and do something, we’ll do it because it’s a way to also make sure that we are visible in the

community and people remember that we exist.” By increasing visibility, even if the program

does not initiate the contact with a potential partner, there is a heightened possibility that

that potential partner will contact the program.

4.7.3.2 Mutually Beneficial Partnerships Take Many Forms

Partnerships should be built from a mutually beneficial relationship. For example, when

working with teachers, a program can meet its goal of working with kids while teachers may

get their own state requirements met, and the students get an activity that is interesting

to them. Program 11 explained, “Usually [the high school teachers are] like, ‘Hey. Can we

come to your classroom?’ And then when I’ve emailed them back, I ask, ‘Oh. Well, where

are you going to be in the curriculum? We can try to do something that aligns with where

you are. Or do you have any standards that you want us to hit?’ And so a few of those

teachers have emailed me back, and they said, ‘Yeah. That would be awesome. We have two

standards that– if you can hit these two standards in your activity, that would be cool.’ And

so that’s going to take some time for me and the other coordinator because now we have

to look at the standards and go look in the demo closet and figure out an activity that we

can do.” Program 15 stated: “[our partners] were really good about reminding us that the

museum serves a lot of ages. And so the student groups are often really good at producing

132

activities that were good for, like, you know, elementary school age, kind of and beyond,

right, so like, second graders, first graders and kind of up, but they weren’t really good at

figuring out activities for two year olds, or four year olds. [Our partners] were really good at

making sure to remind us that we needed activities for that and providing us with resources

to make sure we were meeting that population.”

It is also important to consider how long partnerships last. Partnerships can last many

years, but they can also end, evolve, or splinter off. An example of a partnership that evolved

is Program 10, which says, “We partner[ed] with another organization. Now they’ve taken

it and spun it to better address their group of people. So it doesn’t have the same structure

but it’s still a STEAM camp with a lot of the same ideas.”

4.8

Institutional Connection and Resources

Institutions are the colleges and universities associated with their respective informal physics

and astronomy programs. Programs are often hosted within a specific institutional unit, in

most cases a physics, astronomy, or education department or center. The connection between

the programs and their host institutions includes the resources that institutions provide, the

institutional recognition of its members in the public engagement activities of the informal

programs, and the personnel that are part of both the informal programs and also affiliates

of the institutions. We describe these ideas with the following components: Connection to

the Institution, Physical and Digital Resources, and Funding and Budget.

133

4.8.1 Connection to the Institution

In this component, we focus on how the connection to the institution affects programs and

vice versa. While this component is referred to as Connection to the Institution, sometimes

the nature of this connection is more of a relationship that is mutually supportive of both

entities.

In other cases this connection can be a source of tension, or sometimes both.

Universities also provide resources, including rooms for events, facility maintenance, hosting

of web servers, university insurance, and financial support for programs. We will discuss

these in more detail in the respective components of Physical and Digital Resources and

Budget and Funding.

4.8.1.1 Positive Institutional Relationships can lead to Financial, Resource, and

Volunteer Support

One benefit of the institutional connection for programs is sustained funding over time and

resilience in the face of budget issues. Multiple programs reported that during challenges

from COVID-19, being connected to the department or university budget offered them

protection from closure. This was especially salient in the cases of Programs 9 and 14,

which are planetariums connected with large research universities. For example, Program 9,

states: “Being part of the university, being part of this department that sees us as an asset,

and something that they want to continually support definitely gives us a lot of stability.

It’s, you know, and like, right now, there’s a great example during COVID times, you know,

we have this larger backing department and university to kind of help make sure that we’re

not going to close down. If we were completely separate, that might not be the case.”

Another benefit for informal programs of their connection to a department or center is

134

access to physicists, astronomers, students, and facilities for research and teaching. For

example, being connected to a physics department helps Program 15 recruit the large

numbers of student volunteers it needs to run its open-house style event at a local museum.

Program 10 uses physics research facilities to host middle school students for a summer

program. The expertise of faculty and other researchers is also a benefit to programs -

Program 9 commented on this, saying, “we have very direct access to how this field is

changing and what we’re learning in it and reputable sources who can help our presenters

or our staff kind of learn these topics and make sure we’re communicating them effectively.”

4.8.1.2 Departments and Universities also Benefit from Working with Programs

In the Support of Personnel section, we discussed how programs can be professional

development opportunities for students. The ability to offer this professional development

opportunity is also a benefit to the institution.

Informal programs provide a way for

undergraduate and graduate student volunteers to be trained in science communication and

to gain direct experience communicating with public audiences. Some form of training

happens in all programs in our data set.

In a few cases this professional development is

formalized and recognized by the institution. In Program 12, student volunteers can achieve

an official certificate in educational partnerships for working with the local community

partner; this university-recognized certification can be listed as an item on professional

documents. Other programs are connected to classes, offering a location or experience for

students.

Programs also can be useful in recruitment of students to universities. For example

Program 9 explains that bringing students to its planetarium “has been a pretty impactful

way to highlight what happens at the university and the research that happens or just like

135

creates a very memorable experience. So that’s where we’re, you know, trying to work

with admissions a little bit more to utilize our space as an inspiring place to recruit more

students.”

Informal programs also support physics and astronomy faculty at the institutions. In

several cases, individual faculty had leveraged the institutional connection with the informal

program in broader impacts sections of grant proposals for other types of research. Large

scale physics centers also relied on the success of their informal physics programs when

reapplying for funding, such as with Programs 8 and 10. Programs also benefit faculty by

giving them an outlet to share their work with the public and also connect with communities

of which they are a part. For example, Program 15 explains: “And so I feel like, [this event]

gave us that opportunity to do it as a department, right, as opposed to like, the University

doing it, or the college doing or whatever it was, like, no, this is something we own something

we’re responsible for. And something that is, that is like really nested in the kinds of things

that we do that demonstrate to the public that we are a valuable [resource], we’re valuable,

we’re valuable to them.” There is a sense that these programs can inspire some community

goodwill that is beneficial to the institutions.

4.8.1.3 Power Dynamics can Create Tension Between the Program and the

University

For some programs, however, their relationship with the university can sometimes cause

tension or stress in terms of program design, implementation, and support. We note that

this is especially prevalent in some interviews with student leaders as opposed to faculty

leaders, in particular with situations such as requesting funding.

Multiple student led programs report tension in asking faculty and department

136

administrators for financial support. The student leader for Program 11 explains their

perspective in this way: “So they give us money, but it’s a fight to get money every year.

So basically what we do is we put together a budget and then go ask for money, and then

usually they say no, and then we go to the head, and then the head is like, ‘Okay,’ and

then we get money. Every year they think– this is my viewpoint. I think, every year, they

think that we’re asking for more money than we need. In that sense, I don’t think it’s as

supportive as it could be.” Budget issues specifically can cause tension with programs and

departments. For example, after COVID, Program 15 states: “You know, there’s no, there’s

really no discretionary spending at the department level anymore. And so going and hitting

up the chair for a couple hundred dollars is going to be a pretty hard ask now.”

Even programs that have a generally positive relationship with their university unit can

sometimes face hurdles. Program 9 reports that sometimes planned activities are delayed for

other university events, and recently they’ve had issues with university changes to parking:

“now you have to pay all hours of the day, and that will significantly change our patron

experience. Football games happen, and then you can’t get on campus.” They also have to

conform to certain university policies, and so their signage is not as large or creative as they

would like to draw in audiences.

4.8.1.4 Department Attitudes Towards Public Engagement may be Changing

but still have Room for Improvement

Some program leaders report that program activities are becoming more integrated into the

identity and activities of their respective departments or colleges. For example, Program 9

says: “I think recognizing the importance of outreach is also increasing over time. I would

say that five years ago, it was not as appreciated as it is today by many faculty. So that’s

137

where I would see that, [our program] overall, I think is being increasingly appreciated.”

The program leader also notes that as new faculty members are joining the department,

they are incorporating the planetarium’s work in their own classrooms.

In addition, the

program leader says that faculty are increasingly using the programs as a part of their

public engagement work for grants. “I think all of those things are just sort of adding to

this sense that community outreach is important, and that the department wants to make

time and put energy towards that.”

Program 15 talks about the growth of their program over time, stating “with more

faculty involvement and interest, we, I think we start to realize that like, we can do this as

a department and we can coordinate it together, and we can bring in, we can bring in more

than, you know, introductory physics demonstrations, right, we can actually bring in really

cool science and bring it into a space where families can interact with it and sort of develop

a bit of an understanding of like, What does [our physics department] actually do?”

However, some program leaders, especially student leaders, express a desire for more

support from faculty. Program 11 suggests that faculty in their department could do more

to support the group that is responsible for the department outreach, saying, “I think faculty

people like the idea of [this student group focused on issues of women in physics], and I think

they use that as a token, like, ‘We have a women’s group in the department. We like that.’

But they don’t do anything in their day-to-day life to support that. They would never show

up to [our groups] events . . . So I think they like to tokenize that we have a women’s group

and not be super supportive of it.”

138

4.8.2 Physical and Digital Resources

There are two big categories of resources that are important for running programs - physical

resources and digital resources. Physical resources was the most mentioned category. These

are tangible items that are necessary for programs to run, and they are connected but

separate from program finances which we describe in the Funding and Budget component.

Programs also use digital resources to varying degrees. Digital resources include websites

and apps for public audiences that must be maintained. Programs also use file sharing and

storage to document and pass along digital copies of activities, photos, or other program

information. Also, during informal education activities, some programs use digital tools

others have developed, for example PhET simulations.

4.8.2.1 Community and Campus Venues Affect Program Content and Audience

Participation

The resource common to all the programs in our sample was having a venue, physical space,

and/or building where programs are held. Venues can be off-campus or on campus, or

dedicated STEM education spaces (Ex. planetariums, museums). Off-campus examples

include libraries, school cafeterias and gyms, conference centers, and outdoor areas. Venues

constrain what types of programming can be done in specific rooms or areas. For instance,

the size of a room limits the number of people who can engage in a program. The type of

seating also limits ways for audiences to participate. For example, lecture-style classroom

seating precludes activities that use table-top experiments.

Programs that travel to community spaces (e.g. physics demonstration shows) need to

be flexible in order to accommodate different physical arrangements in varying off-campus

139

venues. For example, low ceilings mean some demos cannot be done safely, lack of outlets

may mean some demonstrations cannot be performed, and available seating for audiences

can affect their ability to see equipment up-close. Because venues may have other events,

program personnel might have limited time for set-up and clean-up, or have to pack in

and pack out all equipment completely, which means any equipment must be transportable.

Additionally, program leaders have to balance the other events at the off-campus venue; for

example, they cannot schedule or create a new program whenever they want but must work

with the venue staff as part of event planning.

Some programs host audiences at the host institution, i.e. on a college or university

campus. Specific program locations are usually in classrooms, or other building spaces. An

affordance of campus venues is having somewhat more control over the space, as program

personnel are usually employees or students at the host institution. However, audiences

must travel to campus for events, which can bring up issues of transportation, parking, and

accessibility.

4.8.2.2 Permanent Venues

for

Informal Physics Programs have many

Affordances

Museum and planetarium spaces allow program leaders to have significant control over the

physical space, which in turn affects the ways audiences are able explore physics activities.

These spaces can be designed to have signage, displays, and seating that prompt certain types

of audience participation. For example, Program 12, which is located at a local museum,

explains: “[the museum] doesn’t have signs on the wall, it has signs like where the bathroom

is, but it doesn’t have signs on like what you’re supposed to learn at it. So it’s very open

ended there.” In permanent venues, program leaders have more control over the creation

140

of new exhibits and content, which can allow them to expand their learning goals or reach

different audiences. Program 9, a planetarium, described working on their lobby renovation

as “creating the space to be a little bit more updated and have exhibits that are maybe a

little bit more interactive.” Program 14 has been able to encourage audiences creativity by

developing exhibits that incorporate art into physics and astronomy programming.

Having a permanent space can also be limiting. Many spaces have restrictions from

university rules and often limited budgets. Venues like planetariums and observatories

require specific staff for running telescopes or dome projection systems. Museum spaces may

need staff trained for the preservation of a historical collection, or expertise in mechanical and

electrical work for interactive exhibits. Permanent venues also require facility maintenance

from use over time. These venues are difficult to maintain without specialized staff, which

often require added costs for salaries and equipment. Some programs have to share their

space with others. Finally, the size, shape, and technological parameters of the physical

building can limit program changes and growth.

Some programs need to expand to

community locations to continue to grow programming and reach more people. For example,

the director of Program 14 describes the limits of their venue: “I feel like right now, we are

kind of hitting our capacity in terms of staff time and space really to do too much more.

But I think where we do have the room to grow and where we would probably see the most

change in five years is what we’re doing out in the community.”

4.8.2.3 Physics Equipment is Integral to Creating a Meaningful Experience for

both Audiences and Program Personnel

Most programs use physics equipment and demonstrations for some or all of their activities.

In many cases, activities are built around using equipment and demos. Program 10 describes:

141

“the focus isn’t the materials, the reading materials, and the focus isn’t the speaking.

Those are just supplementary so that you can get on to using the equipment and supplies.”

Sometimes programs take physics demos for classroom use and adapt them to be appropriate

for kids and safe for use. Other programs use sophisticated equipment to show off current

research, such as microscopes. Museum and planetarium spaces can have big physical

equipment connected to physics ideas. For example, Program 12, which is located at a

science museum, describes one exhibit on quantum computing applications that has a giant

light pegboard with pieces that audience members can put in it and move. In all cases, demos

need to be updated, maintained, and sometimes replaced. Some activities use “at-home” or

upcycled materials, which has been especially useful during COVID. For example, during

COVID, Program 12 had kids create buildings out of cardboard and then drop them off at

the museum.

Importantly, physical equipment is not just for use by audiences; a recurring theme is how

much program personnel find enjoyment using physics equipment and demos themselves.

The director of Program 10, a youth summer camp, states: “For a lot of [the program

volunteers] it– and especially the older grad students or post-docs –it helps remind them of

why they get into science in the first place. Seeing that excitement, playing with the toys

when you get bogged down in research, sometimes it’s good to step back and remember

why you got into it in the first place.” As an example, the student leader of Program

13 commented on their own enjoyment of a particular piece of equipment they use in

demonstration presentations - a Theremin: “[I’m] the one who actually learned how to

play it [laughter].” Program leaders report that student volunteers often choose activities

or demonstrations that are on interesting topics, visually exciting, or showcase surprising

phenomena. For example, Program 13 reports that electricity and magnetism demos are

142

the favorite of their undergraduate student volunteers. Being playful with ‘physics toys’

and sharing that experience with audiences is a major source of fun for physics students in

informal programs.

4.8.2.4 Resource Management is a Time-Consuming Part of Running Programs

General event planning is also a part of most informal physics programming. Food for

volunteers and for participants, travel costs for personnel and for audiences (such as renting

a bus), and consumables for demos can make up some amount of the program budget, or

most of it in the case of Program 11. Some programs also spend money on t-shirts, stickers,

flyers/posters, and related items for advertising and recruiting. Different programs may use

specialized equipment like sound equipment or vans/vehicles for transportation.

One issue that comes up for program leaders is managing these resources. Lead

facilitators can spend significant time on tasks like purchasing supplies, managing equipment,

coordinating with venues, ensuring food is delivered, and planning for transportation. These

aspects of event planning that are unrelated to physics content can be a challenge, as the

Program 10 director states: “teaching in physics does not tell you how to run a summer

camp, right? I don’t know what kind of food we order, where to have the kids dropped off,

how to manage the kids so that the parents feel that they’re safe, all that stuff I had to learn

on the fly.” Program leaders may feel tension between the amount of time they spend doing

event management tasks and the time they spend developing physics activities or interacting

with audiences.

143

4.8.3 Funding and Budget

The Funding and Budget key component is about the monetary funds that a program receives

and how it utilizes them. Within the funding and budget component are fine-grained issues

related to grants, donations, partners, and institutional connections. It encompasses how

the program allocates the money they receive. This category also includes any changes in

funding and budget over time or challenges that program leaders face on these fronts. Here

we discuss the key themes of this component.

4.8.3.1 Programs find Sources of Funding from Local to University to National

Levels

Some informal physics and astronomy programs are funded in part or entirely by national

agencies, the most common being the National Science Foundation (NSF). Funding from NSF

can either be independent STEM education awards, such as the Advancing Informal STEM

Learning award, or it can be part of the broader impacts of a more traditional research

project, where the educational and outreach activities make up a fraction of the overall

budget. NSF grants can often provide substantial funding, in the hundreds of thousands or

million dollar range. For example, Program 9 reports that “a portion of how we are funded

is through grants. And that makes it very easy to be part of a university where, you know,

faculty are receiving grants that require a broader impacts, outreach component, and then

we can serve as that outreach arm for that project. So that’s how we can achieve a lot

of our funding.” Another affordance of national grants are their flexibility, which allowed

some programs to absorb some hits from pandemic. For example, Program 12 states, “the

National Science Foundation has been very, very flexible and very, very understanding of

144

‘Yeah, we know you’re, you know, you’re trying, you’re doing what you’re not quite doing

what you thought you would do, but you’re doing something and yes, keep paying your

graduate students.’ And so we were able to keep paying our graduate students, which was

good.” One challenge, however, of obtaining NSF funding is the amount of work and resources

needed to put together a proposal, and an uncertainty of obtaining funding given the highly

competitive nature of the NSF award process.

Programs also use additional funding sources including a) grants from professional

organizations, like APS, or national foundations, b) smaller grants from local government

and community-oriented organizations c) university, college or department level funding, and

d) individual or business donations. Programs sometimes generate their own revenue from

audience participants through ticket sales or program fees; however, in these cases, programs

are typically aware that this can limit audience attendance and may be in conflict with their

goal to be accessible to all audiences and thus keep the fees small.

4.8.3.2 Budgets are Significantly Impacted Depending on the use of Student

Volunteers or Paid Staff

Personnel are the biggest chunk of budget for most programs unless it is all volunteer staffed.

Programs with large amounts of funding, such as from government agencies, seem to have

more personnel that are paid, such as part-time or full-time staff roles, which can amount to a

full post-doc or higher level salary. Some programs have a model where students are paid for

repeated work, but not for one-off events. Main expenses for programs aside from personnel

are supplies and equipment, such as for demos and hands-on activities, food for kids or

volunteers at program events, and transportation costs related to traveling to off-campus

events. Some faculty noted that informal programming is relatively inexpensive compared

145

to other educational or research activities in department budgets. However, this is typically

true when the program relies on volunteers who work for free. For example, a comment from

a faculty member leading Program 15, which operates with student volunteers and faculty

committed to the program as part of their service: “I don’t think we’ve ever had a money

issue. Because it’s pretty inexpensive to run? Sure. I mean, I think the most expensive

items are the poster prints and the T-shirts.”

4.8.3.3 Multiple Sources of Funding can Support the Stability of Programs

Stability for informal programs can look like being in operation without hiatus, the ability

of a program to consistently offer a certain activity, or having enough personnel to operate

reliably. An important part of how funding affects stability is the number of funding sources

that programs have. Some programs have funding from only a single source this can provide

a reasonable amount of stability, for example in the case of funding from a large NSF grant

that is able to be renewed. However, single source funding, for example from a partnership

with a local business or from only the department budget, can potentially lead to instability.

For instance if the circumstances at the partner or host institution change and budget cuts

are needed, the program may not be seen as part of the core function of the institution.

New sources of funding can lead to new initiatives or the expansion of existing activities;

however, these programs can retract if or when that particular source of funding ends. Some

funding sources have built in timelines, like most grant funding, and program leaders must

consider a plan for continuing the programming after a grant ends. A number of program

leaders reported actively seeking funding from multiple sources, looking to incorporate more

funding sources for program stability in the case that some funding is scheduled to end or

ends unexpectedly. For example, Program 9 reports that “making our kind of sponsorship

146

and donation space more robust is going to be important going into the future so that we

can be a little bit more autonomous and sustainable.”

4.8.3.4 Funding Cuts can be a Challenge

Grants that end and university budget cuts can both affect programs. When budget

cuts happen, typically program cuts are to paid personnel positions and the amount of

programming offered. For example, when Program 8’s national grant funding ended, they

were able to keep running but they had to cut funding for a postdoc and the one semester

fellowship they offered to graduate students for participation. However, being seen as an

important resource to the campus or community is a way to shore up programs against

budget cuts. For example, Program 14: “the more we use the planetarium and the more we

become a valuable asset to many people on campus and in the community, the less likely we

would be a target for state budget cuts and things like that as well. So there is that ulterior

motive in a way for doing this. The ultimate goal is I really want people to learn and have

a really good time learning. But there is also this piece of it. If people are learning in a lot

of different ways and we’re valuable, then that provides us a bit of a safety net.”

4.9

Implications

Thus far, we have presented on each of the 12 key components separately, highlighting key

themes and examples for each. While we find that these 12 components are all important to

programs, the level of importance and the manifestation of each component will vary across

each program. Here, we present on how these components are connected to each other. We

will also discuss how we envision program leaders and informal physics practitioners using

147

this framework as well as future work.

4.9.1 The Interconnectedness of the Key Components

The reality is that these 12 components do not operate independently of each other.

Informal physics programs are complex systems, and as such, these components have strong

connections to each other. For example, a program leader may choose physics topics

specifically to attract audiences. Perhaps the program leader used a form of assessment

to get audience feedback on what audiences find engaging, and this leads the program to do

more astronomy related activities. If that program makes the switch to doing more astronomy

related content, then they will need to make sure that their volunteers are appropriately

trained to cover such material. It could be that they would even need to recruit volunteers

with an astronomy background. These components being closely connected to each other

adds a layer of complexity for program leaders to consider when reflecting about their own

programs. An additional complexity is that program leaders emphasize and prioritize each

component differently.

4.9.2 Considering How Practitioners can Utilize this Framework

As discussed in the Strategies for Assessment component and in prior work [106],

practitioners want to be able to conduct assessment to evaluate aspects of their programs,

but often lack the tools, time, and resources to be able to do so. One of the goals of this

key components framework is to identify aspects that are important across informal physics

programs and that program leaders have influence over. It is important to do no harm in

these programs. While using this framework can help in designing or improving programs

148

and is well intended, it may not be as informed or knowledgeable as the program leader

about specific aspects. These components can serve as guideposts that program leaders can

use to evaluate different aspects of their program.

Identifying the important areas that program leaders should focus on helps to lay the

foundation for the future development of assessment tools and resources, for example,

reflective rubrics. As an example to help guide discussion on the creation of tools, we provide

below sets of questions that accompany each of the key components. These questions are

meant to help program leaders reflect upon their own programs and think about how they

address key features of their own program. For example, below is a list of reflection questions

regarding the Support of Audience Engagement and Learning component:

• What are the spaces where audience members can ask questions in my program? Is it

set-up so that there can be a back and forth conversation?

• How do audiences interact with the personnel? With other audience members?

• What are the ways that audience members can give me feedback?

• What background knowledge does the audience have on the physics topics?

• Are my program materials accessible to audiences with differing needs? Are my

program materials translated into languages that serve my audiences?

A more comprehensive list of reflective questions for each component is provided in

Appendix D. It is worth noting that because of the interconnectedness of the key components,

some questions may have relevance to multiple components. In addition, every program has

its own set of unique features, so not every question will be relevant to every program.

Instead, the idea is that program leaders can reflect upon the questions for the components

that are most pressing for their own situation. We envision a couple different ways that

practitioners can utilize this framework tool.

149

One approach is for the practitioner to go through each component and accompanying list

of questions in Appendix D and reflect on their relevance to the practitioners program. This

process can help the practitioner identify potential areas that they should make more detailed

plans on improving on. This approach of going through each component and question

may be especially helpful for younger programs, practitioners creating a new program, or

practitioners who recently took a leadership role in a program. While potentially tedious,

going through each component and question can help the practitioner to more broadly

establish where the program is at and the direction that it wants to go.

Another approach for using this framework is for the practitioner to reflect on the aspects

of the program that they think are going well and the aspects that they have challenges

with. Upon their reflections, practitioners should think about the relevant components for

that scenario, and then seek the reflection questions in Appendix D for those components.

For example, a program leader may be able to easily get volunteers and the volunteers may

be enjoying their time in the program; however, the program leader may also find it difficult

to keep audiences engaged with activities. In this scenario, the practitioner may consider

prioritizing their focus on the Support of Audience Engagement and Learning, Understanding

Audience Motivations and Needs, and Program Content and Design components over the

Recruitment and Onboarding of Personnel. This approach for using the key components may

be more useful for program leaders who have had more experience leading their program.

4.9.3 Future Work

Program leaders are regularly thinking about how they can improve upon their programs.

Programs are complex and there are many important aspects that program leaders have

to consider with limited time and resources. The development and implementation of this

150

framework is a way to professionalize the field of informal physics education and be used

as a tool for advocacy for more support of informal physics activities. We are particularly

interested in blending this framework with existing frameworks that look specifically at

inclusion, diversity, and accessibility. We believe that this framework will guide and inspire

researchers and practitioners to partner together in creating future tools and resources that

can help program leaders to more easily consider changes to their programs. Communication

is key for these partnerships.

In this regard, it is important to consider feedback and

reflections with the partners, consider the frequency of communication, think about the

kind of input that is being asked for or given, and negotiate and be aware of the voice and

level of input that each member of the partnership has.

151

Chapter 5. Using a Volunteerism Framework

to Understand the Motivations of University

Students who Facilitate Informal Physics

Programs

This chapter was published as an article for the 2023 Physics Education Research Conference

Proceedings [120]. The author order on this published article is Bryan Stanley, Kathleen

Hinko. Some aspects of this published work have been expanded for this chapter.

5.1

Introduction

Physicists and physics students engage with public audiences through a variety of informal

physics programs and activities like school visits, summer camps, open houses, public talks,

after school programs, and more. Participation in these activities are often voluntary for both

the participants and the facilitators. Many volunteers in informal physics programming are

university undergraduate and graduate students [4, 106]. Some students volunteer repeatedly

and for many hours in these spaces throughout their student careers, even though they are

often not earning course credit or being paid [15, 33, 133, 134]. We are working to understand

the nature of the deep and meaningful experiences that students are having. Studies have

looked at the impacts on university student volunteers and have found how some structures

of informal physics programs support physics identity development, sense of belonging, and

career-related skills [32, 33, 59, 60, 61, 62, 63]. Students have different career paths, some of

which are education related, and their experiences from volunteering can impact the careers

152

they pursue.

Some studies have looked at the motivations of university students who volunteer in

informal programs, noting themes such as positive experiences, sharing science, professional

development, and teaching opportunities

[63, 135].

In particular, some studies have

used community of practice [59, 60, 61, 62, 136] and personas [137] frameworks to better

understand the involvement of university students volunteers in informal physics programs.

However, many of these studies analyze students as they are actively involved in their

volunteering and do not look at the careers and long term impacts. Rethman et al. [33]

found that current and former student volunteers reported an increase in motivational beliefs.

Those motivations have strong ties to student excitement and development of interest and

skills that are relevant to becoming a physicist [33].

Here, we interview alumni at various points in their professional careers about their past

experiences volunteering in informal physics programs and how those experiences impacted

them and their career paths. Our overarching research focus is on the different ways that

informal physics programs can affect careers of volunteers. Our initial step in this preliminary

study is to apply a volunteerism motivational framework to 1) see how it maps onto the

informal physics space, and 2) understand how volunteers’ motivations evolve from their

first volunteer experiences in informal physics to their present lives. Before applying to our

larger data set, we first apply this framework to alumni whose careers ended up in educational

fields. For the scope of this paper, we aim to answer the following: How do informal physics

program experiences affect volunteers who go into careers related to education?

153

5.2 Volunteerism Framework

For many facilitators, informal physics is not a part of their main job or responsibilities.

For example, faculty may have teaching or research responsibilities while students may have

coursework or other involvements. The work that most of the facilitators do in their informal

physics programs is in addition to their main job or position [106]. We use the term student

volunteer to describe the university students who are contributing to the functionality of

these programs.

In some cases, but not all, university students may be compensated for

their involvement in their program, however, we still label them as a volunteer as their main

position is being a student and their informal physics involvement is on the side.

Given that participation in informal physics programming is often in addition to

one’s current job and responsibilities, there must be some motivations for the volunteer

to 1) volunteer in the first place, and 2) to stay in that volunteer position. The

volunteerism framework aims to understand those two points for volunteers more broadly.

Clary and Snyder define six categories that describe motivations for volunteering (Values,

Understanding, Enhancement, Career, Social, and Protective), as described in Table 5.1

[78, 79]. Pulling from foundations in psychology, this framework was tested with volunteers

in public health, hospital programs, psychology programs, and business, with populations of

volunteers being non-students, students getting course credit for their service, and students

not getting course credit [78]. While this framework is aimed for volunteering more generally,

Clary and Snyder acknowledge that these categories may appear differently based on the type

of activity the volunteer is participating in [78]. This framework has been used in scientific

spaces, such as citizen science programs and environmental sciences, but was adapted with

some contextual renaming or some context-based additions [80, 81, 82, 83].

154

For this pilot study, we want to test this framework in the context of informal physics

programs to determine if it captures the motivations of student volunteers. Applying this

framework to interviews with alumni helps to understand how those motivations connect to

the volunteers’ career paths. Because we are testing the framework, we are applying it to

a subset of our data. Given that informal physics programs are inherently educational, we

first apply this framework on alumni who had a career in education.

Table 5.1: Adaption of the motivational categories and their descriptions [78, 79, 80, 81, 82,
83].

Categories

Definitions

Career

Preparation and experience for career-related

endeavors.

Enhancement

Increasing positive feelings of oneself.

Protective

Reduced negative feelings from personal challenges.

Social

Building and strengthening relationships with

friends and new people.

Understanding Gaining and implementing new skills and knowledge.

Values

Belief that the person finds important to their life.

5.3 Methods

For data collection, we contacted the lead facilitators of two informal physics programs:

Traveling Physics and After School Physics. Both programs are housed at different large

research universities. Traveling Physics is a traveling program that visits schools with hands-

155

on physics experiments that were built by its team of primarily undergraduate student

volunteers. After School Physics is an afterschool program that partners with multiple

nearby middle schools and high schools. A team of undergraduate and graduate students and

postdocs visit each school once a week to lead hands-on physics activities and explorations.

We contacted these programs about our study because 1) both programs have been

around for decades, 2) both programs stay in contact with their alumni, and 3) the authors

have previous involvement with these programs. Program leaders shared information of

our study to their alumni contact lists in order to recruit participants to be interviewed.

Connections with these participants snowballed to members from two additional programs

who were also interviewed. In total, 25 interviews with alumni were conducted.

The approximately 30-minute interviews were semi-structured. Some interviews were

conducted in person and some were done virtually on Zoom. We asked participants’ about

their past experiences in their program, their career after leaving their university, skills

important to their current work, and memorable experiences the participants had in the

program, with other volunteers, and with audience members. For analysis, both authors

independently coded the interviews with the volunteerism framework using the qualitative

analysis program MAXQDA. Code units were full sentences in which the response met the

definition of a framework category. A code could be multiple consecutive sentences, but

not partial sentences. Sentences could include multiple codes. After each interview, the

authors compared codes, to which there was a high agreement, discussed any discrepancies

and documented common themes found within each category.

For the scope of this paper, we are only analyzing participants who volunteered in

either Traveling Physics or After School Physics and who ended up in an educational-

based career, whether that was formal, informal, or a combination of both. We will present

156

each participants’ trajectory linearly, highlighting the different motivational categories that

manifest throughout. The three participants we will discuss are “Amber,” “Mark,” and

“Claire.”

Amber began her post-secondary career as a student at a two-year college. After taking

an astronomy course, she decided to pursue a degree in physics at a large four-year research

university where she volunteered at Traveling Physics. Her job trajectory went in many

different directions. She worked as lab coordinator, worked in the tech industry, returned to

school to pursue a teaching degree, left the teaching program to open up her own business,

returned to the tech industry, then hired as assistant director for Traveling Physics.

Mark earned his bachelor’s degree in physics. As an undergraduate, he worked as a

teaching assistant for the undergraduate labs. He went to graduate school to continue

studying physics. As a graduate student, he volunteered at Traveling Physics. After graduate

school, he worked at a university as a lecture demonstration specialist, where he teaches some

classes and still participates in public engagement activities.

Claire was a neuroscience major as an undergraduate who wanted to go into education

since early college. As an undergraduate, she volunteered in After School Physics. She is

currently a middle school science teacher.

5.4 Applying Framework to Three Cases

Here, we will discuss the job trajectories of Amber, Mark, and Claire from when they

volunteered in their respective informal physics program to their current position. We present

each persons’ story individually and chronologically, highlighting key instances where the

participant mentioned one of the six motivational categories. We find that all six motivational

157

categories are present across these three interviews.

5.4.1 Amber

As an undergraduate, Amber got involved at Traveling Physics because her tutor was a

volunteer there and he told her that he thought she would enjoy volunteering with the

program. Prior to volunteering in Traveling Physics, Amber had experience working with

kids through nannying and daycare. Building from her enjoyment of working with kids,

one of Amber’s original motivations for volunteering at Traveling Physics was because, “I

get to come in, inspire some folks, and maybe get some young people interested in doing

science.” Values like wanting to inspire young people is one of the motivators for Amber to

get involved with volunteering, but additional categories also help retain her in the program.

As a student, Amber had jobs in retail and pizza delivery; however, “I was done doing

jobs that were just doing jobs.” She had Protective motivation for transitioning to Traveling

Physics to reduce the negative feelings she had working at those other jobs. Instead, she

wanted a job where she could use “my skill set that I thought I was good at. And so by

joining [Traveling Physics], I could just concentrate on school but also enhance my physics

learning. So I could go into, let’s say, my E&M class, and I’m struggling with this electricity

and magnetism concept, and I could come back into [Traveling Physics] and say, ‘How does

this work?’... So now I can move more into a career trajectory, versus I’m just doing jobs to

pay the bills.” Traveling Physics was providing Amber with Understanding motivations by

helping her to build upon her physics knowledge and skills. In addition, she sees these skills

and knowledge to be more helpful in lining up with her Career trajectory compared to her

other job experiences.

After earning her bachelor’s degree, she ran the undergraduate labs before getting a

158

job in the tech industry. After being laid off, she returned to her alma mater to pursue

a teaching degree. During this time, she worked again at Traveling Physics. Based on

her past experiences in the tech industry, Amber came into Traveling Physics with some

additional motivations, specifically with her Values of inspiring women. “For me personally,

because I am a woman in science and something I didn’t come to until I was in the tech

industry, and I saw how few women there were that I worked with, my personal goal was

to go out and inspire young women. So showing them and enabling them that they can do

science.” After a semester of education courses, she decided that she did not want a formal

educational career; however, she had built Social relationships with the other volunteers.

One of these relationships led to her and another volunteer opening up a business together.

Amber eventually left that business and went back to work in the tech industry.

Amber says that when she was back in the tech industry, “I was continually really trying

to get back into [Traveling Physics] because I realized that was the thing I loved in life. I

mean, even my coworkers would say, ‘You go do this thing on some weekends where you

drop in, and you do [Traveling Physics], and all you do is talk about [it]. Why don’t you go

get a job [there]?”’ Her love for Traveling Physics would be an Enhancement motivation,

but it was one that developed over time and played a role in her wanting to return and stay

involved. Not only was her love of Traveling Physics apparent to her, but her tech colleagues

noticed and commented on her enjoyment of the program, even to the point of encouraging

her to pursue the program professionally. She applied for multiple positions at Traveling

Physics and ultimately was hired as an assistant director.

In our interview, Amber reflected on how her past volunteering experiences impacted

her after graduating. While she learned some physics knowledge and skills by building

experiments, she also says that volunteering “enabled me to be okay to continue to fail. It

159

was okay to come in here and work on a project for an entire semester, and the project

doesn’t work, and that’s okay...Gosh, when [my business] imploded, and we weren’t making

any money and that was it, I was like, ‘I don’t know what to do,’ And I’m like, ‘Gosh,

what would [Traveling Physics director] do?’

It’s like, ‘Oh, you learn.’ You pick up the

pieces. You see what’s the thing you enjoy, and you use that as your guiding light to go

forward.” Volunteering helped Amber to gain life skills in overcoming failure, and she used

that Understanding later on in her career.

Amber’s career path was not a straight forward one. She changed jobs multiple times

and worked in multiple different fields. However, throughout her career, she was regularly

involved in Traveling Physics, which helped influence the career choices that she made.

Figure 5.1 is a visual representation of her career path.

160

Figure 5.1: This is a visual representation of Amber’s career path. She first joined Traveling
Physics as a physics undergraduate. After graduating, she worked as a lab coordinator
before transitioning to the tech industry. Later, she pursued a teaching degree, where she
also returned to Traveling Physics because she Valued inspiring younger people. After some
Social connections with another volunteer, she co-started a business. She later returned to
the tech industry, where she dreamt of returning to Traveling Physics in a full-time role.
Ultimately, she got a leadership position at Traveling Physics. A variation of this figure was
originally presented as a poster [138].

161

5.4.2 Mark

As an undergraduate, Mark was asked by the physics department to teach some of the

introductory physics labs as a teaching assistant. This was one of the early moments where

Mark learned that he enjoyed teaching. When he attended graduate school, the Enhancement

from his undergraduate teaching motivated him to get involved in the introductory labs and

working with the Traveling Physics coordinator. “[In undergrad], I knew how the lab ran, so

I helped tell the teaching assistants the next time what we were doing and I would help set

up the labs for everybody, started editing the manuals and realized, hey, I like this. So that’s

why when I saw that when I got to [graduate school] with [Traveling Physics director], I was

very willing to jump in and help and stay in that aspect of it.” In the introductory physics

labs, his responsibilities included managing the lab spaces, teaching some of the labs, and

leading the teaching assistants.

Mark’s motivations for volunteering were more than just feeling good. “I love physics,

I love the doing of it, but if I couldn’t share it with somebody in some fashion, I don’t

think I’d still be in it. That’s what I realized I liked, and I felt like I could do that if I was

with teaching assistants, the lab students, and, and then ultimately outreach...I feel like I’m

sharing it, not dispensing information.” A couple motivations overlap here. He Values the

sharing of physics versus the dispensing of knowledge. He also determined that in order for

him to stay in a physics Career, sharing physics with others was a necessary aspect for him.

Volunteering helped him to Understand how to learn and share physics. “It changed the way

I approach physics when I’m learning it. Because ...when I’m learning it, I’m trying to make

connections now...and help [others] see how it works. The concepts and the principles rather

than just here’s some problem solving skills to get you through and graduate. Because that

162

was more the drive of my undergraduate career. [Traveling Physics] was the flip. It was the

reverse.”

Mark also had Protective motivations from not feeling trusted to do work in research labs.

“In two research labs...they hardly gave me any responsibility or significant responsibility

or asked me to do any. I didn’t feel like I was helpful, part of a team or useful. Joining

[Traveling Physics], all of that switched. They were like, ‘hey, would you like to work on this

or what would you like to work on? OK, pursue it.’ And then they gave guidance, but you

got to explore with it and learn and try and maybe you came back and it was wrong and

they give you advice and you go work on it some more.” Even when he was taking other

coursework and labs as a student, Mark found the manuals and activities to be “cookbook”

and that his role was more about “going through the motions. You were just another cog in a

wheel...I’d never felt that in [Traveling Physics]. [I] always felt useful, appreciated, helpful.”

Traveling Physics provided Mark a place where he felt trusted, able to contribute, and able

to learn and apply his skills.

As a lecture demonstration specialist, he currently does demonstrations for physics

courses and sometimes teaches courses. He also facilitates teaching workshops with K-12

teachers and does public engagement with K-12 schools. Mark reflects on how Traveling

Physics impacted the Career path that he took. “I just want to emphasize again it did steer

me [and] had a huge influence on where I’m at now. The outreach, the teaching, the trying

to relate more to the public and the students rather than just going into research...I’m about

teaching first and foremost. Everything I do is about that now.” As a form of Enhancement,

Mark says “[Traveling Physics] made me who I am and I’m grateful for it.”

163

5.4.3 Claire

She was a neuroscience major who also studied sociology and science education. Since early

college, she had been interested in pursuing an education career. This motivation stemmed

from her Values of impacting kids and getting them interested in science. “I wanted to have

more of a significant like role in in kids lives. And I also wanted to incorporate my like love

of science and so science education really kind of just put all of those together.” She saw an

advertisement for After School Physics and joined because of her Career goals. “I’ve had

that interest in education for a while. Especially trying to combine education, like with my

own passion of science, I think I saw a poster for it and then I saw some people I already

knew, like TA’s and LA’s also attending, and I wanted to check it out and I made a lot of

cool friends with the group and some cool connections as well.” Here, Social was another

motivation for Claire. She already knew people in that particular program and then she

continued to make friends and connections through volunteering.

Claire describes her overall experience volunteering in After School Physics as

“overwhelmingly positive” and that “it was really heartwarming to play with the kids.”

These positive experiences are a form of Enhancement. Some of these positive feelings came

from her Social relationships with the other volunteers, noting that “It was really cool to

connect with other people that were interested in science education.” With a combination

of student volunteers of varying academic ranks, many of whom were physics majors, Claire

mentioned that “there was another girl that attended with me. Another undergrad. She

was in like astrophysics, I think, but we really connected over being the younger ladies in

[After School Physics]. I think we became really good friends.” Even though Claire was a

neuroscience major, she says that in her carpool rides to schools with the other volunteers

164

that they would have a mix of science-related conversations. “It was really cool to just have,

like, science discussions with people that can, like, keep up even if they’re not necessarily in

the same discipline. But I felt like it was a lot of support.”

As a current middle school science teacher, Claire notes that education was already the

Career path that she was going down. “[After School Physics] was one of the first moves

I made to solidify myself in sort of a science education pathway. And so as I was exposed

to pedagogy there, I was just starting to be exposed to it within my own courses. But it

solidified the age group I wanted to work with. The subject matter I wanted to work with.

And I think in some ways the demographic I wanted to work with as well.”

5.5 Discussion

Amber, Mark, and Claire represent three different types of educational career pathways.

Within the three interviews, we find all six categories to be present; however, for each

person, some motivational categories were more prominent than others and manifest in

different ways. For example, Amber and Claire had some form of Career motivation but in

different forms. Amber’s Career motivations began more with physics knowledge and skills

which then evolved into pursuing education, while Claire’s Career motivations in education

were solidified and narrowed down. Mark had prior teaching experiences that he found

positive, but it was those in combination with his Protective motivations from his physics

research lab experience that contributed to his pursut of a career in education. Table 5.2

serves as a brief summary of how each category was present across the three interviews and

which categories were most prominent for each person. The boldface highlights how that

motivation connects to their current work.

165

Table 5.2: Some of the most notable motivational categories for each interview. The bolding
connects that motivation to their current work.

Amber
Traveling Physics
Career : She initially saw the
program as a place to gain
and apply physics knowledge
to prepare for a career in
She
the tech industry.
transitioned into a career
of informal physics.

She wanted to
Values:
inspire young people. After
her experience in the tech
industry, she was more
motivated
inspire
young women through
Traveling Physics.

to

Mark
Traveling Physics
Protective: Program gave
him a feeling of trust and
agency, unlike other research
and physics experiences he
had.

Claire
After School Physics
Career : She joined because
the program already aligned
to her goals of becoming a
teacher. This ultimately
solidified the career that
she wanted.

Understanding: He learned
presentation skills and how
to connect with the people
he is sharing physics with.
Applies these skills in his
current job.

She already knew
Social :
some other volunteers and
then built new relationships
throughout her time in the
program.

Enhancement:
Her tech industry colleagues
were commenting on her love
of being involved with the
Traveling Physics program.
She applied for Traveling
Physics job.

Enhancement: Past positive
teaching experiences helped
motivate him to volunteer
in the program. He then
becomes grateful for how
it made him who he is
today.

Values: Wanted to play a
role in kids’ lives and to get
them interested in science.
Current job is a middle
school science teacher.

166

There are several limitations with this study. In addition to the participants self-reporting

their experiences, most of them are years removed from their time volunteering as students,

so recollection of memories should be taken into consideration. In addition, while the alumni

we discussed here ended up in education-related jobs, that is not the case for all alumni. We

are encouraged by how well the framework illustrated the nuances in people’s motivations,

and how those motivations evolved throughout their time in the program and into their

careers. Similar to other studies that adapted the volunteerism framework to their own

contexts [80, 81, 82, 83], we find that this framework appears to appropriate for the informal

physics context.

5.6 Future Work

Understanding people’s motivations for volunteering in informal physics programs can

provide further contextual information that can be used to further support volunteers. Next

steps include applying this volunteerism framework to our larger dataset, which includes a

variety of job trajectories, some of which are non-education focused. One particular note of

interest I have, which came from me conducting each interview, is the involvement public

engagement work after graduating. Some research participants continued to volunteer in

some type of informal space, but others did not. Some cited other commitments such as

work, spending time with family, etc. Others mentioned having a difficult time finding ways

to get involved in informal spaces, especially if they did not work at, or lived farther away

from, a university.

Most of the participants that I interviewed for this study also talked about how much

fun they had volunteering in their programs. It would be interesting to do a more specific

167

analysis on what “fun” is. For example, do volunteers think building experiments is fun? Is

it socializing with other volunteers? Engaging with kids? All of the above?

As a part of my data collection for this project, I did site visits to Traveling Physics

and After School Physics. I observed that having moments of downtime and group social

opportunities seemed to contribute to the sense of community among the volunteers. For

example, Traveling Physics hosted a group meeting at the beginning of their work day to

discuss their experiences with the previous event. After that meeting, volunteers went to

work on building activities for their next events. Throughout this build time, the personnel

would converse with each other about topics related and unrelated to the program. Traveling

Physics also had a designated time to have lunch together for each of their work days.

At lunch time, the personnel shared stories and jokes with each other, appearing to be

really close to each other. For After School Physics, I rode in the car with the group of

volunteers to one of the schools that they work with. In the car was me, a postdoc, a senior

graduate student, and first-year graduate student, and an undergraduate student. There

were some conversations about the activities that the program was about to do, but there

were also conversations between the postdoc and senior graduate student about job searches,

conversations between the two graduate students about graduate courses, and conversations

with the undergraduate about what life as a graduate student was like. On the car ride after

the event, there were conversations about what went well and poorly with the activities they

just led. There were also conversations about each of the volunteers’ hobbies, which included

hiking and board games. In that car ride, volunteers exchanged numbers and made plans for

having a future game night. There appeared to be strong social bonds being formed among

the personnel members in aspects of programs outside of their main events. I think this is

an area of informal physics programming that should be studied more closely.

168

Chapter 6. Discussion

In Chapter 3, I identified the important structures of informal physics programs, finding

that the personnel play a central role in the functionality of these programs. This laid the

foundation for Chapter 4, where I developed a framework of key components designed to help

informal physics practitioners reflect upon the critical aspects of their programs. Given the

importance of personnel, multiple of these components were directly related to the personnel.

While Chapters 3 and 4 looked at how personnel are important to informal physics programs,

Chapter 5 analyzed how informal physics programs are important to personnel, specifically

in motivating students’ career pursuits. Understanding the motivations of their student

volunteers can help program leaders consider how to better support their personnel.

My hope is that this work contributes to supporting informal physics program leaders, and

in turn, that also helps leaders to support their program personnel. I am both a researcher

and a practitioner. In this chapter, I reflect on my research in the context of my personal

role as an informal physics practitioner, specifically through the lens of my key components

framework. I then include some recommendations for researchers, practitioners, and physics

departments.

6.1 My Positionality

as

an

Informal Physics

Researcher and Practitioner

Similar to the other program leaders reported on in Chapter 3, my work as an informal

physics practitioner was done in addition to my main job responsibilities as a graduate

student. My experience as an informal physics practitioner helped me to ask pointed

169

questions and follow-up questions in interviews, as well as have some contextual insights

during data analysis. While I consider my practitioner experience to be an asset during

the research process, there are also opportunities for potential bias to occur when collecting

and interpreting data. For example, I can often relate to other program leaders when they

discuss challenges that I may have also experienced as a practitioner, and that might make

me sympathetic to their perspective.

Prior to being a physics education researcher,

I was actively involved in public

engagement. As an undergraduate, I was an intern at a traveling science program where

I was a part of a team of staff and undergraduate students who built hands-on science

experiments and brought them to K-12 schools and public events. While I had already

enjoyed education, I was not seriously considering it as a potential career path until I became

involved in public engagement. The years I had spent in this traveling science program helped

me gain an understanding of how to design activities, engage with audiences of varying ages

and backgrounds, and some of the logistical decisions that a program has to make. As a

part of my undergraduate experience in informal physics, I felt a strong sense of community

with other personnel as well as with our audiences. The excitement and joy that I got from

exploring the wonders of science with others helped me to view informal education as a

potential career option, which is one of the motivations for my work in Chapter 5.

Throughout my graduate student career, I continued doing public engagement work, but

this time as a program leader. Since 2020, I have been the public engagement coordinator

for a graduate student organization housed in the physics department. The organization’s

main goal is to promote and support diversity within the physical sciences. To do so, it

hosts a variety of events for university students in physical science programs as well as host

mentoring programs for graduate and undergraduate students. While most of the work in

170

the program focuses on supporting university students, my role was doing work with the

community beyond the university. My public engagement work included visiting primarily

elementary and middle schools as well as participating in open-house style events. Graduate

school is stressful. I treasure the opportunities I had in public engagement. Having kids tell

me that they thought the experiments I built were cool and that they wanted to make their

own has been really heartwarming. Getting others excited about physics has helped me to

also stay excited about physics.

6.2 Using the Key Components Framework to Reflect

on my Experiences as an Informal Physics

Program Leader

In Chapter 4, I developed a framework of key components that describes important aspects

of informal physics programs that program leaders have influence over. Future work includes

using this framework to develop tools that program leaders can use to evaluate their own

programs. The framework was originally presented in Figure 4.2 as a circle of all 12 key

components. The circle is meant to show that all 12 components are connected to each

other. However, all 12 components may not be of equal importance for each program.

For example, the personnel components may be more heavily weighted for a program that

prioritizes undergraduate involvement in community engagement. A different program may

prioritize supporting youth in pursuing studies in STEM, so the program may put more

weight on the audience and program design related components. While it is important to

focus on all 12 components, it is also important for tools and resources to be flexible and

171

malleable to each program’s unique situations and needs. At the moment, we are testing out

potential tool formats, such as reflection questions about each component. In this section, I

tested out this approach by reflecting upon my own experiences leading an informal physics

program.

As described in 4.9.2, I envisioned two ways that a program leader could use these

reflective questions. When I started writing this section, I had planned on going through

each component individually. After writing my reflections for the first couple of components,

I found myself reflecting on experiences that spanned multiple components. I found that since

I already had years of experience in this position, it was easier to reflect on experiences that

I found went well and those that were challenging first. From there, I identified the present

components and reflected on areas of improvement, which I present below.

6.2.1 What Went Great: Goals, Program Content and Design,

Physical and Digital Resources, Funding and Budget,

Assessment

I am listing Goals, Program Content and Design, Physical and Digital Resources, Funding

and Budget, and Assessment together here because I find them to be highly related in

my particular case. When I started in this position, the program only owned a few

demonstrations. Past public engagement activities usually borrowed supplies from the

physics department. One of my goals within the program was to show that science can be

done anywhere by anyone. My approach for doing so was building hands-on demonstrations

that are made out of common household items with the intention that students are inspired

to make their own science experiments at home cheaply.

172

The public engagement funding that I got was provided by the department. Every year,

I submitted a budget proposal for public engagement and I always got what I asked for

financially. I used these funds to buy supplies to build hands-on demonstrations that the

program can claim as its own. One example of an activity is a set of cartesian divers made

out of water bottles and ketchup packets (see Figure 6.1). Another example is exploring

forces with hairdryers and ping pong balls.

When I visited schools, I typically brought a set of demonstrations that I set up around the

classroom. These demonstrations typically covered general physics topics, though sometimes

teachers may request a topic theme (for example, forces) that would influence the activities

I brought. Students self-explore with the activities themselves. My volunteers and I would

move around the classroom conversing with students, ask probing questions, and explore

alongside the students. We would then wrap up the session by having students share out

what they learned and ask any questions they might have.

I used a combination of formal and informal assessment techniques in my program. After

a school visit, I ask the teachers to fill out a feedback form about the experience, which is

a more formal form of assessment. I have had some teachers tell me that they had students

say that they want to go home and make their own experiments, which was one of my goals.

I also rely on informal forms of assessment like observations and casual conversations with

students. While students were exploring with activities or building their own demonstrations,

I would casually talk with individual students and learn what they found interesting. I found

that especially with younger students, they had no problem saying if something was cool,

and especially if something was not cool. These types of observations and interactions helped

me to adapt and alter the activities that I used in following events. For example, I have an

activity where students explore how Slinky springs can be used to make interesting sounds.

173

I have witnessed students get excited and spend a lot of time exploring with the Slinkys,

therefore I make a point to bring that activity to most of my events. A demonstration that

I thought was really cool centered on exploring polarization and digital screens, shown in

Figure 6.2. I found this activity interesting, however, I found it to be one of the activities

that students interacted with the least, so I stopped running it.

174

Figure 6.1: These are two Cartesian divers made out of water bottles and packets of mustard
(can work with other condiment packets). The water bottles are filled with water with a
packet floating inside. The packets float to the top of the bottle (as seen in the bottle on the
right) because there are air bubbles inside the packet. As someone squeezes the bottle (as
shown on the bottle on the left), the packet begins to sink. This is because the air bubble
is getting compressed, taking up less volume, and thus becoming more dense. I have found
this to be a popular activity among students. They seem to enjoy seeing how hard they can
squeeze the bottle. They also often take the bottle around the room and show their friends,
teachers, and/or parents.

175

Figure 6.2: LCD screens, like the screen on this calculator, has a polarizing film on top
that blocks out some of the light on the screen, which is how we are able to read the
display. For this demonstration, I peeled off the polarizing film. This means that when the
calculator is on, no light from the screen is being blocked, making it difficult to read the
screen, appearing as if it is turned off. But if the person looking at the calculator is wearing
polarized sunglasses, the sunglasses block out the light in the same manner as the polarizing
film, allowing the wearer to be able to read the screen. I like this experiment, however, when
I took it to schools, students engaged with it considerably less than the other activities. As
a result, I stopped bringing it on school visits as often and built additional activities that
were similar to other demonstrations that students enjoyed.

176

Reflecting on these components, I find that I maximize my available budget to purchase

the necessary physical resources I need to build hands-on physics demos and activities. These

activities are the centerpiece of the public engagement events that I run. I designed these

activities with the goal of getting students interested in science and to explore science outside

of the classroom on their own accord. Based on my informal and formal assessments, it

appears that I am meeting this goal. That said, I use these forms of assessment to better the

activities and program design, such as upgrading or removing less engaging demonstrations.

6.2.2 What Went Well: Understanding Audience Motivations

and Needs, Support of Audience Engagement and Learning,

Relationship with Partners and Community

When I became the public engagement leader, I had been living in Michigan for less than

a year. I did not have much familiarity with schools in the area and had minimal direct

personal connections with community partners. To get in contact with schools, I relied on

mutual connections. I knew some professors in the College of Education and some other

program leaders who led teacher workshops. I asked them to share information about my

public engagement work with their teacher connections. This led to some connections with

local teachers.

In addition, the university has a science festival program that creates a

contact list of university scientists and programs that teachers can contact about visiting

their schools.

I have registered my public engagement work to be on that list, and each

year, I get about 4-5 teachers that contact me about visiting their classrooms. After I had

worked with a particular school or teacher, there were often times where I was invited back

the following year.

177

When a teacher emailed me about visiting their school, I typically scheduled a phone or

video meeting with them to discuss what made sense in terms of scheduling, format, topics,

number of students, etc. Average visits were about 45-minutes, but some teachers requested

that I visit multiple classes at the school. Some teachers did not have a preference on content,

so I brought whatever I wanted. Other teachers were more specific. For example, I had a

teacher request activities that were related to weather, chemical reactions, and matter since

those were the units that her students were studying at the time. I had another teacher who

was a long-term substitute teacher for elementary music who asked me to visit for the entire

school day to do science of sound demonstrations for all of their students. By meeting with

teachers ahead of time, we were able to set clear expectations of what the visit would look

like, what students would find interesting, and other logistics.

There were, however, some challenges, especially in the beginning of the COVID-19

pandemic, where I was doing virtual visits to schools. This varied in structure based on

health department recommendations and the policies of each school. For virtual visits, I

would join a class via a video conference call. Based on each particular situation, students

were each individually on the video call from their own residence, or students were socially

distanced in their classroom while my video was projected on a screen in a room. For

the virtual events, I led demonstrations and was able to have engaging back and forth

conversations with K-8 students. In the meetings I had with teachers prior to the visit, I

relied on teachers to understand what worked well in the virtual space. For example, the

approaches for getting students to unmute themselves on Zoom were different for middle

school students than they were for 2nd graders. Teachers were able to provide wonderful

insight on what kept the attention and interest of their particular students, which helped

me to develop activities that met the students’ needs and kept them engaged.

178

I found that I was particularly successful with engaging K-8 students. An age group

that I had more challenges with engaging was with high schoolers.

I did a virtual visit

to a set of high school classes, and I had a more difficult time getting students excited

and engaged. I did not quite meet my goal with that student population, I think partially

due to the activities and partially due to the awkwardness of the virtual visit structure.

Generally, I found in-person visits to be more engaging than virtual visits because students

could explore with the demonstrations and activities themselves rather than watching me

on a computer screen. When working with high school students, it was especially difficult to

get much engagement. For example, in trying to ask questions to the students, I was often

met with silence. Mid-visit with the high school class, I decided to shift away from doing

the demonstrations as I was observing that students were not that interested in it. I started

talking a bit more about what a career in physics looks like, thinking that may be of more

interest and relevance to high school students. I got a little more engagement, but by that

point, I had lost some confidence in myself and overall engagement was low.

After this event, I decided to advertise events with physics demonstrations to K-8 classes

rather than K-12. I did some advertising to high school classes about doing visits that were

more focused on careers in physics, with the idea that I could recruit some physics graduate

students to join me in high school classrooms and discuss the different types of physics fields

and careers. I promoted this through the university science festival like my other events, but

I was never contacted about this style of event. For more engaging events for high schoolers,

I would definitely need to rely more on high school teachers. In order to make connections

with high school teachers, I would have to be more deliberate in contacting them directly to

see if they would be interested in collaborating. Building those relationships takes time and

effort to do it right. Given my limited time and the success I was having with K-8 graders,

179

I decided to shift my focus away from working with high school students.

6.2.3 What Went Okay: Recruitment and Onboarding of

Personnel, Connection to the Institution

Since my public engagement work was through a student organization, I primarily recruited

university students to volunteer for events. My main method of recruitment was through

email. For each event, I would send an email to all the graduate students in the physics

department. The email would include event details like time, age group, and general

information about what volunteering responsibilities would include. For any one event, I

usually got about 1-2 people interested in volunteering out of a department of 200+ graduate

students. Sometimes I did not get any physics student volunteers.

Surprisingly, most of my volunteers ended up being from outside of the department. That

is primary because in addition to the public engagement work that I have discussed thus

far, I also created and hosted a podcast where I interviewed 90 graduate students about

their research, interests, hobbies, and how they ended up where they are today. The podcast

started off with me interviewing other MSU physics graduate students about their life stories

and how that connected to their research and graduate student experiences. Over time, I

began interviewing MSU students in other disciplines. I did not know most of these students

in other disciplines prior to interviewing them. After the interview, I would ask them if they

were interested in receiving emails about volunteering opportunities. Everytime I sent an

email to the department looking for volunteers, I also sent an identical email to students I

met through my podcast.

I found that having volunteers from other disciplines lead to some exciting collaborations.

180

I encouraged my volunteers to lean on their own interests and knowledge when facilitating

these informal learning spaces. For example, I got a request from a teacher about doing some

activities that dealt with chemical reactions. I asked my volunteers if they had any activities

that they wanted to create for this visit. I had a volunteer who wanted to lead an activity on

elephant toothpaste. It is a chemical reaction with dish soap, hydrogen peroxide, and yeast.

She chose it because she was a chemical engineering graduate student at the time, so talking

about chemical reactions was something that she was comfortable and knowledgeable with.

I have also had volunteers who want to engage and be role models for kids, but expressed

concerns because they did not have a physics or STEM background. Conversely, I also have

had volunteers who had the content background but were nervous about how to engage

with children. I communicated with my team collectively and individually so that we could

support each other as well as the students we were working with. For example, a local

elementary school requested activities that were about forces. As a physicist, I had some

ideas for activities, but many of my volunteers were experts in other fields. I encouraged

them to design something related to their interests or areas of study. One volunteer was a

medical student who, after meeting and discussing with me, was able to design a series of

water bottles of food coloring connected with straws such that when a bottle was compressed,

it pumped the fluid into the other chambers, much like a heart. The students loved this and

asked so many questions, to which she was able to answer the biology related questions and

I could help answer the physics related questions.

After a few years in my position as a program leader, I ran into a common issue

that other program leaders have: student volunteers graduate or become busy with other

responsibilities. I decided to stop hosting my podcast, which I was doing by myself, which

partially made it difficult for me to build connections with students in other disciplines,

181

which were most of my volunteers. The few physics volunteers that have worked with me

have told me that they enjoyed their volunteer experience, but getting people to volunteer in

the first place has been challenging. I tried other methods of recruitment besides email such

as talking to people directly or promoting public engagement efforts at orientation events

for first-year graduate students, but I did not notice vast improvements in recruitment.

The department has been supportive of my work in many ways. One year, I was awarded

the departmental outreach award. Faculty members have repeatedly reached out to me

saying that they appreciate my podcasting work, even going as far as encouraging prospective

graduate students to listen to it.

I have enjoyed the kind words from the department;

however, most of the verbal support has been for my podcasting and using it for recruiting

prospective graduate students, and there has been less promotion or acknowledgement of

the other public engagement work that I do. As a graduate student, I also did not ask

department faculty to volunteer in activities or ask faculty to encourage other students to

volunteer, primarily due to the awkwardness of that power dynamic.

The main challenge I had was that people were supportive of me being the one who

did public engagement work, but there was less support in helping me do that work. As

I reflect on this experience, I think I needed to do more on demonstrating the benefits of

volunteering, possibly sharing the testimonials of previous volunteers who stated that they

enjoyed their experience. I also think I should have had more direct conversations with the

department about promoting my other forms of public engagement.

182

6.2.4 What Was Challenging: Distribution of Roles and Tasks,

Continued Support of Personnel

Much like some of the program leaders mentioned in Chapter 3, I felt that much of the

tasks related to the public engagement work fell directly onto me. Tasks included writing

proposals to participate in larger public engagement events like the university science festival,

communicating with teachers and other community partners, purchasing supplies, planning

and building activities and demonstrations, recruiting volunteers, and event facilitation.

As discussed in 6.2.3, on occasion, I was able to get some assistance from volunteers in

creating their own activities and demonstrations, however, this primarily happened for the

virtual events, which were more presentation-focused, and typically took place when there

were stricter COVID-19 restrictions that made it difficult to meet in person and exchange

supplies.

For some events, I had about 3-4 volunteers, which was helpful because we were able

to all engage with the audience individually at the same time.

In addition, while some

volunteers were working with some audience members, I could also focus on other tasks, like

preparing for the next audience group that we were going to see, fixing demonstrations that

broke, or talking with teachers and parents. There were some events where I was the only

facilitator. That meant that if a demonstration broke, I had to pull it off to the side and

could not fix it until I returned home. It also meant that there were fewer opportunities to

have individual interactions with every audience member. There were some events, such as

visiting the music class described in 6.2.2, that I was facilitating by myself all day, which

meant that I could not take a break and there was no one to split tasks with. It also meant

that there was not anyone that I could reflect with about how the event was going and some

183

changes that could be made.

The Continued Support of Personnel component was also challenging for me. The public

engagement events were not on a consistent basis, for example, weekly like some other

programs are. That is because for any one school visit, there may be about a day’s worth

of commitment from me with teacher communication, travel, purchasing supplies, building

activities, and the visit itself. As a graduate student, I have limited time and resources to

commit to that voluntarily on a frequent basis. As a result, the most events I facilitated in

a year was nine. It was difficult enough to recruit volunteers for a single event, let alone the

planning process. Therefore, it was also difficult to build much of a community among the

volunteers in this informal space. I had volunteers who said they enjoyed volunteering, but

graduate student schedules are so busy that even the ones who enjoyed their time were only

available to volunteer for a couple of events.

I had a graduate student tell me once that they were interested in volunteering, but

were nervous because they did not have a lot of experience working with kids. They ended

up volunteering for an event and they did great. Before each event starts, I like to do a

short prep meeting with volunteers about what to expect and some tips on working with

the audiences, but this volunteer’s comment got me thinking about how maybe I could host

some form of workshop session in the department about public engagement work to help

train, and potentially recruit, volunteers. By the time that I thought about this, I was part

way through my final year in this position and had very little time to commit to hosting

such a workshop.

Reflecting back on my time as a program leader, if I were to do it all over again, I would

host a public engagement workshop, at least once to see how it went. It could be a way

to recruit some volunteers and build some confidence among some students. Given that

184

university students already cite time constraints for volunteering for events, I am hesitant

on asking them to participate in an additional workshop. On the other hand, it may pique

their interest and make them more willing to dedicate their time.

6.3 Considering Different Framework Representations

The Key Components framework is a complex framework due to the detail and

interconnectedness of the components.

To assist researchers and practitioners in

understanding the complex nature of these components, it can be helpful to provide multiple

representations as each representation may resonate differently for each person. For example,

an alternative representation is the use of metaphors, as I discuss here.

Metaphors can be a useful tool to describe complex systems in a more relatable fashion

[139]. Education studies have found metaphors to serve as tools in understanding student’s

perceptions and conceptual understanding of a situation [140, 141]. The physics community

has a history of using metaphors to describe abstract physics concepts in a more relatable

form. Some examples include comparing the flow of electricity to water flowing through

pipes, describing the history of the universe through clocks and calendars, and Maxwell’s

demon [142, 143, 144]. No metaphor is perfect, but they can still be useful by representing

frameworks in different ways that may resonate with different people. Below, I provide two

metaphor examples to illustrate our key components framework.

6.3.1 Plant Metaphor

Informal physics programs are gardens of opportunity, cared for by the program leaders. A

small, local garden differs from a large botanical garden in the amount of foot traffic, number

185

of plants, types of plants, and layout. The gardener has to decide if they are going to include

plants that are okay to touch, like in a children’s garden, or if they are going to include

only plants that one just observes. Perhaps the gardener includes both. This is similar to

a program leader deciding if physics activities are going to be hands-on or structured as

presentations.

Plants need water. Water can come from rainfall, but it can also come from irrigation.

Can the garden as it currently functions thrive from just one of those water sources, or does

it need multiple sources of water? Does it need a lot or a little water? If the gardener has

a desert botanical garden of cacti and succulent plants, the garden can perhaps thrive even

if some water sources have dried up. Other gardeners may need to seek out other water

sources to keep their garden thriving. These are similar decisions that program leaders have

to consider with funding. Program leaders have to consider how much funding they need

and what they will do if they lose a funding source.

What does the community want to see out of a garden? Are there particular plants they

want to see? Are there opportunities for community members to plant their own plants

in this garden? In this garden, is there one gardener, or are there multiple gardeners who

manage the space. Are the gardeners paid to manage the garden, or is it all volunteer work?

Program leaders also have to consider if their activities are even of interest to the community

and if the community has opportunities to guide the direction of the program. In addition,

program leaders need to consider their personnel, their responsibilities, and support systems.

6.3.2 Car Metaphor

All cars have the same general mechanics: they have wheels, a steering wheel, seats, and

require some form of fuel like gasoline or electricity to run. However, different types of

186

programs are used for different purposes. Perhaps the goal is to engage more with nature, so

the car is designed with off-roading in mind. That requires special attention to aspects like

tires and suspension. Tires are still important to an average street car, but not in the same

capacity. Some vehicles are designed to handle a lot of people while other cars are much

smaller. The bigger the car, the more fuel that it consumes. The driver of the vehicle can

decide where the car goes, but drivers may take varying levels in input and feedback from

passengers and mechanics to decide their next step.

Informal physics program leaders are the drivers of their program. They make decisions

on the direction of the program and which aspects of the program may need upgrades or

maintenance. Some programs have smaller numbers of personnel, audiences, and partners

while other programs have large numbers of those. The larger the program, usually the more

funding it needs in order to operate. Personnel, audiences, and partners can contribute to

maintaining the program and provide feedback on where it is moving to.

6.4 Recommendations

The takeaways of this dissertation are that personnel are crucial to the functionality of

informal physics programs, and that these programs also have positive impacts on their

personnel. However, a common challenge for program leaders is that they are overwhelmed

with responsibilities and tasks, often lacking the support and tools needed to improve and

meet their goals. Chapter 4 is a step in the direction of creating those supports and tools

with the key components framework, but there is still more that needs to be done.

As discussed throughout this dissertation, there are many types of people involved in

the informal physics space, directly and indirectly. There are program leaders, volunteers,

187

audiences,

institutional administrators, community partners, education researchers, and

more. Each of these groups have varying levels of impact on programs, and programs can

have some level of impact on each group. Programs can benefit institutions with recruitment

of students and help faculty to reach broader impact goals. Programs can partner with

local businesses in hosting events that then ultimately bring audiences to those businesses.

Program leaders want to do more assessment and evaluation with their programs, but often

lack the time, tools, and experience to do so, which means that there are potential partnership

opportunities for education researchers to carry out research.

With any sort of working relationship or partnership, it is important for everyone to have

clear expectations of what they are wanting and what they are able to provide. This takes

communication and an awareness of the voice and responsibility that each person has.

I

have found the Degree of Collaboration Abacus Tool [145] to be helpful in thinking about

the distribution of responsibilities of each person in my public engagement, research, and

other collaborative efforts. In their paper about this tool, Doberneck and Dann describe it

as “a visual or metaphorical tool used to account for the valence of the relationship between

two collaborating entitiescommunity and university partners.

In other words, the abacus

tool can visually represent whether, during each step of a shared project, the community or

university partner has more voice in project decision-making or whether both partners share

the work equally” [145].

Imagine an abacus, a ladder-like frame with horizontal rungs that beads are able to

move left and right on. In this metaphor, one side of the abacus represents the voice and

responsibility of one partner, and the other side represents the other partner. Each rung

represents the steps, tasks, or decisions that need to be made on the project that the partners

are working together on. The placement of the beads on each rung represent where the level

188

of voice and power exists. Figure 6.3 provides an example of the abacus tool being used for

outlining the responsibilities for when I partner with school teachers in organizing school

visits [145].

Figure 6.3: This is an example of the abacus tool being used for when I work with a teacher
for coordinating a school visit. For steps like scheduling a date and time, both of us have
about an equal voice as we both have our own time restrictions and availability. When
visiting a school, the teacher I am partnering with is typically the one who coordinates with
locating a physical space, coordinating schedules with other teachers, and gets approval from
school administrators for hosting the visit. I, on the other hand, am responsible for recruiting
other graduate student volunteers and transporting them to the school. This figure models
just some of the task distributions among me and my teacher partner. A similar model can
be used for task distribution among me and program volunteers. This figure was modeled
after the figures published in Doberneck and Dann [145].

A healthy partnership does not necessarily mean that every rung has beads in the middle.

This tool also does not have to be limited to just two partners. Variations of this concept

can be used when multiple groups are involved. There are some steps where it is more

appropriate for a particular partner to have more of a voice. I encourage program leaders to

189

use an abacus-like tool when reflecting on their programs for a couple of reasons. Listing out

and documenting all the decisions and necessary responsibilities helps put into perspective

the volume of considerations that exist in running a program. Using the key components

framework can help to identify the important tasks that are, or should be, done. The

placement of beads can then also put into perspective just how many of these decisions and

responsibilities fall onto the program leaders. Reflecting on this exercise may help program

leaders and other individuals like volunteers, administrators, researchers, and community

members identify areas of the program where responsibility can be adjusted in a more

equitable manner.

Informal physics programs have many positive impacts on physicists, physics students,

and local communities. My work here has identified many of the key structures of these

programs, one of which is the importance of the personnel. Program leaders often experience

challenges with running their programs and wish there was a tool or resource that they

could use to assess their program. The key components framework lays the foundation

for developing such tools. Further collaboration between practitioners, researchers, and

community partners can build off of this work to further support and improve these programs.

190

BIBLIOGRAPHY

[1] P. Bell, B. Lewenstein, A. W. Shouse, M. A. Feder, L. Science, S. Education,
D. Behavioral, and S. Sciences, Learning Science in Informal Environments. National
Academies Press, 2009. Publication Title: Learning Science in Informal Environments.

[2] M. Fenichel and H. A. Schweingruber, Surrounded by science: Learning science in
informal environments. 2010. Publication Title: Surrounded by Science: Learning
Science in Informal Environments.

[3] E. Hargittai, T. F¨uchslin, and M. S. Sch¨afer, “How Do Young Adults Engage With
Science and Research on Social Media? Some Preliminary Findings and an Agenda for
Future Research,” Social Media + Society, vol. 4, pp. 1–10, July 2018.

[4] D. Izadi, J. Willison, N. Finkelstein, C. Fracchiolla, and K. Hinko, “Towards mapping
the landscape of informal physics educational activities,” Physical Review Physics
Education Research, vol. 18, p. 020145, Dec. 2022.

[5] J. E. Bartley, L. M. Mayhew, N. D. Finkelstein, M. Sabella, C. Henderson, and
C. Singh, “Promoting Childrens Understanding And Interest In Science Through
Informal Science Education,” (Ann Arbor, MI), pp. 93–96, 2009.

[6] R. Wulf, K. Hinko, and N. Finkelstein, “Promoting childrens agency and
communication skills in an informal science program,” in Proceedings of PER
Conference, (Philadelphia, PA), pp. 430–433, 2013.

[7] R. Wulf, L. M. Mayhew, N. D. Finkelstein, C. Singh, M. Sabella, and S. Rebello,
“Impact of Informal Science Education on Childrens Attitudes About Science,” in
Proceedings of PER Conference, (Portland, Oregon), pp. 337–340, 2010.

[8] K. P. Dabney, R. H. Tai, J. T. Almarode, J. L. Miller-Friedmann, G. Sonnert, P. M.
Sadler, and Z. Hazari, “Out-of-School Time Science Activities and Their Association
with Career Interest in STEM,” International Journal of Science Education, Part B,
vol. 2, pp. 63–79, Mar. 2012.

[9] R. Dou, Z. Hazari, K. Dabney, G. Sonnert, and P. Sadler, “Early informal STEM
experiences and STEM identity: The importance of talking science,” Science
Education, vol. 103, pp. 623–637, May 2019.

[10] J. B. Ayers and C. O. Price, “Children’s Attitudes Toward Science,” School Science

and Mathematics, vol. 75, pp. 311–318, Apr. 1975.

[11] M. M. Atwater, J. Wiggins, and C. M. Gardner, “A study of urban middle school
students with high and low attitudes toward science,” Journal of Research in Science
Teaching, vol. 32, pp. 665–677, Aug. 1995.

191

[12] Z. Zacharia and A. C. Barton, “Urban middleschool students’ attitudes toward a

defined science,” Science Education, vol. 88, pp. 197–222, Mar. 2004.

[13] M. W. McLaughlin, Urban sanctuaries : neighborhood organizations in the lives and
futures of inner-city youth. San Francisco: Jossey-Bass Publishers, first edition. ed.,
1994.

[14] J. Kahne, J. Nagaoka, A. Brown, J. O’Brien, T. Quinn, and K. Thiede, “Assessing
After-School Programs as Contexts for Youth Development,” Youth & Society, vol. 32,
no. 4, pp. 421–446, 2001.

[15] E. Jensen and N. Buckley, “The Role of University Student Volunteers in Festival-
based Public Engagement,” National Co-ordinating Centre for Public Engagement,
p. 44, 2011.

[16] Y.-T. Pan, K.-K. Yang, Z.-R. Hong, and H.-S. Lin, “The Effect of Interest and
Engagement in Learning Science on Adults Scientific Competency and Environmental
Action,” EURASIA Journal of Mathematics, Science and Technology Education,
vol. 14, Aug. 2018.

[17] J. H. Falk and L. D. Dierking, Learning in Museums. Rowman & Littlefield, 2 ed.,

2018.

[18] G. Thomas and J. Durant, “Why should we promote the public understanding of
science?,” in Science in Public: Communication Culture and Credibility, pp. 1–14,
1 ed., 1987.

[19] “MSU Mission Statement | Board of Trustees | Michigan State University.”

[20] “Broader Impacts | NSF - National Science Foundation.”

[21] R. A. Salmon, R. K. Priestley, and J. Goven, “The reflexive scientist: an approach
to transforming public engagement,” Journal of Environmental Studies and Sciences,
vol. 7, pp. 53–68, Mar. 2017.

[22] L. Keith and G. W. Kerr, “Levelling the playing field: lessons from sport on re-framing
science engagement as a benefit to the individual,” Journal of Science Communication,
vol. 21, p. A03, June 2022.

[23] P. T. Robbins, “The reflexive engineer: perceptions of integrated development,”

Journal of International Development, vol. 19, pp. 99–110, Jan. 2007.

[24] J. D. Adams and P. Gupta, “Informal science institutions and learning to teach: An
examination of identity, agency, and affordances,” Journal of Research in Science
Teaching, vol. 54, pp. 121–138, Jan. 2017.

192

[25] K. A. Hinko, P. Madigan, E. Miller, and N. D. Finkelstein, “Characterizing pedagogical
practices of university physics students in informal learning environments,” Physical
Review Physics Education Research, vol. 12, p. 010111, Feb. 2016.

[26] M. B. Bennett, K. A. Hinko, B. Fiedler, and N. D. Finkelstein, “The Effect of Explicit
Preparation in Pedagogical Modes for Informal Physics Educators,” in Proceedings of
PER Conference, (Cincinnati, OH), pp. 52–55, Jan. 2018.

[27] M. B. Bennett, B. L. Fiedler, and N. D. Finkelstein, “What factors influence
pedagogical methods in informal learning spaces?,” in Proceedings of PER Conference,
(Provo, UT), Jan. 2020.

[28] M. B. Bennett, B. Fiedler, and N. D. Finkelstein, “Refining a model for understanding
and characterizing instructor pedagogy in informal physics learning environments,”
Physical Review Physics Education Research, vol. 16, p. 020137, Nov. 2020.

[29] “Bachelors Degrees in Physics and STEM Earned by Women.”

[30] “Degrees Earned by Underrepresented Minorities in Physics.”

[31] C. Fracchiolla, “Scientists shouldn’t have to choose between public outreach and

research,” Science, vol. 380, no. 6646, 2023.

[32] J. Randolph, J. Perry, J. P. Donaldson, C. Rethman, and T. Erukhimova, “Female
physics students gain from facilitating informal physics programs,” Physical Review
Physics Education Research, vol. 18, p. 020123, Oct. 2022.

[33] C. Rethman, J. Perry, J. P. Donaldson, D. Choi, and T. Erukhimova, “Impact of
informal physics programs on university student development: Creating a physicist,”
Physical Review Physics Education Research, vol. 17, p. 020110, Aug. 2021.

[34] M. B. Bennett, C. Fracchiola, D. B. Harlow, and K. Rosa, “Informal Learning in
Physics,” in The International Handbook of Physics Education Research: Teaching
Physics (M. F. Ta¸sar and P. R. L. Heron, eds.), pp. 12–1–12–28, AIP Publishing
LLCMelville, New York, Mar. 2023.

[35] H. Eshach, “Bridging In-school and Out-of-school Learning: Formal, Non-Formal, and
Informal Education,” Journal of Science Education and Technology, vol. 16, pp. 171–
190, Apr. 2007.

[36] T. Gascoigne, B. Schiele, J. Leach, M. Riedlinger, B. V. Lewenstein, L. Massarani,
and P. Broks, eds., Communicating Science: A Global Perspective. ANU Press, 1 ed.,
Sept. 2020.

193

[37] D. M. Doberneck, C. R. Glass, and J. Schweitzer, “From Rhetoric to Reality: A
Typology of Publically Engaged Scholarship,” Journal of Higher Education Outreach
and Engagement, vol. 14, no. 4, pp. 5–35, 2010.

[38] J. Saltmarsh and M. Hartley, eds., To Serve a Larger Purpose : Engagement for
Democracy and the Transformation of Higher Education. Philadelphia, PA: Temple
University Press, 2011.

[39] H. L. Ainsworth and S. E. Eaton, “Formal, non-formal and informal learning in the
sciences,” tech. rep., Onate Press and Eaton International Consulting (EIC) Inc., 2010.

[40] “APS Public Engagement.”

[41] J. Stilgoe, S. J. Lock, and J. Wilsdon, “Why should we promote public engagement

with science?,” Public Understanding of Science, vol. 23, pp. 4–15, Jan. 2014.

[42] S. Owens, “Engaging the Public: Information and Deliberation in Environmental
Policy,” Environment and Planning A: Economy and Space, vol. 32, pp. 1141–1148,
July 2000.

[43] J. Lezaun and L. Soneryd, “Consulting citizens: technologies of elicitation and the
mobility of publics,” Public Understanding of Science, vol. 16, pp. 279–297, July 2007.

[44] J. Barnett, K. Burningham, G. Walker, and N. Cass, “Imagined publics and
engagement around renewable energy technologies in the UK,” Public Understanding
of Science, vol. 21, pp. 36–50, Jan. 2012.

[45] B. Lewenstein, “Public Engagement,” Jan. 2016.

[46] J. Willison, D. Izadi, I. Ward, K. A. Hinko, and C. Fracchiolla, “Challenges in study
design for characterizing the informal physics landscape,” in Proceedings of PER
Conference, (Provo, UT), pp. 651–656, 2019.

[47] APS FOEP, “FOEP Survey.”

[48] S. R. Singer, N. R. Nielsen, and H. A. Schweingruber, Discipline-Based Education
Research: Understanding and Improving Learning in Undergraduate Science and
Engineering. 2012. Publication Title: National Research Council.

[49] “The History of AAPT.”

[50] K. Cummings, H. Close, and N. Finkelstein, “A Developmental History of Physics

Education Research *,” National Academies, pp. 1–24, 2011.

[51] D. E. Meltzer and V. K. Otero, “A brief history of physics education in the United

States,” American Journal of Physics, vol. 83, no. 5, pp. 447–458, 2015.

194

[52] S. Kanim and X. Cid, “Demographics of physics education research,” Physical Review

Physics Education Research, vol. 16, p. 020106, July 2020.

[53] J. L. Docktor and J. P. Mestre, “Synthesis of discipline-based education research
in physics,” Physical Review Special Topics - Physics Education Research, vol. 10,
p. 020119, Sept. 2014.

[54] J. L. Alzen, L. S. Langdon, and V. K. Otero, “A logistic regression investigation of the
relationship between the Learning Assistant model and failure rates in introductory
STEM courses,” vol. 1, pp. 1–12, 2018. ISBN: 4059401801521 Publisher: International
Journal of STEM Education.

[55] J.-S. S. White, B. V. Dusen, and E. A. Roualdes, “The impacts of learning assistants on
student learning of physics,” in 2016 PERC Proceedings, (Sacramento, CA), pp. 384–
387, 2016.

[56] A. V. Knaub, K. T. Foote, C. Henderson, M. Dancy, and R. J. Beichner, “Get a room:
the role of classroom space in sustained implementation of studio style instruction,”
International Journal of STEM Education, 2016. Publisher: International Journal of
STEM Education.

[57] R. J. Beichner and J. M. Saul, “Introduction to the SCALE-UP ( Student-Centered
Activities for Large Enrollment Undergraduate Programs ) Project,” Proceedings of
the International School of Physics, no. July, pp. 1–17, 2003.

[58] “Informal Physics Education Research.”

[59] B. Prefontaine, C. Fracchiolla, M. Vasquez, and K. A. Hinko, “Intense Outreach:
Identities,” in Proceedings of PER

Experiences Shifting University Students
Conference, (Washington, DC), Jan. 2019.

[60] C. Fracchiolla, B. Prefontaine, and K. Hinko, “Community of practice approach
for understanding identity development within informal physics programs,” Physical
Review Physics Education Research, vol. 16, no. 2, p. 20115, 2020. Publisher: American
Physical Society.

[61] C. Fracchiolla, B. Prefontaine, M. Vasquez, and K. Hinko, “Is Participation in Public
Engagement an Integral Part of Shaping Physics Students Identity?,” in Research and
Innovation in Physics Education: Two Sides of the Same Coin (J. Guisasola and
K. Zuza, eds.), pp. 225–238, Cham: Springer International Publishing, 2020. Series
Title: Challenges in Physics Education.

[62] B. Prefontaine, C. Mullen, J. J. Gven, C. Rispler, C. Rethman, S. D. Bergin, K. Hinko,
and C. Fracchiolla, “Informal physics programs as communities of practice: How can

195

programs support university students identities?,” Physical Review Physics Education
Research, vol. 17, no. 2, p. 020134, 2021.

[63] C. Fracchiolla, S. Hyater-Adams, N. Finkelstein, and K. A. Hinko, “University physics
students motivations and experiences in informal physics programs,” in Proceedings of
PER Conference, (Sacramento, CA), pp. 124–127, 2016.

[64] P. R. Lawrence and J. W. Lorsch, Developing Organizations: Diagnosis and Action.
Organizational Development, Addison-Wesley Publishing Company, Inc., 1969.

[65] E. P. Learned and A. T. Sproat, Organizational Theory and Policy: Notes for Analysis.

Richard D. Irwin, Inc, 1966.

[66] G. R. Jones, Organizational theory, design, and change. Upper Saddle River, NJ:

Pearson, 7th ed ed., 2013. OCLC: ocn748578122.

[67] D. Jaffee, Organization Theory: Tension and Change. Boston, MA: McGraw Hill,

2001.

[68] A. Ilhan, “The Conceptual Framework Of Organizational Theory Analysis: An
Organizational Level Assessment,” Globus An International Journal of Management
and IT, vol. 12, pp. 81–87, Dec. 2020.

[69] H. K. Anheier, Nonprofit Organizations: Theory, Management, Policy. London:

Routledge, 2005.

[70] E. A. Tay¸sir and N. K. Tay¸sir, “Measuring Effectiveness in Nonprofit Organizations:
An Integration Effort,” Journal of Transnational Management, vol. 17, no. 3, pp. 220–
235, 2012.

[71] A. Vernis, M. Iglesias, B. Sanz, and A. Saz-Carranza, Nonprofit organizations :
challenges and collaboration. Basingstoke England ; New York: Palgrave Macmillan,
2006.

[72] C. Fracchiolla, N. Finkelstein, and K. Hinko, “Characterizing models of informal
physics programs,” in Proceedings of PER Conference, (Washington, DC), pp. 1–4,
2018.

[73] D. Izadi, J. Willison, K. A. Hinko, and C. Fracchiolla, “Developing an organizational
framework for informal physics programs,” in Proceedings of PER Conference, (Provo,
UT), 2020.

[74] “Demographics | AmeriCorps,” Jan. 2023.

[75] U. C. Bureau, “At Height of Pandemic, More Than Half of People Age 16 and Over

Helped Neighbors, 23% Formally Volunteered,” 2023. Section: Government.

196

[76] J. Wilson and M. Musick, “The Effects of Volunteering on the Volunteer,” Law and

Contemporary Problems, vol. 62, no. 4, p. 141, 1999.

[77] K. Dempsey-Brench and A. Shantz, “Skills-based volunteering: A systematic literature
review of the intersection of skills and employee volunteering,” Human Resource
Management Review, vol. 32, p. 100874, Dec. 2022.

[78] E. G. Clary and M. Snyder, “Understanding and Assessing the Motivations of
Volunteers: A Functional Approach,” Journal of Personality and Social Psychology,
vol. 74, no. 6, pp. 1516–1530, 1998.

[79] E. G. Clary and M. Snyder, “The Motivations to Volunteer: Theoretical and Practical
Considerations,” Current Directions in Psychological Science, vol. 8, pp. 156–159, Oct.
1999.

[80] D. R. Wright, L. G. Underhill, M. Keene, and A. T. Knight, “Understanding the
Motivations and Satisfactions of Volunteers to Improve the Effectiveness of Citizen
Science Programs,” Society & Natural Resources, vol. 28, pp. 1013–1029, Sept. 2015.

[81] M. C. Domroese and E. A. Johnson, “Why watch bees? Motivations of citizen science
volunteers in the Great Pollinator Project,” Biological Conservation, vol. 208, pp. 40–
47, Apr. 2017.

[82] B. Bruyere and S. Rappe, “Identifying the motivations of environmental volunteers,”
Journal of Environmental Planning and Management, vol. 50, pp. 503–516, July 2007.

[83] S. K. Jacobs, J. S. Carlton, and M. C. Monroe, “Motivation and Satisfaction of
Volunteers at a Florida Natural Resource Agency,” Journal of Park and Recreation
Administration, vol. 30, no. 1, pp. 51–67, 2012.

[84] B. Stanley and K. Hinko, “Informal physics volunteers: Motivations from student to

career in education,” 2023.

[85] L. Hamilton and C. Corbett-Whittier, Using Case Study in Education Research. SAGE

Publications Ltd, 2013.

[86] J. Elliott and D. Lukes, “Epistemology As Ethics In Research And Policy: The Use of
Case Studies,” Journal of Philosophy of Education, vol. 42, no. S1, pp. 87–119, 2008.

[87] B. Flyvbjerg, “Five misunderstandings about case-study research,” Qualitative

Inquiry, vol. 12, no. 2, pp. 219–245, 2006. eprint: 1304.1186.

[88] A. Oancea and R. Pring, “The Importance of Being Thorough: On Systematic
Accumulations of What Works in Education Research,” Journal of Philosophy of
Education, vol. 42, pp. 15–39, Aug. 2008.

197

[89] R. Yin, “Designing Case Studies,” in Case Study Research: Design and Methods,

pp. 24–65, 2009.

[90] R. E. Stake, The Art of Case Study Research. SAGE Publications Ltd, 1995.

[91] B. Yazan, “Three Approaches to Case Study Methods in Education: Yin, Merriam,

and Stake,” The Qualitative Report, Feb. 2015.

[92] L. Bartlett and F. Vavrus, Rethinking Case Study Research : A Comparative Approach.

Taylor & Francis Group, 1 ed., 2016.

[93] P. Sedgwick, “Convenience sampling,” BMJ, vol. 347, pp. f6304–f6304, Oct. 2013.

[94] R. A. Peterson, “On the Use of College Students in Social Science Research: Insights
from a Second-Order Meta-analysis,” Journal of Consumer Research, vol. 28, no. 3,
pp. 450–461, 2001.

[95] R. A. Peterson and D. R. Merunka, “Convenience samples of college students and
research reproducibility,” Journal of Business Research, vol. 67, pp. 1035–1041, May
2014.

[96] I. Simonson, Z. Carmon, R. Dhar, A. Drolet, and S. M. Nowlis, “Consumer Research:
In Search of Identity,” Annual Review of Psychology, vol. 52, pp. 249–275, Feb. 2001.

[97] C. Parker, S. Scott, and A. Geddes, “Snowball Sampling,” SAGE Publications Ltd,

2019.

[98] D. D. Heckathorn, “Respondent-Driven Sampling: A New Approach to the Study of

Hidden Populations,” Social Problems, vol. 44, no. 2, pp. 174–199, 1997.

[99] M. Naderifar, H. Goli, and F. Ghaljaie, “Snowball Sampling: A Purposeful Method
of Sampling in Qualitative Research,” Strides in Development of Medical Education,
vol. 14, Sept. 2017.

[100] B. Stanley, D. Izadi, and K. A. Hinko, “Perspectives on informal programs: How site
visits can help us learn more,” in Proceedings of PER Conference, (Virtual), pp. 503–
508, 2020.

[101] A. Traxler and E. Brewe, “Equity investigation of attitudinal shifts in introductory

physics,” vol. 020132, pp. 1–7, 2015.

[102] C. Lusthaus, M.-H. Adrien, G. Anderson, F. Carden, and G. P. Montalvan,
Organizational assessment : a framework for improving performance. Washington,
D.C.: Inter-American Development Bank, 2002.

198

[103] V. K. Otero, D. B. Harlow, and D. B. Harlowe, “Getting Started in Qualitative Physics

Education Research,” Getting Started in PER, vol. 2, pp. 1–66, 2009.

[104] N. Sangasubana, “How to conduct ethnographic research,” Qualitative Report, vol. 16,

no. 2, pp. 567–573, 2011.

[105] M. Genzuk, “A synthesis of ethnographic research,” Occasional Papers Series. Center
for Multilingual, Multicultural Research (Eds.). Center for Multilingual, Multicultural
Research, Rossier School of Education, University of Southern California. Los Angeles,
pp. 1–10, 2003.

[106] B. Stanley, D. Izadi, C. Fracchiolla, and K. Hinko, “Central role of personnel in informal
physics programming,” Physical Review Physics Education Research, vol. 19, no. 2,
2023.

[107] A. Krishnamurthi and L. J. Rennie, “Informal Science Learning and Education:

Definition and Goals,” After School Alliance, pp. 1–10, 2012.

[108] P. Tamir, “Factors Associated with the Relationship between Formal, Informal, and
Nonformal Science Learning,” The Journal of Environmental Education, vol. 22,
pp. 34–42, Jan. 1991.

[109] B. Butler, “Would Another Name Make a Difference?,” 1996. Publication Title: APS

Forum on Education Newsletter.

[110] J. H. Falk, S. Randol, and L. D. Dierking, “Mapping the informal science education
landscape: An exploratory study,” Public Understanding of Science, vol. 21, pp. 865–
874, Oct. 2012.

[111] E. Andrews, A. Weaver, D. Hanley, J. Shamatha, and G. Melton, “Scientists and Public
Outreach: Participation, Motivations, and Impediments,” Journal of Geoscience
Education, vol. 53, pp. 281–293, May 2005.

[112] S. Allen, P. B. Campbell, L. D. Dierking, B. N. Flagg, C. Garibay, R. Korn,
G. Silverstein, and D. A. Ucko, “Framework for Evaluating Impacts of Informal Science
Education Projects Report from a National Science Foundation Workshop The Division
of Research on Learning in Formal and Informal Settings (DRL),” no. February, 2008.

[113] S. Allen and K. Peterman, “Evaluating Informal STEM Education:

Issues and

Challenges in Context,” New Directions for Evaluation, 2019.

[114] C. Fracchiolla, B. Prefontaine, M. Vasquez, and K. Hinko, “Is Participation in Public
Engagement an Integral Part of Shaping Physics Students Identity?,” in Research and
Innovation in Physics Education: Two Sides of the Same Coin (J. Guisasola and

199

K. Zuza, eds.), pp. 225–238, Cham: Springer International Publishing, 2020. Series
Title: Challenges in Physics Education.

[115] A. A. Bergerson, B. K. Hotchkins, and C. Furse, “Outreach and Identity Development:
New Perspectives on College Student Persistence,” Journal of College Student
Retention: Research, Theory & Practice, vol. 16, pp. 165–185, Aug. 2014.

[116] “About SySTEM 2020.”

[117] S. B. Merriam and E. J. Tisdell, Qualitative Research: A Guide to Design and

Implementation (4th ed.). San Francisco, CA: Jossey-Bass Publishers, 2015.

[118] M. B. Bennett, K. A. Hinko, and D. Izadi, “Challenges and opportunities for informal
physics learning in the COVID era,” Physical Review Physics Education Research,
vol. 17, p. 023102, July 2021.

[119] “TranscribeMe.”

[120] B. Stanley and K. A. Hinko, “Using a volunteerism framework to understand the
motivations of university students who facilitate informal physics programs,” in 2023
Physics Education Research Conference Proceedings, (Sacramento, CA), pp. 332–337,
Oct. 2023.

[121] A. Robertson and W. T. Hairston, “Observing whiteness in introductory physics: A
case study,” Physical Review Physics Education Research, vol. 18, p. 010119, Mar.
2022.

[122] K. Rosa and F. M. Mensah, “Educational pathways of Black women physicists: Stories
of experiencing and overcoming obstacles in life,” Physical Review Physics Education
Research, vol. 12, p. 020113, Aug. 2016.

[123] “Broadening Participation in STEM | NSF - National Science Foundation.”

[124] “Physics Frontiers Centers (PFC) | NSF - National Science Foundation,” Apr. 2022.

[125] “Joint Network for Informal Physics Education and Research (JNIPER) Mission and

Vision.”

[126] A. Lau, “Communication with Alexandra Lau, Senior Program Manager at American

Physical Society.”

[127] K. N. Canfield, S. Menezes, S. B. Matsuda, A. Moore, A. N. Mosley Austin, B. M.
Dewsbury, M. I. Feli-Mjer, K. W. B. McDuffie, K. Moore, C. A. Reich, H. M. Smith,
and C. Taylor, “Science Communication Demands a Critical Approach That Centers

200

Inclusion, Equity, and Intersectionality,” Frontiers in Communication, vol. 5, Jan.
2020.

[128] K. A. Stofer, D. Hanson, and K. Hecht, “Scientists need professional development to
practice meaningful public engagement,” Journal of Responsible Innovation, vol. 10,
p. 2127672, Jan. 2023.

[129] “The Equity Compass | The YESTEM Project.”

[130] E. Aurbach, E. Kuhn, and R. K. Niemer, “Organizing and Reflecting Different Types
of Engagement Activities: The Michigan Public Engagement Framework,” Apr. 2019.

[131] E. Aurbach and J. Kuhn, “Expanding the Utility and Impact of the Michigan Public

Engagement Framework.”

[132] S. V. Chasteen, R. E. Scherr, and M. Plisch, A Study of Thriving Physics Teacher
Education Programs: Development of the Physics Teacher Education Program Analysis
(PTEPA) rubric. College Park, MD: American Physical Society, 2018.

[133] N. D. Finkelstein, L. Mayhew, C. Henderson, M. Sabella, and L. Hsu, “Acting in
Our Own Self-Interests: Blending University and Community in Informal Science
Education,” in AIP Conference Proceedings, (Edmonton, Alberta (Canada)), pp. 19–
22, AIP, 2008.

[134] K. Hinko and N. D. Finkelstein, “Impacting university physics students through
participation in informal science,” (Philadelphia, PA, USA), pp. 178–181, 2013.

[135] A. Muller, D. M. Christman, M. M. Rice, F. Soto-Apolinar, S. Hirsch, and D. J. Arya,
“Undergraduate Motivations for Participating in Afterschool Programs: A Case Study
of The STEMinist Program,” Afterschool Matters, vol. 34, pp. 20–29, 2021.

[136] C. Rispler, B. Prefontaine, and K. A. Hinko, “Understanding university students
identity through engagement in informal physics programs,” in Proceedings of PER
Conference, (Provo, UT), Jan. 2020.

[137] S. El-Adawy, E. C. Sayre, A. C. Lau, and C. Fracchiolla, “Personas for supporting
physicists’ engagement in informal education,” in 2022 Physics Education Research
Conference Proceedings, (Grand Rapids, MI), pp. 157–162, American Association of
Physics Teachers, Sept. 2022.

[138] B. Stanley and K. Hinko, “From volunteer to program leader: a career in informal

physics,” 2022.

[139] G. Lakoff, Metaphors we live by. Chicago: University of Chicago Press, paperback. ed.,

1980.

201

[140] K. Singh, “Metaphor as a tool in educational leadership classrooms,” Management in

Education, vol. 24, pp. 127–131, July 2010.

[141] E. Kemp, “Metaphor as a Tool for Evaluation,” Assessment & Evaluation in Higher

Education, vol. 24, pp. 81–89, Mar. 1999.

[142] D. Gentner and D. R. Gentner, “Flowing Waters or Teeming Crowds: Mental Models

of Electricity,” tech. rep., Bolt Beranek and Newman Inc., 1992.

[143] B. D. Lewin, “Metaphor, Mind, and Manikin,” The Psychoanalytic Quarterly, vol. 40,

pp. 6–39, Jan. 1971.

[144] J. P. Harrington, “Pynchon, Beckett, and Entropy: Uses of Metaphor,” The Missouri

Review, vol. 5, pp. 129–138, June 1982.

[145] D. M. Doberneck and S. L. Dann, “The degree of collaboration abacus tool,” Journal
of Higher Education Outreach and Engagement, vol. 23, no. 2, pp. 93–107, 2019.

202

APPENDIX A. CHAPTER 3 INTERVIEW PROTOCOL

This appendix was originally published as supplemental material for Stanley et al. [106].

Prior to being interviewed, participants in this study filled out a survey, providing

logistical information about their program. Some of these questions follow-up on their survey

responses for clarification and more details. These were semi-structured interviews and the

interviewer would ask follow up questions based on the participant’s responses. Below is

the full list of questions, broken up into themes. Questions were asked from all themes in

all interviews, but based on the content of the response and follow-up questions, not every

single specific question from this list was asked of each interview participant.

Development, history, and goals of the program

• When did you join the [program name]? How did that get started? What was your

motivation for joining?

• Have you seen any drastic changes over time in any aspects of the program?

• What is the history of the program? How did it get started? What was the motivation

to start the program? Who started it?

• In the survey, you described your role in the program as [their survey response]. Could

you explain more on the day-to-day operation? How do you define your relationship

with the rest of the personnel? Are there any other details in that regard you would

like to share with us here?

• In the survey, you described the objective of your program to be [their survey response].

Has that changed over time? In which way and why?

203

Connection to institution and resources

• How is the program connected to the physics department and/or university?

• What other resources are available to the program, provided by the institution, physics

department, physics community at your institution, such as educational, structural

(space, transportation) or personnel?

• Where is the program held? What space do you use for your different activities?

• Has the program always been hosted at the mentioned institution? Why or why not?

• How does the placement within the host institution impact the program?

• What are the most important overall impacts the program has had on the institution?

Has there been any major structural impacts? Has there been any major cultural

impacts?

• Tell me about the culture of your (department/center/national lab). How collaborative

is your institution?

• How do people work together?

• How receptive are members of your institution to the program?

• What is the role of the administration of the institution in the informal program? Have

these roles changed over time? Why?

• What are the characteristics of the institutions culture that has a larger impact on the

program? In which way? Why?

• How do you think your research affects your role in the program?

204

Leadership, volunteers, recruitment and training

• How did you count the number of personnel? Anyone that you did not count? Any

changes over time?

• How is the program structured in terms of the personnel? Do you have staff, volunteers,

etc? Staff? What are their roles in the program?

• Do the personnel receive training for the role they play in the program? What kind of

training? For how long?

• Has leadership changed over time? What is their relationship with the rest of the

personnel, partners, and sponsors?

• How do you recruit facilitators and volunteers? How do you advertise for your program?

How do you normally attract volunteers?

• What are the facilitators and volunteers backgrounds?

(are they undergraduate,

graduate students, STEM, Education, etc)?

• What are the roles of individual physicists (faculty, staff) and university physics

students in the program?

Connections to community partners and funding

• What is the role of community partners with your program? What is their connection

to your program/event? Has this changed over time? In which way and why?

• How would you describe your relationship with your partners?

• What is the role of sponsors with your program? What is their connection to your

program/event? Has this changed over time? In which way and why?

205

• If there could be a community or organization that could provide support to programs

like yours, what kind of support would you like to see?

• How is the program funded? How has this changed over time?

• In your opinion, what factors affect or have affected the sustainability of the program?

Physics content, activity format, and assessments

• Could you give us more information on the physics contents/concepts that your

program aims to cover? Has the content changed over time?

• Is the program based on a particular educational model?

• What are the logistics of the program? (For example, preparation before the activity,

emailing, setting up events...)

• Are there any particular pedagogical techniques emphasized and employed for the

design and implementation of the activities?

• What

educational and discipline-specific

resources are used to inform the

design/development of the activities?

• Do you think that having some assessment/evaluation involved with your program

could have positive effects on the program in the future in any way? Do you have any

assessment ideas in mind that can be used to evaluate the program’s performance or

measure some aspects?

• You mentioned in the survey that your assessment is based on [their survey response],

could you tell us more? What are the expected outcomes? Do you use that information

to make changes in the model/design of the program?

206

• What assessment resources do you draw from? How do you measure outcomes? How

do you measure if the program’s goals have been achieved?

Impact on audiences and volunteers, challenges, and

looking forward

• What is the impact of your program on volunteers? On audiences?

• What makes the program successful? What are the characteristics of the program that

makes it successful? Why?

• Are program volunteers achieving their own goals? How is that determined?

• Has the programs audience demographics changed over time?

• Are audiences achieving their goals? How is that determined?

• Are there any current challenges the program is experiencing?

• Where do you see the program going in the next 5 years?

207

APPENDIX B. CHAPTER 3 OVERLAP TABLE

This appendix was originally published as supplemental material for Stanley et al. [106].

Table B.1: The below table shows the percentage of code overlaps present for each framework
category pair across all three case study programs. The sum of the columns may not add
up to 100% due to rounding.

Pub Physics

Camp Physics

Physics Club

N= 258 overlaps

N = 202 overlaps

N = 56 overlaps

Assessment-Audience

3.1%

Assessment-Institution 0.0%

Assessment-Personnel

2.7%

Assessment-Program

2.3%

Assessment-Resources

0.0%

Audience-Institution

0.8%

Audience-Personnel

5.4%

Audience-Program

12.4%

Audience-Resources

1.9%

Institution-Personnel

7.0%

Institution-Program

8.1%

Institution-Resources

2.3%

Personnel-Program

29.8%

Personnel-Resources

11.6%

Program-Resources

12.4%

2.5%

1.0%

2.5%

1.0%

1.0%

0.5%

8.9%

19.3%

7.9%

11.4%

3.0%

5.0%

15.3%

10.9%

9.9%

208

1.8%

1.8%

3.6%

5.4%

0.0%

0.0%

5.4%

12.5%

0.0%

21.4%

0.0%

3.6%

25.0%

14.3%

5.4%

APPENDIX C. CHAPTER 4 PROGRAM DESCRIPTIONS

Table C.1: We selected 15 programs from our dataset to build our key components
framework. These programs differ in format, content, audience demographics, and institution
type.

Program Format

Topics

Interviewee

Location Audience Type

Institution Type

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Private Liberal

Arts

Large Research

Facility

Public Lecture

Astronomy

Faculty

Midwest Adults

R1 University

Summer Camp

Topics in Current

Physics Research

Public Lecture,

Topics in Current

Demonstration Shows

Physics Research

Paid Staff

Midwest K-12

Large Research

Facility

Paid Staff

Midwest Adults

R1 University

Observatory

Astronomy

Paid Staff

Midwest General

R1 University

Open House,

General physics,

Undergraduate

Planetarium Shows

astronomy

Student

Midwest K-12

R1 University

Demonstration Shows General physics

Faculty

West

General

R1 University

Summer Camp

General physics

Faculty

East

K-12

Public Lecture

Planetarium

Summer Camp

Topics in Current

Physics Research

Astronomy,

planetarium shows

Topics in Current

Physics Research

Paid Staff

West

Adults

Paid Staff

West

General

R1 University

Paid Staff

Midwest K-12

Large Research

Facility

Demonstration Shows,

Open House

General physics

Graduate

Student

Midwest K-12

R1 University

Museum Program

General physics

Faculty

West

K-12

R1 University,

Museum

Demonstration Shows General physics

Paid Staff

Midwest K-12

R1 University

Planetarium

Astronomy

Paid Staff

Midwest General

R1 University

Science Festival

General physics

Faculty

Midwest General

R1 University

209

APPENDIX D. CHAPTER 4 REFLECTION QUESTIONS

Below is a list of questions for each key component that program leaders can use to reflect

upon their program. Not every question may necessarily be relevant to every program leader.

In addition, some questions may be relevant for multiple components.

Goals

• What are the goals of my program?

• Who developed the goals? Are they shared by all program personnel?

• How are the goals of my program being met, and in what ways are they not met?

• What are the contributing factors to the success of my program goals?

• What are the challenges to meeting the program goals?

Program Content & Design

• What physics topics do my audiences find interesting?

• What physics topics do my personnel and I have expertise on?

• Am I able to travel to audiences? Are audiences able to travel to my program?

• Where am I able to host my program?

• How does the physical space where I host my program impact the types of activities I

host?

Strategies for Assessment

• What forms of assessment are utilized in my program?

• What is my program focused on assessing?

210

• What is the purpose or motivation of assessment within my program?

• Are the types of assessments used by my program effective for meeting the goals of

assessment?

• How can my program use results from assessments to make changes or improvements?

Recruitment and Onboarding of Personnel

• What type of volunteer (student, faculty, etc) would be the best fit for the tasks I need

done?

• Are my advertisement methods appropriate and relevant for the type of volunteer I am

trying to recruit?

• What do my volunteers need to be trained on?

• Who is conducting the training?

Leadership? More experienced volunteers?

Combination of both?

• What incentives can I offer?

• What are the benefits I can tell potential volunteers to convince them to volunteer?

• How do the schedules of my volunteers align with when I host my events?

Distribution of Roles and Tasks

• What are the tasks that need to be completed for my program to run?

• Of the volunteers I have, who is able to complete each task?

• Are some of my volunteers able to carry out multiple tasks?

• What is the time commitment for each task?

211

• Do my volunteers have the time available to complete these tasks?

• What is the order of importance of each task?

• Do tasks have to be completed in a particular order or can they be completed in

parallel?

• How many tasks or responsibilities does each personnel member have?

Continued Support of Personnel

• Am I able to give financial support or course credit to my volunteers?

• Do volunteers have autonomy in creating activities?

• What does volunteer mentorship look like? Are volunteers being mentored by program

leadership? Are they mentored by more senior volunteers?

• Do volunteers get leadership opportunities?

Understanding Audience Motivations and Needs

• Who is the audience I am aiming to connect with?

• Who are the audiences nearby?

• What lived experiences and cultural resources does the audience have?

• What are my audiences interested in?

• What do my audiences want to learn about / need to learn about?

• What other activities or events are available or popular with audiences in the area?

• What are the limitations on the audience’s ability to attend events? Is my program

affordable?

212

• What ways does our program content connect to local and national STEM curriculum

or standards?

Support of Audience Engagement and Learning

• What are the spaces where audience members can ask questions in my program? Is it

set-up so that there can be a back and forth conversation?

• How do audiences interact with the personnel? With other audience members?

• What are the ways that audience members can give me feedback?

• What background knowledge does the audience have on the physics topics?

• Are my program materials accessible to audiences with differing needs? Are my

program materials translated into languages that serve my audiences?

Connection to Community and Partners

• Who are my partners? How many partners do I have?

• What are my goals with this partnership? What are my partners’ goals?

• What do I need/want from this partnership? What do my partners need/want?

• What expertise do I bring to this partnership? What expertise do my partners bring

to this partnership?

• How often do I communicate with my partners? What does this communication look

like?

• How long do I anticipate this partnership lasting?

• How have my partnership relationships evolved over time?

213

Relationship with Institution

• How does my institution benefit my program?

• How does my program benefit my institution?

• What are the supports that I would like to see from my institution?

• Am I able to request for support from the institution? (ask for funding, physical space,

volunteers, etc.)

• What does the institution see as the benefits of my program?

• Can my program partner with other institutional events? Ex. Participating in

recruitment events

Physical and Digital Resources

• What are the physical spaces that I need to host an event?

• What are the physical spaces that I need outside of events (lab space, offices, storage

space)?

• What are the limitations of my physical space?

• What type of supplies do I need?

• Do I have an inventory list of the supplies and activities that I have?

• What does my online presence look like (website, social media)? What information

about my program is available?

• How often are my online materials updated?

214

Funding and Budget

• What do I need funding for? Purchasing supplies? Paying personnel? Travel? Food?

• How much funding do I need to carry out my goals?

• What are my sources of funding?

• Do I have one source of funding or multiple sources of funding?

• Does my institution have funding opportunities for my program?

• How long will my funding source last?

• If a funding source ends, what adjustments would I need to make to my program?

215

APPENDIX E. CHAPTER 5 INTERVIEW PROTOCOL

These were semi-structured interviews and the interviewer would ask follow up questions

based on the participant’s responses. Below is the full list of questions, broken up into

themes. Questions were asked from all themes in all interviews, but based on the content

of the response and follow-up questions, not every single specific question from this list was

asked of each interview participant.

Background and Current Work

• Where did you attend for your post-secondary education?

• What was your field of study?

– What led you to this area of study?

• What have you been doing since leaving [post-secondary institution]?

– Can you describe some of your current work responsibilities?

• What are some skills that you find important in your current work?

– Can you provide some examples of how these skills are important to your current

work?

• For program leaders

– Why did you pick this job? Where there any positions that you had that led you

here?

– Tell me about your current position. What are your roles and responsibilities?

216

Informal Physics Experience

• When you were at [post-secondary institution], you were involved in an informal physics

program(s). Can you describe the program?

– What was your involvement in the program?

• What was your motivation for joining [program]?

• What do you feel were the goals of the program?

• What were your goals in the program?

• As a child, do you have any memorable experiences with informal spaces? Examples:

after school programs, summer camps, science museums

– What effect did these experiences have on your educational/career pursuits?

– What effect did these experiences have on your involvement in [program]?

• How would you describe your overall experience in [program]?

– Can you describe a particularly memorable experience you had with [program]?

– Can you describe a particular experience you had engaging with the

audience/community member/public member with [program]?

– Can you describe a particular experience you had interacting with your fellow

members of [program]?

• How are those experiences different from other experiences you had (research, classes,

etc)?

• How have your experiences in [program] influenced, if at all, your current career?

217

– Is there a specific experience you can describe that influenced your career pursuits?

• What did you value most from your experience in [program]?

– Do you ever use anything from those experiences today?

– Did you learn anything that impacted you?

• Are you still involved in informal physics/physics public engagement activities?

– Can you describe them?

– How about non-physics related service/engagement?

– Do you feel that there are opportunities for you to get involved in such

engagements?

• Where do you see your career going?

• Is there anything else you would like to share about your experience in [program] or

your career?

218