COMMUNICATING NATURAL HAZARD RISK: WARNINGS, DECISIONS AND
PRECAUTIONARY BEHAVIOR
By
Robert E. Drost

A DISSERTATION
Submitted to
Michigan State University
In partial fulfillment of the requirements
for the degree of
Geological Sciences – Doctor of Philosophy
2014

ABSTRACT
COMMUNICATING NATURAL HAZARD RISK: WARNINGS, DECISIONS AND
PRECAUTIONARY BEHAVIOR
By
Robert E. Drost
This dissertation consists of four studies that investigate factors influencing
individual decision-making and preferences for delivery of information during severe
weather. The first study population consisted of 49 undergraduate students and focused
on the role semantic and episodic memory plays during severe weather warnings
(tornado), and examined how participants would react during a simulated tornado
warning. The study discovered that increased knowledge of severe weather generally
lead to more prudent decisions during warnings, while greater personal experience with
tornadoes lead to a decrease in taking precautionary measures. The second study was
made up of 181 members attending one of two Geological Society of America annual
conferences. The study investigated geoscientist’s perception of climate issues and the
public’s trust of science communication. The study found that scientists generally agree
that anthropogenic climate change exists, however, the public’s knowledge of climate
issues varies in depth and their trust of the scientific community is wavering. The limited
trust of the science community is of concern since it has the potential to negatively
impact the public during exposure to a variety of natural hazards. The third study
explores the impact of weathercaster gesturing on viewer retention and attention to
material presented during a live weather forecast. The study was composed of thirty-six
undergraduate students and was viewed on Tobii T60 eye tracking equipment in order

to capture gaze data of the participants. Results of the study indicate that gesturing may
be acceptable during broadcasts where non-salient information is being presented since
it may redirect the attention of the viewer. However, when salient information is being
presented, gesturing may redirect viewer attention away from items of importance and
is therefore not recommended. The fourth study examines the impact of a traditional
severe weather warning on viewer attention and retention of the weather information
being presented. Multiple screen elements make up a typical weather warning and may
include radar imagery, live footage, weathercaster information and warning scrolls at the
bottom of the television screen. A traditional televised weather warning was compared
to a novel animated version and an audio only version containing nearly the same
information. All versions were viewed/heard as part of an eye tracking experiment
utilizing a Tobii T60 eye tracker. Eye tracking data were used to ascertain participant
gaze information. Participant knowledge and experience data were collected prior to
viewing the warnings. After viewing one of the three warning treatments, participant
retention and preference data were collected. The study revealed that multiple onscreen elements may lead to a diverse viewer attention pattern when viewing warnings.
Additionally, participants viewing the animated warning retained significantly more
warning information than the other participants. The studies completed in this
dissertation indicate that numerous inputs affect viewer behavior during weather
warnings. It has shown that in addition to knowledge and experience, trust in the
information being delivered, delivery methods, and the make-up of the warning
broadcast all influence how warning information is attended to and retained by the
viewer.

COPYRIGHT BY
ROBERT E. DROST
2014

For my parents, Arthur and Jean Drost, for pushing me to think beyond my comfort
level, and my mom, Marlene Ventura, for giving me a chance for a better life. Most
importantly, for my wife, Vicki Drost, for supporting my childhood dreams.

	
  

v	
  

ACKNOWLEDGEMENTS

During my course of study that has lead to this dissertation I have been the
benefactor of an enormous amount of emotional and intellectual support and
encouragement from my family, friends and faculty. First, I would like to show my
appreciation to my advisor, Julie C. Libarkin, for having the courage to accept and
support a non-traditional student and for the encouragement to realize my life goal of
being a scientist. Although difficult and often times frustrating, I have grown to embrace
the nuances of becoming an interdisciplinary researcher and appreciate the knowledge
and experience brought by others while I completed my research. I would also like to
thank my committee members who possess a wide range of experience and broadness
of discipline that has not only challenged my work, but also improved upon my results
by encouraging me to explore areas unknown to me. First, Tyrone Rooney, who
challenged me to think like a geologist, exposed me to the world of geochemistry, and
was always available to discuss the difficulties I encountered along the way; Aaron
McCright, whose bold suggestions and pinpoint focus on social aspects of my work
helped me think outside the box, and challenged conventional thought in my research
area: and Stephen Thomas, who served on my committee for only a short time, but
supplied the skill, insight and enthusiasm that supported my research direction.
Although my committee was quite diverse in its’ disciplinary background, every member
supported and encouraged my education and research to the greatest possible extent.
The result for me was a much deeper understanding of my own discipline and research
that encompassed a much broader focus enabling collaborative engagement inside and

	
  

vi	
  

outside the academic realm. The inspiration and guidance I have received from my
committee is greatly appreciated and a critical component to my success.
I would like to express my appreciation to the members of the Geocognition
Research Laboratory who have provided thoughtful input and support during the last
five years. In particular I would like to thank Sheldon Turner, Nicole Ladue, Emily Ward
and Christy Steffke, all who made difficult times less difficult and provided support on all
levels during my course of study.
I thank Sheldon Turner for taking time to support my geological inspirations and
partaking in discussions about zombies, drilling to the mantle and insect takeovers
designed to maintain our sanity.
I thank Jay Trobec for providing not only videos of his on-screen persona, but
also for proposing interesting topics worthy of study. I would also like to thank Jay for
his conference support and willingness to fill my spot when I was unable.
I would also like to thank Mark Casteel, Matt Meister and Stephen Thomas for
their support and knowledge on chapter four of my dissertation. Without their novel
approaches, animation skills and real-world input, the study would not have achieved
its’ goals.
As a special acknowledgment I would like to show my deep appreciation for the
Michigan State University Graduate School (Dr. Julius Jackson), the Office of
Geological Sciences (Dr. David Hyndman), the Environmental Science and Policy
Program (Dr. Jinhua Zhao), the College of Natural Science (Dr. Richard Schwartz) and
Dr. Julie Libarkin for the financial and emotional support I received during my final year
when life attempted to interfere with my dreams through illness. I will always remember

	
  

vii	
  

and appreciate the genuine outpouring of concern and selflessness I experienced
during that time. The actions of these individuals support and reinforce my convictions
that Michigan State University is without equal among educational institutions.
Finally, I especially thank my wife Vicki, my kids Ian, Megan, Kevin, Brandi and
Nick for their patience, willingness to listen to ongoing “geology and meteorology”
discussions around the campfire, enthusiasm for my research and for giving up a little
“normality” in their life to support my decision to return to school and become a scientist.

	
  

viii	
  

TABLE OF CONTENTS
LIST OF TABLES

xi

LIST OF FIGURES

xii

Introduction

1

Chapter 1 Memory and decision-making: Determining action when the
sirens sound.
1.1 Introduction
1.2 Theoretical Foundations
1.3 Methods
1.3.1 Participants
1.3.2 Data Collection
1.3.3 Data Analysis
1.3.4 Validity and Reliability
1.4 Results
1.5 Discussion and Conclusions
ACKNOWLEDGMENTS
APPENDICES
Appendix 1A: Semantic Questionnaire (SQ1)
Appendix 1B: Episodic Questionnaire (EQ1)
Appendix 1C: Decision-making questionnaire (D1 and D2)
Appendix 1D: Statement of Reprint
REFERENCES

7

Chapter 2 GSA members on climate change: Where, what, and ways forward?
2.1 Introduction
2.2 Methods
2.3 Results
2.4 Discussion and Conclusions
APPENDIX
REFERENCES

36
36
37
37
38
42
44

Chapter 3 Gesturing During Weathercasts: A “hands down” best practice?
3.1 Introduction
3.2 Methods
3.2.1 Data Collection
3.2.2 Data Analysis
3.3 Results
3.4 Discussion and Conclusions
APPENDICES
Appendix 3A: Retention Survey Questions
Appendix 3B: Analytical Methods

46
47
51
51
54
56
61
66
67
68

	
  

ix	
  

8
9
12
12
13
15
16
17
23
26
27
28
30
31
32
33

REFERENCES

69

Chapter 4 Severe Weather Warning Communication: Factors impacting
audience attention and retention of information during warnings.
4.1 Introduction
4.2 Methods
4.2.1 Data Collection
4.2.2 Data Sources
4.2.3 Data Analysis
4.3 Results
4.3.1 Participant Results – Knowledge, Experience and Trust
4.3.2 Model Results – Retention and Preference
4.4 Discussion and Conclusions
APPENDICES
Appendix 4A: Knowledge Questionnaire (KNOWLEDGEQ)
Appendix 4B: Experience Questionnaire (EXPERIENCEQ)
Appendix 4C: Retention Questionnaire (RETENTIONQ)
Appendix 4D: Traditional Model Questionnaire (PREFERENCEQ - TRAD)
Appendix 4E: Animated Model Questionnaire (PREFERENCEQ - ANIM)
Appendix 4F: Audio Model Questionnaire (PREFERENCEQ - AUDIO)
Appendix 4G: Independent T-test Results
Appendix 4H: Analytical Methods
REFERENCES

73
74
80
82
83
85
89
89
91
97
102
103
104
105
106
108
110
111
113
115

	
  

x	
  

LIST OF TABLES
Table1.1: Factor loadings and communalities for the Ignore Warning Scale. The
strength of the factor loadings and number of items per factor suggest the retention of
one scale, the “Ignore Warning Scale”.
15
Table 1.2: Participants (n=49) and their personal lifetime warning and event
experiences. Results indicate a significant experience with severe weather warnings
and a fair amount of experience with tornadoes.
19
Table 1.3: Results of the Related Samples Wilcoxon Signed Rank Test indicating that
the stimulus had significant impact on the low episodic/low semantic and low
episodic/high semantic groups.
20
Table 1.4: Results of the MANOVA analysis suggesting the main effects were not
significant but the interaction of the two were somewhat significant given the small
number of participants.
21
Table 1.5: Pre-stimulus and post stimulus results illustrating the participants likelihood of
ignoring a warning for both D1 and D2.
22
Table 2.1: Survey questions and response information. Please contact authors for
access to full survey results.
37
Table 3.1: Definitions of gaze data used in this study.

50

Table 3.2: AOI letter designation and corresponding areas in pixels (units squared).
*Areas are identical except for the Weathercaster Hand AOI.
56
Table 3.3: Total fixations, average fixation duration and total fixation duration for both
No gesture and Gesture conditions. No Gesture n=19, Gesture n=17.
58
Table 3.4: Results of the Mann-Whitney U Test Statistics for No Gesture and Gesture
conditions.
59
Table 4.1: Data source descriptions and timing during the study.

82

Table 4.2: Likert scale questions and participant’s (n=90) responses. 1 = strongly agree,
5 = strongly disagree.
90
Table 4.3: Participant’s Likert scale responses to the difficulty and ease of
understanding their severe weather warning model.
1=not difficult, 5=very difficulty.

94

Table 4G: Independent T-Test Results.

112

	
  

xi	
  

LIST OF FIGURES
Figure 1.1: Experiment progression and timing of the questionnaires and slide show
stimulus.
14
Figure 1.2: Participants primary source of weather information. The Internet was the
primary source of weather information, followed by TV with friends and family (F/F)
ranking third.
18
Figure 1.3: D1 and D2 results showing the interaction of experience and knowledge
when faced with the decision of ignoring a tornado warning.
20	
  
Figure 2.1: Image showing the regions of most climate change concerns among GSA
respondents. Darker orange areas are where more respondents shaded in.
38
Figure 3.1: A - No Gesture condition forecast variation and B - Gesture condition
forecast variation. Both forecasts were identical except for the presence or absence of
the weathercaster’s gesturing. Forecasts provided by KELO-TV, Jay Trobec, Chief
Meteorologist.
52
Figure 3.2: A - No Gesture condition with associated AOI and B - Gesture condition with
associated AOI. All but one AOI are identical; the Hands AOI varies based on main
gesturing position in the video.
56
Figure 3.3: A and B are example gaze plots during No Gesture condition and Gesture
condition forecast variations. Each depiction illustrates one participant’s gaze during
each forecast. In Figure 3 A is a gaze plot for Participant Elab2011020a during the No
Gesture condition forecast variation and B shows a gaze plot for Participant
Elab2011006b during the Gesture condition forecast variation.
60
Figure 3.4: Example heat maps for the No Gesture condition (A) and Gesture condition
(B) forecast variations. Each depiction illustrates the study population’s gaze during
each forecast. In Figure 3.4 A is a heat map for study participants during the No
Gesture condition forecast variation and B is a heat map for study participants during
the Gesture condition forecast variation.
61
Figure 4.1: Example of Traditional and Animated media clips. The Audio only clip
contains no images and is not illustrated here.
81
Figure 4.2: Series of images showing participant gaze for both the TRADITIONAL
(Trad) and ANIMATED (Anim) media clip. Individual participant’s gaze is indicated by
the black dots. Clip Trad1 is representative of a “diffuse” gaze pattern; Trad2 is
representative of a “concentrated” gaze pattern. Clip Anim1 is representative of a gaze
“concentrated” on multiple elements; Anim2 is representative of a “concentrated” gaze
pattern.
96

	
  

xii	
  

Introduction
Severe weather warnings provide the projected track of a storm and its’ intensity
with the intent of providing adequate lead-time so individuals may take precautionary
measures to ensure personal safety and protection of property. Ideally, severe weather
warnings would be 100% accurate, consistently provide adequate time for preparation,
and include instructions of precautionary measures necessary to protect life and
property. In addition, the public would heed these warnings and advice without question
regardless of personal intuition.
Unfortunately, severe weather warnings often lead to “false alarms” due to the
unpredictable nature of these events. False alarms may lead to public desensitization of
the severe event and result in the public ignoring the warning or substantially
downplaying its’ threat potential. Individual decisions and behavior in response to a
severe weather warning is often unpredictable due to an individual’s amount of
knowledge and experience with severe weather as well as how the information is being
conveyed during the warning. The science and improvement of forecasting and
issuance of severe weather warnings remains in the hands of institutions dedicated to
this purpose. Discovering what influences public behavior and determining the best
methods to convey severe weather warning information becomes the variable factor
that influences what precautionary measures are taken by individuals during warnings.
In order to clarify what influences individual behavior, a few questions become
apparent: a) how does personal experience and knowledge of severe weather affect
decisions made during severe weather events? b) what information does an individual
find useful when making decisions during severe weather events? c) does the amount

	
  

1	
  

of information conveyed to an individual impact the severe warning message? and d)
what role does the weathercaster serve when delivering the severe weather warning?
The National Oceanic and Atmospheric Administration and the National Weather
Service are dedicated to improving the effectiveness of severe weather warnings.
Constant refinement of warnings designed to increase the effectiveness take place
within these agencies. Recently, storm based warnings that use polygon boundaries
instead of county boundaries to more precisely define areas at risk was implemented to
reduce warning area and economic impact associated with unnecessary warnings.
Additionally, impact based warnings are being piloted that incorporate harsher tones
and more direct warning statements in an attempt to combat public complacency.
Success of both measures has been hard to define. Storm based warnings remain
limited when multiple warnings within a county are issued, necessitating additional
clarification be made to the public. Impact based warnings graphically communicate
storm potential, however they risk frightening the public with bold statements intended
to elicit public response to take protective measures.
Both of these warning enhancements necessitate the need for an increase in
public understanding of severe weather and associated warnings. This increase in
understanding may be accomplished through additional knowledge of threatening
weather and more effective communication techniques.
The present course of study investigates variables associated with building better
severe weather warnings and delivery methods that lead to prudent precautionary
measures by individuals faced with a severe event. It is intended to answer the following
question: How does an individual’s knowledge and experience of severe weather, as

	
  

2	
  

well as warning communication methods influence individual behavior and decisionmaking during a severe weather event?
Four studies have been completed in order to investigate this research question.
The first chapter looks at the roles of memory (semantic and episodic) in influencing an
individual’s behavior during tornado warnings. Semantic memory represents the
accumulated knowledge an individual has on a particular subject area. Episodic
memory represents the individual’s personal experience. The second chapter
investigates the science community’s (expert) attitudes and opinions on public
communication of potential natural hazards using climate change as an example, as
well as public trust of the science community. Chapter three investigates the impacts of
gesturing (body movement by the weathercaster) and the role of TV screen clutter
(multiple on screen media communicating similar information) on participant’s attention
and retention during weather broadcasts. Chapter four looks at screen clutter and
identifies the most helpful features during a warning and determines viewer reliance on
certain modes of warning methods (scroll, radar, meteorologist recommendations). In
addition, alternate delivery formats, traditional TV broadcasts and animation are used to
develop novel approaches to alternative warnings.
Chapter 1: Memory of past events and specific subject knowledge of severe weather
influences individual behavior during severe weather events. This study examined how
participants would behave during a simulated tornado warning (a 5 minute slide show
stimulus showing tornadoes and resulting damage) and compared their decisions based
on the amount of weather knowledge and personal experience. The study consisted of
forty-nine undergraduate students and lasted 1 hour during which they were faced with

	
  

3	
  

decision-making tasks associated with severe weather. Two major outcomes surfaced
from the results: (1) Students who possessed a greater amount of weather knowledge
where more likely to take precautionary measures during a severe weather event than
those with less knowledge, and (2) personal experience with severe weather combined
with a less amount of knowledge tended to cause the students to be less likely to take
precautionary measures during a severe weather event.
Chapter 2: Public perception of inconsistencies among scientist’s views and the lack of
effective communication impacts public perception of science issues. Potential lack of
trust and limited knowledge impact the public’s comprehension of a multitude of geoevents, from climate change to severe weather. This study examined commonalities
among scientists and the public as they view natural hazards associated with climate
change. These served as an indicator to the overall level of trust the public has in
science based information. The public typically receives this information through local
media outlets. Surveys were collected in an annual meeting of the Geological Society of
America (GSA) in 2009 and again in 2011 resulting in a total of 181 surveys completed
by conference attendees. Geoscientists were asked to express their views on climate
change and public perception as well as their understanding of potential issues
surrounding natural hazards and public communication. Major outcomes of the study
indicate that while scientists accept that anthropogenic climate changes exist, the
public’s knowledge is limited and a trust issues exists between the public and the
scientific community. Mainly, the public perceives a lack of agreement among the
scientific community on a multitude of natural hazard topics even though they represent
the main source of science-based issues. Additional methods of communication need to

	
  

4	
  

be explored in order to ensure effective public understanding of science-based issues
and to bridge the gap of public perception of disagreement among scientists.
Chapter 3: Televised media is one of the most accessed sources of weather
information. Despite the commonalities of information being presented by
weathercasters, individual preferences for delivery of weather information varies among
weather broadcasts. This study examined the impact of weathercaster gesturing during
weather forecasts on public retention of the information being presented. In this study,
thirty-six undergraduate students were exposed to two identical weather forecasts. Both
forecasts were viewed on the Tobii T60 eye tracker in order to determine participant
fixation and gaze information. In one of the forecasts the weathercaster remained
relatively motionless, while in the other forecast the weathercaster gestured using his
hands to direct attention to the forecast. Results indicated that while gesturing diverted
viewer attention from specific on screen displays, retention of the forecast material
remained unaffected.
Chapter 4: Wide ranges of natural disasters occur in the United States, inflicting untold
personal and economic damage. Severe weather, in various forms, is experienced
almost daily throughout the United States. Given the number of traditional formats,
varied amount and type of information conveyed during severe weather warnings, public
attention to these potential disasters is often lacking. The final study compared
traditional warnings along with an animated warning in order to assess viewer impact,
preference and retention of the information being presented. The study was comprised
of 90 participants consisting of college students, faculty and staff. Participants
completed questionnaires that gauged their level of weather knowledge and personal

	
  

5	
  

experience. Next, they viewed one of three possible severe weather warning video clips
presented on a Tobii T60 eye tracker. After viewing the nearly identical clips, a retention
questionnaire was completed followed by a questionnaire designed to discover the
participant’s weather warning preferences and usefulness of on-screen information
presented during the clip. Of the three warning clips (traditional, animated and voice
only), participants retained significantly more information during the animated clip. In
addition, gaze patterns were more diffuse during the traditional clip compared to the
animated clip.
Conclusions: The studies completed in this dissertation found that an individual’s
knowledge and experience (memories) impact behavior during severe weather
warnings. However, in addition to an individual’s memories, a multitude of factors also
influence the individual’s comprehension of and attention to severe weather warnings.
Decision-making and resulting behavior may be impacted by not only the
communication techniques and delivery method utilized by the media, but also an
individual’s preferences for information received during a warning. Memory, social,
cognitive and policy inputs all share a role in the understanding and resulting behavior
associated when a severe weather warning is issued. Future studies involving an
evaluation of an individual’s weather knowledge and experience should consider the
role of screen clutter, viewer preferences, usefulness of the information being delivered,
and weathercaster delivery techniques. Together this information may be used to “build”
a better severe weather warning that is more effective in delivering the information and
imparting the motivation to move the public to take precautionary measures insuring
their safety and property.

	
  

6	
  

Chapter 1
Memory and decision-making: Determining action when the sirens sound.
Memories, both semantic, or learned knowledge, and episodic, or personal
experiences, play an important role in an individual’s decision-making under risk. In
addition, varying levels of knowledge and experience exist in each individual. These
memories enable individuals to make informed decisions based on previous knowledge
or experience, and ultimately influence one’s behavior under risk. In this study, fortynine undergraduate students participated in a one-hour, classroom-based experiment
focusing on decision-making. The sample contained n=23 “episodic” participants,
referred to as “high episodic”, who reported having personally experienced a tornado
and n=24 participants, referred to as “low episodic” who had no reported tornado
experience. Incomplete data reported by the remaining participants was not included in
this study. All participants completed a decision-making task both before and after
viewing a 5-minute slide show stimulus related to tornadoes and associated damage.
This decision-making task prompted participants to describe the actions they would
anticipate taking during an actual tornado warning. Prior to the stimulus, high episodic
participants exhibited a marginally higher tendency to ignore a tornado warning than
those participants without episodic, (low episodic) memories. After the tornado
stimulus, all participants reported a greater likelihood to engage in precautionary action
than reported prior to the stimulus. We also find that: 1) those participants with low
episodic memory showed greater precaution than the high episodic memory group; and
2) participants with greater knowledge of tornadoes showed the greatest gains in
anticipated precautionary behavior. This study suggests that increasing a population’s

	
  

7	
  

general knowledge of tornadoes could result in greater individual precaution and overall
safety during a tornadic event.
1.1 Introduction
The National Weather Service (NWS) is tasked with providing the public with
information about marine and atmospheric conditions throughout the U.S. and its
territories. One specific role of the NWS is the forecasting of severe weather and the
administration of warnings in ”the United States, its territories, adjacent waters and
ocean areas, for the protection of life and property…” (NOAA 2010). Generating a
weather warning draws on a wide range of expertise involving science, technology,
social interaction, public policy, and related domains. Although significant effort has
gone into developing issuance criteria used to determine when a severe weather
warning is warranted (NOAA 2010), how the public utilizes warning information in the
face of a hazardous event is unclear (Lazo, et. al. 2009). Researchers have also
acknowledged the importance of prior experiences and knowledge in decision-making
during weather warnings, although their specific influences are not as well investigated
(Dillon 2011; Leiserowitz 2006). Current research involving other types of natural
hazards have also identified the relationship between knowledge/experience and
decision-making and believe it to be a reasonable approach when investigating
behaviors associated with natural hazard risk (Morss 2008 and Whitmarsh 2008).
This study specifically addresses the influence of prior experiences, termed
“episodic memories” here, and knowledge, termed “semantic memories” here, during an
artificial weather warning. Episodic memories represent actual life experiences, and
likely have both knowledge and affective components (Tulving 1972). Semantic

	
  

8	
  

memories are information about the world that has been learned through reading,
media, schooling, and other indirect experiences (Tulving 1972). Specifically, we
designed a classroom experiment that probed both their semantic and episodic
memories and gave participants a sense of immediacy, as if a real tornado warning
were being issued. This experiment addressed the central research question: “What
influences do varying levels of semantic and episodic memories have in influencing
decision-making about tornado warnings?” We hypothesized that those individuals with
personal memories of a tornado would be more inclined to take precautionary measures
than those without such experience; however, results did not support that outcome.
When an individual is faced with a risk, both types of memory play an active role
during analytical and experiential processing of associated decisions, since decisions
draw upon past memories (Marx 2007). Constraints on time, knowledge, and cognitive
load are vital when determining a course of action in risk situations. These impact an
individual’s potential options when responding to warnings by limiting the opportunity to
integrate past experiences and information into decision-making.
1.2 Theoretical Foundations
A number of studies have suggested a link between experiences, knowledge and
decision-making under risk exists. In fact, people have a tendency to make decisions
based on small amounts of previous experience (Hertwig & Pleskac 2010). Navigating
risk situations requires a complete understanding and assessment of the risk and needs
to take into consideration previous experiences and other sources of information
impacting the context of the situation (Kasperson 1996). When exposed to risk,
individuals call upon a wide range of tools in order to assess potential danger, and

	
  

9	
  

determine the appropriate behavior. Often the level of an individual’s knowledge of the
risk, as well as the fear associated with the actual event itself, will be considered before
taking action (Morrow 2009). People are exposed to a variety of risk situations as part
of their daily routine. These situations vary greatly, ranging from business decisions to
medical predicaments, all of which are impacted by experiences and knowledge.
Studies of business managers have shown that decisions often follow intuitive
paths, or are highly context dependent. Management decisions can also be based on
an individual’s fear of being wrong, lack of a common decision-making structure across
groups of individuals, or insufficient knowledge (Riabacke 2006). Medical decisionmaking is another area that can require individuals to make decisions under risk.
Pregnancy, for example, is a situation in which individuals are presented with
information typically categorized as a high or low risk. Prior experience with pregnancy
alleviates some of the unfamiliarity associated with decision-making around risks,
although differences in pregnancies require an individual to evaluate risk in the context
of the current medical condition (Lyerly et, al. 2007). Many of these same factors, such
as knowledge, previous experience, context, and the perceived level of risk influence
decisions encountered during exposure to natural hazards.
Decision-making under natural hazard risk is similar to risk encountered in other
situations. Uncertainty arises due to lack of knowledge, in this case of natural hazards,
as well as the unpredictability associated with the systems and processes that influence
natural hazards. The way in which these hazards impact society and an individual’s
choices related to the hazard is often difficult to ascertain due to the information and

	
  

10	
  

outcomes of past events that are available, many which may represent an imperfect
record of past events (UKCIP 2003).
Both previous experience, or episodic memory, and knowledge, or semantic
memory, lie at the core of decisions made during risk situations. As a result, this study
will take a closer look at how each impacts decisions made during exposure to risk
associated with tornado warnings.
For the purposes of this study, we will focus on specific memories, both episodic
and semantic, and attempt to determine the role they play when an individual is
exposed to tornado warnings. During decision-making tasks, people may draw on
semantic knowledge and/or episodic experience as they make decisions based on their
available memories. When attempting to determine the behavior associated with these
memories, it is difficult to predict the outcome since decidedly different scenarios may
occur despite similar past experience or knowledge. In this study, we hypothesized that
someone who has experienced a tornado will act upon the warning by heeding the
message to take precautionary measures. Conversely, those individuals who have not
previously experienced a tornado and have little knowledge of the actual threat may
choose to ignore the warning. These hypotheses will also take into account the impact
of an individual’s knowledge of severe weather and tornadoes, since it has been shown
to influence decisions (Johnson 1992), and the likelihood that an inverse relationship
between the risk and benefits associated taking precautionary measures during a
tornado warning may exist (Slovic 2000).

	
  

11	
  

1.3 Methods
A purposeful sample of students participated in this study, completing four
questionnaires and participating in a brief stimulus. The first was the Semantic Severe
Weather Questionnaire (SQ1), which determined the student’s level of knowledge about
severe weather. Next was the Episodic Severe Weather Questionnaire (EQ1), which
determined the student’s personal experience with severe weather, in general, and
specifically tornadoes. Following the completion of SQ1 and EQ1, participants
completed a four-point Likert scale Severe Weather Decision Questionnaire (D1),
followed by a slide show stimulus, then completion of the final Severe Weather Decision
Questionnaire (D2). Questionnaire data provided: 1) semantic knowledge of severe
weather (SQ1); 2) personal experience with severe events and tornadoes (EQ1); and 3)
potential responses to tornado warnings, collected both before (D1) and after (D2)
viewing of a slideshow stimulus consisting of tornadoes and related damage. The intent
of the stimulus was to determine the impact of a recent experience on decision-making,
by either enabling prior recollection, or creating a recent experience.
1.3.1 Participants
Forty-nine undergraduate students at a large Midwestern institution participated
in a one-hour, cognitively based experiment focusing on decision-making during a
regularly scheduled classroom time and location. The experimental population averaged
22 years in age and was composed of male (n=21) and female (n=28) participants. The
population consisted of varying levels of academic completion (freshman, sophomore,
junior and senior), all of whom were non-science majors.

	
  

12	
  

1.3.2 Data Collection
Data were collected in four successive steps during class (Figure 1). Participants
were introduced to the cognitive definitions of semantic and episodic memory, given a
brief explanation of the experiment, and informed that participation was voluntary. A
two-page semantic questionnaire (SQ1) was then administered. The questionnaire
consisted of five demographic, three informational and ten weather knowledge
questions mainly centered on severe thunderstorms and tornadoes. The episodic
questionnaire (EQ1) followed and consisted of seven questions focusing on personal
experience with severe weather and tornadoes. After completing the semantic and
episodic questionnaires, the students were asked to complete a decision-making
questionnaire (D1). The decision-making questionnaire prompted students to consider
their actions during a tornado warning, and listed ten potential scenarios of varying
degrees of precautionary measures for students to choose from, and posed two openended questions. D1 was intended to determine actions participants would most likely
take during a tornado warning. Upon completion of these questionnaires, the
participants viewed a 5-minute slide show stimulus containing a series of images of
actual tornadoes and associated property damage. Participants were asked to immerse
themselves in the stimulus as if they and their families were actually experiencing the
tornadoes both physically and emotionally. The slide show stimulus was intended to
invoke a personal experience, or act as a priming event in the case of those with
previous tornado experiences. After viewing the slide show, the students responded
anew to the decision-making questionnaire (D2). The intent of D2 was to determine the
impact of the stimulus on the decision-making process. The session ended with a

	
  

13	
  

deeper explanation of the experiment and a question-and-answer period with the
participants.
Figure 1.1: Experiment progression and timing of the questionnaires and slide show
stimulus.

We investigated the factorability of the ten items contained within the decisionmaking instrument (D1) through exploratory factor analysis (Table 1.1). Criteria
demonstrating factorability were met, including a Kaiser-Meyer-Olkin measure of
sampling adequacy of 0.733, above the 0.6 value recommended for factor analysis.
Bartlett’s Test of sphericity was significant (χ2 (45) = 257.4, p < 0.001). Communalities
were at or above 0.5, except for one just below that value, indicating shared variance
with other items. Given these data, exploratory factor analysis was performed on all ten
items. A one factor solution was indicated by scree plot analysis, while eigenvalue
analysis suggests a three factor solution. Ultimately, after considering the strength of
factor loadings, number of items per factor, and internal consistency of factors, we
retained one scale, the “Ignore Warning Scale”, which contains four items (5,6,7,9;
	
  

14	
  

Table 1.1) and which explains 80.1% of the variance in these four items. This scale has
high Cronbach’s alpha of 0.90. A high score on this scale indicates that a respondent is
likely to ignore a tornado warning, while a low score indicates a less likely probability
that they would ignore the warning. Confirmatory factor analysis on the same items
after the stimulus (D2) confirms the stability of this scale, with equivalent factor loadings
and alpha values at both administrations.
Table 1.1: Factor loadings and communalities for the Ignore Warning Scale. The
strength of the factor loadings and number of items per factor suggest the retention of
one scale, the “Ignore Warning Scale”.
Item
Factor Loading
Communalities
Ignore warning since you see no
.928
.861
apparent danger
Ignore warning since you assume
.941
.885
you are safe in your current location
Ignore warning since there’s a small
chance that you will be hit by a
.931
.867
tornado
Wait for sirens to sound before
.664
.441
seeking shelter
1.3.3 Data Analysis
Participant questionnaire responses were collected and analyzed quantitatively.
First, we describe the results of the semantic and episodic questionnaire using simple
descriptive statistics. Second, a Related Samples Wilcoxon Signed Rank Test was
performed on the decision results both before and after the stimulus. The test is useful
when investigating any change in scores from one time point to another when
individuals are subjected to more than one condition, which in this case was the
stimulus. We considered results for four different populations: the high semantic, high
episodic group, the high semantic, low episodic group, the low semantic, high episodic
group, and the low semantic, low episodic group. Based on these results, a multivariate

	
  

15	
  

analysis of variance (MANOVA) was performed to determine the significance of the
main and interaction effects suggested by the Related Samples Wilcoxon Signed Rank
Test results.
1.3.4 Validity and Reliability
Quantitative research enables a researcher to ask specific questions, generate
hypotheses, collect data in a controlled setting, and subsequently test hypotheses
(Golafshani 2003). Validity in quantitative research is dependent on the ability to
measure and observe the intended variables and reliability is implied when the
experiment and resulting data can be replicated. Criteria for validity and reliability of our
survey instruments and study design have been established. We explain each of these
criteria in detail; see Clark and Libarkin (2011) and references therein for more
information. Note that overall, the findings of the current study are limited to collegeaged populations in the Midwest.
Content validity is the extent to which questions are actually measuring the
construct intended. Often, experts are asked to comment on item validity. In this case,
materials generated by two expert organizations, NOAA and NWS, were used in
developing all three survey instruments. The D1 and D2 utilized information from the
NOAA and NWS tornado preparedness guidelines (NOAA 1995). The EQ1
incorporated items modified from the Enhanced Fujita Scale (NOAA 2009). The SQ1
included items generated by NOAA and NWS (NOAA 1995 and NWS 1995), as well as
misconceptions identified in the research literature (NOAA 1995). All three surveys
were revised based on comments from two geoscientists from the Geocognition
Research Laboratory.

	
  

16	
  

Conclusion validity is the ability to determine the relationship between variables
being researched. Experts exposed to the findings of this research through formal
presentations at professional conferences found the results noteworthy and deserving
of further investigation. In general, experts agreed that our finding of a relationship
between knowledge/experience and decision-making was enlightening; this concurs
with current research on other types of natural hazards (Morss 2008 and Whitmarsh
2008).
Construct validity refers to the ability of instruments to correlate with the
underlying theory of the study. Current understanding of the importance of knowledge
in decision-making suggests that higher knowledge should lead to more effective
decision-making (Johnson 1992). In general, participants with higher SQ1 scores
displayed more effective tornado-warning decisions relative to those with lower SQ1
scores. This effect was mitigated by the impact of personal experience, the focus of this
study.
Reliability of the instruments was evaluated through demonstrated consistency of
tests results with similar populations utilizing comparable instruments. The small
sample size limits internal reliability considerations, although ongoing work will allow for
comparison of findings across populations.
1.4 Results
Results of the Severe Weather Semantic Questionnaire (SQ1) indicate that
participants were on average “somewhat likely” to consult a weather forecast as part of
their daily routine. Participants checked weather forecasts an average of 4.9 times
each week (approximately once daily), and referenced a wide range of sources for their

	
  

17	
  

information about current weather (Figure 1.2). Results also indicate that 63% of
participants understood the difference between the National Weather Service (NWS)
severe weather statements. The two most prominent statements in this study, tornado
watch and tornado warning, should prompt different levels of action; a watch suggesting
conditions exist for tornado development, and a warning indicating a tornado exists.
While the majority of participants understood that a “warning” was the most urgent NWS
statement and required immediate personal action, a substantial number of participants
(37%), were oblivious to the difference, and would not realize the need for taking
precautionary measures during a tornado warning. Based on the results of SQ1,
participants were divided into two groups determined by their SQ1 score. Those
participants who averaged below the overall average, which happened to be 50%, were
classified as low semantic (LS), and those who scored above 50% were classified as
high semantic (HS).
Figure 1.2: Participants primary source of weather information. The Internet was the
primary source of weather information, followed by TV with friends and family (F/F)
ranking third.

	
  

18	
  

Results of the Episodic Severe Weather Questionnaire (EQ1) indicate that
students had a significant amount of experience with severe weather and tornadoes.
Responses to two questions, “Have you personally experienced a severe weather
event” (EpiQ3), and “Have you personally experienced a tornado” (EpiQ5), indicated
that 88% of the participants had personal experience with severe weather warnings and
events, while slightly fewer than 50% had experienced a tornado (Table 1.2). These
questions were used as indicators of overall severe weather and tornado episodic
experience. Participants who indicated no experience with tornadoes were classified as
low episodic (LE), and those participants who indicated experience with at least one
tornado were classified as high episodic (HE). Since this study concerns itself
specifically with tornado experience, the statistical analysis incorporated only responses
to EpiQ5, which specifically asks the participant to state their tornado experience.
Table 1.2: Participants (n=49) and their personal lifetime warning and event
experiences. Results indicate a significant experience with severe weather warnings
and a fair amount of experience with tornadoes.
Severe Weather Event

Tornado Event

% of participants

Severe Weather
Warning
88%

84%

47%

# of experiences

3.4 per year

31.0 lifetime

1.4 lifetime

The main and interaction effects of knowledge of severe weather impacts and
varying levels of personal experience with tornadoes on decision-making before and
after a simulated warning (stimulus) were considered. Essentially, the interaction
allowed for insight into the impact of knowledge and experience when students were
faced with a decision that involved ignoring a tornado warning. The impact of the
interaction emphasized the greater likelihood of participants in the high semantic group
towards taking precautionary measures during tornado warnings (Figure 1.3).

	
  

19	
  

Figure 1.3: D1 and D2 results showing the interaction of experience and knowledge
when faced with the decision of ignoring a tornado warning.

For the Wilcoxon Signed Rank Test, we tested the hypothesis that no difference exists
between the mean of D1 and D2. The test is often used when comparing two related
samples, which in our case is D1 and D2. The test results indicated that those
possessing low episodic and low semantic memories and low episodic and high
semantic memories both experienced significant impacts due to the stimulus. Those
possessing high episodic and low semantic memories and high episodic and high
semantic memories experienced only moderate change after the stimulus (Table 1.3).
Table 1.3: Results of the Related Samples Wilcoxon Signed Rank Test indicating that
the stimulus had significant impact on the low episodic/low semantic and low
episodic/high semantic groups.
Related Samples Wilcoxon Signed Rank Test
Group
N
D1
D2
Statistical significance
Low episodic
11
1.5
1.1
p< 0.016
Low semantic
Low episodic
14
1.1
0.62
p < 0.007
High semantic
	
  

20	
  

Table 1.3 (cont’d)
High episodic
7
Low semantic
High episodic
16
High semantic
All
47

1.0

0.89

p < 0.180

1.4

1.1

p < 0.066

1.3

0.93

p < 0.001

A multivariate analysis of variance (MANOVA) was completed in order to assess the
statistical significance of the independent variables “episodic” and “semantic” and the
interaction term “episodic*semantic” on the dependent variables D1 and D2. This was
chosen since the MANOVA allows us to investigate whether changes in the
independent variables impact multiple dependent variables, and offer insight into the
interactions between dependent and independent variables.
MANOVA results suggested that the main effects, semantic and episodic memory were
not significant with semantic D1=.953, D2=.638, and episodic D1=.654, D2=.521.
However, the interaction of semantic and episodic, D1=.128, D2=.174, memory may be
considered somewhat significant given the relatively small number of participants and
the overall difference in significance indicated by the test (Table 1.4).
Table 1.4: Results of the MANOVA analysis suggesting the main effects were not
significant but the interaction of the two were somewhat significant given the small
number of participants.

	
  

21	
  

In all, participant responses to SQ1 and EQ1, question EpiQ5, indicate
interesting interactions between knowledge and prior tornado experience and
anticipated responses to tornado warnings. These responses were compared using the
previously identified “choose to ignore” warning scale. Participant decision-making prestimulus (D1) indicates that the HE group with a high level of knowledge (HS) was more
likely to take precautionary measures during a tornado warning than the HE group with
a low level of knowledge (LS), while the LE group with a low level of knowledge (LS)
was more likely to take precautionary measures than the LE group with a high level of
knowledge (HS). The interaction between semantic and episodic memory was
significant after the stimulus. Participant decision-making post stimulus (D2) resulted in
interesting movements among the groups. The HE group with a high level of
knowledge (HS) was found to be more likely to ignore a tornado warning than the HE
group with a low level of knowledge (LS), while the LE group with a high level of
knowledge was less likely to ignore a tornado warning than the LE group with a low
level of knowledge (LS). It appears that tornado experience may provide impetus for
participants to take precautionary measures, especially when combined with a high
level of knowledge of severe weather/tornadoes (Table 1.5). It is not known, however,
how long the impact of recent memories, or priming of existing memories, of tornado
experiences provides this effect.
Table 1.5: Pre-stimulus and post stimulus results illustrating the participants likelihood of
ignoring a warning for both D1 and D2.
D1 Pre-stimulus
D2 Post stimulus
Likelihood of
Likelihood of
Ignoring
Ignoring
Experience
Knowledge
Experience Knowledge
Tornado
Tornado
level
level
level
level
Warning
Warning
1=more likely
1=more likely

	
  

22	
  

Table 1.5 (cont’d)
High episodic
High episodic
Low episodic
Low episodic

High
semantic
Low
semantic
High
semantic
Low
semantic

High
episodic
High
episodic
Low
episodic
Low
episodic

3
1
2
4

High
semantic
Low
semantic
High
semantic
Low
semantic

1
3
4
2

1.5 Discussion and Conclusions
Data analyzed from the study points to interesting interactions between
knowledge and experience. Varying levels of severe weather knowledge coupled with
instances, or lack of, personal tornado experience allow for a wide range of potential
outcomes when an individual is faced with a tornado warning. Although the study was
limited by availability of prior research in the topic area as well as sample size, (n=49),
and the relatively narrow scope of the student demographics, the results are still
considered attributable to cognitive and demographic factors that substantiate the need
for further research using refined methods and more diverse participants.
In general, this study suggests that participants with a higher level of semantic
knowledge (HS) will be more likely to take precautionary measures during a tornado
warning, than those who scored lower on the semantic questionnaire (LS). In addition,
students who have previously experienced a tornado (HE) are the least likely to react
favorably to a warning if they possess less knowledge of severe weather. This may be
due to the possibility of the students discounting the apparent danger and relying on the
greater availability of relatively harmless past experiences. After the stimulus, the
episodic group increased their likeliness to react to the warning, but only slightly. The
results tend to support the likelihood of semantic memories, or knowledge, providing

	
  

23	
  

greater impetus for action during a severe weather event, when compared to only
episodic memories, or experiences of tornadoes.
Both semantic and episodic memories play an important role in an individual’s
decision-making during risk situations. Though decision-making is often assumed to
rely more on the recollection of prior experiences, consideration should be given to the
impacts of an individual’s severe weather knowledge. In this study, participants were
faced with a decision-making task both before and after viewing a 5-minute slide show
stimulus of tornadoes and related storm damage. The results indicate a strong
correlation between the level of severe weather knowledge and the likelihood to heed
the warning associated with the event. Overall, previous tornado experience (HE)
indicated a greater tendency to ignore a tornado warning than those who had not
experienced a tornado event.
The stimulus itself provides tantalizing suggestions about the structure that more
effective tornado warnings could take. In this study, the stimulus consisted of tornado
footage and related storm damage that may have impacted the warning response
among participants. While it has been shown that imagery plays a part in decisionmaking (Leiserowitz 2006), it is not the intent of this paper to determine the impact of
the portrayal of tornado images upon the students as an affective component of
memory. Generally both D1 and D2 results illustrate that individuals with greater
knowledge of severe weather events were less likely to ignore the tornado warning,
while those possessing previous tornado experiences were more likely to ignore the
warning, especially if they also had a low level of severe weather knowledge. While it is
possible that a crossover effect due to the short lapse of time between D1 and D2 may

	
  

24	
  

have impacted the likelihood of participants to ignore a tornado warning, it is beyond the
scope of this study to determine how separate treatments may have altered the results,
and is better determined by further study. Recency of prior tornado warnings may
impact the likelihood of action since recent events, even those without an actual tornado
event, are more salient in an individual’s memory than a memory of an actual tornado
experience in the past. If prior tornado or other severe weather events were relatively
minor in intensity, individuals may use this information by anchoring their judgments on
information/experiences that are not relevant to their current situation (Newell 2010).
Greater knowledge of severe weather provides participants with additional means to
determine when precautionary measures should be taken during severe weather. The
conclusions reported in this study reinforce the importance of memories on decisionmaking behavior. Specifically, knowledge is a critical factor when one is considering
taking precautionary measures during a tornado warning. Modifying an individual’s
knowledge, or semantic memory would appear a logical step in influencing
precautionary behavior since this could be accomplished through a variety of
educational or media avenues. Attempting to modify an individual’s personal
experience, or episodic memory would be much more difficult, if not impossible since
this would require direct experience with a tornadic event. Since knowledge appears to
be a significant factor based on the results of this study, future research will address the
limitations of the current study by providing an educational intervention to control for
differences in severe weather knowledge among participants.

	
  

25	
  

This information, coupled with an understanding of the impacts of additional variables
utilized during risk decisions play a significant part in determining future directions in the
development of tornado warning practices.
ACKNOWLEDGEMENTS
This work was completed while in residence at the Geocognition Research Laboratory
at Michigan State University. I thank Emily Geraghty Ward and Julie Libarkin for
assistance with this paper, as well as all students who graciously participated in this
research.

	
  

26	
  

APPENDICES

	
  

27	
  

Appendix 1A: Semantic Questionnaire (SQ1)

	
  

28	
  

	
  

29	
  

Appendix 1B: Episodic Questionnaire (EQ1)

	
  

30	
  

Appendix 1C: Decision-making questionnaire (D1 and D2)

	
  

31	
  

Appendix 1D: Statement of Reprint
At the time of submission of this dissertation, Chapter 1, “Memory and decision-making:
Determining action when the sirens sound” on pages 7 through 32 had been previously
published in the Journal of Weather, Climate and Society on January 2013.
Drost, Robert, 2013: Memory and decision making: determining action when the sirens
sound. Wea. Climate Soc., 5, 43–54.
doi: http://dx.doi.org/10.1175/WCAS-D-11-00042.1
©American Meteorological Society. Used with permission.

	
  

32	
  

REFERENCES

	
  

33	
  

REFERENCES

Clark, S. K., and Libarkin, J.C., 2011: Designing a mixed-methods research instrument
and scoring rubric to investigate individuals’ conceptions of plate tectonics. The
Geological Society of America, Special Paper 474.
Dillon, R., Tinsley, C., and Cronin, M., 2011: Why Near-Miss Events Can Decrease an
Individual’s Protective Response to Hurricanes. Risk Analysis, vol. 31, no. 3.
Golafshani, Nahid. 2003: The Qualitative Report. Volume 8, Number 4, p. 597-607.
Hertwig, R., Pleskac, T., 2010: Decisions from experience: Why small samples?
Cognition, 115, p. 225-237.
Johnson, B., 1992: “Advancing Understanding of Knowledge's Role in Lay Risk
Perception,” American Sociological Association.
Kasperson, R., Kasperson, J., 1996: The Social Amplification and Attenuation of Risk.
The Annals of the American Academy, vol. 545.
Lazo, J. K., Morss, R. E., and deMuth, J. L., 2009: 300 Billion Served
Sources, Perceptions, Uses, and Values of Weather Forecasts. BAMS, p. 785-798.
Leiserowitz, A., 2006: Climate Change Risk Perception and Policy Preferences: The
role of Affect, Imagery, and Values. Decision Research, 1201 Oak Street, Suite
200, Eugene, OR 97401.
Lyerly, et. al., 2007: Risks, Values, and Decision Making Surrounding Pregnancy.
Obstetrics & Gynecology, vol. 109, no. 4.
Marx, S. M., Weber, E. U., Orlove, B. S., Leiserowitz, A., Krantz, D. H., Roncoli, C., and
Phillips, J., 2007: Communication and mental processes: Experiential and
analytic processing of uncertain climate information. Science Direct, Global
Environmental Change 17, p. 47–58.
Morrow, B., 2009: Risk behavior and risk communication: Synthesis and expert
interviews final report for the NOAA Coastal Services Center. SocResearch.
Morss, et al., 2008: Communicating Uncertainty in Weather Forecasts: A Survey of the
U.S. Public. Weather and Forecasting, vol. 23, February, 974-991.
National Oceanic and Atmospheric Administration (NOAA), 2010: The official National
Weather Service Mission Statement. www.noaa.gov.

	
  

34	
  

National Oceanic and Atmospheric Administration (NOAA), 1995: Tornadoes…Nature’s
Most Violent Storms. A Preparedness Guide, including Safety Information for
Schools.
National Oceanic and Atmospheric Administration (NOAA), National Weather Service,
2009: National Centers for Environmental Prediction, Storm Prediction Center.
National Weather Service (NWS) Southern Region, 1995: JetStream – An Online
School for Weather. www.srh.noaa.gov.
Newell, B.R, Pitman A. J., 2010: The Psychology of Global Warming. BAMS, p. 10031014.
Riabacke, A., 2006: IAENG International Journal of Computer Science, 32:4.
Slovic, P., 2000: The Perception of Risk. Earthscan Publications Ltd.
Tulving E., 1972: Organization of memory: Episodic and semantic memory. Academic
Press, p. 381-403.
UKCIP Technical Report 2003: Climate adaptation: Risk, uncertainty and decisionmaking.
Whitmarsh, L., 2008: Are flood victims more concerned about climate change than
other people? The role of direct experience in risk perception and behavioral
response. Journal of Risk Research, vol. 11, no. 3, p. 351–374.

	
  

35	
  

Chapter 2
GSA members on climate change: Where, what, and ways forward?
2.1 Introduction
Climate change is one of the most pressing environmental, economic, and
societal issues of the 21st century. Addressing climate change issues is difficult partly
due to the disconnect between the scientific community and the public’s understanding
and perception of climate change issues. A number of studies have examined the views
of the public and suggest that basic knowledge about climate change is limited and that
many believe there is still no agreement in the scientific community about the possible
causes and impacts (Hamilton, 2011; Whitmarsh, 2009). Fewer studies have looked at
the broad scientific community, but those that have report that climate scientists who
understand the climate process generally accept that anthropogenic climate change
exists and agree that human activity has had a profound impact on Earth’s climate
(Doran, 2009; Oreskes, 2004). However, in order for vital information to be passed onto
the policy makers and voters, the scientists must put forth the effort to inform. Included
in that effort is the need to communicate the agreement scientist’s share on climate
change factors. Studies indicate that only 47% of the American public believes that
there is scientific consensus on climate change (Doran, 2009); this belief needs to be
addressed if scientist’s views are to be accepted by the public. Here we look at the
perceptions of Geological Society of America (GSA) scientists on climate change to
understand: what are their biggest concerns, what regions of the USA will be most
impacted, and how do we bridge the gap between scientists and the public?

	
  

36	
  

2.2 Methods
Surveys were collected in the exhibit halls of the GSA Annual Meetings in 2009
and 2011. GSA has over 20,000 members and each annual meeting attracts nearly
6,000 geoscientists (www.geosociety.org). We collected 181 surveys. Forty-nine
percent of participants were female, and ages ranged from 19 to 70 years. The brief
two-page survey varied slightly between the two years but both asked participants to
indicate on a map where they believe climate change will have the most impact. They
were then asked to answer questions (seen in Table 2.1) about the region they
indicated.
Table 2.1: Survey questions and response information. Please contact authors for
access to full survey results.
Question
Response Type
Shade in one region on the map below that you feel has been or will
Shading/Circling
be impacted by climate change.
This region has been or will be severely impacted by climate change. Likert Scale
I believe the general public is sufficiently informed about the impacts
Likert Scale
of climate change in this region.
Describe the climate change impact(s) that the region you shaded
Open-ended
has or will experience.
Explain what you believe would be the most effective way to increase
Open-ended
public understanding of climate change.
2.3 Results
The most common response to the open-ended question about the impact of
climate change was sea level rise (32%) followed by more severe weather (22%) and
water resource issues (20%). These responses make sense since people generally
show the most concern for “salient, palatable” risks (Seacrest, 2000). The remaining
responses were varied and specific with the next most common being agricultural shifts,
both spatially and temporally (6%). Geographically specific impacts included pine-beetle
expansion and loss of the maple-syrup industry. In all 181 surveys, there were only two

	
  

37	
  

“climate change skeptics” that clearly stated they do not believe anything will occur
because climate change is not happening.
Circled regions from the surveys were digitized into ESRI ArcGIS to visualize the
overall regions of concern. Figure 2.1 represents the overall density of regions circled
across all participants. The focus on coasts is consistent with the impacts given on the
open-ended portion of the survey, including sea level rise and and increase of
hurricanes. Water resource issues are also reflected in the focus on the southwestern
United States. There were minor differences between years based on the location of the
meeting (more focus on the northwest in 2009 and the northern midwest in 2011).
These signals, however, were relatively insignificant compared to the concern over the
coasts.
Figure 2.1: Image showing the regions of most climate change concerns among GSA
respondents. Darker orange areas are where more respondents shaded in.

2.4 Discussion and Conclusions
Survey results indicate that 89 percent of the respondents believe that climate
change presents a significant risk to the public, whereas only about half the general
population is concerned. This difference in perceived risk may be influenced by a
number of potentially mitigating factors shared by both the respondents and the public.

	
  

38	
  

These include personal experiences with climate change (Whitmarsh, 2009) as well as
social and demographic factors (Leiserowitz, 2006). Perhaps the difference is rooted in
the scientist’s understanding and acceptance of the evidence, which may be immune to
influences by informational sources available to the public. Although recent studies
have demonstrated varying beliefs in climate change by Americans (Hamilton, 2011;
Whitmarsh, 2009), substantial doubt and lower perceived risk of climate change still
remain among the population.
As the public’s trusted source, what do these scientists think of the current state
of the public’s awareness of climate change? Most respondents (84%) believe the
public is not adequately informed on the potential climate change impacts in the United
States. This position is reflected by recent studies that indicate the American public is
not well informed on climate change issues (Malka et al., 2009). Although potentially
alarming, the geoscientists were forthcoming with possible solutions to increasing public
awareness.
The majority of survey respondents (52%) believe the public is best informed
through educational means, varying from formal K-12 education to specific public
outreach programs delivered through a variety of methods to enable the greatest
coverage. The remainder of responses varied in the delivery mechanism of climate
information to the public. Some believed in a pure source of information derived from
the scientific community, while others felt the government should take a role in
disseminating the information in an understandable public format. Interestingly, 6% of
the respondents indicated that an actual climate related disaster would serve best to
wake up the public to the risk associated with climate change in their respective region.

	
  

39	
  

Although drastic in comparison with more reasonable alternatives, the impact of
disasters and national attention focused on these events has the tendency to grip the
public’s scrutiny in an immediate and urgent manner. The link between scientists and
the public thirst for knowledge is an opportunity for the geoscience community. The
public generally relies on the media to navigate science-based issues, ranging from
local weather to complex information about geo-happenings, including climate change.
Since scientists usually generate this information, a more direct connection between the
media and scientists, or perhaps alternative methods of providing for the interaction
between scientists and the public would be beneficial. Malka et al. (2009) show that
nearly three-quarters of the public relies on scientists for information because the
complexity and number of issues is too much to fully grasp without conducting research
oneself.
Survey data demonstrates that climate change concerns among geoscientists
are consistent and aligned with current climate science. This community has great
potential to influence public awareness and understanding of climate issues by acting in
unison (Anderegg et al., 2010) and reinforcing the public’s trust (Hamilton, 2011;
Whitmarsh 2009). The GSA’s official position statement on climate change highlights
the opportunities available to members in order to help this cause. These include
participating in professional education, engaging in public education activities,
collaborating with stakeholders, working with other science and policy societies, and
utilizing the most up-to-date sources of climate science (GSA 2010). The impacts of
climate change range from local communities to the global population. With
overwhelming consensus, and armed with the best science, each member of the

	
  

40	
  

geoscience community can find their niche in moving the public toward better
understanding of the risks and solutions for the changing climate.

	
  

41	
  

APPENDIX

	
  

42	
  

Appendix 2A: Statement of Reprint
At the time of submission of this dissertation, Chapter 2, “GSA members on climate
change: Where, what, and ways forward?” on pages 32 through 39 had been previously
published in the Geological Society of America’s publication, GSA Today on January
2013.

Drost, Robert E., and Sheldon P. Turner. "GSA members on climate change:
Where, what, and ways forward?" GSA Today 23.1 (2013).
doi: 10.1130/GSATG157GW.1
©Geological Society of America. Used with permission.

	
  

43	
  

REFERENCES

	
  

44	
  

REFERENCES

William R. L. Anderegg, James W. Prall, Jacob Harold, and Stephen H. Schneider,
2010. Expert credibility in climate change. PNAS 107(27) 12107–12109.
Ariel Malka, Jon A. Krosnick, and Gary Langer, 2009. “The Association of Knowledge
with Concern about Global Warming: Trusted Information Sources Shape Public
Thinking.” Risk Analysis 29, 633-647.
Bray, D., von Storch, H., 2010. A Survey of the Perspectives of Climate Scientists
Concerning Climate Science and Climate Change. GKSS 2010/9.
Doran, P. T., 2009. Examining the Scientific Consensus on Climate Change. EOS, Vol.
90, No. 3.
Geological Society of America. 2010. “GSA Position Statement - Climate Change”.
Retrieved from http://www.geosociety.org/positions/position10.htm
Hamilton, L., 2011. Climate Change Partisanship, Understanding, and Public Opinion,
Casey Institute Issue brief No. 26.
Leiserowitz, A., 2006. Climate Change Risk Perception and Policy Preferences: The
role of Affect, Imagery, and Values. Decision Research, Eugene, OR.
Oreskes, N., 2004. Beyond the ivory tower: The scientific consensus on climate change.
Science, 306 (5702), 1686.
Seacrest, S., Kuzelka, R., Leonard, R., 2000. Global climate change and public
perception: The challenge of translation. Journal of the American Water
Resources Association. 36(2), 253-263.
Whitmarsh, L., 2009. What’s in a name? Commonalities and differences in public
understanding of “climate change” and “global warming”, Public Understanding of
Science, 18, 401–420.

	
  

45	
  

Chapter 3
Gesturing During Weathercasts: A “hands down” Best Practice?
Televised media is one of the most frequently accessed sources of weather
information. The local weathercaster serves as the vital link between the viewing public
and weather data and associated forecast models. As such, weathercaster
characteristics, from vocal cadence to physical appearance, can impact viewer
understanding. This study considers the role of weathercaster gesturing on viewer
attention during weather forecasts. Two variations of a typical weather forecast were
viewed by a total of 36 participants during an eye tracking experiment. The first forecast
variation contained physical gestures towards forecast text by the newscaster (Gesture
condition), while the second variation contained minimal gesturing (No Gesture
condition). Following each eye tracking experiment, participants completed a retention
survey related to the forecast. The resulting data were used to identify specific areas of
interest that participants attended to during viewing and to ascertain how well the
forecast was retained across both conditions. Study results suggest that the
weathercaster’s gesturing during forecasts impacted viewer attention, but did not affect
retention of weather information. Gesturing diverted attention from other areas of
interest within the forecast by encouraging participants to focus on the weathercaster’s
hands. This study indicates that minor modifications to weathercaster behavior can
produce significant changes in viewer behavior. In practice, it appears acceptable to
gesture in situations where there are few areas that may divert viewer attention from
salient to non-salient information. However, in situations where viewer attention needs
to be focused on salient information, gesturing is not recommended.

	
  

46	
  

3.1 Introduction
Recent studies indicate the general population accesses weather information
through personal observation, newspapers, radio, television, and the internet (Lazo, et.
al. 2008 and Drost 2013). Additional work indicates that television is still the most
popular source for weather, traffic, and breaking news despite the availability of many
other media outlets (Pew Research 2011, Harris 2007 and Lazo 2002). As a source of
this information, network or cable television is accessed almost daily in most
households, particularly for newscasts or weather (Lazo, et. al. 2008). As a
consequence, the local or national weathercaster plays a vital role in disseminating
weather, climate, and even general science to the general public (Morrow 2008 and
NRC 2003). With almost 3,000 television stations in the United States (ERI 2011), the
delivery and style of weather forecasts is also highly variable.
Effective transfer of information during weather forecasts may be impacted by a
variety of factors such as how text, animations, pictures and color are used within the
forecast (Morss 2008). Forecasts may utilize text, visuals, or animations to deliver
information, although most rely on the weathercaster speaking directly to the audience
(Trobec 2007). The forecast itself may incorporate technical terms, employ humor, or
use knowledge of local places to relay weather information (Socci 2007). In addition to
the actual delivery of forecast information, elements such as station logos, live video
footage and informational text scrolls may also be displayed to viewers (WMO 2005).
These varying elements and styles of delivery may all potentially impact the
viewing audience’s perception and retention of weather forecast information (Stewart
2006). Previous research has utilized eye-tracking studies to ascertain the roles of

	
  

47	
  

these elements and their impact on viewers learning and retention (Stark et. al. 2007)
and to determine the impacts to comprehension when combining conventional text with
a variety of reinforcing imagery (Yaros & Cook 2011). These studies (Stark et. al. 2007
and Yaros & Cook 2011) confirm that visual representations tend to reinforce viewing
time and fixation on salient features when information is presented both visually and in
text. In addition, individuals spend more time on specific information being presented
when reinforced through the use of gesturing (Louwerse and Bangerter 2000). .
This recording of human eye movements with eye-tracking software provides
insight into aspects of fields of view that attract (or distract) viewer attention. Eye
movement data captures the punctuated progression of eyes across stimuli in the field
of view. Eye movements dart from feature to feature in jerky actions called saccadic
movements (Liversedge and Findlay 2000) that ultimately fall in the intervals between
fixations. A fixation (Table 3.1) occurs when eye movement pauses on a specific area of
the visual field for more than a defined length of time (Tobii 2010), suggesting attention
to an underlying feature. This attention can be inferred to relate to cognitive processes
(Just and Carpenter 1976).
In general, fixation data can be used to infer which aspects of stimuli attract
attention, while other characteristics of eye tracking data provide supporting evidence
for interpreting cognitive mechanisms or rationales (Table 3.1). Of particular interest are
areas of interest (AOI), established based on the content of the forecast and the
weathercaster’s on-screen position depending on which condition was being viewed.
AOI may also be determined by evaluating the number of fixations on a certain area

	
  

48	
  

such as a graphic element, and may be indicative of a participant’s interest in that
element.
Fixation duration may also be used to determine important AOI, providing
evidence for which stimuli are most attractive to users, or possibly most confusing
(Waterworth et. al. 2003). Fixation count represents the actual number of fixations within
an AOI and often impacts other fixation data in the form of higher measured results. In
the form of a graphical output, gaze plots can be used to determine the movements and
positions where the participant was looking during the entire eye tracking session, while
heat maps provide for a visual representation using cool to warm colors (typically green
to red) that indicate how long or how often the participants fixated on an image.
Characteristics of gaze plots such as total path length, time spent on fixations
within an AOI, average fixation time, and the number of occurrences in which a viewer
crossed their own gaze path, are quantifiable and thus make viewer attention on an
image or video comparable. From this data we can ascertain which AOI were viewed
first, which attracted or deflected attention, and the length of time over which each AOI
was viewed. Heat maps representing the visual generalization of gaze data can be
useful in qualitatively identifying which image or scene commanded viewer attention.
Areas of most intense focus are normally represented in red. Apparent heat map
intensities can be visually compared, but resulting qualitative metrics are biased since
the analysis of the patterns is often based on the individual interpretation and should be
replaced using quantitative metrics when possible (Ehmke & Wilson 2007).
This research focused on participant interest and retention of specific forecast
elements (AOI) utilizing gaze data collected during an eye tracking session. The data

	
  

49	
  

were used to identify the link between where and how long a participant looked at the
forecast video and how these were impacted by weathercaster gesturing. The data
were then used to determine whether gesturing impacted participant retention of the
forecast elements contained within the AOI.
The current study considered the relationship between viewer attention and
information retention in response to actual weather forecast communication and
investigated the role of forecaster gesturing on viewer retention. The study builds upon
current research by integrating the outcomes related to the variety of factors that may
affect forecast transference and the impact of gesturing on viewer attention and
retention.
This study will attempt to answer the following research questions: 1) Does
gesturing by the weathercaster increase viewer attention to forecast elements? 2) Does
weathercaster gesturing cause viewers to redirect their attention to particular forecast
elements 2) Does gesturing impact viewer attention and retention of critical forecast
information? 3) How does gesturing, or lack of gesturing, impact viewer attention on
forecast elements?
Table 3.1: Definitions of gaze data used in this study.
Term
Areas of
Interest
Fixation
Fixation
Duration

Data Use as interpreted
by researcher
Identify participant interest
An element of interest contained
in specific forecast
within the forecast clip.
elements.
Also known as gaze cluster centroid: Identify visual attention on
a single location characterizing the
features within the field of
geographic center of a cluster of data view
The length of time a participant spent
fixated on a specific feature.
Determine visual attention
hierarchy
Calculated as the sum of fixations
within a specific feature.
Study definition

	
  
	
  

50	
  

Table 3.1 (cont’d)
Fixation Count

Gaze plots

Heat maps

Number of fixations per feature.
Indication of pattern of eye
movement around an image,
including intervals of no movement
and time spent looking at specific
areas
Spatial map of areas with longest
total gaze

Identify visual attention on
features within the field of
view
Ascertain elements viewed
first, elements that attracted
or deflected attention, and
image comprehension.
Indicate elements that are
particularly attractive.

3.2 Methods
Thirty-six undergraduate students at a large Midwestern institution participated in
the experiment. The mean age of the experimental population was 20 years (SD=1.056)
and was composed of male (n=10) and female (n=26) participants. 75% of the
population was Caucasian, 14% Asian, 8% African American. Participants were nonscience majors with and were paid $25 for their time.
3.2.1 Data Collection
Data were collected in three successive steps during the study. Participants first
completed a survey consisting of demographic questions related to age, gender,
ethnicity, and geoscience experience. Participants were given a brief explanation of the
eye tracking equipment and its function before completing a consent form and engaging
in the experiment.
The study was conducted as part of an hour-long eye tracking experiment that
consisted of multiple studies investigating participant’s familiarities and opinions
regarding climate change and geosciences. During this particular experiment,
participants viewed one of two versions of a nearly identical weekend weather forecast
of approximately 26 seconds in length and were told to watch as if observing an actual

	
  

51	
  

upcoming weekend forecast. In the No Gesture condition (Figure 3.1 - A), the
weathercaster is relatively motionless and speaking directly at the audience. In the
Gesture condition (Figure 3.1 - B), the weathercaster is seen gesturing with his hands
toward the on-screen weather forecast text. The Gesture condition consisted of 17
participants and the No Gesture condition consisted of 19 participants.
A Tobii T60 Eye Tracker was used to collect eye position, time, and validity data
every 1/60th of a second during the experiment. The Tobii T60 determines eye position
using multiple techniques including both bright and dark pupil tracking (Tobii 2011).
Sixty gaze data points were collected for each eye every second. Resulting gaze
calculations were determined by Tobii Studio 3.0 software designed for the eye-tracker
(Tobii 2010). The eye tracker captured viewer visual attention as they interacted with
the forecast. Immediately following the eye tracking session, participants completed a
retention questionnaire used to determine which elements of the forecast they were
able to remember (Appendix 3A). Each eye tracking and questionnaire session ended
with a brief explanation of the purpose of the experiments and an opportunity to ask any
general questions.
Figure 3.1: A - No Gesture condition forecast variation and B - Gesture condition
forecast variation. Both forecasts were identical except for the presence or absence of
the weathercaster’s gesturing. Forecasts provided by KELO-TV, Jay Trobec, Chief
Meteorologist.

	
  

52	
  

For the purpose of this study, seven Areas of Interest (AOI; Table 3.2), within the
forecast were identified: Station Banner, Days of the Week, Scene Image, Forecast,
Forecast Temperature, Weathercaster’s Hands, and Weathercaster’s Face (Figure 3.2 A and 3.2 - B; Table 3.2). Eye tracking data related to each AOI were used to
understand how interactions with each AOI change in the presence or absence of
gesture. ArcGIS, a mapping and spatial analysis software suite commonly applied to
geographic data (ESRI 2001), was used to link AOI with eye tracking data. Each AOI
was digitized using the common mapping tool in ArcGIS to create polygon and
shapefiles that represent each AOI. Each AOI shapefile was then overlain onto a frozen
frame of both forecast videos (Figure 3.2 - A and 3.2 - B). In this particular case, the
videos were relatively static since the weathercaster remained mostly motionless other
than the specific hand and limited head movement. This relative static positioning
allowed us to identify accurate AOI across the forecast video.
After viewing the video, participants completed a questionnaire assessing the
extent to which forecast information was retained (Appendix 3A). The questionnaire
contained seven questions, five related to weather and two related to the
weathercaster’s attire and screen background. Eye tracking data provided location,
duration, and the gaze track of the participants’ visual attention during the videos, while
the questionnaire data provided: 1) participants’ recollection of the weekend weather
forecast; 2) participants’ recollection of non-forecast information (shirt color and
background color).

	
  

53	
  

3.2.2 Data Analysis
Participant questionnaire responses were collected and analyzed quantitatively
(justification of methods are found in Appendix 3B). Eye tracking data was then
imported into ArcGIS and filtered using custom Python scripts. Raw eye tracking data
was filtered to remove invalid data. This can be defined as a poor validity measurement
for one or both eyes during instances of blinking or when the eye tracker cannot
constrain the eye position effectively. Olsson’s (2007) interpolation method was used to
interpolate gaze for intervals where invalid data was removed. The filtered gaze data
was then grouped into fixations to eliminate data points that represent instances when
the participants were not focusing their gaze. We chose to use Salvucci & Goldberg’s
(2000) fixation filter. This filter is based on spatial dispersion of the eye tracking data
and defines a cluster and being a set of sequential eye tracking points that are close in
spatial proximity. We used the same distance threshold of 35 pixels for clustering as per
Tobii Studio eye tracking software default. The minimum fixation time is generally
accepted by visual scientists to be 80ms (White 2008). Our eye tracking data were
collected every 1/60th of a second, requiring 5 sequential points to exceed 80 ms. Each
fixation has a geographic average location, known as the fixation or gaze. The number
of gaze cluster centroids and consequently the time spent within each AOI was then
summarized using ArcGIS’s Select by Location querying tool for each forecast variation.
Demographic and retention surveys were analyzed via simple descriptive
statistics and included a skewness and kurtosis analysis to determine the normality of
the study population. An independent samples T-test was performed to determine if
gender impacted retention questionnaire results; age and ethnicity of the study

	
  

54	
  

population varied only slightly and were not considered as covariates. The total number
of fixations, average fixation duration, and total fixation duration were compared for both
the No Gesture and Gesture conditions in order to determine differences in the amount
of time each group spent fixated on each of the seven AOI.
A Mann-Whitney U Test was performed on the two conditions, No Gesture and
Gesture, in order to determine which had greater participant fixation and fixation
duration for each AOI. This particular test is useful when investigating whether or not a
difference exists between two groups. In this case, the two groups would be the No
Gesture and Gesture forecast variations.
An independent samples T-test was performed to determine differences in
retention scores among the No Gesture and Gesture groups in order to ascertain the
impact of gesturing during each condition. Lastly, gaze plots and heat maps were
generated using ArcGIS, in order to identify patterns of eye movement and determine
elements of each condition found attractive by the study participants. These elements,
known as AOI, were initially identified prior to data collection based on traditional
elements contained in a weather forecast. Once data collection had been completed the
information was used to determine actual AOI location and boundaries based on the
eye gaze of the participants. Raw eye track data exported from Tobii Studio were
converted into ArcGIS-compatible formats using a custom ArcGIS tool written in Python.
The tool iteratively clusters the raw eye tracking data and removes eye position
measurements characterized by low confidence based on the validity code (Tobii 2011).
Gaze cluster centroids representing fixations were connected using ArcGIS10.0’s Points
to Line tool to make gaze plots for each participant (Figure 3.3A, and 3.3B). Gaze plots

	
  

55	
  

represent the path of viewer attention across an image and help to characterize both
saccadic and smooth movements.
Figure 3.2: A - No Gesture condition with associated AOI and B - Gesture condition with
associated AOI. All but one AOI are identical; the Hands AOI varies based on main
gesturing position in the video.

Table 3.2: AOI letter designation and corresponding areas in pixels (units squared).
*Areas are identical except for the Weathercaster Hand AOI.
AOI Name

Letter Designation

No gesture pixels
(units2)

Gesture pixels
(units 2)

Station Banner
Days of the Week
Scene Image
Forecast
Forecast Temperature
Weathercaster’s Hands*
Weathercaster’s Face

A
B
C
D
E
F
G

22220.9
6639.3
22166.0
76787.4
6852.2
22937.1
10064.5

22220.9
6639.3
22166.0
76787.4
6852.2
26167.8
10064.5

3.3 Results
The results of this study include descriptive data of the study population and
findings based on retention and gaze data collected during the study. Retention data
represents the participant’s correct responses to questions regarding forecast elements
and AOI data indicates where and how long participants attended to particular forecast
elements. As a result, both the retention and gaze data may be indicative of the
	
  

56	
  

attention paid to the forecast by the participants. Descriptive data indicated that the
study population was normal based on the results of the skewness and kurtosis results
in which statistical values for gender, age and ethnicity all fell within an acceptable
statistical range of -2 to 2.
Retention survey results indicate that no difference exists between No Gesture
and Gesture forecast variations when comparing the participants’ correct responses to a
total of seven questions that focused on important elements of forecast and nonforecast elements related to the scene itself. An independent samples T-test was
conducted to compare retention scores for the No Gesture and Gesture group. No
significant difference in the scores exists between the No Gesture (M = 2.63, SD = 1.17)
and the Gesture groupings for forecast scores (M = 2.94, SD = 1.29); t(34) = -.770, p =
0.447. No significant difference was found for the non-forecast scores with the No
Gesture (M = 1.16, SD = .688) and the Gesture groupings (M = 1.18, SD = .728); t(34) =
-.079, p = .938. These results suggest that weathercaster gesturing had no effect on
participant retention of forecast information.
In order to ascertain if gender differences might impact retention of the weather
forecast information during the No Gesture and Gesture conditions an independent
samples T-test was performed. The results indicated that gender was not a significant
factor in scores for female (M=2.84, SD=1.15) and male (M= 2.70, SD= 1.41) conditions
in this study; t(34) = 3.19, p = .752. No additional tests were performed on the remaining
demographic data (age and ethnicity) since these data had minimal variation.
Eye tracking data were used to determine the total number of fixations, average
fixation duration, and total fixation duration spent on each AOI across the study

	
  

57	
  

population (Table 3.3). The total number of fixations were greater for the No Gesture
condition AOI compared to the gesture condition AOI, except for the Weathercaster’s
Hands and Face AOIs. The average fixation duration for the No Gesture condition AOI
were also greater than the Gesture condition AOI, except for a slightly higher duration in
the Weathercaster’s hand AOI for the Gesture condition. The total fixation duration for
the No Gesture condition AOI were greater than the Gesture condition AOI for all AOI.
For both conditions, the Forecast temperature, Forecast and Weathercaster’s face AOI
induced the greatest gaze among participants, while the Station banner, Days of the
week, Scene image and Weathercaster’s hands AOI induced the least amount of gaze.
AOI for both conditions were similar except for the Hands AOI, which represented a
larger area (pixels) due to the movement of the weathercaster’s hands during the video.
Table 3.3: Total fixations, average fixation duration and total fixation duration for both
No gesture and Gesture conditions. No Gesture n=19, Gesture n=17.
Area of Interest
(AOI)

Total number of
fixations
No Gesture Gesture

Average fixation
duration
No Gesture
Gesture

Total fixation duration
No Gesture

Gesture

Station banner
Days of the week
Scene image
Forecast
temperature
Forecast

116
115
161
262

76
93
105
242

.44
.66
.53
.54

.18
.26
.23
.25

8.4
12.55
10.16
10.34

3.04
4.39
3.83
4.24

1134

986

.54

.24

10.19

4.07

Weathercaster’s
hands
Weathercaster’s
face

11

39

.18

.19

3.43

3.22

167

177

.96

.36

18.18

6.1

The results of the Mann-Whitney U indicate that in most comparisons, eye
tracking data were similar across the two conditions where Z = 1.96, p <= .05. The only

	
  

58	
  

exceptions to this finding were U scores associated with the “Centroids Hands Percent”,
U = 81.5, p =.006 and the “Fixation Duration Hands”, U = 97, p = .029 (Table 3.4).

Fixation	
  
Duration	
  Face	
  

Fixation	
  
Duration	
  Hands	
  

Fixation	
  
Duration	
  Scene	
  

Fixation	
  
Duration	
  
Temperature	
  

81.5	
  

108	
  

147	
  

115	
  

108	
  

139	
  

107.5	
  

97	
  

117.5	
  

117.5	
  

298.5	
  
-­‐0.514	
  
0.607	
  

282	
  
-­‐1.041	
  
0.298	
  

325.5	
  
-­‐0.827	
  
0.408	
  

271.5	
  
-­‐2.723	
  
0.006	
  

261	
  
-­‐1.709	
  
0.088	
  

337	
  
-­‐0.481	
  
0.645	
  

268	
  
-­‐1.48	
  
0.130	
  

261	
  
-­‐1.067	
  
0.09	
  

292	
  
-­‐0.710	
  
0.472	
  

260.5	
  
-­‐1.712	
  
0.087	
  

287	
  
-­‐2.186	
  
0.029	
  

270.5	
  
-­‐1.307	
  
0.182	
  

270.5	
  
-­‐1.307	
  
0.162	
  

b

.616 	
  

b

.315 	
  

b

.415 	
  

b

.010 	
  

b

.003 	
  

b

.661 	
  

b

.145 	
  

b

.003 	
  

Fixation	
  
Duration	
  
Station	
  Banner	
  

Centroids	
  
Temperature	
  
Percent	
  

135.5	
  

Fixation	
  
Duration	
  Days	
  
of	
  the	
  Week	
  

Centroids	
  
Scene	
  Percent	
  

129	
  

Fixation	
  
Duration	
  
Forecast	
  

Centroids	
  
Hands	
  Percent	
  

145.5	
  

Centroids	
  
Station	
  Banner	
  
Percent	
  

Centroids	
  Face	
  
Percent	
  

Mann-­‐
157	
  
Whitney	
  U	
  
Wilcoxon	
  W	
  
347	
  
Z	
  
-­‐0.143	
  
Asymp.	
  Sig.	
  
0.888	
  
(2-­‐tailed)	
  
b
Exact	
  Sig.	
  
.900 	
  
[2+(1-­‐tailed	
  
Sig.)]	
  
a.	
  Grouping	
  variable:	
  Trial	
  
b.	
  Not	
  corrected	
  for	
  ties.	
  

Centroids	
  Days	
  
of	
  the	
  Week	
  
Percent	
  

Centroids	
  
Forecast	
  
Percent	
  

Test	
  Statistics	
  

Table 3.4: Results of the Mann-Whitney U Test Statistics for No Gesture and Gesture
conditions.

b

.490 	
  

b

.067 	
  

b

.042 	
  

b

.185 	
  

b

.185 	
  

Finally, gaze plots and heat maps were generated using ArcGIS. These were
used to illustrate gaze patterns, fixation duration and forecast elements particularly
attractive to participants. The gaze plots are useful in attempting to understand what
elements were of particular interest to the participant and how they moved their
attention from one element to another. The heat maps are useful in determining which
elements were found attractive by the participant, potentially drawing attention not only
to themselves, but also away from other elements. The gaze plot (Figure 3.3A and
3.3B) depicts one participant’s complete gaze path. Circles represent each fixation; the
size of the circle reflects the duration of fixation where larger circles indicate longer
fixation times. The lines between fixations indicate the pattern of eye movement by the
participant.
In the No Gesture gaze plot, the participant’s attention appears to be heavily
concentrated on the weathercaster’s face and the forecast. Other areas of participant
attention, though somewhat less, are the scene image, days of the week and station

	
  

59	
  

banner. In addition to the heavy concentration, the participant also demonstrates a
pattern of uninterrupted movement between the weathercaster’s face and the forecast.
There is no apparent attention given to the weathercaster’s body or hands.
Similar to the No Gesture gaze plot, the Gesture gaze plot also demonstrates
areas of heavily concentrated attention on the weathercaster’s face and the forecast.
However, the plot appears to show the participant moving from the weathercaster’s
face, stopping in the area close to the hands and then moving to the forecast. There
also appears to be a greater amount of attention focused on the days of the week than
illustrated in the No Gesture plot.
Figure 3.3: A and B are example gaze plots during No Gesture condition and Gesture
condition forecast variations. Each depiction illustrates one participant’s gaze during
each forecast. In Figure 3 A is a gaze plot for Participant Elab2011020a during the No
Gesture condition forecast variation and B shows a gaze plot for Participant
Elab2011006b during the Gesture condition forecast variation.

Heat maps (Figure 3.4) are a eye tracking visualization tool that display a
summary of gaze data across multiple participants. In this study, red on a heat map
indicates the general location where participant gaze data focused most often, green
indicates an area of lesser focus, and no color indicates no data was recorded for these

	
  

60	
  

areas. Areas of high fixation represent potential areas of interest to the participant
because participants spent more time overall visually attending to these areas.
The No Gesture heat map indicates higher concentration of participant’s attention
on the weathercaster’s face and forecast. There also appears to be moderate levels of
attention on the days of the week. The scene image and station banner indicate a lesser
amount of participant attention being paid to these elements.
The Gesture heat map generated results similar to the No Gesture except for a
lesser amount of attention being focused on the weathercaster’s face. There does
appear to be a more pronounced connection between the weathercaster’s face and the
forecast and days of the week, however, when compared to the No Gesture heat map.
Figure 3.4: Example heat maps for the No Gesture condition (A) and Gesture condition
(B) forecast variations. Each depiction illustrates the study population’s gaze during
each forecast. In Figure 3.4 A is a heat map for study participants during the No
Gesture condition forecast variation and B is a heat map for study participants during
the Gesture condition forecast variation.

3.4 Discussion and Conclusions
This study considered the extent to which weathercaster gesturing would impact
viewer attention and retention during a televised weather forecast. After viewing one of
two video clips (No Gesture or Gesture) of an actual weather forecast, participants were

	
  

61	
  

asked seven questions directly related to the forecast they previously viewed. Five of
the questions were related to actual forecast elements and the remaining two pertained
to the weathercaster shirt color and background scene color. Quantitative and statistical
analysis indicated no difference in the retention scores for both the No Gesture and
Gesture conditions.
A closer look at the fixation data (Table 3.3) however reveals some interesting
findings that appear to support gesturing influencing participant attention during the
video clips. Higher total fixations were recorded for all AOI during the No Gesture
condition except for the hands AOI and face AOI. Total fixations for the hands and face
AOI for the Gesture condition were greater than those recorded for the No Gesture
condition. It is possible that the participant’s attention was distracted by the hand
movement, causing them to focus on the hands more and reducing attention on other
elements of the video.
Except for the hands AOI, average fixation duration is lower for all other elements
for the Gesture condition. Previous research indicates that extended fixation duration
may be an indicator of confusion, causing the participant to spend additional time trying
to understand what is being presented. As a result, it may be that participants viewing
the No Gesture condition were somewhat confused by the forecast and required more
time to process the information being presented.
Total fixation duration was higher for the No Gesture condition compared to the
Gesture condition, but retention scores were not significantly different. This may be
attributable to participant confusion experienced during the No gesture condition due to

	
  

62	
  

the lack of weathercaster gesturing, resulting in additional time spent to understand the
forecast.
The most significant differences between both gesture trials were the number of
fixations and the length of time of those fixations on the weathercaster’s face (Face
AOI) and hands (Hands AOI). In particular, the hands gesture encouraged the greatest
difference in viewer attention diversion from the forecast in terms of fixation duration,
between both gesture trials. Gesturing diverts viewer attention from particular points of
interest contained in the weather forecast without negatively impacting viewer retention
of forecast information. Being aware of the impacts of gesturing, weathercasters may be
able to alter viewer behavior using gesturing to aid in transitions associated with
traditional forecasts by altering the viewers focus on specific elements.
It appears that for the purpose of this study that gesturing by the weathercaster
distracts undergraduate participants from the time spent on important weekend weather
information during typical weather forecasts. In addition, participant interaction with on
screen graphics and text may combine with gesturing to alter attention to the various
elements contained within the forecast. While no gain in retention was realized by
forecasts in which gesturing by the weathercaster is prominent (Gesture condition), the
increased fixation on the weathercaster’s hands (Hands AOI), and diminished fixation
on other AOI included in the video, as well the potential impact of graphic elements
(station banner), should be considered by the weathercaster in determining best
practices when developing appealing weather forecasts. Careful analysis of on-air
forecast elements, weathercaster position and subsequent gesturing techniques all
impact viewer attention. Weather forecast elements, weathercaster position and

	
  

63	
  

movements all have the capacity to influence viewer retention and attention of the
material being presented. Specific attention should be paid to elements that may cause
confusion, or increased concentration on the part of the viewer, during weather
forecasts. While gesturing has not been shown to impact retention of the material being
presented, it does have the ability to redirect attention during forecasts. The redirection
may be responsible for reducing confusion or concentration necessary for the viewer to
understand forecast elements.
Since television continues to be a popular source for newsworthy information,
careful thought and consideration should be given to the design elements and posturing
of the on screen personality. This is especially important in cases where the
weathercaster is considered the “science link” to the general public, often being the
case in local and large market televised media. In situations where minimal elements
are competing for viewer attention, gesturing may be appropriate since attention will not
be diverted from salient information. However, in instances where gesturing would
cause viewer attention to be diverted from salient information during the forecast,
gesturing is not recommended. This applies to not only weather related segments, but
the entire newscast as a whole. Maximizing the viewers experience by reducing
confusion and unnecessary concentration on non-salient elements should lead to
increased attention to the information being provided and insure ample retention.
This study supports the future need for additional investigation into weathercaster
delivery methods and broadcast elements and their impact on viewer attention and
retention. The information presented during these broadcasts often contain critical
information pertaining to not only weather related events, but other major happenings

	
  

64	
  

requiring decisions to be made by viewers. Communication that is accurate and easily
comprehended without distraction provides the basis for timely consideration of the
material being presented, allowing for purposeful decision-making.

	
  

65	
  

APPENDICES

	
  

66	
  

Appendix 3A: Retention Survey Questions

	
  

67	
  

Appendix 3B: Analytical Methods
Descriptive statistics were used to explain the results of a demographic survey
consisting of questions related to age, gender, ethnicity, and geoscience experience.
Descriptive statistics are used to describe the base level data of a study and provide
simple summaries. They are typically used as a base level of study information and
provide a basis for further quantitative analysis of the study.
For this study descriptive statistics were used to determine whether the study population
was normal based on a the results of the skewness and kurtosis results in which
statistical values for gender, age and ethnicity all fell within an acceptable statistical
range of -2 to 2.
An Independent samples T-test was performed to compare retention of the forecast and
non-forecast information presented in the clip.
Independent samples T-tests are used to compare the means of two independent
samples, or populations. It is often utilized to test a hypothesis based on the difference
between two different groups (samples).
For this study the retention scores of both No Gesture and Gesture groups were
compared to determine if statistical differences existed between the two groups. An
additional independent samples T test was performed to investigate the impact gender
had on retention of forecast information between male and female participants.
A Mann-Whitney U test was used to compare eye tracking data of AOI identified by the
study. The data was used to determine differences in participant attention to different
forecast elements contained within each clip (No Gesture and Gesture).
The Mann-Whitney U-test is used to determine whether two independent samples of
observations are drawn from the same distributions of collected data. An advantage of
the test is that the two separate samples do not have to contain the same number of
observations.
In this case the number of fixations were compared between the No Gesture and
Gesture groups for the AOI identified by the study to determine if gesturing served to
direct a participant’s attention to a particular AOI.

	
  

68	
  

REFERENCES

	
  

69	
  

REFERENCES

Drost, R. (2013): Memory and decision-making: Determining action when the sirens
sound. Weather, Climate and Society.
Electronic Research Incorporated ERI 2011.
Ehmke, C., Wilson, S. (2007): Identifying Web Usability Problems from Eye-Tracking
Data. Published by the British Computer Society People and Computers XXI –
HCI... but not as we know it: Proceedings of HCI 2007.
ESRI (2001): ArcGIS TM Spatial Analyst: Advanced GIS Spatial Analysis Using Raster
and Vector Data. ESRI White paper J8747.
Harris, (2007): Local television news is the place for weather forecasts for a plurality
of Americans. The Harris Poll #118, November 28, 2007. Available online at
(http://www.harrisinteractive.com/harris_poll/index.asp?PID=839).
Just, M., A., Carpenter, P., A. (1976): Eye Fixations and Cognitive Processes. Cognitive
Psychology. Volume 8, 441-480.
Liversedge, S., P., Findlay, J., M. (2000): Saccadic eye movements and cognition.
Trends in Cognitive Sciences. Volume 4, No.1.
Lazo, J., K., and Chestnut, L., G. (2002): Economic value of current and improved
weather forecasts in the U.S. household sector. Report to the NOAA Office of
Policy and Planning, pp. 213.
Lazo, J., K., Morss, R., E., Demuth, J., L. (2008): 300 Billion Served: Sources,
Perceptions, Uses, and Values of Weather Forecasts. National Center for
Atmospheric Research (NCAR).
Louwerse, M., M., Bangerter, A. (2000): Focusing a\Attention with Deictic Gestures and
Linguistic Expressions. Institute for Intelligent Systems. 1331-1336.
Morrow, B., H., Lazo, J., K., Demuth, J., L. (2008): Communicating Weather Forecast
Uncertainty: An Exploratory Study with Broadcast Meteorologists. Final Report of
Focus Groups Conducted at the 36th AMS Conference on Broadcast
Meteorology Denver, CO June 25-26, 2008.
Morss, R., E., Demuth, J., L., Lazo, J., K. (2008): Communicating Uncertainty in
Weather Forecasts: A survey of the U.S. Public. American Meteorological
Society. Volume 23, pg. 974-991.

	
  

70	
  

National Research Council (NRC), (2003): Communicating Uncertainties in Weather
and Climate Information: A Workshop Summary. National Acad. Press, 68 pp.
Olsson, P. (2007) “Real-time and offline filters for eye tracking,” Master’s thesis, Royal
Institute of Technology, Apr. 2007.
Pew Research Center for the People & the Press (2011): Internet Gains on Television
as Public’s Main News Source. January 4, 2011.
Rashbass, C. (1961): The Relationship Between Saccadic and Smooth Tracking Eye
Movements, Journal of Physiology, Great Britain.
Salvucci, D., Goldberg, J. (2000): Identifying fixations and saccades in eye-tracking
protocols. Proceedings of the Eye Tracking Research and Applications
Symposium 2000 (pp. 71–78). NY: ACM Press.
Socci, A., D. (2007): Are TV stations undermining the standards and credibility of
weather forecasting and meteorology? Bulletin of the American Meteorological
Society, 88, no. 4, 578-580.
Stark A., P., Quinn, S., Edmonds, R. (2007): Eyetracking the news: a study of print and
online reading. St. Petersburg, FL: The Poynter Institute for Media Studies.
Stewart, Al., E. (2006): Assessing the Human Experience of Weather and Climate: A
Further Explanation of Weather Salience. Preprints, AMS Forum: Environmental
Risk and Impacts on Society: Successes and Challenges; Atlanta, GA; Amer.
Meteor. Soc., CD-ROM, 1.6.
Tobii Eye Tracking (2010): An Introduction to Eye Tracking and Tobii Eye Trackers.
Tobii Technology AB.
Tobii T60 and T120 Eye Tracker User Manual, Revision 4 (2011): Tobii Technology AB.
Trobec, J. (2007): Emerging television weather presenting strategies in the United
States. EMS7/ECAM8 Abstracts, Vol. 4, EMS2007-A-00680, 2007 7th EMS
Annual Meeting / 8th ECAM.
Waterworth, E., L., Haggkvist, M., Jalkanen, K., Olsson, S., Waterworth, J., Wimelius,
H. (2003): The Exploratorium: An Environment to Explore your Feelings.
Psychology Journal, Volume 1, Number 3, pg. 189-201.
White, S., J. (2008): Eye movement control during reading: Effects of word frequency
and orthographic familiarity. Journal of Experimental Psychology: Human
Perception and Performance.

	
  

71	
  

World Meteorological Organization (2005): Guidelines on weather broadcasting and the
use of radio for the delivery of weather information. PWS 12, WMO/TD No. 1278.
Yaros, R., A., Cook, A., E. (2011): Attention Versus Learning of Online Content:
Preliminary Findings from an Eye-Tracking Study. International Journal of Cyber
Behavior, Psychology and Learning (IJCBPL). Volume 1, Issue 4.

	
  

72	
  

Chapter 4
Severe Weather Warning Communication: Factors impacting audience attention
and retention of information during warnings.
A wide variety of natural hazards impact the U.S. inflicting personal and
economic tolls on the public. Communicating warnings about impending natural hazards
is an important duty of weathercasters working for news organizations. On television,
hazard warnings are typically conveyed through live radar, storm chasers, live
newscasts, and warning scrolls. The effectiveness of this traditional approach, however,
is unclear, especially given the lack of attention some members of the general public
pay to these warnings. A study comparing individual responses to a traditional warning,
an animated warning, and an audio warning was undertaken to evaluate the impact of
delivery methods, on viewer retention, and viewer preferences during severe weather
warnings. A Tobii T60 eye tracker was used to document visual interactions with onscreen warnings. Results indicate that viewers of the animated warning retained more
pertinent information about the tornado warning than viewers of the traditional warning,
and retention during the traditional warning was equivalent to that of the audio warning.
In addition, gaze patterns for the traditional warning were much more diffuse than for
the animated warning, suggesting that attention was more focused on the animation
than the live video. Retention and preference data suggest that while traditional
warnings do provide sufficient information for people to act upon, inclusion of novel
animated elements may reinforce retention of the warning information. In addition,
modifications to reduce visual complexity of traditional warnings may positively impact
viewer attention to individual warning elements. Future studies will consider the

	
  

73	
  

effectiveness of a hybrid warning containing both traditional and animated components.
Development and analysis of a hybrid warning will allow for identification of warning
elements that positively impact viewer attention and retention of warning information
presented during traditional televised broadcasts. This information may be used to
advance current severe weather warning communication techniques and increase
public awareness during severe weather events.
4.1 Introduction
A wide range of factors influences the effectiveness of severe weather warnings.
Traditional elements, such as live weathercaster reporting, radar imagery and warning
scrolls appearing at the bottom of the television screen are typical of live warning
broadcasts (WMO 2005). Recent developments designed to enhance warning
effectiveness include experimental impact-based warnings that include text containing
enhanced “storm impact” information (CRH/NOAA 2013) and personalized texts to cell
phones (NWS 2010). Despite a long history of traditional television warnings and new
advances made to severe weather warning delivery and communication, significant
impact to property and lives continues to be a major consequence associated with
severe weather throughout the United States (Folger 2013). Improvements to current
warning mechanisms may lead to additional avoidance of these consequences, and
result in a reduction of the lives and property lost during severe weather events. The
communication and delivery of information during a severe weather event share similar
obstacles with a variety of natural hazards.
Natural hazards exist in many forms throughout the United States (USGS 2007).
In fact, the U.S. is rated among the top five countries over the last decade to be hit most

	
  

74	
  

frequently with natural disaster events (Sapir, et. al. 2012). This places the population at
risk from dangers associated with volcanoes, earthquakes, landslides, drought, fire, and
severe weather. Severe weather, in the form of blizzards, hurricanes, and
thunderstorms, pose the most frequently experienced risk (Smith 2013). Of these,
thunderstorms occur most frequently throughout the United States (NOAA 2010).
Thunderstorms produce lightning, tornadoes, straight-line winds, floods, and hail;
together, these average about three billion dollars in insured losses and over 65
fatalities each year (NOAA 2010). In recent years, these losses have grown
considerably and tornadoes have taken a lead role in damage associated with severe
thunderstorms (Munich Re 2012). Recent events, such as the 2011 outbreak in Joplin
Missouri, which claimed 161 lives and the 2013 event in Moore Oklahoma that killed 24
(Kuligowski et. al. 2013), represent a recent increase in severe outbreaks which have
resulted in record property losses and fatalities associated with tornadoes (NCDC
2013).
The destructive properties of tornadoes are the leading cause of property losses
and the second leading cause of thunderstorm related fatalities (Folger 2013). In fact,
during the 2011 extreme tornado outbreak (of which Joplin was a part), Munich Re and
the National Climatic Data Center reported over 20 billion dollars in damage and 552
lives lost due to tornadoes alone (Simmons, et. al. 2011). The ability to reduce the
impact of severe thunderstorms and tornadoes typically rests in efforts that focus on
storm detection and warning systems developed by the National Oceanic and
Atmospheric Administration (NOAA) and the National Weather Service (NWS), in
conjunction with institutions focusing on severe weather detection and prediction (Folger

	
  

75	
  

2013 & NOAA 2013). Recent research, however, cites policy, demographic and
cognitive factors as potential elements that can increase fatalities associated with
tornado fatalities (Folger 2013).
Policy related factors, such as land use and rezoning practices, can influence the
number of tornado fatalities by exposing the population to areas of greater risk of
tornado activity (Brenner 1995). This may occur when land is developed in tornado
prone areas, or existing structures are not required to have tornado safe shelters
included in their design. Since people are more likely to seek shelter during tornado
warnings, compared to other severe weather events, the type of shelter available may
also affect a community’s safety (Folger 2013). Demographic factors, such as income,
may also influence exposure to tornado risk. Manufactured homes, for example, are
more prone to damage by tornadoes than traditional construction. Since a large number
of low- and moderate-income families live in manufactured housing in tornado prone
areas, tornado risk is generally increased in these situations, unless alternate tornadosafe structures are provided by the community (Sutter 2010).
Cognitive factors, such as experience and knowledge, impact decisions made by
the public regarding tornado warnings and may influence response to warnings.
Research has shown that past tornado warning experience may negatively impact an
individual’s response to tornado warnings when compared to those who have never
experienced a tornado warning (Drost 2013). For instance, if previous instances exist in
which an individual experienced a tornado warning with no subsequent tornado event
they may be more likely to dismiss the warning. In some cases they may also look for
additional confirming evidence of a tornadoes existence before seeking shelter (Mileti

	
  

76	
  

and Sorensen 1990). Interestingly, severe weather knowledge positively influences
individual and community response to severe weather warnings (Beer and Hamilton
2002 & Drost 2013). Increased knowledge among individuals and communities about
their potential exposure to tornado risk increases awareness of the danger, and this
heightened awareness promotes more prudent assessment of the potential risk and
better decision-making (Beer and Hamilton 2002). Although severe weather experience
is unique to individuals, knowledge of severe weather is available in many forms;
traditional education, informal exposure through day-to-day activities, and media
coverage of severe events. Knowledge, therefore, can be accessed through TV
weather forecasts, social media, and Internet sources (Lazo 2009). In turn, these
sources rely heavily on National Oceanic and Atmospheric Administration (NOAA) data
that is supplied to the National Weather Service (NWS 2013), providing a direct link
between scientists and the public.
A strategic element of the NOAA mission focuses on understanding and
predicting changes in weather (NOAA 2013). Much of this mission is accomplished
through delivering daily weather information and storm warnings in conjunction with the
National Weather Service (NWS). The NWS supplies a wide range of data and related
weather products that are used by both the government and private sector (NWS 2013).
Media outlets utilize these data to communicate weather information to the public.
Technologies that deliver this information to the public range from personal devices (cell
phones, PDA’s) to traditional newspapers and TV broadcasts (Drost 2013 and Lazo
2009). When utilizing televised broadcasts for severe weather information, the public
may choose from local TV stations or national enterprises such as The Weather

	
  

77	
  

Channel. Regardless of the choice of information, in extreme weather events a concise
and timely forecast is crucial for decision-making to reduce losses to property and of life
(Mileti and Sorensen 1990).
Televised severe weather warning broadcasts utilize a number of visual elements
to communicate information. Television producers assume that viewers have the ability
to attend to both visual elements and verbal information being presented by the
weathercaster simultaneously (Bergen, Grimes and Potter 2005). While accessing
verbal and visual information simultaneously is quite feasible (Baddeley and Hitch
1974), attending to multiple visual elements at the same time represents a challenging
visual task (VanRullen, et. al. 2007). As warning broadcasts become more visually
complex through the addition of multiple screen images or elements (radar, live footage,
warning scrolls, etc.), determining if the additional content provides added benefit to
communicating warning information, or is instead a distracter to effective
communication, becomes essential (Josephson and Holmes 2006). It is possible in
some instances that less information, or fewer methods of conveying information may
be more effective. Reduction of screen clutter represents a potential for improved
viewer attention since the inclusion of multiple visual elements has been shown to
negatively impact the effectiveness of the main broadcast message (IPG 2011) and
reduces viewer comprehension of the information being presented (Bergen, Grimes and
Potter 2005).
Typically, traditional broadcasts include radar imagery, storm chasers, warning
scrolls appearing at the bottom of the screen, and live weathercaster coverage of the
event (WMO 2005). While these approaches attempt to incorporate multiple modes of

	
  

78	
  

communication, their effectiveness and utilization by the public has not been entirely
successful, as witnessed by the public’s frequent lack of response to severe weather
warnings (NOAA 2008 and Lazo 2009). In order to increase the effectiveness of current
warnings, a better understanding of public preferences contained within traditional
broadcasts is necessary.
This study attempts to investigate the effectiveness of warning elements and
delivery methods associated with traditional televised severe weather warning
broadcasts. It endeavors to understand the impacts of individual warning elements on
public knowledge and retention and determine communication preferences during
severe weather warnings. In this work, we compare delivery methods, retained
knowledge, and preferences across three warning types: a traditional televised
broadcast warning, an animated warning, and an audio warning. By understanding the
efficacy of severe weather warning delivery methods, severe warning elements, and
public retention of the information contained within, warnings may be produced that will
reduce loss of lives and property during severe events. In this study, we examined
whether differences in the conveyance (traditional, animated, audio) of severe weather
warnings and message elements (radar, scrolls, live coverage) each contain impact
viewer retention of the information contained in the warning. Participant preferences for
warning information were also investigated to determine what content is considered
most useful when facing decisions during severe weather warnings. This information
was collected utilizing a multiple data collection mechanisms. These include gaze data
collected during eye tracking, participant’s response to retention and preference
surveys, and participant opinions expressed during think aloud questions. Gaze data

	
  

79	
  

were converted to bee swarm outputs in order to ascertain where participant’s attention
was focused during the warning clips. Recording participants eye movements using eye
tracking equipment and software allows for the discovery of where individuals are
looking and for how long, during the severe weather warnings. These fixations occur
when a participant spends a defined length of time on a particular detail contained
within the severe weather warning (Tobii 2010). A participant’s fixation and length of
time spent on a particular detail is often indicative of the attention that is being paid to
that area or item being viewed (Waterworth, et. al. 2003).
Data from the retention and preference questionnaires were used to determine
the amount of warning information participant’s retained after viewing the severe
weather warning, and to identify warning elements they found most useful. Think aloud
responses allowed participants to express their opinions regarding the warning model
they had viewed. Results of the study can inform best practices for constructing a
severe weather warning that positively impacts participant retention.
4.2 Methods
This study occurred during an hour-long eye tracking experiment that consisted
of four separate and unique studies. Participants were recruited through listserv and
email solicitation at a large midwestern institution. Ninety members of the general
public, undergraduate and graduate students, faculty, and staff, were paid $20 each for
their participation. The population consisted of 59% female and 41% male participants.
Ages of the participants ranged from 18 to 46 years, with a median of 23 years.
Seventy-six of the participants were Caucasian, 18% Asian, 4% Black and 2% were
self-coded as unspecified. Seventy-four percent of the participants were non-science

	
  

80	
  

majors. As a whole the study participants had reported completion of a total of 42 Earth
science related courses.
Each participant was exposed to one of three unique severe weather warning
treatments, consisting of two video clips (Figure 4.1) and one audio clip, through
random assignment. All the participants also completed five questionnaires. The audio
and video clips were of similar length (84-95 seconds) and contained nearly identical
audio. Any differences in audio were related to slight differences in the references to
warning visuals. The traditional video clip (TRADITIONAL) was a portion of an actual
severe weather warning broadcast live as an interruption to a normal television
broadcast. The animated video clip (ANIMATED) contained the same warning
information as the TRADTIONAL clip and varied on slightly in length due to the addition
of precautionary recommendations. The Animated clip was completely animated and
contained no “live” components and contained the same audio as the AUDIO clip.. The
audio clip (AUDIO) contained the same audio as the ANIMATED model without an
accompanying visual component. The severe weather warning clips were viewed on a
Tobii T60 eye tracker that allowed capture of eye movements during video clip viewing.
Figure 4.1: Example of Traditional and Animated media clips. The Audio only clip
contains no images and is not illustrated here.

	
  

81	
  

4.2.1 Data Collection
Data were collected in three successive steps during this study; pre-eye tracking,
during eye tracking, and post eye tracking (Table 4.1). First, participants were asked to
complete a weather knowledge and weather experience questionnaire. Next,
participants were given a brief explanation of the Tobii T60 eye tracker before
proceeding to the actual experiment. Participants were then exposed to one of the three
severe weather media treatments. After completing the eye tracking portion of the
study, participants were asked to complete three additional questionnaires; a severe
weather retention questionnaire containing questions directly related to the weather
warning, a weather model preference questionnaire that ranked model elements (radar,
live footage, warning scrolls, etc.) and a questionnaire probing Earth science course
experience and participant demographics. At the conclusion of the study, participants
were asked several think-aloud questions to gain insight into their preferences for
severe weather communication and opinions on the warning clip they previously
viewed.
Table 4.1: Data source descriptions and timing during the study.
Data Sources
Weather knowledge questionnaire

Measurement
Participant knowledge of severe
weather and tornado facts and
precautionary measures to take
during warnings

Timing
Pre-eye tracking

Two questions recorded participants’
thoughts regarding severe weather
terminology.
Weather experience questionnaire

Four questions inventoried
participants exposure to severe
weather, specifically tornadoes
Six questions assessed their
attitudes and emotions towards
severe weather and community
response to warnings

	
  

	
  

82	
  

Pre-eye tracking

Table 4.1 (cont’d)
Video eye tracking

Captured fixation and gaze
information to determine patterns of
attention during video clips

During eye tracking

Retention questionnaire

Participant retention of information
communicated during the video clip

Post eye tracking

Preference questionnaire

Four rank order questions
determined participant preferences
for warning communication and what
is most helpful in making related
decisions. Two Likert questions
assessed the participant’s difficulty in
understanding warnings. One Openended question addressed warning
elements usefulness.

Post eye tracking

Domain experience questionnaire

Recorded geoscience experience
and demographic information

Post eye tracking

Think aloud questions

Captured participants’ reaction and
comments about the media clip they
experienced

Post eye tracking

4.2.2 Data Sources
Data were collected through questionnaires, eye tracking, and think-aloud
questions:
1) The Weather Knowledge Questionnaire (KNOWLEDGEQ) consisted of eleven
questions pertaining to tornado development, severe weather facts, and recommended
precautionary measures. The KNOWLEDGEQ was validated in Drost (2013) and
measures weather knowledge among study participants to determine their
understanding of common weather terms used during severe events.
2) The Weather Experience Questionnaire (EXPERIENCEQ) consisted of twelve
questions divided into three categories: Two questions asked for participant reaction to
the terms “severe weather warning” and “tornado”, four inventoried the participants
exposure to severe weather, specifically tornadoes, and the remaining six evaluated
attitudes and emotions towards severe weather and community response to warnings.
	
  

83	
  

The EXPERIENCEQ questionnaire validation was previously established in Drost
(2013).
3) Exposure to one of three severe weather warning models: Traditional media clip
(TRADITIONAL), Animated video clip (ANIMATED), and an Audio media clip (AUDIO). .
These models were displayed on a Tobii T60 Eye Tracker, which was used to capture
participant fixations and gaze information. These eye tracking data were used to discern
patterns of viewing that suggest attention or distraction during viewing. This information
was then compared to retention data in order to gauge the impact of multiple screen
elements on participant attention and retention and the use of novel animated elements
to increase attention and retention of warning information.
4) The Severe Weather Retention Questionnaire (RETENTIONQ) was made up of ten
multiple choice questions specific to the severe weather warning clip participants were
exposed to during the eye-tracking session. The questions were used to determine what
specific information participants remembered while viewing/listening to their particular
warning model.
5) The Severe Weather Model Preference Survey (PREFERENCEQ) contained four
rank order questions specific to the severe weather warning model participants were
exposed to during the experiment. The PREFERENCEQ was completed after the
experiment and questions focused on the participants’ preferences for elements used in
the warning they had viewed or heard (radar imagery, location determination,
weathercaster delivery, etc.). In addition, two questions assessing participants’ difficulty
in understanding severe weather warnings and an open-ended question asking about
most useful elements of warnings were included. The survey was intended to compare

	
  

84	
  

alternative warning elements to those currently in place, and to determine what
participants consider most useful, or preferred, during severe weather warnings.
6) A demographic survey collected information on participants educational major,
gender, age, ethnicity and family educational background, as well as exposure to Earth
science courses.
7) Participants were asked think aloud questions at the conclusion of the experiment.
These open ended questions allowed participants to express their views and opinions
about the severe weather warning clip they experienced and preferences for severe
weather communication, if any.
4.2.3 Data Analysis
Three types of data were analyzed: eye tracking data, survey data, and thinkaloud data (justification of methods are found in Appendix 4H). Eye tracking gaze data
from the TRADITIONAL and ANIMATED treatments were used to generate participants
gaze points simultaneously over the course of the video clip. This data allows for the
comparison of the gaze patterns of multiple participants for each video warning clip. The
resulting video output produced is known as a bee swarm. These bee swarm videos
were analyzed to determine the extent of attention or distraction experienced by
participants during the experiment. Descriptive statistics were used to explain the
results of the KNOWLEDGEQ, EXPERIENCEQ, RETENTIONQ, PREFERENCEQ and
DEQ data in order to provide summaries of the data collected during the experiment. A
statistical analysis of the data provided by the KNOWLEDGEQ, EXPERIENCEQ was
performed to determine similarity among the participants and the potential impact on
retention. Additional statistical analysis was performed on the RETENTIONQ,

	
  

85	
  

PREFERENCEQ data to ascertain the whether a difference existed among the
participants and the warning model they experienced.
The KNOWLEDGEQ evaluated the participant’s level of familiarity with severe
weather facts, tornado development and recommended precautionary measures. Data
from the KNOWLEDGEQ were used as one of the three pre-measures (knowledge,
experience, trust) performed before the eye tracking session began.
The EXPERIENCEQ was meant to assess the participant’s experience with
warning messages, severe weather, and tornadoes by inventorying actual experience
with these items and assigning a score based on the participant’s responses. Additional
Likert questions contained within the EXPERIENCEQ measured participant emotions
about tornado warnings and tornadoes during the pre-stimuli data collection. Six
questions regarding severe weather warnings were rated on a Likert scale where 1
indicated, “strongly agree” and 5 indicated, “strongly disagree”. A factor analysis was
performed on the six Likert questions contained within the EXPERIENCEQ to determine
if a correlation existed that would lead to a common scale of participant trust in weather
warnings.
The results of the knowledge, experience, and trust data were then analyzed to
determine if they were similar for all models. An independent samples T-test was
performed on the knowledge, experience and trust data to determine if there were any
statistical differences across the three groups. Significant statistical differences may
indicate that knowledge, experience and trust may have the ability to influence
participant performance on other instruments contained within the study.

	
  

86	
  

Next, an independent samples T-test was performed comparing the
RETENTIONQ data among all three warning models to determine if significant
differences existed between the participant questionnaire results. The analysis was
based on questions 1 through 7 of the RETENTIONQ data since the existing model did
not contain specific references to the remaining questions (8 through 10). These
questions queried participants about specific precautionary measures associated with
tornado warnings.
As a final measure of the potential impacts of knowledge, experience and trust
on the three groups performance on the RETENTIONQ, a linear regression was run to
determine if there were any main or interaction effects.
The severe weather model preference questionnaire (PREFERENCEQ) was
used to determine elements of a severe weather warning broadcast that participants
considered most helpful when attempting to understand and make decisions. The
PREFERENCEQ contained similar questions for each weather warning model, but was
altered when necessary due to inherent differences in the three treatments. In these
instances data were recorded as n/a. Participants indicated their preferences by
determining a rank order for each model element.
The PREFERENCEQ also contained two questions that assessed the level of
difficulty in understanding the warning model they viewed and how it compared to a
traditional TV broadcast. They rated their models on a Likert scale where 1 was “not
difficult” and 5 was “very difficult”. A factor analysis was performed on the two Likert
questions contained within the PREFERENCEQ. An independent samples T-test was
performed on “difficulty” questions contained in the EXPERIENCEQ for all three models

	
  

87	
  

to determine if differences existed between the groups. In addition, a linear regression
was performed to ascertain what impacts difficulty may have on the RETENTIONQ
scores.
Individual screen shots taken from the Tobii T60 eye tracker bee swarm data for
both the TRADITIONAL and ANIMATED clips were initially identified based on 5-second
increments during both warning videos. These screen shots were then compared and
paired according to the similarity of the content and audio elements. These example
shots displayed station and weathercaster identification, warning level, current storm
track and location and predicted future storm movements. These elements were all
representative of questions asked on the RETENTIONQ and PREFERENCEQ.
Participant gaze was represented in the bee swarm data by the black dots on the
images and allowed us to quantitatively analyze participants gaze during individual
screen shots.
In order to compare participant gaze for each video warning video model,
individual scenes were captured every five seconds to quantify the subject’s gaze,
resulting in 18 individual shots. Next, screen shots containing identical images or
information exclusive to the individual media scene were eliminated and as a result 15
scene shots were retained. A rubric was developed that allowed researchers to code
the viewer’s gaze as either concentrated or diffuse. Two authors’ coded 15 scene shots
for each warning clip. Each set of clips was coded separately, and a reliability check
yielded an independent rating of 84% agreement. Any inconsistencies discovered
during the process were discussed between the two authors and worked through until a
mutual agreement was reached.

	
  

88	
  

Lastly, participants were asked think aloud questions about the severe weather
warning clip they viewed and preferences for communication during warnings.
4.3 Results
4.3.1 Participant Results – Knowledge, Experience and Trust
The KNOWLEDGEQ and EXPERIENCEQ provided useful information for
determining the level of severe weather information and past experience for the study
population. In addition, trust questions contained within the EXPEREINCEQ provided
insight into the population’s level of confidence in warning information and
recommendations for taking precautionary measures. The difficulty questions contained
within the PREFERENCEQ also provided information on the ease of understanding
traditional severe weather information.
The population exhibited moderate levels of understanding of weather. The
weather knowledge questionnaire (KNOWLEDGEQ) tested understanding of conditions
necessary for tornado development, frequency, timing and weather statements
associated with approaching storms, as well as, common misconceptions. Participants
had a median score of 60% on the KNOWLEDGEQ. Several questions stood out in
participant responses. When asked the typical speed of a tornado and how much
advanced warning is given only 20% of the respondents answered correctly. Somewhat
more disturbing is the fact that only 50% of the respondents understood that a “warning”
is the most urgent National Weather Service Statement.
The weather experience questionnaire (EXPERIENCEQ) consisted of eleven
questions and inventoried participant’s weather experiences and thoughts/emotions
concerning severe weather. The majority of participants reported experiencing some

	
  

89	
  

form of severe weather: results indicated that 86% of the participants experienced a
severe weather warning at least once a year while 50% had experienced an actual
severe weather event. Of that 50%, within the last two years, 40% experienced a severe
thunderstorm, 36% experienced heavy or blowing snow, 23% experienced torrential rain
or flooding, 22% experienced damaging winds, and 3% experienced a tornado.
Additionally, 13% of the participants reported physically experiencing a tornado over two
years ago ranging in strength from an EF0 to EF4 (from the Enhanced Fujita Scale)
based on their judgment that were provided as part of the questionnaire.
In addition to knowledge and experience, the level of trust exhibited by
participants was measured using six Likert scale questions contained within the
PREFERENCEQ (Table 4.2). A factor analysis was performed and retained the first four
of the six questions, yielding a "Trust in Weather Warnings" (Trust) scale calculated
from the average of these four items. For our sample, the scale yielded a Cronbach's
alpha of 0.69, acceptable for use as an aggregate sample-level measure. The four-item
trust scale resulted in an overall average of 2.44 indicating that the participant level of
trust in weather warnings fell between agree and neutral.
Table 4.2: Likert scale questions and participant’s (n=90) responses. 1 = strongly agree,
5 = strongly disagree.
Question
1. I trust the information contained in severe weather warnings.
2. Tornado warnings are indicative of a serious threat to my life and property.
3. A severe weather warning evokes negative emotions for me.
4. Existing community tornado preparedness plans keep me safe.
5. Instinct guides my decisions when faced with severe weather warnings.
6. The image of a tornado evokes positive emotions for me.

Participant’s
Response
1.97
2.27
2.71
2.83
2.67
4.27

Data collected prior to the eye tracking portion of the study indicated all groups
possessed a moderate amount of knowledge, mean score of 6.5 ± 1.5 (maximum of

	
  

90	
  

11), and experience, mean score of 1.49 ± .81 (maximum of 3). A trust mean score of
2.44 ± .72 (maximum of 5) indicated that the groups neither trusted nor distrusted
severe weather warnings. Subsequent independent samples T-tests indicated that no
significant differences existed among the three groups for knowledge, experience and
trust (Appendix G). Differences in these data could have potentially impacted variations
in RETENTIONQ data for the three severe weather warning models.
4.3.2 Model Results – Retention and Preference
The RETENTIONQ determined the amount of information the study population
retained immediately after viewing the severe weather warning clip and provides insight
into the effectiveness of the warning model once other factors that may influence
participant retention are eliminated. The PREFERENCEQ provided information relevant
to participant preferences for severe weather warning delivery methods and content
contained within the actual warning.
Results of the RETENTIONQ varied among the three warning models. Overall,
the participant’s average score for all three models was 38%. Individually, participants
viewing the ANIMATED model scored 46%; those viewing the TRADITIONAL model
scored 35%, while those viewing the AUDIO model scored 36% on the RETENTIONQ.
Independent-samples t-tests were conducted to compare RETENTIONQ scores
for the ANIMATED group, TRADITIONAL group, and AUDIO group. There was a
significant difference in the scores for the ANIMATED (M=4.66, SD=1.10) and the
TRADITIONAL (M=3.5, SD=1.66) groupings; t(58)=-2.94, p=.005. There was also a
significant difference in the scores for the ANIMATED (M=4.66, SD=1.10) and the
AUDIO (M=3.6, SD 1.48) groupings; t(58)=2.87, p=.006). These results suggest that the

	
  

91	
  

animated severe weather warning had a greater effect on participant retention of
information than the traditional severe weather warning and the audio warning. In order
to determine if knowledge, experience and/or trust impacted RETENTIONQ results, a
linear regression was performed. We found that knowledge (Beta=-.052, p>.33),
experience (Beta=.178, p>.11) and trust (Beta=-.067, p>.58) were not significant
predictors for retention, indicating that knowledge, experience and trust had no main or
interaction effects within the groups. RETENTIONQ scores for participants were directly
related to the individual model viewed during the study. The ANIMATED groups
performance was statistically significant to the performance of TRADITIONAL and
AUDIO participants.
The PREFERENCEQ asked participants to list in rank order the most helpful
resources, aspects, and information when attempting to understand information
contained in the warning model they viewed. Resources included a range of four to
seven elements, such as radar imagery and weathercaster in-studio report, depending
on the warning model viewed (Appendix D, E, F). In this instance participants who
viewed the TRADITIONAL and ANIMATED model indicated that the radar image was
the most helpful resource. Since the AUDIO participant viewed no video, the option to
choose that response was not included in their questionnaire. These participants
selected the weathercaster’s live in-studio report of storm location and movement as the
most helpful resource.
Next, depending on the warning model, participant’s ranked three to six elements
as the most helpful aspect in determining where the storm is located or moving in
relation to their location. Examples of these elements are towns and cities, highways

	
  

92	
  

and roads and county boundary lines and are contained in appendix 4D, 4E and 4F.
The TRADITIONAL and ANIMATED group ranked names of towns and cities first, while
the AUDIO group ranked the weathercaster’s description of storm location and
movement first and highways and roads second. This difference may be due to the lack
of visual content associated with the audio warning or unfamiliarity with one’s
geographical surrounding.
Participants were then asked to imagine that they were watching television and
the program was interrupted by a message from the National Weather Service (NWS)
indicating a tornado was predicted for their area. They were then asked to select what
information would be helpful in making a decision on how to respond to the severe
weather warning. Again, these choices are representative of those found in traditional
televised severe weather warnings and examples may be found in appendix 4D, 4E and
4F. In this instance, all three groups (TRADITIONAL, ANIMATED, AUDIO) selected the
radar image, NWS warning message and the forecaster’s recommendation as the top
three informational sources. Considering the same scenario as the previous question,
where they were asked to imagine that they were watching television and the program
was interrupted by a message from the NWS indicating a tornado was predicted for
their area, participants were asked to list in rank order the most helpful information in
making a decision when the NWS predicts a tornado for their area. All groups
(TRADITIONAL, ANIMATED, AUDIO) chose radar image as the most helpful.
Two Likert scale questions were used in the PREFERENCEQ to determine the
participant’s difficulty understanding the severe weather warning model they viewed,
and how they would rate the ease of understanding compared to a traditional TV

	
  

93	
  

broadcast. Responses for question 1 indicated a comparable level of difficulty among all
three groups (TRADITIONAL, ANIMATED, AUDIO) and the video clip they viewed.
Question 2 indicated that the ANIMATED and AUDIO clip are harder to understand than
a traditional TV broadcast, but the TRADITIONAL warning clip ranked on par to the
traditional TV broadcast (Table 4.3).
Table 4.3: Participant’s Likert scale responses to the difficulty and ease of
understanding their severe weather warning model. 1=not difficult, 5=very difficulty.
Question
Model
Participant’s
response
TRADITIONAL
1.93
1) How difficult did you find the severe weather
ANIMATED
2.00
warning to understand?
AUDIO
2.33
TRADITIONAL
1.73
2) Compared to live media (TV) broadcasts of severe
ANIMATED
2.40
weather warnings rate your ease of understanding of
the recorded warning.
AUDIO
2.97
An independent samples T-test was performed on the data to determine if the
data were significantly different from each other. Results for question 1 indicate a
significant difference in the scores for the AUDIO (M=2.33, SD=.80) and the
TRADITIONAL (M=1.93, SD=.74) groupings; t(58)=-2.00, p=0.049 but not for the
AUDIO compared to the ANIMATED (M=2.00, SD=.91) group: t(58)=-1.51, p=.138.
These results suggest that the AUDIO group found their severe weather warning clip
somewhat more difficult to understand than the TRADITIONAL and ANIMATED groups.
Results for question 2 indicate there was a significant difference in the scores for
the ANIMATED (M=2.4, SD=1.13) and the TRADITIONAL (M=1.73, SD=.74) groupings;
t(58)=-2.70, p=.009 and the AUDIO (M=2.97, SD=1.03) and TRADITIONAL(M=1.73,
SD=.74) groupings; t(58)=-5.31, p<=.001 These results suggest that the ANIMATED
and AUDIO groups found the severe weather warning clip they viewed more difficult to
understand than a traditional TV severe weather warning broadcast. Despite the fact
	
  

94	
  

that the ANIMATED group found the warning clip more difficult to understand than the
TRADITIONAL group, they outperformed the TRADITIONAL group on the
RETENTIONQ. It is possible that the difficulty in understanding forced more attention on
the ANIMATED video clip by the viewers (as evidenced by bee swarm data) and
resulted in increased retention.
A final question contained in the PREFERENCEQ solicited participants opinions
on what element of the severe weather warning model they viewed would be most
useful when making a decision on what actions to take in the event of a tornado
warning. Those who viewed the TRADITIONAL model most often-mentioned radar
imagery, storm direction information and the timing and location of the event as most
useful. Participants who viewed the ANIMATED model also mentioned radar imagery
and storm direction information, but differed by then mentioning precautionary
measures as most useful. The AUDIO group mentioned timing and location and
precautionary measures as most useful and made no mention of radar imagery.
In order to compare differences in participant gaze between the two warning
clips, bee swarm data were generated indicating where participants were looking during
the clips Figure 4.2). A comparison of the TRADITIONAL and ANIMATED bee swarm
data indicated that for the TRADITIONAL warning clips participants exhibited a diffuse
pattern of gaze during 33% of the selected clips. In comparison, participants viewing the
ANIMATED warning clips exhibited a diffuse pattern in less than 1% of the selected
clips. Several of the clips contained multiple elements (weathercaster, radar, warning
scroll), which resulted in participant gaze being divided among the individual elements

	
  

95	
  

being displayed. These were not considered to be diffuse as long as participant gaze
was concentrated on the individual elements.
Figure 4.2: Series of images showing participant gaze for both the TRADITIONAL
(Trad) and ANIMATED (Anim) media clip. Individual participant’s gaze is indicated by
the black dots. Clip Trad1 is representative of a “diffuse” gaze pattern; Trad2 is
representative of a “concentrated” gaze pattern. Clip Anim1 is representative of a gaze
“concentrated” on multiple elements; Anim2 is representative of a “concentrated” gaze
pattern.

At the conclusion of the study participants were asked to comment on the severe
weather warning model they had previously viewed. This was meant to provide
information not easily captured as part of a questionnaire and allow the three groups to
express their opinions on the warning model they viewed. The TRADITIONAL group
frequently expressed feelings of confusion and cited the abundance of information being
provided during the warning as “a lot to consider”. The ANIMATED group overall found
the animated warning enjoyable but mention it would be “hard to take seriously”. Lastly,
the AUDIO group mentioned that the lack of visuals contributed to the difficulty in
remembering warning information and that they needed to “concentrate on the
message” to remember what was being communicated.

	
  

96	
  

4.4 Discussion and Conclusions
Effective severe weather warnings are essential to providing the population with
the information needed to take action when threatening weather exists. Warnings that
contain salient information and are delivered to the public on a timely basis provide
ample opportunity for precautionary measures to be taken to protect lives and property.
Traditional severe weather warnings contain multiple elements such as radar, live
reports, warning scrolls, and weathercaster recommendations that convey storm
information to the population. This relevant information allows the public to make
decisions on what steps to take to ensure their safety. This study examined traditional
severe weather warnings and the communication elements they contain to gain a
perspective of how information retention is impacted and what elements are considered
most useful during threatening conditions. Participants viewed a traditional severe
weather warning, animated warning and an audio warning containing nearly identical
severe weather warning information. Additional information pertaining to participant
knowledge, experience and warning preferences was collected for each warning
treatment.
Overall, participant knowledge of basic severe weather concepts was moderate.
Participants demonstrated a lack of knowledge and understanding of weather
statements, tornado speed and lead-time available to take precautionary measures.
Understanding the NWS weather statement information and knowing when to take
precautionary measures is an important first step to ensuring personal safety during
threatening weather. Knowledge of a tornado’s speed and direction figure prominently in

	
  

97	
  

determining an individual’s course of action and heighten the need for immediate action.
Combined, this lack of understanding makes for a potentially disastrous situation.
While the majority of participants had experienced a severe weather warning only
a small percentage had recent exposure to a tornado. As a whole, participant’s trust of
the information contained in the warnings and potential for serious harm is somewhat
neutral. The neutral attitude toward warning information and potential for threats to their
lives and property exposes them to additional danger when combined with the lack of
knowledge about severe weather location and movement. Perhaps the frequency in
which participants had experienced warnings and subsequent lack of understanding the
warning messages themselves contribute to the neutral attitude toward the perceived
potential for harm.
Our data suggest that the ANIMATED treatment was better than TRADITIONAL
at communicating information in the weather warning. The animated severe weather
warning was shown to statistically impact participant retention of severe weather
information when compared to the group who viewed the traditional and audio warning.
Not only did the ANIMATED group retain more of the severe weather warning
information contained in the clip, but bee swarm data also support the evidence of
increased attention on visually salient warning elements contained within the
ANIMATED media clip. Participants viewing the ANIMATED model focused on specific
emerging aspects of the warning in a highly concentrated fashion while participants
viewing the TRADITIONAL model exhibited a somewhat more diffuse focus.
Even though the ANIMATED group retained more warning information than the
TRADITIONAL and AUDIO groups, they indicated that the ANIMATED warning clip was

	
  

98	
  

more difficult to understand when compared to a traditional TV severe weather
broadcast. The bee swarm data may also provide insight into this phenomenon. It is
possible that the novel elements represented by the ANIMATED warning caused a
more focused gaze on the particular warning elements which lead to the participants
increasing the attention required to understand the information being provided by the
media clip. This increased attention may be representative of the difficulty reported by
the participants comparing the ANIMATED warning to a traditional warning broadcast.
The increased attention on warning elements may also be indicative of the increase in
information retention by the ANIMATED group.
The inclusion of novel elements into severe weather warnings provides
opportunities to further investigate the benefits and risks associated with supplementing
traditional warnings with animated elements. Novel elements may initially provide a
positive impact in viewer attention or retention of warning information. As a new source
of communication, viewer’s attention may be drawn to the novel element, resulting in
more effective delivery of weather warnings. However, once the novel element has
become commonplace in warning messages it’s effectiveness may diminish as the
“newness” fades.
Preferences in severe weather warning elements were relatively consistent
among all groups. Participants selected radar imagery, weathercaster information and
NWS warning scroll as sources of storm intensity, location and future track as preferred
sources of warning information. Although the ANIMATED and AUDIO groups warning
models differed slightly with the inclusion of weathercaster recommendations for
precautionary measures, all groups consistently rated this element among the most

	
  

99	
  

helpful for making decisions during a severe weather event. The TRADITIONAL group
would select the “recap of weather information” in place of “weathercaster
recommendations” since recommendations were not included in the PREFERENCEQ
for the traditional media clip. This indicates that updated weather information was of
primary consideration to all groups.
Additional elements rated useful among all groups were information associated
with names of towns and cities, highways and roads, and county boundary lines.
Although this information was ranked highly by all groups, the AUDIO group differed by
selecting the “weathercaster description of location and movement” first. This may be
explained by the context of where the viewer would be listening to an audio warning,
such as an automobile, and/or to the lack of visual imagery associated with the
depiction of these elements.
Participant’s preference of severe weather warning elements contained within
warning broadcasts suggests that warnings may contain information unhelpful to the
viewer, and that information may lead to viewer distraction as evidenced by the
scattered pattern of the bee swarm data in the TRADITIONAL warning clip. In addition,
our study shows that traditional warning messages could be improved by reducing the
number of components shown simultaneously that lead to viewer distraction. This
combination, along with the fact that participant knowledge, experience and trust did not
impact retention during the viewing of the three models, supports the inclusion of novel
animated elements within traditional severe weather warnings. Care should be taken
however, when including animated elements since participants often commented that a

	
  

100	
  

warning that was completely animated would be taken less seriously than a traditional
warning.
These results support the development of a hybrid severe weather warning
model that includes traditional and animated (or novel) elements. In order to accomplish
the development of a hybrid warning we propose further studies that will incorporate
data fielded from a larger population, including heads of households, in order to more
aptly represent the general population. These studies will utilize gaze data to identify the
most useful aspects of traditional and animated warning models and viewer attention
information. This data will be combined with retention and preference data in order to
create a hybrid severe weather warning model which will be compared to a traditional
severe weather warning, and evaluated on its ability to positively impact viewer
retention of warning information and attention to warning elements. It is likely positive
impacts associated with elevated retention and attention will provide a sounder basis of
viewer decision-making during severe weather events. These developments will build
upon previous work in order to advance warning and communication techniques,
leading to more effective public communication during severe weather events.

	
  

101	
  

APPENDICES

	
  

102	
  

Appendix 4A: Knowledge Questionnaire (KNOWLEDGEQ)

	
  

103	
  

Appendix 4B: Experience Questionnaire (EXPERIENCEQ)

	
  

104	
  

Appendix 4C: Retention Questionnaire (RETENTIONQ)

	
  

105	
  

Appendix 4D: Traditional Model Questionnaire (PREFERENCEQ - TRAD)

	
  

106	
  

Appendix 4D (cont’d)

	
  

107	
  

Appendix 4E: Animated Model Questionnaire (PREFERENCEQ - ANIM)

	
  

108	
  

Appendix 4E (cont’d)

	
  

109	
  

Appendix 4F: Audio Model Questionnaire (PREFERENCEQ - AUDIO)

	
  

110	
  

Appendix 4F (cont’d)

	
  

111	
  

Appendix 4G: Independent T-Test Results
Table 4G: Independent T-Test Results.
Independent Ttest (Equal
Model N Mean
variances
assumed)
Knowledge
1
30 6.73
2
30 6.37
Experience
1
30 1.33
2
30 1.50
Trust
1
30 2.56
2
30 2.31
Knowledge
2
30 6.37
3
30 6.40
Experience
2
30 1.50
3
30 1.63
Trust
2
30 2.31
3
30 2.47
Knowledge
1
30 6.73
3
30 6.40
Experience
1
30 1.33
3
30 1.63
Trust
1
30 2.56
3
30 2.48

	
  

F

Sig.

t

1.53
1.30
.76
.82
.71
.83
1.30
2.02
.82
.85
.83
.62
1.53
2.02
.76
.85
.71
.62

.617

.436

1.001 58

.321

.298

.587

-.817

58

.417

.765

.385

1.211 58

.231

4.502 .038

-.076

58

.940

.009

.923

-.618

58

.539

3.099 .084

-.879

58

.383

1.932 .170

.719

58

.475

.157

.693

-.144

58

.155

.921

.341

.436

58

.664

112	
  

df

Sig. (2tailed)

SD

Appendix 4H: Analytical Methods
Descriptive statistics were used to explain the results of the KNOWLEDGEQ,
EXPERIENCEQ, RETENTIONQ, PREFERENCEQ and DEQ.
Descriptive statistics are used to describe the base level data of a study and provide
simple summaries. They are typically used as a base level of study information and
provide a basis for further quantitative analysis of the study.
For this study descriptive statistics were used to describe the level of knowledge and
experience of the study population. It was also used to determine the amount of
information retained after viewing the severe weather models and give insight into the
study populations preferences for having severe weather information communication.
Additional descriptive data also provided information on the gender and age of the study
population. This information was used as a basis for determining additional statistical
analysis of the data collected during the study.
Further statistical analyses were performed on KNOWLEDGEQ, EXPERIENCEQ,
RETENTIONQ, PREFERENCEQ and Trust data to determine differences among the
participants and the model they experienced.
Independent samples T-test’s were performed on knowledge, experience and trust to
determine statistical differences across the three groups. In addition, an independent
samples T-test was also performed comparing RETENTIONQ data among all three
models to determine if differences existed between questionnaire results.
Independent samples T-tests are used to compare the means of two independent
samples, or populations. It is often utilized to test a hypothesis based on the difference
between two different groups (samples).
This study was composed of three different treatments (groups) and the Independent
samples T-test’s were used to determine if significant differences existed among the
groups viewing the three treatments in terms of their knowledge of severe weather,
experience with severe weather and trust of the weather information being
communicated during their specific treatment. This was completed in order to determine
if these variables impacted results of participant retention of the information presented
in their severe weather warning treatment.
A T-test was also performed comparing participant retention scores to determine if
significant differences existed in retention of the severe weather information
communicated among the three groups.
Finally, an independent samples T-test was performed on “difficulty” questions
contained in the EXPERIENCEQ for all three models to determine if differences existed
between the groups.
A linear regression was run to determine if any main or interaction effects of knowledge,
experience and trust were seen on the participant’s performance on the RETENTIONQ.
Linear regressions are used to predict the impact of one or several variables on a
second variable.
For this study, the linear regression was used to determine if participant knowledge,
experience or trust of severe weather had impact on the scores they received on the
retention instrument. In addition, a linear regression was performed to ascertain what
impacts difficulty may have on the RETENTIONQ scores.

	
  

113	
  

A factor analysis was performed on the six Likert questions contained within the
PREFERENCEQ used to determine the participant’s level of trust of weather warnings.
A factor analysis may be used as a data reduction method. In these instances it
attempts to generate a smaller amount of uncorrelated variables from a larger set of
correlated variables. By this method it generates an output of variables that measure
similar things from larger set.
In this study a factor analysis was performed on the six Likert questions. Four of the six
questions were retained during the analysis and the result was a scale that indicated the
participant’s trust in weather warnings.

	
  

114	
  

REFERENCES

	
  

115	
  

REFERENCES

Baddeley, A.D. and Hitch, G. (1974) Working memory. In The Psychology of Learning
and Motivation (Bower, G.A., ed.), pp. 48–79, Academic Press.
Beer, T., Hamilton, R., (2002): Natural Disaster Reduction, Safer Sustainable
Communities: Making Better Decisions about Risk. ICSU Position Paper. IUGG
Commission on Geophysical Risk & Sustainability and ICSU Committee on
Disaster Reduction.
Bergen, L., Grimes, T., Potter, D., (2005): How Attention Partitions itself during
Simultaneous Message Presentations. Human Communication Research. Vol.
31 No. 3, July 2005 311-336.
Brenner, S., Noji, E. K., (1995): Tornado Injuries as Related to Housing in the Plainfield
Tornado. International Journal of Epidemiology. 23: 144.
Central Region Headquarters (CRH/NOAA), (2013): www.crh.noaa.gov/images/crh/IBW
Drost, R., (2013): Memory and decision-making: Determining action when the sirens
sound. Weather, Climate and Society. January 2013.
Folger, P., (2013): Severe Thunderstorms and Tornadoes in the United States.
Congressional Research Service. 7-5700. www.crs.gov. R40097.
IPG Media Lab, (2011): Are all screens created equal? A research study by the IPG
Media Lab.
Josephson, S., Holmes, M., (2006): Clutter or content? How on-screen enhancements
affect how TV viewers scan and what they learn. Association for Computing
Machinery, Inc. ETRA.
Kuligowski, E. D., Phan, L. T., Levitan, M. L., Jorgensen, D. P., (2013): Preliminary
Reconnaissance of the May 20, 2013, Newcastle-Moore Tornado in Oklahoma.
National Institute of Standards and Technology.
Lazo, J. K., Morss, R. E., and deMuth, J. L., (2009): 300 Billion Served: Sources,
Perceptions, Uses, and Values of Weather Forecasts. BAMS, p. 785-798.
Mileti, D. S., Sorenson, J. H., (1990): Communication of emergency public warnings. A
social science perspective and State-of-the- Art Assessment. Oak Ridge National
Laboratory, ORNL-6609.
Munich Re, (2012): Severe weather in North America. Perils, Risks, Insurance.

	
  

116	
  

National Climatic Data Center (NCDC) (2013): www.ncdc.noaa.gov/climateinformation/extreme-events.
National Oceanic and Atmospheric Administration (NOAA) (2010):
www.nws.noaa.gov/os/severeweather/resources.
National Oceanic and Atmospheric Administration (NOAA) (2009): Service Assessment
of the Super Tuesday Tornado Outbreak of February 5-6, 2008.
National Oceanic and Atmospheric Administration (NOAA) (2013):
www.noaa.gov/about-noaa.
National Weather Service (NWS) (2010): Experimental Service Description Document
(SDD). Experimental Mobile Decision Support Services (MDSS). Interactive
NWS (iNWS): Warning Alert SMS Text and Email Messaging Services via Mobile
Device Technologies.
National Weather Service (NWS) (2013): www.nws.noaa.gov/mission.
Sapir, D. G., Vos, F., Below, R., Ponserre, S., (2012): Annual Disaster Statistical
Review: the numbers and trends. Centre for Research on the Epidemiology of
Disasters (CRED) Institute of Health and Society (IRSS) Université catholique de
Louvain – Brussels, Belgium.
Simmons, K., Sutter, D., Pielke, R., (2012): Blown away: monetary and human impacts
of the 2011 U.S. tornadoes. Extreme events and insurance: 2011 annus
horribilis, p107-120.
Smith, D. F., (2013): The Frequency of Severe Weather Events. EQECAT. ABS Group.
Sutter, D., Simmons, K., M., (2010): Tornado Fatalities and Mobile Homes in the United
States. Natural Hazards 53(1): 125-137.
Tobii Eye Tracking (2010): An Introduction to Eye Tracking and Tobii Eye Trackers.
Tobii Technology AB.
United States Geological Survey (USGS) (2009): Natural Hazards – A National Threat.
U.S. Department of the Interior, U.S. Geological Survey Fact sheet.
VanRullen, R., Carlson, T., Cavanagh, P., (2007): The blinking spotlight of attention.
PNAS 19204 - 19209, vol. 104, no. 49.
Waterworth, E., L., Haggkvist, M., Jalkanen, K., Olsson, S., Waterworth, J., Wimelius,
H. (2003): The Exploratorium: An Environment to Explore your Feelings.
Psychology Journal, Volume 1, Number 3, pg. 189-201.

	
  

117	
  

World Meteorological Organization (WMO) (2005): Guidelines on Weather Broadcasting
and the use of Radio for the Delivery of Weather Information. WMO/TD No.
1278.

	
  

118