53.....er , . 3. $9. .

u

.I
3.311.;
. 45‘)!

«WNW. #st 904....»

. . , g ..
Xx... {so}; .vadA-wi «Hm-MWEOL: t

32.135 1‘ a“. lit! z {\I

”3”: Mm,
. a». .
ﬁﬁkﬂy

... hLdumaamun

x
t
V

J?»

 

 

« .
ﬁbﬂ

 

I., o _‘)Pv A. nap:-
9

a. ‘3 g .. , ., . gﬁmﬁ i» .‘

m}

This is to certify that the
dissertation entitled

MEASURING THE IMPACT OF INFORMAL SCIENCE
EDUCATION IN 2008 ON ENVIRONMENTAL KNOWLEDGE,
A'lTlTUDES AND BEHAVIORS

presented by

CHRISTOPHER DAVID WILSON

has been accepted towards fulﬁllment
of the requirements for the

Doctoral degree in ZoologL

 

 

[1/1141] 4/.“

’ Major P r - 2"r’s fgnature
/ "- ﬂ

Date

MSU is an Afﬁrmative Action/Equal Opportunity Institution

 

i 7 ‘—

 

LIBRARY
Michigan State
Dniversity

 

 

 

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE DATE DUE DATE DUE

mattmm

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2/05 p:lCIRC/DaIeDue.indd-p.1

MEASURING THE IMPACT OF INFORMAL SCIENCE EDUCATION IN 2008
ON ENVIRONMENTAL KNOWLEDGE, ATTITUDES AND BEHAVIORS

By

Christopher David Wilson

A DISSERTATION

Submitted to
Michigan State University
in partial fulﬁllment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Zoology

2005

ABSTRACT

MEASURING THE IMPACT OF INFORMAL SCIENCE EDUCATION IN 2008
ON ENVIRONMENTAL KNOWLEDGE, ATTITUDES AND BEHAVIORS

BY

Christopher David Wilson

Despite the emphasis in modern zoos and aquaria on conservation and
environmental education, we know very little about what people learn in these settings,
and even less about how they learn it. Research on informal learning in settings such as
zoos has suffered from a lack of theory, with few connections being made to theories of
learning in formal settings, or to theories regarding the nature of the educational goals.

This dissertation consists of three parts: the development and analysis of a test
instrument designed to measure constructs of environmental learning in zoos; the
application of the test instrument along with qualitative data collection in an evaluation
designed to measure the effectiveness of a zoo’s education programs; and the analysis of
individually matched pre- and post-test data to examine how environmental learning
takes place, with respect to the constructivist view of learning, as well as theories of
environmental learning and the barriers to pro-environmental behavior.

The test instrument consisted of 40 items split into four scales: environmental
knowledge, attitudes toward the environment, support for conservation, and
environmentally responsible behavior. A model-driven approach was used to develop the
instrument, which was analyzed using Item Response Theory and the Rasch dichotomous

measurement model. Aﬁer removal of two items with extremely high difficulty, the

instrument was found to be unidimensional and sufﬁciently reliable. The results of the
IRT analyses are interpreted with respect to a modern validity framework.

The evaluation portion of this study applied this test instrument to measuring the
impact of zoo education programs on 750 fourth through seventh grade students.
Qualitative data was collected from program observations and teacher surveys, and a
comparison was also made between programs that took place at the zoo, and those that
took place in the school classroom, thereby asking questions regarding the role of setting
in environmental education. It was found that students in both program types
signiﬁcantly increased their environmental knowledge as a result of the program, but
only students in the school-based programs signiﬁcantly improved their attitudes towards
the environment.

Analyzing by grade, seventh grade students scored signiﬁcantly lower on all
aspects of the test than the younger students, suggesting a detrimental effect of novel
settings on learning in adolescents. Teacher survey data suggests that teachers place great
importance on how the education program would ﬁt in with their school-based
curriculum, but did little to integrate the program into their classroom teaching.
Observations of the programs revealed some logistical issues, and some concerns
regarding the zoo instructors’ use of curriculum materials.

Analyzing the test data from a constructivist perspective revealed that students
with high incoming environmental attitudes had signiﬁcant increases in environmental
knowledge. That is, students with positive attitudes towards the environment are
predisposed to engage in learning about the environment. Some gender-speciﬁc ﬁndings

are also discussed.

To Alden

ACKNOWLEDGEMENTS

The research presented in this dissertation was made possible by the efforts and
support of many people and organizations. First and foremost are the teachers whose
students completed the tests, and who themselves completed pre- and post-visit surveys,
despite minimal incentives. It is a testament to both Binder Park Zoo and Michigan State
University that all the teachers who were asked to take part in the study did so to the best
of their ability. These teachers include Angela Kangas, Helen Owens, Linda Grubaugh,
Lisa Williams, Lorraine Haddad, Mary Van Dyke, Sally Newville, Vivki DeMeyer,
Wendy McLenithan, and Mrs. Ray. The staff of Binder Park Zoo was accommodating,
ﬂexible and patient throughout, particularly Tom F unke, Greg Guise, and Carol Hill.

My graduate advisory committee members, Dr. Raven McCrory, Dr. Gail
VanderStoep and Dr. Jim Smith have been especially helpful and patient throughout my
graduate work, and I would especially like to thank Dr. Smith for giving me the
opportunity to co-author an NSF grant with him. I would like to express my utmost
gratitude to my advisor, Dr. Richard Snider, for making this, and so many other
opportunities available to me. His vision, knowledge, energy and guidance have helped
me develop both personally and professionally.

I would like to thank my family, especially my parents Marion and David, for
their love, understanding, and faith in my abilities. Thanks are also due to my ﬁ‘iends who
have helped carry me through my PhD. career, especially Mary Martin, Melissa Holmes,

Erich Ottem and Sam Hazen. Finally, I would like to thank my wife, Kalynn Schulz, for

her help and support in every aspect of this work, and for giving me all the motivation to

ﬁnish that I could ever need.

vi

TABLE OF CONTENTS

LIST OF TABLES ............................................................................. x
LIST OF FIGURES ............................................................................ xi
CHAPTER 1 - INTRODUCTION ........................................................... 1
CHAPTER 2 - INSTRUMENT .............................................................. 6
INTRODUCTION ...................................................................... 7
Context in which the instrument will be used ......................... 7
Purpose of the instrument ................................................. 7
Decisions that will be made based on the instrument .............. 8
METHODS .............................................................................. 10
Deﬁnition of the construct .................................................. 10
Previous measures of the construct .................................... 11
lntemal model of the construct ........................................... 13
Hierarchical organization of the construct — a developmental
model .................................................................. 16
Content sampling -Test blueprint ....................................... 17
Types of items ................................................................ 18
Frame of reference .......................................................... 19
Pilot testing ................................................................... 19
Structure of the instrument ................................................ 19
Administration procedures ................................................. 21
Sampling ....................................................................... 21
ANALYSES ............................................................................ 23
Item Response Theory and the Rasch measurement
model ................................................................. 23
Dimensionality ............................................................... 25
Fit ................................................................................ 26
Reliability ...................................................................... 27
Differential item functioning ............................................... 27
RESULTS ............................................................................... 29
Dimensionality ......................................................................... 3O
Reliability ................................................................................ 32
Fit ......................................................................................... 33
Difﬁculty ................................................................................. 35
Differential item functioning ........................................................ 39

VII

DISCUSSION .......................................................................... 41

Content ......................................................................... 41
Substantive .................................................................... 42
Structural ....................................................................... 42
Generalizability ............................................................... 43
External ........................................................................ 44
Consequential ................................................................ 45
CHAPTER 3 - EVALUATION .............................................................. 46
INTRODUCTION ..................................................................... 47
Literature Review and Theoretical Framework ...................... 47
BACKGROUND INFORMATION CONCERNING THE PROGRAM ..... 54
Origin of the program ........................................................ 54
Goals of the program ....................................................... 54
Individuals who use the program ........................................ 55
Characteristics of program materials, activities, and
administrative arrangements ..................................... 56
DESCRIPTION OF THE EVALUATION STUDY ............................. 59
Purpose of the evaluation .................................................. 59
METHODS .............................................................................. 61
Program proﬁle ............................................................... 61
Deﬁnition of variables ....................................................... 62
Discussion of variables and methods of measurement ............ 64
Zoo mission objectives ............................................ 64
Curriculum objectives ............................................. 65
Evaluation design ............................................................ 66
Sampling ....................................................................... 68
RESULTS .............................................................................. 69
Analysis of the test data ................................................... 69
Integration of the program into the school- based
curriculum ............................................................. 76
Teacher satisfaction ......................................................... 78
Program observations ...................................................... 80
Instructors use of scripts .......................................... 80
Organizational issues ............................................. 80
DISCUSSION ......................................................................... 81
Test data ...................................................................... 81
Integration into the curriculum ............................................ 86
Teacher satisfaction ......................................................... 87
Program observations ...................................................... 90

viii

CHAPTER 4 - LEARNING ................................................................. 93

INTRODUCTION ..................................................................... 94
The constructivist view of learning ...................................... 95
The goals of environmental education: knowledge, attitudes
and behaviors ....................................................... 98
Environmental knowledge ........................................ 99
Environmental attitudes ........................................... 100
The barriers to pro-environmental behavior ................. 102
METHODS .............................................................................. 110
RESULTS ............................................................................... 112
Examining the data from a constructivist perspective .............. 113
Effect of gender ............................................................... 1 15
DISCUSSION .......................................................................... 118
CHAPTER 5 - CONCLUSIONS, FUTURE DIRECTIONS, and
LIMITATIONS ......................................................................... 125
CONCLUSIONS ...................................................................... 126
FUTURE DIRECTIONS ............................................................. 129
The Relationship between Formal and Informal
Settings ......................................................................... 129
Knowledge vs. Understanding ........................................... 130
Didactic Lectures ............................................................ 131
Novelty ........................................................................ 132
Working with Teachers ..................................................... 132
LIMITATIONS ......................................................................... 133
Issues of Dimensionality ................................................... 133
Signiﬁcance of Low Correlations ........................................ 135
Agreement Bias .............................................................. 138
APPENDIX A - The test instrument .................................................... 139
APPENDIX B — Item numbers used in the analyses .............................. 142
APPENDIX C — Teacher survey forms ................................................. 144
APPENDIX D — Consent form ............................................................ 146
REFERENCES ................................................................................. 147

LIST OF TABLES
CHAPTER 2

Table 1. Test Blueprint. Examples of items in each category, with each
category being created by the intersection of two dimensions: topics (columns)
and our construct elements (rows) ............................................................ 18

Table 2. Summary of IRT analyses being conducted on the test
instrument .......................................................................................... 28

Table 3. The eigenvalues and the percentage of the total variance explained
for each residual factor ........................................................................... 30

Table 4. Residual factor item loadings ...................................................... 31

Table 5. Difﬁculty indices for each of the 40 items. The ﬁve items with the
lowest difﬁculty are labeled a, and the ﬁve items with the highest difﬁculty are
labeled b .............................................................................................. 35

Table 6. DIF contrasts between pairs of grades for each item. X represents
pairs of grades that are signiﬁcantly different (t = $1.96) with a DIF contrast of |

>.5 | ................................................................................................... 39
Table 7. Comparison of item difﬁculty rankings between two schools .............. 44
CHAPTER 3

Table 1. Variables, indicators and measures ........................................... 63

Table 2. P-values for the one-way ANOVAs examining the effect of the
program on student knowledge, attitude, support, behavior, and total test score,
for each program location. Signiﬁcant p-values are shown in

bold ................................................................................................... 71
Table 3. P-values for the one-way ANOVAs examining the effect of grade on
program impact. Signiﬁcant p—values are shown in bold ................................. 74
CHAPTER 4

Tables 1a & 1b. P-values for the correlations between scores on each
subscale for individual students, n = 228 .................................................... 112

Table 2. P-values for the correlations between pre-test score on each scale,
and the difference between pre-test and post-test scores for each scale, for 228
students .............................................................................................. 113

LIST OF FIGURES
CHAPTER 1

Figure 1. lntemal model of the construct. The model reﬂects the hypothesized
structure of the construct, which is reﬂected in the structure of the test
instrument .............................................................................................. 15

Figure 2. Hierarchical organization of the construct. The model represents the
hypothesized development pathways between elements of the internal
model ................................................................................................... 16

Figure 3. Item Characteristic Curves. a) ICC for a single item; b) ICCs for two

items that differ in their difﬁculty; 0) ICCs for items that differ in their power to
discriminate between examinees having differing levels of ability (after Wright &
Masters, 1982; DeVeIIis, 2003) ................................................................... 25

Figure 4. Test lnforrnation Function for the entire instrument, indicating that the
test is most powerful at differentiating between students with average

ability .................................................................................................... 29
Figure 5. Scree plot of each factor’s eigenvalue ............................................ 30

Figure 6. Item characteristic curves for item with the highest difﬁculty (item 35),

and the item with the lowest difﬁculty (item 12) .............................................. 36
Figure 7. Average difﬁculty for each of the four scales. All scales except Attitude
and Support are signiﬁcantly different .......................................................... 38
CHAPTER 2

Figure 1. Program Theory Model ................................................................ 61
Figure 2. Experimental design (after Campbell & Stanley, 1963) ....................... 66

Figure 3. Interaction bar plots for the effect of program location (Schools on

Safari vs. Zoomobile) and the zoo program (comparison of pre-test and post-test
data) for each scale (knowledge, attitude, support, behavior and total test score).
Data expressed as Mean +l- S.E.M ............................................................. 70

Figure 4a. Plots of the environmental scales with signiﬁcant program effects for

the Zoomobile programs. Zoomobile participants’ posttest scores were

signiﬁcantly higher than pretest scores for Knowledge, Attitude, and Total test
score. Data expressed as Mean +/- S.E.M. Asterisk indicates p < 0.05 ............... 72

xi

Figure 4b. Effect of the Schools on Safari program on student knowledge.

Schools on Safari participants’ posttest scores were signiﬁcantly higher than

pretest scores for Knowledge variable only. Attitude, support and behavior
variables were not effected. Data expressed as Mean +l- S.E.M. Asterisk

indicates p < 0.05 .................................................................................... 73

Figure 5. Effect of the program on each environmental variable by grade. Data
expressed as Mean +/- S.E.M. Asterisks * indicate a signiﬁcant between the

pretest and posttest scores. One asterisk (*) indicates p < 0.1, two asterisks (**)
indicates p < 0.05. Different letters represent signiﬁcant differences between
grades (p < 0.1) ....................................................................................... 75

Figure 6. Primary factors that teachers consider when planning ﬁeld trips ........... 76

Figure 7. Teacher responses on the post-visit survey to questions regarding the
use of pre and post-visit materials, the parallels made to state science objectives,
and the extent to which they had integrated the program into their school-based
curriculum .............................................................................................. 77

Figure 8. Information deﬁcit model of environmental education (after Burgess et
al., 1998) with annotations (gray arrows) showing the impact of Binder Park
Zoo’s two program types ........................................................................... 83

Figure 9. Differences between the way expert and novice teachers use
curriculum materials ................................................................................. 90

Figure 10. Components of the teacher — student — curriculum relationship. After
Remillard & Bryans (in press) ..................................................................... 91

CHAPTER 3

Figure 1. Information deﬁcit model - a basic model of pro-environmental
behavior ................................................................................................ 98

Figure 2. The Hines, Hungerford and Tomera model of pro-environmental
behavior (Hines et al., 1986) ...................................................................... 106

Figure 3. The Diekmann 8 Preisendoerfer “Low cost — High Cost Model” of pro-
environmental behavior. Diekmann and Preisendoerfer (1992) .......................... 107

Figure 4. The Kollmuss & Agyeman model of pro-environmental behavior
(Kollmuss and Agyeman, 2002) .................................................................. 108

Figure 5. Plot for the signiﬁcant relationship between prior environmental attitude

and change in environmental knowledge. The equation for the line is y = -0.004 +
0.112x, the Pearson Correlation Coefﬁcient is 0.116, n = 228 ........................... 114

xii

Figures 6a and 6b. The signiﬁcant effects of gender on the pretest

environmental variables. Boys had signiﬁcantly higher environmental knowledge
than girls (p = 0.09, n = 228), and girls had signiﬁcantly higher support for
environmental issues than boys (p = 0.88, n = 228). Data expressed as Mean +/-
S.E.M. Asterisk indicates p < 0.1 ................................................................ 115

Figures 7a, 7b and 7c. Effect of gender on change in the environmental
variables between the pretest and posttest. Girls increased their environmental
knowledge signiﬁcantly more than boys (p = 0.02, n = 228), as well as increasing
their overall score on all 40 items signiﬁcantly more than boys (p = 0.04, n =
228). There was also a trend (p = 0.12) toward girls having a greater change in

behavior than boys .................................................................................. 117
Figure 8. Information deﬁcit model (after Burgess et al., 1998), with the gray

arrow showing the direction of learning as indicated by this study ...................... 120
Figure 9. A simple cognitive psychology model of attitudes .............................. 121

xiii

CHAPTER 1

INTRODUCTION

Over the past few decades, zoos and aquaria have been in the process of
transforming themselves from being essentially collections of exotic animals pulled from
the wild for the purpose of entertainment and recreation, to being centers of conservation
focused on safeguarding animal diversity. Unfortunately, their ability to directly
inﬂuence wild animal populations through captive breeding and reintroduction programs
is somewhat limited, due to factors such as habitat loss; political, social and economic
issues in developing countries; the loss of genetic diversity in small populations; and
maladaptive behaviors that are common in captive populations.

As such, zoos and aquaria are increasingly coming to the realization that they can
make a more signiﬁcant impact on conservation indirectly via education, that is, teaching
people about threats to the environment and biodiversity, fostering positive attitudes
towards the environment, and ultimately promoting environmentally responsible action.
To meet this need, zoos have rapidly expanding their educational offerings, and now
more than 9 million students each year taking part in on-site education programs.
Consequently, modern zoos and aquaria are offered an unparalleled opportunity to make
a signiﬁcant impact on the public’s awareness and understanding of environmental
issues.

Despite this extensive educational activity, we unfortunately know very little
about what children learn in zoos, and even less about how they learn it. In terms of what
students learn, few research studies have quantitatively examined change in
environmental knowledge, attitudes or behavior as a result of a zoo education program.
Those that have were often conducted “in house,” by zoo staff associated with the

program’s development or administration, thereby making their ﬁndings questionable.

The problem is exacerbated by such studies rarely ﬁnd their way into the research
literature. In terms of how students learn, research on learning in informal settings has
traditionally suffered from a lack of a solid theoretical grounding. Few connections are
made to what we know about learning from the wealth of research in school settings, and
few connections are made to what we know about the nature of the environmental
education goals.

This study addresses these problems by measuring what students learn from
education programs taking place at Binder Park Zoo in Battle Creek, MI, and examines
how students learn by analyzing test data from a constructivist perspective. That is, we
examine how learning is taking by connecting learning as a result of the program with
students’ prior environmental knowledge and attitudes.

The study was designed with the following objectives, divided into three sections
representing the following three chapters. The ﬁnal chapter (Chapter 5) examines the
conclusions of the study with respect to these objectives, and discusses future directions

and limitations of the work.

Development and Analysls of a Test Instrument

0 Objective 1

To develop a test instrument designed to measure the impact on Binder Park Zoo’s
education programs on several components of environmental literacy — knowledge of
nature, attitudes towards the environment, support for conservation, and environmentally

responsible behavior.

0 Objective 2
To analyze the functioning of the test instrument using Item Response Theory and the
Rasch measurement model, and to interpret those analyses with respect to a modern

validity framework.

Evaluation of Blnder Park Zoo’s Education Programs

0 Objectlve 3

To measure the extent to which Binder Park Zoo’s education programs were meeting
their objectives through application of the test instrument in a pre/post design.

0 Objectlve 4
To compare the effectiveness of programs that were presented at the 200, with programs
that were presented at schools.

0 Objective 5

To evaluate how teachers use Binder Park Zoo’s education programs, how they integrate
the program in their school-based curriculum, and how Binder Park Zoo can better

develop a collaborative relationship between formal and informal settings.

Examinlng Learnlng through Analysls of the Evaluation Data
0 Objective 6

To examine the relationships between individuals’ environmental knowledge, attitudes

and behaviors.

. Objective 7

To examine student learning from a constructivist perspective by measuring the impact of
students’ prior knowledge, attitudes and behaviors on learning taking place as a result of

the program.

CHAPTER 2
INSTRUMENT

In this chapter I describe the development, analysis and behavior of a test instrument
designed to measure students ’ environmental knowledge, attitudes and behavior. This
instrument is used as part of an education program evaluation in Chapter 3, and is used

to address theoretical perspectives on learning in Chapter 4.

INTRODUCTION

Context In which the Instrument will be used

The instrument described here was developed to measure the educational impact
of environmental programming offered to schools by Binder Park Zoo in Battle Creek,
Michigan. These programs were designed to address both the 200’s conservation
philosophy, and the science education objectives of Michigan’s public schools. The
instrument is also being designed to allow us to ask questions about how environmental
learning takes place in informal science education settings such as zoos. The measures

described here have been designed for students in grades 4 though 7.

Purpose of the instrument

There are three primary purposes for obtaining these measures. The ﬁrst is to
form part of a more complete evaluation of Binder Park Zoo’s education programs that
has the objectives of: demonstrating the program’s worth (to funding sources and the
community); providing an opportunity for receiving feedback to improve programs; and
promoting conservation education by substantiating claims about its beneﬁts (Jacobson,

1991)

The second purpose of collecting these measures was to develop a tool that may
be used by others to evaluate conservation education programs in zoos and other informal
learning environments. The American Zoo and Aquarium Association (AZA) has
repeatedly called for evaluation of zoo education programs to take place, yet no formal
protocols or procedures are given. Any evaluation that does take place is often
superﬁcial, biased, or severely ﬂawed. Consequently, developing, implementing, and

testing evaluation methods are crucial purposes of this research.

Finally, the instrument was being used to investigate the relationships between
environmental knowledge, attitudes and behavior. The structure of the instrument
(described in more detail below) is such that scores on individual scales can be compared
to address these topical questions in environmental education research. Furthermore, the
instrument is being used to examine learning in informal settings from the perspective of
a constructivist view of learning. To this end, prior knowledge, attitude and behavior
scores (obtained from the application of the instrument in a pretest) are being compared
with change in the same factors between pretest and posttest applications. In this way, we
can examine learning as a generative process, with students building upon their prior

understandings, attitudes and habits.

Decisions based on the measures

The decisions based on these measures will be “low stakes”, and will relate
directly to which program objectives are and are not being achieved. For example, if
programs are successful in inspiring support for conservation, but not action, then Binder

Park Zoo would be prompted to more closely examine the relationship between attitudes

and behavior in the context of conservation, and to adjust the focus of their efforts
accordingly. Decisions about the modiﬁcation of program curriculum materials and

program focus will be primary results of these measures.

METHODS

Deﬁnition of the construct

In educational measurement in any context, researchers are concerned with latent
variables; the underlying constructs that measurements are intended to reﬂect. They are
latent, rather than obvious, because they are not observable, and they are variables, rather
than constants, because some aspect of them, such as strength, is prone to change.
Although we cannot directly observe it, the latent variable is presumed to take on a
speciﬁc value under certain conditions, and studies are designed to estimate that value for
each person at the time and place of test administration. As such, regardless of what
method of data collection or observational tool a researcher employs, we are only

estimating the effectiveness of the environmental education on the latent variables.

The latent trait that the instrument under development is designed to measure can
be broadly perceived as the educational impact of the program, but this can be broken
down into the subcomponents that make up this construct. These subcomponents are
knowledge of nature, attitudes toward nature, support for conservation, and pro-

environmental behavior. To better deﬁne these components:

. Knowledge of Nature: An understanding of wildlife natural history, ecosystem

processes, threats to biodiversity, and an understanding of the complexity of

natural systems.

10

' Attitudes toward Nature: An affective response to the natural world, that is, the
visitor’s feelings toward nature are aroused, resulting in a feeling of increased
emotional association and compassion.

' Support for Conservation: An increased willingness in the visitor or student to
support current environmental efforts.

- Pro-environmental Behavior: Translation of the conservation support into actual
behavior. The visitor student changes their behavior and acts in a more

environmentally responsible fashion.

Previous measures of the construct

When attempting to measure the effectiveness of any education intervention,
investigators need to be concerned with the quality and nature of their measurement
instruments. To be robust, the instrument’s validity and reliability need to be established,
yet these criteria are rarely met by tests used in informal settings. The vast majority of
investigators use new instruments constructed speciﬁcally for their current projects. Thus,
it becomes almost impossible to make meaningful comparisons of different instructional
techniques across studies because the comparability of the measurement instruments is
unknown (Lceming et al., 1993). Scale development should take place only if appropriate
instruments do not exist, so before embarking on developing new scales, researchers must
ensure that alternative instruments are not available (DeVeIIis, 2003). Gray et a1. (1985,
p36.) addressed this problem, and concluded, “For the sake of efﬁciency, continuity,
comparability, and quality, future researchers should feel some responsibility to use and

further develop inventories already in existence.”

11

Beginning in the early 1970’s, a great deal of research has been conducted that
attempts to measure the environmental literacy, concern and behavior of various
individuals and groups. The scaling of environmental attitudes and orientations began
with Maloney & Ward (1973), who developed a series of scales to measure verbal
commitment (which measures what a person is willing to do in reference to
environmental issues), actual commitment (which measures what a person actually does
in reference to these issues), affect (which measures the degree of emotionality related to
such issues, and knowledge (which measures speciﬁc factual knowledge related to
enviromnental issues). Since then, these scales have been modiﬁed and updated, and

applied to measuring environmental learning in a variety of settings.

A number of studies have attempted to measure the effects of speciﬁc
environmental interventions, such as school ﬁeld trips to nature centers, workshops, or
instructional programs. Bogner (1998) describes four primary problems that often are
faced by studies that attempt to measure the effects of an environmental intervention: the
lack of reliable, tested measures; small sample sizes; biases resulting from the program
implementer being the same person as the program evaluator; and failure to collect
follow-up data that tests for retention of information and modiﬁcation of attitudes and
behavior over time. It has been shown on numerous occasions that posttests administered
immediately after the program experience are inconclusive (e.g., Crompton & Sellar,
1981; Leeming et al., 1993; Lewis, 1981). Leeming et al. (1993) echo many of the

sentiments of Bogner, and add further criticisms of inappropriate units of analysis, the

12

lack of concern with establishing the reliability and validity of instruments, and the

effects of experimenter expectancy.

Internal Model of the Construct

One way to represent a construct is in the form of an internal model, which
dissects a construct into its various components. In this way, we make explicit the
assumptions about the internal structure of what we are trying to measure. The function

of the internal model is to:

a) Lay the foundation for structural validity arguments. That is, we can compare
the expected internal structure of the construct (internal model) with the actual
structure (as revealed through factor analysis, for example), and draw

conclusions with regard to the structural validity of the instrument.

b) Provide the basis for item development activities. That is, items are developed
to reﬂect the entire range and structure of the construct, as revealed by the

internal model.

As described above, our construct can be broken down into the components of
knowledge of nature, empathy for nature, conservation support and conservation
behavior. Each of these can be deconstructed further into a number of elements that

represent the internal structure of each of the components.

13

The knowledge of nature component relates directly to the content of the
education program, which provides information about animal adaptations, ecosystem
structure and function, the value of nature to humans, and current issues in conservation.
We can deconstruct the empathy for nature component into the various elements of the
relationship between humans and the natural world, such as moral obligations, sustaining
economic development and interference with nature. Support for conservation can take a
number of forms, including being willing to give time or money, and being prepared to
change one’s behavior or inﬂuence the behavior of others. Finally, environmental
behavior takes that willingness one step further, and is composed of factors such as
recycling, energy usage, conservation work, and environmental charity donations. The

internal model of our construct is illustrated in Figure l.

14

Knowledge of Nature

 
   
    
 

lnteractions/ Animal Threats to Value of Nature
Webs/Ecosystems Adaptations Biodiversity to Humans

Empathy for Nature

   
   
   
     
  

Equity and Humans and

 

Humans Interference Duties to
. Development Economy
wrth Nature Non-humans
Issues over Nature
Program
Objectives —>Support for Conservation
Willingness to WIIIIngness t9 Willingness to WIllIngness to
. . Make a FInanCIal . Help Inﬂuence
GIve TIme / Work . . Change Behawor
ContnbutIon Other

Environmental Behavior

Energy Usage Charitable
Donations
Conservation Recycling
Work Membership of
Voting Environmental

Groups

Figure 1. lntemal model of the construct. The model reﬂects the hypothesized structure
of the construct, which is reﬂected in the structure of the test instrument.

15

Hierarchical Organization of the Construct - a Developmental Model

The four major components of the construct described in the internal model can
be place upon a continuum, with those factors that are easiest for program participants to

achieve at one end, and those that are hardest to achieve at the other end, as shown in

 

 

Figure 2.
Easiest to achieve >Hardest to achieve
l I I I
I I I I
Knowledge Empathy Support for Environmental
of Nature for Nature Conservation Behavior

Figure 2. Hierarchical organization of the construct. The model represents the
hypothesized development pathways between elements of the lntemal model.

We can hypothesize that recall of factual information presented in the program
would be easiest for program participants to achieve, followed by the affective response
to nature as measured by the empathy for nature subscale. It would take an additional step
for this affective response to develop into support for conservation efforts, and a ﬁnal
degree of commitment to actually change one’s behavior in a direction supportive of
environmental protection. If this hypothesized structure is reﬂected in the data, that is, if
program participants with low levels of environmental knowledge also receive low scores

on the other scales, then evidence for structural validity will have been provided.

In addition to serving as the foundation for instrument validity arguments, this

hierarchical model is relevant to our research questions about the relationships between

16

environmental knowledge, attitudes and behavior. The classic information deﬁcit model
of environmental education that proposes a linear pathway from knowledge to attitudes to
behavior (we care about what we know about, and our attitudes determine our behavior)
has been questioned in the literature (see Kollmuss & Agyeman, 2002), and so examining
the relationships between these factors in the context of informal education in zoos will

be interesting.
Content Sampling — Test Blueprint

The topics of items are chosen to reﬂect the knowledge, attitudes and behaviors
that could realistically be expected to be held or achieved by 4th through 7th grade
students. Some items were written speciﬁcally for this instrument, and some items were
taken from previously used instruments that measure similar constructs, which were then
modiﬁed to be age appropriate or updated to reﬂect current environmental science. All of
the knowledge items were written to reﬂect the speciﬁc content of Binder Park Zoo’s
education programs. The attitude, support and behavior sets of items were identical for all
students in the study, whereas the knowledge items differed between classes of students
depending on the program in which they took part. That is, if students took part in the
program focusing on the illegal trade in international wildlife and wildlife products, then

the knowledge items reﬂected this content.
The structure of the internal model for each of the components of our construct

was mirrored in the structure of each set of items; the following test blueprint table

(Table 1) gives examples of how items were categorized.

17

Table 1. Test Blueprint. Examples of items in each category, with each category being
created by the intersection of two dimensions: topics (columns) and our construct
elements (rows).

 

 

 

 

 

 

 

 

 

 

The Natural World Conservation Humans and Economy
over Nature

Endangered species There are many Preserving habitats is
are only found in animals and plants that not a vital part of

Knowledge Aﬁ-ica. have not yet been protecting endangered

discovered. species.
I enjoy watching Protecting the Building highways is so
Empathy television programs environment is not my important that
about animals and responsibility. sometimes forests have
nature. to be destroyed.
I would be willing to I am going to talk to I would never donate
S help protect rare my parents about some of my pocket
upport . . . .
ammals, but not conservation Issues. money to a conservation
plants. organization.
I have helped feed or At home, I separate I don’t make a special
Action shelter animals that trash so it can be effort to buy soda in
live In my yard. recycled. containers that can be
recycled.
Types of Items

Ideally, a rating scale consisting of Likert-type items would be used to measure all

 

but the knowledge component of our construct, but this type of indicator can be
unfamiliar and confusing for the age groups upon which this research will be conducted.
Students also differ in the way they use scales of this type, with many students only
selecting the extreme options (strongly agree, strongly disagree), which can lead to
problems when interpreting results. As such, a simpler dichotomous indicator will be
employed, with all items being altemate-choice True / False statements. Items of this type
are sufﬁciently simple for our target age group, but do have the unfortunate side effect of

increasing examinee performance through guessing. However, because our frame of

18

reference is a normative group, this undesirable consequence will be constant for both

groups.

Frame of Reference

The measures resulting from the posttest instrument described here are interpreted
in relation to the pretest scores for the same individual student. That is, the instrument is
norm referenced, with the normative sample being the same students prior to the
intervention. Norm referencing is suitable in this case because we are not interested in the
student’s actual ability, attitudes or behaviors, but rather the change in these traits that

can be attributed to the program.

Pllot testing

Experts in the ﬁelds of informal education, visitor studies, and the environmental
sciences were used to identify potential problems with items, and to make
recommendations for modiﬁcations, if necessary. These items then were pilot tested on
two groups of students in the desired grade levels, one in Okemos, Michigan, the other in
Clara City, Minnesota. Students were asked to circle words they did not understand, and
otherwise indicate any difﬁculties they had with individual items. Results of this pilot

testing were used to make subsequent modiﬁcations to items.

Structure of the Instrument

Each of the four subscales, representing the four components of the construct of
interest (knowledge, attitudes, support, and behavior) contained ten items. The items

were worded in such a way that if a student were to answer all the items in a way that

19

indicated a high level the construct, that is, showing highly positive environmental
attitudes, behaviors and knowledge, then the student would answer 20 items with True,
and 20 items with False. In this way, half the items indicated a high level of our construct
when endorsed, and the other half represented a high level when not endorsed. The
purpose of this was to avoid acquiescence or agreement bias, that being the tendency to
agree with items irrespective of their content (DeVeIIis, 2003). In addition to this
structure, a balance of positive and negative statements was used, with effort made to
avoid confusing double negatives. Analysis of the performance of the these items is being
conducted because some researchers have found that the beneﬁts of using negatively
worded and reverse polarity items may be outweighed by the costs. The costs include
respondents not noticing the reversed polarity, grammatically confusing statements, or
confusing strength of agreement with expressing the strength of the attribute being

measured, and can result in items performing poorly (DeVeIIis, 2003).

The items were arranged on the instrument by alternating items from each
component: that is, an empathy item, followed by a support item, followed by a behavior
item, followed by a knowledge item, and repeat. This was done to remove any effects of
the structure of the instrument on the results, such as might occur if all 10 knowledge
items were grouped together, followed by 10 attitude items, and so on. On the posttest,
six items were added that addressed the student’s enjoyment and impressions of the zoo

program. A full version of the instrument can be found in Appendix A.

20

Test administration procedures

Once a school contacts Binder Park Zoo to make a program booking, the
evaluation study is explained to them, and the correct number of test instruments is sent
out to them, along with consent forms (the consent form used in this study can be found
in Appendix D). Students complete the pretest at school in the classroom. It would be
inappropriate to ask students to ﬁll out the questionnaire immediately after they arrive at
the zoo (or the zoo arrives at the school), where there is high potential for distractions to
interfere with the validity of responses. Further, it is essential that students complete the
pretest and the posttest in an identical environment. The posttest was administered as
close as possible to four weeks after class participation in the program. This delay has
been deemed sufﬁcient by several authors to allow environmental attitudes and behavior
to become constant (e.g. Bogner, (1998), and Lisowski and Disinger (1991)). Review and
meta-analyses have discarded studies that did not observe time spans at all due to
numerous inherent problems with this method (Crompton & Sellar, 1981; Leeming et al.,

1993).

Sampling

The sample of the evaluation data analyzed in this chapter is from two schools
(Explorer Elementary in Caledonia, MI, and White Pigeon Central Elementary in White
Pigeon, MI), representing a total of 278 students from 15 classes, in grades four, ﬁve and

six. This sample was chosen because:

21

a)

b)

Both schools had taken part in the “Tropical Treasures” program, and
therefore I could use the same knowledge scale, making it possible to analyze
the entire instrument (see “Content Sampling” section above for more details).
Teachers and students had clearly labeled and organized the tests, making the
identity of each student’s class clear, as well as making it possible to match

individual student’s pre-tests and post-tests.

These two schools gave us an adequate sample size, with sufﬁcient variation
within the sample to look for how items functioned differently across groups
(see “Differential Item Functioning” in the Analyses section below for more

details).

22

ANALYSES

Item Response Theory and the Rasch measurement model

The instrument was analyzed using Item Response Theory (IRT) and the Rasch
dichotomous measurement model. Item Response Theory (IRT) is an alternative to True
Score Test Theory (TSTT), also called classical measurement theory (CMT), and is
receiving increasing attention in educational measurement (DeVellis, 2003; Embretson &
Riese, 2000). While classical methods of instrument analysis are concerned with groups
of items (scales), IRT focuses on individual items and their characteristics. Items on IRT
scales are designed to be of varying difﬁculty and require different degrees of ability to
answer them, whereas items on classical scales are assumed to be similar in difﬁculty and
measure the strength of the latent trait in the same way. Classical scales are analyzed by
looking at how many of the similar items a student answered correctly, and we analyze
IRT scales are analyzed by looking at which items in the range of difﬁculty were
correctly answered. Item response theory allows the researcher to assess the performance
of individual items, and identify items for which modiﬁcation or removal is required.
With classical methods, we may be able to determine if an item is performing well or
poorly, but the nature of the deﬁciencies remains largely elusive. IRT can assess the

strengths and weaknesses of items much more speciﬁcally (DeVeIIis, 2003).

The Rasch measurement model (Rasch, 1960; Wright, 1999) is an example of a
latent trait or item response model for dichotomously scored performances (correct or
incorrect); other IRT models exist for rating scales and partial credit items. The Rasch

model depicts the student’s ability as a latent trait that drives the student’s observable

23

behaviors. As do all item response models, the Rasch model posits that a sigmoid

mathematical function depicts the relationship between the ability of student n (13"), the

difﬁculty of one step in item 1' (5,1), and the probability that student n will answer item i

correctly, that is, score 1 rather than 0 (lint 1). We can write the Rasch model as:

exp (Bn " PH)
1 1' exp (I3n — I3”)

 

‘I’ni1 =

The relationship can be pictured via Item Characteristic Curves (ICCs; Figure 3), with the
standard curve shown in Figure 3a Figure 3b, illustrates the relationship in the form of
ICCs for two items with different levels of difﬁculty. The curves (items) differ in the
amount of ability required for a ﬁxed probability of a correct answer, the item
represented by the lighter line being more difﬁcult. Item difﬁculty is now no longer
subjective as it is in classical methods, because we now have points on the X axis that
correspond to probabilities on the Y-axis. The purpose of establishing the difﬁculty of
items is to determine how much ability is required to pass them, regardless of the level of
ability in the sample population. Items that are too easy or too difﬁcult contribute little
information. The level of discrimination of an item (how well items discriminate between
high- and low-ability examinees) can be assessed by examining the slopes of ICCs, since
these slopes dictate how much of an increase in ability is required to bring about a change
in the likelihood of getting an item correct. Figure 3c shows two items having different
slopes (measured at the point of inﬂexion). In this example, we can conclude that the

item represented by the darker line has a higher level of discrimination since a smaller

24

increase in ability is required to increase the probability of a correct answer (DeVeIIis,

 

 

 

 

 

 

 

 

 

2003)

a) SINGLE ITEM b) DIFFICULTY c) DISCRIMINATION
E r:
5 II
8 8, 4 . .
a o.
’5 33>. , .
g 5
-‘ n A

Ability (l3) Ability Ability

Figure 3. Item Characteristic Curves. a) ICC for a single item; b) ICCs for two items that
differ in their difﬁculty; c) lCCs for items that differ in their power to discriminate between
examinees with differing levels of ability (after Wright & Masters, 1982; DeVeIIis, 2003).

In addition to item difﬁculty and discrimination indices, the following analyses

also were conducted on the instrument:

Dimensionality

Because the Dichotomous Rasch Model assumes that the construct being
measured is unidimensional, a Principal Component Analysis (PCA) was conducted in
Winsteps (Linacre, 2001) to determine the dimensionality of the instrument. The results
of this analysis will determine if the instrument should be analyzed as four separate
subscales measuring four separate constructs (knowledge, empathy, support and
behavior), or as a single instrument measuring a single construct. The PCA operates by
identifying items that are highly correlated with each other, and that are distinct from

other sets of items that are also highly correlated. More speciﬁcally, PCA divides the

25

total score variance by ﬁnding the groups of items that maximize the variance explained.
The ﬁrst group of items, or factor, is found from those items that result in the largest
eigenvalue, the second factor by the second largest eigenvalue, and so on. The decision to
retain variables was made based on the recommendations of Stevens (1996), whose
criteria are based on the number of items associated with each factor, and the strength
(loading) of that association. Stevens states that components with three or more loadings
having absolute values greater than .80 are reliable, components with four or more
loadings with absolute values of greater than .60 are reliable, and that components having
ten or more loadings need only have absolute values greater than .40, providing the
sample size is above 150. Kaiser’s criterion also was used in the decision to retain
factors, which states that only those factors should that have eigenvalues greater than
1.00 should be retained. A Scree plot also was produced, illustrating factors before the

asymptote being examined for substantive explanations.

Flt

Fit analyses were performed in Winsteps to establish the degree to which the test
data conform to the Rasch model. The results of this analysis were used to determine if
individual items or persons should be dropped, because the validity of parameter
estimates for these people or items is questionable if they do not perform as the Rasch
model predicts. Possible explanations for this misﬁt may be lucky guesses or carelessness
by test takers, multidimensionality in the test, or simply poor quality items. The criteria
for ﬂagging potentially misﬁtting items were an outﬁt mean square greater than 1.4 (or a
standardized mean square greater than 2.0), low or negative point measure correlations,

or an outlying value on the discrimination index; substantive explanations for these

26

misﬁtting items were then sought. The same procedure was followed for misﬁtting

persons.

Rellabllity

To examine the reliability of the instrument, that is, the consistency of test scores
for a group over repeated applications of the instrument, reliability coefﬁcients and

standard errors for items were requested in the Winsteps output.

Differential Item Functioning

Differential Item Functioning (DIF) analysis examines the extent to which
students in different groups differ in their probabilities of answering a particular item
correctly. The presence of DIF may be the result of test bias (items favor one group over
another), non-equivalence of comparison groups, or other factors. We can examine the
differential functioning of items by comparing the characteristics of ICCs of individual
items across groups, ﬂagging items that differ exhibit DIF, and then identifying potential

substantive explanations.

Table 2 summarizes the [RT validity analyses that were performed on the instrument.

These analyses were conducted with the Winsteps computer program (Linacre, 2001).

27

Table 2. Summary of IRT analyses being conducted on the test instrument.

 

 

 

 

 

 

Analysis Purpose

Identiﬁes items that are too easy or too difﬁcult, and contribute
Item Difﬁculty _ . .

IIIIIC Information.

Identiﬁes how well items discriminate between hi gh- and low-
Discrimination . _ .

ability exarnrnees.

Examines the internal structure of the instrument / suitability of
Dimensionality

the Rasch model.
Differential Item Examines the tendency of items to perform differently across
Functioning groups; identiﬁes bias.

Identiﬁes the proportion of test variance attributable to the
Reliability

latent variable — the degree to which scores are free of error.

 

28

RESULTS

The mean score out of a total of 40 on the instrument was 27.2, with a standard
deviation of 4.9. The highest score on the instrument was 38, and the lowest was 11. The
test information function (Figure 4) shows us that the instrument gives us the most
information about students of average ability, with the peak of the curve being just below
zero (we can obtain the most information about students having slightly lower than

average ability).

Test Information Function

 

Information

 

 

-
v

 

-5 -51'52’0’5 ii'é
Measure

Figure 4. Test Information Function for the entire instrument, indicating that the test is
most powerful at differentiating between students having average ability.

29

Dimensionality

In addition to the Rasch factor, Winsteps extracted ﬁve residual factors with
eigenvalues greater than 1. These eigenvalues, along with the percentages of the total

variance explained by each factor, are shown in Table 3 below.

Table 3. The eigenvalues and the percentage of the total variance explained for each
residual factor.

 

 

 

 

 

 

 

 

Factor Eigenvalue % Variance
Rasch 27.3 40.6
1 2.7 4.1
2 1.9 2.8
3 1.9 2.8
4 1.6 2.3
5 1.6 2.3
Residual
Variance
Total 67.3

A Scree plot of these eigenvalues is shown below in Figure 5.

 

 

 

 

i I
t 30 —. i
I .
l 25 ~ g
Q l
2 20 d i
g l
c 15 J I
a l
o— -‘ I
NJ 10 i I
5 i
l k—O O ,

0 HA I I f V ———_T_’ *— V

 

l

l

, -. ,

5 Rasch 1 2 3 4 5 I
a Factor 1

Figure 5. Scree plot of each factor’s eigenvalue.

3O

The line appears to reach an asymptote after the Rasch dimension; suggesting that
only one factor is present in the instrument. We can examine the possible presence of
other dimensions by examining the items that load onto each residual factor. No factors
had loadings greater than .56, so we use Steven’s criterion that requires components to
have ten or more loadings having absolute values greater than .40 to be retained. Table 4

shows the signiﬁcant item loadings for each residual factor.

Table 4. Residual factor item loadings.

 

 

Number of Items with
Residual loadings greater than 0.4 Item Numbers Interpret?
Factor . . .
Posrtrve Negative
1 4 0 22, 26, 28, 39 No
2 1 2 14, 16, 18 No
3 1 1 19, 32 No
4 1 O 1 No
5 1 0 24 No

 

 

As we can see, Stevens’ criteria for retaining any of the residual factors have not been

met (see page 34 above), once again suggesting that the instrument is unidimensional.

If we can examine the content of the items that load onto each residual factor for
substantive explanations, we can see that the three of the four items loading onto factor 1
are “support” items” (the fourth is a “behavior” item) and the three items that load onto
factor 2 are all “attitude” items. The ﬁrst factor’s items, which all load positively are:

22. When I am older I am going to join a conservation group.

26. I would be willing to help clean up a stream where people had dumped trash.

28. I am going to talk to my parents about conservation issues.

39. I have told friends how they can better care for the environment.

The three attitude items that load onto factor two are:

31

14. I am concerned about pollution from industry.

16. Building highways is so important that sometimes forests have to be
destroyed.

18. In order to feed people, we must clear areas of nature to grow foods like

grain.
One could probably argue that there are sufﬁcient commonalities between these items to
warrant their inclusion as distinct factors, especially the three attitude items that all relate
to resources use and the balance of human economy with nature. However, the small
number of items in each factor, coupled with the eigenvalue data and the low loadings, is
sufﬁcient to conclude that the instrument is unidimensional. As such, analysis of the test

data with the Rasch dichotomous model is valid.

Reliability

The Winsteps output gave a Cronbach’s alpha ﬁgure of 0.73. This reliability
coefﬁcient is respectable, especially because we are not dealing with a high-stakes
situation. The decisions that will be made based on this instrument are those concerned
with improving the program through modiﬁcation of the curriculum and its
implementation, rather than any decisions about passing or failing students that may
warrant a higher reliability coefﬁcient. Although by no means strict criteria, DeVellis
(2003) states that his comfort levels for research scales such as this are: below .60,
unacceptable; between .60 and .65, undesirable; between .65 and .70, minimally
acceptable; between .70 and .80, respectable; between .80 and .90, very good; and for
coefﬁcients above .90, the scale is probably too long. Going by this, we can have

conﬁdence in the reliability of this instrument.

32

Fit

None of the items had an outﬁt mean square greater than 1.4 (with 1.31 being the
highest value for all items), but six of the 40 items had a standardized outﬁt mean square
greater than 2.0, these being items 6, 1, 29, 40, 16, and 11. These items therefore should

be scrutinized for oddities.

Looking at the most misﬁtting response strings, we see a predominance of
unexpected incorrect responses (high scoring individuals getting easy items incorrect),
and only two items having a string of unexpected correct responses (low scoring
individuals getting difﬁcult items correct). The items with misﬁtting correct responses
were items 34 (At the store, I tell my parents to use paper bags instead of plastic bags.)
and 35 (I am not a member of a conservation organization.), which were both also
ﬂagged as having the most unexpected responses. These two items were also the most
difﬁcult (or most more unlikely to be answered in a pro-environmental fashion) items on
the instrument, with only 20.6% of students agreeing with item 34, and only 13.5% of

students disagreeing with item 35.

If we look for substantive explanations for the misﬁt, we can see that item 35 is
certainly problematic for more than one reason. First, it is not entirely clear what a
conservation organization is; second, it is difﬁcult for children in the sample age group to
be members of conservation organizations; and third, the item is a double negative, and
therefore potentially confusing. I therefore I have no reservations about dropping the item
from the instrument, especially because its difﬁculty means that it contributes little

information. As for item 34, even environmentalists are not entirely in agreement on

33

whether or not paper bags are preferable to plastic bags, especially if one reuses the
plastic bags, so the polarity of the item is unclear. The action of “telling my parents” may
also be off limits, because one student left me note saying, “I never tell my parents to do
anything. I ask!” Again, because item 34 contributes little information and is potentially

problematic, it should probably be removed from the instrument.

The items having the most misﬁtting incorrect responses (easy items that high
scoring students got wrong) were all knowledge items, such as item 6 (The canopy of the
rainforest is the layer closest to the ground), item 1 (There are no rainforests in the
United States) and item 8 (Many medicines are made ﬁom rainforest plants). Items 3, 2
and 5 also showed several misﬁtting responses. Each of these items represents
information that was presented in the program, and because the data being analyzed here
are pretest data, it is perhaps not surprising that randomness is present in the responses to
these items. We would have to analyze the posttest data for ﬁt to see if these knowledge

items still required consideration for removal or modiﬁcation.

As for misﬁtting persons, the analyses identiﬁed a small number of students
having misﬁtting response strings. These students also had been ﬂagged at the time of
data entry as having either a) answered a small fraction of the items, b) agreed or
disagreed with all or many of the items, or c) alternated between agreeing and
disagreeing throughout the instrument. Unfortunately, some response patterns such as
these are to be expected, given the age of the students involved, but identifying these

students through the ﬁt analyses allows us to remove them from subsequent data analysis.

34

Difficulty

Table 5 shows the difﬁculty indices for each of the 40 items. Because the
difﬁculty values are the fraction of students getting the item correct, a high value

indicates an “easy” item.

Table 5. Difﬁculty indices for each of the 40 items. The ﬁve items having the lowest
a
difﬁculty are labeled , and the ﬁve items having the highest difﬁculty are labeled .

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Item # Difﬁculty Item if Difficulty
1 63.9 21 88.93
2 89.93 22 37.3b
3 88.1 23 85.1
4 78.0 24 63.7
5 80.7 25 75.0
6 69.7 26 83.1
7 69.4 27 64.6
8 87.1 28 50.3
9 93.13 29 56.5
10 85.9 30 66.8
11 46.7 31 81.2
12 93,93 32 54.5
13 83.8 33 80.1
14 77.1 34 20.6b
15 85.6 35 13.1b
16 54.1 36 65.8
17 45,2“ 37 65.7
18 3,7,5b 38 73.3
19 92,13 39 52.7
20 82.6 40 50.4

 

 

 

 

 

 

35

An example of item characteristic curves (ICC’s) for two items (items 12 and 35)
having different levels of difﬁculty is shown in Figure 6. Because these two items are the

opposite extremes of our item difficulty range, all the other items’ curves fall somewhere

between these two.

ITEM CHARACTERISTIC CURVES

1.0 . -

—ltem 35

 

0.75“

Expected Score

0.25”

 

 

 

 

 

Measure

Figure 6. Item characteristic curves for items having the highest difﬁculty (item 35), and
the item having the lowest difﬁculty (item 12).

From Table 5, we can see that items 34 (At the store, I tell my parents to use paper bags
instead of plastic bags.) and 35 (I am not a member of a conservation organization.),

have particularly high difficulty levels, with only 20.6% and 13.4% of students answering

36

these items in line with the expected pro-environmental response. Because these values
are so low, the items tell us little about our population, and so have minimal usefulness
on our instrument. These two items also were ﬂagged for further examination in the “Fit”
analysis above. Other items having high difﬁculty are 17 (I sometimes step on beetles or
spiders on purpose), 18 (In order to feed people, we must clear areas of nature to grow
foods like grain) and 22 (When I am older I am going to join a conservation group). Item
22 is sufﬁciently similar to item 35 for the high difﬁculty to be due to similar reasons,
and so also should be considered for removal. Item 18 reﬂects an understandable conﬂict
between resource development and environmental protection, and indeed, is not an easy
conﬂict to resolve. As for Item 17, however much it may suggest a lack of empathy for
nature, kids just like crushing bugs!

The ﬁve items having the lowest difﬁculty are:
2. Plants and animals in an ecosystem are dependent on each other for survival.
9. Endangered species are only found in Africa
12. It makes me sad to think that some animals may become extinct.

19. Some animals should not be hunted.
21. I would never donate some of my pocket money to a conservation organization.

There appears to be little in the content of these items that would explain their low
difﬁculty and so one would be reluctant to remove any of them from the instrument based
on that statistic alone. In fact, one would even expect item 21 to be a difﬁcult item, given
the high difﬁculty of items 22 and 35, which both also refer to participating with
conservation organizations. Having said that, items having difﬁculty values as low as
item 12 (93.9% of students agreed with the statement), not only give us little information

about the students, but also reduce the ability of the instrument to detect change when

37

used in the zoo evaluation in a pre/post design. As such, it will probably be advisable to

look at the evaluation results both with and without the inclusion of these “easy” items.

Figure 7 below shows the mean difﬁculty for each of the four scales (knowledge,
attitude, support, behavior). The one-way ANOVA detected a signiﬁcant overall
difference in difﬁculty between scales [F (3,36) = 3.039, p < 0.05]. Fisher’s PLSD was
employed for post-hoe comparisons between individual scales. Behavior items were
signiﬁcantly more difﬁcult than knowledge (p < 0.005) and attitude items (p < 0.09), but
not more difﬁcult than support items. Knowledge, attitude, and support scales did not

differ signiﬁcantly in difﬁculty.

 

 

 

 

0knowledge ttitude suport behavir

Figure 7. Average difﬁculty for each of the four scales. All scales except Attitude and
Support are signiﬁcantly different.

38

Differential Item Functlonlng

The DIF analysis compared the behavior of items between the three grades

present in the sample: 4’“, S"1 and 6’“. Table 6 shows behavior comparisons between each

pair of grades for each item. Flagged comparisons represent pairs that are statistically

signiﬁcant (t = :I:1.96) with a DIF contrast of | >.5 |.

Table 6. DIF contrasts between pairs of grades for each item. X represents pairs of
grades that are signiﬁcantly different (I = :I:1.96) with a DIF contrast of | >.5 |.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ltemif 4"‘—5'h 4”‘—6"’ 5”‘-6"‘ Item# 49—5“ 4"‘-6"’ 5"’—6"‘
1 x 21 x x
2 22 x
3 23
4 24
5 x x 25
6 26
7 x x 27 x
8 28 x
9 29
10 x 30
11 31

12 32

13 x 33

14 34

15 x 35

16 x x 36 x x
17 37

18 38

19 39 x

20 4o

 

 

 

 

 

 

 

 

The DIF analyses identiﬁed 14 items with statistically signiﬁcant differences between

grades, that is, 14 items for which the results of one grade differed signiﬁcantly from

39

another. This is not particularly surprising because a number of studies have found that
environmental knowledge, attitudes and behaviors vary with several factors, two of
which are age and level of schooling (Rickinson, 2001). In fact, we would have probably

been more concerned if all items functioned the same in each grade.

40

DISCUSSION

Validity cannot be evaluated solely on a quantitative basis. That is, validity is not
an intrinsic characteristic of a test, but rather a characteristic of a particular use of a test
score; therefore evidence must be sought that supports the notion that a test score is valid
for a particular use or decision. Messick (1989) describes how validity evidence can be
depicted as a crossing of two facets of decision-making: evidence versus consequences,
and interpretation versus use. To illustrate the interrelatedness of all aspects of validity,
Messick introduced a six—component framework that depicts all aspects of validity as
variations of construct-related evidence. His six aspects of validity are content,
substantive, structural, generalizability, external, and consequential. This framework
expands upon previously used and more simplistic deﬁnitions of validity, as described by
Cronbach & Meehl (1955). Below I brieﬂy describe Messick’s six components, along

with a discussion of the validity evidence that aligns with each component.

Content

Content validity refers to evidence of content relevance, representativeness, and
technical quality. Efforts to insure content validity included pilot testing; review of the
test instrument by environmental biologists, teachers, and measurement experts; and the
Fit analyses conducted using the Rasch model. The pilot testing and content reviews led
to a number of item modiﬁcations prior to these analyses, which greatly improved the
content of the instrument. The Fit analyses ﬂagged a small number of items for

consideration for modiﬁcation or removal. I concluded that two of these (34 and 35)

41

should be removed from the instrument, thereby increasing its content validity. Further
application and analysis of the “new” version of the instrument would be necessary to

fully understand the effect of removing these items.

Substantive

Substantive validity refers to the extent to which the processes involved in
examinees generating a response are representative of and relevant to the intended
construct. Our developmental model predicted that environmental knowledge would be
easier to achieve than a positive attitude toward the environment, which in turn would be
easier to achieve than environmentally responsible behavior. This hierarchy is realized in
the difﬁculty data, with ﬁgure 6 showing expected relationships between the four
subscales.

Although not discussed above, we can also ﬁnd evidence for substantive validity
in the change between pretest and posttest scores as discussed in Chapter 3. The predicted
increase in scores as a result of the zoo education program suggests that the instrument is

indeed measuring what it was intended to measure.

Structural

Messick’s structural validity component refers to the ﬁdelity of the scoring
structure to the structure of the construct, that is, the structure of what it is we are trying
to measure. The internal structure of the construct as represented in the internal model

can be compared to the results of the dimensionality analyses, with consistency between

42

the two being evidence for structural validity. The dimensionality analyses (including
examination of the eigenvalues and factor loadings) revealed that the instrument was
unidimensional, in that the 40 items did not segregate into discrete factors. This suggests
that environmental knowledge, attitudes, support and behaviors are not discrete latent
traits, and that there are signiﬁcant connections and/or correlations between them. This is
in agreement with both our hypothesized internal model, as well as our developmental /
hierarchical model that illustrates the connections between the four. The relationships

between these factors are discussed in more detail in Chapter 4.

Generalizability

The generalizability component refers to which score properties and
interpretations generalize across groups and settings. Our reliability score of 0.73
(Cronbach’s alpha) was concluded to be acceptable, suggesting that the scores were
relatively free of error, and that the majority of the variance in scores was due to the
latent variable. The DIF analyses showed that some items did perform differently across
different age ranges, but because age previously has been shown to be a factor that

inﬂuences environmental literacy, this is not surprising or troubling.

We also can examine the reliability of the instrument by seeing how different
groups rank the forty items in terms of difﬁculty. That is, we can compare the item
difﬁculty rankings for the two schools used in these analyses (Explorer Elementary and
White Pigeon Central Elementary) with agreement between the two being evidence for

the generalizability of the instrument. Table 7 shows these data.

43

Table 7. Comparison of item difﬁculty rankings between two schools, Explorer (n=151),

Central (n=127).

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Ex Iorer Central Ex lorer Central
Item # R afrkl n g Ranking Difference Item # R afiki n g Ranking Difference
1 18 14 4 21 36 36 0
2 37 35 2 22 3 4 -1
3 34 37 -3 23 3O 29 1
4 20 27 -7 24 19 13 6
5 33 22 11 25 24 21 3
6 17 19 -2 26 25 32 -7
7 22 15 7 27 12 17 -5
8 31 34 -3 28 9 8 1
9 40 39 1 29 11 10 1
10 29 31 -2 30 14 20 -6
11 6 7 -1 31 32 23 5
12 38 40 -2 32 13 6 7
13 17 33 -6 33 28 25 3
14 21 24 -3 34 2 2 O
15 35 28 7 35 1 1 0
16 5 18 -13 36 23 1 1 12
17 8 5 3 37 15 16 -1
18 4 3 1 38 16 26 -10
19 39 38 1 39 10 9 1
20 26 30 -4 40 7 12 -5

 

 

 

 

 

 

 

 

The two schools (n1 = 151, n2 = 127) ranked all 40 items astoundingly similarly, with no

signiﬁcant differences between the two in either a paired t-test or a signed rank test

across the set of 40 items.

External

The external validity component includes convergent and discriminant evidence

from comparisons with other methods of measuring the same construct. No other

methods of measuring our latent variables were used in this study Therefore, before

44

further use of this instrument, it would be advisable to interview students after they had
completed the test to see how their written responses corresponded with their intended
and verbally stated explanations. Similarly, scores from other measures, such as
application tests or scenario responses, could be correlated with scores on this instrument
in a multi-method multi-trait matrix in order to provide more evidence of external
validity.

Having said that, the data on the role of the age variable described in Chapter 3,
and the data on the role of gender described in chapter 4, are in agreement with previous

ﬁndings on the role of those factors.

Consequential

Consequential validity refers to the implications of score interpretations, as well
as the actual and potential consequences of test use. Chapter 3 of this dissertation
explores the application of this instrument in evaluating the educational impact of
environmental programming offered by Binder Park Zoo in Battle Creek, Michigan. As

such, the next chapter explores the consequential validity of the instrument.

45

CHAPTER 3
EVALUATION

46

In this chapter I describe a model-driven evaluation study of Binder Park Zoo '3
conservation education programs. The eﬂectiveness of the programs is examined from
the perspective of both the zoo and the schoolteachers. The test instrument described in

Chapter 2 is applied to address some of these questions.

INTRODUCTION

Literature Review and Theoretical Framework

Conservation education is becoming increasingly emphasized in the modern zoo
as the foundation to the fulﬁllment of its mission. With increasing human populations and
decreasing biodiversity, the desire to understand the natural history of animals has been
joined by the need to teach conservation of their populations and ecosystems. To meet
this need, over the past few decades zoos and aquariums have rapidly expanded their
conservation, education and research programs. Now more than nine million students per
year visit on-site education programs at zoos and aquariums (AZA, 2004), which are thus
given the opportunity to make a signiﬁcant contribution to the public’s awareness,

attitude and behavior toward conservation objectives.

The goal of most education programs offered by zoological parks is to increase
awareness and concern about entire ecosystems and their plight, and to shape behavior
concerning the environment and conservation (Bogner, 1998). It is hoped that when such
environmental education is integrated with a natural experience, it will foster positive

attitudes and promote environmental action. Fundamentally, the goal of such educational

47

approaches is to promote environmental literacy and to mold a public that is capable and

willing to take action (Hungerford & Peyton, 1978; Hungerford & Volk, 1990).

Unfortunately, little is known about how zoos’ education efforts translate to
learning outcomes (Braveman & Yates, 1989). Researchers have focused on demographic
information about visitors; how zoos are utilized; how visitors move through zoos; the
way visitors use their time at zoos; and the social nature of visits (Churchman, 1985). The
majority of the literature is either based on self-reports or is entirely descriptive in nature;
educational impact in terms of quantitative grth in knowledge has not been well
studied. Further, the majority of research on education in informal settings has been
conducted in museums; studies conducted in zoos are “extremely sparse” (Bixler, 1995).
Research showing that different settings elicit different learning behaviors makes the
assumption that museum research results may be generalized to zoos suspect (Garner,
1990, Carlin, 1999). Jacobson (1991) points out that “the limited ﬁrnds available for
conservation education demand the effective use of evaluation to ensure successful

programs.”

What research has been performed is variable, both in terms of quality and
conclusions. Improvements certainly have been made since 1963 when Heini Hediger
expressed his sorrow over the "shameful and regrettable fact that most zoological gardens
are still not as yet well organized as they should be for educational work with schools and

with the people who are looking for spare time activities" (Hediger, 1963). Almost 40

48

years later, research is continually being published that suggests we still have a long way

to go.

Koran and Baker (1979) found no evidence supporting the advantage of ﬁeld trips
and museum visits over conventional classroom instruction. Additionally, Tunnicliffe
(1997) found no marked distinction between the content of conversations between
schools groups and family groups at zoos, “strongly suggesting that schools are failing to

make effective use of the educational potential of zoos.”

More positively, Marshedoyle et al. (1982) found that learning as deﬁned by their
study did occur during a school ﬁeld trip to the zoo, although they treated this result with
skepticism in light of the wealth of research that suggests the opposite. The authors also
examined teachers’ objectives for bringing their class to the zoo, as well as their
satisfaction with the program. They found that teachers generally were satisﬁed with the

visit, but also expressed a desire for an expanded role of 200 education services.

Falk (Balling & Falk, 1982; Falk & Dierking, 1997, Falk et al., 1998) has
performed extensive research on various aspects of learning in novel ﬁeld trip
environments. One conclusion of particular relevance to the grade data collected in his
study was the ﬁnding that ﬁfth grade students learned better in a novel environment than
in a familiar setting, whereas they found the opposite result in third grade children. They
concluded that ﬁfth grade children were particularly receptive to education during 'ﬁeld

trips to zoos or museums, and that signiﬁcant learning could occur (Balling & Falk,

49

1982). This view is supported by other research that shows that children’s attitudes
toward conservation begin to develop at an early age (Miller, 1985; Bryant and
Hungerford, 1977; Chawla, 1988; Lyons & Breakwell, 1994). In relation to other studies
on learning on novel ﬁeld trips, Falk and Dierking (1997) found that narrow deﬁnitions
of learning (presentation of facts) might be the cause of previous studies’ failures to ﬁnd
differences in learning. Falk et al. (1998) went on to conclude that there were individual
differences in what is learned during such experiences, and that measures should be based

on degree of change in knowledge.

Bruce Carr, the Director of Conservation Education at the American Zoo &
Aquarium Association (AZA), has reviewed data from recent unpublished investigations
and concluded that “zoos and aquariums do successfully communicate a conservation
message to the public, but the message is often subtle, short-term, and not frequently tied
to speciﬁc understandings developed within the zoo or aquarium” (Carr, 2001). The
AZA, in its accreditation requirements, calls for zoos to evaluate their education
programs on a regular basis to ensure program effectiveness and inclusion of modern

scientiﬁc content (AZA, 2002).

Problems associated with zoo education programs are plentiful and diverse,
although common themes can be identiﬁed: few connections are made to students’
curricula to introduce and reinforce their science learning; little groundwork is
undertaken to prepare students for the educational experience; and insufﬁcient

assessment and modiﬁcation of education programs takes place. This study addresses

5O

each of these three points by conducting a theory-based evaluation of education programs
offered to local schools by Binder Park Zoo, located in Battle Creek, Michigan. Binder
Park Zoo, a private not-for-proﬁt zoo, has perhaps the most developed education
department of all Michigan zoos. It is due to this commitment to advance conservation
through education that Binder Park wishes to evaluate comprehensively its education

programs.

The evaluation requested by Binder Park Zoo was essentially summative in nature
The education department, and those to whom they report, wanted to learn to what extent
they are effective in achieving their program goals. However, we added formative
evaluation elements to the investigation (data collection on how the program is operating,

not just what it is producing) to provide a more complete and informative evaluation.

To thoroughly and accurately evaluate any program, one must develop a full
understanding of that program. One method of achieving this understanding is through
developing a theory of the program that expresses the hypotheses upon which a program
is based in terms of a series of sequential, causal elements (Weiss, 1998). For example, a
program that seeks to reduce the incidence of a disease may be grounded in the
hypotheses that people would become educated about transmission, have methods of
prevention made available to them, and through this they would change a critical lifestyle
element, thereby reducing transmission, ultimately reducing the number of new
infections. Wholey (1987) describes program theory as identifying “program resources,

program activities, and intended program outcomes, and speciﬁes a chain of causal

51

assumptions linking (them)”. The primary advantage of this method is that not only does
one receive a measure of how much change has occurred, one also deve10ps an

understanding of how this change occurred (Weiss, 1997).

The development of the program theory model (Figure I) greatly aided us in
identifying the multiple objectives that the zoo’s education program seeks to meet, as
well as the pathways to, and program elements associated with, those objectives. As such,
a further use for the program theory is to guide the evaluation and aid in identifying the
relevant variables. The primary objective of zoo education programs is to elicit an
affective response from the students, moving them toward greater understanding of
ecological and conservation issues, and ultimately positive action on behalf of the

environment (Berkovitz, 1988).

Jacobson (1991) describes four main outcomes of such theory-based evaluation of

conservation education programs, in that they may:

0 Provide accountability in demonstrating the program’s worth — to funding
sources, the community, and other groups;

0 Offer an opportunity for receiving feedback and improving programs;

0 Further understanding of the process of program development; and

0 Promote conservation education by substantiating claims about its beneﬁts.

52

In addition to endeavoring to achieve each of these outcomes on various levels, in this
study we seek to advance the methods used to evaluate education programs in zoos and
other novel learning environments. The AZA repeatedly has called for evaluation of zoo
education programs to take place, yet no formal protocols or procedures are given. What
evaluation that does take place is often superﬁcial, biased, or severely ﬂawed.
Consequently, developing, implementing, and testing evaluation methods is crucial. A
full set of objectives for the evaluation can be found in the “Purposes of the Evaluation”

section below.

Bogner (1998) describes four primary problems that often are faced by studies of
this kind: lack of reliable, tested measures; small sample sizes; biases resulting from the
program implementer being the same person as the program evaluator; and failure to
collect follow-up data that test for retention of information and modiﬁcation of attitudes
and behavior over time. It has been shown on numerous occasions that posttests
administered immediately after the program experience are inconclusive (e.g., Crompton
& Sellar, 1981; Leeming et al., 1993; Lewis, 1981). These and other methodological

considerations are discussed in the following sections.

53

BACKGROUND INFORMATION CONCERNING THE PROGRAM

Origin of the Blnder Park Zoo Education Program

Binder Park Zoo, over the past twenty years, has developed a series of education
programs that address both the zoo’s conservation philosophy and the science education
objectives of southwest Michigan’s public schools. The programs are aimed primarily at
grade levels K through 7, and deal with subjects such as the international wildlife trade,
ecosystem function, and animal adaptations to the environment. In the past ten years the
program has more than doubled in size, which has been in line with the zoo’s ‘double the

zoo’ mission that led to creation of the award-winning Wild Aﬁica exhibit.

Goals of the program

The ﬁrst set of goals, the zoo mission objectives, are deﬁned in terms that
encompass the conservation philosophy of Binder Park Zoo, along with that of the
American Zoological Association (AZA) and the larger, global conservation community,
that is: “... to nurture empathy, understanding and conservation of nature.” In terms of
education goals, the ‘conservation of nature’ element of the mission statement may be
broken down into ‘support for conservation’ and ‘conservation action.’ The evaluation
instrument (see Chapter 2 for a complete description) that was used to evaluate the zoo
mission objectives was developed from existing instruments, such as those developed by
Maloney and Ward (1975); Maloney et al. (1975); and Bogner & Wilhelm (1996). Some
items were taken directly from these previous studies, some of the items were newly

written, and others were updated and modiﬁed for the age group of the study.

54

Conveniently, the individual elements of the zoo’s mission statement (empathy,
understanding and conservation of nature) closely mirror scales used in previous studies:

ecological attitudes, behavior, and knowledge.

It is no longer sufﬁcient for a zoo to market its education programs to schools
solely under the premise of promoting conservation. Zoo education programs must be
designed also to be of beneﬁt to teachers and schools as they plan their curricula and
prepare their students for standardized tests. Binder Park Zoo attempts to achieve this
through:

a) Aligning the content of their programs with state science objectives, as
deﬁned by the Michigan Essential Goals and Objectives for Science
Education (Michigan State Board of Education, 1991); and

b) By providing teachers with a pre and post-visit instructional materials, to
introduce concepts to prepare students prior to the visit, and to reinforce
concepts back in the classroom.

These goals, therefore, provide us with a second set of objectives that we must evaluate.

Individuals who use the program

Binder Park Zoo primarily attracts visitors and students from southwest and
central Michigan, although visitors attend from around the country and the world.
However, the education programs market themselves solely to the Battle Creek and
surrounding school districts. Despite offering programs to the entire K-12 range, the

education department’s offerings focuses on grades K through 7. Binder Park Zoo offers

55

education programs without regard to race, color, religion, natural origin, disability, age,

gender or sexual orientation.

Characterlstlcs of program materials, activities, and administrative arrangements

The zoo’s programs can be split into two categories: 1) the traditional ﬁeld trip,
referred to by the zoo as Schools on Safari, and 2) ‘zoo-to-school’ programs, that
transport animals and docents to the schools to present the programs there, referred to by
the zoo as Zoomobile programs. Until a few years ago both of these two program types
were both offered during the season when the zoo was open, but in order to expand their
offerings throughout the entire year, the zoo now offers the Schools on Safari programs
only during the season when the zoo is open to the public (April to October) and the
Zoomobile programs during the closed season (October to April). Data on the
effectiveness of both types of program were collected during this study to allow a
comparison of the effectiveness of presenting the program. The content and presentation
of these two program types is essentially identical, but the students in the school-to-zoo
programs are involved in a tour around the zoo, which to some extent allowed us to
address questions regarding the importance of a natural experience in environmental

education.

Binder Park Zoo advertises its programs to schools through television,
newspapers, and by directly contacting schools within the district. The primary source for
ideas for programs is within the zoo, that is, programs are developed to align with the

200’s and educators’ skills, interests and focus, rather than through any needs assessment

56

of the schools’ requirements. Some programs, such as the Suitcase for Survival program,

are nationally distributed and developed outside of the zoo.

8)

b)

All program types consist of the following materials:

“Scripts” — the content that is to be presented to the students is written formally in
what the zoo refers to as scripts. The use of this word is interesting, because it
suggests that the presenters must adhere closely, if not verbatim, to the materials.
This is matter for concern, since research suggests that education can be more
effective when teachers and students interact with curriculum materials, and
construct an enacted curriculum with respect to their prior experiences and

understanding .

Demonstration animals — each program uses live demonstration animals to
illustrate various points, such as adaptations or comparative morphology. These

animals include parrots, kinkajous, and sloths.

Pre- and post-visit materials - Binder Park Zoo provides schools with materials to

prepare the students prior to the visit, and to reinforce concepts back in the

classroom after the visit.

Program instructors vary with regard to their backgrounds; some are former

conservation biologists, and some are recent zoology graduates. Few of the instructors

57

have formal training or experience in education. A docent, who helps with the

demonstration animals and equipment, usually accompanies the instructors.

58

DESCRIPTION OF THE EVALUATION STUDY

Purposes of the evaluatlon

The evaluation was designed primarily to determine the degree to which Binder
Park Zoo is effective in reaching its various related objectives (as described in more
detail in the Program Effectiveness section below) for its education program; to ascertain
which objectives are and are not being met; and to identify explanations for why certain

goals are not being met.

By evaluating the Zoo Mission Objectives, we are measuring how effective the
zoo is in teaching speciﬁc program content, stimulating a positive affective response to
nature, and encouraging environmentally responsible behavior. We also are looking to
gain an understanding of the connection between environmental knowledge, attitudes and
behaviors; to develop a greater understanding of the knowledge with which students enter
the program; and to determine how students’ prior understandings and experiences

inﬂuence the success of the program. These ﬁnal objectives are described in Chapter 4.

The Curriculum Objectives are being examined to learn about how teachers use
the program, and how they incorporate program content into their school-based
curriculum. Interpretation of this data will result in advice fore zoo on how they can
better assist teachers and schools in maximizing the effectiveness of the ﬁeld trip

experience; to develop a greater understanding of the relevance and role of deﬁned

59

curriculum elements to teachers; and to gain a greater understanding of the role of pre

and post visit instruction materials.

By evaluating the 200’s education programs that take place in school classrooms
in addition to those that take place at the zoo, we intend to establish the extent to which
the zoo environment ameliorates Binder Park Zoo’s educational programs and aids in the

achievement of its education objectives.

A further purpose of this study is to aid in the development of methods that may
be utilized to constructively evaluate education programs in zoos, as well as education in
other novel ﬁeld trip environments. The identiﬁcation of multiple program objectives; the
use of the theory-based approach; and the development and analysis of the subscale-
based instrument, will all aid in this purpose. Chapter 2 dealt speciﬁcally with the

development and analysis of the test instrument.

60

METHODS

Program Proﬁle

The program theory model (Figure 1) illustrates the Binder Park Zoo education
program in terms of its inputs, processes and outputs. The program theory is in no way
designed to be tested or proved; rather it is used to guide the evaluation, and aid in

identifying variables and deciding what data to collect.

INPUTS

 

 

- Curriculum materials, “scripts”
. Pre and post visit materials

. Demonstration animals

- State science standards

- Students

I Instructors

 

 

PROCESSES

- Program presentation

- Program location (school or
zoo)

- Use of pre and post visit
materials

I Integration into the school-
based curriculum

 

 

 

 

OUTPUT

I Change in environmental
knowledge, attitudes and
behavior

I Learning relevant to the
school-based curriculum

I Teacher satisfaction

 

 

 

 

Figure 1. Program Theory Model. A theoretical representation of how Binder Park Zoo’s
education programs operate.

61

Furthermore, the theory serves to assist in interpretation of evaluation results by allowing
us to identify failing program elements. Utilizing this theory-based approach to
evaluation, therefore, seeks to provide us with a clear perspective on what, how, and why
change occurred. Please refer to the following sections (Deﬁnition of Variables /

Discussion of Variables) for explanations of the variables within the model.

Binder Park Zoo’s education programs are a diverse group of offerings that
attempt to maximize the potential of the 200’s collection, exhibits and staff for education.
The program process begins with schools receiving brochures containing descriptions of
the various programs, along with the Michigan K-12 Science Objectives (Michigan State
Board of Education, 1991) to which each program corresponds. Teachers select a sub-
program based on factors such as their curriculum needs, their interests, and their
students’ interests. Teachers who repeatedly take their classes to the zoo each year may
try a new sub-program each year, or stick with one they have found successful in the past.
The school pays a per-student fee and arranges a day for the visit. Employed interpreters
present the programs, which take place both in classrooms at the zoo and at the exhibits
throughout the zoo, depending on the selected program’s focus. All programs use animals
and animal artifacts as much as possible, and emphasis is placed on deduction and

discovery.

Definition of Variables

Table 1 shows how the program objectives relate to the variables that were measured in
the evaluation. The methods of measurement and more detailed explanations of the

indicators and instruments can be found below.

62

Table 1. Variables, indicators and measures.

WHEN

 

 

 

 

 

 

 

 

 

 

 

 

 

 

VARIABLES INDICATORS MEASURES MEASURED TO WHOM
Zoo Mission “To nurture empathy, understanding and
Objectives conservation of nature ”
Empathy for Understanding of, Test; empathy
Nature and compassion for for nature Pre- :1; tPOSt' Students
“A ttitude ” nature subscale
Conservation Willingness to T68” .
. conservatron Pre- and post-
Support support conservation . . Students
.. Suppo rt .. efforts support vrsrt
subscale
Conservation Behaving in an T68” .
. . conservatron Pre- and post-
Actlon envrronmentally . . . Students
,. . ,, . action vrsrt
Behavior responsrble manner
subscale
Understanding Knowledge of Ties” prog
. content Pre- and post-
of Nature spec1ﬁc program . . Students
.. ., knowledge vrsrt
Knowledge content
subscale
Use of materials,
Program student engagement, Program During the Zoo
Implementation other inﬂuences on observations visit Educators
student learning
Curriculum “All programs meet Michigan Essential Goals and
Objectives Objectives for Science Education ”
Use of Pre and Post Teacher Pre- and post-
"t t 'al . 't Teachers
Integration into vrsr ma en s surveys v1s1
the Cumculum Use of state science Teacher Pre- and post- T
. . eachers
standard surveys v1srt
Teacher Meeting the teachers Teacher Pre- and post-
. . cumculum needs and . . Teachers
Satisfaction . surveys vr51t
expectations

 

 

 

 

63

Discussion of Variables and Methods of Measurement

a) Zoo Mission Objectives.

The set of subscales developed in the classical studies by Maloney and Ward
(1973), subsequently modiﬁed by Maloney et al. (1975) provides us with the basis to
develop an instrument capable of measuring affective responses to nature, as well as
differentiating between environmental concern and environmental behavior. The
instrument we used to measure the zoo mission objectives is composed of four subscales,
these being empathy for nature, conservation support, conservation actions and
understanding of nature. The items in the ﬁrst three subscales are taken either directly
from previous studies (Maloney and Ward (1973); Maloney et al. ( 1975); and Bogner &
Wilhelm (1996)); modiﬁed from previously used items to insure accurate and
contemporary content as well as suitability for the age groups of children involved in this
study; or they were written speciﬁcally for this study. The fourth subscale, understanding
of nature, involves questions that deal with material covered in each of the speciﬁc
programs, so therefore were all newly written. These subscales differ accordingly with

each program (the instrument and its items can be found in Appendix A and B).

Subscales for each of the zoo mission objectives are used because an adequate
measure cannot be derived from responses to single questions. The assumption with
using scales of this type is that any errors that occur with responses to individual items
will cancel out each other over the suite of items. Summing the rating from each item is

then performed to ﬁnd the total score on each scale (Bogner & Wilhelm, 1996).

64

Each item in the instrument requires a true or false answer; the only items for
which there is a deﬁnite correct answer are those from the Understanding of Nature
(Knowledge) subscale. All other items measure an attitude, a behavior, or the intention or
willingness to perform a behavior. There are ten questions within each subscale, ﬁve of
which require a positive response, and ﬁve negative (if one is to answer all questions in
line with showing a high level of the construct). Please refer to Chapter 2 for a more

detailed description of the test instrument.

b) Curriculum Objectives

Teachers completed a pre- and post-visit survey instrument (see Appendix C),
which included questions about the teacher’s use of pre- and post-visit instructional
materials; the extent to which they were integrating the program into the school-based
curriculum; and their awareness and use of the connections made to state science
education standards. Teachers also were asked about their biggest concerns when
planning a ﬁeld trip, and how they prioritized various factors such as cost, learning, and
enjoyment. The post-visit surveys also asked teachers to provide feedback on their
satisfaction with the program, and to make recommendations for how to improve the ﬁeld

trip experience.

In addition to these teachers surveys, all programs were observed and transcribed
in order to identify unexpected factors that might inﬂuence learning; to see how
instructors adhered to or interacted with the curriculum materials (scripts); and to assess

how engaged the students were in the program.

65

Evaluation Design

For the design to be truly experimental, we would have had to assign some classes
that wished to visit the 200 to a control group, preventing them from participating in the
program. Ethical and economic considerations prevent use of this design, therefore this
study falls into the quasi-experimental category as described by Stanley & Campbell
(1963). Nevertheless, the design is a common and valid approach for such a research
question, and because we were interested primarily in change between the pretest and
posttest, the pretest essentially was functioning as a control. The design of the zoo

mission objectives portion of the evaluation took the form shown in Figure 2:

01 X3 02
03 Xb 04

Figure 2. Experimental design (after Campbell & Stanley, 1963). Where O1 and 03 are
pretests, 02 and 04 are posttests, ‘and X is the treatment, in this case the education

program (Xa represents the Schools on Safari programs, Xb represents the Zoomobile

programs).

It was unnecessary to add a group to control for students learning solely from
repetition of the test. This possibility was minimized because the length of time between
pretest and posttest was sufﬁciently long (over four weeks), and the students did not
expect the posttest. This potential problem was further minimized by randomization of

the posttest question order.

66

Students completed the pretest at school in the classroom. It would have been
inappropriate to ask students to ﬁll out the questionnaire immediately after they arrived at
the 200, where there is high potential for distractions to interfere with the validity of
responses. Further, it was essential that students completed the pretest and the posttest in
an identical environment. The posttest was administered as close as possible to four
weeks after participation in the program. This delay was deemed sufﬁcient to allow
environmental attitudes and behavior to become constant by a number of authors (6. g.
Bogner, (1998), and Lisowski and Disinger (1991). Papers and meta-analyses have
discarded studies that did not observe time spans due to numerous inherent problems with

this method (Crompton & Sellar, 1981; Leeming et al., 1993).

The test instrument was subjected to pilot testing, involving approximately two
hundred 4th through 6th grade students completing the test. Modiﬁcation of the items took
place based on this feedback, with reference to both statistical analysis and comments
from teachers, students and test experts. Refer to Chapter 2 for more information on the

test instrument.

It was ensured at all stages that zoo staff were not involved in the evaluation in
any way, and conversely that the school teachers or evaluators were not involved in the
zoo education program, thereby preventing another source of bias from entering the study

(Lemming et al., 1993, Bogner, 1998).

67

Sampling

During 2002 and 2003, all schools booking programs at Binder Park Zoo for
students in grades 4 through 7 were contacted and asked to take part in the evaluation.
Most schools were happy to take part, although some were unable to participate due to
time constraints, such as the need for parental consent for each student required for
human subjects research (UCRIHS) approval. Two schools completed the pretest, but
were not able to complete the posttest, and so were not included in this study. The
analyses below represent data from nine different schools, each of which sent between 50
and 150 students to participate in the zoo programs. A total of 1503 test instruments
(pretests and posttest combined) were analyzed, from approximately 750 students (not
exactly half, because a small number of students only completed a pretest or posttest due
to absence from school). This data set represents 64,629 item responses. Twenty-two

teachers completed the pre— and post-visit teacher surveys.

68

RESULTS

This results section is split into four parts:
1. Analysis of the Test Data.
2. Integration of the Program into the School-Based Curriculum.
3. Teacher Satisfaction.

4. Program Observations.

1. Analysis of the Test Data

Two-way ANOVAs were conducted to examine the effects of the program
(comparison of pretest and posttest data) and the effect of program location (Zoomobile
vs. Schools on Safari) for each environmental variable (knowledge, attitude, support,
behavior, and “total ” — a composite total score for the entire instrument). Figure 3 shows
the effect of the program and the program location for each of the four environmental
variables and the total test score. Main effects of program location (school or zoo) were
found for knowledge [F (1,1498) = 23.26, p < 0.0001], attitude [F (1,1499) = 28.38, p <
0.0001], support [F (1,1499) = 73.163, p < 0.0001], behavior [F (1,1499) = 10.56, p <
0.005] and total scales [F (1,1499) = 61.36, p < 0.0001]. A main effect of program
(pretest — posttest comparison) also was found for the knowledge scale [F (1,1498) =
37.58, p < 0.0001], with no interaction between location and program. That is, the
increase in student knowledge between the pretest and posttest was not dependent on

program location.

69

 

Pretest

'co

 

 

G)

Knowledge
.‘1 .

 

sos ZMB ' I 363 ZMB

 

 

Support
Behavior

 

 

   

SOS ZMB

 

Total

 

wsos .ZMB

Figure 3. Interaction bar plots for the effect of program location (Schools on Safari vs.
Zoomobile) and the zoo program (comparison of pretest and posttest data) for each
scale (knowledge, attitude, support, behavior and total test score). Data expressed as
Mean +/- S.E.M.

70

Because program impacts for the two locations differed signiﬁcantly on all scales,
the effects of program were analyzed separately for the Zoomobile and Schools on Safari
locations, and the p-values for each of these analyses are listed in Table 2, with the
signiﬁcant effects on the program being highlighted in bold. There was a signiﬁcant
increase in both environmental knowledge [F (1,704) = 28.01, p < 0.0001] and attitude [F
(1,704) = 4.34, p < 0.04] for students in the Zoomobile program, as well as a signiﬁcant
increase in total test score [F (1,704) = 4.90, p < 0.03]. No effects of the Zoomobile
program were found for the support or behavior variables. Students in the Schools on
Safari program showed a signiﬁcant increase only in environmental knowledge [F
(1,794) = 13.16, p < 0.0005]. See Figures 4a and 4b for bar graphs depicting signiﬁcant

program effects.

Table 2. P-values for the one-way ANOVAs examining the effect of the program on
student knowledge, attitude, support, behavior, and total test score, for each program
location. Signiﬁcant p—values are shown in bold.

 

 

 

 

 

 

Zoomobile Schools of Safari
p-values p-values
Knowledge <0.0001 0.0003
Attitude 0.0376 0.9740
Support 0.8467 0.2034
Behavior 0.6055 0.2937
Total 0.0272 0.8837

 

 

 

 

 

71

ZOOMOBILE

 

 

 

 

 

 

  

 

n
.U
.. *
a)
O)
U
2
3
O
C
x
*
Q)
'0
3
'33
<
*

,

_74

g ..
*—

.6-

157 .
.5-

 

 

pre test post test

Figure 4a. Plots of the environmental scales with signiﬁcant program effects for the
Zoomobile programs. Zoomobile participants' posttest scores were signiﬁcantly
higher than pretest scores for Knowledge, Attitude, and Total test score. Data
expressed as Mean +/- S.E.M. Asterisk indicates p < 0.05.

72

SCHOOLS ON SAFARI

*

 

KNOWLEDGE

 

 

 

pre test post test

Figure 4b. Effect of the Schools on Safari program on student knowledge. Schools
on Safari participants’ posttest scores were significantly higher than pretest scores I
for Knowledge variable only. Attitude, support and behavior variables were not
affected. Data expressed as Mean +/- S.E.M. Asterisk indicates p < 0.05.

Because students in the study were in grades 4 through 7, two-way ANOVAs
were conducted to examine the effect of the student grade level and the effect of the
program for each environmental variable. Mean scores for knowledge [F (3, 1494) =
18.706, p < 0.0001], attitude [F (3, 1495) = 4.002, p < 0.008], support [F (3, 1495) =
23.92, p < 0.0001], behavior [F (3, 1495) = 9.815, p < 0.0001], and total test score [F (3,
1495) = 8.990, p < 0.0001], differed signiﬁcantly by grade. Main effects of program were
found for knowledge [F (3,1495) = 31.87, p < 0.0001], and total test score [F (3,1495) =
8.99, p < 0.04]. To examine in more detail the role of grade in program effectiveness,

one-way ANOVAs were conducted comparing the pre- and post-scores for each

73

environmental variable separately for every grade. Signiﬁcant increases are highlighted
in Figure 5, and summarized in Table 3. Fourth grade students showed a signiﬁcant
increase for knowledge [F (1,458) = 9.22, p < 0.003] and attitude [F (1,458) = 2.95, p <
0.09] but not support, behavior, or total. Fiﬁh grade students showed signiﬁcant increase
only on the knowledge scale [F (1,317) = 8.22, p < 0.005]. Sixth grade students
signiﬁcantly increased on both knowledge [F (1,85) = 13.14, p < 0.0005] and total scales
[F (1,85) = 2.83, p < 0.1], but not attitude, support, or behavior scales. Seventh grade
students signiﬁcantly increased only on the knowledge scale [F (1,634) = 12.811, p <
.0005]. In addition, seventh grade students scored signiﬁcantly lower than all the other

grades on all scales except knowledge, as indicated by the letters (a, b and c) in Figure 5.

Table 3. P-values for the one-way ANOVAs examining the effect of grade on program
impact. Signiﬁcant results are shown in bold.

 

 

 

 

Environmental Variable
Knowledge Attitude Support Behavior Overall
(p-value) (p-value) (p-value) (p-value) (p-value)
Grade Fourth 0.0025 0.0864 0.6772 0.4646 0.341 1
Fifth 0.0044 0.7257 0.9960 0.2642 0.2756
Sixth 0.0005 0.3319 0.5599 0.4530 0.0963
Seventh 0.0004 0.5590 0.1 145 0.2658 0.9122

 

 

 

 

 

 

 

 

 

 

74

 

Support

Knowledge

 

 

Attitude

  
  

 

 

7' Fourth ' Flfth

 

 

Behavior

   
 

'5 Fourth Fifth

 

Figure 5. Effect of the program on each environmental variable by grade. Data
expressed as Mean +/- S.E.M. Asterisks * indicate a signiﬁcant between the pretest
and posttest scores. One asterisk (*) indicates p < 0.1, two asterisks (**) Indicates p
< 0.05. Different letters represent signiﬁcant differences between grades (p < 0.1).

75

2. Integration of the Program Into the School-Based Curriculum

On the pre-visit teacher survey, the teachers were asked in open questions what
factors they consider when planning ﬁeld trips such as the trip to Binder Park Zoo. The

common factors that they listed are shown in Figure 6.

III;

Educational Program Transportation Sufﬁcient Student
Value Cost Chaperones Enjoyment

 

16

Number of Teachers
on

 

 

 

 

Figure 6. Primary factors that teachers consider when planning ﬁeld trips.

On the post-visit teacher surveys, teachers were asked about the extent to which
they used the Binder Park Zoo’s pre- and post-visit materials, the parallels made to state
science objectives, and the extent to which they had integrated the program into their

school-based curriculum. Figure 7 shows the results of these questions:

76

Use of Pre- and Post-visit Materials:

El Used Signﬁcantly
' I Used Some
IE Didn't Use

 

Use of Connections to State Science Standards:

El Used
Didn't Use
1:] Not Aware

 

Extent of Integration of the Visit into the School-Based Curriculum:

El Integrated
I Somewhat
I] Not Integrated

 

Figure 7. Teacher responses on the post-visit survey to questions regarding the use of
pre and post-visit materials, the parallels made to state science objectives, and the
extent to which they had integrated the program into their school-based curriculum.

77

3. Teacher Satisfaction

Looking at the post-visit teacher surveys, the teachers were, on the whole, very
satisﬁed with the program, with 59% rating the experience as Excellent, 32% giving the

program a Good rating, and 9% rating the program as Fair.

Responses to the post-visit survey question “How could Binder Park Zoo design
its programs to better meet your education objectives, tie in with your curriculum, or help

prepare your students for the MEAP?” fell into a few discrete of categories:

1. Comments on logistical Issues:

“The room (gym) was not the best location for the program. The sound system did not
work properly and there were too many students present. Some things were diﬂicult to
see or hear. "

”Next year we plan on bringing more chaperones, but that ’s a school issue! ”

P

“Lacks pre program organization prior to entering the zoo. ’

“I would like to see the program done in smaller groups so the students could hear and
see better. "

P

"Let us know the times of the programs and location before we get there. ’

“Our students need lots of exposure and time with these types of activities and
presentations, but in smaller group settings. ”

2. Comments on the range of programs offered:

9

“Oﬂer more programs. ’

"Oﬂer a wider variety. "

It

More programs and better advertisement of programs available ”

78

3. Comments on the content of programs, and correspondence with the school-

based curriculum:

"Could you include some plants in addition to animals and how the food web ties it all
together? "

“Focus on food chains and food webs in the rainforest. ”
”I think the program content tied in well with our science education objectives. "

“Possibly to hear what types of adaptation and/or habitats certain animals in the
rainforest have. ”

“Follow-up activities in the classroom. Graph information and we have the students
interpret it. ”

“The program reinforced the conservation concepts I am teaching rainforest ﬂoor and
levels! Great! "

4. Comments on student Involvement in the program:

“It would have helped If the students had more hands on with different skulls. ”
“Appreciate the hands-on opportunities and student involvement. ”

“The interaction with the children was excellent. "

”Student interaction kept students engaged.

79

4. Program Observations

Instructor’s Use of the Scripts

The program instructors adhered extremely closely to the materials given to them
by the 200, both in terms of program content and structure, and the language used to
present the material. On occasions when different instructors presented the same program
on different occasions to different audiences, very few differences were seen in the
presentations. The only notable observed differences were in the questions asked by the
students, but after answering any questions the instructors always quickly got back to the

script.

Organizational Issues

Teachers and chaperones often seemed confused as to where and when the
programs were taking place. On a number of occasions teachers were frantically rushing
around the education building and students were being shuttled into the zoo classroom
after the program had started, disrupting the presentations considerably. This disorder
frequently led to teachers and students becoming annoyed and frustrated. It should be
noted though that this was only the case with the Schools on Safari programs; the
Zoomobile programs ran very smoothly, with students ﬁltering in and out of classrooms
and assembly halls in an organized manner. It was not, however, clear if the confusion at
the zoo arose as a result of a lack of information being given to the teachers, or through

teachers and chaperones struggling to manage a large population of excited children.

80

DISCUSSION

Test Data

To summarize the major ﬁndings from the test instrument data:

1.

Binder Park Zoo’s education programs are making a signiﬁcant impact on
students’ knowledge of environmental issues and science.

The Zoomobile and Schools on Safari programs both signiﬁcantly impacted
student knowledge to the same extent.

The Zoomobile programs stimulated an affective response, and signiﬁcantly
improved students’ attitudes towards the environment.

All four grades (fourth through seventh) signiﬁcantly increased their scores on the
knowledge variable.

Seventh grade students had signiﬁcantly lower scores on the instrument than all

other grades.

Binder Park Zoo should take great satisfaction from the ﬁrst ﬁnding, especially

because we have seen from many of the studies cited in the introduction to this chapter

that quantitative change in knowledge is an extremely rare commodity in studies on

learning in zoos. The same signiﬁcant impact on students’ environmental knowledge

being found for both the Schools on Safari programs and the Zoomobile programs

(ﬁnding 2) is also encouraging, in that both program locations are effective in increasing

student knowledge. Fostering such awareness of environmental issues is an essential goal

81

of environmental education, because if individuals are to know how to act to lower their
impact on environmental problems, they must have knowledge of the problems and their
causes. That is, a narrative account of phenomena is one essential component of deep
understanding. However, one of the limitations of the instrument used in this study was
its measurement of only this narrative, or information-based aspect of the cognitive goals
of environmental education in the knowledge variable and scale. It would be more
informative to extend this variable to include understanding, which is the student’s ability
to reason and to make connections between data and models through inquiry or
application. Foreseeing the environmental consequences of our actions requires such
reasoning. Correlating such a variable with environmental attitudes and pro-

environmental behavior would perhaps show us a stronger and causal relationship.

The ﬁnding that the Zoomobile programs signiﬁcantly increased students’
environmental attitudes while the Schools on Safari did not is both encouraging and
curious. At ﬁrst glance, the concept that the Zoomobile programs were more effective
than the Schools on Safari programs seems counter-intuitive, in that one would expect a
program presented at the zoo and accompanied by a day at the zoo to be more effective in
increasing students’ feelings of attachment with the natural world than a program
presented in a school classroom, gymnasium or assembly hall. So what might account for
the differences between the effectiveness of the Zoomobile and Schools on Safari
programs? As described in the discussion of teacher comments and program observations
below, the Schools on Safari programs were signiﬁcantly more disorganized and chaotic

than the Zoomobile programs. This certainly should be considered a signiﬁcant factor

82

that may be holding back the impact of the Schools on Safari programs. The issue of
novelty may also be playing a role, in that the novel zoo environment may be not be
conducive to student learning. This aspect is discussed in more detail below in the

discussion of the signiﬁcance of differences in program effectiveness between grades.

The information deﬁcit model (Kollmuss & Agyeman, 2002; Burgess 61 al., 1998)
of environmental education posits a linear progression from environmental knowledge to
environmental attitudes, to pro-environmental behaviors. While there are many problems
with this model (the model is discussed and analyzed at length in the next chapter), we
can use it to consider how successful Binder Park Zoo’s education programs are at
achieving the 200’s conservation education objectives. Figure 8 (below) illustrates the
model, along with annotations (gray arrows) illustrating the results of the test data. If the
ultimate objective of environmental education is to foster environmentally responsible
behavior, then we can see that the Zoomobile programs are making signiﬁcant progress

toward achievement of that goal.

pschoolson Safari>

Environmental Environmental Pro-environmental
Knowledge . Attitudes Behavior

Zoomobile 7 g . >

Figure 8. Information deﬁcit model of environmental education (after Burgess et al.,
1998) with annotations (gray arrows) showing the impact of Binder Park Zoos two

program types.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

83

Should Binder Park Zoo be satisﬁed that their school programs are increasing
students’ environmental knowledge and attitudes in the Zoomobile programs, but only
knowledge in the Schools on Safari programs? Probably not. The potential of informal
learning environments extends far beyond providing information to students and visitors,
especially because that information is available ﬁom an array of other sources such as
television, books and school-based instruction (Webb, 2000). Rather, zoos have great
potential to foster an affective response to nature, and to change students’ attitudes
toward the environment. Learning in zoos can and should extend beyond the cognitive

domain, and into affective and behavioral domains.

Data on the impact of grade level on student learning tells us two things. First,
students in all grades signiﬁcantly increased their scores on the knowledge variable
between the pretest and posttest (Figure 5). Coupled with the Zoomobile / Schools on
Safari data, we can now say that Binder Park Zoo was effective in increasing student
knowledge of environmental issues across the entire range of variables. From an
environmental education standpoint, it is interesting to know that a single program was
effective in increasing knowledge across a fairly wide age range,. The ﬁnding also speaks
to the quality of the instrument in being able to function similarly and detect differences

across the intended age range.

The second ﬁnding regarding grade is that seventh grade students scored

signiﬁcantly lower than all the younger students on the test instrument on all the variables

except knowledge. The seventh grade students scored lower than other grades in both

84

pretest and posttest scores. This ﬁnding supports research on novelty and informal
education that has found that children in grades 4 through 6 are ready for and can thrive
on ﬁeld trips to novel environment such as zoos, whereas children in the grades below
and above often ﬁnd novelty a distraction from learning (Balling et al., 1983). The
hormonal and brain changes occurring during adolescence (Giedd et al., 1999) may well
be too much of an obstacle for the 200’s education program to overcome, but it may just
be that these children need more than one trip to the zoo to overcome the negative effects

of novelty.

85

Integration Into the Curriculum

Figure 6 shows us that the main factors teachers consider when planning ﬁeld
trips to sites like Binder Park Zoo are the educational value of the experience, and the
degree of correspondence with the school-based curriculum. These data from the pre-visit
teacher survey do not, however, tie in with the teachers’ accounts of their use of pre- and
post-visit materials, their use of the connections made to state science objectives, or their
integration of the program into the school-based curriculum, as described on the post-
visit teacher survey, illustrated in Figure 7. Only a small proportion of teachers actively
used the pre- or post-visit materials in the classroom, and few took advantage of the
connections that the programs made to state science benchmarks. Only 15% of teachers
felt they had successfully integrated the 200’s education program into their school-based

curriculum.

So how can we explain the discrepancy between the importance that teachers’ put
on curriculum ﬁt (as seen on the pre-visit survey) and their lack of integration of the zoo
ﬁeld trip into their school-based curriculum (as seen on the post-visit survey)? On ﬁnding
a similar result in a study involving ﬁeld trips to a science center in Vancouver, Canada,
Anderson & Zhang ( 1993) argued that because teachers are required to justify ﬁeld trips
to administrative authority, the high value placed on curriculum ﬁt might be connected
more to the need to secure legitimacy rather than with any genuine desire to integrate the
ﬁeld trip experiences into the school-based curriculum. While this explanation might well
explain the discrepancy between these two sets of results, we believe that more research

involving teacher interviews would be necessary to draw any speciﬁc or ﬁrm

86

conclusions. However, because an isolated ﬁeld trip experience alone is unlikely to have
any signiﬁcant long-term impact on student Ieaming, it is certainly evident that methods
need to be developed that encourage teachers to take advantage of pre— and post-visit
materials, and to greater integrate the ﬁeld trip into their school-based curriculum.

Otherwise the potential of a rich and powerful experience may be lost.

87

Teacher Satisfaction

The teachers were generally pleased with the education program their students
took part in, with 91% of teachers rating the program “Good” or higher. The four
categories of comments do, however, provide us with some information from which to

make recommendations as to how the zoo might improve its offerings.

The “Comments on logistical issues” tell us two things. First, the teachers coming
to the zoo for the Schools on Safari programs needed more information or clearer
instructions on when and where their programs were taking place, both before coming to
the zoo, and upon arriving at the zoo. This ﬁnding was echoed in the program
observations, and possibly reﬂected in the differences seen in the test data. Second, the
teachers found that the Zoomobile programs were conducted in groups that were too
large for the students to take advantage of the interactive elements of the program.
Instead, the students in large groups were reduced to observers rather than participants in

the program.

The teachers’ comments on program content of the programs suggested a desire
for closer correspondence with the school-based curriculum. Science benchmarks are
increasingly moving toward integration and making connections between topics, and this
trend was reﬂected in the teachers’ comments. Teachers wanted their students not only to
learn about individual animals, but also to understand those animals’ roles within the
ecosystem, and how they interact with other plants and animals in food webs.

Furthermore, teachers requested that the program extend beyond biology by giving the

88

students graphical information for them to interpret back in the classroom. Teachers also

called for a wider variety of programs covering different and more diverse topics.

89

Program Observations

Observations of the teachers’ confusion about the time and location of the Schools
on Safari programs have been discussed in the Teacher Comments section above, so I

will not refer to them further here.

The other set of program observations concerned the instructors’ use of the
curriculum materials. Consistent with Binder Park Zoo’s shorthand for these materials as
scripts, it was found that the instructional materials were followed extremely closely by
all instructors, with very little variation in structure or content. Remillard & Bryans (in
press) make a number of distinctions between the way novice and expert teachers use

curriculum materials, which can be placed along a continuum, as shown in Figure 9.

Following Drawing on Participating with

 

4’

Novice Expert

Figure 9. Differences between the way expert and novice teachers use curriculum
materials.

Remillard describes novice and the most ineffective instructors as those who
closely follow instructional materials; enacting a planned curriculum with as much
ﬁdelity as possible. The Expert and most effective teachers, on the other hand,
collaborate and interact with the materials and their students to design the enacted

curriculum. Figure 10 illustrates this participatory relationship.

90

  
 
   
  
 

TEA CHER ‘ CURRICULUM

‘

STUDENTS

Enacted
Curriculum

 

Figure 10. Components of the teacher - student — curriculum relationship. After
Remillard 8 Bryans (in press).

80, unless it is desirable that the zoo instructors present the same identical
program material each time, the question becomes “what can be done to encourage the
instructors to move along the continuum in Figure 9, and to have a more dynamic
relationship with the curriculum materials?” There are three ways this can be achieved.
The ﬁrst is through experience; as instructors become more familiar with the content,
they become more comfortable interacting with the materials. For example, many ﬁrst-
year graduate students follow the lab manual to the letter when teaching undergraduate
introductory biology. As they become increasingly conﬁdent, they integrate their own
experiences and try new activities as their familiarity with teaching the class increases
(personal observation). The second is through training and professional development
activities, which can prompt teachers to think about how they use curriculum materials

and encourage them to try different approaches. The third approach is through developing

91

curriculum materials that encourage instructors to engage in a collaborative /
participatory relationship with the materials and their students. This is no small task,
especially because it is even questionable whether curriculum materials alone can change
teachers’ often stable models of curriculum material use. However, it is worth
considering that it is the materials themselves that are inﬂexible and thus responsible for

overly rigid instruction.

92

CHAPTER 4

LEARNING

93

In this chapter I explore the relationships between environmental knowledge, attitudes
and behavior, in terms of both a linear progression (information deﬁcit) model and a
constructivist view of learning. The test data collected in Chapter 3 forms the basis of

these analyses.

INTRODUCTION

The assertion that research into informal learning in settings as zoos suffers from
a lack of theoretical grounding certainly holds true for many studies published during the
20th century. Researchers predominantly have focused on demographic information about
visitors; how zoos are utilized; how visitors move through zoos; the way visitors use their
time at zoos; and the social nature of visits (Churchman, 1985). The majority of the
literature is either based on self-reports or is entirely descriptive in nature; educational
impact in terms of quantitative growth in knowledge has not been well studied.
Deﬁnitions of learning have been consistently absent from the literature on learning in
informal settings, with the majority of studies being concerned only with the products of
learning rather than the process of learning. Despite the previously somewhat superﬁcial
approach to studying learning in museums, zoos, and nature centers, researchers are
coming to the realization that “knowing how people learn might be more important than
knowing what they learn” (Lucas et al., 1986). It is this change of approach that is
leading to the most signiﬁcant applications of learning theory to research in informal

settings. In this introduction I will discuss:

1. The constructivist view of leaming;

94

2. The goals of environmental education: knowledge, attitudes and behavior;
3. Barriers to pro-environmental behavior; and

4. The application of the constructivist view to environmental education.

The Constructivist View of Learning

For over a hundred years, education researchers have acknowledged that learners
bring their own unique perspectives to the learning setting. James (1890) asserted that
even the environment encountered by infants is not full of equally attractive stimuli, but
rather the child is innately more conscious of certain types of information than others.
Similarly, new knowledge and understanding is constructed in elementary school
children through adults based on the learner’s prior experiences, knowledge, skills, and
beliefs (National Research Council, 2000). That is, the contemporary view of learning
recognizes the role of these previously fashioned factors in the construction of new

knowledge, problem solving, and memory.

This view, referred to as the constructivist view of learning, has had far-reaching
implications throughout formal education. For example, in order to use prior knowledge
as a starting point for new instruction, teachers must develop an understanding of that
knowledge, along with the misconceptions, false beliefs, and naive notions held by their
students. Further, this view urges teachers to encourage students to construct their own

knowledge through active learning rather than strictly constraining the learning process.

95

In the past decade, constructivist learning principles slowly have begun to be
applied to informal settings. Museums, science centers and zoos are inherently
constructivist learning environments, where learning is mediated through interactions
with the physical and social environment, and interpreted with respect to the visitor’s
prior knowledge and experiences. Researchers have recognized the great variability in
visitors to these institutions, in terms of their ages, backgrounds, experiences, and level of
interest. In fact, due to the artiﬁcial structure of school and classroom populations, the
diversity that exists in visitors to free-choice learning institutions is even greater than that
within classrooms (F alk & Adelman, 2003). The Ausubelian maxim (Ausubel 1968): the
single most important factor inﬂuencing learning — ‘what the learner already knows’
applies in zoos as much as it does in schools. This point is supported by Falk and

Dierking (1992), when they comment:

Museum visitors must somehow perceive information before they can store it in
memory. Under normal conditions, people pay attention to things that interest
them. Their interests are determined by their experiences, knowledge and
feelings. This is a classic feedback loop: people learn best those things that they
already know about and interest them, and people are interested in those things

they learn best.

This variability presents educators with many challenges in targeting learners, and

similarly presents researchers with many confounding factors in their attempts to measure

learning. Despite these challenges, it is essential that we develop an understanding of the

96

impact of the variability in visitors’ interest, prior knowledge and experiences on their
learning in free-choice settings. If we do not recognize and account for this diversity in
the visitor population, research can easily lead to a misrepresentation of an institution’s

impact.

Without explicitly referring to constructivist views of learning, Falk and Adelman
(2003) have made one of the few attempts to apply these principles to learning in
informal settings. By acknowledging the variability in prior knowledge and attitudes of
aquarium visitors, they examined how these factors contributed to the degree of change
(learning) due to the visit. It is this sort of approach that I believe is vital if we are to
understand how environmental learning takes place in informal settings and improve the
effectiveness of educational interventions. Before examining in more detail the goals of
environmental education, and the theorized relationships between them, I should make it
clear that my deﬁnition of learning (especially with respect to environmental education)
is not restricted to the cognitive domain, but rather is expanded to include affective and
behavioral components. This is perhaps a slightly unconventional view of Ieaming, but
because the goals of environmental education (described below) reach beyond knowledge
and understanding, and that learning is required to achieve these affective and behavioral

goals, then I believe that our deﬁnition of learning should reﬂect these objectives.

97

The Goals of Environmental Education: Knowledge, Attitudes and Behaviors

The linear progression (shown in Figure 1) from knowledge to attitudes, to
behaviors, represents one of the earliest models of environmental education, and has been
called an information deﬁcit model (Burgess et al., 1998). In fact, if we remove the
“environmenta ” label from this diagram, the progression from knowledge to attitudes to
behaviors represents the fundamental model of many governmental and non-
governmental organizations’ public awareness campaigns, from programs encouraging
the use of car seatbelts to those attempting to reduce the transmission of sexually
transmitted diseases (as such, many of the general implications for environmental

education described in the last section of this paper also may be applied to other settings).

 

 

 

Environmental Environmental Pro-environmental
Knowledge Attitudes Behavior

 

 

 

 

 

 

 

 

 

Figure 1. Information deﬁcit model - a basic model of pro-environmental behavior.

However, soon after their inception in the 1970’s, these models were found to be
ﬂawed, with very few studies ﬁnding a signiﬁcant correlation between the three factors.
Below I describe what is currently known about environmental knowledge and attitudes

independently, and then examine their impact on environmental behavior.

98

Environmental Knowledge

Rickinson (2001) conducted an extensive review of the literature on learners’
environmental knowledge, attitudes, and behaviors. With regard to factual knowledge,

the following trends were found:

I Levels of factual environmental knowledge are generally low.

I Factual knowledge varies by environmental topic area (plants vs. animals, global
vs. local issues).

I Factual knowledge varies across several factors (gender, level of schooling, socio-

economic factors).

The author also looked at studies that examined conceptual understanding of

environmental issues, rather than just factual knowledge, and found the following trends:

I Conceptual understanding of environmental problems is even more limited than
factual knowledge.

I Persistent misconceptions exist regarding the science of environmental issues.

I Difﬁculties in understanding often relate to external inﬂuences (e.g., schooling,
the media).

I Conceptual understanding varies with several factors (e.g., age, gender).

99

Although there are always a few exceptions, most studies ﬁnd that only a small
amount of the variation in environmental behaviors can be explained by environmental
knowledge. Among many studies corroborating this, Kempton et al. ( 1995) surveyed
members of environmental groups and anti-environmentalists, and found that the
knowledge of environmental issues in both groups was surprisingly low. While this study
suggests that environmental knowledge is not a requirement for pro- (or anti-)
environmental behavior or activism, others, such as Diekmann and Preisendoerfer (1992),
have found that a thorough, technical understanding of environmental issues does not
necessarily lead to pro-environmental behavior. Kollmus and Agyeman (2002) assert that
“the longer the education the more extensive is the knowledge of environmental issues.

Yet more education does not necessarily mean increased pro-environmental behavior.”

Environmental Attitudes

Newhouse (1991) deﬁnes an attitude as “an enduring positive or negative feeling
about some person, object or issue,” and is distinguished from a belief, which is deﬁned
as “the information that a person has about a person, object or issue.” In addition to
factual knowledge and conceptual understanding, Rickinson (2001) also looked at
learners’ environmental concern and attitudes. With regard to environmental concern, the
key trends in the literature are:

I Young people see some environmental issues as more important than others

(global greater than local).

I Young people are concerned, and often pessimistic, about the future.

100

I Environmental concerns vary across several factors (e.g., gender, age, geographic

location, socio-economic grouping).

With regard to environmental attitudes, Rickinson’s key ﬁndings were that:
I Young people generally hold positive environmental attitudes.
I Young people are less pro-environmental in relation to issues directly affecting
their own lives.
I Environmental attitudes vary across several factors (e.g., gender, age, socio-

economic grouping, academic ability).

Hines et al., (1987) conducted a meta-analysis of 51 studies examining the
relationship between environmental attitudes and behavior, and found only a modest
correlation between the two. Other researchers don’t go nearly as far. For example Hines
(1980) found very little evidence for a causal relationship between attitudes and related
behaviors in a comprehensive review of the literature. As such, the question becomes,
“why do people behave pro-environmentally?” or perhaps, more interestingly, “what are

the barriers to pro-environmental behavior?”

101

Barriers to Pro-environmental Behavior

Rajecki (1982) was one of the ﬁrst to seek explanations for the discrepancies
between environmental knowledge, attitudes and behaviors. He describes four signiﬁcant

causes:

1. Direct vs. Indirect Experience — Raj ecki asserts that having indirect experience with an
environmental issue (such as formal classroom instruction) will not be as powerful as a
direct experience, and so consequently attitudes formed by direct experiences are more
likely to impact behavior. Fazio and Zanna (in Rajecki, 1982) offer three explanations for

this connection, in that direct experience may

a) provide more and more accurate information;
b) cause one to focus on their own behavior, and that it is that behavior that
drives the attitude, rather than the other way round; and

c) may involve repetition, and may therefore be more memorable.

Similarly, Maitney (2002) found that pro-environmental behavior change is more likely
to persist if it is grounded in and driven by “signiﬁcant and meaningful experience”.
Conversely, behavioral change that is driven by incentives, regulations or anxiety is less

likely to persist.

2. Normative Influences — These are deﬁned as social and cultural inﬂuences that may

prevent a person from behaving in line with their attitudes. Newhouse (1991) provides

102

the example of a banquet where, despite her feelings about wasting food, the author does
not stand up and lecture everyone about how much food they should take. The
implications of these norms on conservation are straightforward, in that when social
norms for a particular behavior exist, then attitudes are a good predictor of behavior.
When they do not exist, and a culture promotes a behavior that is not pro-environmental

or sustainable, then the discrepancy between attitude and behavior will be greater.

The issue of normative inﬂuences also may have implications on the
measurement of attitudes and behaviors, especially when self-reports are used. If there is
a social norm against a certain behavior, then respondents are more likely to acquiesce
and reﬂect that behavior in their responses, regardless of their actual behavior. The last

two of Rajecki’s explanations also deal with measurement issues:

3. Temporal Instability — This factor highlights the reality that peoples’ attitudes change
over time, and that data on attitudes and behaviors should be collected at as short an
interval as possible if meaningful comparisons are to be made. Furthermore, the more
stable an attitude that is drawn on to direct a behavior, the better a predictor of behavior it
is. If attitudes toward a speciﬁc issue are developed from an unstable set of attitudes (e. g.,
an attitude about battery recycling being developed from general attitudes about

recycling), then behavior will be less predictable (Schwartz, 1978).

103

4. Lack of Correspondence Criteria — A consistent problem in many studies that have
failed to ﬁnd any correlation between environmental attitudes and behavior is the use of
measures that do not deal with directly relevant topics. That is, the attitude scales often
do not measure attitudes toward the same environmental issue as the behavior scales. For
example, one would not necessarily expect a high degree of correlation between attitudes
toward reintroducing wolves and recycling behavior, yet often conclusions are drawn
from similar comparisons. Further, the attitudes that are measured are often much more
general than the behaviors, (e.g., Do you care about the environment? vs. Do you recycle
batteries? (Kollmus & Agyeman, 2002)). However, Newhouse (1991) points out that
given our lack of understanding of the relationship between attitudes and behaviors, it is

not always apparent which attitudes correspond with which behaviors.

These above three paragraphs highlight frequent methodological problems in
environmental knowledge / attitude / behavior measurement, and remind us how
complicated it is to design experiments that measure and compare attitudes and behaviors
(Kollmuss & Agyeman, 2002). In fact, Newhouse (1991) asserts that many, if not most,
researchers believe that attitudes do play a signiﬁcant role in shaping behaviors, but

methodological problems often prevent the relationship from being realized in the data.

Hines et al., (1987) expanded Rajecki’s list by introducing the concept of locus of
control, which refers to people’s feelings about how signiﬁcantly they can impact an
issue through their behavior. He distinguishes between people with an external locus of

control (who see change as the result of a higher power, those in authority, or chance) and

104

those with an lntemal locus of control (who see their own behavior as being able to
positively inﬂuence issues). Predictably, Hines found that individuals with an internal
locus of control were more likely to participate in pro-environmental behavior than
people with an external locus of control. Similarly, Gigliotti (1992, 1994) examined the
relationship between environmental behavior and attitudes regarding the ability of
science and technology to solve environmental problems. He found that people who
believe technology provides the solution to environmental problems are less likely to
engage in pro-environmental behavior. Hines et al. ( 1987) also deﬁne other factors, such
as an individual’s sense of personal responsibility, and knowledge of what actions to take
to lower an individual’s impact on an environmental issue, as being signiﬁcant in the
formation of pro-environmental behavior. Figure 2 depicts Hines et al.’s model of
environmental behavior, in which the term “situational factors” refers to issues speciﬁc to
individuals and individual issues, such as optional methods of reacting to an

environmental issue, economic constraints, and social pressures.

105

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Attitudes —' 332223;”
Locus of Personality
Control Factors
Pro-
Personal _ Knowledge of w. Intention to * environmental
Responsibility Issues Act Behavior
Knowledge of
Action 4
Strategies
Action Skills H

 

 

 

 

Figure 2. The Hines, Hungerford and Tomera model of pro-environmental behavior
(Hines et al., 1986)

The issue of the cost of environmentally responsible behavior was raised by
Diekmann and Preisendoerfer (1992), who attempted to explain the lack of correlation
between attitudes and behavior with a low-cost / high-cost model. They deﬁne “cost” as
the time and effort it takes to engage in environmentally responsible behavior, and
suggest that when the cost of the behavior is low, then attitudes are a reliable predictor of

behavior. Figure 3 illustrates this relationship.

106

 

 

Impact of environmental attitude
on pro-envrronmental behavror

Cost of pro-environmental behavior

Figure 3. The Diekmann & Preisendoerfer “Low cost — High Cost Model” of pro-
environmental behavior. Diekmann and Preisendoerfer (1992)

One would conclude from this model that pro-environmental attitudes can positively

inﬂuence low-cost environmental behaviors.

Expanding this notion of cost to include not only time and effort, but also
monetary expenditure, we see that economic incentives may encourage people to act pro-
environmentally, regardless of their knowledge of, or concern for, the issue. For example,
taxation of harmful environmental practices has been used in many countries to
encourage less damaging behavior. Von Weizaecker and Jesinghaus (1992) found that
people in countries with a high gas tax drive less than their counterparts in countries with

a low gas tax.

Kollmuss and Agyeman (2002) proposed that perhaps the most complicated

model of environmental behavior to date (greatly simpliﬁed below in Figure 4) involving

107

a number of “internal factors”, such as knowledge, values, and attitudes; “external
factors”, such as cultural norms and economic and political pressures; being mediated
through a number of barriers, such as lack of incentives or feedback; and shaped by old
behavior patterns. The impact of previous behaviors on future behaviors had previously
not been discussed, but the authors assert that this notion of “old habits” is a potentially

strong barrier that is often overlooked.

 

 

 

 

 

 

 

 

 

 

Internal
Factors
Barriers H Old _, Pro-environmental
behavior behavior
External p attem s
Factors

 

 

 

 

 

 

 

 

 

Figure 4. The Kollmuss & Agyeman model of pro-environmental behavior (Kollmuss and
Agyeman, 2002).

It can therefore be seen that there exists no clear understanding of the
relationships between environmental knowledge, attitudes and behaviors, and that there
are undoubtedly many factors that determine our environmental behavior, some
complementary, some competing. We can develop ever increasingly complicated models
of pro-environmental behavior, but we must question how useful these models are in
promoting more effective environmental education. What is certainly lacking in the
literature is a connection between these theoretical models of environmental learning and

research conducted in the ﬁeld. In this chapter I hope to build a bridge between the two.

108

The test data presented in Chapter 3 provide us with a resource with which to
address many of the questions described above. For example, relationships between
environmental knowledge and environmental attitudes can be examined by looking at
correlations between scores on the scales within individual students. That is, do students
with high levels of environmental knowledge also have positive environmental attitudes?
Further, and more interestingly, a constructivist view of environmental learning can be
examined by correlating, for example, change in attitude score between the pretest and
posttest with the student’s prior knowledge score, thereby asking what proportion of the
variation in attitudinal change can be explained by prior knowledge? Through this
approach, we can begin to analyze quantitatively the assumptions of the information

deﬁcit and other models.

109

METHODS

A sample of the test data collected for the evaluation in Chapter 3 was used to
address the questions in this chapter. This sample was the same as the sample used in
Chapter 2’s instrument analyses, that being the data from two schools: Explorer

Elementary and White Pigeon Central Elementary. That sample was selected because:

a) Both schools had taken part in the “Tropical Treasures” program and, therefore,
had used the same knowledge scale (other schools had taken part in different
programs that used different knowledge items).

b) Teachers and students had carefully labeled and organized their tests, making it
clear in which class each student was, as well as making it possible to match
individual students’ pretests and posttests. With some of the other schools it
would have been almost impossible to match individual student’s pretests and
posttest, which is essential for the following analyses.

c) These two schools gave us an adequate sample size (n = 151 and n = 127), with

sufﬁcient variation within the sample to look at the role of gender and grade.

Students had written their names on their completed tests, so each student’s
pretest was matched with his/her posttest where possible. Some whose tests were
unnamed, illegible, or just didn’t form a matching pair were not included in the sample,
nor were students who had not completed the test correctly (e.g., those who had answered

all items the same, or had alternated responses throughout). This reduced the total sample

110

size from 288 to 228. While we were matching individual students’ pretests and posttests
by name to make the sample analyzed in this chapter, the gender of each student was

recorded.

111

RESULTS

The ﬁrst question that was asked of the matched pretest / posttest sample
concerned the relationships between individual students’ environmental knowledge,
attitudes and behavior. Correlations were made between each pair of scores on each of
the four scales, for both the pretest data and the posttest data. Table 1a below shows the
Pearson correlation coefﬁcients and p-values for the correlations between each pair for
the pretest data, and Table 1b below shows the p-values for the correlations between each
pair for the posttest data (n=228). All the pairs of variables are signiﬁcantly correlated in
the posttest data. In the pretest data, attitude, support, and behavior are all signiﬁcantly
correlated, but the knowledge variable is only moderately correlated with each of these
three variables. Taken together, these two sets of data tell us that a student with a high

knowledge score is likely to have high attitude and behavior scores.

Tables 1a and 1b. Pearson correlation coefﬁcients and p-values for the correlations
between scores on each subscale for individual students, n = 228.

la. Pretest Correlations

Knowl Attitude

Knowl

Attitude 0. 1 07 0.1083

S 0. 1514 0.41
Behavior 0.1 1250

 

1b. Posttest Correlations:

Knowl Attitude
Knowl

Attitude 0.31
S 0.477 0001
Behavior 0.149 0.37 0001

 

112

Examining the Data from a Constructivist Perspective

The constructivist view of learning described above describes learning as being
constructed with respect to prior understandings, attitudes and experiences. As such we
can examine how learning is constructed with respect to environmental knowledge,
attitudes and behaviors by examining the relationships between the pretest scores on each
scale (prior knowledge, attitudes and experiences), and the difference between pretest
and posttest scores (learning). For example, by looking at the relationship between prior
knowledge and attitude change, we are asking the question “is knowledge of
environmental issues a requirement for environmental attitude change, and so is
environmental knowledge an important determinant of environmental attitudes?” Table 2
shows the p-values for the Pearson correlation coefﬁcients between each pair of variables

for the 228 individual students.‘

Table 2. P-values for the correlations between pretest score on each scale, and the
difference between pretest and posttest scores for each scale, for 228 students.

 

Pretest-Posttest Difference

 

 

 

 

 

 

 

Knowledge Attitude Support Behavior
Pretest Knowledge 0070, p=.29 -0.074, p=.26 -0.090, p=.17
Attitude 0.116, p=.08 -0.022, p=.74 0.033, p=.62
Support 0.041, p=.53 -0.086, p=.20 -0.058, p=.39
Behavior -0.013, p=.84 -0.100, p=.13 -0.062, p=.35

 

 

A trend was found in which prior attitude score predicted a change in knowledge
[Knowledge Difference = -.004 + 0.112 (Pre Attitude); p < 0.09]. The plot showing the

slope of this relationship is shown in Figure 5.

113

 

 

 

 

 

 

8 .6.

e 7, O i O

Q .41 o e o

o 1 e i e . 5 e

g, .2. o o o o o c

3 i 6 0M

E 0; --- --O-O--O--O--.--O-f-

g 1 e o e o e o o e

f 42-. o to o o

8 1 I I

% -.4: O

‘L 0 .2 .4 .6 1
Pre Attitude Score

Figure 5. Plot for the signiﬁcant relationship between prior environmental attitude
and change in environmental knowledge. The equation for the line is y = -0.004 +
0.112x, the Pearson Correlation Coefﬁcient is 0.116, n = 228.

114

Effect of Gender

In the pretest data, two signiﬁcant differences were found. First, males had
signiﬁcantly higher incoming environmental knowledge than females [Figure 6a;
F(1,226) = 2.747, p < 0.1]. Second, females had signiﬁcantly higher support for

environmental issues than males [Figure 6b; F(1,226) = 2.941 , p < 0.09].

 

*

 

Knowledge
g,

 

 

 

 

 

Suppon

 

 

 

 

 

Female Male

Figures 6a and 6b. The signiﬁcant effects of gender on the pretest environmental
variables. Boys, had signiﬁcantly higher environmental knowledge than girls (p =
0.09, n = 228), and girls had signiﬁcantly higher support for environmental issues
than boys (p = 0.88, n = 228). Data expressed as Mean +/- S.E.M. Asterisk indicates
p < 0.1.

115

The effectiveness of the program differed between boys and girls for knowledge
and total score variables. Girls gained signiﬁcantly more knowledge than boys [Figure
7a; F(1,226) = 5.798, p < 0.02], and also increased their score on the entire set of items
(total score) signiﬁcantly more than boys [Figure 70; F( 1,226) = 3.958, p < 0.05]. There
was also a trend toward girls increasing their environmental behavior more than boys

[Figure 7b; F(1,226) = 2.420, p = 0.12].

116

 

 

Knowledge

.04“

 

 

 

 

 

 

 

 

 

 

Behavior

 

 

 

 

 

Total

 

 

 

 

 

Female Male

Figures 7a, 7b and 7c. Effect of gender on change in the environmental variables
between the pretest and posttest. Girls increased their environmental knowledge
signiﬁcantly more than boys (p = 0.02, n = 228), and also increased their overall score
on all 40 items significantly more than boys (p = 0.04, n = 228). There was also a
trend (p = 0.12) toward girls having a greater change in behavior than boys.

117

DISCUSSION

To summarize the major ﬁndings of this chapter:

1. Signiﬁcant positive correlations exist between individual students’ environmental
knowledge, attitudes, support and behavior.

2. There is a signiﬁcant positive correlation between individual students’ incoming /
prior environmental attitudes and their change in environmental knowledge
between the pretest and posttest.

3. No other prior conditions signiﬁcantly predicted change in any of the
environmental variables.

4. Boys had signiﬁcantly higher incoming environmental knowledge than girls, but
girls had signiﬁcantly higher incoming support for environmental issues.

5. Girls gained signiﬁcantly more environmental knowledge than boys as a result of
the program, and also increased their score on the entire set of items signiﬁcantly

more than boys.

The ﬁrst ﬁnding, that students’ scores on each scale are positively correlated with
their scores on each of the other scales, is an important development in the study of
environmental education. As we saw in the introduction, such correlations between
individuals’ environmental knowledge and attitudes, and between attitudes and behavior,

are hard to ﬁnd, which has led researchers to question the connections between them.

118

Findings two and three are perhaps the most interesting in this study. From our
constructivist perspective, we asked the question, “If learning takes place through
building upon the learner’s prior experiences, knowledge, skills, and beliefs, then
which of these prior conditions are critical for learning to take place in the field of
environmental education?” Early models of environmental education, such as the
information deﬁcit model (Figure 1), would have us believe that environmental
knowledge is critical for formation of positive attitudes toward the environment (we care
about what we know about) and that positive attitudes toward the environment are critical
for us to behave pro-environmentally (we behave in accordance with our attitudes). We
can, of course, logically deduce from this model that our behaviors would also be
determined by our knowledge — as the name of the model suggests; if only people had
information, they would behave responsibly. Subsequent models re-arranged the
placement of these variables, usually by placing them in the same column of the model,
indicating the uncertainty of which needs to come ﬁrst (e.g., Figures 2 and 4), and little

empirical evidence has helped us clarify how such learning takes place.

The results shown in Table 2 and Figure 5 show that, with respect to the
constructivist view of learning, student’s environmental learning in terms of change
in knowledge takes place by building upon, and with reference to, their prior
environmental attitudes. That is, students with positive attitudes toward the
environment are predisposed to engage in learning about the environment. This
ﬁnding, of course, contradicts the predictions of the information deﬁcit model, which has

learning proceeding in the opposite direction, as shown by the grey arrow in Figure 8.

119

 

< ’ [Evidence from this study

Environmental Environmental . I Pro-environmental
Knowledge Attitudes Behavior

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 8. Information deﬁcit model (after Burgess et al., 1998), with the grey arrow
showing the direction of teaming as indicated by the present study.

While this ﬁnding may be signiﬁcant in the ﬁeld of environmental education,
other disciplines have long known about this relationship between attitudes and learning.
For example, research in advertising has found that when information has higher personal
relevance and we hold positive attitudes toward the topic, we pay more attention to it, and
the easier it is to retain. (e.g., Bumkrant and Sawyer, 1993, Greenwald and Leavitt,
1984). Are such conclusions drawn from the ﬁeld of advertising relevant to learning in
informal settings? Absolutely, says Robert Webb (2000): “Advertising is amazingly
similar to museum exhibit design in that in both cases there is a very brief moment to
catch attention and deliver a message that one hopes will have ﬁxture impact. ” Similarly,
in teaching, teachers are trained and encouraged to make the material they are presenting
appropriate and signiﬁcant to their students in order to take advantage of the power of

personal relevance.

There are a number of implications for environmental education of our discovery
of the importance of attitudes in driving environmental learning. A general psychological
model of attitudes includes three components, as shown in Figure 9:

1. Affect - our feeling toward an object.

120

2. Cognition — our thoughts, beliefs, and expectations about an object.

3. Behavior — the tendency to respond in a particular way to an object.

 

Attitudes

 
    
  

Cognition

  

 

 

 

é

Figure 9. A simple cognitive psychology model of attitudes.

Webb (2000) describes how the affective component is principally controlled by

personal relevance, in that we feel the most emotion toward things to which we are
personally connected. The zoo industry oﬁen presents the statistic that more people
attend zoos and aquaria each year than attend all professional sports games combined, but
while this is undoubtedly a notable claim, it probably would be safe to assume that people
know more about the rules, history and current issues in sports than they do about
biodiversity, ecology and environmental issues. In considering why this is the case, we
can apply our results of this study to presume that people hold positive attitudes toward
their sports teams and ﬁnd them personally relevant, which makes them receptive to

learning and absorbing information about them. Taking Webb’s assertion that personal

121

relevance is the driving factor behind positive attitudes, this underscores the need for
zoos and aquariums to make their educational efforts personally relevant to their visitors
and students. This unfortunately gets us back to the problem of the great variation that
exists in visitors to informal learning settings, because what is important to one person is
not necessarily important to the next. Nevertheless, this highlights the need to describe
and characterize the visitor and student population to at least give us an idea of from

where we are starting.

In addition to trying to understand and build upon students’ and visitors’ attitudes
toward nature, environmental education can, of course, seek to foster positive attitudes
and personal relevance. As described in the introduction, the importance of direct
experiences in this endeavor cannot be understated, in that exposing students to natural
settings and engaging them ﬁrst hand in environmental issues is undoubtedly more
powerﬁrl in nurturing affect than formal classroom education. Chawla (1998) conducted
interviews of professional environmentalists both in Norway and the United States, and
found that the most frequently mentioned formative experiences that led to their interest
and careers were 1) childhood experiences in nature, and 2) experiences with
environmental destruction. Education came in well down the list. The challenge for zoos,
therefore, becomes to use their synthetic versions of the natural world to immerse

students in real-life formative environmental experiences.

Finally, does our ﬁnding on the importance of attitudes in motivating

environmental learning tell us anything about how we can achieve the ultimate goal of

122

environmental education: pro-environmental behavior? If we couple this ﬁnding with the
results of the correlations of the environmental variables within individuals (Tables 1a
and 1b), we get some hope from knowing that individuals with high environmental
knowledge and attitudes also have high levels of pro-environmental behavior. Although
we haven’t shown a causal relationship here, we can be encouraged by knowing that
students who have signiﬁcantly improved their environmental attitudes and knowledge

are likely to behave pro-environmentally.

The results on the effects of gender on test scores showed that a) boys had higher
incoming environmental knowledge than girls, b) girls had higher incoming support for
environmental issues than boys, and c) girls signiﬁcantly increased their environmental
knowledge and score on the total set of items between the pretest and posttest, whereas
boys had no such signiﬁcant increases. These results correspond with the ﬁndings of
Mark Rickinson’s meta-analysis (Rickinson, 2001), as described in the introduction.
Rickinson presents apparently conﬂicting results on the relationship between gender and
environmental knowledge. He describes a study by Gambro and Switzky (1999) that
found that male students had signiﬁcantly higher environmental knowledge than female
students, even when controlled for the number of science classes taken. Rickinson also
describes a study by Connell et al., (1998) which found that female students had a greater
conceptual understanding of environmental issues than male students. These two ﬁndings
appear contradictory, but as we discussed in Chapter 3, there are important differences
between knowledge of facts (as in the Gambro and Switzky study) and conceptual

understanding (as in Connell et al., study). Because our knowledge measures dealt with

123

the former construct, our result of male students having higher incoming knowledge is in

line with Rickinson’s ﬁndings.

Rickinson also describes studies that are in agreement with our ﬁnding that girls
had higher incoming (pretest) levels of support for environmental issues than boys. Four
studies are described (Connell et al., 1999; Clarke, 1996; Hampel et al., 1996; and Chan,
1996), which all ﬁnd that girls are more likely than boys to be willing to act in a way that

would beneﬁt the environment.

The ﬁnal ﬁnding, that girls increased their environmental knowledge between the
pretest and the posttest more than boys, neatly makes the connection between the gender
data and the ﬁndings indicating importance of positive environmental attitudes in driving
learning. If girls care more about environmental issues than boys (as is shown by their
higher incoming support measure), then our model of environmental learning would
predict the observed increase in girls’ environmental knowledge between the pretest and

the posttest.

124

CHAPTER 5

CONCLUSIONS,
FUTURE DIRECTIONS,
8:
LIMITATIONS

125

CONCLUSIONS

With reference to the objectives stated in the Introduction Chapter, the major

ﬁndings from this study are summarized as follows:

Development and Analysis of a Test Instrument
Objective 1

T 0 develop a test instrument designed to measure the impact on Binder Park Zoo's
education programs on several components of environmental literacy — knowledge of
nature, attitudes towards the environment, support for conservation, and environmentally
responsible behavior.
0 Through the characterization of the 200’s educational objectives, use of
numerous model-driven approaches to test development, and pilot testing,
a test instrument was developed to measure various aspects of the
effectiveness of Binder Park Zoo’s education programs.

Objective 2

T o analyze the functioning of the test instrument using Item Response Theory and the

Rasch measurement model, and to interpret those analyses with respect to a modern
validity framework.

0 With the removal of two items with unacceptably high difﬁculty, the test

represents a unidimensional and reliable measure of the educational

impact of the environmental education program, and the analyses provide

numerous sources of validity evidence.

126

Evaluation of Binder Park Zoo’s Education Programs

Objective 3
To measure the extent to which Binder Park Zoo ’s education programs were meeting
their objectives through application of the test instrument in a pre/post design.
0 Binder Park Zoo’s education programs are making a signiﬁcant impact on
students’ knowledge of environmental issues and science.
0 All four grades (fourth through seventh) signiﬁcantly increased their scores on the
knowledge variable.
0 Seventh grade students had signiﬁcantly lower scores on the instrument than all
other grades.
0 Boys had signiﬁcantly higher incoming environmental knowledge than girls, but
girls had signiﬁcantly higher incoming support for environmental issues.
0 Girls gained signiﬁcantly more environmental knowledge than boys as a result of
the program, and also increased their score on the entire set of items signiﬁcantly
more than boys.

Objective 4

To compare the eﬂectiveness of programs that were presented at the zoo, with programs
that were presented at schools.
0 The Zoomobile and Schools on Safari programs both signiﬁcantly impacted
student knowledge of program content.
0 The Zoomobile programs stimulated an affective response, and signiﬁcantly

improved students’ attitudes towards the environment.

127

Objective 5

To evaluate how teachers use Binder Park Zoo ’s education programs, how they integrate
the program in their school-based curriculum, and how Binder Park Zoo can better
develop a collaborative relationship between formal and informal settings.
0 Teachers placed great importance on the educational value of the ﬁeld trip, and
the extent of alignment of the program with their school-based curriculum.
0 Despite this trend, teachers did little to integrate the program into their school-

based curriculum.

Examining Learning through Analysis of the Evaluation Data
Objective 6

To examine the relationships between individuals’ environmental knowledge, attitudes
and behaviors.
0 Signiﬁcant positive correlations exist between individual students’ environmental
knowledge, attitudes, support and behavior. Previous studies have only found
weak connections between these variables.

Objective 7

To examine student learning ﬁom a constructivist perspective by measuring the impact of
students ’ prior knowledge, attitudes and behaviors on learning taking place as a result of

the program.
0 There is a signiﬁcant positive correlation between individual students’ incoming /
prior environmental attitudes and their change in environmental knowledge

between the pretest and posttest. This result is interpreted as suggesting that

128

students with positive attitudes towards the environment are predisposed to
engaging in learning about the environment.
0 No other prior conditions signiﬁcantly predicted change in any of the

environmental variables.

FUTURE DIRECTIONS

The research described in this dissertation leads us a number of avenues for new
research studies, as well as to make a number of recommendations aimed at improving
the effectiveness of environmental education. Some of these directions and

recommendations are described below.

The Relationship between Formal and Informal Settings

The emphasis in the research literature on learning in informal settings has for
years been concerned with the question: “What can be done in schools to prepare students
for, and to improve the effectives, of ﬁeld trips.” That is, researchers examining learning
in zoos, museums and science centers to be focused on using the school system to
maximize the effectiveness of educational interventions that take place outside of the
school. However, in light of the results of this study, this focus appears misguided. If
positive environmental attitudes predispose students to engaging in learning about the
environment, one would look to the richness of experiences in informal settings to
nurture those attitudes, which could then be built upon in formal settings to develop

understanding. That is, considering the amount of time that students spend in formal vs.

129

informal settings, it would seem more appropriate to focus on questions regarding how
informal settings can collaborate to improve school-based instruction, rather than the

other way around.

As such, I believe that more work has to be done to illuminate, develop and
support the relationship between formal and informal settings, as to maximize the

strengths of each institution.

Knowledge vs. Understanding

The cognitive learning variable in this study had the title “knowledge” or
“knowledge of nature”, and referred to a set of facts related to speciﬁc program content
that the student may or not be able recall. Such knowledge goals have traditionally been
the mainstay of American education, where students are presented with a never ending
series of truths, with are set to memory and regurgitated on an exam. In science classes in
particular, students have conventionally been occupied with memorizing a specialized set
of vocabulary (e.g. endoplasmic reticulum, Golgi apparatus), rather than developing an
understanding of the functioning of systems (that cells use a series of complex structures
to build proteins, through instructions encoded in DNA). It is such an understanding that
empowers students to be able to use and apply scientiﬁc knowledge and ways of thinking
in new situations, that is, to become scientiﬁcally literate. As such, education should aim
for active use of knowledge, and it is that active use and application of knowledge that I

am referring to here as understanding.

130

As discussed at length earlier, the goals of environmental education extend
beyond cognitive goals of knowledge and understanding, and reach in affective and
behavioral domains. In examining how each of these variables affected each other, we
found no signiﬁcant relationship between students’ prior knowledge, and their change in
attitudes or behaviors. Instead of merely measuring knowledge, it would be interesting to
develop a set of items to measure students’ understanding of environmental systems and
issues, and examine how students’ environmental understanding is related to, and
inﬂuences, attitudes and behaviors. Foreseeing the environmental consequences of our
actions requires understanding of systems, rather than simple knowledge of facts. As
such, correlating an understanding variable with environmental attitudes and pro-

environmental behavior would perhaps show us a stronger and causal relationship.

Didactic Lectures

The material presented to the students in the programs was essentially in the form
in of a didactic lecture, that is, a formal presentation containing instructive and factual
information. Information was presented to students as a series of pieces of information
that they should know, that knowledge being passed down by someone acting in the role
of a teacher. Even the programs that were presented at the zoo were delivered within a
specially constructed classroom, designed to mimic many aspects of a school
environment. It was therefore not entirely unexpected that the zoo programs failed to
have signiﬁcant impacts on environmental attitudes or behaviors, in that in many ways
they failed to capitalize on the richness of the zoo environment. It is perhaps surprising
that despite having a modern and forward-looking perspective on environmental

education, that Binder Park Zoo presents many of their programs so formally. One of the

131

recommendations of this study is therefore for the zoo to reexamine the format of their
education programs, and consider how they can get students more actively involved in
their education, to start asking and investigating questions, and to have more direct and

meaningful experiences in the natural world.

Novelty

The curious result that the Zoomobile programs (presented at the school) were on
the whole more effective than the Schools on Safari programs (presented at the zoo)
highlights how novelty can be a distraction from learning. The novel content in the
program, coupled with the novel zoo environment, apparently resulted in an unfavorable
learning environment. It will be interesting to examine how best to overcome the negative
effects of novelty, be that through repeated trips to novel environments, preparation prior
to the visit to introduce students to novel concepts, or through embracing novelty and

focusing it to stimulate and engage students, rather than distracting them.

Working with Teachers

The ﬁnding that teachers on the whole did little to integrate the program into the
school-based curriculum, despite them putting a high value on the educational value and
curriculum ﬁt of the ﬁeld trip, suggests that there are barriers that need to overcome in
order to increase the efﬁcacy of the ﬁeld trip experience. The ﬁnding of Anderson &
Zhang (1993) that teachers’ interest in curriculum ﬁt may be more connected with their
need to secure legitimacy for the ﬁeldtrip than any genuine intention to maximize the
educational value of the experience, suggests one avenue to explore in more detail. Since

an isolated ﬁeld trip experience alone is unlikely to have any signiﬁcant long-term impact

132

on student learning, it is certainly evident that methods need to be developed that
encourage teachers to take advantage of pre- and post-visit materials, and to greater

integrate the ﬁeld trip into their school-based curriculum.

LIMITATIONS

Despite this study ﬁnding a number of signiﬁcant and interesting results, we must
pause to consider the legitimacy of these ﬁndings with respect to how the data were
collected, how the data were analyzed, and how those results were interpreted. As with
any research study, the methods employed and the interpretation of the results must be
questioned in order to ascertain the validity of the conclusions drawn by the researcher.
Below I consider a number of issues that may be of concern, and discuss their potential to

reduce the strength of conﬁdence that we have in our conclusions.

Issues of Dimensionality

In Chapter 2, a Principal Component Analysis was conducted to examine the
dimensionality of the instrument to a) determine if the Rasch model could appropriately
be used to analyze other aspects of the instrument’s functioning, and b) to analyze the
internal structure of the instrument and learn about relationships between types of items.
The results of this analysis indicated that the test instrument was unidimensional, that is,
a single factor accounts for most of the variability in item responses, and no signiﬁcant
discrete sets of items (referred to as factors, dimensions, or components) existed. This

result told us that a) the application of the Rasch measurement model was appropriate,

133

and b) that signiﬁcant correlations existed between students’ environmental knowledge,

attitudes and behaviors.

Having reached the conclusion of test unidimensionality, one might question the
legitimacy of comparing student responses to discrete sets of items (e. g. student scores on
the knowledge set of items with the scores on the attitude items) in subsequent chapters.
That is, if the set of 40 items on the instrument has been found to be a measure of a single
underlying construct, can we then deconstruct those 40 items to compare four separate

constructs?

The 40 items on the instrument were written to represent four different aspects of
environmental literacy — knowledge, attitude, support and behavior. Knowing which
layer of a rainforest is referred to as the canopy (a knowledge item), and writing to your
congressman about environmental issues (a behavior item), are undoubtedly measures of
different things. What the unidimensionality result tells us is that they are also measures
of a single thing; that there are substantial relationships between items from each of the
four scales; and that no sets of items sit apart as unrelated constructs. This is not a
surprising ﬁnding, since the discussion on barriers to pro-environmental behavior in
Chapter 4 highlights the long standing awareness and complexity of these relationships.
As such, it is these substantial relationships that are explored in detail in the subsequent

chapters.

134

It may also be the case Principal Component Analysis failed to reveal the true
structure of the instrument. The power of any factor analysis is at least partially a
function of sample size, and our sample of 278 students may not have been sufﬁcient,
especially considering the large number of items on the test. Comrey’s criteria for sample
size in factor analysis (Comrey, 1973) classiﬁes a sample of 100 as poor, 300 as good,
500 as very good, and 1000 as excellent. This problem is compounded by our sample
coming from three different grades, thereby adding another source of variation to the
results. As such, further analysis of the structure of the instrument should be conducted

with a larger sample size from a more homogenous population.

Significance of Low Correlations

The conclusion made in Chapter 4 that students with positive attitudes toward the
environment were predisposed to engage in learning about the environment, was made
based on a p-value of 0.08 on a Pearson correlation coefﬁcient of 0.116. Despite many
research disciplines using 0.05 as their yardstick for determining signiﬁcance, a ﬁgure of
0.08 may be argued to be representative of a signiﬁcant trend for a number of reasons.
First, p-values represent the probability of rejecting a true null hypothesis (a Type I
error), this being something we undoubtedly want to avoid. However, setting our
signiﬁcance level too high (at 5%, or even 1%) increases the probability of Type II
errors, that is, concluding that a relationship is not preset when it really is. Therefore,
when deciding on a signiﬁcance level, one needs to make a context relevant decision as
to which is worse: a Type I or a Type 11 error. In this study, since we are dealing with a
low stakes issue and an exploratory rather than conﬁrmatory piece of research, I believe

that a Type 11 error is a greater threat.

135

Second, the result makes sense. In learning in any context it is intuitive that
students who care about an issue are more likely to be receptive to information
concerning it. As discussed above, the same ﬁnding has been found and applied in
advertising, marketing, and public awareness campaigns for decades. If a borderline level
of signiﬁcance was found for a peculiar or bafﬂing result, we would be more inclined not

to interpret it than with a ﬁnding as logical as ours.

Third, we are dealing in this study with human behavior, where variation is
certainly a greater presence than with studies involving physical properties or chemical
reactions. Furthermore, we are dealing with young children, who add noise to the data set
in ways too numerous to describe (some are discussed above in the Fit Analyses in
Chapter 2). The commonly used ﬁgure of 0.05 is an arbitrary line drawn in the sand, and

is not an appropriate standard for all situations.

Fourth, our hypothesis is directional. That is, we are only interested in increases in
learning as a result of change in prior attitude, not decreases in learning. As such, one
could argue that a one-tailed test is appropriate, thus raising the signiﬁcance of our

correlation coefﬁcient.

The argument could be made that signiﬁcance is always going to be found with
sufﬁciently large sample sizes. However, our sample of 228 students is not particularly
large, and this was the only signiﬁcant result in an analysis involving a total of 12

correlations.

136

A correlation coefﬁcient as low as 0.116 may also be questioned, since such a low
degree of association between two variables may not be meaningful. The interpretation of
any correlation coefﬁcient is based on the slope of the line between two variables with
equal standard deviations. For example, if the correlation between size (variable X) and
speed (variable Y) is 0.5, then a difference in size of one standard deviation will be
associated with a difference in speed of only 0.5 of a standard deviation. Our correlation
of 0.116 corresponds to a change in learning of one twelfth of the change in prior
attitudes. Cohen (1988) asserts that a correlation of 0.5 is large, 0.3 is moderate, and 0.1
is small. Anything below 0.1 may be considered trivial, but correlations of 0.1 and above
may represent a small but signiﬁcant relationship between two variables. We are not
claiming that the only factor inﬂuencing learning in zoos is prior environmental attitudes,
or that prior attitudes can explain all of the variation in learning as a result of the
program. Rather, the contention is that prior attitudes are one signiﬁcant factor among
many that inﬂuence change in knowledge. Prior knowledge, for example, has a
correlation of 0.513 with change in knowledge, and so explains signiﬁcantly more

variation in learning.

It might be argued that variance explained (R2, or the correlation squared) might
be a more suitable scale to represent magnitude of linearity. However, a correlation of
0.116 corresponds to a variance explained of only 1.3%, which does not convey

adequately the magnitude of such a correlation.

137

Agreement Bias

Whenever one tries to measure a construct with social connotations, respondents
may acquiesce to providing responses that they believe are socially acceptable and
desirable. This is particularly the case with Agree / Disagree type items, as well as with
socially critical topics such as environmental protection. Despite the test instructions
explicitly stating that there were no right or wrong answers, the fact that the test was
administered by the children’s’ teacher may have inadvertently inﬂuenced some
children’s perception of the purpose of the test. However, since we are mostly concerned
with change in variables and their correlation with other variables, rather than absolute
values, this issue is of less concern. Further, such undesirable effects are minimized when
the test is not administered immediately after a program, thereby disassociating the test

from the environmental advocacy presented in the program.

138

APPENDIX A

The test instrument

Due to margin constraints, the response options have been omitted. The original

instrument contained a column down the right hand side of the page containing the words

TRUE FALSE — where the students circled their responses.

139

Binder Park Zoo l Michigan State University Education Program Evaluation

First Name Last Name

Grade

1)
2)
3)
4)
5)
6)
7)
8)

9)

 

 

Please reply to the following statements by circling
TRUE if you agree with the statement

FALSE if you disagfere with the statement
I prefer a well cared for lawn to a wild meadow.
I would never donate some of my pocket money to a conservation organization.
I hardly ever switch the light off when I don’t need it anymore.
There are no rainforests in the United States
It makes me sad to think that some animals may become extinct.
When I am older I am going to join a conservation group.
At home, I separate trash so it can be recycled.

Plants and animals in an ecosystem are dependent on each other for survival

I enjoy watching television programs about animals and nature.

10) I would be willing to help raise money for a conservation effort.

1]) I prefer to ask for a ride rather than walk short distances.

12) There are many animals and plants that have not yet been discovered.

13) I am concerned about pollution from industry.

14) In winter, I will turn up the heat before I put on a sweater.

15) At the store, I tell my parents to use paper bags instead of plastic bags.

16) Preserving habitats is not a vital part of protecting endangered species.

17) There should be special nature areas that people are not allowed to enter.

140

18) I would be willing to help protect rare animals, but not plants.
19) I am not a member of a conservation organization.

20) There are very bad winters in the rainforest.

21) Building highways is so important that sometimes forests have to be destroyed.
22) I would be willing to help clean up a stream where people had dumped trash.
23) I don’t make a special effort to buy soda in containers that can be recycled.
24) The canopy of the rainforest is the layer closest to the ground.

25) I sometimes step on beetles or spiders on purpose.

26) I would not be willing to write my congressman about a conservation issue.
27) I have saved water in the last week.

28) The temperature stays around 85 degrees in the rainforest.

29) In order to feed pe0ple, we must clear areas of nature to grow foods like grain.
30)I am going to talk to my parents about conservation issues.

31) I have helped feed or shelter animals that live in my yard.

32) Many medicines are made from rainforest plants.

33) Some animals should not be hunted.

34) On vacation, I would buy plant or animal products as souvenirs.

35) I have told ﬁiends how they can better care for the environment.

36) Endangered species are only found in Africa.

37) Protecting the environment is not my responsibility.

38)I am going to help reduce pollution in my local rivers and streams.

39) I have not used public transportation in the last week.

40) Rainforests are very humid.

141

APPENDIX B
Item Numbers Used in the Analyses

The reader should note that this was not the order of items on the instrument; see the
“Structure of the Instrument” section above for details.

Knowledge items:

1. There are no rainforests in the United States. F

2. Plants and animals in an ecosystem are dependent on each other for survival. T
3. There are many animals and plants that have not yet been discovered. T

4. Preserving habitats is not a vital part of protecting endangered species. F

5. There are very bad winters in the rainforest. F

6. The canopy of the rainforest is the layer closest to the ground. F

7. The temperature stays around 85 degrees in the rainforest. T

8. Many medicines are made from rainforest plants. T

9. Endangered species are only found 1n Aﬁica. F

10. Rainforests are very humid. T

Attitude items:

11. I prefer a well cared for lawn to a wild meadow. F

12. It makes me sad to think that some animals may become extinct. T

13. I enjoy watching television programs about animals and nature. T

14. I am concerned about pollution from industry. T

15. There should be special nature areas that people are not allowed to enter. T

16. Building highways is so important that sometimes forests have to be destroyed. F
17. I sometimes step on beetles or spiders on purpose. F

18. In order to feed people, we must clear areas of nature to grow foods like grain. F
19. Some animals should not be hunted. T

20. Protecting the environment is not my responsibility. F

Support Items:

21. I would never donate some of my pocket money to a conservation organization. F
22. When I am older I am going to join a conservation group. T

23. I would be willing to help raise money for a conservation effort. T

24. In winter, I will turn up the heat before I put on a sweater. F

25. I would be willing to help protect animals, but not plants. F

26. I would be willing to help clean up a stream where people had dumped trash. T
27. I would not be willing to write my congressman about a conservation issue. F

28. I am going to talk to my parents about conservation issues. T

29. On vacation, I would buy plant or animal products as souvenirs. F

30. I am going to help reduce pollution in my local rivers and streams. T

142

Behavior items:

31. I hardly ever switch the light off when I don’t need it anymore. F

32. At home, I separate trash so it can be recycled. T

33. I prefer to ask for a ride rather than walk short distances. F

34. At the store, I tell my parents to use paper bags instead of plastic bags. T

35. I am not a member of a conservation organization. F

36. I don’t make a special effort to buy soda in containers that can be recycled. F
37. I have saved water in the last week. T

38. I have helped feed or shelter animals that live in my yard. T

39. I have told friends how they can better care for the environment. T

40. I have not used public transportation in the last week. F

143

APPENDIX C

Teacher survey forms

Binder Park Zoo / Michigan State University Education Program Evaluation

Pre-visit Teacher form

Name

School

 

Class Grade Level

 

Date of Upcoming Zoo Visit
200 Program You Have Registered For

Date Students Completed First Questionnaire

 

Roughly, what percentage of your students receive assisted meals?

Roughly how long did it take your students to complete the questionnaire?

What do you hope your students will get out of the program?

Are you integrating the program into your curriculum in any way? If so how?

Are you aware of the Michigan Science Objectives the zoo program meets?

Any other program expectations?

144

Binder Park Zoo / Michigan State University Education Program Evaluation

Post-visit Teacher form

Name

 

School

 

Date Students Completed Second Questionnaire

 

Percentage of Students on Assisted Meals

 

Level of Satisfaction with the Program

 

Comments:

Did you integrate the program into your curriculum in any way? If so how?

Were you aware of the Michigan Essential Goals and Objective for Science Education
that the Suitcase for Survival program met?

 

Were these useful to you?

 

If yes, how?

How could Binder Park Zoo design its programs to better meet your education objectives,
tie in with your curriculum, or help prepare your students for the MEAP?

Any other comments?

145

APPENDIX D

Consent Form

Christopher D. Wilson
Dept. of Zoology, 203 Natural Science Building
Michigan State University

Dear Parents/Guardians:

I am a graduate student in the Department of Zoology at Michigan State University under the
supervision of Dr. Richard Snider. As part of my dissertation research, I am conducting a study to evaluate
the effectiveness of Binder Park Zoo’s education programs. We hope to ﬁnd out what students learn during
a visit to the zoo so as that Binder Park Zoo may improve its education programs. Participants in this study
will visit Binder Park Zoo during their scheduled ﬁeld trip.

Before and after the visit, participants in this study will be asked to complete a questionnaire
designed for this research. This will take place during school hours and on school grounds. The teacher will
explain the directions for completing the questionnaire to the participants, and will give them as much time
as need to answer the questions. Participants do no have to answer any questions that they do not wish to.
The questionnaire will take approximately half an hour to complete. One month after the zoo ﬁeld trip,
participants will be asked to take a similar test again, using the same procedures.

During the visit, I will observe the program I will record information regarding questions that
students ask, how receptive students were to the program, and any distractions that may occur. Students
will be unobtrusively observed so that their trip is not disturbed.

I am asking you to permit your child to participate in this study. It is anticipated that there will be
no risks in taking part in this study. There are also no direct beneﬁts or compensation for your child. This is
a voluntary study, so you may withdraw you child, or s/he may withdraw her/himself, from the study
without any penalty or prejudice to you or to her/him. It is preferable, however, that s/he takes part in the
entire study. Participation, or nonparticipation, in this study will not affect her/his grade in any classes.

No student names will be used in reporting the study ﬁndings. The list with numerical codes and
names will be destroyed at the end of the study. I will be the only one to see individual questionnaires.
Group scores and averages will be shared with my committee.

If you have any concerns or questions at any time, please feel free to contact myself at 517 367
0706, or my supervisor, Dr. Snider, at 517 355 8473. Questions of concerns about research participants
rights can be directed to University Committee on Research Involving Human Subjects (UCRIHS), 202
Olds Hall, Michigan State University, East Lansing, MI 48824-1046. Your help with my research is
greatly appreciated. If you agree to allow your child to participate in this research study, please ﬁll out the
bottom of this form & return it to your child's teacher.

Sincerely,

Christopher D. Wilson

I agree to allow my child, to participate in the Christopher D. Wilson’s
study at Binder Park Zoo.

 

Signature: Date

 

146

REFERENCES

Ausubel, David P. (1968). Educational Psychology, A Cognitive View. New York:
Holt, Rinehart and Winston, Inc.

American 200 8 Aquarium Association, AZA (2002). Guide to Accreditation of
Zoological Parks and Aquariums (and Accreditation Standards)
http://www.aza.org/Accreditation/Documents/AccredGuide.pdf

American Zoo & Aquarium Association, AZA (2004). www.aza.org

Bailing, J. D. & FaIk, J. H. (1982). A perspective on ﬁeld trips: Environmental
effects on Ieaming. Curator, 23(4), 229-240.

Bailing, J. D., FaIk, J. H. Aronson, R., White, J., Herman, J., & O’Connell, R. A.
(1983). Improving the quality of a single-visit ﬁeld trip to the National
Zoological Park. Roundtable Reports, 7(4), 19-20.

Berkovitz, A. (1988). The 200 as Noah’s ark, In Druger. M. (Ed.), Science for the
fun of it: A guide to informal science education (pp. 70-77). (ERIC
Document Reproduction Service No. ED 303 318)

Bixler, B. C. (1995). A survey of the present status and potential future of
conservation education programs existing at zoos and aquariums
throughout the United States and Canada. Unpublished Master’s Thesis,
University of Georgia, Athens, GA.

Bogner, F. X., & Wilhelm, M. G. (1996). Environmental perspectives of pupils: the
development of an attitude and behavior scale. The Environmentalist, 16,
95-110.

Bogner, F. X. (1998) The inﬂuence of short-term outdoor ecology education on
long-term variables of environmental perspective. The Journal of
Environmental Education, 29 (4),17-29.

147

Braveman, M. T. & Yates, M. E. (1989). Enhancing the education effectiveness
of zoos. (ERIC document Reproduction Service No. ED 306 132)

Bryant, 0., 8. Hungerford, H. (1977). An analysis of strategies for teaching
environmental concepts and values clariﬁcation in kindergarten. Journal of
Environmental Behavior, 10, 17-36.

Burgess, J., Harrison, C. & Filius, P. (1998). Environmental communication and
the cultural politics of environmental citizenship. Environment and
Planning A, 30, pp 1445-1460.

Bumkrant, R. E. & A. G. Sawyer (1983). Effects of Involvement and Message
Content on Information Processing Intensity, in Richard J. Harris, ed.,
Information Processing Research in Advertising, Hillsdale, NJ: Lawrence
Erlbaum.

Campbell, D. T., and Stanley, JG, (1963) ”Experimental and Quasi-Experimental
Designs for Research on Teaching." In N. L. Gage (ed.), Handbook of
Research on Teaching. Chicago: Rand McNally.

Carlin, K. A. (1999). The impact of curiosity on Ieaming during a school ﬁeld trip
to the zoo. Unpublished Ph.D. Dissertation, University of Florida, FA

Carr, B. (2001) AZA Visitor Research Preliminary Proposal. Unpublished
research proposal, American Zoo and Aquarium Association.

Chan, K. K. W. (1996). Environmental attitudes and behavior of secondary
school students in Hong Kong, The Environmentalist, 16(4), pp. 297-306.

Chawla, L. (1988). Children's concern for the natural environment. Children’s
Environment Quarterly, 5, 13-20.

Chawla, L. (1998). Signiﬁcant life experiences revisited: a review of research on
sources of pro-environmental sensitivity. Journal of Environmental
Education, 29(3), pp. 11-21.

148

Chawla, L. (1999). Life paths into effective environmental action. Journal of
Environmental Education, 31(1), pp15-26.

Churchman, D. (1985). The educational impact of zoos and museums: A review
of the literature. (ERIC Reproduction Service No. ED 259 912)

Clarke, 8. (1996). Environmental attitudes and knowledge of year 11 students in
a Queensland high school. Australian Journal of Environmental Education,
12, Pp. 19-26.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
New Jersey: Lawrence Erlbaum.

Connell, S., Fien, J., Sykes, H. & Yencken, D. (1998). Young people and the
environment in Australia: beliefs, knowledge, commitment and educational

implications. Australian Journal of Environmental Education, 14, pp. 39-
48.

Connell, S., Fien, J., Lee, J., Sykes, H. & Yencken, D. (1999). “If it doesn’t
directly affect you, you don’t think about it”: a qualitative study of young
people’s environmental attitudes in two Australian cities. Environmental
Education Research, 51(1), pp. 95-113.

Cooper, J. (1995) Dialogue tracking: adding a new dimension to visitor tracking.
Paper presented at Visitor Studies Conference, St Paul, MN.

Crompton, J. L., & Sellar, C. (1981). Do outdoor education experiences
contribute to positive development in the affective domain? Journal of
Environmental Education, 12(4), 21 -29.

Cronbach, L. J. & Meehl P. E. (1955) Construct validity in psychological tests.
Psychological Bulletin, 52, 281 -302.

DeVeIIis, R. F. (2003). Scale Development: Theory and Application. Sage
Publications.

149

DeWhite, T. G. & Jacobson S. K., (1994) Evaluating Conservation Education

Programs at a South American Zoo. Joumal of Environmental Education,
25(4). 18-22.

Diekmann, A. 8 Preisendoerfer, P. (1992) Persoenliches Umweltverhalten: Die
Diskrepanz zwischen Anspruch und Wirklichkeit. Koelner Zeitschrift fuer
Soziologie und Sozialpsychologie, 44, pp. 226-251

Embretson, S.E. & Reise, S. P. (2000). Item Response Theory for Psychologists.
Lawrence Erlbaum: Mahwah, NJ.

Falk, J. H. and Dierking, L. (1992) The Museum Experience (Washington:
Whalesback Books).

FaIk, J. H. & Dierking, L. D. (1997). School ﬁeld trips: Assessing their long-term
impact. Curator, 40(3), 211-218.

Falk, J. H., Moussouri, T. & Coulson, D. (1998). The effect of visitor's agendas on
museum Ieaming. Curator, 41(2), 107-120.

Falk, J. H. & Adelman, L. M. (2003). Investigating the impact of prior knowledge
and interest on aquarium visitor Ieaming. Joumal of Research in Science
Teaching. 40(2), 163-176.

Gambro, J. 8., & Switzky, H. N. (1999). Variables associated with American high
school students’ knowledge of environmental issues related to energy and
pollution. Journal of Environmental Education, 30(2), pp. 15-22.

Garner, R. (1990). When children and adults do not use Ieaming strategies:
Toward a theory of settings. Review of Education Research, 60(4), 517-
529

Giedd, J. N., Blumenthal J., Jeffries N.O., et al. Brain development during
childhood and adolescence: a longitudinal MRI study. Nature
Neuroscience, 1999; 2(10), 861 -3.

150

Gigliotti L.M. (1992) Environmental attitudes: 20 years of change? Journal of
Environmental Education, 24(1), pp. 15—26.

Gigliotti, L.M. (1994) Environmental issues: Cornell students’ willingness to take
action, Journal of Environmental Education, 25(1), pp. 34—42.

Gray, D. B., Borden, R. J., 8 Weigel, R. H. (1985). Ecological beliefs and
behaviors: Assessment and change. Wetport, CN: Greenwood Press.

Greenwald, A. G. 8 Leavitt, C. (1984). Audience Involvement in Advertising, Four
Levels. Journal of Consumer Research, 9, 132-140.

Hampel, B., Holdsworth, R., 8 Boldero, J. (1996). The impact of parental work
experience and education on environmental knowledge, concern and
behavior among adolescents. Environmental Education Research, 2(3),

pp. 287-300.

Hediger, H. (1959) Man and animal in the zoo. (translated by Gwynne Vevers
and Winwood Reade) New York: Delacorte Press.

Hines, J. M., Hungerford, H. R. 8 Tomera, A. N. (1987). Analysis and synthesis
of research on responsible environmental behavior: a meta-analysis.
Journal of Environmental Education, 18, 1-8.

Hungerford, H. R., 8 Peyton, R. B. (1978). Teaching environmental education.
Portland, ME: Weston Walch

Hungerford, H. R., 8 Volk, T. L. (1990). Changing learner behavior through
environmental education. The Journal of Environmental Education, 21(3),
8-21.

Jacobson, S. K. (1991) Evaluation Model for Developing, Implementing, and
Assessing Conservation Education Programs: Examples from Belize and
Costa Rica. Environmental Management, 15(2), 143-150.

James, W. (1890). Principles of Psychology. New York: Holt.

151

Jensen, BB. (2002) Knowledge, Action and Pro-environmental Behavior.
Environmental Education Research, 8(3), pp. 325-335

Kempton, W., Boster, J.S. 8 Hartley, J.A. (1995) Environmental Values in
American Culture (Cambridge, MA, MIT Press).

Kollmus, A. 8 Agyeman, A (2002) Mind the Gap: why do people act
environmentally and what are the barriers to pro-environmental behavior?
Environmental Education Research, 8(3), pp. 239-261

Koran Jr., J. J., 8 Baker, S. D. (1979). Evaluating the effectiveness of ﬁeld
experiences. In M. B. Rowe (Ed.), What research says to the science
teacher, #2. Washington, DC: The National Science Teachers
Association.

Langbein, L. I. (1980) Discovering whether programs work: a guide to statistical
methods for program evaluation. Scott, Foreman and Company, Illinois,
USA.

La Trobe, H. 8 Acott, T.G. (2000). A modiﬁed NEP/DSP environmental attitude
scale. Journal of Environmental Education, 32 (1 ), 12 — 20.

Leeming, F. C., Dwyer, W. 0., Porter, B. E., Cobern, M. K. (1993). Outcome
research in Environmental Education: A Critical Review. Journal of
Environmental Education, 24(4), 8-21.

Lewis, G. E. (1981). A review of classroom methodologies for environmental
education. Journal of Environmental Education, 13, 12-15.

Linacre, J. M. (2001). WINSTEPS Computer Program Version 3.21. Chicago:
MESA Press.

Lisowski, M., 8 Disinger, J. F. (1991). The Effect of Field-Based Instruction on
Student Understandings of Ecological Concepts. Journal of Environmental
Education, 23, 19-23

152

Lucas A.M., McManus, P., 8 Thomas, G. (1986). Investigating Ieaming from
informal sources: Listening to conversations and observing play in science
museums. European Journal of Science Education, 8, 341—352.

Lyons, E., 8 Breakwell, G. M. (1994). Factors predicting environmental concern
and indifference in 13-16 year-old. Environment and Behavior, 26(2) 223-
238.

Maloney, M. 8 Ward, M. (1973). Ecology: Let’s hear from the people. American
Psychologist, 28, 583-586.

Maloney, M., Ward, M. 8 Braucht, N. (1975). A revised scale for the
measurement of ecological attitudes and knowledge. American
Psychologist, 30, 787-790.

Maitney, RT. (2002) Mind in the Gap: summary of research exploring “inner”
inﬂuences on pro-sustainability Ieaming and behavior. Environmental
Education Research, 8(3), pp. 299-307.

Marshedoyle, E., Bowman, J. L., 8 Mullins, G. W. (1982). Evaluating
programmatic use of a community resources: The zoo. Journal of
Environmental Education. 13(4), 19-26

Messick, S. (1989). Validity. In R.L. Linn (Ed), Educational measurement (3rd
ed., pp. 13-103). New York: Macmillan.

Michigan State Board of Education, (1991). Michigan Essential Goals and
Objectives for Science Education.

Miller, J. (1985). The development of pre-adult attitudes toward environmental
conservation. School Science and Mathematics, 75, 729-737.

National Research Council, (2000). How People Leam: Brain, Mind, Experience,
and School. National Academy Press, Washington DC.

153

Newhouse, N. (1991 ) Implications of attitude and behavior research for
environmental conservation, Journal of Environmental Education, 22(1),
pp. 26—32.

Rajecki, D.W. (1982) Attitudes: themes and advances (Sunderland, MA,
Sinauer).

Rasch, G. (1960) Probabilistic Models for Some Intelligence and Attainment
Tests, University of Chicago Press, Chicago.

Remillard, J. T. 8 Bryans, M. B. (in press). Teachers’ orientations toward
mathematics curriculum materials: Implications for curricular change. In
Journal of Research in Mathematics Education.

Rickinson, M (2001) Learners and Learning in Environmental Education: a critical
review of the evidence. Environmental Education Research, Vol 7, No 3,
2001.

Schwartz, SH. (1977) Normative inﬂuences on altruism, in: L. BERKOWITZ (Ed)
Advances in Experimental Social Psychology, Vol. 10 (New York,
Academic Press).

Stevens, J. (1996). Applied multivariate statistics for the social sciences (3rd ed.),
Mahwah, NJ: Erlbaum.

Tunnicliffe, S. D. (1997) School visits to zoos and museums: a missed
educational opportunity. lntemational Journal of Science Education, 19(9),
1039-1056.

Von Weizaecker, EU. 8 Jesinghaus, J. (1992) Ecological Tax Reform (New
Jersey, Zed Books).

Webb, R. C. (2001). The nature, role, and measurement of affect. Journal of
Interpretation Research, 5(2), 15-30.

154

Weiss, C. H. (1997) How can theory-based evaluation make greater headway?
Evaluation Review. 21(4), 501-524

Weiss, C. H. (1998) Evaluation. Methods for studying programs and policies. 2nd
Ed. Prentice Hall, New Jersey, USA.

Wholey. J. 8. (Ed.). (1987) Organizational excellence. Stimulating quality and
communicating value. Lexington, MA. Lexington Books.

Wright, 8. D. 8 Masters G. N. (1982). Rating Scale Analysis. Rasch
Measurement. MESA Press. Chicago.

Wright, B. (1999) Fundamental measurement for psychology. In: S.E. Embretson
8 S.L. Hershberger (Eds) The new rules of measurement: What every
psychologist and educator should know, pp. 65-104. (Mahwah, NJ:
Lawrence Erlbaum Associates).

155

IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

3 1293 02736 7766