PPS-:9
(In {155.19.3i-z Ida
$sz 5.3
. . .3»:
:2.
fen”
can a?
.3 ...

r F} u
L») ﬂail“... : 2
J Enliburlr; Q
.5 .
JR .
c .4
i. i I 5:12.“.
Ix‘..:.2:
..

 

 

v. z 5....
, u

3.5,.me Vursuﬁi

1.... . 33.3mm“

. g: m s

:.

 

‘ .3! .3!

.3“.
n. J.

 

 

 

 

h-x

 

{Wilfourlea '7 .5...
. . . (

 

PROCESS—ORIENTED ASSESSMENT TOOLS FOR STUDYING SECOND
GRADERS’ INFORMATIONAL COMPREHENSION

By

Katherine R. Hilden

A DISSERTATION
Submitted to
Michigan State University
in partial fulﬁllment of the requirements
for the degree of
DOCTOR OF PHILOSOPHY

Counseling, Educational Psychology, and Special Education

2008

ABSTRACT

PROCESS-ORIENTED ASSESSMENT TOOLS FOR STUDYING SECOND
GRADERS’ INFORMATIONAL COMPREHENSION

By
Katherine R. Hilden

This dissertation includes two manuscripts that resulted from a single study that
examined two process-oriented assessment tools for informational reading
comprehension. While verbal protocol methodology has contributed greatly to our
understanding of how adult readers comprehend, currently no researchers have utilized
verbal protocols to study how readers in the primary grades comprehend informational
texts. Recently, the Concepts of Comprehension Assessment (COCA), another process-
oriented assessment tool, has been developed to provide insights into how ﬁrst and
second graders comprehend informational texts. Thirty second graders in ﬁve classrooms
in ﬁve elementary schools participated. The study involved a within-subjects design that
ﬁrst involved students thinking aloud as an informational text was read to them and then
completing the COCA. Modified grounded theory analysis was employed to create an
inventory of the comprehension processes that the students reported in the verbal
protocols. The inventory resulted in 23 processes.

The ﬁrst manuscript focuses on comparing the COCA and the verbal protocols
and addresses the following research questions: 1) In what ways are our insights into
informational reading comprehension similar when using the COCA and verbal
protocols; 2) What does each tool uniquely contribute to our understanding of second
graders’ informational reading comprehension; and 3) What do verbal protocols tell us

about second graders’ comprehension processing of informational texts?

To address these questions the analysis focuses on a) a comparison of the
processes measured by the two tools, b) an item-analysis and, c) a comparison of the
verbal protocol composite scores to the COCA total scores. Results of this study reveal
that second graders utilize a range of processes, including some highly sophisticated
comprehension processes, when comprehending informational text. While the study
shows considerable overlap between the COCA and the verbal protocols in the
information they provide, each tool provides unique insights as well.

The second manuscript focused solely on the verbal protocol data and was based
on the inventory of processes. The study identiﬁed different proﬁles of comprehension
among second graders reading informational text. These processes were employed in
meaningfully different amounts and patterns across verbal protocols. Speciﬁcally, data
reveal six distinct reading proﬁles: Active Processors, Active Processors with a
Foundation Process(es), Single Processors, Active and Superﬁcial Processors, Single
Superﬁcial Processors, and Passive Processors. The Active Processor proﬁles are
characterized by ﬂexible utilization of a wide range of strategies and active construction
of meaning. Single Processor proﬁles were dominated by a single process. Superﬁcial
Processor proﬁles made many repeating or irrelevant comments. Finally, Passive
Processors reported a low level of cognitive and affective activity and failed to use a
wide variety of processes. Identifying reading proﬁles may help teachers to group
students and tailor instruction to their comprehension strengths and weaknesses.
Researchers might use results of this study to better understand how informational
reading comprehension develops in primary grade readers by identifying different paths

and levels of sophistication in development.

C0pyright by
KATHERINE R. HILDEN
2008

ACKNOWLEDGEMENTS

When expressing my appreciation for those who contributed to this dissertation, I
am reminded of something that Sir Isaac Newton once wrote in a letter, “If I have seen
further it is by standing on the shoulders of giants.” It is with this sentiment in mind that I
would like to express my gratitude to Dr. Nell Duke, my dissertation director. My
dissertation road was long and winding and it is through her guidance that I successfully
navigated it. In reﬂecting on my doctoral preparation, I realize that she purposefully
applies the principle of “gradual release of responsibility” to her mentoring style.

I also like to thank my committee members, Dr. Mary Lundeberg, Dr. Janine
Certo, and Dr. Mark Reckase for their support. I am conﬁdent that their suggestions and
feedback made my dissertation signiﬁcantly stronger.

I would be remiss if I did not include Dr. Michael Pressley in my
acknowledgements. He was the ﬁrst who identiﬁed my potential in this chosen career. I
only wish he was here to see me achieve my goal. I will forever repeat his mantra, “Focus
like a laser.”

I would like to give a big hug and thank you to my ﬁ'iends and fellow colleagues,
Dr. Annie Moses, Dr. Alison Billman, and Lauren Fingeret. Ladies, where would we be
without one another? Each of you played a role in this dissertation. Know that I am
always grateful. Finally, to my husband, Nathan, words cannot express what your love

and support have meant. You are my rock.

\I

TABLE OF CONTENTS

LIST OF TABLES ................................................................................ viii
LIST OF FIGURES .................................................................................. ix
INTRODUCTION .................................................................................... 1
Overview of Dissertation ................................................................... 1
References .................................................................................... 5

MANUSCRIPT l: MANUSCRIPT ONE: A COMPARISON
BETWEEN VERBAL PROTOCOLS AND THE CONCEPTS

OF COMPREHENSION ASSESSMENT ........................................................... 6
Introduction ................................................................................... 6
What We Know About Adult Reading Comprehension ........................... 6
Developing Reading Comprehension ................................................ 8
Verbal Protocols and Developing Readers .......................................... 9
Concepts of Comprehension Assessment .......................................... 12
Purpose of the Study .................................................................. 15
Methods ..................................................................................... 16
Participants ............................................................................. 16
Data Collection Procedures .......................................................... 17
Materials ................................................................................ 21
Analysis & Procedures ................................................................ 23
Results ........................................................................................ 32
Inventory of Processes for Verbal Protocols ....................................... 32
Item Analysis ........................................................................... 34
Comparing Verbal Protocol Composite Scores to COCA Total Scores. ......._.37
Discussion ................................................................................... 37
Similarities between the COCA and the Verbal Protocols ....................... 38
Dissimilarities between the COCA and the Verbal Protocols ................... 39
What Verbal Protocols Tell Us about Informational Text Reading
Comprehension ........................................................................ 40
Logistics of Administration and Scoring Verbal Protocols and the COCA. . .43
Limitations and Future Directions ................................................... 45
References ................................................................................... 51

MANUSCRIPT 2: PROFILES FOR INFORMATIONAL TEXT

COMPREHENSION IN SECOND GRADE .................................................... 56
Introduction ................................................................................. 56
Verbal Protocol Methodology and Comprehension of Developing
Readers .................................................................................. 57
Reading Proﬁles ........................................................................ 59
Purpose of the Study .................................................................. 60

vi

Methods ...................................................................................... 61

Participants .............................................................................. 61
Data Collection Procedures ........................................................... 62
Materials ................................................................................ 64
Analysis Procedures ................................................................... 67
Results ....................................................................................... 71
Active Processors ...................................................................... 72
Active Processors with a Foundation Process(es) ................................. 73
Single Processors ...................................................................... 75
Active and Superﬁcial Processors ................................................... 76
Single Superﬁcial Processors ........................................................ 77
Passive Processors ..................................................................... 78
Discussion ................................................................................... 79
Instructional Implications ............................................................ 79
Limitations and Future Directions ................................................... 85
References ................................................................................... 91
APPENDIX ......................................................................................... 120
A Script for Think-aloud Sessions ................................................ 120

vii

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

Table 8

Table 9

Table 10

Table l 1

Table 12

Table 13

LIST OF TABLES
Breakdown of Ethnicity Percentages for Participating Schools ............ 95
District Demographics by Percentages and Locale Codes for Schools. . . .96

The Number of Questions per Construct for Salmon and Dragonﬂies
in the COCA ....................................................................... 97

Inventory of Processes from Verbal Protocols ................................ 98

- Rubric for Establishing the Quality of the Verbal Protocols Based on

Students’ Processes ............................................................... 99

Descriptive Statistics for the Four Contributors Measured by the
COCA ............................................................................. 110

Descriptive Statistics for the Processes that Fall under the
Contributors that are Measured by the COCA, but are not Directly
Measured by the COCA ........................................................ 111

Descriptive Statistics for Verbal Protocol Processes that are not
Measured Directly or Indirectly by the COCA .............................. 114

Quantitative Processing Differences between Salmon and
Dragonﬂies ....................................................................... 1 l3

Proportion of Matches, Related Matches, Misses, and Non-Responses
between Verbal Protocols and COCA Items for Salmon ................... 114

Proportion of Matches, Related Matches, Misses, and Non-Responses
between Verbal Protocols and COCA Items for Dragonﬂies .............. 115

Description, Criteria, and Frequency for Verbal Protocol Proﬁles ....... 116
Frequencies Broken Down by Form (Salmon & Dragonﬂies),
Total Number of Processes, Number of Different Processes, and

Maximum Number of Reports for Most Frequently Reported Process
for Reading Proﬁles ............................................................. 117

viii

LIST OF FIGURES

Figure l Percentages for Process Breakdown .......................................... 119

ix

Introduction

Investigating the comprehension processes that young readers utilize has been the
driving focus on my graduate career. I have attacked this issue from three related
directions. First, I have examined the challenges and successes that teachers face as they
begin to implement effective instructional practices that are geared toward improving
their students’ comprehension (Hilden & Pressley, 2007). Second, I have examined how
verbal protocol methodology can inform the ﬁeld about students’ developing
comprehension processes (Hilden, 2006; Hilden & Pressley, 2002). Third, I have been a
part of a research team that has designed and tested two assessments that focus on
informational text comprehension in the primary grades, the Concepts of Comprehension
Assessment (COCA) and the Informational Strategic Cloze Assessment (ISCA) (Billman,
Duke, Hilden, et. a1, 2008; Hilden, Duke, & Billman, et. a1, 2008). My dissertation work
ﬂowed naturally from the intersection of these last two directions. My past verbal
protocol work was done solely with narrative texts. I was curious as to what verbal
protocols could tell us about informational text comprehension of young readers. Also, I
was intrigued with the possibility of studying how the COCA and verbal protocols would
compare with one another since both are process-oriented tools.

Overview of the Dissertation

I wrote this dissertation using an alternative format (Duke & Beck, 1999). The
dissertation consists of an Introduction and two manuscripts that are ready to be
submitted for publication in scholarly journals. In the following, I will brieﬂy describe
the dissertation study as a way to provide a frame of reference for the two manuscripts.

Overview of the Studv

When designing the study, I had two primary purposes in mind. First, I wanted to
compare the COCA and verbal protocols. Second, I wanted to study what verbal
protocols tell us about second graders’ comprehension processing of informational text.
These tools are uniquely situated to ﬁll in a gap in the ﬁeld’s knowledge about how
reading comprehension develops for primary grade students’ reading of informational
text. Because they are both process-oriented tools, they can provide a clearer picture of
the cognitive and affective processing of second graders as they are read informational
texts. As a result, the ﬁrst manuscript, A Comparison between Verbal Protocols and the
Concepts of Comprehension Assessment, addresses the following three research
questions: 1) In what ways are our insights into informational reading comprehension
similar when using the COCA and verbal protocols; 2) What does each tool uniquely
contribute to our understanding of second graders’ informational reading comprehension;
and 3) What do verbal protocols tell us about second graders’ comprehension processing
of informational texts?

As I was analyzing the verbal protocols for the ﬁrst manuscript, I noticed that the
students processed the informational texts in signiﬁcantly different ways. After further
investigation, it appears that there are different proﬁles, or ways of actively processing
text, when second graders encounter informational text. Therefore, the research question
for the second manuscript, Proﬁles for informational text comprehension in second
grade, is “what are different proﬁles of informational reading comprehension in second
grade?” In this way, the second manuscript is an extension of the ﬁnal research question

from the ﬁrst manuscript.

Thirty second graders in ﬁve classrooms in ﬁve elementary schools participated.
The study involved a within-subjects design. In the ﬁrst session, students were trained to
think aloud to a puzzle activity and then thought aloud as an informational text was read
to them. The students then completed the COCA during the second session. Analysis
began with a modiﬁed grounded theory analysis (Strauss & Corbin, 1990) to construct an
inventory of comprehension processes from the verbal protocols. The ﬁnal inventory
consisted of 23 processes.

In the ﬁrst manuscript, I examined which processes overlapped and which
processes were uniquely measured by each tool. I then conducted an item analysis, which
involved analyzing students’ comments during the verbal protocols for pages where there
are COCA items. Finally, I calculated the composite score for the verbal protocols. These
composite scores were based on three measures: the total number of processes reported,
the range in the number of different processes reported, and a quality score. I then
correlated the COCA and composite scores for the verbal protocol scores. Results
showed that second graders apply a wide range of processes when an informational text is
read to them. The study also shows considerable overlap between the COCA and the
verbal protocols when it comes to measuring comprehension strategies, text features, and
processing graphics. However, each tool provides unique insights as well. For example,
the COCA captures vocabulary better, whereas the verbal protocols do a better job of
revealing monitoring and affective processes.

In the second manuscript, a rubric of different reader proﬁles was created once the
verbal protocols had been coded using the inventory discussed above. The rubric

distinguishes between six reading proﬁles: Active Processors, Active Processors with a

Foundation Process(es), Single Processors, Active and Superﬁcial Processors, Single
Superﬁcial Processors, and Passive Processors. The Active Processor proﬁles applied a
wide range of processes, both in type and quantity. The protocols of Single Processors
were dominated by a single process. Active and Superﬁcial Processors and Single
Superﬁcial Processors proﬁles made many repeating or irrelevant comments. Finally,
Passive Processors reported a low level of cognitive and affective activity and failed to
use a wide variety of processes. In addition to providing detailed descriptions of the
proﬁles, I also discuss potential instructional implications for working with these
different types of reading proﬁles.

The study’s conclusions have potential signiﬁcance in several related areas.
Among the most important are the following three areas. First, the results will contribute
to the ﬁeld’s growing understanding of informational reading comprehension in the
primary grades. Second, this study has implications for literacy instruction. Research has
shown teachers typically teach what they assess (e. g., Fredericksen & Collins, 1989).
This is problematic as we do not want primary grade teachers to think that they need not
focus on informational text until the later grades. This study calls much-needed attention
to assessment tools for informational reading comprehension and about variation in
reading proﬁles of informational reading comprehension. Finally, this study provides
valuable information about and future directions for developing verbal protocol
methodology. This methodology that has not typically been used with students in the
early elementary grades, but as is clear from this dissertation, can offer many insights for

this age group.

References

Billman, A. K., Duke, N. K., Hilden, K., Zhang, S., Roberts, K., Halladay, J. L., et. a1.
(2008). Concepts of Comprehension Assessment (COCA). East Lansing: Trustees
of Michigan State University.

Duke, N. K., & Beck, S. W. (1999). Education should consider alternative formats for the
dissertation. Educational Researcher, 28(3), 31-36.

Fredericksen, J. R., & Collins, A. A. (1989). A systems approach to educational testing.
Educational Researcher, 18(9), 27-32.

Hilden, K. (2006, December). Verbal protocols: A window into second graders ’ reading
comprehension of narrative texts. Paper presented at the National Reading
Conference annual meeting. Los Angeles, CA.

Hilden, K., Duke, N. K., Billman, A. K., Zhang, S., Halladay, J ., et al. (2008).
Informational strategic cloze assessment (ISCA). East Lansing: Trustees of
Michigan State University.

Hilden, K. R. & Pressley, M. (2007). Self-regulation through Transactional Strategies
Instruction. Reading and Writing Quarterly, 23, 51-75.

Hilden, K., & Pressley, M. (2002, December). An existence proof that comprehension
strategies instruction can occur in ﬁrst grade. Paper presented at the National
Reading Conference annual meeting. Miami, FL.

Strauss, A. and Corbin, J. (1990). Basics of qualitative research: Grounded theory
procedures and techniques. Sage Publications.

MANUSCRIPT ONE: A COMPARISON BETWEEN VERBAL PROTOCOLS AND
THE CONCEPTS OF COMPREHENSION ASSESSMENT
Many of us in the ﬁeld agree that comprehension can be deﬁned as “the process
of simultaneously extracting and constructing meaning through interaction and
involvement with written language” (p. 11, RAND Reading Study Group, 2002; my
emphasis). This is not to say that reading comprehension is not also an outcome. In fact,
most of the assessment tools that we have today operationalize reading comprehension as
what a student can remember via retelling or question-answering after reading. The end
goal of reading comprehension is certainly important in order to achieve success not only
in school but also in our lives. However, if reading comprehension is not just an outcome,
but rather an active, constructive process, then it makes sense to investigate these internal
mental processes which inﬂuence the outcome. Studying these internal processes is
particularly important if the goal is to improve comprehension instruction. Knowing that
a student struggles with comprehension is not enough. In order to be able to remedy the
comprehension problems through instruction, it is necessary to understand the students’
processing issues that underlie the comprehension difﬁculty. However, studying these
invisible mental processes presents an interesting methodological issue. Most researchers
have gotten past the behaviorists’ belief that because mental processes reside in the black
box of the mind, they are therefore unworthy of study (Friedenberg, 2005). However, the
puzzle remains—how are we to make the invisible visible?
What We Know About Adult Reading Comprehension
In the past couple of decades, much of what we know about reading

comprehension as a process has come from verbal protocols of adult readers. In their

landmark work which synthesized the extant verbal protocol work in reading
comprehension, Pressley and Afﬂerbach (1995) concluded that adult, expert
comprehenders are constructively responsive when they read. This means that expert
comprehenders ﬂexibly apply a repertoire of strategies before, during, and after reading
in response to various text factors (e. g., text difﬁculty, text topic, and the genre of the
text) and the purpose for reading. In this study, I defer to Pressley, Harris, and Marks’
(1992) deﬁnition of strategies as “goal-directed, cognitive operations employed to
facilitate performance” (p. 4). In addition to ﬂexible strategy use, Pressley and Afﬂerbach
found that rich construction of meaning also involves monitoring and evaluative
processes (1995).

Verbal protocols also revealed that as expert readers, we do not passively
internalize the texts and images on the page. Instead, we transact with the text by actively
evaluating, interpreting, reacting and relating to what we read (Rosenblatt, 1978).
Additionally, this transacting with the text is not only a cognitive process but an affective
one as well (Afﬂerbach, 2008; Pressley & Afﬂerbach, 1995; Rosenblatt, 1978). While
this is perhaps easier to imagine with a narrative text, it can also be true for informational
text (Alexander, 1997). Imagine reading about the life cycle of the praying mantis and
ﬁnding oneself disgusted aﬁer reading how the female bites the head off of the male after
mating! This meaning construction in adults is also affected by the purposes for which we
read (RAND Reading Study Group, 2002). For example, when reading an informational
text, we may skim or read deeply and thoroughly depending on whether the text is

answering our questions or not.

The research that compares effective and ineffective readers has also contributed
much to our understanding about the mental processes and strategies that lead to effective
comprehension outcomes (see Duke, Pressley, & Hilden, 2004 for a summary). For
example, we know that struggling readers are not as active when they process text. In
comparisons of novice and expert readers, we ﬁnd that expert readers apply a more
ﬂexible repertoire of strategies than do novices and use the strategies more effectively
(Lundeberg, 1987; Peskin, 1998). Researchers have found that poorer readers make fewer
explanatory inferences that further their understanding of text than their more skilled
counterparts (Laing & Kamhi, 2002; Yuill & Oakhill, 1991). The prior knowledge of the
topic being read not only affects our outcomes of comprehension, it also affects how we
process a text (Anderson & Pearson, 1984). A lack of prior knowledge impacts readers’
abilities to effectively make inferences, make predictions, and use imagery; these skills
are all affected by how much we know about a topic (Duke, Pressley, & Hilden, 2004;
Pressley, et. al, 1992). However, a characteristic of poor readers is that they often fail to
activate the prior knowledge they have or, instead, they connect to a text with prior
knowledge that is tangential at best (Williams, 1993). In summary, we know that when
adults and older students read, their comprehension processes and outcomes are affected
by many individual factors such as readers’ capacities, abilities, prior knowledge and
experience. However, this research has been done primarily with adults with a few
studies of upper elementary grade and high-school students (e. g., Laing & Kamhi, 2002;
Williams, 1993). We actually know very little about the comprehension processes that
young children employ when reading.

Developing Reading Comprehension

Improving students’ reading comprehension is currently a hot topic for
researchers (Cassidy & Cassidy, 2008) and in our schools. It is also one of the ﬁve pillars
of the National Reading Panel Report (2000). While we know a lot about various
predictors and factors that are associated with comprehension as an outcome, not nearly
as much research attention has investigated reading comprehension as a process. A great
deal of research on comprehension and young children has focused on teaching students
strategies and then seeing how this improves children’s comprehension as an outcome.
Indeed, it is clear from the research that teaching students comprehension strategies
improves their understanding and memory of text (for summaries see Pearson & Duke,
2002; National Reading Panel, 2000). Little research has looked at how teaching these
strategies affect the online comprehension process as they read (see Brown, Pressley,
Van Meter, & Schuder, 1996, for an exception). Very few researchers in the past have
asked primary grade students, “What are you thinking about?” when the students are
reading or used other approaches to investigating these online comprehension processes.
Verbal Protocols & Developing Readers

Researchers conduct verbal protocols to gain information about participants’
cognitive processes by having them verbalize their internal states and behaviors while
performing a task (Ericcson & Simon, 1984). Messick (1989) proposed that verbal
protocols, with the insight they provide into mental processes, represent one way to
establish substantive validity (i.e., whether participants actually engage in the theorized
processes measured by an assessment). Typically, verbal protocols involve participants
verbally reporting what they are thinking as they perform a task. It is the researcher’s job

to infer the cognitive processes underlying participants’ behaviors and reported thinking.

Ericcson and Simon (1984) provide a history and standards of excellence for verbal
protocol methodology. Researchers have also provided recommendations for conducting
verbal protocols to study reading comprehension (Afﬂerbach, 2000; Pressley &
Afﬂerbach, 1995; Pressley & Hilden, 2004).

While verbal protocol methodology has greatly informed the ﬁeld about how
adult readers understand text, this methodology has not been used to deepen our
understanding of the reading process of readers in the primary grades. One reason for the
lack of protocol studies with younger students is the assumption that developing readers
are unable to provide useﬁrl verbal protocols. Afﬂerbach and Johnston (1984) reasoned
that because verbal ability is confounded with reading ability, younger and less verbal
participants may produce poor think-alouds. Moreover, verbalizing their thoughts while
reading is thought to overwhelm their cognitive resources, and therefore, this
methodology would not provide an accurate portrayal of their naturally occurring reading
comprehension. Afﬂerbach and Johnston (1984) ﬁrrther stated that young readers may
not provide accurate think-alouds because they may not have sufﬁcient metacognitive
awareness of what they are thinking while reading.

However, despite these trepidations, Afﬂerbach (2000) called for future studies to
investigate the developmental nature of reading comprehension using verbal protocols.
This would presumably include primary grade children. While the reasons discussed
above discourage employing verbal protocol methodology with young students, anecdotal
evidence exists from teachers who claim that ﬁrst and second grade readers can not only
be excellent comprehenders, but can verbalize and discuss the different strategies they

used to make sense of texts (Harvey & Goudvis, 2000; Keene & Zimmermann, 1997;

10

Miller, 2002). For example, Miller writes about her ﬁrst graders who discuss with one
another different types of connections to their background knowledge that they make
while reading (2002). Keene and Zimmermann (1997) even provide a rubric for assessing
students’ think-alouds at the end of their book. Unfortunately, none of Keene and
Zimmermann’s work on think-alouds has been empirically tested to date.

In addition to anecdotal evidence, preliminary research suggests that second
graders can verbalize their thoughts while reading when ﬂuency and decoding demands
are low for students (Brown, Pressley, Van Meter, & Schuder, 1996). The researchers
compared the verbal protocols of second graders who had experienced traditional reading
instruction to transactional strategies instruction (TSI), a form of comprehension
instruction for reading. They triangulated the verbal protocol analysis with analysis of
retellings and standardized assessments. Brown and colleagues concluded that verbal
protocols data converged with the other measures that found that TSI improved reading
comprehension over more traditional instruction. Verbal protocols in this study also
helped answer the more important question of why. This study also provides initial
evidence that second-graders can report their strategy use.

Hilden and Pressley (2002) found a ﬁrst grade teacher whose students were able
to verbalize a rich repertoire of strategies when reading familiar books during a think-
aloud. In fact, her students, with minimal prompting, made as many strategic comments
as the fourth graders in the study.

Hilden (2006) collected verbal protocols from 24 second graders who read at or
above grade-level. These students read aloud and verbally reported their thoughts and

feelings while they read one of ﬁve narrative picture books. Grounded theory analysis

11

(Strauss & Corbin, 1990) resulted in a comprehension processes inventory that consisted
of 7 categories of processes: inferences, connections, and predictions (ICP), questioning,
monitoring, picture strategies, evaluations, synthesizing, and affective comments. Among
other ﬁndings, Hilden discovered that unlike their adult counterparts, only a minority of
these younger readers actively constructed meaning before or after reading. The vast
majority of their comments occurred while reading.

Currently, no researchers have employed verbal protocols to study how primary
grade students comprehend informational texts. This is an important distinction as there
is research evidence that suggests genre impacts the way we process when we read, both
for adults (Kucan & Beck, 1997; Pressley & Afﬂerbach, 1995) and children (e. g., Duke,
Pressley, & Hilden, 2004; Hidi & Hildyard, 1983; Langer, 1996).

Additionally, researchers have not systematically examined how verbal protocols
compare to other comprehension measures (one exception is the previously discussed
work of Brown et. al, 1996), as previously called for by Pressley and Afﬂerbach (1995).
In conclusion, while verbal protocols seem to offer a promising tool for studying reading
comprehension processes in young readers, additional research exploring the utility of
verbal protocols with this population is needed, particularly with informational texts.
Concepts of Comprehension Assessment

Recently, the Concepts of Comprehension Assessment (COCA), a new and
unique assessment tool, has been developed to provide insights into ﬁrst and second
graders’ comprehension processes (Billman, Duke, Hilden, et. al, 2008). This asseSsment
does not use verbal protocol methodology. Rather, the COCA administration involves

asking questions or having students ﬁll in blanks as the book is read to them in order to

12

reveal more about their online construction of meaning. One way that the COCA is
distinctive is that performance does not depend on children’s word recognition ability as
the text is read aloud to students. However, the COCA is not a traditional listening
comprehension task in that students look at the book as it is read to them and respond to
items, many of which cannot be solved by simply listening to the text alone (e. g., reading
the index, or questions regarding graphics). This assessment is also unique in its focus on
informational text within the domain of the life sciences (Billman, Duke, Hilden, et. al,
2008).

The COCA assesses four contributors which are thought to inﬂuence
informational reading comprehension. These four contributors are: text feature knowledge
(e. g., reading and interpreting table of contents, index, & labels), processing graphics,
vocabulary, and comprehension strategies. Within the comprehension strategies
contributor, the authors of the COCA speciﬁcally chose four comprehension strategies to
represent the larger range of strategies: predicting and previewing, integrating prior
knowledge, inferring (as a conscious strategy rather than that which occurs
automatically), and summarizing. Graphics such as diagrams, tables, and maps, have
been identiﬁed as an integral part of informational texts (Duke & Kays, 1998; Purcell-
Gates & Duke, & Martineau, 2007). The processing graphics contributor then refers to
students’ understanding of graphics or illustrations. The text feature contributor measures
students’ abilities to recognize and use navigational text features (e.g., table of contents,
index) as well as other text features common to informational texts (e. g., labels, captions,
pronunciation guides). The vocabulary contributor consists of two related types of

subcontributors. The ﬁrst subcontributor is knowledge of hi gh-utility words such as

13

observe, describe, cycle. Knowledge of these words is obviously important to
informational comprehension across a wide range of topics and texts and varies by
readers (similar to what Beck, McKeown, and Kucan, 2002 call Tier II vocabulary
primarily in reference to narrative text). The other subcontributor of vocabulary assessed
by the COCA is the ability to glean word meaning from text. Examples of this
subcontributor include recognizing when the text provides a deﬁnition of the word and
gleaning word meaning from the glossary. Such vocabulary strategies are important in
informational texts, in which the reader is likely to encounter many unfamiliar words
(Hiebert, 2008). These four contributors were chosen based on either past research
support or because they have been hypothesized to be causally related to informational
reading comprehension. Additionally, the COCA authors chose to focus on these
contributors because they are amenable to teachers’ instruction. Other factors, such as
working memory, are associated with reading comprehension but do not seem as easily
remediated (Pressley, et. al, in press). Still other factors, such as word recognition and
ﬂuency, were not measured by the COCA because they are adequately measured in many
other current reading assessments.

Process-oriented measures, like the COCA, contrast with traditional comprehension
assessments that are typically outcome or product-based, such as recall measures and
answering questions about a text (e. g., the Gates-MacGinitie Test of Reading
Achievement, MacGinitie, MacGinitie, Maria, Dreyer, & Hughes, 2000; the Basic
Reading Inventory, Johns, 1997). These product-based measures occur after one ﬁnishes
reading. Thus, while such measures provide information about how well a reader

comprehended a text (outcome-focused), they do not reveal how or why that particular

14

understanding resulted (process-focused). One exception is the Qualitative Reading
Inventory-4 (Leslie & Caldwell, 2006). This informal reading inventory has an option for
think-alouds that has teachers categorize students’ verbal responses on eight categories.
However, these think-alouds are only available for grades six and higher and evidence of
validity and reliability is not reported.
Purpose of the Study

The dual purposes of this study are to explore 1) what the COCA and verbal protocols
tell us about second graders’ comprehension processes of informational text, and 2) how
these two tools compare. Given a theoretical framework that emphasizes the process-
oriented nature of reading comprehension, verbal protocol methodology seems
particularly well-suited to the study of active meaning-making construction involved in
reading comprehension as the methodology provides a window into how the mind
processes text. The same holds true for the COCA as it was designed for a diagnostic
function that would provide teachers with information about how students are and are not
processing informational text. Both of these tools are in a unique position to ﬁll in a gap
in the ﬁeld’s knowledge about how reading comprehension develops, particularly for
primary grade students’ reading of informational texts. While some comprehension
outcome measures, like retellings, can reveal problems with comprehension strategies,
verbal protocols and the COCA were designed to more clearly capture readers’ strengths
and weaknesses as students construct meaning from texts. In other words, these tools
attempt to address the question: where did their comprehension processes break down
and what were the strategies and processes they used while reading? These questions

have important implications not only for understanding reading development but for how

15

teachers meet the immediate learning needs of their students through scaffolded
instruction.

Speciﬁcally, in this study, I will address three related questions: 1) In what ways
are our insights into informational reading comprehension similar when using the COCA
and verbal protocols; 2) What does each tool uniquely contribute to our understanding of
second graders’ informational reading comprehension; and 3) What do verbal protocols
tell us about second graders’ comprehension processing of informational texts?

Methods

In order to compare the verbal protocols and the COCA tools, I employed a
within-subject design in which students ﬁrst thought aloud and then completed the
COCA. First, I examined what each tool revealed about the comprehension processes of
second graders. This involved looking at what processes are revealed in both the verbal
protocols and the COCA and which are unique to each. Secondly, I compared how well
the verbal protocols and the COCA scores aligned. Also, I did an item—by—item analysis
on the COCA and verbal protocols. This involved looking at the pages in the books
where there are COCA questions to see what students are doing on the verbal protocols.
(For example, when the COCA asked students a processing graphics item, did students
typically make processing graphics comments on the same page during the verbal
protocols, or did they comment about something else, or did they say nothing at all?)
Participants

This study involved 30 second grade students, 11 boys and 19 girls. Teachers in
ﬁve classrooms ranked all of their students on a continuum according to their perceptions

of the students’ informational reading comprehension. Within each classroom, I divided

l6

all of the consenting students into six ascending ability groups, from highest to lowest
perceived ability, and randomly sampled one student from each group. (See Tables 1 and
2 for school demographics.) The rate of parental consent for three of the rooms was 55%.
(I was not able to attain the consent rate for the other two classrooms at the time of
submitting the dissertation). The classes were part of a larger study examining the
validity and reliability of the COCA. The schools for the larger COCA study were
selected to represent a range of school settings in terms of demographics. The
participating classrooms in this study were selected with this same end in mind. The
teachers and administrators were chosen because they had previously indicated their
interest in either participating in research concerning informational text comprehension or
learning more about the COCA more speciﬁcally. Three students were English Language
Learners and none of the students received special education services at the time of the
study.
Data Collection Procedures

The students participated in two individual sessions. In the ﬁrst session, the
students were trained in the verbal protocol procedure and produced a verbal protocol of
one of the COCA books. During the second session, a research assistant administered the
COCA with the other book. I chose not to counterbalance the order of verbal protocols
and the COCA because I did not want the students’ experiences with the COCA to
unduly inﬂuence how students responded during the verbal protocols. There was a
possibility that the COCA questions on the four contributors could inﬂuence the types of
processes that students reported during the verbal protocols. For example, students might

be more likely to report processing graphics processes after being asked multiple

l7

questions that call their attention to graphics in the COCA. However, there seemed to be
less of a chance of the verbal protocols inﬂuencing the COCA since the prompts and cues
were so general (see discussion below for description of prompts).

First Session. The ﬁrst session began when I introduced the think-aloud procedure
by having students think-aloud while solving a 3-D puzzle. At the introduction, students
were asked to talk about whatever they were thinking as they attempted to solve the
puzzle. If students did not independently verbalize, they were cued to say what they were
thinking or feeling approximately once a minute for ﬁve minutes. (See Appendix A for
the complete think-aloud protocol.) The training activity was followed by a brief set of
directions for thinking aloud with the book, which can be found below:

Now before we start, I have some directions. What I am really interested

in is hearing your thoughts and feelings about the book as I read it to you.

So just like the puzzle where you told me what you were thinking and

feeling, I want to know what’s going on inside your mind as you read.

You can stop and tell me what you are thinking whenever you want. That

is okay. Remember, just like the puzzle, there are no right or wrong

answers. Also, do you see these yellow dots? I will stop and ask you to tell

me what you are thinking and feeling when I get to each yellow dot.

Sometimes there may be places where the book is not making you think or

feel anything. If this happens you can shake your head no or tell me that

you are not thinking anything and we will go on. Also, it is okay if you

want to ﬂip through and look at pages. Finally, if you want to hear any of

the pages again just ask and I will reread the page to you. Can you tell me

18

the directions so that I know you understand please? (Clarify any

misconceptions and review directions as needed.)
When listening to students repeat the directions, I checked to see that three main
points were captured: 1) the student can think aloud at any time, 2) the student
should either think aloud after yellow dot or say, “I’m not thinking anything” and,
3) the student can ask to reread.

Aﬁer reading the directions for thinking aloud, I read either Salmon or
Dragonﬂies to the students. Book selection for the verbal protocols was randomly
counterbalanced within and across classrooms, with roughly equal numbers of students
experiencing each form in the ﬁrst session. Students were cued to think aloud on each
page. Each page was marked with a yellow dot in the bottom outside corner (this “yellow
dot” method has been employed in past verbal protocol work, e. g., Olshavsky, 1976-7)
and served as a nonverbal reminder to think-aloud before a student went onto the next
page. If students did not automatically think aloud at the yellow dot, I asked them, “What
are you thinking or feeling?” or directed them to “Think aloud for me.” (This does not
match the COCA assessment, in which students are not asked questions on every page.
However, this cueing decision still allows for a page-by—page comparison between the
COCA and verbal protocols, while allowing for additional data from the pages that do not
have COCA items.) This type of nondirective prompt has been used successfully in past
research and is done in order to cut down on the inﬂuence that the researcher has on the
participants’ verbalizations (e.g., Bereiter & Bird, 1985; Lundeberg, 1987; Myers, Lytle,
Palladino, Devenpeck, & Green; 1990; Olshavsky, 1976-7; Wyatt, Pressley, El-Dinary, &

Stein, 1993).

19

There were several situations in which follow-up prompts were given. First, I
used follow-up prompts in order to clarify what a student was trying to communicate in
order to better understand the underlying processes. For example, when a student
commented that, “I’m like feeling and like thinking about a stream” at the beginning of
the Salmon, I followed up by asking, “Can you say a little more about that?” in order to
clarify what she was thinking. Secondly, I gave follow-up prompts when students
responded nonverbally. For example, when a student smiled I asked, “I see you smiling,
what are you thinking or feeling?” Finally, I also prompted students when I saw their
eyes focusing on a different page than the one I was reading. When a student is
referencing something in an illustration, and I was unclear about where a student was
looking, I asked, “Can you show me where you mean?” or “Can you point to where you
are looking?” These are behaviors that seem to indicate that students are processing the
text or illustrations and I wanted to make sure that I got as complete a picture as possible
of the students’ processing. Some might question whether these follow-up prompts and
yellow dots on every page stimulated thinking where none was occurring at ﬁrst.
However, I took steps to reduce the “talking to talk” phenomenon that I experienced in
previous verbal protocol work (Hilden, 2006) by allowing students to skip thinking aloud
after a page was read. After three no-responses, I reminded the students that, “Remember
that it is really important for you to share what you are thinking and feeling when we get
to these yellow dots.”

In order to guard against fatigue effects, I separated the verbal protocols and the

COCA administration into two separate sessions. The verbal protocols lasted between 9

min 14 s and 21 min 42 s (excluding time taken for directions and training). On average,
the verbal protocols lasted 11 min 46 s for Dragonﬂies and 14 min 26 s for Salmon.

In addition to each page of text, I also asked students what they were thinking and
feeling once I had ﬁnished reading the book. These two cues provide an indication of
how frequently students are processing the text before or after reading, and are important
because processing texts before and after reading is a key characteristic of expert adult
comprehension (Pressley & Afﬂerbach, 1995). In my previous work with verbal
protocols of narrative text (Hilden, 2006), I found that second graders report very little
thinking before or after reading narrative picture-books. I wondered if the same pattern
would hold true for informational text that were read aloud to students.

All of the verbal protocols were audio-taped for later transcription. I also had
black and white copies of the books’ pages for making notes about students’ visual
attention and reading paths (e. g., ﬂipping between pages) during the verbal protocols.
These notes were used to add detail to the audiotaped transcripts (this kind of data has
been used previously in research by Donovan & Smolkin, 2006).

Second Session. The alternative form of the COCA was administered to the
students between two and ﬁve days later. The COCA administration was done by another
researcher. This was done so that I would not be inﬂuenced by students’ COCA results
when working with their verbal protocols. The COCA is administered individually and
takes approximately 10 to 15 minutes to administer depending on the child. (I scored the
COCAs once I had completed my analysis of the verbal protocols.)

Materials

The COCA books, Salmon and Dragonﬂies, were kept in their original form

21

during the ﬁrst session for the verbal protocols in order to maintain the comparison
between the COCA and the verbal protocols. The COCA books were designed to be
parallel in content and in structure. Salmon and Dragonﬂies share the common theme of
animal groups, a topic that is somewhat familiar to most US. students. Both books have
sections on the growth, or life cycle, of the animal. The texts were modeled closely after
existing published trade books in order to maintain authenticity. However, one way that
the COCA books are not like existing published trade books is that the text at the bottom
of each page has been replaced with blank lines. This is done in order to remove the
confounding variable of decoding from the COCA comprehension tasks. Some text still
exists in the book in the form of text features (e. g., labels, headers, Table of Contents,
diagrams). Another way that the COCA books are different than existing trade books is
that they were designed to contain purposeful “mistakes” or places where the text and
graphics have contradictory information. These “mistakes” are one way that the COCA
captures the processing graphics contributor.

When conducting the COCA, the assessor reads the text for each page aloud and
pauses to ask the child 19 questions across 21 (Dragonﬂies) or 25 (Salmon) pages. The
questions were designed to tap the speciﬁc contributors of informational text
comprehension discussed previously. See Table 3 .for a breakdown of the number of
questions per contributor for each COCA book.

Scoring the COCA involves assigning a score of O — 2 for each question using a

scoring guide. Below is an example of a vocabulary item from page 12 of Salmon:

22

Administrator reads: In the ocean the salmon eat shrimp and small ﬁsh.

Animals that only eat other animals are called carnivores. Salmon are

carnivores.

Administrator asks: This page tells you the deﬁnition of a word. What is

that word?

If student does not answer administrator says: Listen closely while I reread

this page.

After rereading, the administrator asks: This page tells you about

carnivores. What does carnivore mean?
For this vocabulary item, students receive a score of 2 if they correctly answer
“carnivores” after the ﬁrst time the question is asked. If students correctly identify
carnivores after the text is written or responds with a correct deﬁnition of carnivores,
such as “animals that only eat other animals”, they receive a score of 1. Scores of 0 are
given when students respond with words other than carnivores or if they fail to produce
an answer after the second prompt. The individual item scores are then tallied to result in
a total number for the COCA. While possible COCA scores range between 0 — 38, the
actual COCA score range for second graders in the present study is 9 - 28 for Dragonﬂies
and 10 — 31 for Salmon. A ﬁnal report on the validity and reliability of the COCA is in
preparation at the time of dissertation submission.
Analysis Procedures

The preliminary phase of data analysis began after I had personally transcribed
the verbal protocols. In this ﬁrst stage, I employed a modiﬁed grounded theory analysis

to construct an inventory of processes that resulted from the verbal protocols (Strauss &

Corbin, 1990). Because the goal of the study was to compare the comprehension
processes measured by the two assessment tools, I started with some pre-formed
categories that correlated with the hypothesized contributors assessed by the COCA
(comprehension strategies, processing graphics, text feature knowledge, and vocabulary).
However, upon sifting through the data from the ground up, several categories emerged
from the data that the COCA was not designed to measure. See Table 4 for the complete
inventory of processes that resulted from the grounded theory analysis.

The transcripts from the protocols were scored during the second stage of analysis
using the ﬁnal reading processes inventory. The basic unit of coding was the turn. Each
turn began when the student began speaking (or had a chance to speak but chose not to).
There were occasions in which students were prompted to think aloud when I noticed
them looking at a different page. A new unit was recorded when this happened. New
units were not coded for prompts that resulted in a one word answer (e. g., “yes/no” or a
clarifying answer), my prompt resulted in a clariﬁcation of a previous unit or if I reread a
page. There were occasions in which my prompting was inadvertently leading or seemed
to take the student off their original train of thinking. When this occurred, I did not count
these turns as new units. (The resulting comments were not coded for processes either.)

The second stage of coding involved coding the individual processes. These codes
reﬂected the strategies and processes inferred from students’ verbalizations. When coding
at the process level, I only coded a strategy once per page. For example, if a student made
a connection to prior knowledge, then I followed up with a prompt, and then the student
made a different connection, I only coded one process: connection to prior knowledge.

Within the same page, if a student used more than one strategy, I coded for multiple

24

processes. For example, a student remarked about dragonﬂies, “That reminds me of how
owls can see good in the dark. And. . .and now I know that they ﬂy very fast”
[Dragonﬂies]; I coded this as a connection to prior knowledge and knowledge
monitoring. Coding occurred at the macrolevel. For example, 1 did not code among the
various types of prior knowledge connections (e. g., connecting to a form of media,
personal experience, or another text) in an effort to maintain the reliability of coding of
the inventory. Also, while some teacher-educators have distinguished between text-to-
text, text-to-self, and text-to-world connections (e. g., Keene & Zimmermann, 2000;
Miller, 2002), these distinctions do not seem to result in qualitatively distinct processes.
One exception to this macro-coding was the monitoring category. For this category, I
noted both the macro and sub category (e. g., monitoring—text monitoring or
monitoring—knowledge monitoring). I was able to reliably distinguish between these
forms of monitoring. These two types of monitoring seemed qualitatively distinct from
one another and focused on different sources of monitoring. Another exception occurred
when coding for intratextuality, which occurred when students made comparisons and
connections with previous pages in the book. Intratextuality falls under the contributor of
processing graphics because these comments were always graphically based. However,
intratextuality was not directly measured by the COCA. Therefore, I wanted to keep
these two categories separate in the analysis. Finally, there were many comments that
reﬂected students’ ineffectual attempts at using the processes. Even though these attempts
were ineffective, I still coded the processes they represented. As explained later, quality

of the processes were analyzed and resulted in a quality score which fed into the

25

composite scores for the verbal protocols. (Additional coding directions can be found in
the coding manual that is available upon request.)

I coded the data dividing the transcripts into two columns, with the verbal
responses in one column, and a place for me to write in the codes in the other column.
When multiple codes were present for a unit, I would underline the part of the verbal
response that went with each code. As part of the iterative process of developing the
inventory of processes, I kept lists of student comments that I found difﬁcult to code for
Salmon and Dragonﬂies. I then discussed these codes with an expert in the ﬁeld of
informational text in order to revise the coding scheme to account for all of the students’
comments. An expert in informational text comprehension development as well as a
colleague in literacy development reviewed the document which led to clarifying some
codes and processes. For example, we discussed the differences between summarizing
and restating extensively. From this conversation, I revised the inventory to distinguish
between the ﬁnal processes of summarizing, restating, and repeating. In order to clarify
the coding process, I created a document that had examples and non-examples of the
processes under question, along with explanations for the way the examples were coded.
These discussions and list of examples and non-examples helped to inform the ﬁnal
version of the coding inventory and was used in the ﬁnal coding of the data.

Revising the inventory of processes occurred until no new categories emerged
from the data, redundancy between categories was eliminated, and I was satisﬁed that the
categories in the inventory adequately depicted the reading comprehension processes

captured by the protocols (Strauss & Corbin, 1990). The ﬁnal inventory was again

26

 

audited by the same expert in informational text comprehension development to ensure
that the processes adequately captured the verbal protocol data.

In order to compare the verbal protocols to the COCA data, 1 divided the
categories into three separate tables. In one table, I placed processes that are the same as
the contributors in the COCA (see previous discussion of the contributors assessed by the
COCA). In a second table, I placed processes that fall under the COCA contributors but
are not among the speciﬁc processes that were directly measured by the COCA (e. g.,
strategies such as questioning, were not directly measured in the COCA but do fall within
the category of comprehension strategies, which is measured by the COCA). I placed
processes that are truly unique to the verbal protocols (i.e., not measured by the COCA)
in a third table. Each table contains a list of processes, descriptions, and examples. An
expert in informational text comprehension audited the inventory for category placement
within the tables. From a grounded theory perspective, the sample size was adequate in
that I reached saturation, or the point where no new categories emerged before I
exhausted coding the collected protocols. Saturation is met when further reviews of the
data fail to result in new categories.

The third stage of coding involved assigning an overall quality score to each of
the protocols. The quality score consists of two factors: accuracy of processes and quality
of processes. The accuracy factor refers to how accurately students are constructing
meaning for informational texts. The quality factor focuses on the overall quality of the
processes reported in the verbal protocol. These factors are in turn comprised of nine
subfactors. See Table 5 for the scoring rubric. For instance, one of the subfactors under

quality is “active and knowledge transforming processes.” Such processes go beyond

27

simply regurgitating what is in the text and reﬂect depth of thinking. Active and
knowledge transforming processes occur when students internalize or elaborate upon the
text. Examples of active and knowledge transforming processes include inferring (e. g.,
about cause-effect relationships), giving elaborative examples, comparing/contrasting
information within the book or with prior knowledge. These examples are similar to
processes that are believed to beneﬁt comprehension that have been coded in previous
verbal protocol work by Cote, Goldman, and Saul, (1998) and Meyers, Lytle, Palladino,
Devenpeck, & Green (1990). Comments that were coded as affective reactions and
activating senses also count as evidence of internalizing the text and are also examples of
“active and knowledge transforming processes.” Alexander (1997) has pointed out the
important role that affective, emotional reactions have in reading comprehension. She
posits that efferent reactions inﬂuence motivation and interest for reading, which in turn
have been shown to impact reading comprehension. In contrast, in the “active and
knowledge-transforming” sub-category, a student would receive a score of zero if he or
she made multiple comments that were coded as irrelevant or repeating statements. The
following comments come from a student who received a score of 2 on the “active and
knowledge transforming processes” subcategory. Upon seeing a diagram of a dragonﬂy
the student commented, “I think they sort of look like ants. How they have a head and a
middle part, and how they have a middle part and they have a back part and legs. They
look like ants.” The same student continues by connecting the dragonﬂies’ spiracles to
the gills of ﬁsh, “It reminds me how ﬁshes breathe through gills in the water, like through
tiny little holes. And that’s it.” These and other comments suggest that this student is

actively connecting the text to her prior knowledge in meaningful ways. In contrast the

28

following comments come from a student who received a score of zero on the “active and
knowledge transforming processes” subcategory. This student’s comments were often
superﬁcial. For example, when encountering the title page to dragonﬂies, the student
comments, “I’m thinking of. . .that this is about dragonﬂies.” When asked if the student
could say a little more, the student simply replied, “no.” Other students who received a
score of one on this subcategory had comments that ranged from superﬁcial and
irrelevant to comments that showed some internalizing of the text.

Another subcategory was “genre accuracy,” which fell under accuracy of
processes. “Genre accuracy” refers to whether students seem to understand the
informational text genre. For example, students received a score of two on “genre
accuracy” when they did not make anthropomorphizing or narrativizing comments and
did not refer to the book as a story. Students received a score of one when they either
included one or two narrativizing or anthropomorphizing comments or referred to the
book as a story. For example, the following child received a score of one for “genre
accuracy” because of anthropomorphizing comments such as, “I’m feeling like the, I’m
feeling like when they die... they’re like kinda scared if they’re gonna die it’s like us.
Like us if we’re scared we’re gonna die.” One student received a score of zero for “genre
accuracy” after she imposed a story structure onto Salmon. During the verbal report, this
student recounted a personal story that is about a time when she and her father went
ﬁshing. Some of the student’s comments were related to the pages in Salmon. For
example, on a page that depicts salmon babies, the student says, “I’m thinking that my
dad’s caught a big ﬁsh and it had babies in it.” On other pages, the story that the student

weaves completely veered off from the text or illustrations. For example, the student

29

reported, “That my little brother, um, once he was in the water, and he went like this
(makes a grabbing motion) and he caught a fish”, on a page that discusses the fact that
salmon are cold-blooded animals.

In order to arrive at a score for the quality measure for each verbal protocol, the
nine subfactors under accuracy and quality each receive a score of 0-2. (The coding
manual for the subfactors is available upon request.) These scores were then added
together to reach a total quality score. The quality score is not based on the quantity of
processes reported. Therefore, a student who only reported nine processes could
hypothetically score higher than a student who reported 35 processes during his or her
verbal protocol.

1 concluded the analysis by establishing a composite score, or total score, for each
verbal protocol. I wanted the composite score to represent what effective comprehension
processing looked like for second graders reading informational texts. Therefore, the
composite score for overall comprehension processing for the verbal protocols was based
on three measures. The ﬁrst measure represents the range in the total number of processes
reported, and captures the overall activity level of processing. The second measure
represents the range in the number of different processes reported. This measure
represents the degree to which the reader is employing a ﬂexible repertoire of processes
when comprehending. The third measure is the total quality score, which, unlike the ﬁrst
two measures, is not determined by the quantity of processes reported. It is important to
note that these three factors represent a set of behaviors that are thought to represent

different, important characteristics of good comprehension processing (Pressley &

Afﬂerbach, 1995).

30

At this point, while research has indicated that activity, ﬂexibility, and quality of
processing all matter (see Pressley & Afﬂerbach, 1995, for a review), it has not indicated
the relative weights that these factors contribute to overall comprehension. For this
reason, I decided to weigh the three factors equally. Because these three factors, total
processes, different processes, and quality scores, had different ranges, I converted each
of the scores into z-scores and then added the z-scores together in order to create a
composite score. These verbal protocol composite scores were then correlated with the
students’ COCA scores.

I estimated interrater reliability at three points based on having a second rater
independently code 10 of the 30 protocols. Interrater reliability was ﬁrst established for
units and was reached by percent agreement of total units coded. Estimated interrater
reliability was 92.3% across a total of 262 coded units.

Next, I estimated interrater reliability for processes. Processes agreement was
scored at the major code level and not the subcode level. For example, the following
comment, “It was when I was little and there was a dragonﬂy sitting on my ﬁnger. I just
remembered that,” would be coded as connection to prior knowledge, not “text-to-self
connection,” which is one kind of connection to prior knowledge. To estimate interrater
reliability for processes, I ﬁrst calculated the percentage of times both raters agreed that a
process was present in a verbal protocol (misses occurred when only one rater coded a
process). Process presence interrater reliability reached 90.2%. Then, of those times when
we agreed that a process was present, I calculated the percentage of times when we

agreed on the actual process. Interrater reliability for process agreement was 88.3%. Total

31

interrater reliability for agreement for process labeling was 77.7% (whether the two raters
correctly agreed on a process’s label).

The third interrater reliability analysis focused on the total quality scores of ten
verbal protocols. I had another rater independently rate 10 of the verbal protocols. For
this measure, I was not interested in students’ scores on individual subfactors, rather I
was interested in the total quality score. In order to calculate interrater reliability for the
quality scores I examined the correlations between our total quality scores. Interrater
reliability, as measured by the Pearson product-moment correlation, was r = .90, p < .01.

Results

Results of this study reveal that second graders utilize a range of processes,
including some highly sophisticated comprehension processes, when comprehending
informational text. While the study shows considerable overlap between the COCA and
the verbal protocols in the information they provide, each tool also provides unique
insights as well. This study’s results are organized into three major sections. First, I
present the comprehension processes found in the verbal protocols. The inventory of
verbal protocol processes was broken into three separate categories: processes that are
directly measured by the COCA, processes that are related to the processes measured by
the COCA but are not directly measured by it, and processes that are not directly or
indirectly measured by the COCA. Second, I discuss the item analysis, which involved
analyzing students’ comments during the verbal protocols for pages where there are
COCA items. Third, I compare the verbal protocol composite scores to the COCA total
scores.

Inventory of Processes for Verbal Protocols

Table 6, 7, and 8 summarize the descriptive statistics for each of the 23 coded
processes. These descriptive statistics are collapsed across Salmon and Dragonﬂies.
Table 6 contains those processes that are directly measured in the COCA. Table 7
represents those processes that are related to the processes measured by the COCA. For
example, while the COCA does not speciﬁcally have any questioning tasks, the
questioning process falls more broadly under the comprehension strategies umbrella.
Table 8 consists of those processes that are uniquely measured by the verbal protocols.
Coding the 30 verbal protocols resulted in a total of 820 coded processes. The processes
in the verbal protocols that are also directly measured by the COCA occurred most
frequently (N = 460), and consisted of seven total processes. The processes that were
related to the COCA items but were not directly related occurred less frequently 01 =
177), and were also comprised of seven total processes. Finally, the verbal protocols
measured nine unique processes that accounted for a total of 183 processes. Figure 1
provides a visual representation of the percentages for each of the type of process.

Table 9 describes the quantitative processing differences between Salmon and
Dragonﬂies. On average, students reported more processes and a wider variety of
processes when Salmon (mean of 32.44 total processes) was read to them as compared to
Dragonﬂies (mean of 21 .36 total processes). While the greater number of processes may
be partly explained by the greater length of the Salmon book, it is not a sufﬁcient
explanation by itself. Salmon is 19% longer than Dragonﬂies, but elicits 53% more
processes. Therefore, other unexplained, text-related factors may also be at work. For this

reason, I will disaggregate the remaining results by form.

33

Also, some of the processes were only observed in one of the two texts.
Questioning the author only occurred in the Dragonﬂies. Anthropomorphizing,
empathizing, and word identification only occurred in Salmon. It is important to note that
all of these processes were low in frequency (occurred ﬁve or fewer times).

Item Analysis

Another way to compare the COCA and verbal protocols is to examine how the
two compare on individual items. This involves mapping out what students report when
thinking aloud on pages where there are COCA items. In order to do this, I kept a record
of matches, related matches, misses, and non-responses. Matches occurred when a
process from a verbal protocol on that page matched the contributors measured by the
COCA on that page. For example, a match would be recorded if a student reported
summarizing during the verbal protocol on a page with a comprehension strategy item in
the COCA. Related matches happened when students reported a process from Table 2
when thinking aloud on a page that fell under the umbrella of the contributor being
tapped by the item on that page. For example, a student made a monitoring comment on a
page where there was a comprehension strategy item in the COCA. Misses occurred
when a student reported a completely different or unique process for a page than was
being measured by the COCA item on that page. For example, a miss would be coded
when a student’s response was coded as Processing Graphics in the verbal protocol on a
page with a comprehension strategies item in the COCA. A non-response arose when a
student failed to think aloud on a page with a COCA item. The difference between non-
responses versus misses is signiﬁcant since the former represents a failure to respond and

the latter represents a mismatch of processes. Table 10 and Table 11 summarize the item

34

analysis ﬁndings for the four COCA contributors. (The rates for these categories are
based on the proportion of the number of items for a particular contributor multiplied by
the number of students who were read Salmon or Dragonﬂies. For example, there were a
total of 21 matches and related-matches for comprehension strategies for Salmon for four
items across 16 students. I divided 21 by 56 to get .38)

The biggest mismatch between verbal protocols and COCA contributors was
vocabulary. This is due in large part to the fact that there were very few processes that
were coded as vocabulary in the verbal protocols. Vocabulary was only coded when the
student called explicit attention to a word or verbalized that they did not know what a
word meant. This only happened a total of 16 times across forms in the verbal protocols.
As a result, on pages where there were COCA items, students usually reported different
processes or no processes at all.

The agreement rates for comprehension strategies and text feature knowledge for
Dragonﬂies are .47 and .36 respectively and represent a medium rate of agreement. The
.71 rate of agreement for processing graphics seems particularly high. This may be due to
the presence of a diagram that contrasts dragonﬂies to damselﬂies. Almost 86% of the
students reported a processing graphics comment for this page. Similarly, 79% of
students noticed that the labels did not match the illustrations on another page that
depicted the environments where dragonﬂies live.

The agreement rates for comprehension strategies (.38) and processing graphics
(.36) for Salmon represent a medium rate of agreement. The high rate of agreement, .81,
for text feature knowledge is based on only two items, the Glossary and Index.

Interestingly, while the majority of students commented on these pages, these comments

35

greatly ranged in their quality and accuracy. Some students demonstrated misconceptions
about the purpose of these text features. One student explained that the number eight after
minnows meant that, “The ﬁshes have eight minnows.” Other students clearly understood
how to use these text features correctly. For example, one student commented that the
index, “tells you what pages, like if you are lost and you need to ﬁnd somewhere you just
go to the index and it tells you where to look and what page it is.” Other students simply
reported that they were not sure what the index was for:

Student: “I don’t know what all that is suppose to be for and everything”

(Points to numbers)

Researcher: “What do you mean? The numbers?”

Student: Yeah and the words.

If item analysis examines what the verbal protocols and COCA have in common,
examining students verbal responses to the pages that did not have COCA items can be
equally illuminating. Because students were asked to think aloud on all of the pages in
the book that had text during the verbal protocols, one can determine whether these pages
afford attention to a particular process(es). There were two pages in Dragonﬂies that did
not have COCA items. Page 1 depicts a dragonﬂy laying eggs on a pond plant, with these
various parts of the illustration labeled. While students used a variety of processes on this
page, there was no particular process used consistently by the students. However, on page
8, which depicts nymphs molting for the last time, 10 of 14 students applied various
comprehension strategies. Thus, if needed, the COCA could potentially ask a
comprehension strategy question on this page. In Salmon, there were six pages that did

not have corresponding COCA questions. Pages 1, 2, and 17 all had 50% or more of

36

students applying comprehension strategies. Interestingly, 5 of the 16 students made
predictions on both page 1 and page 2, evidence that some students actively comprehend
the text as soon as they begin reading. Pages 9, 11, and 18 of Salmon also did not have
COCA questions. On each of pages 9 and 11, 5 of 16 students made restating or repeating
comments. The text on these pages discusses salmon as they prepare to journey to oceans.
Interestingly, these are the only pages where these processes were consistently observed
in Salmon. Finally, on page 18, nearly half the students previewed the diagram on the
following page. These students attempted to make sense of the diagram before the text
for that page was even read. Overall, the additional pages seem to lend themselves to the
comprehension strategies and processes that are directly measured by the COCA.
Comparing Verbal Protocol Composite Scores to COCA Total Scores

In order to calculate a bi-partial correlation between the verbal protocol composite
scores and the COCA Total scores, I ﬁrst created a scatterplot for each verbal protocol
form. Both scatterplots revealed a linear relationship between the composite scores and
the COCA scores. For the students who thought aloud to Salmon and took the
Dragonﬂies COCA, the correlation was r = .646, p < .01. For the students who thought
aloud to Dragonﬂies and took the Salmon COCA form, the correlation was r = .664, p <
.01.

Discussion

In the following section, I will discuss the affordances and constraints that the
verbal protocols and the COCA each offer when it comes to measuring informational
comprehension processes. As expected, there are ways in which the two share similarities

as revealed by the large correlations between the two tools. Also, the two are similar in

37

that there is much overlap between the COCA and the verbal protocols when it comes to
measuring comprehension strategies, text features, and processing graphics. However,
there are ways in which the two offer unique insights into comprehension processing. The
COCA captures the vocabulary contributor better than the verbal protocols. There are
also nine processes that were uniquely captured by the verbal protocols. I will then
brieﬂy discuss what verbal protocols tell us about second graders’ informational text
comprehension. Finally, I will conclude with future directions for informational text
comprehension research and verbal protocol research.

Similarities between the COCA and Verbal Protocols

When discussing similarities between the two tools, it is important to point out
that the correlations between the composite scores of the verbal protocols and the total
scores of the COCA are large according to traditional standards (Cohen, 1988). However,
these correlations are especially large given that students were read two different texts
and experienced two different forms of assessments. Further, while both assessments are
process-oriented, these correlations were especially surprising given that there are two
different types of responses: directed questions, and thinking aloud.

The item analysis also revealed much overlap between the verbal protocols and
COCA when it comes to studying three of the four contributors measured in the COCA,
comprehension strategies, text features, and processing graphics. The item-analysis
showed substantial agreements between the verbal protocols and the COCA for these
contributors. In this way, verbal protocols provide substantive validity evidence for the
COCA because many students seemed to engage in the processes hypothesized by the

COCA (Messick, 1989). When reading these particular informational texts, second

38

graders showed evidence of using comprehension strategies, paying attention to text
features, and processing the graphics when thinking aloud.
Dissimilarities between the COCA and the Verbal Protocols

Vocabulary is the one contributor in the COCA that is not as well captured by the
verbal protocols. The COCA forces students to pay explicit attention to vocabulary in the
texts. However, the verbal protocols simply did not elicit many comments that fell under
the vocabulary process. This is a limitation of the verbal protocols in this study. Research
has indicated that a causal link exists between vocabulary and comprehension
(Blachowicz & Fisher, 2000). However, the current verbal protocols were unable to
provide much insight into this process.

Another dissimilarity between the verbal protocols and the COCA are the nine
processes that were revealed only in the verbal protocols. While some of these processes
such as, anthropomorphizing and empathizing, occurred rarely, others occurred quite
frequently (e. g., restating, repeating, irrelevant comments, and narrativizing). Future
research might study how these processes relate to comprehension outcomes. For
example, one might assume that students who anthropomorphize the animals and impose
a narrative structure on the text might have lower comprehension outcomes. However,
this is a hypothesis that needs to be conﬁrmed by future research. Additional research
might investigate how restating and repeating contribute to or inhibit reading
comprehension at this stage of development.

It is interesting that three of the processes from Table 3, anthropomorphizing,
empathizing, and word study, only occurred in the Salmon text. When looking back at the

anthropomorphizing and empathizing comments, I noticed that all the comments are in

reference to either prey being eaten or hunted, or the salmon dying at the end of their life
cycle. For example, a student said, “I’m thinking, ‘ouch’ that oughta hurt the salmon”
(coded as empathizing), after reading a page about how bears and eagles are enemies of
salmon. While there is a page in Dragonﬂies that references how their bodies help them
keep away from animals, Salmon pays more explicit attention to prey-predator
relationships. Also, there is no mention of the end of the dragonﬂies’ life cycle in the
Dragonﬂies text. Interestingly, this theme of dying seemed to evoke the
anthropomorphizing and empathizing processes. However, it should be noted that while
these processes were coded in Salmon, only a small percentage of students reported using
these processes. Examining the text associated with the anthropomorphizing and
empathizing comments suggests that subtle differences in book content can yield
signiﬁcant differences in comprehension processes (see further discussion below).
What Verbal Protocols Tell Us about Informational Text Reading Comprehension

One important contribution of this study is that children as young as second grade
can provide informative verbal protocols. Recall that Afﬂerbach and Johnston (1984)
posited that younger readers may produce poor read-alouds. However, the current study
demonstrates that when the text is read to them, second graders are capable of reporting
their comprehension processes.

Second, the verbal protocols in this study support the claim that second graders
are capable of transacting with informational texts in sophisticated ways. One fascinating
aspect of the verbal protocols is how frequently students would use comprehension
processes together. For example, students would often combine previewing with

processing graphics. Future research should investigate how young readers combine

40

processes, instead of studying the processes only independently. Evidence for
sophisticated processing also lies in the signiﬁcant overlap between the comprehension
processes described in the inventory and the comprehension strategies that adult readers
use (Pressley & Afﬂerbach, 1995). For example, 75% of second graders in this study
actively constructed meaning of the Salmon text as soon as they began reading (within
the ﬁrst two pages of the book that occur before the Title page). This early processing is
similar to when effective adult readers make predictions when beginning to read a new
text (Pressley & Afﬂerbach, 1995).

While the verbal protocols can capture the sophistication of second graders’
comprehension processes, this methodology is also sensitive to the range in quality of
processing, as demonstrated by this study. For example, some students reported primarily
“knowledge transforming,” and text-related comments, while other students’ verbal
protocols were characterized by superﬁcial comments and a lack of monitoring. This
attention to quality seems especially important if this methodology is going to have
practical applications for inﬂuencing educators’ comprehension instruction.

Finally, the open-ended nature of the verbal protocol highlights some
comprehension processes that are not typically captured in verbal protocol, work such as
affective reactions. The verbal protocols are also somewhat unique in that they
distinguish between different types of monitoring as students construct meaning from
informational text.

Affective reactions. One of the types of processes revealed in the verbal protocols
was affective reactions. Rosenblatt posited that there are two primary stances to reading

(1978). The ﬁrst stance, efferent reading, occurs when the reader is reading to identify

41

and collect information from a text. The second stance, aesthetic reading, focuses on
“living through” the text and includes the reader’s empathetic responses to a text.
Unfortunately, this Reader Response theory (1978) is often simpliﬁed to the point where
efferent reading is associated with informational texts and aesthetic reading is associated
with narratives. For example, Harvey and Goudvis (2007) in their recent edition of
Strategies that work, a reading comprehension text designed to inform teachers’
instruction, wrote, “When we read efferently, we are reading to ‘take away’ pieces of
information or to synthesize big ideas. It is the stance we take when we are reading
informational text... The ﬁction and literature reading we do is what Rosenblatt refers to
aesthetic reading” (p. 58-9). However, as this study clearly demonstrates, it is possible to
combine an efferent and aesthetic stance when reading informational text. 7 out of the 30
students reported affective reactions to the text. One student even made four affective
comments during his think-aloud. Alexander (1997) also explains that affective reactions
to informational text can be tied to motivation for reading, which affects reading
comprehension. Reacting emotionally to informational texts may not be something we
frequently think about, but maybe it is something that we need to pay more attention to.
For instance, more attention to how a range of affective reactions impact reading
comprehension outcomes is warranted. At the very least, we need to make sure that
teachers understand that affective reactions to text do not solely belong in the realm of
narrative texts.

Monitoring. Another process captured clearly in the verbal protocols was the
monitoring processes (knowledge monitoring, text monitoring, and rereading).

Monitoring one’s understanding of text is foundational to reading comprehension (see

Pressley & Afﬂerbach, 1995 for a review). The National Reading Panel Report also lists
monitoring as one of seven strategies that appear “to be effective and most promising for
classroom instruction” (p. 4-42, 2000). In the verbal protocols, these monitoring
processes occurred frequently. All but four students reported at least one monitoring
comment during their verbal protocol. The verbal protocols collected in this study reveal
different kinds of monitoring at work for young children. Speciﬁcally, the verbal
protocols distinguish between when students are monitoring their understanding of the
text, versus when they are monitoring how the text relates to their knowledge base. This
seems like a valuable distinction for both researchers and teachers. For teachers who
speciﬁcally want to know more about how their students monitor their comprehension
while reading, verbal protocols might serve as a valuable tool. In terms of the ﬁeld of
comprehension research, it would be interesting to see how these distinctions in
monitoring each contribute to comprehension outcomes.
Logistics of Administration and Scoring Verbal Protocols and the COCA

When considering the affordances provided by the verbal protocols and the
COCA, it is important to take into consideration the practical logistics associated with
administering and scoring each of these tools. The administration time length for each of
the assessments is similar, although it should be noted that there seems to be more
variability associated with the verbal protocols. The COCA takes approximately 15
minutes to administer and the verbal protocols last anywhere between 9 to 21 minutes.
Currently, it is beyond the scope of this study to compare how long it takes for teachers to
score or code the two assessments. As a researcher who has extensive experience coding

both assessments, I can attest to the fact that there seems to be a learning curve associated

43

with both when ﬁrst learning to code the assessments. These assessments become easier
and faster to score with practice. This seems to be an important consideration to
communicate with teachers when they are ﬁrst learning about the assessments. An
important next step for verbal protocol research is to investigate how efﬁciently and
reliably teachers can code and score the verbal protocols.

One of the major logistical costs associated with the verbal protocols over the
COCA is the time it takes to transcribe the verbal protocols. Individual protocols
typically took at least an hour to transcribe (with many taking over two hours to
completely transcribe). Meyers and colleagues note that practitioners do not have the
time or resources to transcribe tapes accurately (Meyers, Lytle, Palladino, Devenpeck, &
Green, 1990). In their attempt to offer a practical solution, the researchers hand wrote all
responses and questions during the verbal protocols (it should be noted that students were
only responding to one sentence at a time and did not experience lengthier passages of
text with illustrations, as is the case in the present study). When they compared these
handwritten and audio-taped transcripts, they found that there was close agreement
between these two methods (92% of content words were identical and 82.7% of total
words were identical in each pair of transcripts). Furthermore, they noted that the
transcriptions were consistent across the researchers. Therefore, the results of their study
were based on the handwritten protocols. Because verbal protocols seem to provide
additional valuable insights to reading comprehension that other assessments may not tap,
studies such as this one that examine approaches that attempt to make the collection and
analysis of verbal protocols more feasible are needed. Ideally, future studies could

examine the reliability of teachers as they code the verbal protocols in real time.

44

Limitations and Future Directions

One limitation of this study is the extent to which the results can be generalized
beyond the current sample of second graders. The participating students came from
primarily suburban districts. Also, while the schools vary on a continuum for
demographic variables, the school populations consist largely of Caucasian students.
Finally, no students who receive special education services are represented in the current
sample. In these ways, the sample of the current study is restricted.

There is also information that I failed to collect which would have been helpful in
determining the generalizability of the current study. For example, in the future I will
work to get the rate of parental consent for two of the classrooms. Next, while I have
information about individuals who are English Language Learners and have Special
Education designations, I did not collect other information such as individual
demographic information and students’ reading assessment results, that might allow me
to more clearly depict who these students are. Finally, I did not collect classroom
observations. Observations of factors such as teachers’ informational text comprehension
instruction, the amount and type of informational text available to students, and the
amount of informational text writing in which students participated, would likely be
helpful in determining the generalizability of this study’s results. For example, it may be
that some students were better able to verbalize their thinking because sharing one’s
thinking when reading is an established reading procedure in some of these ﬁve?
classrooms. Future research could link teachers’ instructional practices to the processes

measured by verbal protocols.

45

Processing graphics. In applying the verbal protocol methodology, this study is
unique because the coding system for the verbal protocols explicitly accounts for
students’ attempts to make sense of graphics. Interestingly, in the past, verbal protocol
researchers have not paid much attention to how graphics inﬂuence meaning
construction. In fact, the majority of verbal protocol studies seem to be based on texts
that do not have graphics (Kucan & Beck, 1997 ; Pressley & Afﬂerbach, 1995). This
presents an issue when studying how young children make sense of text as they read,
because most texts for young children contain graphics. Authentic informational texts
almost always contain realistic illustrations or photographs, and often include graphical
devices, such as diagrams (Purcell-Gates, Duke, & Martineau, 2007). Therefore, it is
perhaps not surprising that processing graphics was the most frequently reported process
by this second grade sample. While the coding scheme accounts for occasions when
students are processing graphics, it does not delineate the many ways that students are
using graphics to comprehend the books. For example, using a graphic to conﬁrm
something found in the text seems qualitatively different than using the graphic to make a
prediction. Additionally, I coded processing graphics separately from other processes in
order to making the coding process simpler. However, students frequently used
processing graphics in tandem with another process. Therefore, I highly recommend that
future verbal protocol studies focus on the different ways that students process graphics
and relate these graphical processes to comprehension outcomes for informational texts.

Verbal protocols and text sensitivity. In developing the COCA, Salmon and
Dragonﬂies were designed to be as similar as possible given the differences between the

two animals. For example, when designing the books, we chose two animals that

46

experience multiple, distinct stages in their life cycles. Also, when designing the two
COCA books, we often compared the text and illustrations. For example, there are
diagrams in both forms that label the parts of the animals. (In addition, the illustrator was
the same in both texts, so there is a similar artistic style in Salmon and Dragonﬂies.)
There are also several pages in both books that explicitly deﬁne vocabulary words.
Finally, there are glossaries and indexes in both forms of the book.

Despite these design similarities, the verbal protocols were different in the two
forms. The students who were read Salmon tended to report more processes and a wider
range of processes on average as compared to students who were read Dragonﬂies.
Therefore, the verbal protocol data suggests that there are text differences signiﬁcant
enough to prompt differences in comprehension processing. In other words, something
about the texts elicits differences in the way students process the two COCA forms. One
possible explanation for the differences may lie in the information presented in the books.
As discussed previously in the verbal protocols, students made affective and empathizing
comments on the pages in Salmon that discuss prey relationships and the death of the
Salmon at the end of their life cycles. Similar information is not presented in Dragonﬂies.
Also, it may be possible that students have more experience or prior knowledge with ﬁsh
than insects. Such differences in prior knowledge may partly explain the processing
differences in the two forms.

An important question arises, how much of these differences are due to the text
sensitivity of the verbal protocols? In other words, what does any one verbal protocol tell
us about a student’s comprehension processing of another similar text? Myers, Lytle,

Palladino, Devenpeck, and Green (1990) asked a similar question about the stability of

47

processes across verbal protocols of narratives for fourth and ﬁfth-graders. They found
that ﬁve of the six categories of “moves” (similar to what I call processes) had intraclass
correlations of .72 or above. They concluded that moves were stable across the three
stories, especially when the moves occurred with some frequency. Similar verbal
protocol work could be done with informational texts and younger readers.

Of course we know that the way we comprehend is dependent on a variety of text
factors (see RAND Reading Study Group for a review, 2002). However, if verbal
protocols are to be helpful to teachers, there needs to be some generalizability across
texts. Future research could study the stability of processing across multiple texts for
developing readers. Establishing processing stability seems an important next step if
verbal protocols are to be used as a practical assessment tool for teachers in the future.

Sensitivity to instruction. Research that would establish whether these assessments
are sensitive to instruction would strengthen their validity and also serve to increase their
usability to teachers. Currently, work is underway to investigate the impact of ﬁrst and
second grad teachers’ administration and use of the COCA over a school year (Bolt,
Duke, & Billman, 2008). However, similar work has not been conducted with verbal
protocols. While researchers have used think-alouds as a method to improve students’
comprehension (see Kucan & Beck, 1997 for a review), researchers have not typically
looked at whether verbal protocols are sensitive to instruction (see Brown, Pressley, Van
Meter, & Schuder for an exception). For example, if teachers gradually release a strategy
to students over the course of several weeks, do students report this strategy more
frequently, and with a higher quality, when thinking aloud? As an anecdote from the

present study, while collecting verbal protocols in one classroom, I noticed that most of

48

the students were asking multiple questions when thinking aloud in the form of “I
wonder” statements. Afterwards, I asked the teacher if they were currently working on
the questioning strategy. She informed me that the class had just ﬁnished studying
questioning as a part of her reading comprehension instruction and that she used the “I
wonder” language as a way to introduce the strategy. Linking verbal protocols to
instruction represents a promising direction for verbal protocol research, and is an
important step to establishing verbal protocols as a useful assessment tool for teachers.

Verbal protocols and comprehension outcomes. These last two future directions
are a part of a larger need for verbal protocol research. A crucial next step for verbal
protocol research lies in tying the results of verbal protocols to comprehension outcomes.
Pressley and Afﬂerbach made a similar observation in 1995. Unfortunately, very few
researchers have taken this next step. My previous work with narrative texts (2006) is an
exception. However, I found no signiﬁcant correlations between students’ strategy use
and retellings as measured by propositions recalled from the stories. Meyers and
colleagues (1990) also took this step, ﬁnding both positive and negative correlations
between categories of comprehension processes as measured by think-alouds and passage
comprehension. Part of the problem in establishing correlations between verbal protocols
and outcomes is methodological in nature. The vast majority of verbal protocol work is
done with very small sample sizes, which makes it difﬁcult to ﬁnd statistically signiﬁcant
correlations. For example, in Pressley and Afﬂerbach’s review of verbal protocols in
reading, only 6 of 38 studies had 30 or more participants.

Also, part of the problem of linking individual processes to comprehension

outcomes might lay in the fact that much of past verbal protocols research has not

49

attended to the quality of processes reported. Whether or not a student applies a particular
process may not be as important as the quality of that process. For example, Williams
(1993) found that the comprehension processes of students with learning disabilities
tended to be more tangentially related to the text than the processing of average achieving
students. Both the students with learning disabilities and the average achieving students
were using similar strategies. However, the average achieving students were more
effective at applying them. Most students in the primary grades are novices at applying
these strategies, and are not likely to apply them all equally successfully at once. In other
words, they do not resemble many of the effective adult readers that comprised Pressley
and Afﬂerbach’s model of “constructively responsive reading” (1995). Perhaps this is not
surprising, given that Pressley and colleagues noted that such self-regulated strategy use
takes years to master in school (1992). Thus, what processes seem to hinder or aid
comprehension outcomes at this stage of development is difﬁcult to answer when issues
of quality seems so crucial. Therefore, a strength of the current study is that I have shown
that quality of processing can be articulated on a continuum and can be reliably scored.
Future research could examine the relationship between quality scores and outcomes.

Reading comprehension development is a complex web affected by multiple
related factors. Only through concerted research efforts which use multiple

methodologies and measures can we continue to untangle it.

50

References

Afﬂerbach, P. (2008). Understanding and using reading assessment, K-12. Newark, DE:
International Reading Association.

Afﬂerbach, P. (2000). Verbal reports and protocol analysis. In M. L. Kamil, P. B.
Mosenthal, P. D. Pearson, & R. Barr (Eds), Handbook of reading research (Vol.
111, pp. 163-179). Mahwah, NJ: Lawrence Erlbaum Associates.

Afﬂerbach, P., & Johnston, P. (1984). Research methodology: On the use of verbal
reports in reading research. Journal of Reading Behavior, 1 6, 307-322.

Alexander, PA. (1997). Knowledge seeking and self-schema: A case for the motivational
dimensions of exposition. Educational Psychologist, 32, 83-94.

Anderson, R. C., & Pearson, P. D. (1984). A schema-theoretic view of basic processes in
reading. In P. D. Pearson (Ed.), Handbook of reading research (pp. 225-291).
New York: Longman.

Beck, I. L., McKeown, M. G.,& Kucan, L. (2002). Bringing words to life: Robust
vocabulary instruction. New York: Guilford Press.

Bereiter, C., & Bird, M. (1985). Use of thinking aloud in identiﬁcation and teaching of
reading comprehension strategies. Cognition and Instruction, 2, 131-156.

Billman, A. K., Duke, N. K., Hilden, K., Zhang, S., Roberts, K., Halladay, J. L., et. al.
(2008). Concepts of Comprehension Assessment (COCA). East Lansing: Trustees
of Michigan State University.

Blachowicz, C. L., & Fisher, P. J. (2000). Vocabulary instruction. In M. L. Kamil, P. B.
Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of Reading Research,
(Vol. 111, pp. 503-523). Mahwah, NJ: Lawrence Erlbaum Associates.

Bolt, 8., Duke, N. K., Billman, A. K. (2008, December). Using informational reading
comprehension assessment to inform instruction and improve achievement: The

COCA in action. Paper presented at the National Reading Conference, Orlando,
FL.

Brown, R., Pressley, M., Van Meter, P., & Schuder, T. (1996). A quasi-experimental
validation of Transactional Strategy Instruction with low achieving second-grade

readers. Journal of Education, 88, 18-37.

Cassidy, J., & Cassidy, D. (February/March 2008). What’s hot for 2008. Reading Today,
25, 10-11.

51

Center for Educational Performance & Information. (2006-7). Pupil Headcount Data
(SRSD). Retrieved July 9, 2008, from http://michigan.gov/cepi/0,l607,7-113-
21423_30451_30460---,00.html

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
Mahway, NJ: Lawrence Erlbaum Associates.

Cote, N., Goldman, S. R., & Saul, E. U. (1998). Students making sense of informational
text: Relations between processing and representation. Discourse Processes, 25,
1-53.

Donovan, C. A., & Smolkin, L. B. (December, 2006). Children ’s selections and reading
of science trade books. Paper presented at the National Reading Conference
annual meeting. Los Angeles, CA.

Duke, N. K., & Kays, J. (1998). “Can I say ‘Once upon a time’?”: Kindergarten children
developing knowledge of information book language. Early Childhood Research
Quarterly, 13, 295-318.

Duke, N. K., Pressley, M., & Hilden, K. (2004). Difﬁculties with reading comprehension.
In C. A. Stone, E. R. Silliman, B. J. Ehren, & K. Apel (Eds.), Handbook of

language and literacy development and disorders (pp. 501-520). New York:
Guilford.

Ericcson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data.
Cambridge, MA: The MIT Press.

F riedenberg, J. (2005). Mind as a black box: The Behaviorist approach. In J. Friedenberg, &
G. Silverman, (Eds.), Cognitive science: An introduction to the study of mind (pp. 85-
88). Thousand Oaks, CA: Sage Publications.

Harvey, S., & Goudvis, A. (2000). Strategies that work: Teaching comprehension to
enhance understanding. Portland, MN: Stenhouse Publishers.

Hidi, S. E., & Hildyard, A. (1983). The comparison of oral and written productions in
two discourse types. Discourse Processes, 6, 91-105.

Hiebert, E. H. (2008). The word zone ﬂuency curriculum: An alternative approach. In
M.R. Kuhn and RI. Schwanenﬂugel (Eds.), Fluency in the classroom (pp. 154-
170). New York: Guilford.

Hilden, K. (2006, December). Verbal protocols: A window into second graders ’ reading
comprehension of narrative texts. Paper presented at the National Reading
Conference annual meeting. Los Angeles, CA.

52

Hilden, K., & Pressley, M. (2002, December). An existence proof that comprehension
strategies instruction can occur in ﬁrst grade. Paper presented at the National
Reading Conference annual meeting. Miami, FL.

Johns, J. (1997). Basic Reading Inventory: Pre-primer through grade twelve and early
literacy assessments (9th ed.). Dubuque, IA: Kendall/Hunt Publishing Company.

Keene, E., & Zimmermann, S. (1997). Mosaic of thought: Teaching comprehension in a
reader ’s workshop. Portsmouth, NH: Heinemann.

Kucan, L., & Beck, 1. L. (1997). Thinking aloud and reading comprehension research:
Inquiry,
instruction, and social interaction. Review of Educational Research, 67, 217-299.

Laing, S. P., & Kamhi, A. G. (2002). The use of think-aloud protocols to compare
inﬂuencing abilities in average & below-average readers. Journal of Learning
Disabilities, 35, 437-448.

Langer, J. (1996). Reading, writing, and understanding: An analysis of the construction
of meaning. Written Communication, 3, 219-267.

Leslie, L., & Caldwell, J. (2006). Qualitative Reading Inventory-4 (QRI-4). New York:
Addison Wesley Longrnan, Inc.

Lundeberg, M. A. (1987). Metacognitive aspects of reading comprehension: Studying
understanding in legal case analysis. Reading Research Quarterly, 22, 407-432.

MacGinitie, W. H., MacGinitie, R. K., Maria, K., Dreyer, L. G., & Hughes, K. E. (2000).
Gates-MacGinitie Reading Tests. Rolling Meadows, IL: Riverside Publishing.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp.
12-103). London: Collier Macmillan.

Meyers, J ., Lytle, S., Palladino, D., Devenpeck, G., & Green, M. (1990). Think-aloud
protocol analysis: An investigation of reading comprehension strategies in fourth-

and ﬁfth-grade students. Journal of Psychoeducational Assessment, 8, 112-127.

Miller, D. (2002). Reading with meaning: Teaching comprehension in the primary
grades. Portland, ME: Stenhouse Publishers.

National Center for Education Statistics. (2005-6). Search for schools, colleges, and
libraries Retrieved July 2, 2008, from http://nces.ed.gov/globallocator/

National Reading Panel. (2000). Teaching children to read: An evidence-based
assessment of the scientiﬁc research literature on reading and its implications for

53

reading instruction: Reports of the subgroups. Washington, DC: National
Institute of Child Health and Development.

Olshavsky, J. B. (1976—1977). Reading as problem solving: An investigation of
strategies. Reading Research Quarterly, 12, 654—674.

Pearson, P. D., & Duke, N. K. (2002). Comprehension instruction in the primary grades.
In C. C. Block & M. Pressley (Eds.), Comprehension instruction: Research-based
best practices, (pp. 247-258). New York: Guilford.

Peskin, J. (1998). Constructing meaning when reading poetry: An expert-novice study.
Cognition and Instruction, 16, 235-263.

Pressley, M., & Afﬂerbach, P. (1995). Verbal protocols of reading: The nature of
constructively responsive reading. Hillsdale, NJ: Erlbaum.

Pressley, M., Duke, N., Gaskins, I. W., F ingeret, L. Halladay, J ., Hilden, K., et.al. (in
press). Reading remediation: Why we must get beyond the simple view of reading
and visions of how it might be done. In C. R. Reynolds & T. B. Gutkin (Eds.),
Handbook of School Psychology (4th ed.). New York: John Wiley & Sons.

Pressley, M., El-Dinary, P. B., Gaskins, 1., Schuder, T., Bergman, J., Almasi, L., et. al.
(1992). Beyond direct explanation: Transactional instruction of reading
comprehension strategies. Elementary School Journal, 92, 511-554.

Pressley, M., Harris, K. R., & Marks, M. B. (1992). But good strategy instructors are
constructivistsll Educational Psychology, 4, 1-32.

Pressley, M., & Hilden, K. (2004). Verbal protocols of reading. In N. K. Duke, & M. H.
Mallette (Eds.), Literacy research methodologies (pp. 308-321). New York:
Guilford.

Pressley, M., Wood, E., Woloshyn, V. E., Martin, V., King, A., & Menke, D. (1992).
Encouraging mindful use of prior knowledge: Attempting to construct
explanatory answers facilitates learning. Educational Psychologist, 27, 91-110.

Purcell-Gates, V., Duke, N. K., & Martineau, J. A. (2007). Learning to read and write genre-
speciﬁc text: Roles of authentic experience and explicit teaching. Reading Research

Quarterly, 42, 8-45.
f

RAND Reading Study Group. (2002). Reading for understanding: Towards and R & D
program in reading comprehension. Santa Monica, CA: Rand Education.

Rosenblatt, L. (1978). The reader, the text, the poem. Carbondale, IL: Southern Illinois
University Press.

54

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory
procedures and techniques. Newbury Park, CA: Sage.

U. S. Census Bureau. (2006). American Community Survey. Retrieved on September, 11,
2008 from http://nces.ed.gov/surveys/sdds/acs06/acs06ref.aspx

U. S. Department of Education, National Center for Education Statistics. (2005-6). The
common core of data (NCES 2007-354). Washington, DC: Author.

Williams, J. P. (1993). Comprehension of students with and without learning disabilities:
Identiﬁcation of narrative themes and idiosyncratic text representations. Journal
of Educational Psychology, 85, 631-641.

Wyatt, D., Pressley, M., El-Dinary, P. B., & Stein, S. (1993). Comprehension strategies,
worth, credibility monitoring, and evaluation: Cold and hot cognition when
experts read professional articles that are important to them. Learning and
Individual Differences, 5, 49-72.

Yuill, N., & Oakhill, J. (1991). Children '5 problems in text comprehension. Cambridge,
England: Cambridge University Press.

55

MANUSCRIPT 2: PROFILES FOR INFORMATIONAL TEXT COMPREHENSION IN
SECOND GRADE

Reading comprehension has been deﬁned as the “process of simultaneously
extracting and constructing meaning through interaction and involvement with written
language” (RAND Reading Study Group, 2002, p. 11). Furthermore, this process is
inﬂuenced by three factors: the reader, the text, and the activity in which the reading is
occurring (RAND Reading Study Group, 2002). The abilities and dispositions that an
individual brings to reading, such as one’s background knowledge, motivation, cognitive
capacities (e. g., memory, ﬂuency, oral language) all inﬂuence how a reader
comprehends. Text features such as the semantic and syntactic difﬁculty, length, and
topic can also inﬂuence reading comprehension. Additionally, the genre of the text also
affects how we comprehend (Kucan & Beck, 1997; Pressley & Afﬂerbach, 1995).
Finally, the activity, or purpose for which reading is done and the consequences attached
to it also impact reading comprehension (RAND Reading Study Group, 2002). However,
these three factors do not act independently but rather inﬂuence one another. And it is
how these factors uniquely weave together that makes reading comprehension so
fascinating and complicated to study.

Verbal protocol methodology has greatly informed what is known about how
expert adult readers construct meaning from texts. Researchers who use verbal protocols
are typically interested in ﬁnding out what is going on in the mind as a person completes
a task. Verbal protocols have a long history in the ﬁeld of cognitive problem-solving
(Ericcson & Simon, 1984). Since the 805, researchers have applied this methodology to

the ﬁeld of reading to learn more about the cognitive processes involved in reading

56

comprehension. From Pressley and Afﬂerbach’s (1995) seminal work that synthesized
verbal protocol research, we now know that expert readers are constructively responsive
when they read. This means that they apply a repertoire of comprehension strategies
before, during, and after reading. And that they ﬂexibly apply this repertoire based on
various text factors and their purposes for reading. Furthermore, from these verbal
protocol studies we know that effective comprehenders monitor their on- going
comprehension and have ﬁx-up strategies for when their comprehension breaks down.

While we have a clear vision of the processes involved in expert reading
comprehension in adults, the vision for how reading comprehension develops in young
readers is much less clear. Cain and Oakhill (2007) reviewed studies that examined how
individual factors such as phonological, metalinguistic, semantic, and syntactic skills
impact children’s reading comprehension. They concluded that there are not deﬁnitive
trajectories for reading comprehension development and that we need better models of
development.
Verbal Protocol Methodology and Comprehension of Developing Readers

One area of reading comprehension development that has not received extensive
attention is the study of the online comprehension processes used by younger readers.
Verbal protocols represent a potentially fruitful methodology for studying these online
processes. However, this methodology has not been as extensively used with young
readers, particularly children in second grade and below. This lack of protocol studies
with developing readers is likely due in part to the assumption that they are incapable of
providing useful verbal protocol data. Afﬂerbach and Johnston ( 1984) reasoned that

verbalizing their thoughts is likely to overwhelm the developing readers’ cognitive

57

resources, and therefore, the methodology would not result in an accurate description of
their reading comprehension processes. Furthermore, Afﬂerbach and Johnston reasoned
that young readers may not be sufﬁciently metacogrritively aware to accurately report
their thinking.

Despite these reservations, Afﬂerbach (2000) called for future verbal protocol
research to study the developmental nature of reading comprehension. Presumably this
would include primary grade children. While the reasons discussed above discourage
employing verbal protocol methodology with young students, positive research evidence
exists for the potential use of this methodology. One study found that second graders
were capable of verbalizing their thinking as they read an illustrated Aesop’s fable
(Brown, Pressley, Van Meter, & Schuder, 1996). These researchers triangulated the
verbal protocol with other assessment data.

In my past verbal protocol research (2006), I found that second graders who read
at or above grade level could verbally report their thoughts and feelings while reading a
narrative picture book aloud. Similarly, Alverrnann concluded that the second graders’ in
her study, “seemed relatively at ease with the task” of thinking aloud (p. 186, 1984). In
this verbal protocol study Alverrnann analyzed second graders’ verbal protocols to
investigate strategies that students used to comprehend narratives in basal readers.

In contrast, the present study is unique because it focuses on the comprehension
processes that students apply to informational texts. Currently, researchers have not used
verbal protocol methodology to focus on how primary grade students construct meaning
from informational texts. The issue of genre is an important one as research suggests that

genre impacts the way we process as we read, both for adults (Kucan & Beck, 1997;

58

Pressley & Afﬂerbach, 1995) and children (e. g., Duke, Pressley, & Hilden, 2004; Hidi &
Hildyard, 1983; Langer, 1996).
Reading Proﬁles

Few researchers have used verbal protocols to develop proﬁles of reading
comprehension. Wade (1990) developed a taxonomy of ﬁve proﬁles of narrative reading
comprehension using verbal protocol methodology. Wade had struggling second to ninth
grade readers think aloud they read short segments of stories. These proﬁles focused on
cognitive top-down and bottom-processing and focus primarily on students’ connections
to prior knowledge and ability to monitor their comprehension. The ﬁve proﬁles
included: good comprehenders, the non-risk taker, the non-integrator, the schema
imposer, and the storyteller. Wade characterized Good Comprehenders as interactive
readers who monitor their comprehension. In contrast, the Non-risk Takers assumed
passive roles because they did not go beyond the text. Wade concludes that these readers
“either lack or underutilize their background knowledge and over rely on the text to
suggest an appropriate schema (p. 447). The Non-Integrators did not connect the
hypotheses they made to one another or the text. The Schema-Imposer is a top-down
processor who applies a schema early on and holds onto it despite incoming conﬂicting
information. Finally, the Storyteller allows their prior knowledge to take over the
information in the text to the point where their comprehension does not resemble the
original text. The present study uses a wider lens when looking at comprehension in that I
did not limit my investigation to only cognitive strategies, but included other processes

linked to comprehension.

59

Wade, Trathen, and Schraw (1990) used cluster analysis to create an empirical
taxonomy of college students as they studied a lengthy expository text. This taxonomy
consisted of six clusters, or proﬁles, of students: Good Strategy User, the Information
Organizer, the Flexible Reader, the Text Noter, the Mental Integrator, and the
Memorizer. These proﬁles were organized around the three types of strategies or tactics
that they applied when reading: text-noting tactics (e. g., highlighting, underlining,
outlining, diagramming), mental-leaming tactics (e. g., mental integration, connecting to
background knowledge, imaging, and self-questioning), and reading tactics (e. g., varying
reading speed and re-reading). The Good Strategy Users used the widest range of study
tactics and were more likely to provide the purposes for using each tactic. The
Information Organizers also reported many mental learning and text-noting tactics, but
rarely reported reading tactics. Their comprehension tended to focus on organizing the
main ideas. Flexible Readers placed heavy emphasis on reading tactics. They frequently
reported changing their reading speed and re-reading ﬂexibly according to their changes
in understanding of the expository text. The Text Noters almost exclusively used text-
noting processes. The Mental Integrators relied heavily on the mental-leaming tactics
and only reported a few of the other types of tactics.

Purpose of the Study

Recently when I collected verbal protocols, or think-alouds, from second graders
as they processed an informational text, I noticed how differently the students went about
constructing meaning. The fact that students uniquely transacted with the texts should not
be surprising. Indeed, as stated previously we have come to expect that individuals

uniquely transact with the text as they read, as this is one of the tenets of modem

60

comprehension theory (Pressley & Afﬂerbach, 1995; Rosenblatt, 1978). However, what
is not well known are the different ways that second graders construct meaning when
informational texts are read to them. Therefore, in the current study I sought to address
the following research question: What are different proﬁles of informational reading
comprehension in second grade? Answering this question has potentially noteworthy
implications for developmental reading comprehension theory and instruction. Better
understanding of the proﬁles of active meaning making in developing readers will help
the ﬁeld identify different patterns and levels of sophistication in development.
Additionally, ﬁrture research could link these processing proﬁles to comprehension
outcomes. Also, the ability to pinpoint students’ strengths and weaknesses in
comprehension processes would likely enable teachers to streamline their comprehension
instruction. If future research could link proﬁles with comprehension outcomes, then
additional research could investigate how these proﬁles could be used to inform teachers’
reading comprehension instruction.
Methods

In this study, I investigated different proﬁles of informational reading
comprehension in second grade. I used grounded theory analysis (Strauss & Corbin,
1990) to develop an inventory of reading comprehension processes that the second
graders used when processing an informational text. The second step consisted of
identifying processing patterns students used when constructing meaning from the text.
From these emerging patterns, I created proﬁles that classiﬁed the protocols.

Participants

61

This study involved 30 second grade students from ﬁve classrooms. Within each
classroom, I divided the consenting students into six ascending ability groups. These
rankings were based on teachers’ perceptions of students’ informational reading
comprehension. I then randomly sampled one student from each group. This random
sampling resulted in 19 girls and 11 boys who participated in the study. Three of the
students were English Language Learners and none of them received special education
services in the 2006-07 school year. See Tables 1 and 2 for student and school
demographics.

Data Collection Procedures

This study was a part of a larger study that compared second graders’ verbal
protocols to their scores on the Concept of Comprehension Assessment (COCA), a newly
developed process-oriented reading comprehension assessment for informational text
(Billman, Duke, Hilden, et. a1, 2008). In the COCA, students answer questions or ﬁll in
blanks as an informational text is read to them. These questions and blanks were
purposefully designed to assess four hypothesized contributors to informational reading
comprehension: comprehension strategies, processing graphics, text feature knowledge,
and vocabulary. The COCA authors chose four strategies to represent the larger range of
comprehension strategies that effective comprehenders apply when reading. These
strategies included: previewing and predicting, connecting to prior knowledge, inferring,
and summarizing. Items that assess processing graphics measure students’ understanding
of illustrations and graphics such as, diagrams, tables, maps, and illustrations. Text

feature knowledge refers to the ability to recognize and apply common text features to

informational texts (e. g., Table of Contents, index, labels, captions, and pronunciation

62

guides). Finally, the vocabulary contributor consists of two related items. The ﬁrst type
of vocabulary item examines students’ knowledge of high-utility words that cut across
many domains. The second type of vocabulary item measures students’ ability to infer
word meaning from the surrounding text and graphics.

In the larger study, students participated in two individual sessions spaced one to
three school days apart. In the ﬁrst session, the students were trained in the verbal
protocol procedure and produced a verbal protocol to one of the two forms of the COCA,
Salmon and Dragonﬂies (these books are described in greater detail later). During the
second session, a research assistant administered the other form of the COCA. For the
purposes of this study, only the ﬁrst session is of interest.

The ﬁrst session began by asking students to think-aloud while solving a 3-D
puzzle. This served as an introduction to thinking aloud. Before assembling the puzzle,
students were asked to verbalize whatever they were thinking or feeling while putting it
together. Students were cued approximately once a minute if they did not independently
verbalize what they were thinking or feeling. (See Appendix A for the Think-Aloud
Protocol.) After completing the training activity, the students were given the following
directions for thinking aloud with the book:

Now before we start, I have some directions. What I am really interested

in is hearing your thoughts and feelings about the book as I read it to you.

So just like the puzzle where you told me what you were thinking and

feeling, I want to know what’s going on inside your mind as you read.

You can stop and tell me what you are thinking whenever you want. That

is okay. Remember, just like the puzzle, there are no right or wrong

answers. Also, do you see these yellow dots? I will stop and ask you to tell

me what you are thinking and feeling when I get to each yellow dot.

Sometimes there may be places where the book is not making you think or

feel anything. If this happens you can shake your head no or tell me that

you are not thinking anything and we will go on. Also, it is okay if you

want to ﬂip through and look at pages. Finally, if you want to hear any of

the pages again just ask and I will reread the page to you. Can you tell me

the directions so that I know you understand please? (Clarify any

misconceptions and review directions as needed.)

I checked to make sure that the students included three main points when
repeating the directions: 1) the student can think aloud at any time 2) the student
should either think aloud after yellow dot or say, “I’m not thinking anything” 3)
the student can ask to reread.

Alter establishing the directions for thinking aloud, I read either Salmon or
Dragonﬂies to the students. Book selection was randomly assigned within each class with
roughly equal numbers of students experiencing each form. Students were cued to think
aloud on each page. A nonverbal reminder to think-aloud was present on each page in the
form of a yellow dot in the bottom outside comer (this “dot” method has been employed
in past verbal protocol work, e. g., Olshavsky, 1976-7). Reminders to think aloud were
also given on each page when students did not verbalize independently. This verbal cue
took the form of “What are you thinking or feeling?” or directed them to “Think aloud for
me.” Nondirective prompts such as these have been used successfully in the past and is

believed to minimize the inﬂuence that the researcher has on the participants’ verbal

64

reports (e. g., Bereiter & Bird, 1985; Lundeberg, 1987; Myers, Lytle, Palladino,
Devenpeck, & Green; 1990; Olshavsky, 1976-7; Wyatt, Pressley, El-Dinary, & Stein,
1993). A frequent criticism of verbal protocols is that they are thought to stimulate
thinking that would not otherwise occur in the absence of verbalization (Nisbett &
Wilson, 1977; Scardamalia & Bereiter, 1983). In light of this criticism, I attempted to
reduce the “talking to talk” phenomenon that I experienced in previous verbal protocol
work (Hilden, 2006) by allowing students to skip thinking aloud after a page was read. If
students failed to think-aloud on three pages I reminded them that, “it is really important
for you to share what you are thinking and feeling when we get to these yellow dots.”

Follow-up prompts were given in several situations. First, follow-up prompts
were given in order to clarify what a student was trying to communicate. For example,
when a student commented that, “I already knowed that” in Salmon, I followed up by
asking, “You already knew what?” in order to clarify what she was thinking. Secondly, I
followed up when students responded nonverbally. For example, I asked, “I saw you kind
of frown on that page while I was reading. What made you do that do you think?”
Finally, when students focused their attention on a different page than the one I was
reading I would use a follow-up prompt, “I saw you looking at this other page (p. 17)
when I was reading this page (p. 16). Can you tell me what you were thinking when you
looked over there? (p. 17)” Similarly, when I was unclear as to what a student was
referencing in an illustration, I would ask, “Can you show me where you mean?” or “Can
you point to where you are looking?” These non-verbal behaviors seem to indicate that
the student is processing the text or illustration. I wanted to make sure that I got as

complete a picture as possible of the students’ processing.

65

In addition to the yellow dots on each page, I also asked students to think aloud
once I had ﬁnished reading the book to them. Continuing to process a text after reading is
a characteristic of expert adult readers (Pressley & Afﬂerbach, 1995). I wanted to see
whether second graders would also think aloud once the reading was complete. In my
past verbal protocol work (Hilden 2006) I found that second graders only infrequently
thought aloud once they had ﬁnished reading.

The verbal protocols lasted between 9 min 14 s and 21 min 42 s (excluding time
taken for directions and training). On average the verbal protocols lasted 11 min 46 s for
Dragonﬂies and 14 min 26 s for Salmon. (The greater length of protocols may be partly
explained by the greater number of pages in the Salmon book. Salmon is 19% longer than
Dragonﬂies in text length, and results in 23% longer verbal protocols.)

The verbal protocols were audio-taped for later transcription. I also took notes
regarding students’ visual attention and reading paths (e. g., ﬂipping between pages) on
black and white photocopies of the books during the verbal protocol collection. I then.
used these notes to add details to the transcripts as I transcribed from the digital
recordings. (Donovan & Smolkin have used a similar data source in previous work;
2006)

Materials

Students were read either Dragonﬂies or Salmon. These two texts were designed
so that they have parallel content and structure. They are both about animals that are
likely to be somewhat familiar to most US. students. They both have sections on the life
cycle of the focus animal. These informational texts were closely modeled after published

trade books in order to maintain their authenticity. However, the books were originally

66

designed for the COCA assessment. As such, these books are not like existing trade
books in a few ways. First, the text at the bottom of each page has been replaced with
blank lines. However, some text still exists in the book in the form of text features (e.g.,
labels, headers, diagrams, and the index). When introducing the book, I ask students if
they notice anything different about the book compared to others they have read, as a way
to point out that the text has been replaced by lines. Second, some of the pages have
places where the graphics are arranged in such a way as to be at odds with the text. For
example, in Dragonﬂies, there is a page where there are three illustrations that depict
environments of dragonﬂies: forests, deserts, and mountains. The corresponding labels
are mixed up. In the COCA, the students are asked if they notice a mistake on these
pages. Prior to starting the verbal protocols, I cue the students that there might be
mistakes in the book, “in this book there may be a few pages where the authors or
illustrator made mistakes. For example, the picture may not match what the words say on
the page. If you see a mistake, be sure to let me know.”
Analysis Procedures

I began data analysis by analyzing the verbal protocols using a modiﬁed grounded
theory analysis (Strauss & Corbin, 1990). This resulted in an inventory of processes that
the second graders in this sample used to construct meaning from the informational texts.
I began with some pre-formed categories from the COCA assessment (Billman, Duke,
Hilden, et. al, 2008). These pre—formed categories included: comprehension strategies
(predicting, summarizing, connecting to background knowledge, and inferring),
vocabulary knowledge and strategies, text feature knowledge, and processing graphics.

Also, when I came across processes that I had trouble labeling, I consulted reading

67

comprehension research that had labeled similar processes in the past. For example, I
noticed a subtle difference between when students asked questions about the content in
the book versus when they asked questions about the way the author or illustrator crafted
the text or illustration. Inﬂuenced by the instructional comprehension strategy,
questioning the author (Beck, McKeown, Sandora, Kucan, & Worthy, 1996), I decided to
distinguish between these processes. When consulting the reading comprehension
research did not help with labeling processes, I came up with descriptive labels on my
own (e. g., narrativizing, and anthropomorphizing).

Revising the inventory of processes was a recursive procedure that continued until
no new categories emerged from the data, category redundancy had been eliminated, and
I was satisﬁed that the inventory successfully captured the reading comprehension
processes in the verbal protocols (Strauss & Corbin, 1990). Once I was satisﬁed with the
inventory I had an expert in the ﬁeld audit it to ensure that the inventory adequately
represented the verbal protocol data. Several small changes in the description of the
individual processes resulted from the audit. The ﬁnal inventory consists of 23 processes.
The inventory of processes includes a description and examples of each of the 23
processes. This inventory can be found in Table 4.

Next, I applied the coding processes to each verbal protocol. These codes
reﬂected the processes inferred from students’ verbalizations. When coding processes, I
only coded a process once per page. Therefore, if a student asked multiple questions on
one page, I only coded one questioning process. This did not preclude multiple processes
for one page. For instance, I coded the following exchange as both inferring and

intratextuality.

68

Student: The eagles and bears like to eat salmon just like the other page. It
said that salmon are smaller than bears and bears are bigger so the bears
will eat the salmon.

Researcher: You said, “just like the other page.” What other page were

you talking about? Can you show me?

S: Sure. (Flips to page 10). See on this page that animal is bigger than that,

and that, and that [points to minnows]. And those animals are smaller.

And if it is smaller, if the animal is bigger it will eat it. And the bear is

bigger than the salmon.

Coding occurred at the macrolevel. For example, when a student
interpreted the index, I coded the comment as text feature knowledge, not index.
One exception to this macro-coding was monitoring. I broke this process down
into three subprocesses (monitoring—text monitoring, monitoring—knowledge
monitoring, and monitoring—rereading). Another exception was when I coded for
intratextuality. This process happened when students connected pages in the
book. Intratextuality falls under the larger process of processing graphics.
However, intratextuality is not directly measured by the COCA, so I kept the two
processes separate in the analysis. Finally, there were many occasions when
students ineffectively attempted to use a process. I still coded these as the
processes the student was attempting. (The complete coding manual is available
upon request).

Interrater reliability was estimated on ten protocols that were randomly selected.

To calculate IRR for the processes, I ﬁrst calculated the percentage of processes that we

69

agreed were present (misses occurred when only one rater coded a process). This form of
interrater reliability reached 90.2%. Then, of those times when we agreed that a process
was present, I calculated the percentage of times when we agreed on the label for the
process. Interrater reliability for labeling the processes was 88.3. Total interrater
reliability for agreement for process labeling was 77.7% (whether the two raters correctly
agreed on a process’s label).

Once I had created an inventory of processes and coded the verbal protocols, I
designed a rubric for the proﬁles and then assigned verbal protocols to these categories.
When coding the verbal protocols, I quickly noticed that students seemed to be going
about creating meaning from text differently. Some students seemed very active and
resembled the “constructively responsive readers” that Pressley and Afﬂerbach describe
(1995). Others seemed to talk a lot, but there was not much substance to their comments.
In contrast, other students seemed to not have much to say at all. As I coded the
protocols, I kept notes on my reactions to what made each of these readers stand out.
Once the coding was complete, I reviewed my notes and developed categories based on
the number of processes used, the range of processes used, and the nature of processes
used. I revised the categories until the proﬁles were mutually exclusive (that a student’s
verbal protocol did not meet the criteria for more than one proﬁle) and saturation was
reached (all of the readers/verbal protocols were accounted for by the proﬁles).

The ﬁnalized rubric consists of six distinct proﬁles: Active Processors, Active
Processors with Foundation Process(es), Single Processors, Active and Superﬁcial
Processors, Single Superﬁcial Processors, and Passive Processors. In order to reach a

single process threshold or superﬁcial processor threshold (seven total irrelevant or

70

repeating comments), a student must report a process seven times. Seven processes seem
to be an appropriate number for these proﬁles, which represents between quarter and a
third of the cued speaking opportunities (with this equaling the number of pages plus the
post cue). It makes sense then, that using a process seven or more times would
signiﬁcantly affect the meaning construction of that book. Similarly, seven seems to be
the appropriate number for establishing the number of different strategies required for the
Active Processors proﬁle as it represents roughly a third of the 23 total strategies in the
inventory. The number of total processes needed to reach the Active Processor proﬁle is
simply the number of pages in the book plus one (opportunity to make a comment once
the book was ﬁnished). It seems reasonable that actively processing students should
average at least one process per page. This proﬁle does not require a student to speak on
every page, rather students should average at least one process per page.

Once I was satisﬁed that these proﬁles adequately captured the different
proﬁles in my data, I had an expert in informational reading comprehension audit
the rubric to ensure that the proﬁles captured the data and were mutually
exclusive. See Table 12 for descriptions and criteria for the six proﬁles.

Results

This study demonstrates that second graders are capable of verbalizing their
thinking and feelings when being read an informational text. Their comprehension
processes ranged from the highly sophisticated to the superﬁcial, with distinct patterns or
proﬁles of reading processing for different readers. The following section is dedicated to

further describing these different informational reading proﬁles. Table 13 consists of a

71

frequency count and descriptive statistics for the proﬁles. In order to better capture the
proﬁles, I will also provide an example of a student who ﬁt each proﬁle.
Active Processors

The students who qualiﬁed for this proﬁle (along with the Active Processors with
Foundation Process(es)) were the most active as measured by the total number of
processes. However, they did not rely heavily on any one process, rather they ﬂexibly
used a wide variety of processes. This active and ﬂexible application of processes is a
hallmark of effective readers in adult verbal protocol literature (see Pressley &
Afﬂerbach, 1995 for a summary). Additionally, while not a speciﬁed criteria for the
proﬁle, all of the active processors except for one also connected to their background

knowledge.

Ben1 was an Active Processor who was read Dragonﬂies. I chose to discuss
Ben’s verbal protocol because he represents a typical Active Processor. Ben’s verbal
protocol resulted in 29 total processes, of which twelve were different. Ben’s ﬁrst
comment is on the ﬁrst page where she makes a prediction, “Um, I’m feeling like the
story is giving an idea about like a life cycle or something like a dragonﬂy.” He goes on
to apply other comprehension strategies as well as report affective reactions, “I’m feeling
that like, kinda good because I’m learning more about them [dragonﬂies]” (double-coded
as knowledge monitoring and affective reaction). When I asked him what made him say
that on that particular page, he went onto summarize what he had learned thus far in the
book. A deﬁning characteristic of Active Processors is that they frequently will apply

more than one strategy per page and that these processes will be used in tandem together.

 

1 . . .
All of the students’ names have been changed to protect the particrpants’ privacy.

72

Another example of this is when the student looked at a diagram of a damselﬂy and made
the following comment:

I sorta knew it because in class we studied insects and I knew it because

different parts of the body. And so this is their head and this is their thorax

and this is their abdomen [student points to different parts of the body as

she reads the labels], so different parts of their body.

This comment was double coded as prior knowledge and text feature knowledge because
he read the labels associated with different parts of the dragonﬂy’s body.
Active Processors with a Foundation Process(es)

These students met all of the same criteria as the Active Processors. However,
these students were different in that they relied heavily on at least one process. Eight of
the eleven students relied heavily on a single process. These processes included,
processing graphics, restating, monitoring, prior knowledge, inferring, and questioning.
The remaining three students relied heavily on two processes, processing graphics and
either restating, monitoring, or questioning. By far the most common foundation process
was processing graphics. Of the 11 students who qualiﬁed for this proﬁle, six reported
using processing graphics a minimum of seven times. This is perhaps not surprising
given that processing graphics was the most commonly reported strategy, with 156
occurrences. To give a frame of reference, the next most reported process was prediction
with 82 occurrences. This just highlights how crucial illustrations were to how students
constructed meaning from the text.

Anna was an Active Processor with Foundation Process(es) who was read

Salmon. She reported 39 processes, of which 12 were different. She relied heavily

73

on questioning, which accounted for 14 of the total processes. Often these
questions took the form of “I wonder” statements. Again, Anna was typical of her
proﬁle. Similar to the Active Processors, Anna’s reports on a single page
frequently reﬂected more than one process. For example, on a page that has a
diagram depicting the life cycle of Salmon, Anna commented:
I wonder why they die and why. . .and why they, and I wonder why they
die. And um. . .and.. and I um. There is going to be a pattern because they
are eggs, and then they will hatch and have the yolk, then they turn into
babies and then they are like almost like an adult, and then they are an
adult and they turn red and then at the top [refers to picture at 12 o’clock,
during the entire comment she follows the pictures with her ﬁnger in a
clockwise position].
On this page Anna starts by self-questioning. Then after a pause during which she
collects her thoughts she makes a prediction that there will be a pattern to the life cycle.
Anna conﬁrms her prediction on the next page.
Researcher: You’re smiling. Why are you smiling?
Student: Because I said there was like going to pattern that it was going to
start again and that’s it!
This was rare, as students did not often conﬁrm predictions or answer their
questions. Similar to the Active Processors, Anna monitors her comprehension
and growing knowledge of salmon. Towards the end of the book she comments,
“I never knew. I only thought there was one type of salmon. I didn’t know there

were all of these types.”

74

Single Processors

Unlike Active Processors, Single Processors are not as active or ﬂexible in how
they construct meaning from text. Typically these students report fewer total processes
and a more restricted range of different processes. On average, Active Processors and
Active Processors with Foundation Process(es) reported 30.7 and 36.5 processes and
used 11.7 and 11.8 and different processes respectively. In comparison, Single
Processors on average reported 22.2 processes and used just 8.0 different processes.
Another rough estimate of activity is the number of pages on which students did not
make any comments. The Active Processors and the Active Processors with Foundation
Process(es) on average only skipped 3.09 pages. In contrast, the Single Processors
skipped over twice as many pages, 7.6 pages on average. Like the Active Processors with
Foundation Process(es), the verbal protocols of Single Processors are dominated by a
single process. These single processes included, depending on the reader, predicting,
questioning, and connecting to prior knowledge. Interestingly, these processes are all
comprehension strategies and are frequently emphasized in teachers’ reading
comprehension instruction (e.g., see teacher guides such as Keene & Zimmermann, 1997;
Miller, 2002). These processes seem to play an even more important role for the Single
Processors than the Active Processors because the meaning construction is largely based
on a single process.

I read Dragonﬂies to Nadia, a Single Processor. Of the 18 pages on which Nadia
commented, she made predictions on seven of them. She made a range of predictions. For
example, she used the Table of Contents to predict what the book will be about, “I think

this is going to be a lot of different stuff like stages of a dragonﬂy [reading heading] and

75

there are going to be a place where it is going to be about the eggs [reading heading].”
She also seemed to understand the informational genre when she predicted that, “I’m
thinking that it’s going to um, tell more facts about the nymphs and um, that’s all
Interestingly, Nadia did not conﬁrm or revisit her predictions. It is important to note that
while Nadia largely relied on prediction, she used a few other processes such as
summarizing, processing graphics, inferring, and demonstrating text feature knowledge.
Active and Superﬁcial Processors

The Active Processors and Superﬁcial Processors proﬁle sounds like an
oxymoron but is a real proﬁle found for one child in this study. These students meet all of
the criteria for the Active Processors proﬁle. They have a high rate of activity and also
demonstrate a wide range of processes. They also show some evidence of monitoring
their comprehension, as effective comprehenders do. However, their high activity rate is
due in part to numerous irrelevant and/or repeating comments. While only one student in
this sample met this proﬁle, I am suspect that this is a proﬁle that many teachers could
identify with. These are students who have a lot to say about what they are reading, but
not all of it is likely to improve their comprehension in the long term. These students are
learning how to be active thinkers when reading, but they have not yet learned how to do
it selectively in a way that will beneﬁt their comprehension.

Jude was an Active and Superﬁcial Processor who was read Salmon. While Jude
qualiﬁed for the Active Processor proﬁle by making a total of 27 processes and using 12
different processes, he also made eight irrelevant comments. For example, after reading a
page packed with information about alevins, baby salmon, the student simply replied, “I

think they are cute.” Similarly, after reading about how salmon are full grown when they

76

are about three years old, Jude looked at me and said, “Wet ﬁshies.” This is not to say
that Jude was incapable of deeper thinking. For example, when reading about how young
salmon eat zooplankton Jude made the following connection, “I know. On Sponge Bob
Square Pants they are called plankton. That’s what these are [points to zooplankton
label].” Some of the comments that were not coded as irrelevant also seemed superﬁcial
in nature. For example, after Jude identiﬁed a graphic that labeled the parts of a salmon
as a diagram I used a follow-up prompt to which Jude replied, “Just diagrams.” While
this statement was coded as processing graphics, it seems more superﬁcial compared to
other students who interpreted the diagram by discussing the parts of a salmon and made
connections to their prior knowledge on this page.

Single Superﬁcial Processors

Single Superﬁcial Processors are those who meet the criteria for single processor
but also have a high rate of irrelevant comments and/or simply repeat the text. This
proﬁle is dominated by a single process and superﬁcial comments.

Cindy met the criteria for the Single Superﬁcial Processor proﬁle and was read
Salmon. Of the 26 total processes coded, only ﬁve different processes were represented.
She made an overwhelming 11 narrativizing comments and 8 irrelevant comments. (It is
important to note that other single processes are theoretically possible.) Combined these
represent 73% of her reported processes. Cindy is an English Language Learner, however
her protocol was one of the longest. Her tie to the text was tenuous at best. In order to
construct meaning to the text, she told a parallel story about going ﬁshing with her
family. For example, towards the beginning of the book the authors discuss how salmon

hunt for minnows when they are about one year old. Cindy’s response was:

77

Student: Um, my dad caught one of these before but except it was caught

on a tree.

Researcher: It was what?

Student: My brother caught one of that. And then after he said it’s not, it’s

not good. Because once we buyed one from the store, one like that from

the store and it wasn’t good. So when we went ﬁshing and my brother

caught one of it and he said it was not good so we threw it back in the

water.
Some of her responses were tied to the illustrations. For example, there is a
picture that depicts a salmon on its back as it makes a gravel nest for eggs. Cindy
commented that, “The ﬁsh that my brother got, the people threw it, and the other
people came, um, they ﬁshed it and then they took it home and then they ate it
and it’s not good.” Several other students also commented that it looked like the
illustration showed a ﬁsh that had been thrown into the water. However, they did
not incorporate this observation into an ongoing story. Overall, Cindy’s comments
were very picture-based and largely ignored the written text.
Passive Processors

Passive Processors are those students who show a low rate of activity as
measured by the total number of processes reported. On average, Passive Processors
reported just 18.1 processes for 21 (Dragonﬂies) to 25 (Salmon) pages of text. While
their range in processes used is narrower than the Active Processors proﬁle, they are still
demonstrating that they have numerous different processes in their repertoire, 8.4

different processes on average. Of the nine students who met the criteria for this proﬁle,

78

only three students reported fewer than eight different processes. This demonstrates that
these students can engage in a range of different processes, they just rarely do so.
Additionally, all but one student made at least one monitoring comment. Similar to the
Single Processors, these students were more likely to skip thinking aloud to pages than
their Active Processors. On average the Passive Processors failed to think aloud on 7.2
pages compared to the 3.1 pages skipped by the Active Processors. However, unlike the
Single Processors, the Passive Processors do not seem to rely on any particular strategy.

Rebecca was a Passive Processor who was read Dragonﬂies. Rebecca reported 17
processes, of which seven were different. Rebecca inferred several times throughout the
course of the book. When we read about baby dragonﬂies, or nymphs, she concluded
that, “That you can’t see dragonﬂies when they are just babies.” She also showed
evidence of processing graphics. On the diagram that compares dragonﬂies to
darnselﬂies she pointed out that, “their tails are different when they are babies. This one
has a tail like a ﬁsh [points to damselﬂy] and this one has a pointy tail [points to
dragonﬂy].” She also noticed that the labels for the dragonﬂies’ environments are
reversed. Aside from these glimpses of processing, Rebecca was relatively non-active.
Like many of the Passive Processors, Rebecca’s verbal protocol is characterized at most
by one process per page. Of the 21 pages in Dragonﬂies, Rebecca only used multiple
processes on four pages. Rebecca chose not to think aloud on eight of the pages. While
she is not a completely passive reader, her pattern of activity is relatively low and
inconsistent.

Discussion

Instructional Implications

79

Differentiated instruction is only possible when a teacher has a clear idea of her
students’ strengths and weaknesses. Developing proﬁles of reading comprehension of
informational text has the possibility of providing teachers with valuable information that
could improve the way they meet their students’ instructional needs. Below, I have
brieﬂy outlined some questions that one might consider when interpreting the reading
proﬁles. I also propose some instructional implications for the different proﬁles.
However, it should be noted that these proﬁles of reading are only based on one verbal
protocol of one text. I strongly encourage teachers to examine students’ patterns of
processing across texts before labeling a student as having a particular proﬁle.

Active Processors. These students were active, ﬂexible, and independent when
applying comprehension processes to the informational texts in this study. Because the
comprehension processing of these students was arguably the strongest in this study,
teachers might be less concerned about the comprehension processing of these students.
However, teachers of Active Processors need to make sure that these students are active
across a variety of genres, topics, and purposes for reading. Also, because the individual
processes in this study are not sensitive to differences in quality, teachers need to do
some additional investigating to make sure that these students are using the
comprehension processes effectively and efﬁciently. For example, when a student makes
a prediction, does she check on that prediction later on in the text? Similarly, is there a
qualitative difference between a student who names a graphic as a “diagram” and a
student who interprets that diagram to summarize the life-cycle of an animal?

Active Processors with Foundation Process(es). These students are also active in

how they are constructing meaning from the informational texts in this study. However,

80

they rely heavily on a single process as they construct meaning. Additional research is
needed to ﬁrrther investigate this proﬁle. It may be this additional active proﬁle leads to
similar comprehension outcomes as the Active Processors proﬁle. It may not matter that
students rely on one process, as long as they have an overall high activity rate, report a
wide variety of processes, and show some indication of monitoring. However, if this
proﬁle leads to poorer comprehension outcomes, then different instructional implications
may be warranted.

Single Processor. These students tend to use fewer processes, both in range and in
total, compared to their Active Processor peers. However, similar to the Active
Processors with Foundation Process(es), when they are actively constructing meaning
from the text, they rely heavily on a single process. These students would likely beneﬁt
from learning about, and practicing additional strategies. They have shown that they can
actively process an informational text, they may just need additional scaffolding to apply
a wider variety of processes on a more consistent basis.

Superﬁcial and Active and Superﬁcial Processors. These students are making a
signiﬁcant number of irrelevant comments and/or are just repeating what the text says.
For those students who report many irrelevant comments, it is important to get them to
monitor their own comprehension. Teachers might teach these students to ask
themselves, “How does that help me understand the text better?” whenever they report a
process in order to help them become more text-focused. Teachers need to help these
students realize that being an active thinker is not enough when reading. Readers also
need to apply processes effectively. Those students who are simply repeating the text

need to be taught how to internalize the text. Teachers should question whether the

81

students who repeat frequently have much prior knowledge on the text topic. It may be
that a lack of prior knowledge or other challenge is interfering with the meaning-making
process. If the students do have prior knowledge but fail to activate it, teachers should
work with them on making strong connections between the text and their prior
knowledge.

Single Superﬁcial Processor. Earlier, I discussed Cindy’s verbal protocol. She
made an overwhelming number of narrativizing and irrelevant comments. In some ways,
Cindy resembles Wade’s Schema Imposer proﬁle (1990). The Schema Imposer is a
reader who “holds on to an initial hypothesis despite incoming information that conﬂicts
with that schema” (p. 118, 1990). In Cindy’s case, when we began reading about Salmon,
Cindy made the connection to a time when she and her Dad went ﬁshing. However, she
continues to attempt to ﬁt this story to the Salmon text, even to the point of completely
ignoring the text (hence the high number of irrelevant comments). While Cindy seems to
understand that comprehension is an active process, the comprehension processes are
tenuously related to the actual text. The way that Cindy processes Salmon is also
reminiscent of Williams’ studies (1993, 1998) of adolescent readers with reading
disabilities. Similar to Cindy, Williams found that the comprehension of these students
was frequently peppered with tangential background connections. These idiosyncratic
background connections also seemed to hamper students’ abilities to make accurate
predictions and identify the themes of narratives. Cindy’s verbal protocol was almost
completely devoid of any comprehension strategy use. Interestingly, Cindy’s COCA

score was also very low. This is an indication that it is not simply a matter of Cindy not

82

being self-regulated in her use of comprehension processes. Rather, her processing issues
seem to be more pervasive and signiﬁcant.

Students like Cindy would likely beneﬁt from guided comprehension instruction
that begins with how to effectively connect to background knowledge. Miller (2002)
encourages her primary grade students to distinguish between connections that help the
reader understand just a “little” part of the text versus “big” connections that help the
reader connect to important information in a text. Being able to make well-grounded
connections will likely assist Cindy (and students like her) as she moves onto additional
comprehension strategies.

Passive Processors. Students who fall under the Passive Processor proﬁle are
characterized by a relatively low activity rate when constructing meaning from
informational texts. This proﬁle seems related to what Buly and Valencia (2002, 2004)
deem as “word callers” In their sample, 18% of students fell into this proﬁle. Word
callers are students who can read accurately and quickly but who fail to read for meaning.
In other words, the words wash over them passively. (It should be noted that we do not
know the decoding abilities of the students in this study. Therefore, it is possible that
these Passive Processors may look more like what Buly and Valencia call “disabled
readers” because they also have decoding issues along with comprehension issues.
However, for the current study, these second graders fell into the Passive Processor
proﬁle in the absence of decoding or ﬂuency demands.)

It would be interesting to compare the Passive Processors performance on the
COCA to their verbal protocol performance. Compared to verbal protocols, which are

very nondirective, the COCA directs students to apply different comprehension processes

83

on each page (Billman, Duke, Hilden, et. al, 2008). If a Passive Processor scored high on
the COCA, a teacher might conclude that the student’s passive performance on the verbal
protocols may have resulted from a lack of self-regulated strategy use since the student
processing was high when given directed prompts on the COCA. In this case, the teacher
would want to encourage these students to self-regulate their comprehension processes
when reading. For example, a teacher could ask a Passive Processor to mark places
where he is actively thinking about what he is reading. This teacher could later
conference with the student on his independent processing. In comparison, if the student
scored low on the COCA in addition to being qualiﬁed as a Passive Proﬁle, this would be
indicative of more serious comprehension issues. Such students fail to be Active
Processors (or are highly ineffective in their processing) even when given speciﬁc
prompts. Teachers need to make sure that these students realize that “good reading” goes
beyond reading ﬂuently, but that “good readers” are thoughtful readers.

Passive Processors may beneﬁt from some intensive strategy and process
instruction. These students may need to see additional modeling of what active
processing looks like. Direct explanation of how the processes and strategies work may
also be beneﬁcial to these students. They need to know when, where, and how to use
these strategies and processes effectively (Duke & Pearson, 2002). Finally, these students
need much guided practice in order to learn how to implement these processes when
reading. If thinking in terms of the gradual release of responsibility model, the teacher is
likely to have to carry the burden of the strategies or processes at ﬁrst when working with

these students (Pearson & Gallagher, 1983).

84

Interestingly, Buly and Valencia found that 60% of the Word Callers in their
investigation were English Language Learners (2002). None of the three ELL students in
this study fell into the Passive Processor proﬁle. Instead, two of the three ELL students
were Active Processors with Foundational Process(es) and the last was a Single
Superﬁcial Processor. Additional verbal protocol work should be done in order to
investigate how comprehension processing is different for ELL students as compared to
native English-speaking students; especially in the early grades when comprehension is
developing. There has been a history of verbal protocol work that compares reading in a
native language as opposed to reading in a second language; however this work has
focused on older readers and not on informational texts (Davis & Bistodeau, 1993;
Jimenez, Garcia, & Pearson, 1996).

Limitations &. Future Directions

The need for additional study to examine relationships between verbal protocols
(including proﬁles) and comprehension outcome measures is particularly great. Pressley
and Afﬂerbach (1995) identiﬁed this as a high-need area within verbal protocol work.
They argued that correlating verbal protocols with outcome measures would provide a
way to validate the verbal protocol ﬁndings. Unfortunately, they also found that very few
researchers have linked verbal protocols to comprehension outcome measures. Meyers
and colleagues (1990) represent one exception to this rule. They established positive and
negative between categories of processes for fourth and ﬁfth grade students’ think—alouds
and their passage comprehension. For example, they found negative correlations between
passage comprehension and students’ inaccurate paraphrasing. They found strong

positive correlations between inferring and revising of previous thinking and passage

85

comprehension. In contrast, Wade, Trathen, and Schraw (1990) used cluster analysis to
identify ﬁve comprehension proﬁles. However, these proﬁles were not signiﬁcantly
associated with memory for text. However, Pressley and Afﬂerbach (1997) advise that
we cannot conclude from this that Wade, Trathen, and Schraw’s attempt at validation
failed. Instead, they site the small sample size in the “good comprehender” proﬁle led to
small statistical power that made it all but impossible to detect anything than very large
effects. This issue represents an interesting conundrum in verbal protocol research. The
vast majority of verbal protocol work is done with very small sample sizes. In fact, Wade
and colleagues study was somewhat an anomaly in this way. Their study had 67
participants, which is very large for verbal protocol research. The majority of verbal
protocol work is done with fewer than 30 participants (Pressley & Afﬂerbach, 1995). For
future related research, a cluster analysis could be performed on a large number of verbal
protocols for younger readers with the proﬁles from this study. Also, future research
needs to investigate methodologies and analyses that would make correlating verbal
protocols and outcomes easier. For example, one potentially fruitful line of future
research may lie in correlating verbal protocol results with a wider range of
comprehension outcomes. Relationships with the ability to answer inferential and
critical/evaluation questions, for example, could be explored.

The ability to examine the relationship between verbal protocol proﬁles and
comprehension outcomes is important for several reasons. First, we need to know that
these proﬁles are consequential if we want teachers to attempt to alter their instruction
based on their students’ proﬁles. While we might assume that Active Processors would

have better comprehension outcomes than Single Processors or Superficial Processors,

86

this is an empirical question that is not addressed in the present study. Second, if the
proﬁles are validated with respect to outcome scores, it would then make sense to
examine the proﬁles in relation to different groups of students. For example, are readers
with learning disabilities or are English Language Learners likely to have certain
proﬁles? Finally, we need to know whether these proﬁles are associated with reading
comprehension development. For example, is it the case that students usually start off as
Passive Processors, then progress to the Single Processor proﬁle before becoming Active
Processors? These are all future directions for research.

The texts used in the study may be interpreted as a limitation. The alterations to
the COCA texts (and related directions prior to the think-aloud) may have unduly
inﬂuenced the verbal protocols. There was likely more text monitoring than there would
have been without “mistakes.” Also, because the books were read aloud to the students, it
is unclear from this study how decoding abilities and the comprehension processes would
interact. For example, it would be interesting to see how decoding issues for struggling
readers would interact with how they process the text. This brings us to a wider issue in
the ﬁeld. Reading comprehension is such a dynamic and complex process that depends
on various reader and text-related factors. The books used for this study were
informational texts that focused on the life-cycle of animals (life sciences). Future verbal
protocol studies could see how these children would process informational texts on other
science topics such as the solar system (physical sciences), or how they would process
pseudonarratives on these animals. In other words, future research could compare how
children process different genres and topics. Similarly, research could compare how

students’ processes are affected by their motivation, prior knowledge, and other

87

individual factors. Finally, processing proﬁles of students in this study were based on one
verbal protocol. Future research should examine the generalizability of the proﬁles to
other texts. No one study can begin to capture the complexity of reading comprehension
development. Rather, only by using a variety of methodologies will we begin to more
fully understand how various reader and text factors impact how reading comprehension
develops.

One limitation of the current study is the small sample size of 30 second graders.
It is possible that additional verbal protocols may have revealed other reading proﬁles.
While the present sample did not include any students that ﬁt this proﬁle, it is
theoretically possible. This proﬁle would include verbal protocols that include at least
seven irrelevant and repeating comments, and would fail to meet the Active Reading
Proﬁle In this proﬁle, these students would fail to be active and ﬂexible in their processes
they report. The comments would not likely be beneﬁcial to their comprehension of the
text. These verbal protocols would rely heavily on repeating and irrelevant comments.
Superﬁcial Processors who repeat the text frequently resemble Wade’s Non-risk Taker
(1990). She describes such a student as one who “assumes a passive role by failing to go
beyond the text to develop hypotheses.” However, the Superﬁcial Processor would also
make irrelevant comments that are not tied in any way to the text.

More research is needed to investigate individual strategy and process
development. Throughout this study, I noticed many occasions in which students seem to
be applying a process inefﬁciently. In memory research, Waters (2000) states that
children can be inefﬁcient in their strategy use by applying a strategy in a simpler or less

effective form than their older peers. She then provides several examples of research in

88

which children have inefﬁciently applied memory strategies. From this verbal protocol
study and my past work (2006), I see a similar trend with comprehension strategies.
Perhaps it is not surprising that in second grade, children are learning about how to use
new comprehension strategies. However, not all children automatically apply them
effectively at ﬁrst. Several questions emerge. First, can teachers effectively and reliably
distinguish between efﬁcient and inefﬁcient strategies? Is it the case that students’
comprehension processes naturally become more efﬁcient over time through practice? Do
students beneﬁt from tailored instruction geared towards improving the efﬁciency of the
processes? If so, what would such instruction look like? In this case, it looks like basic
and applied research could greatly inform one another.

Another future direction for research in reading comprehension development is to
examine whether it is (and how to make it) practical for teachers to use verbal protocols
as a means to identify proﬁles of children in their classes. From looking at the verbal
protocol research, the trend seems to be for researchers to collect verbal protocols from
students that they have never met. One exception to this trend is a group of studies that
involve teaching students to think-aloud as a strategy to improving their reading
comprehension (Baumann, Seifert-Kessell, & Jones, 1992; Oster, 2001; Schunk & Rice,
1985, Silven & Vauras, 1992). While, verbal protocol researchers go to great lengths to
minimize their inﬂuence, and make students comfortable, one possible research direction
would be to see if students thought aloud differently when their teachers were present
instead of unknown adults. Secondly, if teachers could effectively and reliably collect
verbal protocols, this would be a step toWards using verbal protocols as a classroom

assessment tool. Finally, if teachers were to collect verbal protocols they might have

89

unique insights into the proﬁles and instructional implications. At the very least, a future
step might involve sharing a range of verbal protocols with some effective second grade
teachers and have them identify the students’ strengths and weaknesses as well as discuss

instructional implications for working with these students.

90

References

Afﬂerbach, P. (2000). Verbal reports and protocol analysis. In M. L. Kamil, P. B.
Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol.
3, pp. 163-179). Mahwah, NJ: Lawrence Erlbaum Associates.

Afﬂerbach, P., & Johnston, P. (1984). Research methodology: On the use of verbal
reports in reading research. Journal of Reading Behavior, 16, 307-322.

Alexander, P. A. (1997). Knowledge seeking and self-schema: A case for the
motivational dimensions of exposition. Educational Psychologist, 32, 83-94

Alverrnann, D. E. (1984). Second graders’ strategic preferences while reading basal
stories. Journal of Educational Research, 77, 184-189.

Baumann, J. F., Seifert-Kessell, N., & Jones, L. A. (1992). Effect of think-aloud
instruction on elementary students’ comprehension monitoring abilities. Journal
of Reading Behavior, 24, 143-171.

Beck, I. L., McKeown, M. G., Sandora, C., Kucan, L., & Worthy, J. (1996). Questioning
the author: A yearlong classroom implementation to engage students in text.
Elementary School Journal, 96, 385-414.

Bereiter, C., & Bird, M. (1985). Use of thinking aloud in identiﬁcation and teaching of
reading comprehension strategies. Cognition and Instruction, 2, 131-156.

Billman, A. K., Duke, N. K., Hilden, K., Zhang, S., Roberts, K., Halladay, J. L., et. al.
(2007). Concepts of Comprehension Assessment (COCA). East Lansing, MI:
Trustees of Michigan State University.

Brown, R., Pressley, M., Van Meter, P., & Schuder, T. (1996). A quasi-experimental
validation of Transactional Strategy Instruction with low achieving second- grade
readers. Journal of Education, 88, 18-37.

Buly, M. R. & Valencia, S. W. (2004). Behind test scores: What struggling readers really
need. The Reading Teacher, 5 7, 520-531.

Buly, M. R. & Valencia, S. W. (2002). Below the bar: Proﬁles of students who fail state
reading assessments. Educational Evaluation and Policy Analysis, 24, 219-239.

Cain, K., & Oakhill, J. (2007). Introduction to comprehension development. In K. Cain &
J. Oakhill (Eds.), Children 's comprehension problems in oral and written
language: A cognitive perspective (pp. 3-40). New York: Guilford.

91

Center for Educational Performance & Information. (2006-7). Pupil Headcount Data
(SRSD). Retrieved July 9, 2008, from http://michigan.gov/cepi/0,1607,7-113-
21423_30451_30460---,00.html

Davis, J. N. & Bistodeau, L. (1993). How do L1 and L2 reading differ? Evidence from
think aloud protocols. The Modern Language Journal, 7 7, 459-471.

Donovan, C. A., & Smolkin, L. B. (December, 2006). Children ’s selections and reading
of science trade books. Paper presented at the annual meeting of the National
Reading Conference, Los Angeles, CA.

Duke, N. K., & Pearson, P. D. (2002). Effective practices for developing reading
comprehension. In A. E. Farstrup & S. J. Samuels (Eds.), What research has to
say about reading instruction (3rd ed., pp. 205-242). Newark, DE: International
Reading Association.

Duke, N. K., Pressley, M., & Hilden, K. (2004). Difﬁculties with reading comprehension.
In C. A. Stone, E. R. Silliman, B. J. Ehren, & K. Apel (Eds.), Handbook of

language and literacy development and disorders (pp. 501-520). New York:
Guilford.

Ericcson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data.
Cambridge, MA: The MIT Press.

Hidi, S. E., & Hildyard, A. (1983). The comparison of oral and written productions in
two discourse types. Discourse Processes, 6, 91-105.

Hilden, K. (2006, December). Verbal protocols: A window into second graders ’ reading
comprehension of narrative texts. Paper presented at the annual meeting of the
National Reading Conference, Los Angeles, CA.

Jimenez, R. T., Garcia, G. E., & Pearson, P. D. (1996). The reading strategies of bilingual
Latina/o students who are successful English readers: Opportunities and obstacles.
Reading Research Quarterly, 31 , 90-112.

Keene, E., & Zimmermann, S. (1997). Mosaic of thought: Teaching comprehension in a
reader ’s workshop. Portsmouth, NH: Heinemann.

Kucan, L., & Beck, 1. L. (1997). Thinking aloud and reading comprehension research:
Inquiry.

instruction and social interaction. Review of Educational Research, 6 7, 217-299.

Langer, J. (1996). Reading, writing, and understanding: An analysis of the construction
of meaning. Written Communication, 3, 219-267.

92

Lundeberg, M. A. (1987). Metacognitive aspects of reading comprehension: Studying
understanding in legal case analysis. Reading Research Quarterly, 22, 407-432.

Meyers, J ., Lytle, S., Palladino, D., Devenpeck, G., & Green, M. (1990). Think-aloud
protocol analysis: An investigation of reading comprehension strategies in fourth-
and ﬁfth-grade students. Journal of Psychoeducational Assessment, 8, 112-127.

Miller, D. (2002). Reading with meaning: Teaching comprehension in the primary
grades. Portland, ME: Stenhouse Publishers.

National Center for Education Statistics. (2005-6). Search for schools, colleges, and
libraries Retrieved July 2, 2008, from http://nces.ed.gov/globallocator/

Nisbett, R. E. & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on
mental processes. Psychological Review, 84, 231-259.

Olshavsky, J. E. (1976—1977). Reading as problem solving: An investigation of
strategies. Reading Research Quarterly, 12, 654—764.

Oster, L. (2001). Using think-alouds for reading instruction. The Reading Teacher, 55,
64-69.

Pearson, P.D., & Gallagher, M. (1983). The instruction of reading comprehension.
Contemporary Educational Psychology, 8, 317-344.

Pressley, M., & Afﬂerbach, P. (1995). Verbal protocols of reading: The nature of
constructively responsive reading. Hillsdale, NJ : Lawrence Erlbaum.

RAND Reading Study Group. (2002). Reading for understanding: Towards and R & D
program in reading comprehension. Santa Monica, CA: Rand Education.

Rosenblatt, L. (1978). The reader, the text, the poem. Carbondale, IL: Southern Illinois
University Press.

Scardamalia, M., & Bereiter, C. (1983). Child as coinvestigator: Helping children gain
insight into their own mental processes. In S. G. Paris, G. M. Olson, & H. W.
Stevenson (Eds.), Learning and motivation in the classroom (pp. 61-82).
Hillsdale, NJ: Lawrence Erlbaum.

Schunk, D. H., & Rice, J. M. (1985). Verbalization of comprehension strategies: Effects
on children’s achievement outcomes. Human Learning, 4, 1-10.

Silven, M., & Vauras, M. (1992). Improving reading through thinking aloud. Learning
and Instruction, 2, 69-88.

93

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory
procedures and techniques. Newbury Park, CA: Sage.

U. S. Census Bureau. (2000). 2000 Census School District Tabulation. Retrieved on
September, 11, 2008 from http://nces.ed.gov/surveys/sdds/ref00.asp

U. S. Census Bureau. (2006). American Community Survey. Retrieved on September, 11,
2008 from http://nces.ed.gov/surveys/sdds/acs06/acs06ref.aspx

U. S. Department of Education, National Center for Education Statistics. (2005-6). The
common core of data (NCES 2007-354). Washington, DC: Author.

Wade, S. E. (1990). Using think alouds to assess comprehension. The Reading Teacher,
43, 442-51.

Wade, S. E., Trathen, W., Schraw, G. (1990). An analysis of spontaneous study
strategies. Reading Research Quarterly, 25, 147-166.

Waters, H. S. (2000). Memory strategy development: Do we need yet another deﬁciency?
Child Development, 71, 1004-1012.

Williams, J. P. (1993). Comprehension of students with and without learning disabilities:
Identiﬁcation of narrative themes and idiosyncratic text representations. Journal
of Educational Psychology, 85, 631-641.

Williams, J. P. (1998), Improving the comprehension of disabled readers. Annals of
Dyslexia, 48, 213-238.

Wyatt, D., Pressley, M., El-Dinary, P. B., & Stein, S. (1993). Comprehension strategies,
worth, credibility monitoring, and evaluation: Cold and hot cognition when

experts read professional articles that are important to them. Learning and
Individual Differences, 5, 49-72.

94

Table l

Breakdown of Ethnicity Percentages for Participating Schools

 

 

School White, Black Hispanic Other
non-
Hispanic
1 90.3 5.1 3.9 1.8
2 95.4 2.0 2.0 < 1
3 -- -- -- --
4 96.9 1.2 < 1 1.2
5 96.4 < 1 1.3 1.3

 

Note. These statistics are at the school level. Statistics for School 3 were not available.
However, this school is in the same district as School 5. (National Center for

Education Statistics, 2005-2006).

95

Table 2

District Demographics by Percentages and Locale Codes for Schools

 

 

Schools Free & Residents with Residents Children NCES
reduced Bachelor’s lower than below Locale
luncha degree or high school poverty Code
hi gherb educationb lineC
1 66 18 33 36 City: Small
2 23 27 4 8 Suburb: Large
4 5 34 7 6 Suburb: Large
3 21 20 6 6 Suburb: Large
5 16 20 6 6 Suburb: Large

 

Note. Demographic data is reported at the district level.

aFree & Reduced Lunch data is at the school level. (Center for Educational

Performance & Information, 2006-7)

bUS 2000 Census Bureau

cUS Census Bureau (2005-6)

96

Table 3

The Number of Questions per Construct for Salmon and Dragonﬂies in the COCA

 

 

Processing Text Comprehension
COCA Form Graphics Features Vocabulary Strategies
Salmon 4 2 9 4
Dragonﬂies 3 4 7 5

 

97

0.020 0

$8506. oxamcxonmammﬁds V0102 3.080000

 

0088mm

001.0338

 

00339038: mq08m000

60:80:05. 8 03.00 00:00:80.?” may—00:"
00:8on 8 0 2.0080 830 am; 0:00:04 360 00.
0:00:05. 0 82: 0m 3000. “0030:0— 3:500:00.
04 0:00:04 80: Q:800x€0:QV.

3.030003% 0:0. hxmmmnazm” £0003 0033030
0:05 2:0" 0:0 :00: :0: :0 0:05 on 2:0" :0:
:0000: :00: 0: ”:0 0:00:00. :8 368. 8:980
00 8308 0: 000:9 0800000? 04 08395 0
888 E52000: 00 300:8 07205 000:8-
00000 2:: hw00000§® $0005.03. 0033030
08 204000 00 2:0" 5:: :030: :92. 0a 2:0:
03.2.3000: 888 :0mmm $0: 00:80:.

bxmmozhmmu. 0. 5” ..H:o<.8 5:0 00. ER 9:83:00 2:0: 93.40 mam"
00:0 m:0000:m 0:000 mi: 00803 00:39.93 0080 0i 0». 0:500:08 0:0
00.0 01.88%...

0.0500. n. S $2.0 000300 0:00" Noon—00:83 0:00. CB. 3:8: 0.0
0:000 03:00.0 £08 0: mugma w0:. 00.0 0:00 0:30. 0:0 :30 0:03. 0:...

DEW0:\:.00. :. Tr :78 man: 83 00. 00308:: 368 0m 0008:0000. 0.8
80: 00:5 ":0" 8.0 mag? 0030 ":00 08 2:0 0:0 0030 ":00 08 80 0:0
0030 ":00 08 :80 0:0 80...

bxamozﬁmmm. u. 3
.5 80:50.0. 80 0». :005 :02 83. 0:02 :03 0:0 0:80— ”:00 83.8
00:0:m 0:05. :00. z :00 :03. 0010 0:0 0:5,...

bnghmmm. 0. 7 :CB. 3: 80::m 58 ":0 m801m m0<0:m 0: 0000 0:00:
58 0 :8 ova—o 0n 003000:m :08 0 90m0::<...

bSmoﬁﬁmm. :. m“ ..>:0 :02 0.8 000—005 0» 0m moEm 8 mo 05 0:0 3:0 0
0:08 0:0 0:3. :08 8 308 0:0 ":0: 003.40 ":0 0:00 0:0" :0: :08 ommm
0m00:...

98

0.020 A 803.0:

333.0%” .3000 0033030 0:0: ”0:0 30 3::
0m m00mm0m 0:00: 000333 008:0 :0: 03:003.
3080 3 30 :03 00 @0030. 30,033m £00 030:
0: 00033 8 006—03 2:0. 003033m :00003
$000000 00 0 :00. 2000 33 030: 00003:
301033 00: :0 80: on 300300330000.

“0550:03m” 00330:" 0030000 30 mi 00
303 0000 00. 0 00000: 0m 30 :00? 0033030
00:00:00 30.03000: $03 3000 30: 0:0
00:00:00 0m 8::

Dwamgxﬁmu. 0. 3
90003” 03:3 0 £00 0mg.

03000030: 2:00 00 v.00 3000..V
9000:: 0:00 30% 200.0 m03m 8 30: 0m03.

M00500. 0. N
00000030: 0 :00000 30: v.00 €000 W300 80:3m 0<0~ 0" 3mm :0m0.
€000 v.00 33:3m 03.33m $30: <0: 2000 80:3m 0: 3mm :0m0
3,30 :0m0v..0
3000:: C3. .. H33: 3000 000 :0: 0:0 30v~ 000 3:00.. .. 30030
3:00 023033m 8,200.00 30 0mmm. >00 30v. 000 m03m 8 00
003033m 8 303.

M0380. 0. m” :0 33: 30% 00". 30v. :03 030: :0: :000000 30% 00:.”
:03 30 Em :30? . 0.3 33:3m 30m 8 00” 3 30 033 Em mm: mm
m03m 8 00: 3..

0030003300. 0. No” 20:0 003005000 0:0 00200300 0:0 00300300 000
:30 30 0030. .203. 0030 0.83 0mmm 0:0 30: 30v. 0030 8 ES :30
:03000 0:0 30: 30% 30: 30:. 03: 2.0 0300 0:0 30: 30% :00030
0005...

99

.320 A 303.3.

©§3§§m§€za~mw§mu > 9.830: mcoﬁ Em
”9301:8530? 2H 2033.. mm m 8:55:
@338.

Qammzoizm Sm $382 ‘25 magma ﬁrm m
£830: £85 moBaEEm 9m ”:99. £38 3
E8323 982.

7338le
00382.8 89:. 23m: 3538 3253 En:
gaoamgamsm om 90 85 85 55532..

53 iozzolzm” 00339: 83on
8:?ng mﬁaBBEm $03 999. 9m 85 ca
55530:.

wm§§m§mn 0:? comma ﬁrm: maﬁa
323:8 3 9a manna on 08:38 Gang—mm
:6 £32: QE :0» :93 En 8x”

Rxoimawm Eczzelzm” 00352: 332
S Roomsmwom 3838 0». can: Kai—mama ca
253mg #03 @8305 9555 or

MRSQF u. 5” :18 Sosamlnm €320 a3 9a 222 we»:

bummozhmmm. u. on 3: Soaaaaam : :8 8.93 8955 552 So 395%
$8 o». :8 356:.

DBWQQNRM. w. a” :CB. _ 95x 92. : _ 95x 93 x mm ~35 :2 5
gamma Umomcmo _ man 58 9a ms: 05 man x 528 ES m @809 . .. EBB—d.
OE .25; co” m ammo: 9058 8 :5 39:8 ow Ea moqomc gm :5; :2 m
38$ @058 8 90 9803. 12.5 38% meg :08 Bi So @803 momm =08
anamomaam 93 :5 3958 2.0 83303...

:45 8332: mm mead—0-083 SE. hxonmahzm mwmbinﬁ

@6383 u. 3, :0»: v5: Run»: 3:

bxamozxmmm. u. m“ :— Ems.» 55$ 92 ammo—.9:

100

.588 A 802.8.

NV 88888 82 80 88 2. 508.28:
002.83 08305 8858.

.2800 008888 @0082? 828.0. .4 E84
80$: 8 888:8 8 00:82 8 80 .0008

mnncaazm @053” 00880:» 83006 02888
$88. 080:. 0a 8:0: 00:08. 3350 0088—0.
308% 8: 828m 208 :2 08002 8 80
2:35 8805.

30028880
@808 96.85% 0080:2828 8:088:28 0m
208 802—080 3 08888 80 8088m 0m m
(<08 82 mm :2 82.80: m: 80 8x2.

OW
map—:08 38 80 830.89% 88 8 88:. (<08
80::8m 0m 888:3: 2080.

CW
m8008 800888 82 80% 00 :2 802 80
80:88 2. m 80050 208.

$008,488 08038

308.38% 03:83.. SSS.” 008888 982
88m88 88828: 08580: 8 80 8x» 8:
$0030. 0:; 88882 80 £838 80:5 Va
880 8 88:00 2. 80 805. 8878 920 0a
808 8888 @8033. 2. :00 80 8x2 8

b‘0m0zhmmu. 0. 2w” .4 98w _ 802 228m :02 5008.8 78 can: a2:m
5mm 800050 _ E20." ER 885 .802 802 808 8: 08mm 8: _ 2mm .82
@083 88 808 8: 00 8mm G02» «3: «76 80 ES m 0882 8: 08m.
8: 312% 80 8:8 802 808...

Danmgmmmm. 0. on 8.8 80:5 €8mm. . 2.8 80:8 28%.. 8 8v~ :08. m
08 89 808 8 8% :08. GA“ «<32 :08 z 80— 588 .2500 80. ES 80%
8.0 858m 8% m:m0~m.s

@2503. 0. 3” J 802 82 80 880 mm 80 800N 80 80 88:8 mm 80 m5...

63.80.040.08. 0. 5” :8: _.8 80:5 82 2:08 83. 88:80 #08 mm
00:00 "028080 0: 80: 28080:...

@0503. 0. 3” 35:2 80m 8:285 8080.. A00:Ea-02_0: <38
0:028:83.

bxnmgxmmm. 00. 3 R 3” 2.200: 8:0 2.0 80 880 8: 80: 8mm :8
23.088 S30: 80% :8 288m. Ham 0:0 :8 w 8.: 58 0 mm: €086 8
808855 80 88 0:0 :8 m 008? 8: 6088 8 “Haggai...

101

H88 A 808.“;

88888 28 maﬁa?

388.88% Q8128. NEESREQSm

9:88 8&8m 888588 88 888883
8288 ummam. 3.88 88888 88 388 on
£88,888 8&3 @8308.

N83 5888.3. 38:8an

80:88 88888 808 88 828 ow 888:8.
88x. @8me. 88? 8888. 88 88859888
mcam. 3.8 8850 Q8 ow 88 88:8 mm :8
ooaoay

@6883 u. m
$888” $8 888; 8&8 888 8 88.
”888888 ’58 888; 8&8 8:8“.v
9:88” A5328 8 :8 8m. $888 8 1mg many 3.3.8 8% m8 8
88 88. >88 8 8mm 38 98% m8 8 :8 8888 £886 8 88 3% o:
88 .83.
”888888 >8 8: 8mm 98 :8 888; 8&8 888. $3 8884
98 8&8 88%
9:88” W888 z 88 r 88 o: 88 8888. :8 :8 8?
3.78 mm &8 888 um Bosxoasma

M888? 8. 3
8888” H8 8E8 88 888 E8 8 88 m&88 .88 E8 :8 889
Ema. r 88 88 m&88 m8 m8&_2 88 8&8 88 888 88 Emmﬂ. mo
:8 83m 2:— 88 98 888:.
”88888: <0: 88. :88 E8 88 09$ .88.: 9:8 088—. Ema
288 8: 855 828V 08 mo: 88% 8%
magnum mean. cum? 8 3mm :5. m8 0: 88 68mm. 88 888— mm
Eme 88 88. >88 88. 88 88. 88 :88 88»? m8 m8&_8~. >8
mm x mm 88:2. Wm 88 888 mm .8me 8 SE 88 z 88 :8 88.5 Eme
98 88 888.

b88883 0885” J 88* 98 :8 882 8: 88 @8888 8 mo 88
mm 98 889. :88 888 22% :8 8888 E84 88888 88% 8:5
80w 3% 88 88 2:8 8 838...

98,838.88. 8. 2 Q8885 88$ % omuaosmv” £28 88 808: :8

102

888 A 828.8.

8.858822%

”:88 88888 :8 88 2. :8 5:88:88.
088888 88: 8:8 :8 88 0:. £888 2.
:mmmm888m 8:: :8 8853 0: $8: :8 88888:
8 :8 coo—n.

$88.88 8888:

8:88 88888 on 88888: 88:88 8 :8
8:88 8% :8 88w. 3.88 mm 8388 $08 :8
88:88:88 8888. £88 :8 88:88 862.8
mow—8m 8888: :8: 8 888889

:Exxcuosghiumzm
888888168 :88. 8388188m 8: 88283
8 8:88,»:

MSEEENSW

92:88 88858 $8: 888m 0: 882.8 moo—8m
8888: 8: 8:88. .38 8 :8 888-88: 8::
8.888 88:28.

:88 8 8 8o. . H58. :88 :mmmm: 8:88 m2. o8<<8m “8%...

.8883. :88” .438 mm m 88 door 8 8»: 2:8: 88:88 888 mm: 8 :8
888...
9:883:88. v. on :— 88; :88 Zoo: :38 8 :8 28mm 8088

28:32 _ 88w 8 :Bmos:8m _ :23 88 8v. 8: 88.: _ 88 88 £88.
£88. £88...

b88888. u. 3” :18 m8: _ m8 8 88: :8 coo—n 808.8 z...

M8883. u. :w. .._.8 88:8 «:8: :8v. :8. . .:8v..8 :8 888 888: z,
:8v..8 mo8m :8. rd :8 .8. 58 cm 3. $8.8 8M8: $8.8 moss: :8...

888:. n. :N. i 88: cm: 8: :8 5:8 8:88...

103

.888 8 808.3.

mmhnzzm

008888 8888 3868888 0:. :8 85 0:. m
389:3 3mm. m888 888—88 88 58m 02:
£089

mmhmaazm
m888 8888 m :83 0:. :8 88 8888 0n 2::
<8Q 82 883 2858.

:8»: 885382.03
mqmﬁommmm 8 39:8 08 :02 8 8»: 208% H88

88898 8.8 :0: 888: 8 £9: 88888.

Za§aa$~8m

mag—:2: 8888 m 805. m825Qm888m~ 0:8 :8
c:888=:8m 0:. :8 802. .288 0088888 85: :8
:8: 8 :8 85 0a @888 0: m Ema 882.288 80:8

:8 88:88 8 588883. 8:88 :08 :0" :8

mango 82:8 018888 8:8 2:8: :mmocmmmzm :8

8:88.

8888:: 0038838

008588 :8 8 :0" :8: 8 :8 :00.» 0a 8 808 :8

8:33. 88:. .38 308mm 8 2::. 80:8: 8 :8
88:8 0». 089 900889

OW
008888 08v. 888m 0388 8 22:89

.8883. u. 3. £28: mom—om 0: :8: 60:88 83 878...: _ :88 :8v. :8
8 :88 :8 :08 0: :88. >8: 8 =8 8. 8: x moom 828. 88:8 :8: m2
Emma...

9888:. u. m” 2C8. . :83 2:88. 2:8: :8: 98 82: 28 v83 0:: :8:
:8: 8a 88:28...

.8888. 08893.” 38:88 800888 :8: .38 8 8 98:88.8 8: :8:
:8 88m :9 8»: :8 883

MESS». u. :o“ 2:.8 88.88 88 88%.: 8: maﬁa: £0 8: :8: :8: m8
m: 88:88 .8: m888...

.8883. u. _ T :39). :83. ummmm m8 :884:

.8883. U. NM £48 £8 80w :8 2.? 883—8...

 

104

Hugo A Aoosndv.

208. Comm 338 m8 :mzomNoa.

Eggnog won noamsm Egg—5” 005336 0: En 9035 wow MESS» 9. NA. :05 3 k :3 m8 comma 8 58655.

008838 on Ea 20950530: mean on w. Z 283 .3 o: bxnmozxmmm m3 non—am 8 <o8cEmQ. drama OOO> :23 m8 comma mm

Egg—3%.

105

H88 m

582.3an maniaizm Sm QtemQ £188 V818“: 5888? wag o: @3838 . 388,83

 

 

N
>no§mom

Q88 m. manna Bmwmm .8

$6883 52.83385 3
gngovoBoBENmsm
855858 8 88m :2
88a 8 =8 coow mm m
m8Q.

~=8€83=m U. magma 80588?

HQ: mama—8.0. 58888 8x" 835%

:5” m8 8:58:88 :8:
ﬁrm; £085. 58? 88
R88 ow oosnmamv.

mains" on? Bmwom TN
58.82385 on
8308308388
83888 CW 888 8 :8
coax 8 m m8Q

maﬁa”: 8585388 m B?
om 80:88 88 88888
8i 8888 582—258. 0%
x mm £883 :02 Ban: 8
£283 582m 89: 8§
8858 c8858 888858
m8 88888 8 98
#528: cm 85 8858
8.m.. saw—now m cmowmnoga
838283.

means" 38 8 5688 m
82% $3888 on coo—n.
$3.82? 83.3888.
8&9, maﬁaowoBOnnENOm
n8 81:55.

magma 8888:”? mam 8
moo—=88; 885:9 8x"
mam—88m 28: mm 88
@8me man— 888x.

106

H.868 m 808.8.

_:8€8::m
Um8m88m

7:8-
8:08:28

magma 8:8 8
88:88:~ 5888:
:8m888. 988::
888:8me 8: 88m: H
.8888: £88 8x: 8
2:828:08 :o :2 888:.
AU8mo::88. v. a 8 5“
8:8? u. M E. :8.

:. 38888988 88 8Q

88 8:: 88 88188:-
8883 «$8: :8v~ 88:

£888 88% 8 @8888?
8:588 o: “388:8

8.m.. 8:6? 888 8:85.

wagon: 8858 00,—.
8888 o: :8me 5:88
:88 88 2:88:28: E:
mag—om 8 888:8
8888? $88.:
8888 o: 8888 :88:
88 :2 888:8

38888.8: 88:8

8 8 88: c». :8 .0888:
88:98 888888
88883 3
88m:888::m 8 9888.
P: 88:88. ..€8: :8
28% 88 8: 2.588: a:
:8 ma: 9.8m :88 9: 8::
:8? $38: 88.8% :83
was. 888m :88
Emman...

988:: 8888:”;
888888; 88888
:m8m88m 3 :88 :2 88v~
8888: 8 8883
888888: 888:8: m: :8
E8258. 988:: 8:8:
88:88;~ B88: 8 88%
W: 88818: E8888.
988:» 88 :2 888:8
88v~ .8888: 5:88 8x:
:08 :2 :88: 2:888:98.

. 988:? 88888 88:

m: :88 :8: _
88888.8: 88E :8
8:88 W: :8 8x: on
558:8. .38 :888 0m
:8 88888388 8 88%
mo: 8:88 8 z 8888 8
:8 88m: :88 ow :8 8x8.

107

:88 m 808.8.

 

088:
.88888: 8. 0088888 88 .8: 8 :8 88 8 8. > 82 888888 88 8. 788:8 8888 8:0:
0088888 1:888:08. 88398:: 888: 8 8m :10.. 802888
:8 88. 8:88.. 808. 8:80:08: 88 0:: 8:8
88 8.88:: 88:. 0: 88308:: 8:88: 8
:8 88. :8 88:8.
.228 :8:8 :8 288
:8 :00: 88 m08m 8 8
moo: :80:8 88:8
:8: £88 808 8 .88
5.8 8:: 883....
882888- c. m888 :8 808: 802888 c. m888 :8 8 8:. 0:. :0. 78:: 0: :8 :8888
8.888818 88:88:88 88:8: :8888 802888 88: 88 :88818 8:88
0088888 8:: 08: 888 8 82 :88818 8888818 8883 8:81.818: 88..
8888 A8888 8 :88888 8:: :8888 888888888
8888:? 88888 A8888 8 888883.
1.8888 888888.
:8888 0:. o. m888 8888888 8: o. m888 18888:: 0. 788818 88888 88
788818 0:m08m 808818 :888 8888 808818 88888:: 88 0a
2802888 808818. 88 8888. 88:888.

808818. 0.. 888883 88:3
8:088 :: 21:. :8<8:m
8888. 988818
88888 080.. 8: 8: 8:2
88 808818 8:88.8m0..:.v

108

HmEa m 905.8.

552—89“
zoaﬁonbm
moocmam o:
Zoﬁ

goioama

barman @
gméanmsm
Ocawnosm

WES—ammo Boszonam
83ng moocm o: 2:3 50
£32: :8 $8.58 #08 90
8x”. wen «ESE? :H man.»
.892 :5”. . .3

magma mum—G ~59 $83un 8
8523 m" 5me 25 8x?
858a 9.830: 3 $99.
ammoasm 8 85 8:82 2,
mama—ism.

Kaela—ammo 803815
85595 Ham—w moo—pm on
ﬁrm: So manna :mm 5858
$2: :5 8x“. @252: 33
:95 9:9 368 ow
Boaxonam 88838 man:
8 90mm 9m: Bang Erma Ea
magma mm: comm :2 552.
m3 @882? .4 now; 552
:02 93‘ g :5: ammwd
932: 88 so" mozoé cu
o: @8305 955%.

magma 8km 8x7858a
95303 9: comm so"
323: 8 2522 980
98395.

a. manna :32 8:88
on 23: £530 :8
.833.

109

a. magma £92. 908
so" 8w 8x78583—
ncnmaoam on :8
85325 ~5on m8
mcuonmoma man 30
magma 8 3088.

 

Table 6

Descriptive Statistics for the Four Contributors Measured by the COCA

 

 

Number of
students who Range in
Process Total times used process occurrences Mean and
coded at least once per student (SD)
Comprehension
Strategies
Connections 73 20 0-12 2.43
to Prior (2.81)
Knowledge
Previewing 82 29 0-9 2.73
& Predicting (2.07)
Inferring 60 24 0-9 2.00
(1.98)
Summarizing 22 12 0-4 0.73
(1.11)
Vocabulary 16 1 1 0-3 0.53
(.86)
Processing 15 1 30 2-15 5.03
Graphics (2.67)
Text Features 56 26 0-5 1.87
(1.20)

 

110

Table 7

Descriptive Statistics for Processes that Fall under the Contributors that are

Measured by the COCA, but are not Directly Measured by the COCA

 

 

Number of
Total students who Range in
Process times used process occurrences Mean and
coded at least once per student (SD)
Comprehension
Strategies
Questioning & 64 14 0-14 2.13
Wondering (3.61)
Questioning 2 1 0-2 0.07
the Author (.37)
Activating the 8 4 0-3 0.27
Senses (.69)
Monitoring
Text 28 20 0-4 .93
Monitoring (.91)
Rereading 8 6 0-3 .27
(.64)
Knowledge 43 19 0-7 1.43
Monitoring (1 .79)
Processing
Graphics
Intratextuality 24 12 0-4 0.80

(1.16)

 

111

Table 8

Descriptive Statistics for Verbal Protocol Processes that are not Measured Directly or

 

 

Indirectly by the COCA
Number of
Total students who Range in
Process times used process occurrences Mean and
coded at least once per student (SD)
Evaluating 14 9 0-6 0.47
(1.14)
Affective Reaction 12 7 0-4 0.40
(.93)
Anthropomorphizin 4 2 0-2 0.13
g (.51)
Empathizing 5 3 0—3 0. 1 7
(.59)
Restatin g 60 20 0— 1 0 2.00
(2.57)
Repeating 19 8 0-5 0.63
(1 .3 5)
Word Identiﬁcation 5 4 0-2 0.17
(.46)
Narrativizing 26 6 0-12 .87
(2.54)
Irrelevant 3 8 14 0-8 1 .27
Comments (2. 16)

 

112

Table 9

Quantitative Processing Differences between Salmon and Dragonﬂies

 

' a
Total Processes Different Processes

 

Salmon (N = 16) 521 173
Mean 32.56 10.81
(SD) (8.19) (2.69)
Range 16 - 42 5 — 15

Dragonﬂies (N = 14) 299 125
Mean 21.36 8.93
(SD) (6.92) (2.02)
Range 9 — 36 6 — 12

Aggregate 820 298
Mean 27.33 9.93
(SD) (9.41) (2.55)
Range 5 - 42 5 — 15

 

aDifferent processes indicates the sum of the total number of different processes

reported by the students.

113

Table 10
Proportion of Matches, Related Matches, Misses, and Non-Responses between Verbal

Protocols and COCA Items for Salmon

 

 

Comprehension Processing Text Across
Strategies Graphics Vocabulary Features Contributors

Matches &

Related .38 .36 .10 .81 .27
Matches

Misses .39 .52 .65 .16 .54
Non- .23 .13 .24 .19 .20
Responses

 

114

Table 11
Proportion of Matches, Related Matches, Misses and Non-Responses between Verbal

Protocols and COCA Items for Dragonﬂies

 

 

Comprehension Processing Text Across
Strategies Graphics Vocabulary Features Contributors
Matches &
Related .47 .71 .03 .36 .32
Matches
Misses .18 .19 .58 .30 .36
Non- .35 .10 .39 .34 . 33

Responses

 

115

.0020 5

00000000000. 9.0.0010. 0:0 000020000030. V0300 033000 3.00.08

 

 

030.00 08000000 000000
>00<0 .2080 880008 3000000 80 0 $000 30% 0m 900000 380 80 00 F000 0 05.0000:
0300808 830808 000 000 00070—0N 00083000« 300088. 3.08 00048008 30%.; 0 003

300008 #03 90 000:. .0080 080008 03
0080 B03330 90: 0030000000000 000
0.0 00000035 200» 0000N 0000 3 9000 0100.
§0€_00m0.

>090 0300808 .2080 030008 000 0090—0\ 000 300020.

2:: 00300000 0083090 300000. E 00050:. 900 040

0308808 050 330.5 0003? 00 00 _00m~ 000 03008.
00000000: 030088 080200 000—000”
0300800m @3058. 30003000. 000335
0000 5002—008. 0000030? «80090. 000
00800005.

ﬂaw—0 0300808 .200 B0000=m 00830000 0». 0080 080008
8 000000000 0% 0 ﬂaw—0 03008. .0080
030088 000—000 $0335 0% 00:00
0000000000. 0000000000. 000 000<000m 0000
53%—00,00.

0m 90 300. NM 00.00808 0: 90 0200305.
90000» 380 0<0~0m0 00 008 0 03008
000 00m0 + 080 SN 638098 00 mm 0.04
@0303.

9.0003 380 000—8 00 E08 0 3000330
0033000 9003 3000300? 00.00005. 00
302—008 Bowmaoasmv.

$00000 3008 000000 004 00.0.0 0300800
>25

88 00 E080 _ 03008 0 330303 00. 0
038.

@0003 008 20,—. 300" 000000 0.00 000<0
030083 >20
88 0 000m? 03008 0” 8080 0 038

116

00000 5 003.3.
>00<0 000

90000000—
0300808

m00m00 90000003
0300808

0803 0300808

.0080 830008 3000 30 000000 0.00 000<0
0300808 03 080 3008 000003
80000000. 0033008.

700033 0008000000 8 00300—0~ 0 33 00.
0 83m? 03008 000 80000000: 0033008.

.0080 830008 0000: 0 .02 _0<0_ om
00m000<0 000 0000003 00008\ 000 08 000
083m 0 $000 <000Q 00. 030088.

90003 3008 000000 won 000<0 030889
>20

3008 00 0080 0 3.00033 0003..
00000000 0033008.

m3003 3008 000 000000 0.00 800m_0
0300800 >20

30.08 00 00080 0 080033 00000
~000000m 0033008.

90003 008 000 3000 000000 mow 000<0
030083 >20

3:8 8 .80 0 800m? 03008 308 300 0
038.

 

117

Table 13
Frequencies Broken Down by Form (Salmon & Dragonﬂies), Total Number of
Processes, Number of Different Processes, and Maximum Number of Reports for Most

Frequently Reported Process for Reading Proﬁles

 

Total Processes Different Processes Maximum Process

 

Proﬁle N (Means & SD) (Means & SD) (Means & SD)
Active Processors 3 30.67 11.67 5.00
(2D, IS) (5.69) (.58) (1.00)
Active Processors with 11 36.45 11.80 10.09
foundation process(es) (3D, 88) (5.96) (1.60) (2.84)
Single Processors 5 22.20 8.00 7.40
(2D, 4S) (5.78) (1.10) (0.55)
Active and Superﬁcial 1 27 12 8.00
Processors (0D, lS) (--) (--) (--)
Single Superﬁcial 1 26 5 11
Processor (0D, 1S) (--) (--) (--)
Passive Processors 9 18.11 8.44 4.55
(8D, 1S) (4.20) (2.24) (1.24)

 

118

Figure 1

Percentages for Process Breakdown

Unique
Processes
22%

COCA
Processes
56%

COCA-
Related
Processes
22%

 

Note. Images in this dissertation are presented in color.

119

Appendix A
Script for Think-aloud Sessions

The researcher introduces herself and asks the student to accompany her to a quiet place
where they will not be disturbed. Researcher obtains student’s verbal assent. The
researcher begins by asking the child if he or she wants to hear his or her voice on the
tape recorder as a way to introduce the recording device. The researcher then brieﬂy
demonstrates how the tape recorder works.
T raining for Puzzle Activity

The researcher introduces the puzzle activity.

“Today I am going to watch as you try to put this puzzle together. You

are going to work on it for about ﬁve minutes. It is a really hard puzzle

so you may not be able to ﬁnish it in ﬁve minutes. It is not important

how fast you can do it or even if you can solve it. Rather, I really want to

know about what you are thinking as you put the puzzle together. Since I

cannot read your mind or see inside your head, I would really like for

you to tell me what you are thinking and feeling as you put the puzzle

together. There are no right or wrong answers. In fact, anything you can

tell me would be helpful as I am interested in learning more about how

students like you think. You can stop and tell me what you are thinking

any time. But sometimes I will stop you and ask you to think aloud for

me. (Researcher will ring bell once a minute for ﬁve minutes.) Before

starting the researcher asks, “Is there anything you’re thinking before

you start?”

120

After the child answers researcher replies, “Great, go ahead and try to put it together.”
After each minute the researcher will ask, “Stop for a moment and tell me, what are you
thinking about, or what are you trying to do right now?”

After the ﬁve one-minute segments the researcher says, “Now I want you to stop
working on the puzzle. Is there anything else you want to say about what you are
thinking or feeling before we move on?” After student’s reply researcher says “I will
show you how to solve the puzzle once we are done with the next activity” (if student is
unable to solve the problem in the allotted time). The researcher removes the puzzle
from the table so it will not be a distraction for the student.

Before Think-Aloud for Book

“I really appreciate you telling me what you were thinking as you put the puzzle
together. That was good practice for what we are about to do. I love to read books to
second graders. I am going to read this book to you. Do you notice anything that is
different about this book than other books you’ve read before?” (Student should notice
that the words are missing and that there are lines on the bottom of the page). “I am
going to read the words that go on each page. I will point to the page that I am reading.”
(Demonstrate for student: page 8 & 9 for Salmon, page 4 & 5 for Dragonﬂies). “Also,
in this book there may be a few pages where the authors or illustrator made mistakes.
For example, the picture may not match what the words say on the page. If you see a
mistake, be sure to let me know.”

Researcher introduces the directions for thinking aloud.
“Now before we start, I have some directions. What I am really

interested in is hearing your thoughts and feelings about the book as I

121

 

read it to you. So just like the puzzle where you told me what you were
thinking and feeling, I want to know what’s going on inside your mind as
you read. You can stop and tell me what you are thinking whenever you
want. That is okay. Remember, just like the puzzle, there are no right or
wrong answers. Also, do you see these yellow dots? I will stop and ask
you to tell me what you are thinking and feeling when I get to each
yellow dot. Sometimes there may be places where the book is not
making you think or feel anything. If this happens you can shake your
head no or tell me that you are not thinking anything and we will go on.
Also, it is okay if you want to ﬂip through and look at pages. Finally, if
you want to hear any of the pages again just ask and I will reread the
page to you. Can you tell me the directions so that I know you

understand please?”

(Clarify any misconceptions and review directions as needed). When student repeats the

directions check to see that three main points are captured: 1) Can think aloud at any

time, 2) Have to think aloud after yellow dot or say, “I’m not thinking anything” and, 3)

Can ask to reread.

“I showed you the tape recorder before and now I would like to record you as we read.

This will help me remember what you talk about later. Do you have any questions?”

During the Mink-Aloud.

F ollow-up prompts:

The researcher asks for clariﬁcation when the meaning of the student’s comment

is unclear. For example, the researcher may ask for further explanation by

122

 

repeating the student’s comment and saying, “Can you tell me a little more
about what you are thinking?”, “What made you think that?”, or “What did you
mean when you said, [insert child’s words]?”

0 The researcher prompts when student responds nonverbally (e. g., makes a face,
shrugs, eyes dart between pages, ﬂipping pages) or makes a noise (e.g., laughs,
grunts). For example, if a student’s eyes are traveling between pages the
researcher will prompt, “I noticed that you were [insert behavior]. What were
you thinking when you were doing that?”

Referencing Illustrations:

0 When a student points to an illustration or graphic, the researcher marks the
place in a photocopy of the book for later reference. If the researcher is unclear
about where a student is looking she asks, “Can you show me where you
mean?” or “Can you point to where you are looking?”

Prompting at yellow dots:

«- At each yellow dot the researcher asks: “What are you thinking or feeling?” or
“Think-aloud for me.” (Sometimes it is not necessary to prompt at this point
because the student gets in the habit of thinking aloud once he/she reaches the
yellow dot. At that point asking the prompt can interrupt the child’s talking and
thoughts.) If a child chooses to skip talking-aloud more than three times (at
yellow dots) the researcher reminds the student that, “Remember that it is really
important for you to share what you are thinking and feeling when we get to
these yellow dots.”

After the T kink-Aloud.

123

 

 

“Now that we are ﬁnished reading the book do you have any other thoughts or feelings
that you would like to share with me?” The researcher models how to put puzzle back
together again with help from the student. The session ends by thanking the child and
asks if she or he wants to hear his or her voice on the tape recorder. The student then

returns to the classroom.

124

 

 

  

    
 

A

llilillllllll11111111
3 1

293 02

    

VERSIT‘V L‘I‘BRA‘HIES
: 1 1‘ H :1
llzlllllllll l

956

9294

 

LIL-riv‘