EXAMINATION OF TEST-RETEST RELIABILITY OF A COMPUTERIZED
NEUROCOGNITIVE TEST BATTERY
By
Yusuke Nakayama

A DISSERTATION
Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of
Kinesiology – Doctor of Philosophy
2013

ABSTRACT
EXAMINATION OF TEST-RETEST RELIABILITY OF A COMPUTERIZED
NEUROCOGNITIVE TEST BATTERY
By
Yusuke Nakayama
Context: Test-retest reliability is a critical issue in utilization of computerized neurocognitive
assessments employing pre-participation baseline test followed by a series of post-concussion
tests. Low test-retest reliability was reported by Broglio et al. (2007) for one of the most widely
used neurocognitive test battery, Immediate Post Concussion Assessment and Cognitive Testing
(ImPACT) with specific testing intervals.
Purpose: The purpose of this study was to re-examine the test-retest reliability of ImPACT
between baseline, 45 days and 50 days after baseline.
Methods: A total of 88 physically active college students (54 male, 34 female) volunteered for
this study. A repeated-measures design was used with test group (administration of ImPACT at
baseline, 45days after the baseline, and 50 days after the baseline) as the independent variable
and ImPACT composite scores (Verbal Memory, Motor Processing Speed, and Reaction Time)
as the dependent variables. The pre-testing survey was administered to assess the participants’
physical activity level and physical and mental condition on the testing day. Following the pretesting survey, the Rey’s 15-item memory test was administered before each ImPACT
administration as an effort measurement. Following ImPACT, the post-testing survey was
administered to assess the testing environment.
Results: Intraclass correlation coefficients between baseline to Day 45, Day 45 to Day 50,
baseline to Day 50, and overall intraclass correlation coefficient were reported as follows: visual
memory (.76, .69, .66, .78), verbal memory (.72, .66, .60, and .75), motor processing speed

(.86, .87, .84, .90), and reaction time (.68, .83, .72, and .81). All intraclass correlation coefficient
values exceeded the threshold value of .60 for acceptable test-retest reliability. The pre and posttesting survey with 0 to 10 likert scales revealed the participants were moderately stressed
(average 4.64) and mildly fatigued (average 3.45), with low level of distraction (average 1.73)
while completing the ImPACT test. With an exception of one participant, all participants scored
perfect on Rey’s 15-item Memory test.
Conclusion: Results of this study suggest that ImPACT is a reliable neurocognitive test battery
over 45 and 50 days after baseline assessment. The current study’s findings agree with other testretest reliability studies that have reported acceptable intraclass correlation coefficients over
different testing intervals, which support the utilization of ImPACT for the multidisciplinary
approach to concussion management.

ACKNOWLEDGEMENTS

I would like to acknowledge the effort of my dissertation committee Dr. Tracey Covassin,
Dr. Sally Nogle, Dr. Philip Schatz, and Dr. Jeffery Kovan for their help and guidance with this
dissertation.
To Dr. Tracey Covassin, it may be a clinch, but it is so true that I would not have been
able to complete this project and degree without you. Your positive and cheerful character with
exceptional patience guided me through one of the biggest challenges in my life.
To Dr. Sally Nogle, it was my honor to learn about athletic training from you, and have
you on my committee. Your ability to fulfill numerous responsibilities at high standard is just
fascinating, and I will model after you in my career.
To Dr. Philip Schatz, thank you for serving on my committee, and your valuable advices
and suggestions especially in the area of statistics. Your help was absolutely imperative for this
project. I hope I will have an opportunity to convey thankfulness in person.
To Dr. Jeffery Kovan, even though it was a last minute request, you generously accepted
to become my committee. Thank you for serving on my committee and sparing your valuable
time for helping me. It has been my honor to have a distinguished professional like you.
To Jeff Monroe, thank you for looking after me these four years. I appreciate your letting
me work at the football team, and helping my lectures and the dissertation. I enjoyed all the
conversations I had with you, and I will try to be an athletic trainer whom you can be proud of.
To Sue Halsey, Carol Christofferson, Verna Lyon, Darlene Howe, and Jan Davenport,
thank you for your courteous support. You always welcomed me with warm smiles, and made
Room 134 the most comfortable place on campus.

iv

To Roger Hinds of the New York Knicks, thank you for being my role model as an
athletic trainer and as a person. Your presence has been one of the biggest motivations for me to
cope with these challenging four years. I will push myself to become like you.
To Masayuki Fujihashi, my treasured friend and mentor, thank you for introducing me to
the Michigan State University. These great four years would not have started without your
guidance. Your ceaseless effort as a clinician always inspires me to do the same.
To my parents, Sumio and Hatsuyo. Thank you for your endless support and instilling
your hard work ethic in me. When I left Japan eight years ago, nobody could imagine me
obtaining a doctoral degree from one of the finest universities in the States. I hope you are proud
of this achievement.
And finally, thank you Fiona for all your support. I cannot put into words how much I
appreciate what you have done for me and our family. This is our achievement and you deserve
huge credit for it. I sincerely wish I could put your name next to mine for this doctorate degree.
Thank you Ty and El, our precious sons, for bringing so much joy and a lot of laughter to our
home. I am looking forward to the journey that we are going to take together.

v

TABLE OF CONTENTS

LIST OF TABLES ..................................................................................................... viii
LIST OF FIGURES ...................................................................................................... ix
CHAPTER I INTRODUCTION .................................................................................... 1
Purpose of the Study ............................................................................................... 7
Hypotheses.............................................................................................................. 7
Limitations .............................................................................................................. 8
Assumption and Delimitations ............................................................................... 8
Definitions .............................................................................................................. 8
CHAPTER II LITERATURE REVIEW .................................................................... 11
Introductory .......................................................................................................... 11
Historical Overview of Definitions ...................................................................... 10
Epidemiology of Concussion................................................................................ 13
High School vs. College ...................................................................... 14
Practice vs. Game ................................................................................ 15
Type of Sports ..................................................................................... 16
Sex Difference in Concussion Incidences ........................................... 19
Pathophysiology ................................................................................................... 23
Signs and Symptoms of Concussion .................................................................... 25
Evaluation and Management of Sports-Related Concussion ............................... 31
Sports-Related Concussion Consensus and Position Statement ......... 37
Overview of Neuropsychological Testing for Concussion ................................... 38
Paper and Pencil Neurocognitive Tests ............................................... 40
Computerized neuropsychological Assessment of Concussion .......... 42
Automated Neuropsychological Assessment Metrics(ANAM) .......... 43
CogSport.............................................................................................. 46
Concussion Resolution Index (CRI) ................................................... 48
Immediate Post Concussion Assessment and Cognitive Testing
(ImPACT) ............................................................................................. 50
Physical Activity and Neurocognitive Performance ............................................. 55
CHAPTER III METHODOLOGY ............................................................................. 60
Research Design ................................................................................................... 60
Participants ........................................................................................................... 60
Inclusionary criteria............................................................................. 60
Exclusionary criteria ........................................................................... 61
Instrumentation ..................................................................................................... 61
Immediate Post Concussion Assessment and Cognitive Testing
(ImPACT) ............................................................................................. 61
Rey’s 15 Items Memory Test............................................................... 64
Physical Activity Level Survey ........................................................... 64
vi

Physical/Mental Condition Survey...................................................... 64
Post-Testing Survey............................................................................. 65
Procedures ............................................................................................................ 65
Deta Analysis ........................................................................................................ 65
Evaluation of Hypotheses ..................................................................................... 66
CHAPTER IV RESULTS ........................................................................................... 67
Overview .............................................................................................................. 67
Demographic Data ................................................................................................ 67
Testing Intervals ................................................................................................... 68
Evaluation of the Hypotheses ............................................................................... 68
Results of Computerized Nurocognitive Testing (Hypotheses 1 - 4) .. 68
Results of the Rey’s 15-item Memory Test (Hypothesis 5) ................ 73
Evaluation of Survey Data and Total Symptom Score ......................................... 74
Physical Activity Level ....................................................................... 74
Condition of the Participants and Testing Environment ..................... 75
Evaluation of Association between ImPACT Composite Scores and Survey
Data and Toal Symptom Score ............................................................................. 78
CHAPTER V DISCUSSION ...................................................................................... 80
Introduction .......................................................................................................... 80
Comparison of ICC Values with Other Studies .................................................... 80
Comparison with the Study of Broglio et al. (2007) ........................... 80
Comparison with Elbin et al. (2011) and Schatz et al. (2010) ............ 84
Test-Retest Reliability Difference across the Composite Scores ........ 84
Discussion of Survey Data ................................................................................... 85
General Discussion.............................................................................. 85
Statistical vs Clinical Significance ...................................................... 87
Implication of Findings ........................................................................................ 89
Limitations ............................................................................................................ 90
Suggestions for Future Research .......................................................................... 91
Conclusion ............................................................................................................ 92
APPENDICES ............................................................................................................. 93
APPENDIX A: Human Subjects Consent Form ................................. 94
APPENDIX B: Figure A.1. Rey’s 15-item Memory Test ................... 96
APPENDIX C: Pre-Testing Survey..................................................... 97
APPENDIX D: Post-Testing Survey ................................................. 100
REFERENCES .......................................................................................................... 101

vii

LIST OF TABLES

Table 1. Grading system from AAN (1997), Cantu (1986&2001),
and CMS (1994)........................................................................................................... 35
Table 2. Return to play guideline from AAN (1997), Cantu (1986&2001),
and CMS (1994)........................................................................................................... 36
Table 4.1. Demographic Information for Age, Height, and Weight ............................ 68
Table 4.2. Means and Standard Deviations for ImPACT Composite Scores at
Baseline, 45 days after the Baseline, and 50 days after the Baseline .......................... 69
Table 4.3. Pair-wise Comparison of ImPACT Composite Scores at Baseline,
45 days after the Baseline, and 50 days after the Baseline .......................................... 70
Table 4.4. Intraclass Correlation Coefficients for ImPACT Composite Scores
between Baseline and Day 45, Day 45 and Day 50, Baseline and Day 50, and
Overall.......................................................................................................................... 72
Table 4.5. Cronbach’s Alpha for ImPACT Composite Scores between Baseline
and Day 45, Day 45 and Day 50, Baseline and Day 50, and Overall .......................... 73
Table 4.6. Means and Standard Deviations for Rey’s 15-item Memory Test, and
Self-Reported Effort Level .......................................................................................... 74
Table 4.7. Means and Standard Deviations for the amount of the exercise at
Baseline, 45 days after the Baseline, and 50 days after the Baseline
(minutes/week) ............................................................................................................. 75
Table 4.8. Summary of the Condition of the Participants and Testing
Environment ................................................................................................................. 76
Table 4.9. Pair-wise Comparison of the Condition of the Participants and
Testing Environment at Baseline, 45 days after the Baseline, and 50 days
after the Baseline.......................................................................................................... 77
Table 5.1. Comparison of ICCs Values between the Current Study and
Broglio et al. (2007) ..................................................................................................... 81
Table 5.2. Comparison of ICC Values across the Test-Retest Studies ........................ 85

viii

LIST OF FIGURES
Figure 4.1. Composite Score for ImPACT Verbal Memory at Baseline, 45 days
after the Baseline, and 50 days after the Baseline. For interpretation of the references
to color in this and all other figures, the reader is referred to the electronic
version of this dissertation ........................................................................................... 70
Figure 4.2. Composite score for ImPACT visual memory at Baseline,
45 days after the Baseline, and 50 days after the Baseline .......................................... 71
Figure 4.3. Composite score for ImPACT motor processing speed at Baseline,
45 days after the Baseline, and 50 days after the Baseline .......................................... 71
Figure 4.4. Composite score for ImPACT reaction time at Baseline,
45 days after the Baseline, and 50 days after the Baseline .......................................... 72
Figure 4.5. Condition of the participants and testing environment at Baseline,
45 days after the Baseline, and 50 days after the Baseline .......................................... 77
Figure A.1. Rey’s 15-item Memory Test ..................................................................... 96

ix

CHAPTER I
INTRODUCTION
Sporting activities are second to motor vehicle crashes as the leading cause of traumatic
brain injury (TBI). In the 1990’s there were approximately 300,000 sports-related concussions
reported annually in the United States. Recently it has been estimated that 1.6 to 3.8 million mild
TBIs occur in sports and recreational activities among individuals aged 15 to 24 years in the
United States annually (Langlois, Rutland-Brown, & Wald, 2006). Increases in the number of
athletes participating in sports, increased public awareness about concussion through media and
programs such as the Heads UP program by Centers for Disease control and Prevention (CDC),
and the improvement in concussion diagnosis are thought to be the reasons for this significant
increase in concussions.
Even though the origin of the term is clear (concutere, means dash together, to shake
violently or concussus, means action of striking together), defining a concussion has been a
difficult task due to its complex nature of symptoms and cognitive impairments associated with
concussion. To date, researchers and clinicians have not come to a consensus on the exact
definition of sport-related concussion. In the latest International Conference on Concussion in
Sport (CIS) a concussion was defined as a complex pathophysiological process affecting the
brain, induced by traumatic biomechanical forces (McCrory et al., 2009). The CIS group also
suggested five common features of sport-related concussion including: 1) concussion can be
caused by direct impact to the head, face, neck, or elsewhere on the body with an “impulsive”
force transmitted toward the head; 2) concussion typically results in the rapid onset of short-lived
impairment of neurological function that resolves spontaneously; 3) concussion may result in
neuropathological changes, but the acute clinical symptoms largely reflect a functional

1

disturbance rather than structural injury; 4) concussion results in a graded set of clinical
syndromes that may or may not involve LOC; and 5) concussion is typically associated with
grossly normal structural neuroimaging studies (McCrory et al.,2009).
Epidemiological studies have been conducted based on the data from Reporting
Information Online (RIO) for high school population and National Collegiate Athletic
Association (NCAA) Injury Surveillance System (ISS) for collegiate population. The latest
epidemiological study utilizing the RIO reported that approximately 2.5 concussions per 10,000
athlete exposures (A-Es) occurred in this population (Marar, McIlvain, Fields, & Comstock,
2012). For collegiate population, a study utilizing the NCAA ISS summarized 16 years (1988 to
2004) of injury data from 15 NCAA sports and reported approximately 3.4 concussions per
10,000 A-Es (Hootman, Dick, & Agel, 2007). Even though cautions are needed for interpretation
of the data (e.g., number and type of sports studied), these data suggest that collegiate athletes
may have slightly higher injury rates than high school athletes.
The signs and symptoms of concussion typically fall into 4 categories: physical, cognitive,
emotional, and sleep disturbances (Almasi & Wilson, 2012). Typical signs and symptoms of a
concussions include headache, nausea, dizziness, fatigue, vomiting, balance problems, and
sensitivity to light and noise as common physical symptoms (Barth et al., 1989; Cantu, 1998a;
Maddocks & Dicker, 1989). Mental fogginess, difficulty in concentrating, decreased processing
speed, and memory difficulties are common cognitive symptoms (Maruta, Lee, Jacobs, & Ghajar,
2010). Emotional symptoms including personality change, sadness, nervousness, and sleep
disturbance may be observed in some concussed athletes (Guskiewicz et al., 2006; Lovell et al.,
2006; McCrory et al., 2005).

2

Detecting probable concussion is important to prevent more serious consequences, such
as second-impact syndrome (SIS) that often results in brainstem herniation and death (Cantu,
1998). Even though self-report from athletes becomes such an important part for detecting
probable concussion, the underreported nature of concussion has long been a problem in sport
participation. There are several reasons why athletes do not report their concussion such as
underestimation of seriousness of injury, motivation to stay in competition, cluelessness of the
signs and symptoms of concussion (McCrea, Hammeke, Olsen, Leo, & Guskiewicz, 2004). Due
to this underreported nature of concussion, estimation of concussion incident is thought to be a
conservative estimate. Evaluation and management for concussed athletes have been improved
with utilization of computerized neurocognitive assessments and individualized management
protocols.
Traditionally, major on-field markers for concussion have been thought to be loss of
consciousness (LOC) and post traumatic amnesia (PTA) (McCrea, Kelly, Randolph, Cisler, &
Berger, 2002). In fact, the status of LOC and PTA have been used in most of the grading scales
to determine grade and severity of concussion. However, the most recent consensus statement
from the International Conference on CIS stated that the status of clinical post-concussive
symptoms is more important than the presence or duration of PTA and LOC (McCrory et al.,
2009). These findings have contributed to the improvement of management guidelines that used
to put emphasis on the status of PTA and LOC.
Management guidelines published over the past two decades are largely based on the
opinions of groups of experts rather than on empirical research (Collins, 1999); therefore,
consensus statements have suggested the abolishment of historically utilized concussion grading
scales (Aubry et al., 2002; Mccrory et al., 2005; 2009). In consequence, the management and

3

return to play strategies for sports-related concussion have been refined through re-evaluation
and revision of prior guidelines. Currently, a medically supervised step-wise return-to-play
process is similarly recommended by the CIS and National Athletic Trainers’ Association
(NATA) position statements (Aubry et al., 2002; McCrory et al., 2005; 2009). Based on the
possibility of symptoms returning from progressive physical and sport-specific exertion, a total
of six stages were recommended from complete rest with no activity until asymptomatic (stage 1)
to full contact training after medical clearance (stage 5) and return to play (stage 6) ( McCrory et
al. 2009; Guskiewicz et al., 2004). Injured athletes proceed to the next level if they remain
asymptomatic at the current stage or they must go back to the previous asymptomatic stage if
their concussion symptoms return. The progression resumes after 24 hours of physical and
cognitive rest (McCrory et al.,2009). Along with the development of step-wise progression for
return to play protocols, the utilization of computerized neurocognitive test batteries is another
area of improvement in concussion management.
Neuropsychological assessments have evolved from traditional paper and pencil
measurements (e.g., Paced Auditory Serial Addition Test, the Trail-Making Test A and B, and
the Digit Symbol Test etc.) to computerized neuropsychological assessments in order to
overcome the weaknesses of traditional methods such as high cost, difficulty of administration
and evaluation, learning effects, and sensitivity (Collins & Hawn, 2002; Hinton-Bayre, Geffen,
Geffen, McFarland, & Friis, 1999; McCrea et al., 2005). There are several computerized
neurocognitive assessment programs available including the Automated Neuropsychological
Assessment Metrics sports medicine battery (ASMB), CogSport, the Concussion Resolution
Index (CRI), and the Immediate Post Concussion Assessment and Cognitive Testing (ImPACT).
These computerized neuropsychological test batteries measure several domains of cognitive

4

function including working memory, attention, concentration, short-term verbal and non-verbal
memory, and reaction time.
Currently, utilization of computerized neurocognitive assessments employing preparticipation baseline test followed by a series of post-concussion tests have become a crucial
element of the multidisciplinary approach to concussion evaluation and management
(Guskiewicz et al., 2004; McCrory et al., 2009). The advantage of employing baseline testing is
to minimize possible confounding factors such as age, sex, and individual’s cognitive strength
and weaknesses (Bernhardt, 2000; Guskiewicz et al., 2004). Obtaining baseline testing data
enables more individualized concussion management.
The ImPACT test battery is a widely used neuropsychological assessment program that
evaluates multiple aspects of neurocognitive function including concentration, attention, memory,
visual motor speed, and reaction time. The ImPACT test battery consists of three main parts:
demographic data, Post-Concussion-Symptom Scale (PCSS), and the neuropsychological tests
modules. Some variables obtained in the demographic data consist of subjects’ sport, medical,
and concussion history. The PCSS is comprised of 22 symptoms rated on a seven-point Likert
scale (0 to 6), and is used to document and track concussion symptoms. The neuropsychological
modules are comprised of six tests that measures memory, attention, concentration, visual-spatial,
and working memory. ImPACT generates four composite scores: verbal memory, visual memory,
visuomotor speed, and reaction time. Impulse control composite score is also generated in order
to detect an invalid test due to test takers’ lack of effort or too many errors. Currently, the
ImPACT test can be administered online, which enhances the ease of use of this assessment tool.
Several studies have been conducted regarding sensitivity, specificity, reliability, and validity of
ImPACT test battery.

5

High sensitivity (81.9%) and specificity (89.4%) (Schatz, Pardini, Lovell, Collins, &
Podell, 2006) and good construct validity with standardized neuropsychological tests (r=.34=.59, p<.001) (Maerlender et al., 2010) of ImPACT have been reported. As for the reliability,
with a 7-day time span, Iverson et al.(2003) reported moderate to high Pearson test-retest
correlation coefficients for the following composite scores: verbal memory=.70, visual
memory=.67, reaction time=.79, processing speed=.89 and post-concussion scale=.65. With a 4month time span, Miller et al. (2007) reported no significant difference in all composite scores
when 80% confidence interval was used (p=.04 for visual memory and reaction time, p=.05 for
processing speed, p=.06 in verbal memory). With a 1-year time span, Elbin et al. (2011) reported
ICC values of .85 for motor processing speed, followed by reaction time (.76), visual memory
(.70), verbal memory (.62), and post-concussion symptom scale (.57). Finally, with a 2-year time
span, Schatz et al. (2010) reported ICC of .74 for motor processing speed, followed by reaction
time (.68), visual memory (.65), verbal memory (.46), and then total symptom score (.43). Even
though ICC and correlation coefficients varies over studies and composite scores, the
recommended acceptable ICC threshold (.6) by Baumgartner et al. (2001) is satisfied in most of
the cases.
However, Broglio and colleagues (2007) reported low ICC for all ImPACT composite
scores through three test administrations: baseline, 45 days after the baseline, and 50 days after
the baseline. The ICC for each composite score were as follows: verbal memory (.23 for baseline
to day 45, and .40 for day 45 to day50), visual memory (.32 and .39, respectfully), motor
processing speed (.38 and .61, respectfully), and reaction time (.39 and .51, respectfully). These
low ICC values may have been attributable to methodological problems caused by administering
three different computerized neurocognitive assessments in succession (ImPACT, Headminder’s

6

CRI, Concussion Sentinel). Also, the uncontrolled physical activity level of the participants
might have affected the result of this study as studies have reported the positive effect of physical
activity on cognitive function of young adults (Hillman, Kramer, Belopolsky, & Smith, 2006;
Kamijo & Takeda, 2009).While other studies that showed acceptable ICC and correlation
coefficients used athletes as their study participants, Broglio and colleagues (2007) used “healthy
college students” with no description of their physical activity level. To date, no study has been
conducted to re-examine the test-retest reliability of ImPACT for mid-term (45-50 days) after
Broglio et al.(2007) reported low ICC even though their methodological problem has been
pointed out numerous times. In addition, no study has controlled for physical activity level and
other possible confounding variables such as hours of sleep and caffeine intake.
Purpose of the Study
The purpose of this study is to re-examine the test-retest reliability of the ImPACT
neurocognitive test battery between baseline, 45 days and 50 days after baseline on physically
active college students.
Hypotheses
H1.There will be acceptable test-retest reliability(ICC≥.60) in verbal memory composite scores
over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.
H2.There will be acceptable test-retest reliability (ICC≥.60) in visual memory composite scores
over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.

7

H3.There will be acceptable test-retest reliability (ICC≥.60) in motor processing speed scores
over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.
H4.There will be acceptable test-retest reliability (ICC≥.60) in reaction time composite scores
over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.
H5.There will be no difference in test takers’ effort level as measured by the Rey’s 15-item
memory test over three test administrations (baseline, 45 days, 50 days).
Limitations
The limitations of this study will include: 1) participants will be physically active
individuals who satisfy the ACSM recommendations, 2) the participants will be college students;
therefore the results may not be applicable for the other age groups; 3) the results of this study
will largely depend on the effort level of participants on the neurocognitive tests.
Assumption and Delimitations
This study will make the following assumptions: 1) the participants will perform to the
best of their ability on the ImPACT tests at all three testing administrations; 2) the Rey-15s Test
will detect the effort level of participants on the ImPACT tests.
Definitions
Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)—a computerized
neuropsychological test battery, for measuring attention and processing speed in athletes with
concussions (Iverson et al., 2005)
Impulse Control Composite Score— score represents the total number of errors of omission or
commission on the go/no-go test and the choice reaction time test, which is used to identify

8

athletes who are seriously confused about test instructions or who might have made numerous
right-left confusion errors (Iverson, Lovell, & Collins, 2005)
Intraclass correlation coefficient (ICC)—a univariate measure estimate of the agreement
between scores on the same test at two points in time (Broglio et al., 2007)
Loss of consciousness—a state of brief coma in which the eyes are typically closed and the
athlete is unresponsive to external stimuli (Lovell, 2009).
Physically active individual—individual who meets basic recommendation on cardiorespiratory,
resistance, and neuromotor exercises from the American College of Sports Medicine (Garber et
al., 2011)
Post Concussion Symptom Scale— the total score derived from the measurement on 22
commonly reported symptoms (e.g., headache, dizziness, “fogginess”) by seven-point likert scale
(Lovell & Collins, 1998)
Post traumatic amnesia—the period of amnesia following the injury until the athlete regains
normal continuous memory functioning (Lovell, 2009)
Processing Speed Composite Score— the composite score represents the weighted average of
three tasks that are done as interference tasks for the memory paradigms (Iverson, Lovell, &
Collins, 2005).
Reaction Time Composite Score— the composite score represents the average response time (in
milliseconds) on a choice reaction time, a go/no-go task, and the previously mentioned symbol
match task (Iverson, Lovell, & Collins, 2005).
Verbal Memory Composite Score— the composite score that represents the average percent
correct for a word recognition paradigm, a symbol number match task, and a letter memory task
with an accompanying interference task (Iverson, Lovell, & Collins, 2005).

9

Visual Memory Composite Score— the composite score is comprised of the average percent
correct scores for two tasks; a recognition memory task that requires the discrimination of a
series of abstract line drawings, and a memory task that requires the identification of a series of
illuminated X’s or O’s after an intervening task (mouse clicking a number sequence from 25 to 1)
(Iverson, Lovell, & Collins, 2005).

10

CHAPTER II
LITERATURE REVIEW
Introductory
Although the literature covers a wide variety of aspects regarding sports-related
concussion, this review will focus on seven main areas in order to explore the current knowledge
base of concussion and how computerized neurocognitive test battery has been utilized as a part
of multi-facet management of this complex injury. This literature review will support the
rationale of examining the test-retest reliability of a computerized neurocognitive testing battery.
The review will start with a historical overview of definition and epidemiology of concussion,
followed by pathophysiology, signs and symptoms, evaluation and management, overview of
neuropsychological testing for concussion, and finally an overview of test-retest reliability and
validity of computerized neurocognitive test batteries.
Historical Overview of Definitions
The study of brain damage has a long history. The physicians of the Hippocratic school
observed in ca. 400 B .C that an incised wound in one temple produces a spasm in the opposite
side of the body (Chadwick & Mann, 1950). The term “concussion” originates from the Latin
stem of concutere (to dash together, to shake violently) or concussus (action of striking together)
and it does not include any clinical or pathological consequences (Pearce, 2008). In the 16th
century, its symptoms were first described in a systematic fashion by Berengario de Carpi and
Ambrose Paré, who are often credited with popularizing the term “concussion”(Levin, Benton,
& Grossman, 1982). The term commotio cerebri was used to describe the effects of brain injury
without the presence of a skull fracture, and considered different from other traumatic effects on
the cerebrum because of its little intensity and short duration (Denny-Brown & Russell, 1941).

11

In 1966, Congress passed the Traumatic Brain Injury Act of 1966, and the term traumatic
cerebral concussion was introduced into federal law. Since then the term cerebral concussion
has been used interchangeably with mild traumatic brain injury (Maroon et al., 2000).
Defining concussion has been a difficult task because of the complex nature of symptoms
and cognitive impairments associated with this injury. Even though no consensus on the exact
definition of sport-related concussion has been made to date, it has evolved over time as
scientific researchers have been studying the nature of this injury. For example, historically,
concussion involved a loss of consciousness (LOC) and it was a requirement in order for a
concussion to be diagnosed. However, studies have shown that LOC is not as common as once
reported (Guskiewicz, Weaver, Padua, & Garrett, 2000), with only approximately 9% of all
concussions resulting in loss of consciousness. The American Academy of Neurology (AAN)
defines concussion as a traumatically induced alteration in mental status (e.g., confusion,
amnesia) that may or may not include a loss of consciousness (American Academy of Neurology,
1997). In 2001, the Concussion in Sport (CIS) group, a panel of medical experts including
neurologists, neuropsychologists, and athletic trainers was assembled with the purpose of
forming a basis of a comprehensive systematic approach for managing sports-related
concussions by evaluating current literature. The first International Conference on Concussion in
Sport was held in Vienna in 2001 and concluded that a concussion may or may not involve LOC.
Since the inaugural conference, the CIS group has held two more conferences in Prague in 2004,
and most recently in Zurich in 2008. Through the three conferences, a concussion has been
consistently defined as “a complex pathophysiological process affecting the brain, induced by
traumatic biomechanical forces”. After the first meeting, the CIS group published five common
features of concussion that incorporate clinical, pathological, and biomechanical constructs to

12

supplement the definition of this injury. These defining features of sport-related concussion
include: 1) concussion can be caused by direct impact to the head, face, neck, or elsewhere on
the body with an “impulsive” force transmitted toward the head; 2) concussion typically results
in the rapid onset of short-lived impairment of neurological function that resolves spontaneously;
3) concussion may result in neuropathological changes, but the acute clinical symptoms largely
reflect a functional disturbance rather than structural injury; 4) concussion results in a graded set
of clinical syndromes that may or may not involve LOC; and 5) concussion is typically
associated with grossly normal structural neuroimaging studies (Aubry et al., 2002). These
features of a concussion have been consistent through the three conferences; however, the
statement regarding neuroimaging has changed slightly. In the first and second conference, there
was a statement “concussion is typically associated with grossly normal structural neuroimaging
studies”, which was changed to “no abnormality on standard structural neuroimaging studies is
seen in concussion” in the third conference (Aubry et al., 2002; McCrory et al., 2005; 2009.).
Epidemiology of Concussion
In this section, the most recent prevalence and incident estimates for concussion in high
school and collegiate athletics will be reviewed in order to identify the population at risk of
sports-related concussion. Incidence of sports-related concussion has been reported through
epidemiological studies that used Reporting Information Online (RIO) for high school
population and National Collegiate Athletic Association (NCAA) Injury Surveillance System
(ISS) for collegiate population. In many studies athlete exposure (A-E), an exposure defined as
one athlete playing in one game or practice, is used in order to study the actual injury rate, rather
than percentages. Injury rates are a good measure of injury because of the differences in the
number of players and practice sessions and games in different sports. According to the National

13

Federation of State High School Associations (NFHS) and NCAA, athletic participation rates
continue to rise in both high school and collegiate population with 7,667,955 and 444,077
athletes competing, respectively (NFHS, 2011 & NCAA, 2011). As a result, the incidence of
sport-related concussion is also expected to rise. Sports are second to motor vehicle crashes as
the leading cause of traumatic brain injury (TBI), and it has estimated that 1.6 to 3.8 million mild
TBIs occur in sports and recreational activities annually among people aged 15 to 24 years
(Langlois, Rutland-Brown, & Wald, 2006). In the 1990’s, the estimate was approximately
300,000 because only TBIs associated with LOC was included at that time (Langlois, RutlandBrown, & Wald, 2006).
Epidemiological studies have reported several factors that influence the risk of sustaining
a concussion, such as the competition level (high school vs. college), sports setting (competition
vs. practice), and sex (female vs. male). The remaining portion of this section will review those
epidemiological studies.
High School vs. College
Epidemiological studies have been conducted in both high school and collegiate
populations. A very recent study conducted by Marar and colleagues (2012) examined injury
data from a large, nationally dispersed sample of US high schools (n=100) during the 2008-2010
academic year. The researchers investigated the epidemiology of concussions in high school
athletes. They utilized the High School RIO, an Internet-based sports injury surveillance system,
and reported 1,936 concussions and 7,780,064 A-Es. Results of this study indicated that
approximately 2.5 concussions per 10,000 A-Es occur at the high school level (Marar et al. 2012).
For collegiate population, Hootman and colleagues (2007) utilized the NCAA ISS to
summarize 16 years (1988 to 2004) of injury data from 15 NCAA sports. For concussion, the

14

researchers found incidence rates that range from 1.7 (1988-89, 1989-90, 1992-93) to 4.1 (200001, 2001-02) per 10,000 A-Es. A significant finding of this study found that from the 1988-89 to
2003-04, the injury rate doubled (1.7 to 3.4 per 10,000 A-Es) with an average annual increase of
7.0% of concussions reported.
While the previous two studies examined either high school or college athletes, Gessel
and colleagues (2007) examined the injury data with both populations. Athletes from 100 US
high schools and 180 US colleges that used to compare the rate of concussion between high
school and collegiate athletes during 2005-2006 academic year. The concussion rate was 4.7 and
6.1 per 10,000 A-Es in high school and college athletes, respectively. However, concussions
represented a higher proportion of all injuries sustained in high school athletes (8.9%, 396
concussions /4,431 total injuries) compared to collegiate athletes (5.9%, 482 concussions/8,293
total injuries).
These data suggest that collegiate athletes may have slightly higher injury rates than high
school athletes, but concussions most likely represent a greater proportion of injuries at the high
school level. It is worth pointing out that there are differences in the access to the medical
professionals between high school and college settings. The higher rate of concussions at the
collegiate level may be due to the fact that collegiate settings tend to have greater access to
sports medicine professionals, which lead to more opportunity to diagnose concussions rather
than differences in actual number of concussions.
Practice vs. Game
Several studies have also examined concussion incidences and concussion rates in
practice and competition. In general, both concussion incidence and rates were consistently
higher in competition than in practice with a few exceptions. Marar et al. (2012) reported that

15

among 1,936 concussions reported from a nationally disperse sample of US high schools, 1,289
(66.6%) occurred in competition while 647 (33.4%) concussions were reported during practice.
In the same study, injury rate was also higher in competition (6.4 per 10,000 A-Es) than practice
(1.1 per 10,000 A-Es) for all sports except cheerleading which had a higher injury rate in practice
than in competition (1.2 versus 1.4 per 10,000 A-Es).
A higher concussion incidence rate in competition than practice was observed in the
study of Gessel and colleagues (2007). In the high school setting, 5.3 concussions per 10,000 AEs in competition and 1.1 concussions per 10,000 A-Es in practice were reported, whereas 10.2
concussions per 10,000 A-Es in competition and 2.8 concussions per 10,000 A-Es in practice
were reported in the college setting.
While the comparison of injury rate between competition and practice was a part of
outcomes in many studies, one study was conducted that specifically compared the practice and
competition injury rates among high school athletes (Rechel et al. 2008). Rechel et al. (2008)
found that concussions account for a greater proportion of competition injuries compared to
practice injuries (12.0% vs. 5.9%, proportion ratio=2.02). This trend became evident in certain
sports, especially in boys’ soccer (15.6% vs. 2.3%, PR=6.94) and girls’ basketball (19.0% vs.
3.3%, PR=5.83). The researchers thought this trend is due to the increased amount of physical
contact that takes place during competition. Softball and volleyball players were the only
exceptions to this trend. In softball and volleyball, concussions constituted a greater proportion
of practice injuries than competition injuries (PR=.05: 95%CI=.01-.44 for softball, PR=.36:
95%CI=.06-2.24 for volleyball).
Type of Sports
Regardless the competition level and sports setting, certain sports have been identified as

16

high risk sports for concussion in terms of: higher number of concussion incidence, higher
concussion rates, and greater proportion that concussion represents in total injuries across
various sports. Of all sports played in the United States, footbadll consistently is associated with
the greatest number of sports-related concussion (ie., prevalence of concussion) at the high
school and college levels, ranging from almost 50% to as high as nearly 75% of total
concussions reported (Hootman, Dick, & Agel, 2007; Lincoln et al., 2011). Moreover, between
the 1982 to 1983 season and the 2007 to 2008 season, a total of 35,641,573 high school athletes
and 1,929,069 collegiate athletes competed in American football (Cantu & Mueller,
2009).Considering the intensity of football, as well as the fact that football has the largest
number of participants, it is not surprising that football has the highest number of concussions in
both high school and collegiate settings.
In the college setting, Hootman and colleagues (2007) reported 9,150 concussions over a
16-year period through the NCAA ISS, and fall football accounted for 48.1% of total
concussions (4404/9150). They also reported that spring football was associated with the second
greatest number of concussions (612/9150) followed by women’s soccer (593/9150), men’s ice
hockey (527/9150), and men’s soccer (500/9150). Even though women’s ice hockey had the
highest injury rates (9.1 concussions per 10,000 A-Es), only 79 concussions were reported,
which accounted for 0.9% of the total concussion reported.
In the high school setting, Marar and colleagues (2012) reported 1,936 concussions with
47.1% (912/1936) of the total concussions occurring in football, followed by girls’ soccer (8.2%,
159/1936), boys’ wrestling (5.8%, 112/1936), and girls’ basketball (5.5%, 107/1936). No
concussions were reported from boys’ volleyball. However, sports high in actual number of
concussions does not necessarily mean those sports have a higher risk of concussion than other

17

sports. Therefore, injury rate must be compared to discuss the risk of concussions across the
sports.
In terms of injury rate, several studies have suggested that women’s ice hockey has the
highest injury rate of concussion at the college level (9.1 per 10,000 A-Es) (Gessel, Fields,
Collins, Dick, & Comstock, 2007; Hootman et al., 2007; Lincoln et al., 2011). However, it must
be noted that the NCAA ISS data collection for women’s ice hockey began in 2000 to 2001 and
was only studied over a 3 year period while the other sports were studied over a 16 year period.
This may be a reason for the higher injury rate as well as the increased awareness of concussion
and improvement in diagnosis of concussion. In fact, the number of concussions reported was 79
and it was the second lowest among the all sports studied. If women’s ice hockey is excluded, in
the college setting , spring football had the highest incidence rate (5.4 concussions per 10,000 AEs) followed by men’s ice hockey(4.1) and women’s soccer (4.1), and in-season football (3.7)
(Hootman et al., 2007). It is worth mentioning that even though approximately 8 times more
concussions were reported from in-season football compared to ice hockey (4,404 vs. 527), the
incidence rate was higher in men’s ice hockey compared to in-season football (3.7 vs. 4.1). In
high school, football had the highest concussion rate with 6.4 concussions per 10,000 A-Es,
followed by boy’s ice hockey (5.4), and boys’ lacrosse (4.0) (Marar, McIlvain, Fields, &
Comstock, 2012). It must be noted that even though the total number of concussions reported
from boy’s ice hockey was only the seventh highest, the injury rate was the second highest after
football.
The total number of injuries varies across sports, which provides another insight into the
epidemiology of concussion. At the high school level, concussions represented the greatest
proportion of total injuries among boys’ ice hockey (22.2%) followed by girls’ lacrosse (21.1%) ,

18

cheerleading (20.3%), and football (number was not given but estimated around 17% from graph)
(Marar et al., 2012). In the collegiate setting women’s ice hockey (18.3%) was followed by
men’s ice hockey (7.9%), women’s lacrosse (6.3%), and in-season football (6.0%) (Hootman et
al., 2007). Along with the aforementioned caution regarding the data collection for women’s ice
hockey, it must be noted that Marar et al. (2012) did not include women’s hockey in their data
collection. By contrary, cheerleading, ranked third in the study by Marar et al. (2012) was not
studied in the study by Hootman et al. (2007).
When concussion data were compared across studies, the type and number of sports
studied become an important factor, which is a limitation for those studies that included only
sports that have relatively high concussion rates. The data from both studies of Gessel et al.
(2007) and Rechel et al. (2008) were limited to five boys’ sports: football, soccer, basketball,
wrestling, and baseball, and four girls’ sports: soccer, volleyball, basketball, and softball. The
study by Marar et al. (2012) revealed that the concussion rates in boys’ ice hockey and lacrosse,
and girls’ lacrosse were higher than the concussion rates in all the sports studied by Gessel et al.
(2007) and Rechel et al. (2008), excluding football. Also, the sports that have relatively low
concussion rate, such as swimming and diving, track and field, and gymnastics tend to be
omitted from studies. In order to obtain accurate epidemiological data for practice and
competition, researchers should not limit the study samples to athletes in major sports.
Sex Difference in Concussion Incidences
Title IX as part of the Equality in Education Act of 1972 triggered significant increase for
women to participate in sports, and its rise in participation is continuing. NCAA reported
between 1988 and 2004, women’s participation in sports increased by 80% while men’s
participation increased by only 20% (Hootman et al., 2007). During the 2010-11 academic year,

19

191,131 female athletes participated in NCAA sports (NCAA, 2011). In the high school setting,
NFHS reported that girls’ participation in the 2010-11 academic year was 3,173,549, which was
a record-breaking number. In addition this number represents a more than tenfold increase from
the 1971-72 academic year (NFHS, 2012). This increase in participation numbers resulted in
increased incidence of sports-related injuries in women, including concussion.
Epidemiological studies have been conducted in order to understand differences between
female and male athletes with respect to injury incidence, severity, and mechanism. Although a
significant debate exists over the influence of sex on incidence and severity of brain injuries,
general trends have shown that women are at a greater risk for sustaining concussions compared
to men (Covassin, Swanik, & Sachs, 2003a; Gessel et al., 2007; Hootman et al., 2007; Marar et
al., 2012).
During the 2 year study period of Marar et al. (2012), 1,432 concussions were reported
from boys’ sports with an injury rate of 3.1 per 10,000 A-Es, and 504 concussions were reported
from girls’ sports with an injury rate of 1.6 per 10,000 A-Es. It must be noted that more than
60% of concussions in boys’ sports were from football, which was the highest injury rate (912
concussions with 6.4 per 10,000 A-Es). However, when the data were summarized into sex
comparable sports: soccer, basketball, baseball/softball, swimming and diving, and track and
field, girls’ sports yielded more incidences of concussions as well as higher injury rates (336 vs.
235, 1.7 vs. 1.0 concussions per 10,000 A-Es).
The same trend was reported in the previously mentioned epidemiology study conducted
by Gessel and colleagues (2007). They reported that for the sports played by both sexes, girls’
sports had a higher rate of concussions and represented a greater proportion of total injuries than
boys (soccer: 3.6 vs. 2.2 concussions per 10,000A-Es and 21.5% vs. 15.4%; basketball: 2.1 vs.

20

0.7 concussions per 10,000 A-Es and 9.5% vs. 2.81%; softball/baseball: 0.7 vs. 0.5 concussions
per 10,000 A-Es and 5.5% vs. 2.9%).
A recent study by Lincoln et al. (Lincoln et al., 2011) examined the incidence and relative
risk of concussions. The researchers gathered injury data from 25 high schools in a large public
school system over an 11 year period (1997-2008) for 12 high school boys’ and girls’ sports.
During the 11 year period, 2,651 concussions were reported with 1,986 concussions from boys’
sports, and 665 concussions from girls’ sports. However, similar to what the other studies have
reported, when the data was summarized into sex comparable sports, girls had a higher rate of
concussion than their counterpart with the ratio of 2.1 in soccer, 1.7 in basketball, and 1.9 in
softball/baseball. Lacrosse was the only exception, in which girls had a lower concussion rate
compared to boys (2.0 vs. 3.0 per 10,000). The researchers thought this exception is due to the
different rules, protective equipment, and nature of play between boys’ and girls’ game. These
data suggest the presence of sex differences in the risk of concussion at the high school level,
with female athletes having a higher risk of sustaining a concussion. These sex differences have
also been explored in collegiate populations.
Covassin et al. (2003a; 2003b) examined injury data from NCAA ISS to compare sex
differences in the incidence of concussions among collegiate athletes during the 1997-2000
academic years. During the 3 academic years, 304 concussions were reported from women’s
sports and 254 concussions were reported from men’s sports. They found that concussions
represented a greater portion of total injuries for the women’s sport when they were compared
with their male counterpart: lacrosse (13.9% vs. 10.1%), soccer (11.4% vs. 7.0%), and basketball
(8.5% vs. 5.0%) (Covassin et al., 2003b). Also, for these three sports, female athletes were found

21

to be at a greater risk for sustaining a concussion during games than male athletes (Covassin et
al., 2003a).
Similar results were reported from a study conducted by Hootman et al. (2007), which
summarized 16 years of NCAA injury data in order to identify modifiable risk factors. The
researchers reported a greater percentage of total injuries and higher injury rates in women’s
sports compared with their male counterpart: lacrosse (6.3% vs. 5.6%, 2.6 vs. 2.5 concussions
per 10,000 A-Es), soccer (5.3% s. 3.9%, 4.1 vs. 2.8), basketball (4.7% vs. 3.2%, 2.2 vs. 1.6), and
softball/baseball (4.3% vs. 2.5%, 1.4 vs. 0.7). The data clearly suggest that female athletes have a
higher risk of sustaining concussions compared to male athletes.
There are several possible reasons for these sex differences. Females have smaller head
mass and weaker dynamic stabilization of the neck and head compared with males (Mansell, et
al., 2005; Tierney et al., 2005; Tierney et al., 2008). Less head mass and neck strength should
result in greater head accelerations upon force application based on Newton’s second Law of
Motion (F=ma); therefore, researchers have proposed strengthening the neck muscles as a
method for decreasing the risk of sustaining concussions (Cross & Serenelli, 2003; Viano,
Casson, & Pellman, 2007).
Major limitations of the previous epidemiology studies are (1) number and type of sports
included, which has been previously mentioned and (2) reliability of using AE. Researchers need
to be careful when they use AE especially in high-contact sports such as football because the
chance of sustaining a concussion will be different between a full contact practice and a walk
through practice even though they might be counted equivalently as one practice. Finally, it is
also worth mentioning the underreported nature of this injury. Many individuals including
athletes suffering from mild or moderate traumatic brain injury do not seek medical advice. For

22

example, a study reported that 47.3% of high school football players confidentially reported their
symptoms after a blow to the head due to the reasons such as: 1) a player thinking the injury was
not serious enough to seek medical care, 2) motivation to stay in competition, and 3) lack of
awareness of the symptoms of concussion (McCrea, Hammeke, Olsen, Leo, & Guskiewicz,
2004). Therefore, the published epidemiological data for sport-related concussion are thought to
be a conservative estimate (Langlois et al., 2006; McCrea et al., 2004).
Pathophysiology
Dynamic loading caused by either direct impact to the head or a sudden movement of
the head produced by an impact elsewhere causes acceleration forces that can damage the brain,
which is the most common mechanism of head injury (Almasi & Wilson, 2012). The underlying
injuries resulting from head trauma can be classified into the primary injury, which is a result of
the direct mechanism at the time of incident, and the secondary injury, which is combination of
systemic extracranial insults and intracranial physiological and biomechanical changes (Giza &
Hovda, 2001). In this section, the secondary injury, specifically the neurometabolic cascade
following concussion is reviewed.
A complex cascade of ionic, metabolic, and physiologic events occurs immediately after
cerebral concussion. The primary elements are: binding of glutamine to the N-methyl-Daspartate (NMDA), neural depolarization, abrupt ionic imbalance, increased triphosphate
production adenosine (ATP) demand, and cellular-energy crisis (Giza & Hovda, 2001).
Glutamine is the most abundant neurotransmitter in the brain and NMDA is negatively
associated with neurological outcome in the animal model (Blaylock & Maroon, 2011; Greve, &
Zink, 2009). Binding of glutamine to NMDA results in further depolarization of neurons with the
efflux of potassium and influx of calcium (Giza & Hovda, 2001). The neuronal cell damage from

23

excessive release of excitatory neurotransmitters is called excitotoxicity. Generation of high
levels of reactive oxygen species (ROS) and reactive nitrogen species (RNS), lipid peroxidation
products (LPPs), prostaglandins, and nitric oxide (NO) are involved in the process of
excitotoxicity, which is associated with microtubule dysfunction, membrane injury, dendritic
retraction, synaptic loss, mitochondrial dysfunction, calcium dysregulation, and apoptosis
(Blaylock & Maroon, 2011).
Calcium dysregulation plays a major role in excitotoxicity because calcium affects a wide
range of cellular functions. High mitochondrial calcium may result in swelling, depolarization,
and loss of function (Verweil et al., 2000) while the accumulation of calcium worsens the energy
crisis and even lead to cell death (Giza & Hovda, 2001). Generation of ROS and decreased
mitochondrial respiratory capacity may be caused by increased intracellular calcium (Nicholls,
2008). Abnormalities in calcium homeostasis may last as long as 30 days after traumatic brain
injury (Sun et al., 2008).
To restore the ionic balance, the sodium-potassium pump needs to work overtime
requiring increased amount of ATP, which leads to increased glucose metabolism. The need for
ATP is compromised by a decrease in cerebral blood flow that is from cerebral vasoconstriction
due to increased sympathetic outflow to the cephalic vasculature following the fluid-percussion
brain injury (Shibata, Einhaus, Schweitzer, Zuckerman, & Leffler, 1993). This tenuous
equilibrium between brain healing and the need for increased blood flow and glucose is called
“concussion penumbra”(Grady, 2010). Imbalanced glucose supply and demand also cause
cellular energy crisis. Along with impaired autoregulation and reactivity to vasomotor agents of
the brain, cellular energy crisis is a mechanism for the brain’s vulnerability to injury following
an initial trauma (Giza & Hovda, 2001).

24

Signs and Symptoms of Concussion
Common signs and symptoms of concussion have been discussed in sports-related
concussion literature. The signs and symptoms of concussion typically fall into 4 categories:
physical, cognitive, emotional, and sleep disturbances (Almasi & Wilson, 2012). Common
physical symptoms reported by concussed athletes include: headache, nausea, dizziness, fatigue,
vomiting, balance problems, and sensitivity to light and noise (Barth et al., 1989; Cantu, 1998a;
Maddocks & Dicker, 1989). Mental fogginess, difficulty in concentrating, decreased processing
speed, and memory difficulties are common cognitive symptoms (Maruta, Lee, Jacobs, & Ghajar,
2010). In addition to physical and cognitive symptoms, concussed athletes may experience
personality change, sadness, nervousness, and sleep disturbance (Guskiewicz et al., 2006; Lovell
et al., 2006; McCrory et al., 2005). Of all the common concussion symptoms, headache is the
most common symptom, with frequency between 40% and 86%, followed by fatigue, feeling
slowed down, drowsiness, and cognitive problems (Fazio, Lovell, Pardini, & Collins, 2007).
According to the American Academy of Neurology, some of the signs and symptoms may last
only minutes to hours (i.e., headache, dizziness and poor balance, nausea, vomiting); whereas,
other signs and symptoms may last days to weeks (i.e. anxiety and nervousness, irritability,
difficulty concentrating and remembering, disorientation, fatigue, light-headedness, light
sensitivity, tinnitus).
For many years, LOC and post traumatic amnesia (PTA) have been used as major onfield markers for concussion (McCrea, Kelly, Randolph, Cisler, & Berger, 2002), and most of the
grading scales used to determine grade and severity of concussion were based on these two
symptoms. LOC can be described as paralytic coma (Ommaya & Gennarelli, 1974), and
represents a state of brief coma in which the eyes are typically closed and the athlete is

25

unresponsive to external stimuli (Lovell, 2009). PTA may be associated with loss of memory for
events either preceding or following injury, which determines the amnesia as either retrograde or
anterograde amnesia (Lovell, 2009). PTA is common following LOC and sometimes occurs even
without LOC (Kelly, 2001). Historically, LOC and PTA have been a key component of
concussive injury. However, studies have shown that LOC and PTA may not be present in
concussion as often as it was once thought, and their role and significance in the detection and
management of concussion have changed. Guskiewicz and colleagues (2000) found that among
1,003 diagnosed concussions in college and high school football players, only 8.9 % and 27.7%
of concussions were associated with LOC and PTA, while headache was associated with 86% of
concussions, followed by dizziness (67%), confusion (59%), and disorientation (48%).
In addition to the lower prevalence of LOC and PTA, researchers have also found that
LOC and PTA have weak association with other signs and symptoms of concussion. Collins et
al. (1999) compared neurocognitive performance between groups of concussed patients: those
who experienced LOC at the time of injury, no LOC, or uncertain LOC. The results indicated
that regardless of the presence of LOC at the time of injury, all patients demonstrated poor
neurocognitive performance, and no significant difference across LOC groups. Furthermore,
Guskiewicz et al.(2001) assessed postural stability and neurocognitive function of 36 concussed
collegiate football athletes using the Sensory Organization Test, the Balance Error Scoring
System, and various neuropsychological tests: Trail-Making Test, Wechsler Digit Span Test,
Stroop Color Word Test, and Hopkins Verbal Learning Test. The results indicated that LOC and
PTA were not associated with increased deficits or slowed recovery on measures of postural
stability or neurocognitive function. Moreover, the most recent consensus statement from the
International Conference on Concussion in Sport stated that the presence or duration of PTA

26

alone may not be as important as the status of the clinical post-concussive symptoms. Also, the
significance of LOC was not referred to as exhibiting it or not, rather in its duration (if it is >1
minutes) that may result in modifying management (Mccrory et al., 2008).
In addition to the importance of post-concussive symptoms that were pointed out in
recent consensus statements, several studies have followed the post-concussive symptoms
longitudinally. Studies found that different signs and symptoms recovered in different manners,
but the trend has not been conclusive. One study with high school football players found that
memory impairment often lasted up to 10 days post injury while other cognitive functions,
reaction time, and processing speed, returned to baseline level on approximately 6 days post
injury (Sim, Terryberry-Spohr, & Wilson, 2008). Another study found that the concussed
athletes were asymptomatic at day 7 post injury even though verbal memory, motor processing
speed, and reaction time were still impaired (Covassin, Elbin, & Nakayama, 2010). Covassin et
al. (2010) also found that reaction time took the longest to return to baseline levels (up to 21 days)
while memory function needed the longest recovery time in the study by Sim et al.(2008) (10
days).
Another research effort was made to predict prognosis of concussion from initial postconcussion signs and symptoms. Lau and colleagues (2012) examined the injury data of 108
male high school football athletes in order to determine cutoff scores of symptom clusters
derived from the Post-Concussion Symptom Scale and a neurocognitive test scores (Immediate
Post-concussion Assessment and Cognitive Testing: ImPACT) that will predict a protracted
recovery (>14 days) in concussed athletes. They found statistically significant cutoff scores for
migraine cluster, cognitive cluster, visual memory, and processing speed at 75%, 80%, and 85%

27

sensitivity to predict protracted recovery. Moreover, migraine cluster includes headaches and
dizziness and the cognitive cluster includes fatigue, which is consistent with other researchers.
A cohort study by Collins et al. (2011) was similarly designed with the same population
as Lau et al. (2012), but used only on-field signs and symptoms as predictors for a protracted
(>21 days) versus a rapid (<7 days) recovery in concussed high school football players. On-field
signs and symptoms included: headache, fatigue, confusion, LOC, PTA, imbalance, visual
problems, personality changes, sensitivity to light and noise, numbness, vomiting, and dizziness.
The researchers found that only on-field dizziness was associated with increased risk of a
protracted recovery (OR=6.34).
A similar study was conducted in National Hockey League (NHL) players. Benson et al.
(2011) conducted a prospective longitudinal study during the 1997-98 to 2003-04 seasons to
examine the relationship between initial post-concussion signs and symptoms and time loss.
There were 559 physician-diagnosed, regular season, in-game concussions during the seven
years of study. The researchers found that headache and fatigue were significant predictors of
time loss of more than 10 days (odds ratio: 2.17 and 1.72, respectively). Even though they did
not reach the statistical significance at the 10 day cut-point, presence of amnesia and abnormal
neurologic examination (i.e. cerebellar examination, tamdem gait, rhomberg, etc.) were
significant predictors of time loss. The data suggest that some signs and symptoms may be worth
more attention than others in order to predict prognosis of concussion. However, when you
consider the number of research studies that have investigated how to diagnose concussions, very
little research has examined predicting prognosis in concussed athletes. Thus, more research with
different population and standardized methods are warranted to determine more accurate
prediction in concussion prognosis.

28

Efforts have been made to increase concussion awareness and knowledge. For example,
in 2005, the CDC launched the Heads UP program to help educate youth coaches, high school
coaches, parents, athletes, school administrators, and medical professionals, and its effectiveness
has been reported (Covassin, Elbin, & Sarmiento, 2012; Sarmiento et al., 2010). Covassin and
colleagues (2012) reported that the Heads Up material improved 77% of youth sports coaches’
ability to identify athletes who may have a concussion, and made 63% of youth sports coaches
view concussions as being more serious. In another study, Sarmiento et al. (2010) reported that
even though most of the coaches who responded to their survey study had more than 10 years of
experience coaching, more than one third (34%) of coaches reported that they learned something
new about concussions from the material. Also, 82% of coaches who implemented the materials
to educate athletes reported that the materials were very or extremely useful. These results are
significant because sport coaches may play an important role in detecting concussion as sports
medicine professionals may not always be present, especially at youth and high school settings.
The sports medicine professionals are responsible for detecting, evaluating, and managing
concussed athletes; however, cooperation from athletes and coaches is imperative for the best
result.
A significant amount of effort have been made to improve the management of concussed
athletes. These efforts will be truly rewarded when detection of concussion is improved as well.
However, even though self-report from athletes is a key component of detecting probable
concussions, the underreported nature of concussion has been reported (McCrea et al., 2004).
McCrea and colleagues (2004) reported that 47.3% of high school football players did not report
their symptoms after a blow to the head due to lack of knowledge about signs and symptoms of
concussion as well as motivation to stay in competition. Therefore, athletes may hide their signs

29

and symptoms of a concussion or may not be sufficiently aware of the signs, symptoms, and
potential catastrophic effects of concussion.
Detecting, evaluating, and managing concussed athletes is essential to prevent more
serious consequences. Second-impact syndrome (SIS) is described as diffuse cerebral swelling
leading to the rapid development of cerebral vascular congestion, which in turn causes increased
intracranial pressure that often results in brainstem herniation and death (Cantu, 1998). Although
very rare, SIS can occur if a concussed individual sustains a second head injury before their first
concussion fully resolves (Cantu & Voy, 1995; Cantu, 1998). A recent study analyzed the 30year (1980-2009) US National Registry of Sudden Death in Young Athletes , and reported 17
deaths of high school athletes who sustained a concussion shortly before fatal head trauma (SIS)
(Thomas et al., 2011).
There has been a claim that the term “second-impact syndrome” is misleading and the
syndrome should be referred to as “diffuse cerebral swelling” (McCrory, 2001; McCrory, Davis,
& Makdissi, 2012). There are several possible reasons for this rationale. First, the evidence for
SIS is all anecdotal case reports. Second, all 17 cases were reported in North America and no
case have been reported from other countries. Third, the brain of SIS patients often accompanies
a subdural hematoma, which is a structural injury. Lastly, no clear link between “second” impact
to death is established (McCrory et al., 2012). Further debate is expected. However, regardless
what the syndrome is called, there is a vulnerable window after the initial head trauma due to
metabolic dysfunction as was previously mentioned in the pathophysiology section of this review
of literature. Therefore, the early detection and proper management of sports-related concussion
are essential to reduce the risk of catastrophic consequences.

30

Evaluation and Management of Sports-Related Concussion
Due to concussions’ complex nature, the evaluation, management, and return to play
decisions are one of the biggest challenges for sports medicine professionals. Over the past two
decades, approximately 20 management guidelines have been published, which are largely based
on the opinions of groups of experts rather than on systematic research (Collins, 1999). However,
as a result of reevaluation and revision of prior guidelines through empirical studies, the
management and return to play strategies for sports-related concussion have been refined.
Specifically, the utilization of computerized neurocognitive test batteries, imaging techniques,
and individualized management recommendation have been incorporated in new management
guidelines. Furthermore, consensus statements have suggested the abolishment of historically
utilized concussion grading scales. In this section, both historical and current management
guidelines will be reviewed.
Historically, the severity of concussion has been assessed by grading or assigning a
numerical value such as Grade 1 (mild), Grade 2 (moderate), and Grade 3 (severe), and the
majority of return to play guidelines corresponds to the grading system. Clinicians suggested
grading scales to unify the concussion management. However, researchers have criticized this
uniformed management for several reasons, including lack of consensus, over-reliance on LOC
as a primary marker of severity, inability to account for individual difference in recovery rate and
symptom presentation, and lack of empirical support (Aubry et al., 2002; Collins, 1999; Kelly,
2001; McCrory et al., 2005). However, the common grading and return to play guidelines are
still worth being reviewed as they are important part of the history of concussion management
guidelines.

31

The AAN (1997), Cantu’s Grading Scales (1986; 2001), and the Colorado Medical
Society Guidelines (1994) have become the most recognized guidelines in sports-medicine
professionals. All three grading scales have LOC and PTA as the primary markers of concussion,
although the duration of LOC and PTA differs across the guidelines. The grading scales are
summarized in Table 1.
Typically grading scales also have their own return to play guidelines that are based on
their grading system (see table 2). It must be noted that the AAN, Cantu’s, and Colorado Medical
Society Guidelines permit mildly concussed athletes to return to the same game/practice if they
did not experience any LOC or PTA, the symptoms are resolved on the sideline within 15
minutes, and they successfully complete all sideline mental assessments. In addition to the study
that reported the low prevalence of LOC and PTA in concussion (Guskiewicz, Weaver, Padua, &
Garrett, 2000), and their weak association with other signs and symptoms of concussion
(Guskiewicz, 2001; Collins et al., 1999), studies have questioned the return to play guidelines
that are based on the grading systems over-reliance on LOC and PTA.
Leiniger et al. (1990) compared neuropsychological outcomes between minor head injury
patients with and without LOC in order to investigate the relative impact of LOC on the
development of neuropsychological dysfunction after head injury. The researchers administered
eight neuropsychological tests to patients who were assigned into 3 groups based on their level
of consciousness immediately after the injury: patients who experienced LOC (concussion group,
n=31), patients who experienced disorientation or confusion but no LOC (mild concussion group,
n=22), and a demographically matched control group (n-=23). No significant difference between
concussion group and mild concussion group were found on any of the tests, while mild
concussion group scored significantly poorer than the controlled group on five of the eight tests.

32

This study suggested that LOC has no impact on distinguishing patients at greater or reduced risk
for neuropsychological outcomes.
Similarly, Lovell et al. (1999) reported the failure to find any relationship between LOC
and neuropsychological function in their study with 383 concussed patients who were assigned
to: LOC, no LOC, and uncertain LOC groups based on their level of consciousness at the time of
injury. Neuropsychological tests, including the visual reproduction, digit span, and logical
memory subtests of the Wechsler memory scale, the Trail Making test, Wisconsin Card Sorting
test, Hopkins Verbal Learning test, Controlled Oral word Association, and the Galveston
Orientation and Amnesia test were administered. While mildly decreased performance on speed
of information processing, attention process, and memory in all three groups were observed, no
significant differences were found between the groups in any of the test results. The results of
this study added more evidence to the question on the significance of LOC as a predictor of
severity of concussion.
While the presence of LOC as a predictor of concussion severity has been questioned,
Collins et al. (2003) found that the presence of amnesia is predictive of symptoms and
neurocognitive deficits following concussion. A total of 78 concussed athletes were assigned into
good post-injury presentation group (no measurable change relative to baseline, n=44) and poor
post-injury presentation group (10 point increase in symptom reporting and 10 point decrease in
memory functioning, n=34) based on ImPACT memory and symptom composite scores 2 days
after injury. The value of on-field disorientation, PTA, retrograde amnesia, and LOC was
evaluated as predictor variables for determining whether the concussed athletes fall into good or
poor post-injury presentation group. The results showed that athletes who exhibited retrograde
amnesia and PTA were over 10 and 4 times more likely to demonstrate poor presentation,

33

respectively. On the other hand, experience of LOC or brief disorientation (<5 minutes) did not
predict the presentation group. The result of this study indicated that refinement of sports
concussion grading scales that heavily rely on LOC is necessary.
In addition to the researches that questioned the usage of LOC to determine and predict
the severity of sports-related concussions, Lovell et al. (2004) evaluated a commonly used
grading scale itself, not its components, in terms of the athletes’ neurocognitive functions
measured by a computerized neuropsychological testing battery (ImPACT). The researchers
followed the neuropsychological functions of high school athletes with Grade 1 concussions
during the first week of recovery. A series of ImPACT tests were administered to 43 athletes
diagnosed with a Grade 1 concussion based on AAN grading scale at 36 hours and 6 days postinjury to compare to preseason baseline testing. At 36 hours post-injury, statistically significant
decline in memory (P < .003) and increase in self-reported symptoms (P < .00001) compared to
baseline performance were observed. Also, even though statistically non-significant, subtle
declines on reaction time and processing speed at 36 hours post-injury were reported. At the
second follow up, no significant difference in memory compared to the baseline, and
significantly fewer symptoms than the first follow up were reported. This study suggested that
the return to play guideline for Grade 1 concussion accompanies the risk of premature return.

34

Table 1. Grading system from AAN (1997), Cantu (1986&2001), and CMS (1994)
Severity of Grade
Guideline
Cantu
(1986)

1 (mild)
(1) No LOC

2 (moderate)
(1) LOC < 5 min

(2) PTA < 30 min

OR

3 (severe)
(1) LOC > 5 min
OR

(2) PTA > 30 min
Cantu
Revised
(2001)

(1) No LOC
(2) PTA* or postconcussion signs and
symptoms < 30 min

(2) PTA > 24 hrs

(1) LOC < 1 min

(1) LOC > 1 min

OR
(2) PTA* > 30 minutes,
but < 24 hrs

OR
(2) PTA* > 24 hrs
OR
(3) Symptoms > 7 days

(1) Confusion with
amnesia.
(2) No LOC

(1) Transient confusion

(1) Transient confusion

(2) No LOC

(2) No LOC

(3) Concussion symptom
or mental status change
resolves in < 15 min.

AAN

(1) Confusion without
amnesia
(2) No LOC

CMS

(3) Concussion symptom
or mental status change
resolves in > 15 min.

*retrograde or anterograde

35

(1) LOC (any duration)

(1) LOC (brief or
prolonged)

Table 2. Return to play guideline from AAN (1997), Cantu (1986&2001), and CMS (1994)
Severity of Grade
Guideline

1(mild)

2(moderate)

3(severe)

Cantu
(1986)

Athlete may return to play
that day in select situations
if normal clinical
examination at rest and
exertion. If symptomatic,
athlete may return to play
in 7 days.

Athlete may return to
play in 2 wks if
asymptomatic at rest
and exertion for 7 days.

Athlete may return to play
in 1 mo if asymptomatic
at rest and exertion for 7
days.

Cantu
Revised
(2001)

1st concussion: Return to
play if asymptomatic for
1wk
2nd concussion: Return to
play after asymptomatic
for 1wk
3rd concussion: Minimum
of 1 month; may return to
play if asymptomatic for
1wk

Terminate season; may
return to play next season
if asymptomatic

CMS

Remove athlete from
contest and evaluate
immediately and every 5
min. Permit athlete to
return if amnesia or
symptoms do not appear
for 20 min.

1st concussion: Return
to play in 2 wks if
asymptomatic for 1wk
2nd concussion:
Minimum of 1mo; may
return to play if
asymptomatic for 1wk;
consider terminating
season
3rd concussion:
Terminate season; may
return to play next
season if asymptomatic
Remove athlete from
contest and disallow
athlete to return. Permit
athlete to return to
practice after 1 wk if
asymptomatic.

AAN

Examine athlete
immediately for mental
status changes. Return to
contest if no symptoms or
mental status changes at 15
min.

Remove athlete from
game and disallow to
return. Athlete can
return in 1 wk if
asymptomatic.

Transport athlete to
hospital. Permit athlete to
return to play if
asymptomatic 1 wk (brief
LOC) or 2 wks
(prolonged LOC).

*retrograde or anterograde

36

Transport athlete to
hospital. Perform
neurological examination
and observe overnight.
Permit athlete to return to
play after 2 wks if
asymptomatic.

Sports-Related Concussion Consensus and Position Statements
The criticism and shortcomings of concussion management guidelines have been
addressed in more recent consensus papers and position statements (Aubry et al., 2002;
Guskiewicz et al., 2004; McCrory et al., 2005;2009). In 2002, a group of physicians,
neuropsychologists, and sports administrators organized the 1st International Conference on
Concussion in Sport in Vienna, Austria. The Concussion in Sport (CIS) group discussed key
elements to concussion evaluation and management in order to try and reduce the morbidity
secondary to sports-related concussion. One of the most important conclusions of the meeting
was that the previously published concussion management guidelines were not adequate to
assure proper management of every concussed athlete (Aubry et al. 2002). Moreover, this group
advocated for the abolishment of concussion grading scales.
Regarding return to play on the same day of a concussion, the CIS group concluded that
the concussed athlete who is still demonstrating any signs and symptoms of concussion should
not be allowed to return to play in the current game or practice regardless the presence of LOC
and PTA (Aubry et al., 2002; McCrory et al., 2005; 2009). Concussed athletes not being
recommended to return to play on the same day of their concussion was also agreed by the
position statement of National Athletic Trainers’ Association (NATA) on management of sportsrelated concussion (Guskiewicz et al., 2004).
The CSI and the NATA position statements similarly recommended a medically
supervised step-wise return-to-play process (Aubry et al., 2002; Mccrory et al., 2005; 2009). The
recommended return-to-play step-wise process following a concussion includes: 1) No activity,
complete physical and cognitive rest until asymptomatic, 2) Light aerobic exercise such as
walking or stationary cycling at less than 70% of predicted max heart rate, no resistance training,

37

3) Sport specific exercise without head impact, 4) Non-contact training drills and possible load
progressive resistance training, 5) Full contact training after medical clearance, and 6) Return to
play (McCrory et al. 2009). These stages are based on the possibility of symptoms returning
from progressive physical and sport-specific exertion (Guskiewicz et al., 2004). Injured athletes
are recommended to continue to proceed to the next level if they remain asymptomatic at the
current stage. If concussion symptoms return, the athlete should go back to the previous
asymptomatic stage and resume the progression after 24 hours of physical and cognitive rest
(Mccrory et al.,2009). Unlike the old return to play guidelines, this step-wise guideline has
flexibility to respond to individual differences in recovery from concussion.
Consensus statements and position papers have stressed that sports medicine
professionals should take a multidisciplinary approach when managing concussion (Guskiewicz
et al., 2004; Mccrory et al.,2009.). Along with objective balance assessment, neuroimaging,
genetic testing, the value of neuropsychological testing in concussion evaluation and
management has been shown to be effective (Meehan, d’ Hemecourt, Collins, Taylor, &
Comstock, 2012). Recently, a number of computerized neuropsychological testing programs
have been designed, and their clinical application has become common practice. In the next
section, an overview of neuropsychological tests in concussed athletes, including reliability and
validity of those testing programs will be reviewed.
Overview of Neuropsychological Testing for Concussion
Neurocognitive assessment is a crucial element of multidisciplinary concussion
management as it provides sports medicine professionals with objective information that cannot
be obtained from self-reported symptoms and post-injury behavior. Neurocognitive assessments
have also been suggested to help determine the long-term cognitive recovery from concussion

38

(Guskiewicz et al., 2004). Neurocognitive assessment batteries evaluate multiple aspects of
cognitive function. For example, researchers have found decreases in psychomotor speed,
attention and concentration, working memory, mental set shifting, information processing speed,
reaction time, executive functioning, and memory after an individual incurs a concussion
(Collins, 1999; Echemendia, Putukian, Mackin, Julian, & Shoss, 2001; Erlanger, Kaushik, et al.,
2003; Iverson, Lovell, & Collins, 2005). Researchers have also found that some cognitive
functions may be more sensitive to the immediate effects of concussion than others. A meta
analysis conducted by Beleanger and Vanderploeg (2005) reviewed 21 studies and 790 reported
concussions in order to examine the impact of sports-related concussion across six cognitive
domains: orientation, attention, executive functioning, memory acquisition, delayed memory,
and global cognitive ability. Results suggested that delayed memory, memory acquisition, and
global cognitive functioning were most sensitive to the immediate effects of concussion.
The pre-participation baseline assessments followed by a series of post-concussion
assessments have become the current standard of concussion management. Researchers have
strongly suggested to employ baseline testing as it enables the sports medicine professionals to
compare the concussed athletes’ current cognitive function to his/her own baseline cognitive
function in order to track recovery from concussion, which makes the concussion management
more individualized (Guskiewicz et al., 2004). One of the advantages of employing baseline
testing is its ability to minimize possible confounding factors such as age and sex, which could
exist if normative data is used. In addition, employing baseline testing allows the sports medicine
professionals to take an individual’s cognitive strength and weakness into account so that post
injury deficits can be differentiated from pre-morbid weakness (Bernhardt, 2000). Currently, the
use of baseline and serial post-concussion testing are implemented at the high school, collegiate,

39

and professional level as a key component of a comprehensive approach to sports-related
concussion management to ensure the athletes’ safe return to play.
Advancement of neurocognitive assessment for sports-related concussion from traditional
paper and pencil tests to computerized neuropsychological testing enabled the use of baseline
and serial post-concussion testing for large number of athletes. The rest of chapter will review
the traditional paper and pencil test, and several computerized neuropsychological test batteries.
In addition, sensitivity, specificity, and test-retest reliability of each test will be summarized. For
purposes of this chapter, sensitivity is defined as the probability that a test result will be positive
when a concussion is present, and specificity is defined as the probability that test result will be
negative when a concussion is not present (Schatz, Pardini, Lovell, Collins, & Podell, 2006).
Test-retest reliability is defined as the extent to which scores on the battery remain stable over
time (Randolph, McCrea, & Barr, 2005).
Paper and Pencil Neurocognitive Tests
Traditional neuropsychological assessment of concussion employs paper and pencil
measures. A study by Barth et al. (1989) was one of the first to prospectively assess
neurocognitive performance in concussed athletes. Four paper and pencil neurocognitive tests
including the Paced Auditory Serial Addition Test, the Trail-Making Test A and B, and the Digit
Symbol Test were administered to approximately 2,300 college football players. The same four
tests were re-administered within 24 hours, 5 days, and 10 days post injury to all concussed
athletes (n=183), and 48 age-matched non-injured controls. Results showed decreases in auditory
attention and visuomotor speed at 24 hours post injury in concussed athletes, however, these
deficits were resolved by 5 days post injury. In spite of its limitation of including non-athletes as
controls, this study was considered a hallmark study because it established the use of an athlete’s

40

pre-season baseline levels of cognitive performance for comparison to post-concussion levels,
which influenced the later sports-related concussion researchers.
After the hallmark study by Barth et al. (1989), other researchers prospectively assessed
cognitive recovery from sports-related concussion. Collins et al. (1999) prospectively
administered a battery of eight paper-and-pencil neurocognitive tests to 393 collegiate football
players in order to evaluate post-concussion recovery. The testing battery included the Hopkins
Verbal Learning Test (HVLT, verbal learning and delayed memory), the Trail-Making Tests A
and B (visual scanning and executive functioning), the Digit Span Test (attention and
concentration), the Symbol Digit Modalities Test (information processing speed), the Grooved
Pegboard Test for dominant and non-dominant hands (bilateral fine motor speed), and the
Controlled Oral Word Association Test (CWAT, word fluency). Follow-up testing was
administered to16 concussed football players within 24 hours of injury, 3, 5, and 7 days post
injury. Results revealed significant impairments in verbal learning and delayed memory, and
moderate declines in executive function and processing speed were found at 24 hours and 3 days
post injury. Concussed athletes in this study were found to have their cognitive decrements and
post-concussion symptoms return to baseline levels by 5 to 7 days and 3 to 5 days, respectively.
While paper and pencil assessments have served an important role in sports-related
concussion management, some problematic issues in both clinical and research have been
identified. Generally, paper and pencil assessments are time consuming and costly, and also
require a trained on call clinical professional for proper administration and evaluation (Collins &
Hawn, 2002). Furthermore, these assessments were susceptible to learning effects (Hinton-Bayre,
Geffen, Geffen, McFarland, & Friis, 1999), and its sensitivity to concussion have been
questioned when brief batteries are administered, which is common in sports medicine in order

41

to test the entire athletic teams (McCrea et al., 2005). Specifically, McCrea et al. (2005) reported
that a brief test battery that included the Hopkins Verbal Learning Test, the Trail Making Test
Parts A and B, the Symbol Digit Modalities Test, the Stroop Color Word Test, and the
Controlled Oral Word Association Test was reported to be 23% sensitive to concussion when
administered two days post injury. As a result of the above mentioned issues, paper and pencil
assessment was replaced by computerized neurocognitive assessment program for sports related
concussion management.
Computerized Neuropsychological Assessment of Concussion
In order to overcome the weaknesses in traditional paper and pencil neuropsychological
assessment of concussion, several computerized neuropsychological assessment and symptom
evaluation systems have been developed including the Automated Neuropsychological
Assessment Metrics sports medicine battery (ASMB), CogSport, Concussion Resolution Index
(CRI), and the Immediate Post Concussion Assessment and Cognitive Testing (ImPACT). These
batteries assess reaction time as well as several domains of neuropsychological function
including working memory, attention, concentration, visual spatial, and short-tem verbal and
non-verbal memory.
There are several advantages in using computerized neuropsychological assessments over
the traditional paper and pencil tests. First, the test administration, data collection, and
comparison are easier than the traditional assessments. Second, computerized
neuropsychological batteries offer increased alternate forms through randomization of items
which decreases any learning effects. Third, test takers can complete the task more efficiently
due to less frustration and stronger likelihood of engaging and maintaining interest (Schatz &
Zillmer, 2003). Finally, computerized assessments are able to detect slight differences in the

42

patients’ performance (ie., reaction time) that is not possible by a human examiner (Schatz &
Zillmer, 2003). However, computerized assessments also have its disadvantages such as creating
a false sense of ability that anyone can diagnose neuropsychological deficits, encouraging a
passive stance rather than taking an active role in the process, and inability to assess efferent
language or verbal modalities (Schatz & Putz, 2006). Despite these disadvantages, computerized
neuropsychological assessments have gained popularity in the past decade and several
computerized neurocognitive testing programs have been available commercially for concussion
management. The rest of this section will review each computerized neuropsychological testing
battery.
Automated Neuropsychological Assessment Metrics (ANAM)
ANAM is the standard clinical subset of the Office of Military Performance Assessment
Technology’s Tester’s Workbench. It is a computer based cognitive assessment tool designed to
detect the accuracy and efficiency of cognitive processing in variety of situations (Levinsons &
Reeves, 1997). Test administration takes approximately 20 minutes to complete. Originally,
ANAM was introduced by the US Department of Defense for military research to examine the
cognitive performance of soldiers who were exposed to extreme environmental stressors or
neurotoxins, such as 30 days of undersea missions, fatigue during desert storm bomber missions,
and being wounded with depleted uranium bullets (Levinsons & Reeves, 1997). A study that
collected neuropsychological data using ANAM from 961 soldiers pre- and post-deployment to
Iraq reported that post-deployment soldiers exhibited cognitive deficits in sustained attention,
verbal learning, and visual spatial memory along with significantly higher confusion and tension
(Vasterling et al., 2006).

43

Levinson and Reeves (1997) investigated the sensitivity of ANAM in patients with
acquired brain injury. Participants in this study include 22 individuals with a motor vehicle
accident and two who suffered from a vascular accident undergoing a traumatic brain injury
rehabilitation program. Participants had been classified as marginally impaired (n=8), mildly
impaired (n=7), or moderately impaired (n=7) based on their performances on
neuropsychological evaluations (Cognitive Assessment System) and observations of an
individual’s interactions in social settings as well as personal interviews. Then ANAM was
administered two times separated by a 2 to 3 month interval to examine its ability to classify the
patients by comparing with the original classification (impaired, mildly impaired, and
moderately impaired). At the first administration, the results of ANAM classified 91% of the
patients into groups accurately. Furthermore, classification rate was 100% when mild and
moderately impaired patients were combined as one group. At the second administration, the
results of ANAM classified 86.36% of the patients into groups accurately. This study revealed
the ability of ANAM in distinguishing the severity of acquired head injury.
The construct validity of ANAM has been supported by a study done by Bleiberg and
colleagues (2000). In this study, the relation between ANAM and a set of traditional clinical
neuropsychological tests (Trail Making Test part B, Consonant trigrams total score, Paced
Auditory Serial Addition test, the Hopkins Verbal Learning test, and the Stroop Color-Word test)
were examined. The strongest correlation for mathematical processing (MTH), Sternberg
memory procedure (STN), and spatial processing (SPD) were found with Paced Auditory Serial
Addition test (.663, .447, and .327, respectively). The strongest correlation for matching to
sample (MSP) was found with Trail Making Test part B (-.497). All correlations were

44

statistically significant at p>0.01 level. This study indicated strong concordance between ANAM
and traditional neuropsychological measures.
The clinical utility of ANAM goes beyond military research and is also used in a variety
of illnesses including multiple sclerosis, systemic lupus erythematosus, Parkinson’s disease,
Alzheimer’s disease, migraine headaches, and brain injuries (Kane, Roebuck-Spencer, Short,
Kabat, & Wilken, 2007). ANAM-sports medicine battery is one of the multiple alternate forms
of ANAM, and it consists of simple reaction time (SRT), code substitution (CDS), code
substitution-delayed (CDD), continuous performance test (CPT), mathematical processing
(MTH), matching to sample (MSP), spatial processing (SPD), Sternberg memory procedure
(STN), and procedural reaction time (PRM) (Cernich, Reeves, Sun, & Bleiberg, 2007). Each
subtest provides an accuracy score (percent correct) and a throughput score (number of correct
responses per minutes).
Segalowitz and colleagues (2007) examined the test-retest reliability of ANAM sports
medicine battery in a group of 29 adolescents. The researchers administered ANAM twice during
the same time of day over a one week interval. The intra class correlations (ICC) and the Pearson
correlation coefficient for each composite score were calculated for test-retest reliability. The
highest ICC was reported in MSP (.72), followed by CPT (.65), MTH (.61), CDS (.58), SRT2
(.47), and SRT (.44). The highest Pearson correlation coefficient was reported in CDS (.81),
followed by MSP (.72), CPT (.70), CDD (.67), SRT2 (.50), and SRT (.48). Even though the high
throughput score was reported (.87), the results revealed the variability of test-retest reliability
for individual subtests of ANAM.
Another study assessed the reliability of ANAM sports medicine battery through testretest methods using a military sample. The average test-retest interval for this study was 166.5

45

days. Results revealed a wide range of ICC for each subtest with the highest ICC in MTH (.87),
followed by MSP (.66), SPD (.60), CPT (.58), STN (.48), and SRT (.38) (Cernich et al., 2007). It
is important to point out that in both studies, there are low ICC values reported for reaction time.
The validity and reliability of some subtests of ANAM and ANAM sports medicine
battery were below the acceptable value, which is suggested to be .60 to .70 (Baumgartner &
Chung, 2001). Further studies are warranted in relation to maturation issue (frequency of
baseline update), effect of confounding factors such as stress, fatigue, and sleep deprivation on
the test performance (Cernich et al., 2007).
CogSport
CogSport is a testing battery developed by CogState (1999) measuring psychomotor
function, speed of processing, visual attention, vigilance, and verbal and visual learning and
memory. The battery employs a series of “card games” to examine cognitive function: simple
reaction time, complex reaction time, one-back and continuous learning. CogSport can be
administered without a trained personnel and takes only approximately15 minutes.
Collie et al. (2003) determined the reliability of CogSport measurement by calculating
ICC on serial data collected in 60 healthy youth volunteers at intervals of 1 hour and 1 week. In
the same study, construct validity was also determined by calculating ICC coefficients between
CogSport outcome variables and performance on traditional paper and pencil assessment tools
(Digit symbol Substitution test and the Trail Making Test Part B) in 240 professional athletes
competing in the Australian Football League. The result showed high to very high ICC in
CogSport speed indices at intervals of 1 hour and 1 week (.69-.90), while Cogsport accuracy
indices displayed lower and more variable ICC (.08-.51). Variety of correlation coefficient
between CogSport measures and the Digit symbol Substitution test were reported with the

46

highest in decision making (r=.86), followed by working memory speed (.76), psychomotor
speed (.50), and learning speed (.42). Smaller and less variable correlation coefficient between
CogSport measures and the Trail Making Test Part B were observed with the highest in
psychomotor speed (.44), decision making (.34), working memory speed (.33) and learning
speed (.23).
Axon Sports is a recently launched company with CogState providing them with a
Computerized Cognitive Assessment Tool (Axon Sports CCAT). Like Cogsport, this test battery
also employs a “card game” to evaluate cognitive domains including processing speed, attention,
learning, and working memory. Makdissi et al. (2010) conducted a prospective study and tracked
the recovery of 78 concussed male Australian football players utilizing the baseline and post
concussion data of the Axon sport CCAT and traditional paper and pencil tests (the Digit Symbol
Substitution Test and the Trail Making Test Part B). While concussion associated symptoms
lasted an average of 48.6 hours (95% CI: 39.5-57.7 hours) and symptoms and cognitive deficits
on the traditional paper and pencil test had largely resolve at 7 days post injury, 17.9% of players
still displayed significant cognitive decline on the Axon sport CCAT. The longer cognitive
deficits on the Axon sport CCAT compared with the traditional paper and pencil test on average
2 to 3 days was reported. This study implied greater sensitivity of Axon sport CCAT to cognitive
impairment following concussion compared to the Digit Symbol substitution test and the Trail
Making Test Part B.
Broglio and colleagues (2007) examined the test-retest reliability of three computerized
neuropsychological testing batteries: Immediate post Concussion Assessment and Cognitive
Testing (ImPACT), Concussion Resolution Index (CRI), and Concussion Sentinel, which is an
earlier version of CogSport. In this study, these computerized neuropsychological test batteries

47

were administered successively (the order of administration was not specified) at 3 occasions
with clinically relevant intervals: baseline, 45 days after baseline, and 50 days after baseline.
Along with the neuropsychological testing, the Memory and Concentration Test for Windows
(MACT), which is a computer-based effort assessment was administered in order to control for
suboptimal effort. ICC were obtained using data from 73 participants and their demographics
were as follows: age=21.39±2.78 years, height=170.95±9.00 cm, body mass=69.09±15.07 kg
and total self reported SAT score = 1168.17 ± 99.76. A history of diagnosed concussion was
reported from 12 participants, and number of concussions ranged from 1 to 5. Based on 7 tasks,
Concussion Sentinel develops 5 output scores: reaction time, decision making, matching,
attention, and working memory. The results showed wide range of ICC. When baseline and day
45 were compared, the highest ICC was observed for working memory (.65), followed by
reaction time (.60), decision making (.56), attention (.43), and matching (.23). When day 45 and
day 50 were compared, the highest ICC was reported for matching (.66), followed by working
memory (.64), decision making (.62), reaction time (.55), and attention (.39). The MACT
implied that good effort was made by participants throughout the study. However, the effect of
successively administered three neuropsychological testing batteries on the participants’
performance was pointed out as a possible methodological flaw.
Concussion Resolution Index (CRI)
The CRI, developed by HeadMinder, Inc., is a web-based computerized
neuropsychological assessment battery designed for sports medicine professionals who manage
and monitor resolution of sport-related concussion symptoms. The CRI is comprised of six
subtests: reaction time, cued reaction time, visual recognition 1and 2, animal decoding, and
symbol scanning. Symbol scanning measures simple and complex reaction time, visual scanning,

48

and psychomotor speed. These six subtests form three CRI indices: Psychomotor, Speed Index,
Simple Reaction Time. In addition to cognitive testing, the CRI collects demographic
information, medical history, concussion history, and symptom report. Test duration is
approximately 25 minutes for baseline and 20 minutes for post concussion assessments.
Concurrent validity was established using traditional neuropsychological paper and
pencil-based tests. Correlation for CRI Psychomotor Speed Index were 0.66, 0.60, 0.57 and 0.58
for the Single Digit Modality Test, the Grooved Pegborard Test (dominant and nondominant
hand), and the WAIS-III Symbol Search subtest, respectively (Erlanger et al., 1999).
Correlations for CRI Complex Reaction Time Index were 0.59 and 0.70 for the Grooved
Pegboard test for dominant and nondominant hand, respectively (Erlanger et al., 1999).
Correlations for CRI Simple Reaction Time Index were 0.56, 0.46, and 0.60 for Trail Making
Test Part A, and the Grooved Pegboard Test (dominant and nondominant hand), respectively
(Erlanger et al., 1999)
Erlanger et al. (2003) reported that test-retest reliabilities with a 2-week interval were .82
for psychomotor speed, .70 for simple reaction time, and .68 for complex reaction time. Erlanger
et al. (2001) also examined the sensitivity of the CRI in detecting change between baseline and
post concussive assessment. The CRI has a 77% sensitivity to concussion. Thus, initially the CRI
was found to be a valid and reliable method of identifying changes in psychomotor speed and
processing speed subsequent to sports-related concussion.
However, in a latter study by Broglio et al. (2007), the reliability of the CRI was
questioned. In this study, Broglio and colleagues reported extremely low ICC (.03) for Simple
Reaction Time Error score when day 45 and day 50 were compared to baseline. Low ICC were
also reported for Simple Reaction Time Errors score (.15) and Complex Reaction Time Error

49

score (.26) when baseline and day 45 were compared. However, as it has been previously
mentioned, this study may be flawed by methodological problems.
Immediate post Concussion Assessment and Cognitive Testing (ImPACT)
The ImPACT test battery evaluates multiple aspects of neurocognitive function including
post concussion symptoms, attention, memory, visual motor speed, and reaction time. The
ImPACT consists of three main parts: demographic data, Post-Concussion-Symptom Scale
(PCSS), and neuropsychological tests. The demographic data section provides the sports
medicine professionals with subjects’ sport, medical history, and concussion history information.
The PCSS is comprised of 22-concussion symptoms with a 7-point likert scale (0 to 6). Athletes
self-report their concussion symptoms from not experiencing (0) to severely experiencing (6) the
symptom. The neuropsychological tests are comprised of six modules that tests attention,
concentration, working memory, visual recognition, and reaction time. From the six modules,
five composite scores are generated: verbal memory, visual memory, visuomotor speed, reaction
time and impulse control. Impulse control composite score is included to determinate if an
athlete purposefully does not try hard on the test or makes too many errors resulting in an invalid
test. Recently, an online version of the ImPACT test has been developed, which enhanced the
ease of use of this assessment tool. Compared to other computerized neurocognitive tests, a
larger population of clinicians and researchers have used this cognitive tool. In addition, there
has been several studies conducted regarding sensitivity and specificity, reliability, and validity
of ImPACT test battery.
Schatz and colleagues (2006) examined the diagnostic utility of the composite scores and
the PCSS of the ImPACT in a group of 72 concussed student-athletes and 66 non concussed
student-athletes. The concussed athletes primarily included football players (73%) and the

50

majority of the non-contact sports included athletes participating in track and field and tennis
(79%). All athletes were administered a baseline or pre-injury evaluation that included an
ImPACT test. All concussed athletes were tested within 72 hours of sustaining a concussion.
Concussions were diagnosed by certified athletic trainers or team physicians based on the criteria
of the American Academy of Neurology. Data was analyzed to measure the ability of the five
ImPACT composite scores and PCSS to classify concussion status. A stepwise discriminant
analysis found one discriminate function identifying PCSS, processing speed composite, visual
memory composite, and impulse control composite as significant factors (p=.0001) with 85.5%
of concussion cases correctly classified. Approximately 82% of participants in the concussion
group and 89% of participants in the control group were correctly classified. Thus, the
classification results revealed the sensitivity of ImPACT is 81.9% and the specificity is 89.4%.
In order to examine the construct validity of ImPACT, Maerlender et al. (2010) compared
scores on the ImPACT test battery to a neuropsychological battery and experimental cognitive
measures in 54 healthy male athletes. The neuropsychological battery included the California
Verbal Learning test, the Brief Visual Memory Test, the Delis Kaplan Executive Function
system, the Grooved Pegboard, the Paced Auditory Serial Attention Test, the Beck Depression
Inventory, the Speilberger State-Trait Anxiety Questionnaire, and the Word Memory Test. The
experimental cognitive measures included the N-back and the verbal continuous memory task.
The following scores were generated: neuropsychological verbal memory score,
neuropsychological working memory score, neuropsychological visual memory score,
neuropsychological processing speed score, neuropsychological attention score,
neuropsychological reaction time score, neuropsychological motor score, and
neuropsychological impulse control score. The results showed significant correlations between

51

neuropsychological domains and all ImPACT domain scores except the impulse control factor.
The ImPACT verbal memory correlated with neuropsychological verbal (r=.40, p=.00) and
visual memory (r=.44, p=.01), the ImPACT visual memory correlated with neuropsychological
visual memory (r=.59, p=.00), the ImPACT visual motor processing speed and reaction time
score correlated with neuropsychological working memory (r=.39, p=.00 and r=-.31, p=.02) and
neuropsychological process speed (r=.41, p=.00 and r=-.37, p=.01), and neuropsychological
reaction time score (r=.34, p=.00 and r=-.39, p=.00). It must be noted that the
neuropsychological domain scores for motor, attention, and impulse control were not correlated
with any ImPACT composite scores. Overall the results suggest that the cognitive domains
represented by ImPACT have good construct validity with standard neuropsychological tests that
are sensitive to cognitive functions associated with mTBI.
While good sensitivity and specificity, and construct validity of ImPACT have been
reported, its test-retest reliability has been shown to be inconsistent. Iverson et al.(2003)
examined the test-retest reliability over a 7-day time span using a sample of 56 nonconcussed
adolescent and young adults (29 males and 27 females, average age: 17.6 years). The participants
completed the ImPACT test battery on two occasions with a 7 day interval. The researchers
reported that the Pearson test-retest correlation coefficients and probable ranges of measurement
effort for the composite scores: verbal memory=.70 (6.83pts), visual memory=.67 (10.59pts),
reaction time=.79 (.05sec), processing speed=.89 (3.89pts), and post-concussion scale=.65
(7.17pts). There was a significant difference between the first and 7 day retest on the processing
speed composite (p<.003) with 68% of the sample performing better at the 7 day retest interval
than at the first test. This study was limited by the variation of testing intervals. Specifically,

52

29% of the participants were retested within 3 days, 43% within 4 days, 82% within 7 days, and
95% within 11 days. Although the median was 7 days (average=5.8 days).
Miller et al. (2007) conducted a test-retest study with a longer time period (4 month) with
in-season athletes. The researchers administered a series of ImPACT tests to 47 non-concussed
Division III football players at preseason (before the first full-pads practice), midseason (6 weeks
into the season), and postseason (within 2 weeks of the last game). The results indicated no
significant differences in verbal memory (p=.06) and in processing speed (p=.05) over the 3
testing occasions. However, the scores for visual memory and reaction time showed significant
improvement as the season progressed (p=.04 for both composite score). Even though the
statistical difference was found at the P level of .05, when an 80% confidence interval was used,
the ImPACT results could be interpreted as stable over a 4 month time period in football players.
The test-retest reliability of ImPACT has been examined with even longer time periods. Several
researchers have suggested shorter re-administration of neurocognitive baseline (ie., every year)
assessment for adolescent athletes due to their on-going cognitive development and maturation.
In response, Elbin et al. (2011) investigated the 1-year test-retest reliability of the online version
of ImPACT using the baseline data from 369 varsity high school athletes. The researchers
administered the ImPACT approximately 1.2 years (range, 0.5-2.35 years) apart as required by
the participants’ respective athletic programs. Results showed that motor processing speed was
the most stable composite score with ICC of .85, followed by reaction time (.76), visual memory
(.70), verbal memory (.62), and PCSS (.57). Considering the fact that ICC of .60 and .70 have
been recommended as minimum acceptable ICC (Baumgartner & Chung, 2001), this study
suggested that ImPACT baseline data can be used reliably for 1 year in high school athletes. Low

53

ICC for PCSS (.57) was not unexpected because it has been found to show some variability at
baseline (Covassin, Schatz, & Swanik, 2007; Swanik, Covassin, Stearne, & Schatz, 2007).
The test-retest study of ImPACT with the longest time period was conducted by Schatz
et al. (2010), which examined two baseline data intervals, approximately 2 years apart. This
study included 95 collegiate athletes. Results showed that motor processing speed is the most
stable composite score over 2 years with ICC of .74, followed by reaction time (.68), visual
memory (.65), verbal memory (.46), and then total symptom score (.43). Most of these findings
are consistent with the study by Elbin and colleagues (2011). Even though the ICC for verbal
memory did not reach the “acceptable” threshold (.60), with use of regression based methods,
none of the participant’s scores showed significant change. Also, reliable change indices
revealed that only a small percentage of participants showed significant change (0 to 3%). This
study suggested that college athletes’ cognitive performance remains stable over a 2-year time
period.
In contrast to these findings that support the test-retest reliability of ImPACT, Broglio et
al. (2007) reported low ICC for all ImPACT composite scores. Using 118 healthy college
students, the researchers examined test-retest reliability of ImPACT and two other
neuropsychological tests (Concussion Sentinel and CRI) through three administrations: baseline,
45 days after the baseline, and 50 days after the baseline. For ImPACT, the researchers reported
ICC ranging from .28 to .38 (baseline to day 45), and .39 to .61 (day 45 to day 50). The ICC for
each composite score were as follows: verbal memory (.23 for baseline to day 45, and .40 for day
45 to day 50), visual memory (.32 and .39, respectfully), motor processing speed (.38 and .61,
respectfully), and reaction time (.39 and .51, respectfully). As it has been mentioned previously,
these low ICC values may have been attributable to methodological problems caused by

54

administering three different computerized neurocognitive assessments in succession (ImPACT,
Headminder’s CRI, and Concussion Sentinel).
While the acceptable test-retest reliability of ImPACT for short term (7days) and long
term (1 and 2 years) have been reported (Iverson et al., 2003; Schatz et al., 2010; Elbin et al.,
2011), no study has been conducted to re-examine the test-retest reliability of ImPACT for midterm (45-50 days) after Broglio et al. (2007) reported low ICC. Re-examination of test-retest
reliability of ImPACT for mid-term is warranted without the confounding factors that have been
pointed out in the study of Broglio et al. (2007). In the next section, another possible
confounding factor, which is the effect of physical activity on cognitive function will be
reviewed.
Physical Activity and Neurocognitive Performance
The benefit from participating in physical activities is not limited to the reduction of risk
for physical disorders such as cardiovascular disease, colon and breast cancer, and obesity.
Studies have also found the positive effect of physical activity on cognitive function. In animals,
it has been shown that exercise increases neurogenesis in a brain structure, which result in
improving learning and memory (Praag, Christie, Sejnowski, & Gage, 1999; Vaynman &
Gomez-Pinilla, 2006). In human research, the effects of physical activity on cognitive function
have been studied with various age groups ranging from children to elderly.
A meta-analysis was conducted on 44 studies to determine the effect of physical activity
on cognition in school-age children (aged 4-18 years). In this study physical activity included
resistance/circuit training, perceptual-motor training, physical education program, and aerobic
activity. Cognitive function included perceptual skills, intelligence quotient, achievement, verbal
tests, mathematic tests, memory, developmental level, and academic readiness. The results

55

indicated a statistically significant positive relation between physical activity and all cognitive
performance (Effect Size=.32, SD=.27) with the exception of memory (Sibley & Etnier, 2003).
The above mentioned meta-analysis study is consistent with a more recent meta-analysis
study conducted by Fedewa and Ahn (2011) which reported effect size of .35 (95%CI:.27-.43)
under mixed-effect model, and .32 (95% CI: .26-.37) from cross sectional/correlation data. In
this study, a total of 59 studies were analyzed in order to quantitatively synthesize the research
on physical activity and cognitive outcomes in children (aged 3-18 years). The physical activities
were categorized into total physical fitness, development, strength, flexibility, and cardio.
Cognitive outcome measures included intelligence quotient, total achievement,
vocabulary/spelling/language/art achievement, reading achievement, mathematics achievement,
science achievement, grade-point average, and other such as creativity. The results showed a
significant and positive impact of physical activity on children’s cognitive outcomes.
The study of physical activity and cognition in middle-age and older adults have been
conducted mainly with the purpose of maintenance of cognitive function and reduction of the
risk for age-associated cognitive disorders such as Alzheimer’s disease and vascular dementia.
Regular physical activity can be thought to be a neuroprotective therapy due to significant
benefits of long-term, regular physical activity on cognition, dementia risk, and dementia
progression (Ahlskog, Geda, Graff-Radford, & Petersen, 2011). A relatively recent meta-analysis
of prospective cohort studies reported the protective effect of physical activity in midlife against
the risk of neurocognitive diseases (Hamer & Chida, 2009). The results showed the relative risk
of Alzheimer’s disease and dementia in adults who routinely engaged in physical activity in their
midlife compared with sedentary adults was .55 (95%CI: .36-.84) and .72 (95%CI: .6-.86),
respectively.

56

Compared to the other age group, there is a dearth of research on the effect of physical
activity on cognition in young adults (Hillman, Erickson, & Kramer, 2008). The study of Kamijo
and Takeda (2009) is one of the few studies that have examined the relationship between
physical activity level and the cognitive function of executive control in young adults. In this
study, the physical activity level of 40 young adults (mean age=21.1) were evaluated using the
International Physical Activity Questionnaire (IPAQ) long form, which assesses leisure time
physical activity, domestic and gardening activities, work-related physical activity, and
transport-related physical activity. Based on the IPAQ, participants were categorized in a
physically active group (n=20, 9229.3kcal/week) and sedentary group (n=20, 1698.2kcal/week).
The participants’ electroencephalogram were measured during a spatial priming task, which
asked the participants to respond to the location change of O while ignoring X (distracter) on the
computer screen. This priming paradigm has been used to investigate facilitatory effects for
target (positive priming, location of O does not change while location of X changes,), inhibitory
effect (negative priming, location of O changes while location of X does not change), and control
(both location of O and X change). This priming effect on reaction time and P3 latency was
assessed as indicators of cognitive function, specifically executive control. The results revealed a
larger negative priming effect on reaction time and P3 latency in the active group compared with
the sedentary group, while positive priming effects were only observed in the sedentary group.
This study indicates that physical activity has a beneficial effect on the cognitive processes on
executive control in young adults.
Hillman and colleagues (2006) also used event-related brain potentials to examine the
effect of physical activity level on processing speed and the updating of memory representation
during the performance of switch and non-switch trials. In this study, the young adults who

57

participated in aerobic exercise more than five hours per week were categorized in physically
active group (n=18, mean age=19.4), and those who participated in aerobic exercise less than 1
hour per week were categorized in sedentary group (n=16, mean age=19.4). Participants
switched back and forth between two different tasks: determining whether the digit presented in
the computer screen was greater or less than five; determining whether the digit presented was
odd or even. Electroencephalogram was measured during the task. There were faster P3 latencies
and greater amplitude of P3 component in the physically active group compared with sedentary
group, which suggests regular physical activity improved executive control such as working
memory, interference control, task coordination, and inhibitory control.
Since the effect of physical activity on cognitive function have been reported not only for
older adult populations (Kramer, Erickson, & Colcombe, 2006) but also for young adults
(Hillman, Kramer, Belopolsky, & Smith, 2006; Kamijo & Takeda, 2009), a test-retest reliability
study must be conducted with physically active individuals in order to eliminate any effect of
physical inactivity on cognitive function in the test-retest reliability study for ImPACT.
Moreover, using physically active individuals will be more representative of athletes who are the
primary users of the ImPACT neuropsychological test battery. Physical activity level of the
participants were not taken into account in the study of Broglio et al. (2007), whereas the
participants of other test-retest reliability were athletes (Iverson et al., 2003; Schatz et al., 2010;
Elbin et al., 2011). In addition to the previously mentioned methodological problem,
administrating three neuropsychological testing successively in a testing occasion, and
uncontrolled physical activity level of the participants may have affected the result of the study
of Broglio et al. (2007). This leads to the purpose of the present study, which is to re-examine the

58

mid-term test-retest reliability of ImPACT by replicating the study of Broglio et al. (2007) while
controlling for the previously mentioned confounding factors.

59

CHAPTER III
METHODOLOGY
Research Design
A repeated-measures design will be used to evaluate the test-retest reliability of the
ImPACT neuropsychological test battery. The independent variable will be test group (baseline,
45 days after the baseline evaluation, and 50 days after the baseline evaluation). The dependent
variables will be the composite scores of ImPACT including verbal memory, visual memory,
motor processing speed, and reaction time. In addition, the Rey’s 15-item memory test will be
used before administration of ImPACT to determine effort from test-takers. Thus, dependent
variables will also include the number of items that test-takers reproduced in Rey’s 15-item
memory test.
Participants
The participants selected for this study will be 120 college students who meet the
inclusionary criteria that satisfy the following categories of basic recommendation for physically
activity based on the American College of Sports Medicine (Garber et al., 2011).
Inclusionary criteria
The participants must meet the following basic recommendation: 1) Cardiorespiratory
exercise—at least 150 minutes of moderate intensity exercise (4.8-7.1METs and PRE 12-13) or
75 minutes of vigorous intensity exercise (7.2-10.1 MET and PRE 14-17) per week. One
continuous session and multiple shorter sessions of at least 10 minutes per session are both
acceptable to accumulate the desired amount of exercise. Participants should meet the following
two recommendations: 2) Resistance Exercise—two to four sets of 8-12 repetitions on major
muscle group such as chest, shoulders, back, hips, legs, trunk, and arms two or three days week

60

using a variety of exercises and equipment. 3) Neuromotor exercise—two or three days per week
of exercises that involves motor skills (balance, agility, coordination and gait). Multifaceted
physical activities such as tai chi and yoga involve variety combinations of neuromotor exercise,
resistance exercise, and flexibility exercise. The ACSM recommends 20-30 minutes per day to
be sufficient for neuromotor exercise.
Exclusionary criteria
Participants will be excluded from the study if they have the following: 1) diagnosed with
a learning disability, color blindness, attention deficit disorder, psychological disorder, brain
surgery, or a major neurological condition (demylinating disease, acute disseminated
encephalomyelitis), 2) diagnosed with a concussion within 6 months before or during the study,
3) any athlete with a severe history of intracranial pathology (e.g., subdural hematoma) as
determined by a positive CT or MRI, or 4) English was not their primary language.
Instrumentation
Immediate Post Concussion Assessment Cognitive Testing (ImPACT)
The computerized neurocognitive assessment program used will be the online version of
ImPACT (ImPACT Applications, Inc., Pittsburgh, PA). The ImPACT program consists of three
main sections including 1) demographic section, 2) postconcussion symptom scale (PCSS)
section, and 3) neurocognitive test modules. In the demographic section, participants will be
asked to input demographic and descriptive information such as years of experience playing
sport, history of alcohol and drug use, learning disabilities, attention deficit hyperactive disorder,
major neurological disorder, and previous concussion history. In the PCSS section, participants
will be asked to self-report a total of 22 concussion related symptoms based on how they are
feeling at the moment using a 7-point Likert scale (0=not experiencing this symptom, and

61

6=severely experiencing this symptom). The third category consists of six neurocognitive
modules that are described below. Participants will be instructed to complete the given tasks as
quickly and accurately as possible with their best effort.
ImPACT module one (Word discrimination). Attention processes and verbal recognition
memory will be evaluated in the first module. Participants will be asked to memorize a list of 12
words (target words) that stay on the screen for 750 ms each. Each word shows up twice. Then a
list of 24 words that comprised of 12 target words and 12 non-target words will be presented and
participants will be asked if those words were in the original 12 words by answering “yes” or
“no”. This task is for immediate recall memory. For delayed recall testing, another list of 24
words that are comprised of 12 target words and 12 non-target words will be presented and
participants will be asked if those words were in the original 12 words by answering “yes” or
“no”. Tests are scored based on a total percentage of correct answers.
ImPACT module two (Design memory). Visual recognition memory and attention
processes will be evaluated in the second module measures. The testing process of this module is
exactly the same as module one except that designs are used instead of words. Immediate recall
memory and delayed recall memory will be evaluated by total percentage correct.
ImPACT module three(X’s and O’s). Visual working memory and visual processing
speed will be evaluated by using a distractor task (choice reaction time) and memory task (visual
memory). Athletes are presented a random screen of X’s and O’s with three yellow X’s and/or
O’s. A distractor task is then presented. Participants will be asked to click the Q key on the
keyboard if a blue circle is presented and click the P key on the keyboard if a red square is
presented. After completing the distractor task, the same memory screen will show up and
participants will be asked to click on the three previously illuminated yellow X’s and/or O’s.

62

This process will be repeated four times. The correct identification of the X’s and O’s will be
scored as visual memory, and reaction time for the distractor task will be scored as reaction time.
ImPACT module four (Symbol matching). Nine common symbols (triangle, square, arrow,
etc) matched with nine numbers (1-9) will be presented on a screen. Below these pairings, a
symbol is randomly presented and participants will be asked to click on the matching number as
quickly as they could while at the same time remembering the symbol/number parings. The
number will light up in different colors (green=correct number, red=incorrect number) to
reinforce correct performance. Following the completion of 27 trials, the symbols will disappear
from the top grid, and then randomly reappear below the grid. Participant will be asked to click
on the number that matches the symbol. This module provides an average reaction time score
and score for the memory recall.
ImPACT module five (Color match). Choice reaction time, impulse control and response
inhibition will be measured in this module. First, participants will be asked to respond by
clicking a red, blue, or green button on the screen as the words (RED, GREEN, and BLUE) are
presented. The purpose of this test is to assure that the following tasks would not be affected by
color blindness. Following the first task, a word will be displayed on the screen in the same or
different colored ink (RED in red, or RED in green, etc). Participants will be asked to click on
the mouse as quickly as they could only when the word is presented in the matching ink. A
reaction time score and a score for the number of errors will be obtained from this module.
ImPACT module six (The letter memory). Visual motor response speed and working
memory will be measured in this module. Three random letters will be presented on the screen
and participants will be asked to remember the letters. Then participants will complete a
distractor task, which is clicking the numbered button in a randomized 5 X 5 grid in backwards

63

order starting with 25 to 1. The numbered grid will disappear after 18 seconds, and then
participants will be asked to type in the three letters that appeared on the screen before the
distractor task. This procedure will be repeated five times. This module generates a memory
score (total number of correctly identified letters) and a score for the average number of correctly
clicked number in 18 seconds per trial from the distractor test. The entire ImPACT
neuropsychological test takes approximately 20-25 minutes.
Rey’s 15-Item Memory Test
Before administration of the ImPACT program, the Rey’s 15-item memory test will be
administered in order to evaluate test takers’ effort. The test consists of 15 symbols on a piece of
paper in five rows of three categorically related symptoms. The symbols are easily identified
letters, numbers, or figures (see Appendix A). Patients are exposed to the symbols for 30
seconds, and then asked to reproduce them without worrying about their order for two minutes.
Test-takers who reproduced fewer than nine items will be considered malingering (Goldberg &
Miller, 1986; Bernard & Fowler, 1990), which indicates lack of effort in this present study.
Physical Activity Level Survey
Participants will complete a survey regarding their physical activity level in order to
assure the participants would be physically active throughout the study period. The intensity,
duration, and frequency of cardiorespiratory exercise, resistance exercise, and neuromotor
exercise will be assessed using this survey (see Appendix B).
Physical/Mental Condition Survey
Along with the physical activity survey, in order to control variables that may affect
participants’ performance on ImPACT, information such as physical fatigue level, stress level,
food consumption, and caffeine intake of the testing day will also be gathered (see Appendix A).

64

Post-Testing Survey
Participants will also complete a short post-survey consisting of four questions. The first
question will ask if the participant was distracted during the ImPACT and Rey’s 15-item
memory test. The second question will ask the participant if they had any technical difficulties
(ie., slow mouse/computer). Finally the participants will be asked if the instructions were clear
on the ImPACT and Rey’s 15-item memory test tests (see Appendix C).
Procedures
This study will receive approval from the Institutional Review Board of Michigan State
University prior to recruiting participants. Prior to participation, all participants will be required
to sign the appropriate IRB approved consent forms. All participants will report to the same
reserved computer laboratory for a total of three visits. There will be an approximately 45 days
between the first and second session, and an approximately 50 days between the first and third
session. Each session will last for approximately 40-45 minutes. All participants will complete
Rey’s 15-item memory test, demographic and physical activity survey, and the ImPACT
neuropsychological test battery, which will be administered on a desktop computer with an
optical mouse.
Data Analysis
Descriptive statistics will be performed for all tests to determine means and standard
deviations. SPSS Version 16.0 (SPSS Inc., Chicago, IL) will be used to obtain ICC for each
composite score. Specifically, a 2-way random effects analysis of variance ICC for absolute
agreement will be calculated in order to estimate the test-retest reliability of ImPACT for
baseline to day 45 assessments and for day 45 to day 50 assessments. ICC will range from 0 to 1,
with an ICC closer to 1.00 indicating a stronger reliability. There are a variety of

65

recommendations for ICC interpretation. Baumgartner and Chung (2001) suggest an ICC of .60
to be minimally acceptable. Therefore, in the present study, an ICC of .60 will be used as the
minimum acceptable ICC value. Test-takers who reproduced fewer than nine items will be
considered malingering (Goldberg & Miller, 1986; Bernard & Fowler, 1990), which indicates
lack of effort in this present study. Statistical significance will be set a priori at p=.05.
Evaluation of Hypotheses
H1. There will be acceptable test-retest reliability(ICC≥.60) in verbal memory composite scores
over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.
H2. There will be acceptable test-retest reliability (ICC≥.60) in visual memory composite
scores over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.
H3. There will be acceptable test-retest reliability (ICC≥.60) in motor processing speed scores
over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.
H4. There will be acceptable test-retest reliability (ICC≥.60) in reaction time composite scores
over three test administrations (baseline, 45 days, 50 days) among physically active
individuals.
H5. There will be no difference in test takers’ effort level as measured by the Rey’s 15-Item
memory test over three test administrations (baseline, 45 days, 50 days).

66

CHAPTER IV
RESULTS
Overview
This research was conducted to investigate the test-retest reliability of the ImPACT
neurocognitive test battery between baseline, 45 days and 50 days after baseline on physically
active college students. The following chapter will describe the demographics of the sample and
the ICC for the ImPACT test battery.
Demographic Data
There were a total of 127 subjects who participated in this study; however, data from 39
participants were excluded from further analysis for various reasons including missing one of the
two post-baseline tests (n=31), invalid baseline (n=1), a value greater than 30 on Impulse Control
scores (n=2), not maintaining the required physical activity level (n=2), and technical problem
during testing (n=3). Therefore, there were a total of 88 participants (54 male, 34 female). The
majority of participants were Caucasian (75%), followed by African American (6%), Hispanic or
Latino (3%), Asian American (2%), and 2% of participants reported their ethnicity as “other.”
History of concussion(s) was (were) reported from 21 participants with an average of 1.76
previous concussions (SD = + 1.04) that ranged from 1 to 4 previous injuries; however, no
participants sustained a concussion during the testing period or in the 6 months prior to
participating in this study. A summary of demographic data for age, height, and weight can be
found in Table 4.1.

67

Table 4.1. Demographic Information for Age, Height and Weight
Age (yrs)

Height (cm)

Weight (kg)

Group

Mean

SD

Mean

SD

Mean

SD

Males (n=54)

20.05

+1.96

179.96

+7.59

78.56

+11.33

Females (n=34)

19.77

+1.02

164.72

+7.69

63.66

+9.38

Total (n=88)

19.95

+1.66

174.07

+10.65

72.80

+12.83

Testing Intervals
The testing intervals between the baseline test and the second test was 44.2 ± 0.4 days,
and the baseline test and the third test was 49.81±0.40 days. Also, each participant except for
three reported for each test session at the same time of day. In addition, location and the
computer used were consistent throughout the test period.
Evaluation of the Hypotheses
The following behavioral results (H1 through H5) are combined into the categories of the
computerized neurocognitive test results and the Rey’s 15-item memory test results.
Results of Computerized Neurocognitive Testing (Hypotheses 1 – 4):
There will be acceptable test-retest reliability (ICC≥.60) in computerized neurocognitive test
performance (ImPACT Composite Scores: verbal and visual memory, motor processing speed,
reaction time)over three test administrations (baseline, 45 days, 50 days)among physically active
individuals.
Hypotheses 1 through 4 were supported as the ICC values on the all ImPACT composite
score were equal or higher than .60, which indicates acceptable test-retest reliability. Among the
composite scores, motor processing speed reported the highest overall ICC value with .90, and
visual memory reported the lowest overall ICC value with .75. Specifically, the highest ICC

68

value (.87) was reported from motor processing speed between day 45 and day 50, and the
lowest ICC value (.60) was reported from visual memory between baseline and day 50. With
exception of verbal memory, the ICC values from day 45 to day 50 were higher than the ICC
values from baseline to day 45, and from baseline to day 50. The means, standard deviations, and
range for each ImPACT composite score can be found in Table 4.2, and the mean differences of
each ImPACT composite score between testing days can be found in Table 4.3 (see Figures 4.14.4). The ICC values for each ImPACT composite score from baseline to day 45, day 45 to day
50, and baseline to day 50 can be found in Table 4.4. The Cronbach’s alpha values were also
calculated for each ImPACT composite score from baseline to day 45, day 45 to day 50, and
baseline to day 50, and the summary can be found in Table 4.5.

Table 4.2. Means and Standard Deviations for ImPACT Composite Scores at Baseline, 45 days
after the Baseline, and 50 days after the Baseline
Baseline

45 days

50 days

Mean+ SD

Range

Mean+ SD

Range

Mean+ SD

Range

Verbal
Memory

85.75±10.31

65-100

88.33±10.07

55-100

89.43±8.53

66-100

Visual
Memory

76.07±12.23

41-99

77.95±13.25

49-99

73.20±12.62

46-99

Motor
Processing
Speed

41.16±6.02

28.5853.15

42.57±6.13

29.2852.33

43.01±5.91

29.3852.58

Reaction
Time

0.59±0.09

0.98-0.46

0.58±0.07

0.75-0.47

Impulse
Control

5.15±3.49

0-20

5.65±4.1

0-29

0.97-0.44 0.58±0.08

0-17

5.57±3.81

69

Table 4.3. Pair-wise Comparison of ImPACT Composite Scores at Baseline, 45 days after the
Baseline, and 50 days after the Baseline
Mean Difference (p-value)
Baseline vs Day 45

Day 45 and Day 50

Baseline vs Day 50

Verbal Memory

2.60 (.006)*

1.10 (.256)

3.71 (.000)*

Visual Memory

1.89 (.135)

-4.75 (.001)*

-2.86 (.047)*

Motor Process Speed

1.41 (.001)*

.439 (.307)

1.85 (.000)*

Reaction Time

.011 (.248)

-.007 (.261)

.003 (.695)

Impulse Control

-.420 (.236)

-.080 (.819)

-.500 (.262)

* p< .05

Figure 4.1. Composite Score for ImPACT Verbal Memory at Baseline, 45 days after the Baseline,
and 50 days after the Baseline. For interpretation of the references to color in this and all other
figures, the reader is referred to the electronic version of this dissertation.
90

Composite Score

89

89.43

88

88.33

87
86

85.75

Verbal Memory

85
84
83
Baseline

Day 45
ImPACT Test Time

70

Day 50

Composite Score

Figure 4.2. Composite Score for ImPACT Visual Memory at Baseline, 45 days after the Baseline,
and 50 days after the Baseline
79
78
77
76
75
74
73
72
71
70

77.95

76.07

73.2

Baseline

Day 45
ImPACT Test Time

Visual Memory

Day 50

Figure 4.3. Composite Score for ImPACT Motor Processing Speed at Baseline, 45 days after the
Baseline, and 50 days after the Baseline
43.5
Composite Score

43

43.01

42.5

42.57

42
41.5
41
40.5

Motor Processing Speed
41.16

40
Baseline

Day 45
ImPACT Test Time

Day 50

71

Figure 4.4. Composite Score for ImPACT Reaction Time at Baseline, 45 days after the Baseline,
and 50 days after the Baseline

Composite Score

0.6

0.59

0.59
Reaction Time

0.58

0.58

0.58

Day 45

Day 50

0.57
Baseline

ImPACT Test Time

Table 4.4. Intraclass Correlation Coefficients for ImPACT Composite Scores between Baseline
and Day 45, Day 45 and Day 50, Baseline and Day 50, and Overall
Verbal
Memory

Visual
Memory

Motor
Processing
Speed

Reaction
Time

Impulse
Control

Baseline to Day 45

.76

.72

.86

.68

.74

Day 45 to Day 50

.69

.66

.87

.83

.80

Baseline to Day 50

.66

.60

.84

.72

.57

Overall

.78

.75

.90

.81

.79

72

Table 4.5. Cronbach’s Alpha for ImPACT Composite Scores between Baseline and Day 45, Day
45 and Day 50, Baseline and Day 50, and Overall
Verbal
Memory

Visual
Memory

Reaction
Time

Impulse
Control

.73

Motor
Processing
Speed
.87

Baseline to Day 45

.77

.69

.74

Day 45 to Day 50

.69

.68

.87

.83

.80

Baseline to Day 50

.69

.59

.86

.73

.57

Overall

.80

.76

.91

.81

.79

Results of the Rey’s 15-item memory test (Hypothesis 5):
There will be no difference in test takers’ effort level as measured by the Rey’s 15-item memory
test over three test administrations (baseline, 45 days, 50 days).
Hypotheses 5 was supported as the moderate ICC values for test takers’ effort level and
almost no variance in the Rey’s 15-item memory test were reported. Effort level was calculated
using the Rey’s 15-memory test and asking the participant to rate their effort level on a scale of
0-10. The ICC values for test takers’ self-rated effort level were .84, .83, .78, and .74 for overall,
baseline and day 45, day 45 and day 50, and baseline and day 50, respectively. Cronbach’s Alpha
values were exactly the same as the ICC values. As for the Rey’s 15-item memory test, all
participants scored full score (15points) throughout the testing period with an exception of one
participant who scored 12 points at the second test. Due to its little variance, it was unable to
calculate ICC value for the Rey’s 15-item memory test. The means, standard deviations, and
range for Rey’s 15-item memory test and self-reported effort level can be found in Table 4.6.

73

Table 4.6. Means and Standard Deviations for Rey’s 15-item Memory Test, and Self-Reported
Effort Level
Baseline

45 days

50 days

Mean+ SD

Range

Mean+ SD

Range

Mean+ SD

Range

Rey’s 15
Memory Test

15±0.00

15-15

14.97±0.32

12-15

15±0.00

15-15

Self Reported
Effort Level

9.56±0.87

3-10

9.53±.87

7-10

9.27±1.20

4-10

Evaluation of Survey Data and Total Symptom Score
Along with ImPACT and Rey’s 15-item memory test, additional data was collected from the
participants in order to evaluate the participants’ physical and mental condition as well as the
quality of the testing environment.
Physical Activity Level
Participants were assessed their physical activity level throughout the study period based on
the amount of physical activity including moderate cardio exercise (4.8-7.1METs and PRE 1213), vigorous cardio exercise (7.2-10.1 MET and PRE 14-17), resistance exercise (two to four
sets of 8-12 repetitions on major muscle group), and neuromotor exercise (balance, agility,
coordination and gait). Participants who did not meet the set criteria were excluded from further
analysis (n=2). The means and standard deviations for the amount of the exercise at baseline, 45
days after the baseline, and 50 days after the baseline (minutes/week) can be found in Table 4.7.

74

Table 4.7. Means and Standard Deviations for the amount of the exercise at Baseline, 45 days
after the Baseline, and 50 days after the Baseline (minutes/week)
Baseline
Type of the
Exercise
Moderate
Cardio

45 days

Mean+ SD

Mean+ SD

50 days
Mean+ SD

133.75±136.72

112.13±95.92

102.24±88.58

Vigorous
Cardio

217.44±193.94

196.53±154.65

198.66±182.94

Resistance
Training

175.63±120.95

163.41±112.05

177.82±144.51

Neuromotor
Exercise

69.54±119.89

75.52±121.29

79.46±133.62

Condition of the Participants and Testing Environment
Along with the physical activity, in order to evaluate variables that may affect participants’
performance on ImPACT, information such as physical fatigue level, stress level, food
consumption, and caffeine intake of the testing day were gathered. Also, in order to evaluate the
testing environment, distraction level and presence of technical problems during testing were
assessed. The means, standard deviations, and range of those variables can be found in table 4.8
and figure 4.5. Pair-wise comparisons revealed significant differences for total symptom score
between day 45 and day 50 (mean difference: 2.06) and between baseline and day 50 (mean
difference: 1.97), physical fatigue level between baseline and day 45 (mean difference: -.80) and
between day 45 and day 50 (mean difference: .50), distraction level between day 45 and day 50
(mean difference: -.57) and between baseline and day 50 (mean difference: -.45). The mean
differences and p values can be found in table 4.9, and their statistical and clinical significance
will be discussed in next chapter. The overall ICC values showed that the physical and mental
condition of the participants were consistent throughout the testing period (.90, .85, .73, .80,

75

and .64 for ImPACT total symptom score, stress level, physical fatigue level, food consumption,
and caffeine intake, respectively) with an exception of hours of sleep (.56). Also, low distraction
level (1.58, 1.55, and 2.06 at baseline, day 45, and day 50, respectively) with acceptable
consistency (overall ICC of .65) was reported. Total of three subjects reported technical
problems including difficulty of pointing the mouse (n=1), slow response of mouse (n=1),
freezing of the ImPACT program (n=1), subsequently they were excluded from further analysis.

Table 4.8. Summary of the Condition of the Participants and Testing Environment
Baseline
Mean+ SD

45 days

Range

Mean+ SD

50 days

Range

Mean+ SD

Range

Total
Symptom

10.88±10.93

61-0

10.97±11.79

53-0

8.91±11.19

47-0

Hours of
Sleep

6.99±1.53

0-10

7.23±1.92

3-15

6.78±1.62

0-9

4.98±2.21

0-9

4.55±2.38

0-10

4.40±2.74

0-10

3.12±2.30

0-8

3.81±2.61

0-9

3.41±2.68

0-10

7.0±2.8

0-10

6.8±3.5

0-10

6.5±3.5

0-10

4.2±1.8

0-8

4.4±1.7

0-10

3.8±1.9

0-9

1.58±1.70

0-8

1.55±1.97

0-9

2.06±2.21

0-8

Stress

a

Physical
a

Fatigue
Food
b

Intake
Caffeine
c

Intake

a

Distraction
a

rated on scale of 0-10, where 0 is not stressed/fatigued/distracted at all and 10 is very
stressed/exhausted/distracted
b

rated on scale of 0-10, where 0 is not typical food intake at all and 10 is very typical food
intake
c

rated on scale of 0-10, where 0 is much less than typical caffeine intake, 5 is typical
caffeine intake, and 10 is much more than typical caffeine intake

76

Table 4.9. Pair-wise Comparison of the Condition of the Participants and Testing Environment at
Baseline, 45 days after the Baseline, and 50 days after the Baseline
Mean Difference (p-value)
Baseline vs Day 45

Day 45 vs Day 50

Baseline vs Day 50

Total Symptom

-.09 (.918)

2.06 (.006)*

1.97 (.030)

Hours of Sleep

-.24 (.455)

.46 (.455)

.22 (.455)

.357 (.115)

.214 (.254)

.571 (.023)*

-.80 (.008)*

.500 (.077)

-.298 (.276)

.244 (.486)

.280 (.382)

.524 (.143)

-.143 (.549)

.548 (.011)*

.405 (.084)

.119 (.639)

-.571 (.023)*

-.452(.044)*

Stress

a
a

Physical Fatigue
b

Food Intake

c

Caffeine Intake
a

Distraction
a

rated on scale of 0-10, where 0 is not stressed/fatigued/distracted at all and 10 is very
stressed/exhausted/distracted
b
rated on scale of 0-10, where 0 is not typical food intake at all and 10 is very typical food
intake
c
rated on scale of 0-10, where 0 is much less than typical caffeine intake, 5 is typical caffeine
intake, and 10 is much more than typical caffeine intake
*The mean difference is significant at the .05 level

77

Figure 4.5. Condition of the Participants and Testing Environment at Baseline, 45 days after the
Baseline, and 50 days after the Baseline
12
10
Stress

Likert Scale

8

Physical Fatigue
Food Intake

6

Caffeine Intake
Distraction

4

Total Symptom Score (pts)
Hours of Sleep (hours)

2
0
Baseline

45 days

50 days

Evaluation of Association between ImPACT Composite Scores and Survey Data and Total
Symptom Score
Pair-wise comparison revealed mean differences with statistical significance in some of
the ImPACT composite scores, survey data, and total symptom score between testing days.
Linear multiple regressions analysis with score difference in ImPACT composite scores between
testing days as the dependent variables and corresponding score difference in survey data and
total symptom score that showed statistical significant mean difference in the previous analysis
as the independent variables was conducted in order to assess if any of the statistically significant
changes in survey data and total symptom score explain the score changes in ImPACT composite
scores. The analysis revealed a significant relationship between the change of visual memory
2

composite score and the change of distraction level (R =.066, B=-1.40, t=-2.46, and p=.016)

78

with correlation of -.256 (p=.008) between 45 and 50 days after the baseline, and between the
2

change in verbal memory composite score and the change of total symptoms score (R =.050,
B=.247, t=2.125, and p=.036) with correlation of .223 (p=.018) between baseline and 50 days
after the baseline.

79

CHAPTER V
DISCUSSION
Introduction
The primary purpose of the current study was to re-examine the test-retest reliability of
the ImPACT neurocognitive test battery between baseline, 45 days and 50 days after baseline on
physically active college students while assessing the participants’ physical and mental condition
on the testing day. The ICC values on each composite score from the current study met or
exceeded the .60 level, which is the indication of acceptable test-retest reliability (Baumgartner
& Chung, 2001), and were higher than the ICC values obtained by Broglio et al. (2007). In
chapter 5, this primary result will be discussed in relation to the relevant literature on test-retest
reliability of the ImPACT neurocognitive test battery. While acceptable test-retest reliability was
obtained, performance fluctuation was observed through the testing period in the absence of
concussion. In order to explore the performance fluctuation, the survey results will be reviewed
and discussed in relation to the relevant literature on the effects of the physical and mental
condition on neurocognitive function. Finally, clinical implications of these findings and
suggestions for future research will be proposed.
Comparison of ICC Values with Other Studies
Comparison with the Study of Broglio et al. (2007)
ICC values from the current study were higher than the ICC values obtained by Broglio
et al. (2007) for all of the composite scores measured. The ICC ranged from .60 to .87 for the
current study, while .15 to .61 for the study by Broglio et al. While all ICC values in the study of
Bloglio et al. were greater between day 45 and day 50 than between baseline and day 45 (ICC
values between baseline and day 50 was not available), the same trend was not observed in the

80

current study, which suggests that a shorter testing interval (45 days vs 50 days) does not
necessarily result in higher ICC values in ImPACT composite scores. The comparison of ICC
values between two studies can be found in Table 5.1.

Table 5.1. Comparison of ICC Values between the Current Study and Broglio et al. (2007)
Verbal
Memory

Visual
Memory

Motor
Processing
Speed

Reaction
Time

Impulse
Control

Baseline to Day 45
Current Study
Broglio et al. (2007)

.76
.23

.72
.32

.86
.38

.68
.39

.74
.15

Day 45 to Day 50
Current Study
Broglio et al. (2007)

.69
.40

.66
.39

.87
.61

.83
.51

.80
.54

Baseline to Day 50
Current Study
Broglio et al. (2007)

.66
N/A

.60
N/A

.84
N/A

.72
N/A

.57
N/A

Overall
Current Study
Broglio et al. (2007)

.78
N/A

.75
N/A

.90
N/A

.81
N/A

.79
N/A

There are several factors that may contribute to the differences between Broglio et al.
(2007) and the current study. First, there is a notable difference in the primary reason of data
exclusion between the two studies. While invalid ImPACT baseline assessment accounted for
64% of the data exclusion (29 of 45 exclusions) in the study of Broglio et al., invalid baseline
(i.e., flagged by ImPACT as invalid) assessment accounted for less than 3% (1/39) of the data
exclusion in the current study. When a value greater than 30 on Impulse Control is included as a
reason for data exclusion (n=2), test takers’ performance related reasons (an invalid baseline and

81

a value greater than 30 on Impulse Control) accounted for less than 8% (3/39) of data exclusion
in the current study, which is still considerably low compared with the study of Broglio et al. It
was not clear whether a value greater than 30 on Impulse Control scores were included as an
indicator of invalid assessment in the study of Broglio et al. Rather, in the current study, missing
appointments was the major reason for data exclusion, which accounted for 79% (31 of 39
exclusions) of all excluded subjects.
Schatz and colleagues (2012) examined the prevalence of invalid baseline based on the
ImPACT Version 2.0 Clinical Users’ Manual, and reported that 4.1% of 1388 collegiate athletes
had at least one of the indicators for invalid assessment including a value greater than 30 on
Impulse Control scores. Also, other test-retest reliability studies with different testing intervals
reported comparably low rate of invalid ImPACT baseline assessment including 5.9% rate of
invalid baseline in 117 collegiate athletes in the study of Schatz et al. (2010) and 7.8% rate of
invalid baseline in 484 high school athletes in the study of Elbin et al. (2011). While the current
study reported 0.8% (1 of 127) of invalid baseline, it is obvious that 26% rate of invalid
baseline in the study of Broglio et al. (2007) is notably high when it is compared to not only the
current study, but also other studies.
Second, the high number of invalid baseline data in the study of Broglio et al. (2007)
calls into a question the quality of test taker’s performance on ImPACT including their level of
effort. It is reasonable to assume that their participants were not able to provide their best
performance due to fatigue and stress caused by three consecutive administrations of
neurocognitive tests. One may argue that invalid data were excluded from analysis so that the
analyzed data was reliable. Erdal (2012) suggested that ImPACT test takers can report
significantly lower scores than their baseline assessments without detection by validity indicators,

82

which indicates that having valid data at two testing points does not necessarily mean their
performance are the same.
Third, even though Broglio et al. (2007) stated that the participants exhibited good effort
on all days of testing based on the results from Memory and Concentration Test for Windows
(MACT), there is no evidence that MACT can accurately assess the effort that is required to
complete ImPACT. In fact, in the current study, test takers’ effort was assessed by the Rey’s 15item memory test that is used for detecting malingering (Goldberg & Miller, 1986; Bernard &
Fowler, 1990) and all three participants who reported invalid ImPACT assessment had a perfect
score on the Rey’s 15-item memory test. Thus, this indicates that the Rey’s 15-item memory test
was not sensitive to assess the effort that is required for ImPACT.
Finally, the administration of the three neuropsychological tests and the MACT was not
clearly stated in the study of Broglio et al. (2007). If the authors randomized the order of the
three test administrations at every testing occasion, it could have affected the ICC values. If
ImPACT was administered as the first test of three at baseline and as the last test of three at the
day 45, it is not hard to expect a better performance at baseline than day 45. At the same time, if
the authors did not randomized the order of testing administration, the reported ICC values on
ImPACT composite scores were reliable only if ImPACT was the first neurocognitive test
administered so that it was not affected by the previously administered neurocognitive test.
Moreover, unclear timing of MACT administration calls into a question the reliability of effort
assessment in their study. As a result, Broglio’s et al. research design of administering three
different neurocognitive tests consecutively contributed to the low reliability/ICC of ImPACT.

83

Comparison with Elbin et al. (2011) and Schatz et al. (2010)
While Miller et al. (2007) used 80% confidence interval instead of ICC, Elbin et al. (2011) and
Schatz et al. (2010) reported ICC values of ImPACT composite scores with 1-year and 2-year
testing interval. ICC values reported by Elbin et al. exceeded the cut-off value of 0.60 for all
composite scores, and the same result was reported in the study of Schatz et al. with an exception
of Visual Memory (ICC=.46). With a few exceptions, the ICC values obtained in the current
study were generally greater than these two studies, and several factors could explain the
differences. First, the current study used a shorter testing interval compared to the study of Elbin
et al. and Schatz et al. This shorter test period could have resulted in high ICC. Second, Elbin et
al. used high school athletes with a mean age of 14.8 years compared to the current study which
used collegiate students. The adolescent brain is still developing which could have affected the
ICC values. Third, data in the current study was obtained from a volunteered non-athlete
population while both the other two studies used volunteered athletes. Thus, physical fitness of
the participants could have attributed to the differences in ICC values.
Test-Retest Reliability Difference across the Composite Scores
The majority of research studies examining ICC of the ImPACT test, including this study,
suggest that some ImPACT composite scores have higher test-retest reliability than other
composite scores. The highest ICC value was always found in Motor Processing Speed with an
exception of the study by Broglio et al. (2007) between baseline to day 45. In contrast, the lowest
ICC was always reported from either Verbal Memory or Visual Memory when Impulse Control
is excluded. It is important for clinicians to acknowledge that the test-retest reliability
systemically varies across ImPACT composite scores. The summary of the ICC values across the
studies can be found in Table 5.2.

84

Table 5.2. Comparison of ICC Values across the Test-Retest Studies
Study &
Testing Interval

Miller et al. (2007)
Baseline to120 days

Verbal
Memory

Visual
Memory

Motor
Processing
Speed

.62

Schatz et al. (2010)
Baseline to 2 year

.46

Broglio et al. (2007)
Baseline to Day 45
Day 45 to Day 50
The current study
Baseline to day 45
Baseline to day 50
Day 45 to day 50
Overall

b

.70

.85

b

.65

.23
.40

b

.32

.76
.69
.66
.78

.72

N/A

a

.68

N/A

.38
a

.39

N/A

a

b

.76

.74

.39
.51

.15
.54

.68
.80
.72
.81

.74
.80
.57
.79

a

.61

b

a

.86

b

a

.66

.87

b

a

.60

.84

b

a

.75
b

Impulse
Control

No statistical difference with 80% confidence interval

Elbin et al. (2011)
Baseline to 1 year

a

Reaction
Time

.90

the highest ICC value among the composite scores obtained at a time
the lowest ICC value among the composite scores obtained at a time

Discussion of Survey Data
General Discussion
In order to simulate athletes, the current study was conducted with physically active
individuals. Physical activity level of the participants at each testing sessions was assessed in
order to confirm that the participants satisfied the ACSM guideline for physical activity
recommendation. The majority of participants satisfied and kept their required physical activity
level throughout the testing periods. Three participants reported no resistance training at some

85

testing sessions, but were still included in data analysis due to their high amount of vigorous
intensity exercise (more than 150 minutes per week). Even though the average of moderate
intensity exercise fell below the recommended 150 minutes per week (average of 116 minutes
per week) , the shortfall of moderate intensity exercise was compensated by high amounts of
vigorous intensity exercise that averaged more than twofold greater than recommended 75
minutes per week (average of 204 minutes per week). The average amount of resistance training
was 171 minutes per week, and a participant was assumed to satisfy the recommended amount of
resistance training (two to four sets of 8-12 repetitions on major muscle group two or three days
week) if he or she reported more than 30 minutes of resistance training per week. While it was
not an exclusionary criterion, the average amount of neuromotor exercise surpassed the
recommended 20-30 minutes of two to three times per week with approximately 75 minutes per
week.
Kamijo and Takeda (2008) reported a beneficial effect of physical activity on cognitive
function in young adults. In their study, the energy consumption of the participants was
measured by the International Physical Activity Questionnaire (IPAQ), and the participants in
the active group averaged 6592.1kcal/week of estimated leisure time energy consumption
whereas the participants in the sedentary group had an average of 582.4kcal/week in the same
domain. However, the IPAQ takes into account daily walking thus elevating their energy
consumption. In the current study, the average of participants’ estimated energy consumption
based on their body weight, duration of the physical activity, and intensity of the physical
activity (5.9METS for moderate intensity exercise, resistance training, and neuromotor exercise,
and 8.6METS for vigorous exercise) were 5176.3, 4754.6 and 4739.3 kcal/week at baseline, day
45, and day 50. Thus participants in the current study are thought to be physically active enough

86

to receive a benefit on their cognitive function. Therefore, the aim of using physically active
participants in order to simulate athletes who are thought to benefit on their cognitive function
due to their high physical activity level was accomplished.
Test performance can be influenced by the participants’ physical and mental condition, as
well as by the testing environment. The effect of exercise induced fatigue on cognitive function
has been studied with different methods, intensity of exercise, cognitive tasks, and timing of
cognitive task administration. A meta-regression analysis reported the complex relation between
exercise and cognition (Lambourne & Tomporowski, 2010). Using ImPACT as the measurement
instrument, Covassin et al. (2007) reported a limiting effect of maximal exercise on verbal
memory composite score with statistical significance; therefore, vigorous exercise prior to the
test was discouraged. Attempts were also made to minimize the effects of environment by testing
a participant in the same reserved quiet computer lab at the same time of day for all test sessions
with average of approximately five participants at a time. Based on the survey data rated on a
scale of 0 to 10, as a general impression, the participants were moderately stressed (average ~5)
and mildly fatigued (average ~3.5) , and series of ImPACT were administered under low level of
distraction (average~1.5). The relationship between the data from survey/total symptoms score
and ImPACT composite scores will be discussed in the next section.
Statistical vs Clinical Significance
Even though some differences in participants’ physical and mental condition between
testing days were statistically significant, they can be viewed as non clinically significant. The
mean statistical difference for total symptom score was an average of 2.0 which is extremely low
considering the range was between 0 and 132. The mean statistical difference for physical
fatigue and distraction was on average 0.50 out of a scale from 0-10. Thus all mean differences

87

for physical fatigue level and distraction level were less than one, which is smaller than its rating
increment, and the mean differences for total symptom score were also considerably small at
approximately two. Therefore, despite their statistical significances, the participants’ physical
and mental condition can be considered clinically stable during the testing period along with the
other physical, mental, and environmental conditions that had no statistical significant
differences between testing days.
While the effect of fatigue induced by acute exercise has been studied, the effect of
general physical fatigue on cognitive function has not been studied. In the current study, the
mean of participants’ physical fatigue level on average was 3.5 on a scale of 0 to 10. The
multiple regressions revealed that there were no significant relationships between the changes in
physical fatigue level and any of the changes in ImPACT composite scores, even when the
difference of physical fatigue level was statistically significant between day 45 and 50. This
result may be attributed to the small mean difference of physical fatigue levels between testing
days.
Unlike the physical fatigue level, significant relationships were observed between the
changes in visual memory composite score and the changes in distraction level between day 45
and 50, and the changes in verbal memory composite score and the changes in total symptom
score between baseline and day 50 ; however, neither of these relationships were strong. The R2
and B values for distraction level was .066 and -1.40, which indicates only 6.6% of score change
in visual memory can be explained by the change in the distraction level, and one increase in
distraction level results in 1.4 point decrease in visual memory. Also, the R2 and B values for
total symptom score was .05 and .247, which indicates only 5% of the score change can be
explained by the change in the total symptom score, and one increase in total symptom score

88

results in .247 increase in verbal memory. Only two statistically significant but weak interactions
between changes in ImPACT composite scores and changes in survey/total symptom score data
were observed. Therefore, in the current study, changes in the participants’ physical and mental
condition assessed through the survey and total symptom score do not explain the changes in
ImPACT composite score.
Implications of Findings
While utilization of computerized neurocognitive assessments employing preparticipation baseline test followed by a series of post-concussion tests has become a crucial
element of the multidisciplinary approach to concussion evaluation and management
(Guskiewicz et al., 2004; McCrory et al., 2009), there has been an on-going discussion and
debate on the test-retest reliability of computerized neurocognitive tests among clinicians and
researchers. This study’s findings support the test-retest reliability of one of the most widely
used computerized neurocognitive test batteries, ImPACT, by providing acceptable ICC values
from baseline to 45 and 50 days. The results of this study also contradict Broglio et al.’s (2007)
low test-retest reliability using the same testing intervals. Moreover, the current study’s findings
agree with other test-retest studies by Millar et al., (2007), Elbin et al. (2011), and Schatz et al.
(2010), which suggests that ImPACT can be a reliable tool used for the multidisciplinary
approach to concussion management. However, clinicians must be aware that computerized
neurocognitive assessments are not intented to be a stand alone diagnostic measure (McCrory et
al, 2009).
When clinicians utilize ImPACT for concussion management including return-to-play
decisions by comparing baseline and post-concussion data, they should feel more confident in
knowing that the ImPACT test has been found to have good test-retest reliability. In the past,

89

clinicians had difficulty accepting that the changes athletes exhibited on ImPACT postconcussion were truly due to their concussion, rather than a flaw in the ImPACT test. However,
only trained clinicians should be responsible for supervising ImPACT tests and reading the
results of this computerized neurocognitive test.
Limitations
Several limitations are recognized that may compromise the external validity of the
findings from the current study. With respect to the subjects studied in the current study, the
most limiting factor is that the participants were physically active college students, not athletes.
The ImPACT test is utilized with high school, collegiate and professional athletes, rather than the
typical physically active college student. Even though the participants are physically active
based on the ACSM recommendation, caution should be taken when attempting to generalize the
findings to the athletic population. Also, it is important to note that the current results were
obtained from college students and should not be generalize to other age groups, especially to
younger populations whose brains are still developing. In addition, while Broglio et al. (2007)
assessed the participants’ SAT scores, it was not assessed in the current study. Academic
achievement of the participants could have affected the result and be a potential confounder.
Another limitation of the current study involves the validity of self-reported data
including the participants’ mental/physical conditions in testing days and their physical activity
level. Regarding the physical activity level, even though the participants were not sedentary
population who tend to overestimate the intensity of their physical activity (Duncan et al. 2001),
the highest level of intensity in the scale represents activities >7.2 METs, which was not specific
enough to differentiate high and vigorous physical activities.

90

Finally, test takers’ level of effort was measured by Rey’s 15-item memory test and a self
reported survey. However, due to all three participants who reported an invalid baseline and/or a
value greater than 30 on Impulse Control score had a perfect score on the Rey’s 15-item memory,
the effort assessment in the current study was thought to be not sensitive enough for ImPACT
administration.
Suggestions for Future Research
Given the limitation of the current study regarding the sample population, additional data
should be collected using the same research design with athletic populations and different age
groups to increase external validity. Also, future research would need to investigate the testretest reliability of ImPACT with different time intervals, especially between the current study
and the study of Elbin et al. (2011) whose testing interval was 1-year.
It would be beneficial to examine the test-retest reliability of ImPACT using
asymptomatic participants who have a history of multiple concussions. As athletes with
concussion history have a higher risk of sustaining another concussion (McCrory et al. 2009),
they have a higher chance of utilizing ImPACT. Therefore, obtaining test-retest reliability from
this population may enhance the clinical application of ImPACT. In the current study, nine
participants had a history of multiple concussions (2 to 4), but the sample size was too small to
draw any statistical conclusions.
Lastly, even though it was weak, a significant association of environmental distraction
and ImPACT performance was observed in the current study. The effects of test environment on
ImPACT performance need to be studied further because in reality the optimal environment may
not always be available. For example, in an athletic setting, baseline data tend to be obtained in a
group setting that may create some distraction from teammates, whereas post-concussion testing

91

is more likely to be in an individual setting. It would be beneficial to study the testing
environment quality standard for ImPACT in order to ensure test-retest reliability in a real world
setting.
Conclusion
This research replicated the study by Broglio et al. (2007) that reported a low test-retest
reliability of one of the most widely used neurocognitive testing battery, ImPACT, while
assessing the test takers’ physical/mental condition and the quality of testing environment. The
current study reported ICC values ranging from .60 to .87 with physically active collegiate
students who were moderately stressed, mildly fatigued, and slightly distracted by the
environment. These findings are in contrast to Broglio et al. who reported ICC values rangeing
from .23 to .61. The ICC values obtained from the current study is considered to be more reliable
because Brogio’s et al. had methodological flaws including administering three neurocognitive
test batteries successively. According to the current findings, ImPACT has acceptable test-retest
reliability when it is administered at baseline, 45 days after baseline, and 50 days after baseline,
which support the utilization of ImPACT in concussion management as a part of
multidisciplinary approach.

92

APPENDICES

93

APPENDIX A: Human Subjects Consent Form
Examination of Test-Retest Reliability of a Computerized Neurocognitive Test Battery
Informed Consent for Participant
For questions regarding this study,
Please contact:
Tracey Covassin Ph.D, ATC
Department of Kinesiology
Michigan State University
East Lansing, MI 48824
Phone: (517) 353-2010
E-mail: covassin@msu.edu

For questions regarding your rights
as a research participant, please contact:
MSU's Human Research Protection Program
Michigan State University
408 W. Circle Dr. Room 207 Olds Hall
East Lansing, MI 48824
irb@msu.edu
Phone: (517) 355-2180
Fax: (517) 432-4503

Purpose of Study: The purpose of this study is to examine the test-retest reliability of one of
the major computerized neurocognitive test batteries used for concussion management
(ImPACT) through three test administrations (baseline, 45 and 50 days after the baseline).
Consent: Your participation in the research study is voluntary, as you may choose not to
participate at all, or may refuse to answer certain questions, or discontinue participation at any
time without penalty.
General Experimental Procedures: You will be administered the ImPACT test battery on a
computer three times (baseline, 45 days and 50 days after the baseline). The testing consists of
three categories: Demographic, current symptoms, and neuropsychological modules (e.g., word
memory, design memory, symbol matching, and so forth). The neuropsychological modules
consist of six domains of cognitive functions that form 5 composite scores: Verbal memory,
Visual memory, Motor processing speed, Reaction time, and Impulse Control. Prior to each
administration of ImPACT, you will be administered the Rey’s 15-Item Memory Test, which
consists of 15 symbols on a piece of paper for you to memorize them in 30 seconds, and then
reproduce them in two minutes. You will also complete a short demographic questionnaire that
asks questions pertaining to physical activity level, caffeine and alcohol intake, exercise, and
stress level the day of testing. Finally you will complete a very short post-questionnaire that
asks questions about whether you were distracted by the surroundings during the testing,
faced any technical issues, and understood the test instructions. Testing will be conducted in a
quiet computer laboratory under direct supervision. The length of each testing session will be
approximately 30 to 40 minutes. Total time for three test sessions will be approximately two
hours. ImPACT is not meant to reflect a person's intelligence or level of achievement. Subject's
participation may be terminated at the request of the investigator is they are not able to
complete all three testing sessions.
Instruction for the participants: Due to possible adverse effects of physical exhaustion on
cognitive function, you will be instructed to avoid vigorous exercise and to be well rested for
the day of testing.

94

Possible Risks: There will be little or no discomfort to you. You will be asked to answer
questions about learning disabilities and neurological problems. Please be assured that you may
choose to not answer certain questions and still continue to participate in this study. All
answers are strictly confidential and will not be released to anyone other than those listed in
the confidentiality section below.
Benefits: You will not directly benefit from participation in this study. However, the result of
this study will contribute to the development of the ImPACT test battery and concussion
management.
Compensation: With the completion of participation, you will automatically be entered into a
drawing to win a signed Michigan State University basketball or football.
Confidentiality/Anonymity: Participation in this study is completely voluntary. The only people
who have access to your answers are the certified athletic trainer, faculty members, HRPP
auditors, other funding auditors, or the research integrity office. However, your identity and
information recorded during the study will remain confidential. Confidentiality will be
protected by: (a) results will be presented in aggregate form in any presentations and
publications; and (b) all data will be stored in a computer that has a password necessary to see
confidential data. Your confidentiality will be protected to the maximum extent allowable by
law. You may also discontinue participation at any time without penalty. Your participation in
this research project will not involve any additional costs to you or your health care insurer.
Additionally, it may be necessary for the Michigan State University Institutional Review Board
to perform a confidential audit of the research data and protocols. Data will be kept under
double lock and key for 5 years after the close of the study.
Institutional Contacts: If you have questions or concerns about your role and rights as a
research participant, would like to obtain information or offer input, or would like to register a
complaint about this study, you may contact, anonymously if you wish, the Michigan State
University's Human Research Protection Program at 517-355-2180, Fax 517-432-4503, or e-mail
irb@msu.edu or regular mail at 408 W. Circle Dr. Room 207 Olds Hall, MSU, East Lansing, MI
48824.
If you have any concerns or questions about this research study, such as scientific issues, how
to do any part of it, or to report an injury, please contact the researcher (Tracey Covassin at
517-353-2010 or email covassin@msu.edu or regular mail at 105 IM Circle, MSU, East Lansing,
MI, 48824). I voluntarily agree to participate in the study.
I, ____________________________ have read and agree to participate in this study as
described above. (Please Print Your Name)
___________________________

_____/_____/_____

(Please Sign Your Name)

(Date)

95

APPENDIX B: Figure A.1. Rey’s 15-item Memory Test
Rey’s 15-item Memory Test
Please memorize the symbols as many as you can in 30 seconds. You will be asked to
reproduce them without worrying about their order.

96

Figure A.1. (continued)
Rey’s 15-item Memory Test Answer Sheet
Test Taker #_________________Date___________

Please reproduce the 15 symbols as many as you can without worrying about the order.
You have 2 minutes to reproduce.

97

APPENDIX C: Pre-Testing Survey
Examination of Test-Retest Reliability of a Computerized Neurocognitive Test Battery
Pre-testing Survey
Test Taker #_________________Date___________
The purpose of this survey is to gather information that might affect your performance on
ImPACT. Your identity and information recorded during the study will remain confidential.
Data will be kept under double lock and key.

1. Please indicate your general exercise intensity, duration, and frequency
Cardiovascular exercise
•
•

Moderate intensity (harder than brisk walking) ______min/week
Vigorous intensity (e.g., jogging >5mph, sports such as basketball, soccer, tennis,
swimming) ______min/week

Resistance exercises (including body weight exercise such as pushups, sit-ups, and squat)
•

______min/session ______days/week

Neuromotor exercise (balance, agility, and coordination exercise)
•

______min/session

______ days/week

2. Please rate your current level of stress at this moment?
Not Stressed at all
0

1

2

Very stressed
3

4

5

6

7

8

9

10

3. Did you do vigorous exercise today?
Yes

No

If yes, what exercise and how long? __________________

98

4. Please rate your level of physical fatigue at this moment?
None
0

Exhausted
1

2

3

4

5

6

7

8

9

10

5. Did you eat today?
Yes

No

6. Please indicate if your food consumption was typical to your normal food
consumption?
Not at all
0

1

Very Typical
2

3

4

5

6

7

8

9

10

7. Please indicate your alcohol intake in last 12 hours?
Not at all
0

1

Moderate
2

3

4

5

6

A lot
7

8

9

10

8. Please indicate if your caffeine intake was typical to your normal caffeine intake?
Much less
0

1

Typical
2

3

4

5

Much more
6

7

8

9

10

Thank you. Please start ImPACT test on the computer screen. After ImPACT test, please
complete the post-testing survey

99

APPENDIX D: Post-Testing Survey
Examination of Test-Retest Reliability of a Computerized Neurocognitive Test Battery
Post-testing Survey
The purpose of this survey is to gather information that might have affected your performance
on ImPACT. Your identity and information recorded during the study will remain confidential.
Data will be kept under double lock and key.
1. Please rate how distracted you were by the surroundings during the testing?
Not at all
0

1

Distracted a lot
2

3

4

5

6

7

8

9

10

2. Were there any technical issues that affected your performance (e.g. slow mouse
response)?
Yes

No

If yes, please explain ___________________________________

3. Were the instructions clear for the Rey’s 15 items memory test?
Yes

No

If no, which one/s? ____________________________________

4. Were the instructions clear for the ImPACT test?
Yes

No

If no, which one/s? ____________________________________

5. Please rate your level of effort on taking ImPACT test?
Did not try at all
0

1

2

Tried my best
3

4

5

6

7

8
100

9

10

REFERENCES

101

REFERENCES

Ahlskog, J. E., Geda, Y. E., Graff-Radford, N. R., & Petersen, R. C. (2011). Physical exercise as
a preventive or disease-modifying treatment of dementia and brain aging. Mayo Clinic
Proceedings, 86(9), 876–884.
Almosnino, S., Pelland, L., & Stevenson, J. M. (2010). Retest reliability of forcetime variables of neck muscles under isometric conditions. Journal of Athletic Training,
45(5), 453–458.
American Academy of Neurology. (1997). Practice parameter: the management of concussion in
sports (summary statement). Report of the Quality Standards Subcommittee. Neurology,
48, 581-585.
Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K., Kelly, J., Lovell, M.
(2002). Summary and Agreement Statement of the First International Conference on
Concussion in Sport, Vienna 2001. British Journal of Sports Medicine, 36(1), 6–7.
Barth, J. T., Alves, W., Ryan, T., Macciocchi, S. N., Rimel, R. W., Jane, J. A. (Eds.).
(1989). Mild head injury in sports: Neuropsychological sequelae and recovery of
function. New York, NY: Oxford University Press.
Baumgartner, T. A., & Chung, H. (2001). Confidence cimits for intraclass reliability
coefficients. Measurement in Physical Education and Exercise Science, 5(3), 179–188.
Belanger, H. G., & Vanderploeg, R. D. (2005). The neuropsychological impact of sports-related
concussion: a meta-analysis. Journal of the International Neuropsychological Society:
JINS, 11(4), 345–357.
Benson, B. W., Meeuwisse, W. H., Rizos, J., Kang, J., & Burke, C. J. (2011). A prospective
study of concussions among National Hockey League players during regular season
games: the NHL-NHLPA Concussion Program. Canadian Medical Association. Journal,
183(8), 905–11.
Bernard, L. C., & Fowler, W. (1990). Assessing the validity of memory complaints: Performance
on brain-damaged and normal individual on Rey’s task to detect malingering. Journal of
Clinical Psychology, 49, 432-436.
Blaylock, R., & Maroon, J. (2011). Immunoexcitotoxicity as a central mechanism in chronic
traumatic encephalopathy-A unifying hypothesis. Surgical Neurology International, 2(1),
107.

102

Bleiberg, J., Kane, R. L., Reeves, D. L., Garmoe, W. S., & Halpern, E. (2000). Factor analysis of
computerized and traditional tests used in mild brain injury research. The Clinical
neuropsychologist, 14(3), 287–294.
Bleiberg, J., Cernich, A. N., Cameron, K., Sun, W., Peck, K., Ecklund, L. J., Reeves, C.
D. (2004). Duration of cognitive impairment after sports concussion. Neurosurgery, 54(5),
1073–1080.
Broglio, S. P., Ferrara, M. S., Macciocchi, S. N., Baumgartner, T. A., & Elliott, R. (2007). Testretest reliability of computerized concussion assessment programs. Journal of Athletic
Training, 42(4), 509–14.
Cantu, R. C. (1992). Cerebral concussion in sport. Management and prevention. Sports Medicine
(Auckland, N.Z.), 14(1), 64.
Cantu, R. C., & Voy, R. (1995). Second impact syndrome--a risk in any contact sport. The
Physician and Sportsmedicine, 23, 27–35.
Cantu, R. C., & Mueller, F. O. (2009). The prevention of catastrophic head and spine injuries in
high school and college sports. British Journal of Sports Medicine, 43(13), 981–986.
Cantu, R. C. (1998). Second-impact syndrome. Clinics in Sports Medicine, 17(1), 37–44.
Cernich, A., Reeves, D., Sun, W., & Bleiberg, J. (2007). Automated neuropsychological
assessment metrics sports medicine battery. Archives of Clinical Neuropsychology, 22,
Supplement 1(0), 101–114.
Chadwick, J., & Mann, N. W. (1950). Hippocrates in English: The medical works of hippocrates,
a new translation from the original greek made for english readers. Oxford: Blackwell.
Collie, A., Maruff, P., Makdissi, M., McCrory, P., McStephen, M., & Darby, D. (2003).
CogSport: reliability and correlation with conventional cognitive tests used in
postconcussion medical evaluations. Clinical journal of sport medicine: official journal
of the Canadian Academy of Sport Medicine, 13(1), 28–32.
Collins, M. W. (1999). Current issues in managing sports-related concussion. JAMA: The
Journal of the American Medical Association, 282(24), 2283–2285.
Collins, M. W, Grindel, S. H., Lovell, M. R., Dede, D. E., Moser, D. J., Phalin, B. R.
(1999). Relationship between concussion and neuropsychological performance in college
football players. JAMA: The Journal of the American Medical Association, 282(10), 964–
970.
Collins, M. W, & Hawn, K. L. (2002). The clinical management of sports concussion.
Current sports medicine reports, 1(1), 12–22.

103

Collins, M. W., Iverson, G. L., Lovell, M. R., McKeag, D. B., Norwig, J., & Maroon, J.
(2003). On-field predictors of neuropsychological and symptom deficit following sportsrelated concussion. Clinical journal of sport medicine: official journal of the Canadian
Academy of Sport Medicine, 13(4), 222–229.
Covassin, T., Elbin, R. J., & Nakayama, Y. (2010). Tracking neurocognitive performance
following concussion in high school athletes. The Physician and sportsmedicine, 38(4),
87–93.
Covassin, T., Elbin, R. J., & Sarmiento, K. (2012). Educating coaches about concussion in
sports: Evaluation of the CDC’s “Heads Up: Concussion in youth sports” initiative.
Journal of School Health, 82(5), 233–238.
Covassin, T., Schatz, P., & Swanik, C. B. (2007). Sex difference in neuropsychological
function and post-concussion symptoms of concussed collegiate athletes. Neurosurgery,
61(2), 345–351.
Covassin, T., Swanik, C. B., & Sachs, M. L. (2003a). Sex differences and the incidence of
concussions among collegiate athletes. Journal of Athletic Training, 38(3), 238.
Covassin, T., Swanik, C. B., & Sachs, M. L. (2003b). Epidemiological considerations of
concussions among intercollegiate athletes. Applied Neuropsychology, 10(1), 12–22.
Covassin, T., Weiss, L., Powell, J., & Womack, C. (2007). Effect of a maximal exercise test on
neurocognitive function. British Journal of Sports Medicine. 41 (6), 370-374.
Cross, K. M., & Serenelli, C. (2003). Training and equipment to prevent athletic head and neck
injuries. Clinics in Sports Medicine, 22(3), 639–667.
Denny-Brown, D., & Russell, W. R. (1941). Experimental cerebral concussion. Brain, 64(2-3),
93–164.
Duncan, G. E., Sydeman, S. J., Perri, M. G., Limacher, M. C., & Martin, A. D. (2001). Can
sedentary adults accurately recall the intensity of their physical activity? Preventive
Medicine, 33 (1), 18-26.
Echemendia, R. J., Putukian, M., Mackin, R. S., Julian, L., & Shoss, N. (2001).
Neuropsychological test performance prior to and following sports-related mild traumatic
brain injury. Clinical journal of sport medicine: official journal of the Canadian
Academy of Sport Medicine, 11(1), 23–31.
Elbin, R. J., Schatz, P., & Covassin, T. (2011). One-year test-retest reliability of the online
version of impact in high school athletes. The American Journal of Sports Medicine,
39(11), 2319–2324.

104

Erlanger, D., Feldman, D., Kutner, K., Kaushik, T., Kroger, H., Festa, J., Barth, J., et al. (2003).
Development and validation of a web-based neuropsychological test protocol for sportsrelated return-to-play decision-making. Archives of Clinical Neuropsychology, 18(3),
293–316.
Erlanger, D., Kaushik, T., Cantu, R., Barth, J. T., Broshek, D. K., Freeman, J. R., & Webbe, F.
M. (2003). Symptom-based assessment of the severity of a concussion. Journal of
Neurosurgery, 98(3), 477–484.
Erlanger, D., Saliba, E., Barth, J., Almquist, J., Webright, W., & Freeman, J. (2001). Monitoring
resolution of postconcussion symptoms in athletes: preliminary results of a web-based
neuropsychological test protocol. Journal of Athletic Training, 36(3), 280.
Fazio, V., Lovell, M., Pardini, J., & Collins, M. (2007). The relation between post concussion
symptoms and neurocognitive performance in concussed athletes. NeuroRehabilitation,
22(3), 207–216.
Garber, C. E., Blissmer, B., Deschenes, M. R., Franklin, B. A., Lamonte, M. J., Lee, I.-M.,
Nieman, D. C., et al. (2011). Quantity and quality of exercise for developing and
maintaining cardiorespiratory, musculoskeletal, and neuromotor fitness in apparently
healthy adults. Medicine & Science in Sports & Exercise, 43(7), 1334–1359.
Gessel, L. M., Fields, S. K., Collins, C. L., Dick, R. W., & Comstock, R. D. (2007). Concussions
among united states high school and collegiate athletes. Journal of Athletic Training,
42(4), 495.
Giza, C. C., & Hovda, D. A. (2001). The neurometabolic cascade of concussion. Journal of
Athletic Training, 36(3), 228.
Goldberg, J. O., & Miller, H. R. (1986). Performance of psychiatric inpatients and intellectually
deficient individuals on a task that assessed the validity of memory complaints. Journal
of Clinical Psychology, 42, 792-795.
Grady, M. F. (2010). Concussion in the adolescent athlete. Current Problems in Pediatric and
Adolescent Health Care, 40(7), 154–169.
Greve, M. W., Zink, B. J., Greve, M. W., & Zink, B. J. (2009). Pathophysiology of traumatic
brain injury, Pathophysiology of traumatic brain injury. A Journal of Translational and
Personalized Medicine, 76(2), 97–104.
Guskiewicz, K. M. (2001). Postural stability assessment following concussion: one piece of the
puzzle. Clinical journal of sport medicine: official journal of the Canadian Academy of
Sport Medicine, 11(3), 182–189.

105

Guskiewicz, K. M., Bruce, S. L., Cantu, R. C., Ferrara, M. S., Kelly, J. P., McCrea, M., Putukian,
M. (2006). Research based recommendations on management of sport related concussion:
Summary of the National Athletic Trainers’ Association position statement. British
Journal of Sports Medicine, 40(1), 6–10.
Guskiewicz, K. M., Weaver, N. L., Padua, D. A., & Garrett, W. E. (2000). Epidemiology of
concussion in collegiate and high school football players. The American Journal of
Sports Medicine, 28(5), 643–650.
Guskiewicz, K. M., Bruce, S. L., Cantu, R. C., Ferrara, M. S., Kelly, J. P., McCrea, M.,
Putukian, M. (2004). Recommendations on Management of Sport-related Concussion:
Summary of the National Athletic Trainers’ Association Position Statement.
Neurosurgery, 55(4), 891–896.
Hillman, C. H., Erickson, K. I., & Kramer, A. F. (2008). Be smart, exercise your heart: exercise
effects on brain and cognition. Nature Reviews Neuroscience, 9(1), 58–65.
Hillman, C. H., Kramer, A. F., Belopolsky, A. V., & Smith, D. P. (2006). A cross-sectional
examination of age and physical activity on performance and event-related brain
potentials in a task switching paradigm. International Journal of Psychophysiology, 59(1),
30–39.
Hinton-Bayre, A. D., Geffen, G. M., Geffen, L. B., McFarland, K. A., & Friis, P. (1999).
Concussion in contact sports: reliable change indices of impairment and recovery.
Journal of clinical and experimental neuropsychology, 21(1), 70–86.
Hootman, J. M., Dick, R., & Agel, J. (2007). Epidemiology of collegiate injuries for 15 sports:
Summary and recommendations for injury prevention initiatives. Journal of Athletic
Training, 42(2), 311–319.
Iverson, G. L., Lovell, M. R., & Collins, M. W. (2003). Interpreting change on ImPACT
following sport concussion. The Clinical Neuropsychologist, 17(4), 460–467.
Iverson, G. L., Lovell, M. R., & Collins, M. W. (2005). Validity of ImPACT for measuring
processing speed following sports-related concussion. Journal of clinical and
experimental neuropsychology, 27(6), 683–689.
Kamijo, K., & Takeda, Y. (2009). General physical activity levels influence positive and
negative priming effects in young adults. Clinical Neurophysiology, 120(3), 511–519.
Kane, R. L., Roebuck-Spencer, T., Short, P., Kabat, M., & Wilken, J. (2007). Identifying and
monitoring cognitive deficits in clinical populations using Automated
Neuropsychological Assessment Metrics (ANAM) tests. Archives of Clinical
Neuropsychology: the Official Journal of the National Academy of Neuropsychologists,
22 Suppl 1, S115–126.
106

Kelly, J. P. (2001). Loss of consciousness: Pathophysiology and implications in grading and
safe return to play. Journal of Athletic Training, 36(3), 24
Kelly, K. D., Lissel, H. L., Rowe, B. H., Vincenten, J. A., & Voaklander, D. C. (2001). Sport and
recreation-related head injuries treated in the emergency department. Clinical Journal of
Sport Medicine: Official Journal of the Canadian Academy of Sport Medicine, 11(2), 77–
81.
Kramer, A. F., Erickson, K. I., & Colcombe, S. J. (2006). Exercise, cognition, and the aging
brain. Journal of Applied Physiology, 101(4), 1237–1242.
Lambourne, K & Tomporowski, P. (2010). The effect of exercise-induced arousal on cognitive
task performance: A meta-regression analysis. Train Research, 1341, 12-24.
Langlois, J. A., Rutland-Brown, W., & Wald, M. M. (2006). The epidemiology and impact of
traumatic brain injury: a brief overview. The Journal of Head Trauma Rehabilitation,
21(5), 375–378.
Lau, B. C., Collins, M. W., & Lovell, M. R. (2012). Cutoff scores in neurocognitive testing and
symptom clusters that predict protracted recovery from concussions in high school
athletes. Neurosurgery, 70(2), 371–379.
Lau, B. C., Kontos, A. P., Collins, M. W., Mucha, A., & Lovell, M. R. (2011). Which On-field
signs/symptoms predict protracted recovery from sport-related concussion among high
school football players? The American Journal of Sports Medicine, 39(11), 2311–2318.
Leininger, B. E., Gramling, S. E., Farrell, A. D., Kreutzer, J. S., & Peck, E. A. (1990).
Neuropsychological deficits in symptomatic minor head injury patients after concussion
and mild concussion. Journal of Neurology, Neurosurgery, and Psychiatry, 53(4), 293.
Levin, H. S., Benton, A. L., & Grossman, R. G. (1982). Neurobehavioral Consequences of
Closed Head Injury. New York: Oxford University Press.
Levinsons, D. M., & Reeves, D. L. (1997). Monitoring recovery from traumatic brain injury
using automated neuropsychological assessment metrics (ANAM V1.0). Archives of
Clinical Neuropsychology, 12(2), 155–166.
Lincoln, A. E., Caswell, S. V., Almquist, J. L., Dunn, R. E., Norris, J. B., & Hinton, R. Y. (2011).
Trends in concussion incidence in high school sports a prospective 11-year study. The
American Journal of Sports Medicine, 39(5), 958–963.
Lovell, M. (2009). The management of sports-related concussion: Current status and future
trends. Clinics in Sports Medicine, 28(1), 95–111.

107

Lovell, M. R., Collins, M. W., Iverson, G. L., Johnston, K. M., & Bradley, J. P. (2004). Grade 1
or “ding” concussions in high school athletes. The American Journal of Sports Medicine,
32(1), 47–54.
Lovell, M. R., Iverson, G. L., Collins, M. W., Podell, K., Johnston, K. M., Pardini, D., Pardini, J.
(2006). Measurement of symptoms following sports-related concussion: Reliability and
normative data for the post-concussion scale. Applied Neuropsychology, 13(3), 166–174.
Maerlender, A., Flashman, L., Kessler, A., Kumbhani, S., Greenwald, R., Tosteson, T., &
McAllister, T. (2010). Examination of the construct validity of Impact computerized test,
traditional, and experimental neuropsychological measures. The Clinical
Neuropsychologist, 24(8), 1309–1325.
Makdissi, M., Darby, D., Maruff, P., Ugoni, A., Brukner, P., & McCrory, P. R. (2010). Natural
history of concussion in sport markers of severity and implications for management. The
American Journal of Sports Medicine, 38(3), 464–471.
Mansell, J., Tierney, R. T., Sitler, M. R., Swanik, K. A., & Stearne, D. (2005). Resistance
training and head-neck segment dynamic stabilization in male and female collegiate
soccer players. Journal of Athletic Training, 40(4), 310–319
Marar, M., McIlvain, N. M., Fields, S. K., & Comstock, R. D. (2012). Epidemiology of
concussions among united states high school athletes in 20 sports. The American Journal
of Sports Medicine, 40(4), 747–755.
Maroon, J. C., Lovell, M. R., Norwig, J., Podell, K., Powell, J. W., & Hartl, R. (2000). Cerebral
concussion in athletes: Evaluation and neuropsychological testing. Neurosurgery, 47(3),
659–669; discussion 669–672.
Maruta, J., Lee, S. W., Jacobs, E. F., & Ghajar, J. (2010). A unified science of concussion.
Annals of the New York Academy of Sciences, 1208(1), 58–66.
McCREA, M., Barr, W. B., Guskiewicz, K., Randolph, C., Marshall, S. W., Cantu, R., Onate, J.
A.(2005). Standard regression-based methods for measuring recovery after sport-related
concussion. Journal of the International Neuropsychological Society, 11(01), 58–69.
McCrea, M., Hammeke, T., Olsen, G., Leo, P., & Guskiewicz, K. (2004). Unreported concussion
in high school football players: Implications for prevention. Clinical Journal of Sport
Medicine: Official Journal of the Canadian Academy of Sport Medicine, 14(1), 13–17.
McCrea, M., Kelly, J. P., Randolph, C., Cisler, R., & Berger, L. (2002). Immediate
neurocognitive effects of concussion. Neurosurgery, 50(5), 1032–1042.
McCrory, P. (2001). Does second impact syndrome exist? Clinical journal of sport medicine:
Official Journal of the Canadian Academy of Sport Medicine, 11(3), 144–149.
108

McCrory, P., Johnston, K., Meeuwisse, W., Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T.
(2005). Summary and agreement statement of the 2nd international conference on
concussion in sport, Prague 2004. British Journal of Sports Medicine, 39(4), 196–204.
McCrory, P., Meeuwisse, W., Johnston, K., Dvorak, J., Aubry, M., Molloy, M., & Cantu, R.
(2009). Consensus statement on concussion in sport: the 3rd international conference on
concussion in sport held in Zurich, November 2008. Journal of Athletic Training, 44 (4),
434-438.
McCrory, P., Davis, G., & Makdissi, M. (2012). Second impact syndrome or cerebral swelling
after sporting head injury. Current Sports Medicine Reports, 11(1), 21–23.
Miller, J. R., Adamson, G. J., Pink, M. M., & Sweet, J. C. (2007). Comparison of preseason,
midseason, and postseason neurocognitive scores in uninjured collegiate football players.
The American Journal of Sports Medicine, 35(8), 1284–1288.
Lovell, M. R., Iverson, G. L., Coolins, M. W., McKeaq, D., & Maroon, J. C. (1999). Does loss of
consciousness predict neuropsychological decrements after concussion? Clinical journal
of Sport Medicine : Official Journal of the Canadian Academy of Sport Medicine, 9(4),
193.
National Collegiate Athletes Association. (2011). NCAA participation rates going up. Retrieved
August 21, 2012, from:
http://www.ncaa.com/news/ncaa/article/2011-11-02/ncaa-participation rates-going
National Federation of State High School Association. (2011). 2010-11 high school athletics
participation survey results. Retrieved Augst 21, 2012, from:
http://www.nfhs.org/content.aspx?id=3282
Nicholls, D. G. (2008). Oxidative stress and energy crises in neuronal dysfunction. Annals of
the New York Academy of Sciences, 1147(1), 53–60.
Olivier, P. E., & Du Toit, D. E. (2008). Isokinetic neck strength profile of senior elite rugby
union players. Journal of Science and Medicine in Sport, 11(2), 96–105.
Ommaya, A. K., & Gennarelli, T. A. (1974). Cerebral concussion and traumatic
unconsciousness correlation of experimental and clinical observations on blunt head
injuries. Brain, 97(4), 633–654.
Pearce, J. M. S. (2008). Observations on concussion. European Neurology, 59(3-4), 113–119.
Praag, H. van, Christie, B. R., Sejnowski, T. J., & Gage, F. H. (1999). Running enhances
neurogenesis, learning, and long-term potentiation in mice. Proceedings of the National
Academy of Sciences, 96(23), 13427–13431.
Randolph, C., McCrea, M., & Barr, W. B. (2005). Is neuropsychological testing useful in the
management of sport-related concussion? Journal of Athletic Training, 40(3), 139.
109

Rechel, J. A., Yard, E. E., & Comstock, R. D. (2008). An epidemiologic comparison of high
school sports injuries sustained in practice and competition. Journal of Athletic Training,
43(2), 197–204.
Sarmiento, K., Mitchko, J., Klein, C., & Wong, S. (2010). Evaluation of the centers for disease
control and prevention’s concussion initiative for high school coaches: “Heads Up:
Concussion in high school sports.” Journal of School Health, 80(3), 112–118.
Schatz, P. (2010). Long-term test-retest reliability of baseline cognitive assessments using
impact. The American Journal of Sports Medicine, 38(1), 47–53.
Schatz, P., Pardini, J. E., Lovell, M. R., Collins, M. W., & Podell, K. (2006). Sensitivity and
specificity of the impact test battery for concussion in athletes. Archives of Clinical
Neuropsychology, 21(1), 91–99.
Schatz, P., & Putz, B. O. (2006). Cross-validation of measures used for computer-based
assessment of concussion. Applied Neuropsychology, 13(3), 151–159.
Schatz, P., & Zillmer, E. A. (2003). Computer-based assessment of sports-related concussion.
Applied Neuropsychology, 10(1), 42–47.
Segalowitz, S. J., Mahaney, P., Santesso, D. L., MacGregor, L., Dywan, J., & Willer, B. (2007).
Retest reliability in adolescents of a computerized neuropsychological battery used to
assess recovery from concussion. NeuroRehabilitation, 22(3), 243–251
Shibata, M., Einhaus, S., Schweitzer, J. B., Zuckerman, S., & Leffler, C. W. (1993). Cerebral
blood flow decreased by adrenergic stimulation of cerebral vessels in anesthetized
newborn pigs with traumatic brain injury. Journal of Neurosurgery, 79(5), 696–704.
Shors, J., T. (2004). Learning during stressful times. Learning Memory, 11, 137-144.
Sim, A., Terryberry-Spohr, L., & Wilson, K. R. (2008). Prolonged recovery of memory
functioning after mild traumatic brain injury in adolescent athletes. Journal of
Neurosurgery, 108(3), 511–516.
Sun, D. A., Deshpande, L. S., Sombati, S., Baranova, A., Wilson, M. S., Hamm, R. J., &
DeLorenzo, R. J. (2008). Traumatic brain injury causes a long-lasting calcium (Ca2+)plateau of elevated intracellular Ca levels and altered Ca2+ homeostatic mechanisms in
hippocampal neurons surviving brain injury. European Journal of Neuroscience, 27(7),
1659–1672.
Swanik, C. B., Covassin, T., Stearne, D. J., & Schatz, P. (2007). The relationship between
neurocognitive function and noncontact anterior cruciate ligament injuries. The American
journal of sports medicine, 35(6), 943–948.

110

Thomas, M., Haas, T. S., Doerer, J. J., Hodges, J. S., Aicher, B. O., Garberich, R. F., Mueller, F.
O. (2011). Epidemiology of sudden death in young, competitive athletes due to blunt
trauma. Pediatrics, 128(1), e1–e8.
Vasterling, J. J., Proctor, S. P., Amoroso, P., Kane, R., Heeren, T., & White, R. F. (2006).
Neuropsychological outcomes of army personnel following deployment to the Iraq war.
JAMA: the Journal of the American Medical Association, 296(5), 519–529.
Verweij BH;Muizelaar JP;Vinas, F. (2000). Impaired cerebrl mitochondrialfunction after
traumatic brain injury in humans. Journal of Neurosurgery, 93.
Viano, D. C., Casson, I. R., & Pellman, E. J. (2007).Concussion in professional football.
Neurosurgery, 61(2), 313–328.
Woodward, T. S., Bub, D. N., & Hunter, M. A. (2002). Task switching deficits associated with
Parkinson’s disease reflect depleted attentional resources. Neuropsychologia, 40(12),
1948–1955.

111